Share this
by Pieter-Jan Speelmans on January 21, 2021
The streaming wars are on. If you don’t believe me, just look at the sheer amount of different streaming services which are available, especially as this amount is ever growing. While the average budget spent on streaming services is rising as well, there is no doubt that streaming services need to differentiate in order to attract and retain subscribers. Where content can be licensed in an exclusive way, and the fiercest competition is illegal distribution, there are other important factors to be taken into account, especially the hygiene factors such as platform support. The real question is: how do we keep costs in check while still ensuring this hygiene factor is fulfilled? In this article, a first in a larger series on cross platform media delivery, we’ll dive into some of the challenges we bumped into while helping our customers deploy their media experiences across vast amounts of platforms.
The different viewpoints to categorise platforms
First things first: which platforms are we actually talking about? There are different ways to categorise devices. Often you see overviews listing form factor based categories such as “Desktop”, “Mobile”, “Smart TV”, “Connected devices” and “Gaming consoles”. In other reports, you will see a split based on technology used (such as “Web”, “Android”, “iOS” and “Roku”), screen size (“Mobile”, “Laptop”, “TV”) or splits based on input methods (“Mouse & Keyboard”, “Touch”, “Remote”). The biggest challenge here is that each categorisation has its merits: form factors are easy to link to physical devices, the technology split is easy for developers to link technologies where screen sizes and input methods are highly important from a UI/UX perspective. There is a reason however why these devices are categorised: the amount of combinations of manufacturer, model, OS, OS version, ... is huge.
When we look at the numbers, an important thing to note is the growth of consumption of streaming content directly through smart TV applications. Years ago, streaming started as something you would do solely on your desktop or laptop. The rise of mobile devices caused a significant shift. Today, desktops account for about 10% of the total viewing time. Mobile streaming across smartphones and tablets accounts for about 15%. Early 2019, desktops still accounted for 14% and mobile still accounted for 23%. So where is the other 75% today? And where are people shifting to? Well, it’s in the devices attached to the big screen sitting in our living rooms (and bedrooms, kitchens, bathrooms, …). Even in this category however, there is a big shift which is becoming apparent.
Comparing viewing time on the big screen, only 10% was done using the smart TV directly in 2019. All other usage was related to connected devices such as dongles, casting devices and gaming consoles. Comparing this to the latest numbers, we can see that this percentage has doubled in 2020, with about 20% of the viewing time on the big screen happening through apps on the smart TV. Reasons for this are simple: connected devices often require additional cables, remotes and smart TVs are ever becoming more common, removing the need to enhance the intelligence of your TV.
The challenge I call “Version Hell”
It’s great to know which platforms you want to target. Unfortunately, at that point, your work isn’t done. Platforms come in different versions and flavours. Take the example of Samsung’s smart TVs running Tizen (on which we wrote an entire dedicated article here), spanning about 50% of the smart TV market. On average, a smart TV sits in a living room for 8 years. Devices will be bought the first two years after introduction, and will gradually be replaced until only a few stragglers remain. As a result, the 50% market share is actually split in (for example) 5% being 2020 series devices, 10% being 2019 series, 9% being 2018 series, 8% being 2017, and so on. So does this fragmentation matter?
The media landscape is evolving extremely fast. A device which is 8 years old was shipped with technology which is 8 years old. Often, these devices are not updated. Software updates, especially in the big screen landscape are rare: vendors such as Samsung and LG focus on sales numbers. They don’t see it as a service where they always provide the latest and greatest capabilities on their old devices: they actually benefit from consumers wanting new capabilities, retiring their old device and… purchasing a new one. There are exceptions, such as Roku and Apple TV devices which are kept up to date with forced automatic updates, but these exceptions are rare.
With older devices being present in the landscape, and those devices not having the latest and greatest features, different platform versions can differ in capabilities significantly. There are different ways how this becomes apparent. Some of the most notable ones are related to decoder capabilities, DRM capabilities and streaming protocol support. We’ll go through these in the remainder of this article.
Decoder challenges when comparing platform versions
Decoder capabilities can differ between versions greatly. This is not surprising. New codecs get launched and mature regularly. For example, at the start of 2020 the first AV1 enabled devices started appearing. It will take a long time before the entire set of devices viewers use will support this codec. Similarly, older devices are still around which don’t support the HEVC codec, a commonly used codec to deliver 4k content. As a result, using these codecs across all devices in the ecosystem is not possible, potentially forcing you to set up (or maintain) streams using other codecs such as AVC to achieve compatibility with older versions and devices.
Not just codecs, but also hardware capabilities evolved a lot over the past years. While this shows in codec compatibility, this also shows in other areas more related to performance of the decoder. For example, older devices often cannot seek to a frame accurately due to the lack of fast decode capabilities. This can result in either long seeking times or higher times to the first frame, which are highly undesirable from a QoE perspective.
Similarly, handoff between different decoder configurations has evolved a lot over the last years. This becomes especially apparent when looking at server side ad insertion scenarios where codecs or DRM pipelines are being switched. This is a capability which is crucial for smooth server side ad insertion (SSAI) experiences. For older devices (usually pre-2018, but this can differ depending on vendors), this capability is not available. The impact? Well, it’s quite big: decoding pipelines often need to be torn down and re-initialised completely, leading to black frames and stalling behaviour. Enabling smooth transitions between ads and content can be extremely hard. We struggled with this first hand, and have been working to enable these smooth transitions where possible in THEOplayer for our customers to ensure the best possible user experience.
It’s important to consider these differences in decoders. Depending on the versions you aim to support, it can hinder use cases significantly, such as not allowing you to do SSAI smoothly (which reduces revenue potential), requiring you to provide different encoder configurations (which increases cost), and can ruin user experience (which could increase customer churn). These parameters should weigh in when deciding to support a platform, or not, and can even be leveraged to reduce total cost by dropping support for older versions which are not used any more.
DRM challenges on old devices
When distributing premium content, DRM is a must have. Contracts to obtain content rights are often loaded with (hardware) DRM requirements, especially when it comes to high quality content. Over the years, DRM systems have evolved. Years ago, Widevine was still a separate company, providing Widevine Classic and systems such as Verimatrix ViewRight were commonly used and supported. These days, DRM usage has standardised towards Widevine Modular, PlayReady and Fairplay. Even between these different flavours, standardisation is happening, with common encryption being used across the board and the difference in encryption mode (AES-CBC vs AES-CTR) collapsing into a common support for AES-CBC.
Older devices can however not benefit from the recent standardisation. This can cause significant headaches when targeting older devices. A clear example is webOS 1.0 and 2.0 which officially support only Widevine Classic (although PlayReady support is available with some caveats). As these devices are as recent as 2014 and 2015 models, still sold well into 2016, these are still quite recent. The amount of platforms using these legacy DRM systems is luckily dwindling, resulting in a pretty decent coverage when going for the holy trinity of Widevine Modular, PlayReady and Fairplay.
On the horizon, there is however another issue. With AES-CBC (often called CBCS) becoming the more common encryption mode in the DRM landscape, there is a big promise of unifying streaming pipelines to deliver the same media data with CMAF/CBCS combinations across all platforms. However from a device support standpoint, we’re not there just yet. PlayReady added support for CBCS in version 4. Support within Widevine has grown as of version 15. In practice, this means that most devices which became available in 2020, or received a software update in 2020, are very likely to have support. The ecosystem is definitely growing, but the disadvantage here is that devices remain around as they grow older, and will stay around for at least another 5 odd years. The lack of CBCS support on older devices will as a result hinder adoption for the years to come, forcing the availability of parallel AES-CTR streams for older platforms, and thus increasing the cost to support these platforms.
Flavours of protocols and support across devices
Similar to encoders, DRM systems and so many other things, streaming protocols are far from static. The last few years, low latency has gotten a lot of attention with the spawn of LL-DASH and LL-HLS, but even before that, there were many evolutions. Similarly, server side ad insertion became a lot more popular over the last years, and compatibility guidelines to use HLS’ EXT-X-DISCONTINUITY and DASH’s Periods, highly useful for SSAI, started popping up. When looking back slightly further we see some additional fundamental changes with support for fragmented MP4 (and CMAF) in HLS and requirements for subtitle support (in various old and new formats) and alternative audio tracks popping up. A long list of additions to the requirements in only a few years.
In contrast, a lot of platforms, and especially embedded devices such as smart TVs, are on a slow update cycle. Not even talking about devices being sold historically, most platforms update their software once per year, and this is only for new hardware, sold that year. Older devices are often left behind in the race to new capabilities. With old devices making up a large portion of the devices in the field, the impact on capabilities is not to be underestimated: often the capabilities which are needed by streaming services these days are not supported on the platform natively.
The list of incompatibilities which pop up is long. On one end, there are the compatibility issues with streaming protocols. Old devices were often created in a time when streaming protocols such as Smooth Streaming were popular. MPEG-DASH and HLS were a mere shim of what they are now, making it no surprise device vendors chose to implement extensive support for Smooth Streaming and only had a reduced interest in fine tuning the implementation for other protocols.
In order to circumvent the versioning problem there are a few approaches which are used. A first approach we see is a costly one and involves setting up legacy streams, specific for the targeted platform and version. It increases the cost to support the platform and results in growth of different streams, increasing complexity of the backend, and again the cost as a result.
Luckily, there is another approach. Most platforms do allow for third party players to be loaded. This does provide a solution and allows to broaden the support for the required capabilities. With THEOplayer, support for the latest MPEG-DASH and HLS can be brought to Tizen and webOS devices going back to early versions. It allows you to go even beyond basic playback capabilities and even bring SSAI and rich monetisation options to the platform. While there are still platforms where this is not possible, for most common versions, it does reduce cost of supporting these versions significantly. It provides a good baseline to bring cross platform support at a fraction of the cost compared to setting up custom streams.
Where to go from here?
As you can see, there are plenty of challenges to go around when looking to expand your platform footprint. From THEO’s side, we work to ensure these challenges are taken from the board as much as possible. We’ll provide some further insights on how this can be done in a cost efficient way in the following blog posts we will write in this series. If in the meantime you have any questions: let us know, and our team is happy to assist.