LL-HLS Series: The Evolution of LL-HLS

HTTP Live Streaming, or HLS, was initially created and released by Apple in 2009 to solve the problems of scaling. The HLS protocol has become one of the most popular protocols used today, and is widely supported. It’s biggest disadvantage, as with other similar HTTP based streaming protocols, is latency. In this blog post we will discuss the evolution of Low Latency HLS, how the different variants and updates have changed over time, and how they have impacted the video streaming industry. To read more about how HLS works, check out our previous blog post.

The Need for Lower Latencies

While HLS has become widely adopted over the last years, the market changed. Latency has become a bigger problem, leading to spoiled experiences. Strategies and improvements were suggested to combat latency, and people in the industry have been actively looking for solutions to deliver video at lower latency.

It was Twitter’s Periscope platform that first implemented a number of these improvements and bundled them into LHLS in 2016. While a proprietary version by Periscope, it resolved some drawbacks within HLS and aimed to provide low latency in the range of two to five seconds. The protocol was developed to be compatible with the HLS standard in such a way that default players across different platforms could fall back to standard HLS behaviour. LHLS introduced two approaches in order to reduce latency within LHLS:

1. Leveraging HTTP/1.1 chunked transport for segments
2. Announcing segments before they are available

While reducing the segment size could be another possible approach, it has its limitations. In a best case scenario, each segment starts with an Intra frame (IDR-frame), which allows a player to start the playback of a segment immediately, without the need to wait and download an earlier segment. As I-frames are significantly larger than predicted frames (P-frames), reducing the segment size (and adding in more I-frames) would increase the overall bandwidth used.

The approach used by Periscope tackled the problem of a large buffer, as a part of that buffer was announced up front, without it being available. As a result, this actually reduced the mandatory backoff from the live point, but also the ability to build a buffer. As such, a player should know the location for a segment as soon as possible. By anticipating segment creation, and already listing the location of future segments, a player can anticipate which files need to be loaded. Announcing the segments early, players which were not LHLS aware could still play the stream as if it were a normal HLS stream, and still get an improvement in latency.

This update removed both the latency introduced by segmenting the video stream, as well as the latency introduced by the playlist. Additionally, the approach is scalable over standard CDNs, given they support HTTP/1.1 chunked transfer. To read more about the first LHLS update, check out THEO’s blog post.

The Answer from the Industry

Based on the progress Periscope made with LHLS, a larger group of companies started to experiment with combinations of chunked transfer and HLS. Out of this, a community-based L-HLS specification was drafted. While based on the same concepts as Periscope's LHLS, it introduced a new tag for future segments instead of replacing the last segments in the playlist. This allows to remove all impact on players not compatible with the L-HLS specification and enables them to build a large buffer as with normal HLS.

While this alternative version of HLS was being finalised, in 2019, Apple released a protocol extension for Low Latency HLS later referred to as LL-HLS. The extension still provided backwards compatibility with regular HLS, but gave an Apple approved (and Apple ecosystem-compatible) approach to low latency with HLS. Players which do not understand the protocol extension will play back the same streams, with a higher latency. This allows publishers to offer a single HLS solution both for optimised and non-optimised players. In essence, the update was quite simple: segments are allowed to be split up into smaller parts, which can be downloaded as the larger segment is constructed.

The biggest challenges in this first version of LL-HLS came with the HTTP/2 Push requirement. With the announcement of the protocol extension, Apple stated this requirement would be “widely adopted” by CDNs, however this wasn’t usually the case. The new protocol extension required a number of HTTP/2 features (including multi stream control, H2 push and H2 ping). Many of the major CDN’s were not ready for this, which made large scalability seem like a huge headache for most in the industry. It also caused additional problems for use cases such as SSAI and content replacement.

The industry felt Apple missed out on a simpler solution using chunked transfer encoding (CTE). Using a CTE-based solution would have streamlined the process of delivering low latency content on both HLS and MPEG-DASH streams. The extension also meant a delay in implementation. For SSAI, the specification required an intimate collaboration between the playlist manipulator and the CDN. Tying together both playlist generation and data delivery (through segment push) would require new algorithms and scalability strategies for CDN providers. To read a full update of all the changes, check out THEO’s blog post, or this blog post made by our friends at MUX.

The Latest LL-HLS Update

In early 2020, Apple announced an update to the draft LL-HLS specification. After the concern and friction within the industry, the update removed the HTTP/2 Push requirement and instead introduced a new tag to announce upcoming segments. This update made the LL-HLS protocol very similar to the community version of low latency L-HLS.

The new tag introduced, was called #EXT-X-PRELOAD-HINT. With this tag, a server publishing a low latency HLS stream is able to announce the most likely location of the next media data which will become needed to continue playback. This allows a player client to perform a request, allowing the data to flow in as soon as the next part of a segment becomes available. This process can then be repeated, allowing the removal of additional round-trip time when loading new media data (which was the main reason to use HTTP/2 push).

The updated spec required significant changes to those (THEO included) who had already started implementation of LL-HLS tools. The advantages due to the new change however are significant for the industry and will remove the HTTP/2 push adoption barrier, simplifying the deployment of the technology. At the end of April, Apple published the updated RFC of HLS, including LL-HLS, which makes large future changes unlikely. This allows the industry to go all-in on LL-HLS without running the risk of having to make significant changes in architecture.

Other important updates of LL-HLS spec include:

• The usage of the new tags (EXT-X-PART, EXT-X-PRELOAD-HINT, EXT-X-SERVER-CONTROL, EXT-X-SKIP, etc.) has been specified in use together with "normal" HLS streams. This means some of these can also be used without low latency mode (i.e. the ability to support blocking playlist reloads and delta playlists.).

• The addition of CAN-SKIP-DATERANGES tag to allow skipping of daterange tags.

• There is a LL-HLS profile which has been defined. It contains most of the data as it was in the preliminary specification for LL-HLS:

  • Blocking playlist reload is no longer mandatory, but is recommended.
  • Multiple preload hints can be listed for the same type.
  • It is no longer defined how long parts must remain in the playlist (parts must be removed after 3 target durations, and should appear at "live edge", but "live edge" is not defined).
    

The Adoption of LL-HLS Today

The newest LL-HLS update presented another advantage by easing compatibility with the major alternative low latency solutions currently in the market, such as LL-DASH and community L-HLS. In fact, the new LL-HLS specification feels like a superset of L-HLS, which also makes use of a tag to announce the location of the next segment. With that being said, it seems that the industry is giving up on community L-HLS, and the race has begun to move forward and adopt the newest update of the Apple LL-HLS spec. 

With Apple’s recent solidification of the changes from the preliminary extension into the official update of the HLS specification, it's safe for people to start investing in this.

What's Next?

Now that we’ve discussed how HLS, LHLS, and LL-HLS have evolved and changed over the last decade, we want to go more in-depth on how exactly the most up-to-date spec works, problems it’s fixed, what challenges and opportunities the spec brings to the table. Watch this space for the next blog.

THEO currently has a beta player available that supports the newest LL-HLS update. If you’re interested, don’t hesitate to contact our experts.

Interested in learning more about implementing a Low Latency solution?

On-Demand Webinar: Apple’s LL-HLS is Finally Here

Hosted by our CTO Pieter-Jan Speelmans

Contact our LL-HLS Experts