How to detect active text track cues

This article describes how you can use the THEOplayer API to detect the active text track cues.

The TextTrack API provides this functionality. More specifically, as a developer, you'll subscribe to the enter and exit events in the TextTrack API, or to the cuechange event.

Implementing this functionality can be a use-case for developers who want to build their own UI, and insert and style their subtitles with maximum freedom. Alternatively, you may require the access the active cues for analytics purposes, or to render it outside the video player.

Another common use-case is to detect the active text track cue of timed metadata. This article also discusses this use-case.

SDKs

THEOplayer allows you to track text track changes on the following SDKs.

Web SDK	Android SDK	iOS SDK	tvOS SDK	Android TV SDK	Chromecast SDK	Roku SDK
Yes	Yes	Yes	Yes	Yes	Yes

Implementation for subtitles and closed captions

The TextTrack API is available across all of our 4 base SDKs. As described in the introduction, to detect which text track cues are active, you want to leverage the enter and exit events in the TextTrack API, or the cuechange event.

Note that this subsection focuses on detecting active cues for subtitles and closed captions. Go to the subsection on "implementation for timed metadata" if you rather want to track timed metadata like ID3, emsg, EXT-X-DATERANGE and EventStream.

• Web SDK
• Android SDK
• iOS SDK
• Roku SDK

Web SDK

The implementation of the Web SDK applies to all web-based platforms, including Tizen and webOS.

The Web SDK exposes the TextTrack API through player.textTracks. This textTracks property is a TextTrackList that inherits from the TrackList. This TrackList dispatches the events from the TrackListEventMap. This TrackListEventMap contains the addtrack event, as well as the change and removetrack event.

In the callback of your addtrack event, you want to track the addcue event through the TextTrack interface. Then, in the callback of your addcue event, you want to track the enter and exit events through the TextTrackCue interface. The enter event is dispatched when a cue becomes active and the exit event is dispatched when a cue becomes inactive. You can fetch the actual content in the callback of your enter event through its content property.

Alternatively, in the callback of your addtrack event, you could track the cuechange event through the TextTrack interface. Then, in the callback of your cuechange event, you want to iterate over the active cues. For each active cue, you could also query its content property.

The code below allows you to detect the active text track cues.

player.textTracks.addEventListener("addtrack", function (e1) {
  const track = e1.track;
  track.addEventListener("addcue", function (e2) {
    const cue = e2.cue;
    cue.addEventListener("enter", function (e3) {
      console.log(e3, e3.cue.content);
    });
    cue.addEventListener("exit", console.log);
  });
  track.addEventListener("cuechange", function (e2) {
    const cues = e2.track.activeCues;
    for (let i = 0; i < cues.length; i++) {
      console.log("cuechange active cue", i, cues[i]);
    }
  });
});
// ...
// player.source = ...

You want to invoke these event handlers before you configure your stream, because your video player might have already dispatched the event before you were able to subscribe to it.

Android SDK

The implementation of the Android SDK applies to all Android-based platforms, including Android TV and Fire TV.

The Android SDK exposes the TextTrack API through player.getTextTracks(). This getTextTracks() method returns a TextTrackList that inherits from the TrackList. This TrackList dispatches the events from the TextTrackListEventTypes. The TextTrackListEventTypes contains the ADDTRACK event, as well as the TRACKLISTCHANGE and REMOVETRACK event.

In the callback of your ADDTRACK event, you want to track the ADDCUE event through the TextTrack interface. Then, in the callback of your ADDCUE event, you want to track the ENTER and EXIT events through the TextTrackCue interface. The ENTER event is dispatched when a cue becomes active and the EXIT event is dispatched when a cue becomes inactive. You can fetch the actual content in the callback of your ENTER event through its getContent() method.

Alternatively, in the callback of your ADDTRACK event, you could track the CUECHANGE event through the TextTrack interface. Then, in the callback of your CUECHANGE event, you want to iterate over the active cues. For each active cue, you could also query its content property.

The code below allows you to detect the active text track cues.

player.getTextTracks().addEventListener(TextTrackListEventTypes.ADDTRACK, e1 -> {
    TextTrack track = e1.getTrack();
    track.addEventListener(TextTrackEventTypes.ADDCUE, e2 -> {
        TextTrackCue cue = e2.getCue();
        cue.addEventListener(TextTrackCueEventTypes.ENTER, e3 -> {
            System.out.println(e3 + " " + e3.getCue().getContent());
        });
        cue.addEventListener(TextTrackCueEventTypes.EXIT, e3 -> {
            System.out.println(e3);
        });
    });
    track.addEventListener(TextTrackEventTypes.CUECHANGE, e2 -> {
        TextTrackCueList cues = e2.getTextTrack().getActiveCues();
        for (int i = 0; i < cues.length(); i++) {
            System.out.println("cuechange active cue " + i + " " + cues.getItem(i));
        }
    });
});
// ...
// player.setSource(...)

You want to invoke these event handlers before you configure your stream, because your video player might have already dispatched the event before you were able to subscribe to it.

iOS/tvOS SDK and Legacy iOS/tvOS SDK (4.12.x)

The implementation of the iOS SDK applies to all iOS-based platforms, including iPadOS and tvOS.

The iOS SDK exposes the TrackTrack API through player.textTracks. This textTracks property is a TextTrackList. This TextTrackList dispatches the events from the TextTrackListEventTypes. The TextTrackListEventTypes contains the ADD_TRACK event, as well as the CHANGE and REMOVE_TRACK event.

In the callback of your ADD_TRACK event, you want to track the ADD_CUE event through the TextTrack interface. Then, in the callback of your ADD_CUE event, you want to track the ENTER and EXIT events through the TextTrackCue interface. The ENTER event is dispatched when a cue becomes active and the EXIT event is dispatched when a cue becomes inactive. You can fetch the actual content in the callback of your ENTER event through its content property.

Alternatively, in the callback of your ADD_TRACK event, you could track the CUE_CHANGE event through the TextTrack interface. Then, in the callback of your CUE_CHANGE event, you want to iterate over the active cues. For each active cue, you could also query its content property.

The code below allows you to detect the active text track cues.

player.textTracks.addEventListener(type: TextTrackListEventTypes.ADD_TRACK) { (e1) in
    var track : TextTrack = e1.track as! TextTrack
    track.addEventListener(type: TextTrackEventTypes.ADD_CUE) { (e2) in
        let cue = e2.cue
        cue.addEventListener(type: TextTrackCueEventTypes.ENTER) { (e3) in
            print(e3, e3.cue.content)
        }
        cue.addEventListener(type: TextTrackCueEventTypes.EXIT) { (e3) in
            print(e3)
        }
    }
    track.addEventListener(type: TextTrackEventTypes.CUE_CHANGE) { (e2) in
        let textTrack : TextTrack = e2.track as! TextTrack
        let cues = textTrack.activeCues
        cues.forEach { (cue) in
            print("cuechange active cue", cue)
        }
    }
}
// ...
// player.source = ...

You want to invoke these event handlers before you configure your stream, because your video player might have already dispatched the event before you were able to subscribe to it.

Roku SDK

This subsection is in maintenance. Reach out to our team if you need help.

Implementation for timed metadata

The implementation for timed metadata is identical to the one for subtitles and closed captions, except for three things.

1. You don't necessarily use the enter event. Some types of timed metadata, for example ID3 cues, only related to one specific moment, and those types only have an exit trigger.

2. You might need to set the mode of the relevant text track to hidden, as documented in "How to programmatically enable or disable text tracks". Some types of timed metadata, for example EXT-X-DATERANGE and EventStream, are disabled by default. You should add a condition to your addtrack callback to decide whether you want to set your track to hidden.

3. You should set hlsDateRange to true if you want to detect EXT-X-DATERANGE tags in an HLS stream in your PlayerConfiguration or stream configuration. Refer to the Web, Android or iOS documentation for more info.

The article on "how to track id3 cues / tags" might be useful to learn more about detecting ID3 tags specifically.