Media Playback

TAL provides a device-agnostic media playback API via the MediaPlayer class. This can be used to play video and audio files supported by the device.

The framework only supports the the playback of one item of media at a time. Video can only be played in full screen mode.

Accessing the media playback API

You will need to use the correct media player modifier in the device’s configuration file.

Access the media player through the application’s Device object:

this._mediaPlayer = RuntimeContext.getDevice().getMediaPlayer();

For example, creating a component to playback media:

require.def(
    "exampleapp/appui/components/examplemediaplayer",
    [
        "antie/widgets/component",
        "antie/runtimecontext",
        "antie/devices/mediaplayer/mediaplayer"
    ],
    function(Component, RuntimeContext, MediaPlayer) {
        var ExampleMediaPlayer = Component.extend({
            init : function init () {
                init.base.call(this, "ExampleMediaPlayer");
                this._mediaPlayer = RuntimeContext.getDevice().getMediaPlayer();
            }
        });
    return ExampleMediaPlayer;  
});        

Playback States

The MediaPlayer has a number of playback states:

MediaPlayer.STATE = {
    EMPTY:      "EMPTY",     // No source set
    STOPPED:    "STOPPED",   // Source set but no playback
    BUFFERING:  "BUFFERING", // Not enough data to play, waiting to download more
    PLAYING:    "PLAYING",   // Media is playing
    PAUSED:     "PAUSED",    // Media is paused
    COMPLETE:   "COMPLETE",  // Media has reached its end point
    ERROR:      "ERROR"      // An error occurred
};

The media playback state changes as API methods are called. Different API methods have different effects depending on the playback state. Transitions to the STOPPED and ERROR state have been limited in the state diagram below for clarity. Transitions occur synchronously in response to API methods, except those marked by an asterisk (*) which result from asynchronous behaviour such as buffering or at the end of media.

The playback state can be accessed using getState().

In state EMPTY

• On entry to state: clear source url
• Call initialiseMedia(url) : store the url and transition to STOPPED
• Call playFrom(), beginPlayback(), beginPlaybackFrom(), pause(), resume() or stop(): transition to ERROR

In state STOPPED

• On entry to state: Cancel all playback and fire event type MediaPlayer.EVENT.STOPPED
• Call beginPlaybackFrom(time) : request playback from ‘time’ (clamped to the available range), transition to BUFFERING
• Call reset() : transition to EMPTY
• Call beginPlayback() : begin playback from wherever the device can (could be anywhere for Live, usually media start for VOD), transition to BUFFERING
• Call initialiseMedia(), playFrom() pause(), resume() or stop(): transition to ERROR
• Device metadata/start-buffering/finish-buffering : stay in STOPPED

In state BUFFERING

• On entry to state: fire event type MediaPlayer.EVENT.BUFFERING
• Call playFrom(time): seek to time (clamped to the available range), and remember to resume playback and transition to PLAYING when sufficient data is available
• Call stop() : transition to STOPPED
• Call pause() : When buffering is complete, remember to pause playback and transition to PAUSED when sufficient data is available
• Call resume() : When buffering is complete, remember to resume playback and transition to PLAYING when sufficient data is available
• Call beginPlayback(), beginPlaybackFrom(), reset() or initialiseMedia(): transition to ERROR
• If sufficient data is available for playback, transition to either PLAYING or PAUSED as required
• If the implementation can determine the duration before it’s ready to play, then it should fire event type MediaPlayer.EVENT.STATUS

In state PLAYING

• On entry to state: fire event type MediaPlayer.EVENT.PLAYING
• Call pause() : pause playback and transition to PAUSED
• Call playFrom(time) : seek to time (clamped to the available range) and transition to BUFFERING
• Call stop() : transition to STOPPED
• Call resume() : do nothing
• Call beginPlayback(), beginPlaybackFrom(), reset() or initialiseMedia(): transition to ERROR
• At regular intervals (< 1s): fire MediaPlayer.EVENT.STATUS
• On media completion: transition to COMPLETE
• If buffering starts, transition to BUFFERING

In state PAUSED

• On entry to state: fire event type MediaPlayer.EVENT.PAUSED
• Call resume() : resume playback from current time and transition to PLAYING
• Call playFrom(time): seek to time (clamped to the available range) and transition to BUFFERING
• Call stop() : transition to STOPPED
• Call pause() : do nothing
• Call beginPlayback(), beginPlaybackFrom(), reset() or initialiseMedia(): transition to ERROR

In state COMPLETE

• On entry to state: fire event type MediaPlayer.EVENT.COMPLETE
• Call stop() : transition to STOPPED
• Call playFrom(time): seek to time (clamped to the available range) and transition to BUFFERING
• Call beginPlayback(), beginPlaybackFrom(), reset(), initialiseMedia(), pause() or resume() : transition to ERROR
• Device metadata/start-buffering/finish-buffering : stay in COMPLETE

In state ERROR

• On entry to state: fire event type MediaPlayer.EVENT.ERROR
• Call reset() : transition to EMPTY
• Call initialiseMedia(), beginPlayback(), beginPlaybackFrom(), pause(), resume(), stop() or playFrom() : transition to ERROR
• Device metadata/start-buffering/finish-buffering : stay in ERROR

Initialising the media (audio or video)

To initialise the media, the MediaPlayer must be in the EMPTY state. You initialise the media by using the initialiseMedia() method. It takes a mediaType (MediaPlayer.TYPE.VIDEO or MediaPlayer.TYPE.AUDIO), a url, the mimeType of the file and a DOM element to render the media into.

For example, to load a video:

this._mediaPlayer.initialiseMedia(MediaPlayer.TYPE.VIDEO, "http://example.com/video.mp4", "video/mp4", sourceContainer);

If initialising the media is successful, the playback state will change to STOPPED and an event will be emitted of type MediaPlayer.EVENT.STOPPED.

From the STOPPED state, MediaPlayer can be used to access the properties of the media source (getSource(), getMimeType(), getMimeType());

To change the media source, applications must ensure the MediaPlayer is in the STOPPED state and call reset() to transition to the EMPTY state before calling initialiseMedia() with a different URL.

Playing and seeking through media

The beginPlaybackFrom(seconds) method will attempt to play the media from the provided time (in seconds).

For example, to play media from the start:

this._mediaPlayer.beginPlaybackFrom(0);

This will cause the playback state to change to BUFFERING and an event emitted of type MediaPlayer.EVENT.BUFFERING.

beginPlayback() can also be used. This is sometimes useful when beginPlaybackFrom(seconds) fails to work because it cannot determine the duration of the media, for example, on a live stream.

When the device has loaded enough of the video to begin playback, the MediaPlayer will transition to the PLAYING state and an event of type MediaPlayer.EVENT.PLAYING emitted.

The device may have a slow network connection, meaning the rate of video download is slower than the playback speed. When this occurs, the MediaPlayer will enter the BUFFERING state asynchronously and an event of type MediaPlayer.EVENT.BUFFERING (as above) will be emitted. By adding an event callback to listen to these events, you can update your application’s interface to, for example, show a buffering spinner.

playFrom(seconds) can be used to seek to different points in the media. If the seconds parameter is larger than the duration of the media, the value will be clamped and playback will begin from just before the end. Requests to seek within one second of the current time will be ignored to ensure consistent behaviour across devices.

Hiding the background

Some devices always composite HTML on top of video playback, so if you use any sort of non-transparent background, playback will not be visible.

If this is the case, you will need to remove the background just before playback, then restore on stop.

One way to do this is via a background-none CSS class:

.background-none {
    background-image: none !important;
    background-color: transparent !important;
}

_removeBackground: function () {
    if (this._haveRemovedBackground === undefined &&
     this._mediaType !== MediaPlayer.TYPE.AUDIO) {
        var application = this.getCurrentApplication();
        var device = application.getDevice();
        device.addClassToElement(document.body, 'background-none');
        application.getRootWidget().addClass('background-none');
        this._haveRemovedBackground = true;
    }
},

_restoreBackground: function () {
    var application = this.getCurrentApplication();
    var device = application.getDevice();
    device.removeClassFromElement(document.body, 'background-none');
    application.getRootWidget().removeClass('background-none');
    this._haveRemovedBackground = undefined;
}

Stopping media

To interrupt playback you can call:

this._mediaPlayer.pause();

The MediaPlayer will transition to the PAUSED state. From this state you can choose to resume playback:

this._mediaPlayer.resume();

If you want to close down the video player completely, for example if you are navigating to a different component, you should call

this._mediaPlayer.stop();

This will transition the MediaPlayer to the STOPPED state.

Media playback events

MediaPlayer will emit events to notify the application of changes in the playback state.

To listen for these events in your application, use the addEventCallback() method.

this._mediaPlayer.addEventCallback(this, function(event) {
    //handle events here, e.g. by switching on using event.type
});

The event object has the following properties: { type, currentTime, range, url, mimeType, state} where type is a string specified in the MediaPlayer.EVENT enum:

Sentinels

Some devices do not respond immediately to requests to play, pause or seek through media or do not emit expected events, such as entering buffering. To solve this problem, MediaPlayer will monitor the state of the device to keep it in sync with the API’s state. As a result, the API will fire one of the sentinel events listed below. By listening to for these events in your callback, you can detect deviant media playback behaviour on the device.

An example application snippet using media playback events to display the type of event in a label

this._mediaPlayer.addEventCallback(this, function(event) {
    if(event.type === MediaPlayer.EVENT.BUFFERING) {
        this._showBufferingIcon();
    }
    else if(event.type === MediaPlayer.EVENT.PLAYING) {
        this._hideBufferingIcon();
    }
});

Getting the length of the media

To get the duration of the media, in seconds, use getDuration().

To get the available range in the media that can be seeked in, use getSeekableRange(), this will return the following object for a non-live video:

{
    start:0,
    end:/* duration in seconds */
}

getSeekableRange() is especially useful for live streams, on which the seekable window changes as the live stream progresses.

Errors

API errors (e.g. calling pause() while in the STOPPED state) are treated as fatal errors and the media player transitions to the ERROR state and stops all playback. After this, the player must be reset().

However, device errors (network errors, playback errors, media incompatibility etc.) are raised as error events in the API, but they do not cause a transition to the error state. This is because many device errors are non fatal, and playback can continue normally afterwards. It is recommended that apps similarly treat these error events as notifications, and do not display modal dialogs or end playback just because of a device error event.