BlogBlog
HTML5 Video Skinning – A Case Study
(See the demo.)
Recently RealEyes developed a custom Flash video player for the Monterey Bay Aquarium. We also developed an HTML5 solution for cases when a user might not have Flash Player. In these situations we use the HTML5 video tag and let the browser provide the video player.
The Problem
The limitation here is that each browser’s video player has a unique control bar – and none of these match the look and functionality of the control bar in the Flash video player that we created for the client. See the differences below.
This is how the Flash Player looks.
This is the native video player control bar for the Chrome browser.
The Solution
To create a consistent control bar we used javascript and jQuery (with a little jQuery UI along the way) to manipulate the Media Element API and replace the default native control bar with our own.
File Set Up
The dependencies for this are jQuery and jQueryUI. We handle all our custom control bar creation and functionality in a custom jQuery plugin named videoskin.js. Then we style our control bar with a separate CSS file, videoskin.css.
Let’s break down the code.
First let’s take a look at our very basic HTML. Within the head tags we load jQuery, jQueryUI and our custom plugin ‘videoskin.js’.
On line 8 we apply our plugin to our video tag, which in this case had the ID of test-video.
Within the body tags we’ve got our HTML5 ‘video’ tag with a couple of different source urls specified to keep most browsers happy. Notice that there is no HTML for our custom control bar – that comes later!
<html>
<head>
<script src="http://code.jquery.com/jquery-1.9.1.min.js"></script>
<script type="text/javascript" src="lib/jquery/jquery-ui-1.8.2.min.js"></script>
<script type="text/javascript" src="lib/jquery.videoskin.js"></script>
<script>
$(document).ready(function() {
$('#test-video').videoSkin();
});
</script>
</head>
<body>
<div id="container">
<video id="test-video" controls>
<source src="http://players.edgesuite.net/videos/big_buck_bunny/bbb_448x252.mp4" type="video/mp4">
<source src="http://players.edgesuite.net/videos/big_buck_bunny/bbb_448x252.ogg" type="video/ogg">
<p>Your browser does not support the video tag.</p>
</video>
</div>
</body>
</html>
Now let’s jump to our jQuery plugin code. Here we are using jQuery to inject the html for the new control bar – a series of new DIVs seen on line 8.
// express the video element as a jQuery object
var $t = $(this);
//dynamically add html to style and manipulate
//player container
var $video_wrap = $('<div></div>').addClass('video-player');
//player controls
var $video_controls = $('<div class="video-controls"><div class="video-seek"><div class="video-timer">00:00</div><div class="volume-box"></div></div>');
//add em
$t.wrap($video_wrap);
$t.after($video_controls);
Now, if we switch back to the HTML, we’ll see the code for our new control bar.
<div class="video-controls" style="display: block;">
<a class="video-play" title="Play/Pause"></a>
<a class="video-rewind" title="Rewind 30"></a>
<div class="video-seek">
<div class="video-timer">00:00</div>
<div class="volume-box"></div>
</div>
This gives us our basic control bar which we can style and add our custom icons using typical CSS.
.video-controls {
display: block;
clear: both;
float: left;
padding-top: 5px;
position: relative;
width: 100%;
}
.video-play {
background: url('images/icon_play.png') no-repeat;
display: block;
float: left;
height: 15px;
margin-right: 8px;
width: 12px;
}
So now we’ve got a new custom control bar – let’s add the functionality. Thanks to the Media Element API this is pretty straightforward. We create listeners and functions for the buttons and use jQueryUI to help us with the seek and volume sliders. Note on line 54 that we can show our custom controls now that the functionality is set up.
//create vars for each video player element to add listeners to
var $video_container = $t.parent('.video-player');
var $video_controls = $('.video-controls', $video_container);
var $play_btn = $('.video-play', $video_container);
var $video_seek = $('.video-seek', $video_container);
var $video_timer = $('.video-timer', $video_container);
var $volume = $('.volume-slider', $video_container);
var $volume_btn = $('.volume-button', $video_container);
var $rewind_btn = $('.video-rewind', $video_container);
$video_controls.hide(); // keep the controls hidden
//play-pause function
var vidPlay = function() {
if($t.prop('paused') == false) {
$t[0].pause();
} else {
$t[0].play();
}
};
//play-pause listeners
$play_btn.click(vidPlay);
$t.click(vidPlay);
$t.bind('play', function() {
$play_btn.addClass('paused-button');
});
$t.bind('pause', function() {
$play_btn.removeClass('paused-button');
});
$t.bind('ended', function() {
$play_btn.removeClass('paused-button');
});
//seek slider - requires jquery UI
var createSeek = function () {
if($t.prop('readyState')) {
var video_duration = $t.prop('duration');
$video_seek.slider({
value: 0,
step: 0.01,
orientation: "horizontal",
range: "min",
max: video_duration,
animate: true,
slide: function(){
seeksliding = true;
},
stop:function(e,ui){
seeksliding = false;
$t.prop("currentTime",ui.value);
}
});
$video_controls.show(); //show the video controls
var seekWidth = $t.width() - 150; //adjust width of seek bar for different movie widths
$('.video-seek').width(seekWidth);
} else {
setTimeout(createSeek, 150);
}
};
createSeek();
Our client wanted a button that would rewind 30 seconds when clicked. This was easy to accomplish thanks to the MediaElement API exposing the currentTime property.
//rewind 30 seconds
var rewindThirty = function() {
var rewoundTime = $t.prop('currentTime') - 30;
var posRewoundTime = Math.max(0,rewoundTime);
$t.prop("currentTime",posRewoundTime);
$video_seek.slider('value', posRewoundTime);
};
$rewind_btn.click(rewindThirty);
Finally we remove the native control bar.
$t.removeAttr('controls');
Conclusion
While there are some great jQuery plugins for skinning HTML5 video available, it’s really not difficult to create your own lightweight solution with only the features that you need and with full control over the look and feel.
Here are some more valuable resources on getting started:
Custom Video Controls with jQuery
Custom Controls for HTML5 Video
Custom HTML5 Video Player
Why Architecture
Architecture is not a word often associated with user interface. We talk about user interface (UI) and user experience (UX) design and theory instead. However, when thinking about building a performance driven app, a good UI architecture is as vital as a well-architected data backend and well-written code. This article presents one method of making sure your UI is well-developed and sound.
Recently, we completed a project that targeted a device with very limited hardware specifications. The client’s primary concern was a good user experience and a performant application. This meant instant feedback and no lag on a device that potentially could get bogged down by high demand UI interactions and fancy display elements.
So, we set off to create a solid UI architecture that would be as efficient as possible.
CPU vs. Memory
When discussing performance, there are always two main questions:
- How much memory will the application utilize?
- What will be the CPU utilization of the application?
Of course there are other concerns like the quality of your code and back end architecture, but that is a whole other set of articles and outside the scope of this article.
The first step in a well built UI is defining the capabilities of your target device(s). Does it have a top of the line CPU/GPU? Or is CPU going to be a limiting factor for your app. How much system memory is available? How much memory will be available for your application to utilize?
Once you have answered these questions you will be able to make better UI architecture decisions and profile your application throughout the development cycle. Keeping the two questions above in mind, there are techniques that can be used to keep your UI performant and avoid common CPU or memory hogs.
Our Findings
Vectors or Bitmaps
Using bitmaps will give a performance boost at the expense of memory. Alternatively if memory is the limiting factor, using vectors will help the memory usage.
Since we knew heading into this project that we wanted to use bitmaps because CPU was going to be the most limiting factor, we decided to do some testing and see what kind of memory and CPU hits various drawing methods would have on performance.
The 8 different tests we ran used one of three PNGs:
- A blank image – 4 KB
- A square glow effect – 8 KB
- A rounded glow effect – 8 KB
In the charts below, glow means either the square or rounded glow PNG was used.
Ranked by best processing time (ms).
Ranked by least memory usage (bytes).
Square corners were overall the clear winners in memory consumption and processing time. This was with a bitmap and opaque background as well as with the drawing API. Masks used between 2.5 and 4 times the memory of other drawing methods. They also took the longest processing time.
Filters
In order to prevent the need for filters, work with designers from the beginning. Make sure they understand the design requirements and limitations.
We were able to get the design firm on board from the beginning to create a UI that didn’t require a lot of the problem areas. In this way we avoided design elements that required filters or alpha blending modes.
We also were able to get a design that used as few assets as possible. Most buttons were designed so that hit states were a percentage darker than the up state. Disabled states were lighter. This allowed us to use one graphical asset rather than three.
Tweens
Tweens are a fairly heavy performance hit. Of course we wanted to use tweens. Let’s be realistic, tweens can really add to your user experience.
In order to make sure our tweens were not a huge hit on performance we did several things:
There are already performant tween libraries out there. Most likely someone else has already done it really well, so why not take advantage of it. We decided to use the gTween library from Grant Skinner.
Second, we limited the amount of tweens we used to places where the effect significantly added to the user experience. This is the hardest and most important aspect of building your UI. Just because you can do it doesn’t mean you should. Identifying the places where a tween or graphical effect actually adds to the design is very important.
Finally, we employed a bitmap system. Instead of tweening a display object with all of it’s children, we created a bitmap, tweened the bitmap and then replaced the bitmap with the actual display objects at the end of the animation. Before creating the bitmap we also changed the stage quality to be temporarily higher than the normal StageQuality of the project. Due to the limited nature of the device, StageQuality was normally set to low. In order to get better anti-aliasing and smoothing of our bitmaps we temporarily set StageQuality to high. This resulted in a very smooth animation with no visible lag due to the device’s limited hardware capabilities.
Choosing the Right Display Object for the Job
Choose your display objects wisely. There are a variety of display objects each with varying code weight at the disposal of an ActionScript developer. Try to pick the one that will work best for your needs with the least amount of unnecessary functionality and code weight. A new Shape object will be about half the size of a Sprite or MovieClip. Sprites are just slightly smaller than MovieClips.
You can also choose the lightest built in class and customize it to meet the needs of your application specifically to help keep your application performant.
Transparency
Oh the alpha property. It can be your best friend in design and one of your worst enemies in performance. When flash renders the display list, in order to calculate how a symbol will display it must look at every other object in the display list that is positioned under that symbol. This adds unnecessary strain to the CPU by increasing the processing required on each render.
So how do we avoid having transparency turn into a performance hit? One really great option is the opaqueBackground property. This will make the background of your current display object fill in with whatever color you want. This prevents the renderer from having to look at everything below the current object.
Tip:
Be very cautious with transparency on text. This was one area we did not compromise on. If the text need to be 50% then we determined what color that corresponded to and used TextFormat to reapply the color.
The Event Model
Events and event dispatching can really increase the CPU utilization of your application. When an event is dispatched an event object is instantiated which can be a CPU intensive process. Then, events that bubble like MouseEvent must check for listeners for every object in the current display list hierarchy. In other words the object that was actually clicked and all of its parent objects. This also consumes CPU resources. This is an issue that affects many areas of an app, but is especially relevant to UI that handles user interactions.
There are several things that can be done. For custom events, avoid bubbling events if possible. For display events like MouseEvent, you can manage the mouseEnabled and mouseChildren properties. By disabling mouse events for objects that are not currently available for interaction, you reduce the number of events dispatched. In addition, in the bubble phase of a display event, you can call stopImmediatePropagation or stopPropagation on the event in order to reduce the number of objects it checks against for event listeners.
Avoid Build-in Event Dispatching
Another option to consider is using Robert Penner’s Signal Library. We don’t usually use signals to replace native events, such as mouse events. However it is an option. We do use signals extensively to replace custom events because they are more efficient. With Signals we can get away with less object instances being generated on user interaction which results in lowered memory consumption. Also, object instantiation is a very costly process and lessening the need for it results in less taxing CPU utilization throughout the runtime life of the application.
Putting it all together
So at this point, you are probably wondering what the point of using ActionScript is if all the great display tools are hard on performance? One thing to remember is that this project was only targeting one device with limited hardware. Often, you app won’t have so many constraints on performance for your target platforms and can take advantage of all the great display tools ActionScript provides. However the performance enhancements that we employed show that Flash can still be used to create rich and powerful user interfaces regardless of a target device(s) resources or lack of resources.
The key is to analyze the use case of the application and find the best balance between performance and retaining a great look and feel for your app. That is why UI architecture is as important as any other part of your application. Hopefully the examples above will help you when analyzing the UI architecture of your next Flash application.
This list is by no means exhaustive. These are simply some of the top issues and impactful fixes that we had in the development process of one application we’ve built. For further study, a great resource is the Optimizing Performance for the Adobe Flash Platform white paper which presents a detailed account of performance enhancements when working with Flash.
It’s that time of year when the excitement at RealEyes for the Adobe MAX conference begins to brew. And this year looks like it’s going to be another exciting and inspiring event filled with great networking opportunities and motivating sessions. Our very own David Hassoun will be passing his knowledge onto all those interested in Adobe Primetime and the Primetime Media Player. If you want to hear about Adobe’s next generation of online video offering, make sure to not miss this session!
Creating a Multiscreen Video Player for Adobe Primetime
Wednesday, May 8th @ 9:30 AM
Learn how Adobe Primetime Media Player provides the foundation classes and services to deliver premium video to Flash, iOS, and Android enabled screens, satisfying the needs of audiences today who view content on many devices and expect high-quality video, delivered immediately, within a unique user experience.
Join community leader David Hassoun, founder of RealEyes, as he walks through video player development techniques that:
- Monetize with seamless ad insertion
- Implement rich video analytics
- Optimize content for devices on mobile networks
- Protect HD content with Adobe Access
Sign up here to register for the session.
In addition to David’s Presentation, the RealEyes team will strongly be represented in attendance by many of our developers. Make sure and follow them on twitter and get their thoughts of the conference in real time.
- David Hassoun @hotkeys
- John Crosby @jccrosby
- Noah Gietka @getitdesign
- Jun Heider @coderjun
- Elyse Koker @elysekolker
- Jordan Pagels @designerjordan
- Nils Thingvall @turbidwater
See you there!
Sample Files
Late-Binding Audio
OSMF 1.6 and higher supports the inclusion of one or more alternative audio tracks with a single HTTP video stream. This practice, referred to as “late-binding audio”, allows content providers to deliver video with any number of alternate language tracks without having to duplicate and repackage the video for each audio track. Users can then switch between the audio tracks either before or during playback. OSMF detects the presence of the alternate audio from an .f4m manifest file, which has been modified to include bootstrapping information and other metadata about the alternate audio tracks.
The following article will guide you through the process of delivering an alternate language audio track to an on-demand video file (VOD) streamed over HTTP using the late-binding audio feature. You should be familiar with the HTTP Dynamic Streaming (HDS) workflow before beginning. Please refer to the following articles on HDS for more information:
Getting Started
To get the most out of this article, you will need the following:
- Flash Player 10.1 or higher download the latest Flash Player here
- Flash Media Server 4.0 or higher with Apache 2.2 installed, and configured for HTTP Dynamic Streaming download Adobe Media Server here
- Flex 4.6 SDK or higher download the latest Flex SDK here
- OSMF 1.6 or higher download the latest OSMF build here
- The LateBindingAudio_VOD.fxp Flash Builder project (included in the exercise files)
- A video encoded to a .f4v, .flv, or other .mp4-compatible format (or use “Obama.f4v”, included in the exercise files)
- A separate audio track encoded to a .f4v, .flv, or other .mp4-compatible format (or use “Spanish_ALT_Audio.mp4”, included in the exercise files)
Overview
After completing this article you should have a good understanding of what it takes to stream on-demand video with alternate audio tracks over HTTP. At a high level, this process includes:
- Packaging your media files into segments and fragments (.f4x)
- Creating a master (set-level) manifest file (.f4m)
- Editing the media tags for the alternate audio tracks within the master .f4m to prepare them for late-binding audio
- Uploading the packaged files to Adobe Media Server
- Including a cross-domain.xml file on the server if the media player is hosted on a separate domain from Adobe Media Server
- Playing back the video using the master .f4m as the video source, and switching between audio tracks using OSMF
Packaging the media files
When streaming media using HDS, files first need to be “packaged” into segments and fragments (.f4f), index files (.f4x), and a manifest file (.f4m). Adobe Media Server 5.0 or later can automatically package your media files for both normal on-demand and live streaming with the included Live Packager application (live), and JIT HTTP Apache module (vod). However, in order to achieve late-binding audio, the manifest for the main video file needs to be modified so that it includes information about the alternate audio tracks.
To package media that supports late-binding audio, you use the f4fpackager, a command line tool built into Adobe Media Server. The f4fpackager accepts .f4v, .flv, or other mp4-compatible files, and is located in the rootinstall/tools/f4fpackager folder within Adobe Media Server.
Next, you will use the f4fpackager to create packaged media files. You can use your own video and audio assets for this step, or you can use the “Obama.f4v” (video), and “Spanish_ALT_Audio.mp4”(alternate audio) included in the exercise files.
Running the f4fpackager
The process for packaging media files on Windows and Linux is similar:
- From command line target the f4fpackager executable for execution in the [Adobe Media Server Install Dir]/tools/f4fpackager (Windows), or set the LD_LIBRARY_PATH to the directory containing the File Packager libraries (Linux).
2. Enter the name of the tool, along with any arguments. For this example, you’ll only need to provide the following arguments for each input file:
- The name of the input file
- The file’s overall bitrate (Alternatively, you could add this information manually later)
- The location where you’d like the packaged files to be output (If you omit this argument, the File Packager simply places the packaged files in the same directory as the source files)
3. Run the packager on the main video .f4v file. At the command prompt, enter arguments similar to:
C:\Program Files\Adobe\Adobe Media Server 5\tools\f4fpackager\f4fpackager –-input-file=”C:\Obama.f4v” –-bitrate=”546” –-output-path=”E:\packaged_files”
4. Next, run the packager again, this time to package the alternate audio track:
C:\Program Files\Adobe\Adobe Media Server 5\tools\f4fpackager\f4fpackager –-input-file=”C:\Spanish_ALT_Audio.mp4” –-bitrate=”209” –-output-path=”E:\packaged_files”
5. Click “Enter” to run the f4fpackager
*Note: individual media files are packaged separately, meaning you run the packager once for the main video file, “Obama.f4v”, and then again for the alternate audio file, “Spanish_ALT_Audio.mp4”
Figure 1.0: Packaging media with the f4fpackager tool
You should now see the packaged output files generated by the File Packager in the directory you supplied to the arguments in step #2. Packaging the source media included in the exercise files should output:
- Obama.f4m
- ObamaSeg1.f4f
- ObamaSeg1.f4x
- Spanish_ALT_Audio.f4m
- Spanish_ALT_AudioSeg1.f4f
- Spanish_ALT_AudioSeg1.f4x
Creating the “master” manifest file
Next, you will modify “Obama.f4m” to create a master (set-level) manifest file that will reference the alternate audio track.
- Using a text editor, open the file “Spanish_ALT_Audio.f4m”
Note: If you skipped the previous section on packaging media, you can use the manifests included with the exercise files in LBA/_START_/PackagedMediaFiles_START)
2. Copy everything within the bootstrapInfo and media tags in “Spanish_ALT_Audio.f4m”.
<bootstrapInfo
profile="named"
id="bootstrap4940">
AAAB+2Fic3QAAAAAAAAAGQA...
</bootstrapInfo>
<media
streamId="Spanish_ALT_Audio"
url="Spanish_ALT_Audio"
bitrate="209"
bootstrapInfoId="bootstrap4940">
<metadata>
AgAKb25NZXRhRGF0YQgAAAAA....
</metadata>
</media>
Figure 1.1: Copy the bootstrapInfo and media tags from the alternate audio manifest file
3. Paste the bootstrapInfo and media tags into “Obama.f4m” to reference the Spanish language track.
<?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0">
<id>Obama</id>
<streamType>recorded</streamType>
<duration>133</duration>
<bootstrapInfo
profile="named"
id="bootstrap4744">
AAAAq2Fic3QAAAAAAAAABAAAAAPoAAA....
</bootstrapInfo>
<media
streamId="Obama"
url="Obama"
bitrate="546"
bootstrapInfoId="bootstrap4744">
<metadata>
AgAKb25NZXRhRGF0YQgAAAAAAAhk....
</metadata>
</media>
<bootstrapInfo
profile="named"
id="bootstrap4940">
AAAB+2Fic3QAAAAAAAAAGQAAAAPo....
</bootstrapInfo>
<media
streamId="Spanish_ALT_Audio"
url="Spanish_ALT_Audio"
bitrate="209"
bootstrapInfoId="bootstrap4940">
<metadata>
AgAKb25NZXRhRGF0YQgAAAA....
</metadata>
</media>
</manifest>
Figure 1.2: Paste the bootstrapInfo and media tags from the alternate audio .f4m into the main video’s manifest file to create the master .f4m
4. Add the following attributes to the media tag for the Spanish language track within “Obama.f4m”:
- alternate=”true”
- type=”audio”
- lang=”Spanish”
<media streamId="Spanish_ALT_Audio" url="Spanish_ALT_Audio" bitrate="209" bootstrapInfoId="bootstrap4940" alternate="true" type="audio" lang="Spanish">
Figure 1.3: Edit the alternate audio’s media tag to prepare it for late-binding audio
In the above step, alternate=”true”, and type=”audio” allow OSMF to parse through “Obama.f4m” and see that there is alternate audio available. Logic within the included example player, which you’ll be using to play the video in a later step, uses lang=”Spanish” to populate a dropdown menu with the available alternate audio stream.
5. Save the file “Obama.f4m”. This is now the master manifest file, and it will be what you will reference to play the video and alternate audio content with OSMF.
Upload the packaged files to the server
6. Next, you will need to upload all of the packaged files to a folder within the webroot/vod directory of Adobe Media Server. On Windows this default location is C:\Program Files\Adobe\Adobe Media Server 5\webroot\vod. Later on you will point OSMF to the master manifest within this directory in order to play the video.
Figure 1.4: Upload all of the packaged media files to a location within the webroot/vod directory of Adobe Media Server
Verify the delivery of the .f4m file
At this point, all of the packaged files should be uploaded to a directory on the server within /webroot/vod. It’s a good idea to test whether-or-not the server is delivering the manifest file properly, and you can do that by entering the path to the .f4m file in the address bar of a browser.
To test the delivery of the manifest, open the .f4m in a browser from the webserver. On a local development machine the URL would be something like:
http://127.0.0.1/vod/Obama.f4m
If you’ve entered the URL correctly, and the server is properly serving up the .f4m, you should see the manifest’s xml. Notice the alternate audio’s media and bootstrap info tags you added earlier, as well as the additional attributes in the media tag:
*Note: Safari will not display XML by default
<manifest xmlns="http://ns.adobe.com/f4m/1.0"> <id>Obama</id> <streamType>recorded</streamType> <duration>133</duration> <bootstrapInfo profile="named" id="bootstrap4744"> AAAAq2Fic3QAAAAAAAAABAAAAAPoAAAAAAACB3QAAA… </bootstrapInfo> <media streamId="Obama" url="Obama" bitrate="546" bootstrapInfoId="bootstrap4744"> <metadata> AgAKb25NZXRhRGF0YQgAAAAAAAhkd… </metadata> </media> <bootstrapInfo profile="named" id="bootstrap4940"> AAAB+2Fic3QAAAAAAAAAGQAAAAPoAAAAAAACCwAAAAAA… </bootstrapInfo> <media streamId="Spanish_ALT_Audio" url="Spanish_ALT_Audio" bitrate="209" bootstrapInfoId="bootstrap4940" alternate="true" type="audio" lang="Spanish"> <metadata> AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJh… </metadata> </media> </manifest>
Figure 1.5: Verify that the server is delivering the .f4m properly by entering the path to the manifest in your browser’s address bar
*Note: The above example URL does not point to “/hds-vod” like it would for HDS content that needs to be packaged just-in-time as the client application requests it. This is because “/hds-vod” is a location directive for Apache that tells the server to look for content in the /webroot/vod directory, and package it for delivery. The jithttp module in Apache responsible for just-in-time packaging isn’t needed for this example, as the source files have been already packaged manually.
Include a cross-domain policy file (Optional)
In order to access content from Adobe Media Server using a Flash-based media player that is hosted on a separate domain from the server, the player needs permission in the form of a cross-domain policy file hosted on the server. Below is an example of a cross-domain policy file that allows access from any domain. You may want to use a more restrictive cross-domain policy for security reasons. For more information on cross-domain policy files, see Setting a crossdomain.xml file for HTTP streaming.
Figure 1.6: Include a crossdomain.xml file in the webroot directory of Adobe Media Server
- Open “crossdomain.xml” from the LBA/_START_/ folder in the included exercise files in a text editor.
- Examine the permissions, and edit them if you wish to limit access to include only specific domains.
- Save the file, and upload “crossdomain.xml” to the /webroot directory of Adobe Media Server.
Test the video using an OSMF-based media player
Now it’s time to test the video and the alternate audio streams using the included sample media player. The sample player is provided as a Flash Builder project, and is based on the LateBindingAudioSample application that comes as part of the OSMF source download(OSMF/samples/LateBindingAudioSample). You can find the included sample player in LBA/_COMPLETED_/LateBindingAudio_VOD.fxp in the exercise files folder.
- Import the file “LateBindingAudio_VOD.fxp” into Flash Builder and run the project.
- Enter the URL of the master manifest located on your server in the “.f4m source” field.
- If the player can see the path to the .f4m file, the Play button will be enabled, and the alternate languages dropdown menu will show a Spanish audio option.
- In no particular order, click “Play”, and choose “Spanish” from the alternate languages dropdown menu.
- The video should start to play, and you should see “Switching Audio Track” being displayed in the player under the languages menu.
- The audio should switch to the Spanish track, while the video continues to play normally.
Figure 1.7: Play the media within the included sample application. Enter the URL for the manifest file and click Play. Use the language selection dropdown menu to switch to the alternate audio stream.
Where to go from here
This article covered the steps necessary to attach late-binding audio to an HTTP video stream using a pre-built OSMF-based media player. You should now have a good understanding of how to package media files for delivery over HTTP, as well as what needs to be done on the server side to deliver late-binding audio. In the next article, you will learn all about the media player, and the code within it that makes late-binding audio possible.
In addition to on-demand video, OSMF supports late-binding audio for live video, multi-bitrate video, and DVR. For more information about HTTP Dynamic Streaming, late-binding audio, and OSMF, please refer to the following resources:
To kick off the year, it seemed only appropriate to write about something that we at RealEyes are overly, awkwardly, nerdily, perhaps even a little disconcertingly passionate about – Web Video and the Players that play it.
There are nearly 30 players that are currently available for implementation. Given the state of development, and the various licenses and pricing options it’s quite a challenge to find the perfect one for your project. A list of the currently viable video players can be found here.
With (almost) all the major browsers having shifted to a very quick release cycle, there are updates coming all the time these days. While this is absolutely good for the state of HTML5/CSS3 implementation and a number of other things, it unfortunately doesn’t seem to be getting us that much closer to one good, truly universal option for video yet. Flash of course will work in all major desktop browsers and render consistently across all of them, assuming that it’s installed. Once you start considering mobile though, things are not nearly as simple. Flash will probably work on a desktop, but the cool kids are all switching to HTML5 <video> tags, iOS devices are awfully picky, and Android has it’s own set of peculiarities. Rumor has it that Ubuntu phones will be entering the market later this year, and they will almost certainly have their own quirks.
With all of that in mind, how do you gracefully deliver video to all those possible browsers and devices with a consistent video player interface, and still keep your life as simple as possible?
This is where the current cornucopia of cross-browser video players comes in. Using one of the many players available, it is possible to create a consistent experience that will play everywhere with only one or two video source files and a very manageable amount of code.
Getting down to business, let’s introduce some of the players that seem the most promising, based on their ease of use, range of features, and developer-friendly licenses. Keeping those points in mind, here are a few of our recommendations.
MediaElement.js
MediaElement.js (or MEjs) was created by John Dyer, and provides an impressive number of playback solutions, including HTML5 Video, Flash, and Silverlight. It’s ease of skinning and setup, wide range of delivery options, and very friendly license, make it a perennial favorite, and it is endorsed by the likes of Chris Coyier and Paul Irish.
Easy to install and setup
Getting MEjs up and running is as simple as downloading the current release, adding the required Javascript and CSS files to your site, and calling them with the following code:
In the <head>:
<script type="text/javascript" src="jquery.js"></script> <script type="text/javascript" src="mediaelement-and-player.min.js"></script> <link href="mediaelementplayer.css" rel="stylesheet" />
In the <body> where you’d like to place your video:
<video width="320" height="240" poster="poster.jpg" controls="controls" preload="none">
<source src="myvideo.mp4" type="video/mp4" />
<source src="myvideo.webm" type="video/webm" />
<source src="myvideo.ogv" type="video/ogg" />
<object width="320" height="240" classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0">
<param name="flashvars" value="url=/wp-admin/myvideo.mp4&poster=/wp-admin/poster.jpg" />
<param name="src" value="http://www.realeyes.com/wp-includes/js/tinymce/plugins/media/moxieplayer.swf" />
<param name="allowfullscreen" value="true" />
<param name="allowscriptaccess" value="true" />
<embed width="320" height="240" type="application/x-shockwave-flash" src="http://www.realeyes.com/wp-includes/js/tinymce/plugins/media/moxieplayer.swf" flashvars="url=/wp-admin/myvideo.mp4&poster=/wp-admin/poster.jpg" allowfullscreen="true" allowscriptaccess="true" />
<img src="myvideo.jpg" width="320" height="240" title="No video playback capabilities" alt="" />
</object>
</video>
And finally, start it all up with this (outside of the <head> or some versions of iOS will have issues):
// jQuery method
$('video').mediaelementplayer();
// normal JavaScript
var player = new MediaElementPlayer('#player');
Easy to skin
MEjs comes packaged with 4 skins already made, one default, and three others that emulate popular existing players. If you would like to customize it beyond that, all of the skinning is handled with CSS so it’s very easy to create your own. There’s not a lot of documentation provided, but the CSS for the included skins is very clear and very well laid out, so it can easily provide the reference you need.
VideoJS
VideoJS was created by Steve Heffernan along with Zencoder Inc. and has ties to the Brightcove CDN. This means that while it is wholly open source, and can be used entirely separate from anything related to them, it plays really nicely with their services, should you happen to be using them.
Fairly easy skinning
Skinning VideoJS is almost as easy as skinning MEjs, but unfortunately, the way it’s configured, you will either have to refactor the class names applied to video player elements, or make edits directly to the default CSS file, which isn’t optimal. On the other hand, VideoJS definitely scores points for how thoroughly commented their un-minified CSS file is. You certainly won’t have to guess at which values will effect which elements.
Lightweight
Another gold star for VideoJS is how lightweight it is, even including a Flash Player (many of the other available options that didn’t make our list offer the capability, but require you to add a Flash Player separately). While it lacks the Silverlight support offered by MEjs, that’s probably going to be a minimal concern in most use cases.
Ease of install
Given VideoJS’s support from Brightcove, there’s a CDN hosted option available, making it entirely possible to use without even needing to download it. One point to keep in mind though, is that if you want to use https, you’ll have to self-host instead of using the CDN version. Assuming for the moment though, that you’re using the CDN version, install is as simple as adding the following code to your page:
In the <head>:
<link href="http://vjs.zencdn.net/c/video-js.css" rel="stylesheet" /> <script type="text/javascript" src="http://vjs.zencdn.net/c/video.js"></script>
And then in the <body>:
<video id="my_video_1" width="640" height="264" controls="controls" preload="auto" poster="my_video_poster.png" data-setup="{}">
<source src="my_video.mp4" type="video/mp4" />
<source src="my_video.webm" type="video/webm" />
<object id="my_video_1" width="640" height="264" classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0">
<param name="src" value="http://www.realeyes.com/wp-includes/js/tinymce/plugins/media/moxieplayer.swf" />
<param name="flashvars" value="url=/wp-admin/my_video.mp4&poster=/wp-admin/my_video_poster.png" />
<param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="true" />
<embed id="my_video_1" width="640" height="264" type="application/x-shockwave-flash" src="http://www.realeyes.com/wp-includes/js/tinymce/plugins/media/moxieplayer.swf" flashvars="url=/wp-admin/my_video.mp4&poster=/wp-admin/my_video_poster.png" allowfullscreen="true" allowscriptaccess="true" />
</object>
</video>
Honorable Mentions
Both of these players are well regarded, though they are not open source, and while they have free options available, they require a Commercial License to remove the parent company’s branding:
- Flowplayer
- JW Player
So depending on your needs, they shouldn’t be ruled out, but when there are such good Open Source options available… *refer to any open source software forum and insert compelling, well thought out point and/or tirade here*
Media Encoding
Finally, while we’re not going to go in depth this particular article, if you have questions about encoding your video in multiple formats for cross-platform delivery, here are a few options that will have more information on their respective sites:
And the newly announced
Amazon Elastic Transcoder (beta)
This is just a quick overview of what’s available now, and some of our picks. There are, of course, new things popping up all the time that we’ll be checking out. Even with the two players we addressed, there are whole books that could be written about their implementation and styling.
RealEyes Swap Player
Last but not least, we have developed our own player for use by our clients. We may be just a touch biased, but we think it’s one of the best available options, and we work really, really hard at keeping it ahead of the game.
This sample player is based on the excellent SMP (Strobe Media Playback) media player that can be downloaded and used for free.
One of the great things about the SMP player is that it is so easy to embed and configure. Using SWFObject you simply point to the SMP SWF and pass the url of your media. You can pass other configuration variables but these are limited, which can be frustrating.
The latest SMP release (2.0) has a very cool new feature – a JavaScript API. This gives the developer with some JS experience the ability to control the player.
In our example we have a basic SMP player which we have extended using the JS API. You’ll notice the ability to play/pause/switch media and the indicators for current state and runtime. Thanks to the JS API we can access the MediaPlayer Class without using any ActionScript code.
For our current state and runtime indicators we simply set up event listeners and the corresponding functions:
player = document.getElementById(playerId);
player.addEventListener(“mediaPlayerStateChange”, “onMediaPlayerStateChange”);
function onMediaPlayerStateChange(state, playerId)
{
var newstate = player.getState();
document.getElementById(“currentState”).innerHTML = newstate;
}
The play/pause button uses a simple onclick event handler:
document.getElementById(“play-pause”).onclick = function(){
var state = player.getState();
if (state == “ready” || state == “paused”) {
player.play2();
}
else
if (state == “playing”) {
player.pause();
}
return false;
};
Our media switcher re-embeds the SWP swf so that we can pass in a new poster frame URL.
function changeSrc(url,purl)
{
// Create a StrobeMediaPlayback configuration
var parameters =
{ src: url
, autoPlay: false
, controlBarAutoHide: false
, poster: purl
, javascriptCallbackFunction: “onJavaScriptBridgeCreated”
};
// Embed the player SWF:
swfobject.embedSWF
( “../StrobeMediaPlayback.swf”
, “strobeMediaPlayback”
, 640
, 480
, “10.1.0″
, {}
, parameters
, { allowFullScreen: “true”}
, { name: “strobeMediaPlayback” }
);
}
Our sample player only scrapes the surface of the powerful JS API. (This example illustrates how a completely JS-driven control bar can be created.) The SMP team has done a good job of making the power of the MediaPlayer accessible while doing some smart things like making calls from StrobeMediaPlayback to javascript asynchronous and the reverse calls synchronous optimize performance.
Want to learn more?
Adobe Media Server 5 (formerly known as Adobe Flash Media Server) is now officially released and shipping. Along with a name change that clearly demonstrates its ubiquity across all platforms, AMS 5 has a great list of improved and new features that are available right out of the box. Coupled with its existing feature set, AMS 5 picks up right where FMS 4.5 left off and continues its delivery of media to multiple platforms with several protocols that can fit a large number of needs.
Here is a complete list of the Adobe Media Server 5 release features:
Expanded media streaming options
- Protected RTMP (pRTMP)
- 24/7 live streaming support
- Adobe Access key rotation and output protection (Adobe Access license required)
- Protected HTTP Dynamic Streaming for Adobe Flash and AIR (PHDS)
- Protected HTTP Live Streaming for Apple iOS (PHLS)
- Adaptive bitrate manifest support
- Multi-protocol manifest generator/pre-packaging tools (Flash and iOS)
- Adobe Access 4 DRM ready
- 608/708 closed captioning support
Enhanced communication features
- On-Demand stream packaging for HTTP (Flash and iOS)
- Audio extraction for HLS (required for Apple App Store approval)
- SIP Gateway support with G711 coding in Flash Player
- High quality audio/video capture support
- Scalable P2P introduction services for Flash
- Multicast ingest and recording
Server platform
- Robust HTTP Media Origin services
- Advanced disk management for HTTP (Flash and iOS)
- Optimized Server configuration for live HDS
- Native 64-bit only
The Adobe Media Server 5 product line has the following editions available:
Adobe Media Server 5 Standard (formerly known as Adobe Flash Media Streaming Server) Even though it’s entry level, don’t let that fool you. With AMS 5 Standard, you can broaden your reach to more audiences by streaming RTMP and HTTP to Adobe Flash Player compatible and iOS devices.
Ideal for:
- Internal video delivery on a small scale
- One-way broadcasting
- Video bloggers
- Educators
Adobe Media Server 5 Professional (formerly known as Adobe Flash Media Interactive Server) The proverbial “Flagship” of the product line, AMS 5 Professional is both a robust and affordable solution. Leverage protected HTTP delivery internally and/or externally to Flash Player compatible and iOS devices and scale to fit needs. Target larger audiences and deliver DRM protected content with Adobe Access to implement a single workflow solution.
Ideal for:
- Large broadcasters running HTTP origin services
- Broadcasters who need DRM protection over HTTP to all devices
- Large enterprise deployments with multicast support
- Small to medium sized businesses
Adobe Media Server 5 Extended (formerly known as Adobe Flash Media Enterprise Server) Leverage a large scale, unified communications network and deliver with high-quality voice and video capabilities to SIP-enabled devices through the integrated Adobe Flash Media Gateway. Add massive scalability and reduce network congestion with P2P capacity for large enterprise broadcasts to several locations over several networks.
Ideal for:
- VoIP network services integration enabling traditional calls with Adobe Flash applications
- Massive-scale communication applications
- Large enterprises with global live delivery
- Immediate and complete media delivery control and content protection
Adobe Media Server 5 Starter (formerly known as Adobe Flash Media Development Server) In limited capacities, leverage all of the features available in all editions for Proof of Concept and development. Not to be used for production.
Ideal for:
- Evaluation purposes only
- Proof of Concept
- Application level development
- Testing and/or staging
Let RealEyes help you determine which Adobe Media Server 5 edition meets your immediate and/or long term needs, and how to get started.
Adobe Media Server 5 begins shipping June 21st!
As some of you may or may not know, Adobe Media Server 5 was officially announced last month. This blog post from Kevin Towes (FMS/AMS Product Manager at Adobe) also gives a little more insight:
http://blogs.adobe.com/ktowes/2012/05/announcing-adobe-media-server-5-and-adobe-access-4.html
To better clarify, the Adobe Media Server product line (formerly known as Flash Media Server) began shipping June 21st 2012, in a staggered release format. The Linux version for new licenses is available immediately. The Windows version and upgrades will ship later this year. There are very few resellers able to offer the new Adobe Media Server 5 at this time, and it just so happens that RealEyes is one of them, so contact us for further details and questions.
It is also important to note that all Flash Media Server products will be getting a name change with this release as well. To avoid any confusion, the name changes are as follows:
- Adobe Flash Media Development Server → Adobe Media Server 5 Starter
- Adobe Flash Media Streaming Server → Adobe Media Server 5 Standard
- Adobe Flash Media Interactive Server → Adobe Media Server 5 Professional
- Adobe Flash Media Enterprise Server → Adobe Media Server 5 Extended
Some of the features of this version release include:
- Dynamic copy protection for Flash, Android and now Apple (DRM)
Reach a wider audience with a premium video experience consistently across devices, TVs and desktops with a single infrastructure to protect with a single protection solution.
- Simple publishing workflows for protected HTTP streaming
Use the same source media and live streams to deliver and protect full adaptive bitrate experiences to Adobe Flash, Android and Apple devices to help reduce storage and infrastructure costs.
- Standalone offline packaging utilities for HDS and HLS
New HLS packaging utility with integrated encryption to prepare your media content and lock it down with a wide variety of protection options including DRM with Adobe Access 4
- On-Demand stream packaging
Publish faster, reduce storage costs and save time by publishing video once with full adaptive bitrate support.
Contact us if you would like to learn more about the Adobe Media Server.
Adobe Media Server 5 –
Deployment, Management
and Delivery
RealEyes is putting on an Adobe Media Server (AMS) 5 training program this June. This custom course was created from our 7+ years of working with FMS and AMS servers and their use in real-world deployments. Come join us for our inaugural AMS 5 class this June on the 18th and 19th.
With AMS 5 due out on June 21st, you can be ready for all that the new tool has to offer the day it comes out, and hit the ground running with your new deployment.
Do you currently have an FMS 4.5 or older deployment? Come get up to date on the newest features and functions and be ready to bring your deployment up to AMS 5.
To view our class and register go to: Deployment, Management and Delivery
You can always reach out to us at info@realeyesconnect.com or 303-433-1216 if you have any questions.
RealEyes Media & Why HTTP Dynamic Streaming
RealEyes Media is a developer lead digital consultancy who focuses on on digital media delivery and software application development, consultation, & training. RealEyes builds interactive software solutions specializing in streaming media. If you need a solution on the web, desktop, mobile or connected device we have the expertise to realize your vision.
In the current technical environment it has become evident that media delivery across devices & platforms is in a state of transition and growth. This growth includes not just the devices and users that are consuming the media, but the technologies and transports that are providing the media across the board. Technologies like HTTP Dynamic Streaming (HDS) are blazing the trail that will ultimately lead the consumer to their content. Currently the capabilities provided by HDS provide developers and content providers with the broad access that is required to reach different devices and markets.
What is HTTP Dynamic Streaming?
HTTP Dynamic Streaming or HDS gives the content provider the ability to deliver packaged streaming media over the HTTP protocol to multiple platforms while saving on server infrastructure, bandwidth and administration costs. HDS can also remove the need need for expensive and or high maintenance media servers. The recorded and live content is packaged in a fragmented format allowing for the video to be delivered to the consumer in a streaming, seek-able, and multi-bitrate format. In addition to the high level of delivery options, HDS can be packaged with DRM to protect the media and can be deployed to CDNs to allow for even greater user experience while consuming media.
HTTP Dynamic Streaming Limitations
The work we have been doing with HDS packaged content and delivery has opened the doors to extend the use and integration of HDS packaged content. There are two stumbling blocks that we have encountered with the current state of HDS. Both problems center around the reliance on web server & web server extension to deliver the HDS packaged content. HDS requires an Apache web server & the HTTP Origin Module to deliver the media content to client players. The (simplified) playback sequence for HDS content is:
- The client requests an .f4m file that provides the location and initial data for playback.
- From that data the client player then determines what segment/fragment to request for the current timecode.
- The Apache web server hands this request off to the HTTP Origin Module to extract the needed F4V fragment data from the segment from on the web server and returns that to the web server.
- The Apache web server then provides the response to the client player.
- The client player then parses the F4V content into a playable format and plays it back.
The second stumbling block is playback of HDS content from a local file system. Currently there is no way to extract the F4V fragment data from the packaged segment files. These two use cases highlight the current deployment and usage limitations for HDS with the reliance on Apache and the Origin module. Removing the reliance on Apache and the Origin module not only allows for a greater number of and more flexible deployment options, but can simplify deployments and allow for more cost effective storage and delivery.
Removing the Limitations
The first step to extending the delivery options for HDS content is to get the packaged content into a format that doesn’t require the HTTP Origin Module. This means extracting the fragments from the segment files. RealEyes has created a tool set that can extract the fragments from an HDS packaged file set. The fragments are extracted in the exact same manner as the HTTP Origin Module meaning that the reliance on Apache and the Origin module has been removed. Once the fragments have been extracted, they can be deployed to any server that can respond to HTTP requests. This has been tested with Amazon S3, IIS and Apache (without the origin module installed) and has worked without issue.
The next step is local playback of HDS content. Once we have the fragments, we can play that content back, but for long-form media content this can mean hundreds of fragment files. A set of ActionScript3 libraries have been created to allow for the download and packaging of HDS content for local playback. The download can be accomplished from normal HDS delivery or from HDS content that has had the fragments pre-extracted. As the fragments are downloaded they are written to disk into a file format that allows for interrupted downloads as well as playback during download on windows and mac operating systems.
Local playback while downloading is accomplished through an additional ActionScript3 library created specifically for read and write operations on multiple files asynchronously and has been tuned for high performance. Playback has been tested with short-form video content as well as feature length video content. Great pains were taken to work within the file formats and specifications so things like DRM still work even with content that has had fragments extracted, downloaded and saved locally for playback.
A visualization of the download and playback for HTTP Dynamic Streaming content.
What can you do with it?
Some of the use cases that are possible with the extended deployment and use with these tools and libraries include:
- Local playback of HDS content
- Simplified content deployment
- Content deployment to cost effective services (Amazon S3)
- Simultaneous download and playback of long-form video
- Live HDS Streaming and download for DVR-like for local playback
- Bandwidth and storage space savings
What is Next?
Plans for the toolset include creating web services for fragment extraction, updating the libraries and tools to make sure they work with OSMF 2.0 (currently they have been tested on OSMF 1.6). Although HDS is a powerful and flexible way to deliver content, we are continuing to embrace formats and specifications that increase the reach and availability of digital media. We are an MPEG-DASH promoter and plan to work with this format heavily. We are always looking to the next development and hope to be part of the innovators pushing media delivery to the next level.
Follow Us!