Development

Recently I was challenged with creating a series of nested pie-charts from a single set of data as a Proof of Concept for a client. A colleague suggested using the D3 javascript library and it has proven itself extremely useful for its ability to transform data into visual SVG elements with a minimum of coding. The real magic is in D3’s ‘selectAll’ method – as we will see!

The Final Product

Here’s the completed POC: http://jsfiddle.net/RealEyesDev/7tvfrvh9/ There are 4 pie-charts, each one composed of 3 individual pie-charts (appearing as rings), each with its own style but representing the same data.

The Data

This is dataset that we use to build the charts.
// Define the data
var dataset = [
[[8,6], [8,6]],
[[25,10], [25,10]],
[[45,15], [45,15]],
[[8,6], [8,6]]
];

The HTML

<div id="chart-container"></div>
This is it! This DIV represents the container where our charts will live – D3 will do the rest!

The Magic!

Let’s create our 4 main SVG charts from the dataset. Why 4? Because there are 4 top-level elements in the dataset. Let’s see how D3 uses its built-in selectAll function to loop through the dataset and join the data to new SVG elements.

var svg = d3.select("#chart-container").selectAll("svg")
.data(dataset)
.enter().append("svg:svg")
.attr("width", (r + m) * 2)
.attr("height", (r + m) * 2)
.attr("class", function(d, i) {return i})

SelectAll is a very powerful function. First it’s important to understand that it will look for matching DOM elements (like jQuery). So our initial selection d3.select("#chart-container") finds our div. The subselection, .selectAll("svg") finds no match. Wait – what? Yes, we are selecting elements that don’t exist yet!

This is important because the next 2 lines .data(dataset) and .enter().append("svg:svg") tell D3 that we want to create (append) new SVG elements – one for each top level element in our dataset – and join our data values to these new SVG elements. This is very powerful – and we’re not done with our selectAll function yet!

Next, let’s add some attributes to each of our dynamic SVG objects.
.enter().append("svg:svg")
.attr("width", (r + m) * 2)
.attr("height", (r + m) * 2)
.attr("class", function(d, i) {return i})

The first 2 attributes are fairly straightforward. We’re giving our SVGs a width and height calculated by a radius and margin value set elsewhere. The last line, however, is a bit more interesting. It has a function that uses two values, ‘d’ and ‘i’. What? Well ‘d’ represents the data value that we’ve joined to this object and ‘i’ is the index. What we’re doing is simply giving each SVG object a CSS class – ‘0’ for the first element, ‘1’ for the next, etc… This will come in handy later if we need to perform any DOM manipulation on specific objects.

Let’s go one step beyond!
//ADD A YELLOW OUTER CIRCLE AND G ELEMENT TO EACH SVG
.each(function(d) {
d3.select(this).append("svg:g")
.attr("class","container-g")
.attr("transform", "translate(" + (r + m) + "," + (r + m) + ")")//;
.on("click", zoomIt)
.each( function(d) {
buildPies(this, d);
});

Still within our selectAll function, we use the ‘each’ function to add a ‘g’ element for each SVG, give it a class, translate its origin to the center, attach a click handler and, finally, build a pie-chart based on the related data value. Whew!

<div id="chart-container">
<svg width="220" height="220" class="0"><g class="container-g" transform="translate(110,110)"></g></svg><svg width="220" height="220" class="1"><g class="container-g" transform="translate(110,110)"></g></svg>...etc...etc....

Here’s what we have so far. Our empty chart-container div has evolved considerably. The next step is to build the pie-charts with our buildPies function which we’ve just called and passed in our data for each SVG.

//FUNCTION TO BUILD OUR 2 RINGED INNER PIES
function buildPies(n,d){
var piePan = d3.select(n);
piePan = piePan.selectAll("g")
.data(function(d) { return d; })
.enter().append("g") //CREATE EACH INNER AND OUTER RING
.attr("class", function(d, i, j) {return "ring-"+i+j;});
var path=piePan.selectAll("g")
.data(function(d) { return pie(d); })
.enter().append("path")//MAKE THE SLICES
.attr("fill", function(d, i, j) { return z((i+1)*j); })//i); })
.attr("stroke-width", function(d, i, j) {return j;})
.attr("class", function(d, i, j) {return "slice-"+i+j;})
.attr("d", function(d, i, j) {
j = (j%2===0) ? 1 : 0; //WE ONLY WANT TO KNOW IF THIS IS INNER OR OUTER RING
return arc.innerRadius(87*(j*1))
.startAngle(function(d) { return d.startAngle + Math.PI; }) //ROTATE PIE TO BEGIN AT 6:00
.endAngle(function(d) { return d.endAngle + Math.PI; }) //ROTATE PIE TO BEGIN AT 6:00
.outerRadius(80+(cwidth*(j*1)))(d);
});

Again we are using D3 selectAll functions to append pie-charts to our SVG G elements. You might have noticed that the dataset seemed to contain duplicate values: [[8,6], [8,6]]. This was intentional and tells D3 that we want 2 pie-charts in each SVG (including more sets of values would create more charts). We do this by using the ‘data’ function like this: .data(function(d) { return d; }) and appending a G element with a unique CSS classname. Next we select each G element and create the pies using D3s built-in pie-chart method: d3.layout.pie(). Next each pie gets sliced based on the data values and each slice given a dynamic colored fill.

Wrap Up

There is more code here worth exploring (using d3.scale to set dynamic colors, the zoom function etc…) but I wanted to focus on the power of the D3 selectAll function. Once you wrap your head around how it joins data it becomes a very powerful tool and opens the door to the limitless visualizations that have made D3JS so popular. Just take a look at these examples to get inspired!

Wanted – Entry-level Front-end JavaScript Developers

Posted on October 28, 2014 at 11:12 am in Blog, Development

If this sounds like you, please take a sec to send us your resume: http://realeyes.com/frontenddev Contract, internship, part-time, and full-time will all be considered.

OVERVIEW

A couple weeks ago we started off our series of posts exploring the current availability, options, and capabilities of streaming Alternate Audio Tracks with HTML Video – In case you missed it, be sure to check out Part 1 for a high-level overview.

In this week’s post, we’ll be continuing on with a more in depth look at the alternate audio track options available in Playlist-based Video Formats (HLSHTTP Live Streaming, HDSHTTP Dynamic Streaming, MPEG-DASHDynamic Adaptive Streaming over HTTP), with a focus on HLS – the format created by Apple.

PLAYLIST-BASED VIDEO FORMATS

Playlist-based formats have become a popular choice for serving HTML5 video in recent years, with HLS being the preferred format defined by Apple for delivering video to both web-based and native applications.

One of the strongest selling points of playlist-based formats, is the flexibility and efficiency they offer. As opposed to a more traditional format, such as a simple MP4 (more advanced encoding options are possible), which would contain the full video at a single bitrate to be served as a progressive download, playlist-based formats at their top-level, are simply manifests containing a list of available media segments, with the possibility of defining various playback options.

I won’t go into the full details of bitrates and playback decisioning here, since that goes well beyond the scope of a single article, but as far as it pertains to our topic of Alternate Audio Tracks, let’s examine this HLS sample.

#EXTM3U

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac_sinewave/prog_index.m3u8"

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="en",URI="subtitles/eng_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/fra/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="fr",URI="subtitles/fra_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="es",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/spa/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="es",URI="subtitles/spa_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="ja",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/jpn/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語 (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="ja",URI="subtitles/jpn_forced/prog_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=263851,CODECS="mp4a.40.2, avc1.4d400d",RESOLUTION=416x234,AUDIO="bipbop_audio",SUBTITLES="subs"
gear1/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=28451,CODECS="avc1.4d400d",URI="gear1/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
gear2/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=181534,CODECS="avc1.4d401e",URI="gear2/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
gear3/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=297056,CODECS="avc1.4d401f",URI="gear3/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=1030138,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1280x720,AUDIO="bipbop_audio",SUBTITLES="subs"
gear4/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=339492,CODECS="avc1.4d401f",URI="gear4/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=1924009,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1920x1080,AUDIO="bipbop_audio",SUBTITLES="subs"
gear5/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=669554,CODECS="avc1.4d401f",URI="gear5/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=41457,CODECS="mp4a.40.2",AUDIO="bipbop_audio",SUBTITLES="subs"
gear0/prog_index.m3u8

This is an m3u8 playlist: the filetype associated with HLS streams, provided by Apple. You’ll notice that the two highlighted lines both declare a TYPE of AUDIO, along with matching GROUP-IDs and LANGUAGE declarations. They start to differ once reaching the NAME declaration, BipBop Audio 1 and BipBop Audio 2 respectively. From there, the properties declared are AUTOSELECT and DEFAULT, which in this case tell a video player that BipBop Audio 1 should be automatically selected at the start of playback, and noted in selection dialogs as the default audio track. You will notice that the property declarations for BipBop Audio 2 contain one property more than BipBop Audio 1, which is URI. The value of which, actually points to another m3u8 playlist, using a path relative to our initial m3u8.

The m3u8 playlist definitions for the default Audio Track (BipBop Audio 1) are called out as the Audio corresponding to each of our defined Video Bitrates.

The various video bitrate definitions (called out in lines 18, 22, 26, 30, and 34) in turn point to their own m3u8 playlists which contain segment information for the video at that bitrate. The video playlist corresponding to RESOLUTION=1920×1080 contains the following segment information:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-I-FRAMES-ONLY
#EXTINF:3.003,
#EXT-X-BYTERANGE:119944@564
main.ts
#EXTINF:3.003,
#EXT-X-BYTERANGE:88548@679996
main.ts
#EXTINF:3.003,
#EXT-X-BYTERANGE:118628@1328972
main.ts
#EXTINF:3.003,
#EXT-X-BYTERANGE:88360@2007652
main.ts
#EXTINF:3.003,
#EXT-X-BYTERANGE:117876@2658132
main.ts
#EXTINF:3.003,
#EXT-X-BYTERANGE:87044@3335308
...
...
...
main.ts
#EXTINF:3.003,
#EXT-X-BYTERANGE:122764@409462496
main.ts
#EXTINF:1.2121,
#EXT-X-BYTERANGE:106784@410170692
main.ts
#EXT-X-ENDLIST

If we examine the contents of the BipBop Audio 2 playlist alternate_audio_aac_sinewave/prog_index.m3u8:

#EXTM3U
#EXT-X-TARGETDURATION:11
#EXT-X-VERSION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:9.98458,
#EXT-X-BYTERANGE:86920@0
main.aac
#EXTINF:10.00780,
#EXT-X-BYTERANGE:136595@86920
main.aac
#EXTINF:9.98459,
#EXT-X-BYTERANGE:136567@223515
main.aac
#EXTINF:10.00780,
#EXT-X-BYTERANGE:136954@360082
main.aac
#EXTINF:10.00780,
#EXT-X-BYTERANGE:137116@497036
main.aac
#EXTINF:9.98458,
#EXT-X-BYTERANGE:136770@634152
...
...
...
main.aac
#EXTINF:9.98458,
#EXT-X-BYTERANGE:144309@25679417
main.aac
#EXTINF:10.19356,
#EXT-X-BYTERANGE:147569@25823726
main.aac
#EXT-X-ENDLIST

You will see that it contains a set of segment definitions for an AAC file. which correspond to the video segment definitions laid out in the various video bitrate’s m3u8 files.

While the secondary m3u8 playlist defined for BipBop Audio 2 should in fact work as a standalone HLS stream, it only contains definitions for an audio track, and as such will not play back any video if it is defined as the source for a tag.

In the instance of this playlist, the way that HLS approaches these two audio streams, is that BipBop Audio 1 is the audio originally included with the video tracks which are defined in our root m3u8 playlist. In the event that the alternate is selected, the original (default) audio track will be ignored, in favor of BipBop Audio 2 – in order for this to work correctly, it is necessary to ensure that the timing of the two tracks is identical during the encoding process, so that proper syncing can be handled by the player.

ACCESSING ALTERNATE AUDIO TRACKS

Unfortunately, at this time, as best as I can determine, the only option for accessing Alternate Audio Tracks in an HLS stream is through the iOS native video player. This is slightly different from the player available inline, and is reached when you play back any HTML video on iPhone, or activate fullscreen video on iPad.

iOS7 vs. iOS8

Unfortunately, the release of iOS8 (and iOS 8.02) have a bug which, for the time being, causes Alternate Audio Tracks to not be displayed in the Tracks List of a video.

WORKAROUNDS AND ALTERNATE METHODS

It is possible to set up a workaround which will synchronize the playback of two media sources, and through that, fake the availability of Alternate Audio Tracks which can be accessed via desktop browsers, but that’s a slightly different approach, and does not make use of the internally defined Alternate Tracks of an m3u8.

OTHER FORMATS

HDS and MPEG-DASH both operate on the same principles as HLS, and while their alternate audio tracks seem to be slightly easier to access – my research suggests that you can append a URL variable to define which track of an HDS or MPEG-DASH resource is played, thus far, neither format is used as prominently as HLS.

STAY TUNED

In the next part of this series, we will examine the relatively new WEBM format which is a container formate, and the options available for Alternate Audio Tracks in that paradigm.

HTML Video Check-in – iOS 7 vs. iOS 8

Since iOS 8 went live on the 17th and I updated a few of my devices over the weekend, I decided to do some quick testing of web video playback. I wanted to see if there were any little, undocumented changes that would affect our custom, cross-platform video player, or our general approach to working with HTML video – like the changes to exiting fullscreen video that came in the update from iOS 6 -> iOS 7. 1

Overall, things seem pretty much the same between iOS 7 -> iOS 8, and in a quick runthrough, REPlayer looks to be working just fine.

Cannot Access Alternate Audio Tracks

One interesting change to note, especially since it relates directly to our current series on Alternate Audio Streams in HTML Video, is that the native interface (iOS default controls used when video is fullscreen) for selecting Sub-Title/CC tracks – or Alternate Audio tracks when they’re available – no longer seems to recognize/display the audio tracks in iOS 8.

iOS7 vs. iOS8

Sub-Title selection still works just fine, but the Audio Section (and Audio Tracks) do not display in iOS8. We confirmed this by verifying our test m3u8 still contains Alternate Audio tracks in the manifest. Viewing the same video on a device running iOS7 will display, and allow the selection of, both Sub-Title and Audio Tracks, while iOS8 will only display the subtitle tracks.

Off the bat, I’m assuming this is a bug, not a feature, and it will be addressed in future updates, though it could also be a result of the transition from QTKit to AVFoundation as the new iOS Media Framework. 2
One other possible cause for the discrepancy, is the different versions of WebKit used between the two. 3

As of this writing, this does not seem to be a known issue according to the release notes.

Stay Tuned

Be sure to check back on Wednesday 10/1 as we continue our series on Alternate Audio Tracks in HTML Video – addressing some of the options and implementations available for providing user-selectable alternate audio streams using various formats, and suggest solutions for reaching the widest number of browsers and devices.

This week we’ll be featuring an in-depth writeup of alternate audio in HLS and other playlist-based formats.


Notes and non sequiturs
1

In iOS6 – when you switched to fullscreen video, there were 2 options available for exiting fullscreen:

  • One was to tap the “Exit Fullscreen” icon in the lower right side of the control bar (Two arrows on a diagonal that were pointing inwards towards each other – the inverse of the icon used to enter fullscreen)
    • This would exit fullscreen, and maintain the current playback state of the video, i.e., if the video was playing in fullscreen, it would continue to be playing after leaving fullscreen – if the video was paused in fullscreen, it would remain paused after leaving fullscreen
  • The other was to tap on the text-button “DONE” in the upper left of the fullscreen interface
    • This would exit fullscreen and pause the video, regardless of current playback state

In iOS7 – the “Exit Fullscreen” icon was removed, and the only option was to use “DONE” – this meant that whenever you exited fullscreen in iOS7, the video would be paused every time. Meaning that an extra tap on the Play Button was necessary in order to resume playback.

2

AVFoundation was added in iOS 7 and existed alongside QTKit, though developers were strongly encouraged to make the switch – Have not yet found explicit documentation of the status/availability of QTKit in iOS8

3
  • User Agent String of an iPhone 5S running iOS 8.0 reports WebKit v600.1.4
    • Full User Agent String –
      Mozilla/5.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12A365 Safari/600.1.4
  • User Agent String of an iPhone 5S running iOS 7.1 reports WebKit v537.51.2
    • Full User Agent String –
      Mozilla/5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D167 Safari/9537.53

OVERVIEW

As streaming video continues to mature and become a more widely used medium for delivering major events, it is becoming more important to not only deliver a clear, reliable video, but to begin augmenting it with some of the extra possibilities it affords.

Multiple Language Tracks

One of those extras is the possibility of including alternate audio tracks that can be selected during playback. These can be used to offer video in multiple languages, provide a commentary track, and in many other creative ways.

We’ve previously written about some of the options available for Captions and Subtitles – That article is available here.

While there are many different HTML Video formats currently in use, most of which offer some form of support for multi-track audio, there is unfortunately not yet any simple, straightforward solution for providing a consistent experience across browsers and devices.

This post is intended as the overview to a series looking more in depth at the capabilities of various formats and some of the current options for providing user-selectable multi-track, or alternate track audio with HTML video.

DESKTOPS AND DEVICES

As with HTML Video in general, one of the biggest challenges in delivering content is the wide variety of browsers and devices in use today, and the various ways in which they approach media playback.

One of the intended features of the HTML5 <video> element is the optional inclusion of various <track> tags nested within the element –

<video>
 <src>primary-source.m3u8</src>
 <src>alternate-source01.webm</src>
 <src>alternate-source02.mp4</src>

 <track id="enCaptions" kind="captions"
 label="English Captions"
 src="captions/sintel-en-us.vtt" />

 <p>final fallback message</p>
</video>

This property can be a little mis-leading as an option for providing alternate audio tracks. Although the naming, and a quick glance at a sample can give the impression that this would be a particularly straightforward way of defining audio options, in all of my research it seems to be solely intended for text tracks (closed captions, sub-titles, chapter information, etc.).

DESKTOP BROWSERS

Overall, the recent developments in modern browsers has been great for HTML video. While there is still much fragmentation as to which browser will play which format, all of the major modern browsers have grown a lot of functionality for the handling of HTML Video in the last few years.

While the media spec is far from finalized, many of it’s core concepts are available, and will allow you to inspect and manipulate your video elements through JavaScript.

In general, through the HTML5 Media APIs you should have access to the properties currentSrc, audioTracks, textTracks, and videoTracks among others – though that will unfortunately depend on which video format is being used.

One more small hurdle in providing alternate audio streams, is that in my experience the browser’s default HTML video control bar does not provide users with an interface method of accessing the alternate streams. In order to offer alternate streams, you will need to either create a custom control bar, or use a player that provides that option.

In the case of some formats, such as HLS, alternate audio streams may only be available via the device’s (as opposed to the browser’s) native player.

DEVICE BROWSERS

Device browsers offer just as many, if not more challenges than desktop browsers when it comes to streaming video. Along with the fractious state of currently available devices and their preferred formats, content in the browser usually has more limitations than what is possible in a native app. Even the native browser of most devices only has access to a subset of the playback capabilities and options available in a full app.

Third party browsers in iOS are even consigned to using a WebView as opposed to the full WebKit implementation used by iOS Safari. Meanwhile, although Android is generally listed as supporting HLS, actual playback results are spotty at best.

VIDEO FORMATS

There is a wide variety of video and audio file formats in use today, this is a quick overview and rough system of classification of some that this series will attempt to address in more detail.

HLS, HDS, and MPEG-DASH (Playlist Based)

HLS (HTTP Live Streaming), HDS (HTTP Dynamic Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP) are playlist based formats which provide the <video> element with a playlist of media fragments and options, rather than with a media file directly. By their very nature, they are designed to provide playback instructions for multiple options and variations of media. This means that the file size of the initial playlist stays very small regardless of how many bitrates or alternate tracks are included.

Example of an HLS m3u8 playlist

#EXTM3U

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac_sinewave/prog_index.m3u8"

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="en",URI="subtitles/eng_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/fra/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="fr",URI="subtitles/fra_forced/prog_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
gear2/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=181534,CODECS="avc1.4d401e",URI="gear2/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
gear3/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=297056,CODECS="avc1.4d401f",URI="gear3/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=1030138,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1280x720,AUDIO="bipbop_audio",SUBTITLES="subs"
gear4/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=339492,CODECS="avc1.4d401f",URI="gear4/iframe_index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=41457,CODECS="mp4a.40.2",AUDIO="bipbop_audio",SUBTITLES="subs"
gear0/prog_index.m3u8

While these do allow for responsive playback in their supported browsers and video players, and can easily contain many audio and video tracks, they have an interesting side effect when it comes to browser-based playback – As the media file isn’t being directly provided to the <video> element, attempting to inspect or manipulate the audioTracks or videoTracks properties will generally return undefined.

Selecting Alternate Audio Streams

In regards to HLS specifically, although it can offer a number of alternate audio tracks to access easily through the native iOS player, in a browser (Safari >6.0 on Mac is the only desktop browser to currently support HLS), or even the desktop version of Quicktime, those tracks are essentially unreachable. Ironically, one of the best options for desktop-based HLS playback is to route it through a Flash player, such as JWPlayer Premium and Enterprise Flash-based HLS or OSMF. This will not only allow for HLS playback in browsers other than Safari, but seems to be the only option outside of the native iOS player for accessing HLS alternate audio streams.

WEBM (Container Format)

WebM is an open media format which defines the file container structure to provide VP8 compressed video and Vorbis compressed audio streams. Although the general idea is similar to the playlist-based options above, in that alternate tracks can be defined and included, a WebM file generally contains the media to be played within itself, so file size can quickly become a consideration when including alternate tracks.

That said, WebM files are intended specifically for web-based playback, and as such, tend to play very nicely with the spec of the Media APIs, at least in initial testing in Chrome. WebM files will generally return the expected results when querying audioTracks, textTracks, or videoTracks.

STAY TUNED

In upcoming posts we’ll go more in depth with some of the options and implementations available for providing user-selectable alternate audio streams using various formats, and suggest solutions for reaching the widest number of browsers and devices.

Sublime Text – Search Strings and Filtering

Posted on September 10, 2014 at 3:52 pm in Blog, Development

Sublime Text Tips

Searching with Regular Expressions & Filtering Search Scope

For use in Sublime Text’s “Find In Files…” Dialog (⇧⌘F)

Find in Files Dialog Panel

Sublime Text offers many, many options for navigating within your project’s code or searching within either a single file, or the entire codebase of the project. The “Find” Menu offers over a dozen different options for Find and Replace, and depending on which you choose, there are 3 separate “Find” Dialog Panels available.

In this post, I’m going to show a few quick, introductory tips for using the “Find in Files…” Panel. Searching from this Panel will allow you to search across all files in your project, or (as we’ll cover in the second section) select a sub-set of files for a quicker search with more relevant results.

Regex Search Strings

Regular Expressions (which I’ll often refer to as “regex”) offer some incredible capabilities for searching text of any sort. While the material covered in this post is largely beginner level regex, a full introduction to Regular Expressions is far beyond the scope of a single article – I would recommend a resource such as RegexOne.com if you aren’t familiar with Regular Expressions at all.

The actual syntax of regex can vary slightly between coding languages (PHP and JavaScript for instance), but for this article’s purposes Sublime Text uses Perl Regular Expression Syntax from the Boost library. Full documentation can be found at boost.org, which you will want to reference when creating your own search expressions, or modifying and extending the examples offered below.

It’s important to note that in order to use these strings in your searches, you will need to enable Regular Expressions in search by ensuring that the “Regular expression” button (functions as a checkbox) is selected. The appearance can vary depending on the theme you are using with Sublime Text, but generally the button displays either “*” or “.*” as an icon.

Enable Regex Button
While the appearance of these Option Buttons can change depending on the theme in use, I do not recall ever seeing a Sublime Theme change the location or order of the buttons.

These are some useful templates for regex strings to use in the “Find:” field.

Search “Near”

Frequently, I end up searching for code that I have a rough memory of, but not one specific enough to quickly come up with a single, efficient thing to search for. However, I can almost always remember the general shape and structure of things well enough to come up with a pretty unique pair of things to search for that should only occur in close proximity to one another a few times throughout the project.

Variation 1 – Very Basic

// Variation 1 - Full String
\b[[term-1]]\W+(?:\w+\W+){[[1]],[[10]]}?[[term-2]]\b

// Breakdown of regex
  \b                  // Opening word boundary
          [[term-1]]      // Search Term 1
  \W+                 // Any NON-alphanumeric character (\W) with one or more repetitions (+)
  (?:                 // Open non-marking sub-expression
    \w+               // Any alphanumeric character (\w) with one or more repetitions (+)
    \W+               // Any NON-alphanumeric character (\W) with one or more repetitions (+)
  )                   // Close non-marking sub-expression
  {                   // Open definition of min/max words between search terms
          [[1]],          // minimum quantifier
          [[10]]          // maximum quantifier
  }?                  // Close definition of min/max words between search terms
          [[term-2]]      // Search Term 2
  \b                  // Closing word boundary

// NOTE: Be sure to remove the [[brackets]] when inserting
//        both your search terms and qualifiers

Returns all instances of [[term-1]] that are at least [[1]] but no more than [[10]] words away from [[term-2]] –

  • Just replace [[term-1]] and [[term-2]] with the strings you want to search for
  • [[1]] and [[10]] are quantifiers which define the minimum and maximum number of words allowed between the two search terms to return a match
    • Using [[1]] as the minimum would mean that any instances where the two words were adjacent would not be considered a match
    • You would need to use [[0]] instead
  • NOTE – Be sure to remove the brackets [[ ]] when inserting search terms and choosing quantifiers

Variation 2

// Variation 2 - Full string
\b(?:[[term-1]]\W+(?:\w+\W+){[[1]],[[10]]}?[[term-2]]|[[term-2]]\W+(?:\w+\W+){[[1]],[[10]]}?[[term-1]])\b

// Breakdown of regex
  \b                  // Opening word boundary
  (?:                 // Open non-marking sub-expression
          [[term-1]]      // Search Term 1
    \W+               // Any NON-alphanumeric character (\W) with one or more repetitions (+)
    (?:               // Open non-marking sub-expression
      \w+             // Any alphanumeric character (\w) with one or more repetitions (+)
      \W+             // Any NON-alphanumeric character (\W) with one or more repetitions (+)
    )                 // Close non-marking sub-expression
    {                 // Open definition of min/max words between search terms
          [[1]],          // Quantifier - minimum # of words between search terms
          [[10]]          // Quantifier - maximum # of words between search terms
    }?                // Close definition of min/max words between search terms
          [[term-2]]      // Search Term 2
    |                 // OR Operator
          [[term-2]]      // Search Term 2
    \W+               // Any NON-alphanumeric character (\W) with one or more repetitions (+)
    (?:               // Open non-marking sub-expression
      \w+             // Any alphanumeric character (\w) with one or more repetitions (+)
      \W+             // Any NON-alphanumeric character (\W) with one or more repetitions (+)
    )                 // Close non-marking sub-expression
    {                 // Open definition of min/max words between search terms
          [[1]],          // Quantifier - minimum # of words between search terms
          [[10]]          // Quantifier - maximum # of words between search terms
    }?                // Close definition of min/max words between search terms
          [[term-1]]      // Search Term 1
  )                   // Close non-marking sub-expression
  \b                  // Closing word boundary

// NOTE: Be sure to remove the [[brackets]] when inserting
//        both your search terms and qualifiers

Same as above, but will find the terms regardless of which order they are in –

  • Variation 1 will only return instances where [[term-1]] comes before [[term-2]]
  • Variation 2 essentially just adds an OR and then repeats the search string with the [[term-]]values flip-flopped

Variation 3

// Variation 3 - Full string
\b([[term-1]]|[[term-2]]|[[term-3]])(?:\W+\w+){[[1]],[[10]]}?\W+([[term-1]]|[[term-2]]|[[term-3]])\b

// Breakdown of regex
  \b                  // Opening word boundary
  (                   // Open group of Search Terms
          [[term-1]]      // Search Term N
    |                 // OR Operator
          [[term-2]]      // Search Term N
    |                 // OR Operator
          [[term-3]]      // Search Term N
  )                   // Close group of Search Terms
  (?:                 // Open non-marking sub-expression
    \w+               // Any alphanumeric character (\w) with one or more repetitions (+)
    \W+               // Any NON-alphanumeric character (\W) with one or more repetitions (+)
  )                   // Close non-marking sub-expression
  {                   // Open definition of min/max words between search terms
          [[1]],         // Quantifier - minimum # of words between search terms
          [[10]]         // Quantifier - maximum # of words between search terms
  }                   // Close definition of min/max words between search terms
  ?\W+                // Any NON-alphanumeric character (\W) with one or more repetitions (+)
  (                   // Open group of Search Terms
          [[term-1]]      // Search Term N
    |                 // OR Operator
          [[term-2]]      // Search Term N
    |                 // OR Operator
          [[term-3]]      // Search Term N
  )                   // Close group of Search Terms
  \b                  // Closing word boundary

// NOTE: Be sure to remove the [[brackets]] when inserting
//        both your search terms and qualifiers

Allows you to search for more than just 2 values –

  • Results will be returned if any 2 words in your list are found within the defined proximity of each other
  • This example only adds 1 extra word, but using the syntax illustrated, you could add as many extras as you’d like
    • I wouldn’t recommend going much higher though – for both performance and logic reasons 1
  • As compared to Variation 2 only the list of search terms gets repeated, as opposed to the entire set of search parameters, so that’s better-ish
  • It occurs to me that I’m not 100% certain what will happen if all three search terms are found within the defined range of proximity
    • My guess though, would be that it shouldn’t break anything too thoroughly, you would just end up with a set of results that included each combination listed as a separate match, even though they all pointed back to the same chunk of code 2

I’d be willing to bet that there’s a much more succinct way of accomplishing all this, but these ones work, and I’m still very much in the early stages of learning regex – That being said, if you have any suggestions for improving those strings, I’d love to hear them.


 

Include/Exclude Filters

Filter files and folders to search

These are some useful strings to use in the “Where:” field to make your search run faster, and avoid cluttering your results with, say, the entirety of jQuery in it’s minified format, or all of the libraries you’re pulling in with npm/bower/etc.

Include File Types

*.js,*.tpl.html

Say you’re looking for a style attribute that’s getting added dynamically and throwing off your selectors, because it gets added as an inline style instead of toggling a class – based off of that you know that it’s almost certainly in either a full-blown .JS file, or a script tag somewhere – Combine that with the knowledge that even though our (hypothetical) values are used pretty commonly throughout the CSS, there are probably only a handful of instances that it occurs in the JS. So you could exclude all the CSS files ( -*.css ) from your search and hopefully trim down the number of search results, keeping them more focused (and more likely to be relevant) as well as reducing the number of files being searched by a little bit. However, excluding CSS explicitly will still search every single file of any other file-type.

Exclude Minified Files

-*.min.css,-*.min.js

-OR-

-*.min.*

Exclude Component Libraries

-*/bower_components/*,-*/npm-modules/*

Because there’s (usually) a lot of them, and they’re long but minified – so you still end up with the full code being considered as “context” for your search results. Also, you shouldn’t be editing them anyway.


Conclusion

Although we really just scratched the surface of what Sublime Text and Regular Expressions can do, I hope this quick overview of some of Sublime Text’s more advanced searching capabilities has given you some ideas of how to search more efficiently based on your own preferences, or the needs of a specific project.


 

Tangents and non sequiturs
1

I’m sure it affects overall search performance, and in general, adding more possibilities to match against will result in more matches being returned – which is good, I suppose, if you’ve run a more limited search already and the code you’re hoping for is not showing up – but if you get back a huge list of questionable relevance then that defeats the entire purpose of searching in the first place.

2

So, actually, I bet you could probably write a search with enough parameters, and a large enough range of proximity that (depending on your system specs and the size of your code base) it would in fact hang or crash Sublime Text – but I think you’d have to be trying fairly deliberately to do that

Debugging on Mobile with Weinre

Posted on August 27, 2014 at 10:54 am in Blog, Development

The emulation tools in Google Chrome’s Dev Tools are a great way to do a first round check of your Javascript app’s behavior on tablet and phone. However, there is no substitution for actually testing on the device. Between all the different iOS and Android configurations, not to mention the cloud of other manufacturers and OSes, you’re bound to discover some peculiar behavior that only occurs on the device. What to do?

There are different ways you can plug in and hook up your devices for debugging on your computer. However, iOS has its way, Android has its way, and this approach is as fragmented as the device market. Managing all that is a chore, and often hard to setup. One nice tool that exists for debugging Javascript and HTML cross platforms is weinre. Weinre is a proxy server of sorts that allows you to do remote debugging of your device. You can see the DOM, console output, and even execute Javascript commands. This makes debugging that mysterious mobile bug a bit easier.

Here’s the general steps to setting up:

  1. Install weinre. It’s a server that runs on your dev machine.
  2. Start it up via the command line. Make sure to pick a port for it to run on that doesn’t conflict with other processes running. It defaults to port 8080. If you want to alter the port it is running on, you can use the following command: weinre --boundHost -all- --debug true --httpPort 8081
  3. Put a script embed in your app or page that brings in the script from the server, using the server’s IP address: <script src="http://127.0.0.1:8081/target/target-script-min.js#anonymous"></script>
  4. Visit the server in a browser on your dev machine to make sure it’s running: http://127.0.0.1:8081, for example.
  5. Then when you visit the app on your device, it should show up as a connected user on the weinre server and you can then access Javascript debugging tools.

I found this how-to guide from Nokia to be very valuable in setting up weinre. It has some more in-depth instructions to help get you on your way past the difficulties of debugging on your device and take some of the sting out of mobile development.

Extract: Project Parfait is Now an Official Part of Creative Cloud

A few months ago, in April of this year, Adobe unveiled an experiment called Project Parfait – a web-based service that allows you to upload and view Photoshop Files with a fantastic set of tools for inspecting elements and extracting assets. If you haven’t gotten to check it out yet, I wrote a full Overview and Introduction to Project Parfait which covers its core features and usage in detail.

In the months that Project Parfait has been available, it’s become a fairly regular part of my workflow and is usually the first thing I open to check out new comps. In short, I’ve gotten more attached to it than is generally wise with Betas and experimental projects.

Thankfully, as of today, Project Parfait is becoming an official part of Adobe’s Creative Cloud and will bring it’s full functionality to Creative Cloud Files as a feature called Extract.

Extract in CC

Updates

Since my initial article Project Parfait/Extract has gotten a couple more neat little features added. Besides a lot of little details and refinements, there are a few things worth calling out.

Layer Comps

Project Parfait/Extract will recognize Layer Comps, and allow you select available views of your PSD.

Layer Comps

If you upload a file with Layer Comps defined, you will notice that an extra dropdown selector appears to the left of the size and scaling information just above the viewing area. This will allow you to select any existing Layer Comps and update the view accordingly.

Layer Comp Selector Dropdown

The dropdown selector will only appear for files that have Layer Comps included when they are uploaded, so you will need to make sure that’s done from within Photoshop beforehand.

For a quick intro to creating and using Layer Comps, try this recent video tutorial from Adobe.

Keyboard Shortcuts

There has also been a small set of Keyboard Shortcuts implemented, which open up some handy ways of navigating between elements, allow you to easily zoom in and out, and enable a pixel-level eyedropper tool for extracting very specific color values.

Extract Keyboard Shortcuts

Using Extract in Creative Cloud

To use Extract:

  • Login to Adobe’s Creative Cloud.
  • Go to the Files section and upload a PSD.
  • View the PSD and click on the “Extract” tab.

You should see a familiar interface almost identical to Project Parfait, but, running inside Creative Cloud. From this view, you can also share the PSD with any email address. The recipient of the PSD will not need to be an Adobe Creative Cloud subscriber.

Sharing Functionality

Up until now, even though Project Parfait has been using your regular Adobe ID for login and access, it has not been possible to directly access files in your Creative Cloud storage, and files uploaded to Project Parfait have not been available from within Creative Cloud.

With the direct integration into Creative Cloud as Extract, you should now be able to directly open any PSDs in your storage, or that have been shared with you.

It was also not previously possible to share files uploaded to Project Parfait, without sending the file separately and everyone uploading it to their individual account. With Extract becoming a part of Creative Cloud, all the CC sharing functionality should become available.

One very nice little feature is that if you share a PSD with someone, they will be able to access the Extract features directly from the link, regardless of whether or not they have a Creative Cloud account.

Wrap Up

All in all, it looks like Project Parfait should be making it’s move into the Creative Cloud as Extract in pretty much the same state as it exists now. There should be some nice crossover features added as the Creative Cloud ecosystem continues to grow. According to the current plans, as Project Parfait becomes Extract it should remain more or less complete. In what I’ve seen about the changes, it has been called out that:

  • Users will not lose any functionality from Project Parfait and they will be able to share these PSDs with team members.
  • The service will remain available to all users of Creative Cloud including the free tier of users.

As I understand it, the full features of Project Parfait should be available today from within Creative Cloud as Extract. The Project Parfait service is slated to remain active until sometime in October, so that there’s time to migrate over any files, but after that it will only be available from within Creative Cloud as Extract.

Recently, I was tasked with building a video player that would play live streams via IP Multicast on a supported network and automagically switch to Unicast on an unsupported network. Problem is, with IP Multicast the clients will make a connection and just wait around for data without bombing out. This is because the clients are connected to the IP Multicast address space via their network hardware and not a server endpoint in many other types of streaming.

In the past, this type of configuration might be implemented through a connection timeout in the video player logic. However, I wanted a seamless and immediate way to fall back without making the user have to wait. Enter Apache mod_rewrite.

The general workflow I wanted to follow was this:

  1. The end user hits the video player page on the Apache server
  2. The video player seamlessly and immediately point itself at the right stream.
  3. Everyone’s happy

I accomplished the above with a little mod_rewrite magic in my Apache config.

First, I needed to make sure clients on specific subnets would play back the live stream using Unicast. Second, I needed to properly redirect all other clients to the live stream using IP Multicast. Also, I needed to make sure that VOD requests would be ignored.

Here’s a gist of my rewrites along with some commentary.

Enjoy!

CSS Filters – Current Implementation and usage

Posted on July 16, 2014 at 11:32 am in Blog, Development

OVERVIEW

Almost two years ago, CSS filters were introduced and available as a preview in advance browser versions, such as Chrome Canary. Even using the latest builds, it was still necessary to enable extra flags for the features to work as concisely defined filters. SVG filter effects and Filter Elements can be used to produce similar results (-webkit-filter effects are essentially shorthand presets built using Filter Elements), but for the scope of this article we’ll stay focused on the current -webkit-filter options.

AVAILABLE FILTERS

I’ve created interactive demos of the current filter effects so that you can see them in action and get an idea of the values each one uses. To control the filters, simply drag the slider under each example image. The filter effect, along with the value it’s using, will update in realtime.

In order to play nicely with the data binding used to make things update in realtime, these previews currently only support the implementation for WebKit browsers (Safari, Chrome, Opera) at the moment.

GRAYSCALE

See the Pen CSS Filter – Grayscale by Jordan Pagels (@designerJordan) on CodePen.

SEPIA

See the Pen CSS Filter - Sepia by Jordan Pagels (@designerJordan) on CodePen.

SATURATE

See the Pen CSS Filter - Saturate by Jordan Pagels (@designerJordan) on CodePen.

HUE-ROTATE

See the Pen CSS Filter - Hue-Rotate by Jordan Pagels (@designerJordan) on CodePen.

INVERT

See the Pen CSS Filter - Invert by Jordan Pagels (@designerJordan) on CodePen.

OPACITY

See the Pen CSS Filter - Opacity by Jordan Pagels (@designerJordan) on CodePen.

BRIGHTNESS

See the Pen CSS Filter - Brightness by Jordan Pagels (@designerJordan) on CodePen.

CONTRAST

See the Pen CSS Filter - Contrast by Jordan Pagels (@designerJordan) on CodePen.

BLUR

See the Pen CSS Filter - Blur by Jordan Pagels (@designerJordan) on CodePen.

DROP-SHADOW

See the Pen CSS Filter - Drop-Shadow by Jordan Pagels (@designerJordan) on CodePen.

BROWSER SUPPORT

Browser implementation is still far from standard, but WebKit browsers (Chrome, Safari, Opera) support prefixed filter styles in all current mainstream versions. There are possibilities for implementation in Firefox and IE as well, but the approach is not nearly as straightforward, and there’s a fair bit more code involved. To achieve the same effect across these browsers, you’ll need to use SVG Filters or Filter Element syntax.

It is worth noting that this is entirely different than the Microsoft IE -ms-filter property in terms of implementation and browser handling.

CSS Filter Effects are supported in recent versions of Chrome (18.0+), OSX Safari (6.0+), Opera (15.0+), iOS Safari (6.0–6.1+), and Android Browser (4.4+) as reported by caniuse.com. Firefox is listed as offering Partial Support since version 3.6, although that is using the SVG url() syntax.

IE is listed as not supporting CSS Filter Effects, even into 11.0 - although this MSDN Article from 2011 previews SVG filter implementation that seems to have been slated for IE10.

IMPLEMENTATION

The CSS Filter Effects, as addressed in the examples, are currently implemented in WebKit browsers only and use the following syntax style:

-webkit-filter: [filter name](value);
filter: [filter name](value);

Filter Effects using an SVG filter with the url() syntax are not quite so straightforward and require a separate <svg> element (either in an external file, or included in the page’s HTML) in order to work. The Mozilla Developer Network provides a great article on Applying SVG effects to HTML content.

WRAP UP

Given the wide variance in implementation and guaranteed support, CSS filters (and their cousins) are still a ways from being advisable for any key functionality of a site. That being said, caniuse.com currently lists browser support at almost 65% (includes partial support). With this in mind, CSS filters create some very interesting possibilities for Progressive Enhancement of carefully considered elements.