RealEyes Media

Mobile Flex: View Data

Posted on May 16, 2011 at 9:05 am in Development

From the previous post you should know how to navigate from 1 view to the next using the ViewNavigator.  Now, you want some data in that view right? No problem, this is where the View object’s ‘data‘ property comes into play. Setting the data property is accomplished by passing the data object, in addition to the View’s class name, into the pushView() method on the navigator object.

Example:

navigator.pushView(MyNewView, dataObject);

This effectively calls the setting for the data property of the new View (MyNewView) object that is created.

Managing View Data

You could work with the data property on the View object directly. For instance, if the data object passed into the View via the pushView() method was a simple user object that contained a name property, you could bind the name property to a label control.

Example:

<s:Label id="name_lbl" text="{data.name}" />

Overriding the Data Property Setter

Usually though, you’d want to override the setter for the data property. Then you can type your object and work with it in a better manner.

Example:

protected var user:User;
override public function set data(value:Object):void
{
 super.data = value;
 user = value as User;
}

<s:Label text="{user.name}" />

So now we’ve got the data in the view. The next step is to manage the state of each view. With mobile apps you can’t count on the view staying around, so we’ll need to keep a tight control on the state of each view. That way we can bring the user right back where they expect to be when they come back to the app after a call for example. In the next post we’ll look into how to do this. Stay tuned.

Foundations in Digital Audio-Part 1: Introduction to the Series

Posted on March 30, 2011 at 11:51 am in Development, Media Solutions

RealEyes is pleased to announce the launch of a new series dedicated to making sense of the vast world of digital audio. This series will consist of an ever-expanding collection of posts that aim to provide information about digital audio principles in the form of articles and tutorial-walkthroughs. Topics to be covered include:

• Acoustic-to-Analog-to-Digital conversion
• Soundwaves
• Db
• Nyquist Theorem
• Sample rate/bit depth
• Recording best-practices
• Editing
• Audio effects
• DAW’s (digital audio workstations)
• Exporting to different filetypes
• MIDI
• Audio for the web
• Audio and Actionscript
• Much more!

…Stay tuned!

For a recent project, RealEyes was asked to transcribe the speech from a video provided to us by a client. The transcribed text was to be included in the video and presented in subtitle fashion. To complete this task, we relied on the new speech-to-text capabilities that are available in Adobe Premiere CS4, and Soundbooth CS4. We also utilized some motion graphics firepower with After Effects CS4, (with the help of a customized expression, courtesy of Dan Ebberts’ excellent Developer Connection article, XMP metadata in Creative Suite 4 Production Premium).

In this tutorial, we’ll walk through the process of transcribing speech from a Quicktime video file (.mov) using Adobe Premiere CS4, and creating subtitles in the video from the transcription using Adobe After Effects CS4. We’ll incorporate the use of an expression, courtesy of Dan Ebbert, (slightly modified to suit our needs), that is needed to display the transcribed text in our video.

In order to get the most out of this tutorial, you’ll need the following:

• Adobe Premiere Pro CS4
• Adobe After Effects CS4
• Expression courtesy of Dan Ebbert
• BarackInLax.mov (original source video “Mp4″ saved as “.MOV”)

Creating the Transcription in Adobe Premiere CS4

Create a New Project:

1. Start Adobe Premiere Pro.
2. From the Welcome screen, select New Project.
3. In the New Project panel, specify a name and location for the project.
4. Click OK.
5. In the New Sequence dialog box, choose DV-NTSC/Standard 48kHz.
6. Click OK.

Import Your Video Clip:

1. Choose File > Import, then navigate to the “BarackInLax.mov” clip.
2. Double-click the video file to import it into Premiere.
3. Once the file is in the Project panel in Premiere, double-click it again to open it in the Source Monitor.
4. Use the transport controls in the Source Monitor to preview the video for a few seconds. Be sure to pay close attention to the audio during the preview (so that you can later tell whether-or-not Premiere did a good job with the speech transcription.)

Open Metadata View:

1. Choose Window > Metadata and check to see that the Metadata View is enabled.
2. The Metadata View should have Three main sub-headings: “Clip”, “File”, and “Speech Transcript”. If you import a video clip into your project that has been previously transcribed, that transcription metadata will be displayed in the “Speech Transcript” panel. Of course, we haven’t transcribed our clip yet, so this panel is blank for now.

Create the Transcription:

1. Make sure that your video clip is selected and highlighted in the project panel, and click “Transcribe” in the Speech Transcript panel of the Metadata View.
   
   The Metadata View – Click to Enlarge
   
   
2. A dialog box appears, giving you a few options. Choose the proper language setting based on your video’s speech content.
3. Choose between high quality processing(slower), and medium quality processing(faster).
4. If your video clip contains multiple speakers, you can choose to check the “identify speakers” checkbox.
5. Click OK.
   
   Transcription Dialog
6. Transcriptions done in Adobe Premiere and Adobe Soundbooth are handled by Adobe Media Encoder. Adobe Media Encoder automatically launches when you complete the transcription dialog.
7. Your video clip should load into Adobe Media Encoder’s queue automatically, with the appropriate format and preset for speech transcription already set.
8. Click “Start Queue”.
9. Adobe Media Encoder will now encode your clip, adding the speech transcription metadata to the video file.
   
   Adobe Media Encoder (Video Asset Loaded By Premiere) – Click to Enlarge

Working With The New Transcription Metadata

Correcting Mistakes In The Speech Transcript
At this point you should see the transcription metadata in the Speech Transcript panel of the Metadata View in Premiere. It is likely that the outputted text doesn’t quite match what was said in the video clip so some editing may be in order. Single-clicking on any word in the transcribed text highlights that word, and displays some potentially valuable metadata at the bottom of the Speech Transcript panel. Information about when the selected speech occurred in the timeline, as well as the duration of that text, is displayed.

By double-clicking on a word in the transcription, you can edit the text to match what was actually said in the video. Right-clicking gives you more options. For example, often you’ll find that you’ll need to replace two words with one, or insert a missing word before or after another word, etc. You can use the options available to you by right-clicking on a word in the transcription to make these kinds of edits. This can be a tedious task, as it involves meticulously going through the video content to match it with the transcribed speech output. For this reason, we haven’t edited our example transcription for accuracy, but for the purposes of this tutorial, it will work fine.

Video Transcription Text (Needs Editing) – Click to Enlarge

Adding The Transcribed Speech To Your Video With After Effects CS4

Create a New Project:

1. Open After Effects CS4.
2. Close the Welcome Screen dialog box.

Import Your Transcribed Video Clip and Create a New Composition

1. Choose File > Import, then navigate to the “BarackInLax.mov” clip.
2. Drag the newly imported clip over the “Create a New Composition” icon at the bottom of the project panel. This automatically creates a new composition with settings that match those of your video clip.
3. Adobe Media Encoder has included the new transcription metadata in the video file itself, so you should now see that it appears in the timeline. (you may need to adjust the timeline zoom in order to see the text clearly)
   
   Setting Up a New Composition in After Effects – Click to Enlarge

Create a New Text Layer to Display the Transcription Text

1. Select the Horizontal Type Tool from the tool panel and click on your video in the Composition panel in the area where you’d like the transcription text to be displayed.
2. Make sure the Paragraph panel is open, and with the new text layer still selected, ensure that “Center text” is selected.
3. At this time, feel free to set the font style and font size for your text layer. (You can also hold off on this step if you’d rather wait to see how the transcribed text looks after it’s displayed in your video).
   
   New Text Layer Created – Click to Enlarge

Using an Expression to Display the Transcribed Speech
An “expression”, in After Effects, is a small piece of software that you use in your project to efficiently control a single layer property. By adding an expression to a layer, you can create animations that might have otherwise required you to set hundreds of individual keyframes. Expressions written for After Effects are based on basic JavaScript. You can find more information about using expressions here, here, and here.

Modifying the Sample Expression
In order to make Dan’s expression work for our project, we need to change a couple of lines of code. First, we need to make sure that the expression references our video, and not the one Dan used in his example.

1. Change the line that reads: L = thisComp.layer(“Legato_Ames_BTS_02_trans.mov”);to this: L = thisComp.layer(“BarackInLax.mov”);so that it references our video clip.
2. Next, take a look at this line of code: max = 5; // number of words to displayThis tells After Effects to display the transcription with a maximum of Five words at a time. That works for us, so we’ll keep it as-is. However, if you’d like a different setting for your project, you can make that adjustment here.
3. Finally, let’s look at this bit of code toward the end of the expression: s += L.marker.key(i).comment + ” “;Part of what this line does is to determine the spacing between words in the displayed text in the video. You can control how much space goes between each word by manipulating the whitespace between the quotation marks at the end of the statement. Currently there are two spaces between the quotes, which may or may not spread your displayed text out too much, depending on what you’re going for. For our project, we left only one space between the quotation marks, and that seemed to work well.
4. Confirm that the necessary changes have been made to the expression, and save.
   
   The Expression With Adjustments Made For Current Project – Click to Enlarge

Prepare the Text Layer for the Expression
The expression will be copied and pasted into a single property of the text layer in the timeline.

1. First, open the text layer’s property panel by clicking on the sideways triangle next to the layer name, and then again on the sideways triangle next to the “text” property to open this property.
2. Alt-click (Windows) or Option-click (Mac OS) the stopwatch icon to the left of the Source Text property’s name.
3. Notice that doing this reveals the property “text.sourceText” in the timeline.
4. Make sure that “text.sourceText” is highlighted, and replace the current setting with that of your edited expression by pasting the expression over “text.sourceText”.
   
   Timeline Ready To Have Expression Added – Click to Enlarge

Observe the Transcribed Speech in Your Video Clip
Move the Current Time Indicator to a spot on the timeline that contains transcribed text. If everything worked correctly, you should see the transcribed text displayed in the video in the Composition panel. At this time, you may wish to make adjustments to your text layer’s font size, etc., to optimize the display of text in your video.

The Transcribed Text Shows Up in The Video – Click to Enlarge

Moving On

The speech-to-text capabilities found in Adobe Premiere CS4, and Adobe Soundbooth CS4 represent a nice leap forward in terms of what’s now possible when working with video. Transcribing a video’s speech is just one of the many new possibilities that we have. The ability to search for text within a video, and to use this metadata in custom applications being two more exciting prospects that come to mind.

Please consider the following resources to learn more about using metadata in your multimedia projects:

• XMP metadata in Creative Suite 4 Production Premium Excellent article written by Dan Ebbert on metadata workflows, and using expressions in After Effects.
• MotionScript.com Dan’s site. Great resource for learning about expressions, scripting, and animation.
• After Effects Help Page (expressions) Comprehensive guide to using After Effects.
• Premiere Pro Help Page (speech to text metadata) Comprehensive guide to using Premiere.