at&t developer program · wml over cdpd and wap were the only options for delivering media...

AT&T Developer Program

Mobile Media – Taking Advantage of the Latest Multimedia Technology

White Paper

Document Number 1.0

Revision 0.4

Revision Date 09/15/09

© 2007 AT&T Knowledge Ventures

Legal Disclaimer

This document and the information contained herein (collectively, the "Information") is provided to you (both the individual receiving

this document and any legal entity on behalf of which such individual is acting) ("You" and "Your") by AT&T, on behalf of itself and

its affiliates ("AT&T") for informational purposes only. AT&T is providing the Information to You because AT&T believes the

Information may be useful to You. The Information is provided to You solely on the basis that You will be responsible for making

Your own assessments of the Information and are advised to verify all representations, statements and information before using or

relying upon any of the Information. Although AT&T has exercised reasonable care in providing the Information to You, AT&T does

not warrant the accuracy of the Information and is not responsible for any damages arising from Your use of or reliance upon the

Information. You further understand and agree that AT&T in no way represents, and You in no way rely on a belief, that AT&T is

providing the Information in accordance with any standard or service (routine, customary or otherwise) related to the consulting,

services, hardware or software industries.

AT&T DOES NOT WARRANT THAT THE INFORMATION IS ERROR-FREE. AT&T IS PROVIDING THE INFORMATION TO YOU

"AS IS" AND "WITH ALL FAULTS." AT&T DOES NOT WARRANT, BY VIRTUE OF THIS DOCUMENT, OR BY ANY COURSE OF

PERFORMANCE, COURSE OF DEALING, USAGE OF TRADE OR ANY COLLATERAL DOCUMENT HEREUNDER OR

OTHERWISE, AND HEREBY EXPRESSLY DISCLAIMS, ANY REPRESENTATION OR WARRANTY OF ANY KIND WITH

RESPECT TO THE INFORMATION, INCLUDING, WITHOUT LIMITATION, ANY REPRESENTATION OR WARRANTY OF

DESIGN, PERFORMANCE, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, OR

ANY REPRESENTATION OR WARRANTY THAT THE INFORMATION IS APPLICABLE TO OR INTEROPERABLE WITH ANY

SYSTEM, DATA, HARDWARE OR SOFTWARE OF ANY KIND. AT&T DISCLAIMS AND IN NO EVENT SHALL BE LIABLE FOR

ANY LOSSES OR DAMAGES OF ANY KIND, WHETHER DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, PUNITIVE,

SPECIAL OR EXEMPLARY, INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS

INTERRUPTION, LOSS OF BUSINESS INFORMATION, LOSS OF GOODWILL, COVER, TORTIOUS CONDUCT OR OTHER

PECUNIARY LOSS, ARISING OUT OF OR IN ANY WAY RELATED TO THE PROVISION, NON-PROVISION, USE OR NON-USE

OF THE INFORMATION, EVEN IF AT&T HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH LOSSES OR DAMAGES.

White Paper 1.0 Rev. 0.4 ii

Revision History

© 2009 AT&T Intellectual Property All rights reserved.

AT&T and AT&T logos are trademarks of AT&T Intellectual Property.

All marks, trademarks, and product names used in this document are the property of their respective owners.

Date Revision Description

09/15/09 0.1 This document is currently at the Draft stage.

10/09/09 0.4 Draft revision

10/19/09 0.4b Copyedited final draft.

White Paper 1.0 Rev. 0.4 iii

Table of Contents

1. ...................................................................................................................................................1 Forward

1.1 .......................................................................................................................................1 Audience

1.2 ......................................................................................................................1 Contact Information

1.3 .....................................................................................................................................1 Resources

1.3.1 ..............................................................................................................2 AT&T Resources

1.3.2 ..............................................................................................................2 Other Resources

1.4 ....................................................................................................................2 Terms and Acronyms

Introduction ...................................................................................................................................................4

2. ..................................................................................................................5 Selecting the Best Approach

3. .....................................................................................................................................7 Adobe Flash Lite

3.1 .................................................................................................8 Adobe Flash Development Tools

3.2 .............................................................................................................................8 Media Features

3.2.1 ..............................................................................................9 Motion and Sound in Flash

3.3 ..................................................................................10 Advanced Development with ActionScript

3.4 .......................................................................................11 Accessing Device Hardware Features

3.4.1 ..............................................................................................11 The fscommand Function

3.4.2 ............................................................................................12 The fscommand2 Function

3.5 ...........................................................................................13 AT&T Device Support for Flash Lite

3.6 .............................................................................................14 More Information About Flash Lite

4. ..............................................................................................................15 Microsoft Silverlight for Mobile

4.1 ....................................................................................................15 Silverlight Development Tools

4.2 ...................................................................................................15 Writing Silverlight Applications

4.3 ...........................................................................................17 AT&T Device Support for Silverlight

4.4 .............................................................................................18 More Information About Silverlight

5. .............................................................................................................19 Web Applications with HTML 5

5.1 .........................................................................................................19 Media Features in HTML 5

5.1.1 .....................................................................................................19 The <video> Element

5.1.2 ....................................................................................................20 The <audio> Element

5.1.3 ..................................................................................................20 The <canvas> Element

5.2 ...............................................................................................21 AT&T Device Support for HTML5

5.3 .......................................................................................................21 More Information on HTML5

6. ...................................................................................................................................22 Streaming Media

White Paper 1.0 Rev. 0.4 iv

Table of Contents

White Paper 1.0 Rev. 0.4 v

6.1 ..........................................................................................................22 Streaming Media Formats

6.2 .............................................................................................................................22 Best Practices

7. .......................................................................................................................24 User-Originated Content

7.1.1 .........................................................................24 Content Created by J2ME Applications

7.1.2 ...............................................................................25 Content Created on the BlackBerry

7.1.3 ..............................................................25 Content Created on Windows Mobile Devices

8. .................................................................................................27 Embedding Media in Your Application

8.1 .............................................................................................27 Adding Media as Part of the Install

8.2 .....................................................................................................27 Adding Media After the Install

8.3 .............................................................................................................................27 Best Practices

9. ............................................................................................................................................29 Conclusion

Figures

Figure 1 ............................................................................................................. 17 Silverlight Architecture

Tables

Table 1 — Terms and Acronyms .................................................................................................................. 2 Table 2 — Comparing Development Approaches ........................................................................................ 5 Table 3 — Flash Lite Versions...................................................................................................................... 7 Table 4 — Supported Platform Video Formats ........................................................................................... 10 Table 5 — ActionScript Versions ................................................................................................................ 11 Table 6 — fscommand2 Commands .......................................................................................................... 12 Table 7 — Silverlight Desktop Releases..................................................................................................... 15

Mobile Media

1. Forward

Mobile devices grow increasingly powerful and capable each year. While it seems rudimentary now, when devices introduced support for black and white bitmap images a decade ago, they offered developers and user-interface designers new and compelling ways to communicate with mobile users.

With the introduction of color-capable devices (the Sony Ericsson T68 and Motorola T720) on the AT&T network in 2002, designing media-rich experiences has become a very important aspect of mobile application development. Today’s powerful mobile devices, coupled with AT&T’s network, provide developers the ability to create stunning applications.

This document describes the tools and services available to help mobile application developers deliver a complete media experience on mobile devices.

1.1 Audience

This document is aimed at software developers, software architects, and technical managers who are familiar with software application development on mobile devices using common media types.

1.2 Contact Information

If you have questions or comments about the information in this document, please send an e-mail message to [email protected] and reference the title of this document in your e-mail.

1.3 Resources

This document gives an overview of the landscape for media on mobile devices. Much more information is available from a variety of resources, including those at AT&T as well as other sites on the Internet.

White Paper 1.0 Rev. 0.4 1

mailto:[email protected]

Mobile Media

1.3.1 AT&T Resources

MOBILE WEB 2.0

http://developer.att.com/devcentral/tools_technologies/network/docs/Mobile_Web_2_0_White_Paper.pdf

MARKUP LANGUAGES TUTORIAL

http://developer.att.com/devcentral/tools_technologies/downloads/docs/Markup_Languages_Tutorial.pdf

EMERGING MOBILE APPLICATION ARCHITECTURES

http://developer.att.com/devcentral/community/webcasts/docs/Emerging_Mobile_Application_Architectures.pdf

AT&T DEVICE LIST

http://developer.att.com/developer/device_list.jsp

1.3.2 Other Resources

ADOBE FLASH LITE

http://www.adobe.com/products/flashlite/

MICROSOFT SILVERLIGHT FOR MOBILE

http://silverlight.net/learn/mobile/

MICROSOFT SILVERLIGHT

http://www.silverlight.net/

MICROSOFT EXPRESSION STUDIO

http://www.microsoft.com/expression/

1.4 Terms and Acronyms

The following table defines terms and acronyms used in this document.

Table 1 — Terms and Acronyms

Term or Acronym Definition

2.5G 2nd Generation telephony defining data services








http://developer.att.com/developer/device_list.jsp

http://www.adobe.com/products/flashlite/




Mobile Media

Term or Acronym Definition

3GP Video container format specified by 3GPP

3GPP Third Generation Partnership Project

AAC Advanced Audio Coding

AMR Adaptive Multirate – a codec defined by 3GPP

API Application Programming Interface

CDPD Cellular Digital Packet Data

ECMA European Computer Manufacturers Association

EXIF Exchangeable Image File

FLV Flash Video

H.264 An MPEG-4 video compression standard

HDML Handheld Device Markup Language

HTML Hypertext Markup Language

J2ME Java 2 Micro Edition

JPEG Joint Photographic Experts Group

MMS1 Multimedia Messaging Service

MMS2 Microsoft Media Server

MPEG Moving Picture Experts Group

OTA Over The Air

PCM Pulse Code Modulation

RTSP Real Time Streaming Protocol

SWF Shockwave Flash

W3C World Wide Web Consortium

WAP Wireless Application Protocol

WML Wireless Markup Language

WMV Windows Media Video

WPF Windows Presentation Foundation

WPF/E Windows Presentation Foundation/Everywhere (Silverlight Codename)

XAML Extensible Application Markup Language

XHTML Extensible Hypertext Markup Language

XHTML-MP XHTML Mobile Profile

XML Extensible Markup Language


Mobile Media

Introduction

In the early days, as mobile devices began to support data services, HDML and WML over CDPD and WAP were the only options for delivering media content onto the handset.

These languages could offer only basic black and white bitmap image support. WML was later extended to offer support for color images, but by this time, new markup languages were gaining hold and offering a superior experience, such as XHTML-MP and, later, true HTML support with the introduction of the mobile browsers we know today.

Java on mobile devices in the form of J2ME offered developers control over the devices’ image and audio capabilities, particularly important as smart phones with powerful processors and large color displays gained popularity.

Today’s handsets feature broad support for popular media using a variety of techniques. These include runtimes such as J2ME, Flash, Silverlight, and others, powerful mobile browsers that support full HTML implementations, as well as SDKs for writing native applications offer developers tremendous flexibility in delivering compelling experiences on what are essentially handheld computers.


Mobile Media

2. Selecting the Best Approach

Modern mobile devices can handle nearly the same content used for desktop PCs, either natively or with the assistance of network-based transcoding gateways. As well, compiler conversion utilities make it possible to port desktop applications to mobile devices. Although this support greatly eases the developer’s task of writing mobile applications from scratch, best results come from thinking about how your application should work on a mobile device. Mobile application developers who make thoughtful and efficient use of the screen, processing capabilities, and the data network tend to produce compelling applications consumers love to use.

In this document, we will introduce the reader to the variety of tools and technologies available to add media of all types to their application. We discuss Flash Lite, Adobe’s mobile version of their nearly ubiquitous Flash desktop player, Microsoft’s Silverlight Mobile, HTML 5, as well as integrating with Java and native applications.

In selecting the best development method, it is important to consider a variety of factors, such as:

Does the user experience demand precise control over the display, or is it acceptable for the application to look slightly different on various devices?

Will users need to download software in order to use the application?

Will the application be compatible with the wide range of devices in the ecosystem or just a few, specialized devices?

Table 2 — Comparing Development Approaches

Method Benefits Considerations

Adobe Flash Lite

Offers precise control over the presentation layer

Supports vector-based graphics

Can use existing Flash projects as a starting point

Commonly available

May require some users to download software to use your application

May not be supported on all common devices

May require more processing capabilities than other approaches

Java J2ME Runtime Offers control over presentation layer and

May require some users to download software to


Mobile Media

Method Benefits Considerations

audio

Allows application to run on a wide variety of devices

Commonly available

use your application


Visual appearance of application controls may differ between devices

Some fragmentation issues to consider

Microsoft Silverlight

Offers precise control over the presentation layer and provides powerful built-in effects

Supports vector-based graphics

Likely requires users to download software to use your application

May not be supported on all common devices


Native Application

Offers greater control over user experience

Provides access to some hardware features of the device

Can be a complex undertaking

Limits support to the platform for which application is written

Web Application (HTML) Offers the broadest reach

of all methods Simple coding language

Does not offer precise control over user experience as browser fidelity varies by device

We hope that by using some of the tools and tips presented in this document, your mobile application can provide a better, more compelling experience for the customer.


Mobile Media

3. Adobe Flash Lite

Flash content is widely deployed on the Web in dynamic Web sites, interactive advertisements, and other applications where precise control over interactivity, animation, video, and sound are needed.

Flash Lite is a mobile version of Adobe Flash Player, which enables rich interactive content on many mobile devices. Flash Lite has been designed from the start as a compact, highly optimized subset of Flash built to work on constrained mobile devices.

Flash Lite was launched in 2003 for NTT DoCoMo’s 505i handset in Japan. In 2006, AT&T shipped its first Flash Lite devices including the Nokia 6126 and Sony Ericsson W700i. The early releases of Flash Lite were basic and offered limited device integration. Today, with the recent release of version 3.x, Flash Lite offers a robust and capable solution which competes well with Java J2ME.

Flash Lite has gone through three major revisions since its introduction. Table 3 shows notable features added with each.

Table 3 — Flash Lite Versions

Flash Lite Version

ActionScript Version

Release Date Notable Features

1.1 1.0 Jun 2004 Based on Flash Player 4. Basic media features, data loading, and device access. No video. No XML support.

2.0 2.0 Dec 2005

Based on Flash Player 7. Complex apps and content. Supports device video, audio, images. Save data. Shared objects. Synchronized sound. XML support.

2.1 2.0 Dec 2006 XMLSocket support.

3.0 2.0 Feb 2007 Based on Flash Player 8. Supports FLV video. Improved security (sandbox).

3.1 2.0 Feb 2009 Based on Flash Player 9. H.264 and AAC audio support. Local SWF can connect to network.


Mobile Media

3.1 Adobe Flash Development Tools

Developing Flash Lite requires Adobe Flash Professional and Adobe Device Central. Currently, Flash Professional is the only authoring tool available to design, build, and publish Flash and Flash Lite applications. Flash Professional comes with Device Central, an integrated emulator application used to test Flash Lite content on a variety of simulated devices.

Adobe Flex, the standards-based MXML framework and compiler for creating desktop PC Flash applications, is popular among developers and organizations because it can be integrated with automated build systems and features an open source version. However, Flex does not currently support the Flash Lite runtime. Because both Flash and Flash Lite are written in the same development environment, developers already familiar Flash will likely find developing Flash Lite applications for mobile devices to be a smooth and comfortable transition.

Developers new to Flash but experienced in more traditional development environments such as Java or C++ will likely find the Flash authoring environment quite different than anything they’re used to. Authoring Flash is more akin to working with layered graphics editing tools such as Photoshop or timeline-based video editors like Adobe Premiere. Indeed, a compiled Flash application is typically called a “movie” whether or not there is video content in the app.

Essentially, Flash development centers around a “stage” where imported media assets are controlled and viewed over time. Commands from ActionScript (the Flash scripting language embedded in one or more layers), user input, device events, or elapsed time are used to trigger behaviors in the application.

3.2 Media Features

At its core, Flash is designed to fit into the media design and creation process, so it naturally offers excellent media handling capabilities. The development tools easily import a variety of media formats, and a suite of media creation tools is also available.

One of the main benefits of Flash is its support for vector graphics, which is particularly important for mobile because this permits high-quality graphics with very small file sizes.


Mobile Media

Vector graphics are lines, points, curves, and shading described using mathematical formulas, which become visible on a screen when the device draws (rasterizes) the image by decoding the mathematical formula. Bitmapped graphics are internally represented as a two-dimensional grid of pixels drawn directly to the screen. Information must be stored about each pixel — even for ”blank” space. The larger the dimensions and resolution of a bitmapped image, the larger the file size. Because vector graphics are represented by a mathematical formula, vector graphics can have very small file sizes.

The work of creating the image is performed in software on the display device. This also means vector graphics can be scaled, rotated, and otherwise transformed without loss of quality, although rendering them does require more processing power than displaying bitmapped images.

3.2.1 Motion and Sound in Flash

In Flash, applications are often called “movies” due to the way the content and code is assembled along a timeline in the development tool. All images, code fragments (ActionScript), user interface elements, and so on are associated with a “frame,” which is a moment in time, similar to a film cell in a movie. When a Flash application is run, it begins by playing the first frame and proceeds step by step along a timeline through each frame.

Flash supports multiple ways of animating the bitmap or vector images in a presentation. To give the effect of motion, Flash allows the developer to show and hide layers and move and transform objects across frames and over time. To make the process of creating time-based animations easier, Flash supports tweened animations, whereby the developer describes the beginning and end points of an animation and the authoring tool does the work of calculating the interstitial frames. For example, to create the effect of an image sliding across the screen, the developer simply places the image in the start position, then moves the image to another location and selects “Create Motion Tween.”

Flash Lite also supports audio and video media types. Videos can be embedded in the Flash SWF file itself, stored on the device file system, or accessed from a remote network location and progressively downloaded using HTTP or streamed using RTSP.

When rendering video, developers should be aware that Flash Lite version 2 places video ”in front” of all other objects and will not allow other Flash elements to overlay the video.


Mobile Media

Flash Lite offers programmatic control over video playback, including:

play()

stop()

pause()

resume()

close()

Note that volume control is not supported in the Flash Lite 2.x player.

Flash Lite version 3.x adds support for the Flash FLV format and renders video within the player rather than relying on native playback support in the device as in previous versions. The inclusion of the FLV format also means developers now have a common video format that will play on any device that supports Flash Lite 3.x.

Table 4 — Supported Platform Video Formats

Platform Video Codecs Audio Codecs Max Frame Rate

1.1 1.0 Jun 2004 Based on Flash Player 4. Basic media features, data loading, and device access. No video. No XML support.

2.0 2.0 Dec 2005

Based on Flash Player 7. Complex apps and content. Supports device video, audio, images. Save data. Shared objects. Synchronized sound. XML support.

2.1 2.0 Dec 2006 XMLSocket support.

3.0 2.0 Feb 2007 Based on Flash Player 8. Supports FLV video. Improved security (sandbox).

3.1 2.0 Feb 2009 Based on Flash Player 9. H.264 and AAC audio support. Local SWF can connect to network.

3.3 Advanced Development with ActionScript

While much of the design of a Flash Lite application can be done using the graphical tools available in Adobe Flash Professional, to make more sophisticated applications, the developer will need to use ActionScript.


Mobile Media

ActionScript is the programming language used in Flash and Flash Lite applications to control the content timeline, respond to user and device events, and perform logic and data handling as required by the application. ActionScript is a standard part of Flash and can be written within the Adobe Flash Professional IDE.

In its first version, ActionScript was a simple language with variables, loops, conditionals, etc. However, since version 2.0, ActionScript has grown to become a true object-oriented language patterned after ECMAScript, supporting classes, inheritance and types. Along with the usual basic data types (such as strings, numbers, and booleans), more complex types like Array, Date, TextField, MovieClip, XML, and NetConnections are possible. The latest version of ActionScript adds packages, namespaces, regular expressions, and a virtual machine for runtime type checking.

Table 5 — ActionScript Versions

Flash Lite Platform ActionScript Version

1.0 ActionScript (Flash 4)

1.1 ActionScript (Flash 4) + additional APIs

2.0/2.1 ActionScript 2.0 (Flash 7) + fscommand2

3.0 ActionScript 2.0 (Flash 8)

3.4 Accessing Device Hardware Features

Flash applications can communicate with the underlying device hardware via the fscommand and fscommand2 functions in ActionScript 2.x and later. On mobile devices, these commands allow applications to access some player and mobile-specific information as well as activate the vibration feature on most devices.

When designing an application, developers should keep in mind that hardware features accessible using fscommand vary by device. Additionally, in some cases, the Flash Lite application (SWF) must be digitally signed to gain access to hardware features.

3.4.1 The fscommand Function

The fscommand function takes two arguments, namely, a command string and a parameters string. The most useful command is launch, used to launch an application resident on the device. The syntax of the launch command is device-


Mobile Media

specific because the full path to the application is required. If Flash is executing commands when it receives the fscommand, it will handle fscommand after it finishes processing the current the frame.

In this example, we instruct the native Web browser on a Nokia S60 to load Yahoo.com:

on(keyPress "9") { status = fscommand("launch", "z:\\system\\apps\\browser\\browser.app,http://yahoo.com"); }

3.4.2 The fscommand2 Function

The fscommand2 is similar to fscommand, but it can take any number of arguments (command dependant). Additionally, fscommand2 does not need to wait for the Flash frame to finish rendering but can execute as soon as the command is called.

A subset of these commands is shown in Table 6.

Table 6 — fscommand2 Commands

Command Description

ExtendBacklightDuration Extends the duration of a backlight for a specified period of time.

FullScreen Sets the size of the display area to full screen.

GetBatteryLevel Returns the current battery level as an integer.

GetDevice Gets the name of the device (i.e., “Nokia 6650”).

GetDeviceID Sets a parameter that represents the unique identifier of the device (for example).

GetFreePlayerMemory Returns the amount of heap memory.

GetMaxBatteryLevel Returns the maximum battery level of the device.

GetMaxSignalLevel Returns the maximum signal strength level as a numeric value.

GetMaxVolumeLevel Returns the maximum volume level of the device as a numeric value.

GetNetworkConnectionName Returns the name of the active or default network connection.


Mobile Media

Command Description

GetNetworkConnectStatus Returns a value that indicates the current network connection status.

GetNetworkGeneration Returns the generation of the current mobile wireless network (such as 2G or second generation of mobile wireless).

GetNetworkName Sets a parameter to the name of the current network.

GetNetworkRequestStatus Returns a value indicating the status of the most recent HTTP request.

GetNetworkStatus Returns a value indicating the network status of the phone.

GetPlatform Sets a parameter that identifies the current platform.

GetMaxSignalLevel Returns the maximum signal strength level as a numeric value.

GetSignalLevel Returns the current signal strength as a numeric value.

GetTotalPlayerMemory Returns the total amount of heap memory.

GetVolumeLevel Returns the current volume level of the device as a numeric value.

Quit Causes the Flash Lite Player to stop playback and exit.

ResetSoftKeys Resets the soft keys to their original settings.

SetFocusRectColor Sets the color of the focus rectangle to any color.

SetInputTextType Specifies the mode in which the input text field should be opened.

SetSoftKeys Remaps the softkeys of a mobile device.

StartVibrate Starts the phone's vibration feature.

StopVibrate Stops the current vibration, if any.

3.5 AT&T Device Support for Flash Lite

AT&T first offered support for Flash Lite with the introduction of the Sony Ericsson W810i and the Nokia N75. Initially, the Sony Ericsson W810i offered support for Flash Lite in the browser application, while the Nokia N75 supported Flash Lite in a standalone player on the device. The latest versions of Flash Lite for these devices offers support for Flash in the browser and can also be used to power animated screensavers, and, in some cases, animated wallpaper.


Mobile Media

Flash Lite is supported by most of the recent Nokia and Sony Ericsson devices sold by AT&T. Some devices such as the 8925 (Tilt) and 8525 include the Flash Lite player on the device, although not all manufacturers preload it. Microsoft Windows Mobile devices also support Flash Lite.

3.6 More Information About Flash Lite ADOBE FLASH LITE FEATURES http://www.adobe.com/products/flashlite/version/flashlite_feature_comparison.pdf


http://www.adobe.com/products/flashlite/version/flashlite_feature_comparison.pdf

Mobile Media

4. Microsoft Silverlight for Mobile

Microsoft Silverlight is a relatively new technology offered by Microsoft, which provides functionality similar to Adobe Flash on Windows PCs and Mac OS X. For mobile devices, Microsoft is creating Silverlight for Mobile, which promises a rich experience on mobile devices. Future Windows Mobile and Nokia S60 devices will ship with Silverlight when it is released.

Silverlight on the desktop has been available since 2007 and has seen two major releases in that time.

Table 7 — Silverlight Desktop Releases

Version Release Date Notable Features

1.0 Sep 2007 Core, MP3 & WMA audio, WMV video, JavaScript scripting

2.0 Oct 2008 .NET CLR, adaptive streaming, codec extensions

3.0 Sep 2009 More controls, AAC, H.264, out-of-browser running

4.1 Silverlight Development Tools

Microsoft Visual Studio and Microsoft Expression Studio are the tools developers use to write Silverlight applications. These tools, particularly Visual Studio, are fully-featured IDEs familiar to any C++ developer. For those who prefer Eclipse, a Silverlight IDE plug-in called “Eclipse Tools for Microsoft Silverlight” (eclipse4SL) is available.

According to Microsoft, developers who build Silverlight applications for the Web or for mobile devices will find procedures and development environments to be the same — the only difference being a matter of optimizing applications for mobile form factors1.

4.2 Writing Silverlight Applications

Silverlight applications are created in a manner similar to Windows Presentation Foundation (WPF) applications for the desktop. Silverlight for Mobile uses a subset of WPF commands accessible via the mobile browser plug-in architecture.

1 http://silverlight.net/learn/mobile/


Mobile Media

An application is first defined in XAML (Extensible Application Markup Language), which describes the UI, graphics, animations, events, and more. User interface event handling is written using any .NET language, such as Visual Basic, C#, or JavaScript. In addition to event handling, custom user interface controls can be created using any .NET language. Figure 1 shows the Silverlight architecture.


Mobile Media

Figure 1 Silverlight Architecture

Source: Microsoft

4.3 AT&T Device Support for Silverlight


Mobile Media

Support for Microsoft Silverlight on mobile devices is coming soon. Nokia S60 devices currently support Silverlight 2.0 applications, and the third party plug-in “Skyfire browser” brings Silverlight support to Windows Mobile devices in advance of the official Microsoft release. Additionally, Nokia promises its S60 platform will include support for Silverlight around the same time as the Windows Mobile release.

AT&T will support Silverlight on mobile devices with the official release of Microsoft Silverlight for Mobile.

4.4 More Information About Silverlight

MICROSOFT SILVERLIGHT


MICROSOFT SILVERLIGHT FOR MOBILE


MICROSOFT EXPRESSION STUDIO






Mobile Media

5. Web Applications with HTML 5

HTML 5 is the first major advancement of the Web’s standard markup language since HTML 4.01 was published in December 1999. It radically updates and enhances the standard, while at the same time making it cleaner and more standardized by deprecating old functions and introducing parsing rules to better handle compatibility.

HTML 5 began as a series of concepts for creating desktop-like applications on the Web. The Web standards body, the W3C, adopted the HTML 5 specification as the starting point for the new HTML specification to replace HTML 4.0.1. The W3C HTML 5 working group published the public draft of the specification in January 2008. While work will continue on the specification for some time, pieces of functionality will be included with successive browser releases as they become available.

While Adobe Flash Lite and Microsoft Silverlight require browser plug-ins, the promise of HTML 5 is direct, in-browser support for many of the advanced visual programming features similar to those in the Adobe and Microsoft technologies. Developers already familiar with HTML will be able to develop powerful, media-rich applications that are nearly indistinguishable from competing runtimes.

5.1 Media Features in HTML 5

HTML 5 adds a host of new definitions for control over visual presentations, such as the <video>, <audio>, and <canvas> tags, support for timed media playback, offline storage, and more.

The new definitions, controlled via JavaScript, offer native support for audio and video playback, drag-and-drop manipulation of graphics and more, within the browser, without requiring proprietary plug-ins such as Flash, Silverlight, or QuickTime. (Note that on the iPhone, audio or video elements are played using the built-in QuickTime player, rather than playing in a frame inside a Web page.)

5.1.1 The <video> Element

Use the <video> element to set video to play directly in the browser. HTML 5 offers several fall-back scenarios to handle browser compatibility. For example, multiple video sources can be specified using the <source> element to tell the


Mobile Media

browser to try each source in turn until it finds one it can play. Additionally, exception cases can be added in markup to, for example, show a URL or load the Flash or QuickTime plug-ins to play the media in the event the browser does not support playback directly.

Note that because only some video codecs are supported in HTML 5, you must ensure your content either complies with the codecs supported or implement one of the fall-back strategies discussed above.

Here is an example use of the <video> element: <video width=”320” height=”240” poster=”images/poster.jpg” controls=”true”> <source src=”funny_video.ogv” type=”video/ogg”/> <!—Most Browsers <source src=”funny_video.mp4” type=”video/mp4”/> <!—Apple support  </video>

5.1.2 The <audio> Element

Use of the <audio> element is similar to the <video> element. Here is an example use of the <audio> element:

<audio controls=”true”> <source src=”funny_audio.ogg”/> <!—Most Browsers <source src=”funny_audio.mp3”/> <!—Apple supports whatever Quicktime can play  </audio>

5.1.3 The <canvas> Element

The <canvas> element enables a dynamic drawing area that can be controlled by JavaScript. The canvas can be used to draw graphics and animations directly in the browser window without using a proprietary plug-in like Flash or Silverlight.

For example, some Web-based collaboration sites use the <canvas> feature of HTML 5 to create group drawing spaces for sharing ideas.

Mobile Media

5.2 AT&T Device Support for HTML5

Many Web application development companies, including Google and others, are using the features of HTML 5 now, even though the HTML 5 specification is an ongoing project.

Similarly, some browsers offer support for HTML 5. For example, the Apple iPhone offers robust support for HTML 5 (currently the only AT&T device to do so).

Still, there are some challenges with compatibility. Microsoft has not made any announcement on supporting the new media elements; the World Wide Web Consortium (W3C) has abandoned standardization of video and audio codecs; and there is still some disagreement on the format for open-source video (Ogg Theora versus H.264). Nevertheless, while support for the new <video>, <audio>, and <canvas> elements is sporadic, support is improving on a regular basis.

5.3 More Information on HTML5

HTML 5 OVERVIEW

http://dev.w3.org/html5/spec/Overview.html

CANVAS TUTORIAL

https://developer.mozilla.org/en/Canvas_tutorial


http://dev.w3.org/html5/spec/Overview.html

https://developer.mozilla.org/en/Canvas_tutorial

Mobile Media

6. Streaming Media

Streaming media is the process whereby an audio or video file is sent from server to client in a serial fashion and played on the client as it is received, without waiting for the full file to download. In traditional streaming, except for the use of a cache, the received portions of the streamed file are not saved on the client once played (they are discarded). A similar technique called ”progressive download” allows a client to begin playing a file before it has fully downloaded. In this case, the full file is eventually transferred and saved. In either case, the experience to the user is similar, and the term “streaming media” is often used to describe ”progressive download” implementations as well as true streaming implementations.

6.1 Streaming Media Formats

On the desktop, Flash video is typically used to render Web video, and this is the format used by the popular YouTube Web site. On mobile, the most widely supported format is 3PG, a variation of the MPEG-4 definition.

3GP video is a standardized MPEG-4 stream typically encoded in H.263 video and AMR audio, and while other codecs are allowed, these are the most popular.

Although it varies by device manufacturer, Windows Mobile 6.0 and earlier versions do not support 3GP video or the RTSP protocol. To stream video to these platforms, Windows Media Video (WMV) format with the Microsoft Media Server (MMS) protocol is used.

Still other codecs and containers are supported on mobile, such as RealNetworks RealVideo, and some can offer superior video quality or other features. However, for applications targeting the broadest range of devices, 3GP video is the best choice.

6.2 Best Practices

The high-speed AT&T data network has the capacity to stream video to subscriber handsets. To ensure customers have a great experience with streaming media, follow these best practices.


Mobile Media

Firstly, ensure your target audience can play the video formats you intend to stream. For 3GP audio or video, target devices must support the 3GP format, and the media must be streamed from a host server supporting RTSP, the Real Time Streaming Protocol. YouTube or other paid hosting services can provide this service. In the case of YouTube, devices must support RTSP/3GP with H.263/AMR.

For developers who plan to include streaming media support within applications, here are some general tips and guidelines:

If possible, implement network speed detection and switch to a lower bandwidth stream when clients are connected to slower 2.5G networks

Small ”thumbnail-sized” video or very low resolution video is not engaging, so set minimum bandwidth requirements for your application and provide good exception handling (such as an alternate experience) if there is not enough bandwidth to provide quality video

Keep the user informed of status, with messages such as “buffering,” “closing,” etc., to show that your application is making progress and is not ”hung”

Note that many J2ME devices may not allow full screen playback, especially on devices which allow the orientation to be changed

If possible, launch the device’s native player, which typically offers the best device integration, such as full-screen playback and familiar user controls


Mobile Media

7. User-Originated Content

Today’s advanced mobile devices offer a great new capability: media creation. The ability for users to shoot video on their mobile devices and upload moments later to social networking sites such as Facebook is rapidly changing the landscape in terms of the way media is created and viewed.

Here we give an overview of content creation and sharing on some popular device platforms.

7.1.1 Content Created by J2ME Applications

The Java J2ME platform is available on the largest number of AT&T devices, and through an extensive set of Java Specification Requests (JSRs), it offers access to audio capture, image capture, and storage functions on device hardware, as well as network functions for sharing the captured media.

The Mobile Media API (JSR 135) (http://developers.sun.com/mobility/midp/articles/mmapioverview/) provides the access needed to capture sound or image data on supported devices. This API uses a player object created with a special capture://device?encoding URL (or locator) parameter to acquire audio or camera data.

The following table shows some locator examples:

Locator String Description

capture://audio Capture audio using default encoding (platform dependant).

capture://audio? encoding=pcm&rate=11025&bits=16&channels=1

Capture 11kHz mono audio in PCM.

capture://video? encoding=jpeg&type=exif Capture a JPEG with exif format.

The data can be saved to an internal memory buffer or directly to the device’s file system using a properly configured RecordControl object.

Once instantiated, the player’s UI control can be obtained by invoking the getControl(“VideoControl”) method. To take a photo, images are obtained by invoking the VideoControl’s getSnapshot() method. To record audio or


http://developers.sun.com/mobility/midp/articles/mmapioverview/

http://developers.sun.com/mobility/midp/articles/mmapioverview/

Mobile Media

video on devices supporting video recording, the Player’s start() and stop() methods are used.

Once obtained, uploading is a matter of using standard practices to post the data via HTTP or sockets. Another option available to J2ME developers is to upload or share media using MMS on devices implementing WMA 2.02 (JSR 205).

Keep in mind that Windows Media does not know the MMS message size limit (typically 600k for AT&T devices), so developers might need to resize images or create smaller images using image size options when creating the Player object.

7.1.2 Content Created on the BlackBerry

Recording audio on the BlackBerry platform (version 4.2 and later) is identical to recording audio via J2ME as BlackBerry also implements JSR 135. This process is described in detail in the BlackBerry Multimedia Guide3.

Obtaining a photo on BlackBerry requires the developer to invoke the Camera application via the Invoke class, wait for the user to obtain the image, and listen to the Camera application for the image capture event. Once the image is captured, the application can retrieve the file from the file system. In version 5.0, video recording using JSR 135 will be supported as described in the 5.0 Beta BlackBerry Multimedia Guide4.

7.1.3 Content Created on Windows Mobile Devices

For Windows Mobile devices, the best way to capture audio, video, and photos is to use the provided dialogs. This is simpler to implement and usually gives the user better control over the device than custom controls or other means.

For audio capture, use P/Invoke to activate the Voice Recorder Control5. When creating the control, a file name is passed to the application. After the control has been dismissed, the filename passed in will point to the recording. Audio can also be captured by using the Waveform Audio Functions6.

2 http://developers.sun.com/mobility/midp/articles/wma2/ 3 http://docs.blackberry.com/en/developers/deliverables/1184/Multimedia_guide.pdf 4 http://docs.blackberry.com/en/developers/deliverables/8543/BlackBerry_Java_Application-5.0_MultimediaGuide_Beta-US.pdf 5 http://msdn.microsoft.com/en-us/library/ms836147.aspx 6 http://msdn.microsoft.com/en-us/library/ms925318.aspx


Mobile Media

For photo and video capture, the CameraCaptureDialog7 offers similar capabilities to the Voice Recorder Control. In addition, other options can be specified, such as resolution or video time limit.

One limitation when using this Dialog is the lack of support for multiple photos or videos. It is possible to capture photos and videos via the DirectShow APIs, but this requires the developer to create a capture user interface, and the custom interface may not be able to control the camera with the same precision as the native dialog. Additionally, some codecs may or may not be supported when using the DirectShow APIs.

7 http://msdn.microsoft.com/en-us/library/microsoft.windowsmobile.forms.cameracapturedialog.aspx


Mobile Media

8. Embedding Media in Your Application

For many applications, it makes sense to include media elements in your interface to make the experience more dynamic. This may consist of several images, audio, or video. The way these items are used within the application can affect the user experience significantly.

8.1 Adding Media as Part of the Install

Formerly, mobile applications were restricted in download (binary) size and memory footprint. With faster networks and more powerful devices in use today, setting limits on these attributes is not as common.

By embedding media into the application from the outset, the application will have all media elements available on first startup, which results in a more responsive experience. This is the best approach for applications such as games.

8.2 Adding Media After the Install

For applications where media needs to be “refreshed” every so often, it is advantageous to download the assets as required when the application runs.

Another advantage is that the application size can be much smaller than those which include media. However, care must be taken to ensure a fast and responsive user experience. Include media in your application if it is required on startup. Use a blended method where all or most of the media is delivered with the installation package but stored in an application database. Allow the application to periodically check for updates to these cached media elements and update over-the-air as needed.

8.3 Best Practices

Whichever method best suits your application, the following tips may come in handy:

Combine multiple files into one package if possible. On the current mobile networks, the speed and latency of downloading one larger file versus several smaller files is significant.


Mobile Media

Make the application as responsive as possible by minimizing wait time as resources download. Ensure assets are resident in the application for the most common usage scenarios. Display a progress screen, or better, allow useful functionality while the application is downloading assets.

Keep in mind application size limits. For example, the iPhone is limited to 10MB downloads when connected to EDGE or 3G networks.


Mobile Media


9. Conclusion

As today’s powerful mobile phones become increasingly capable, they are becoming increasingly important in the daily lives of consumers and business users alike. Similarly, users’ expectations of the software applications which run on these devices are ever-increasing as well.

Fortunately, as we have outlined in this document, there are rich development environments and equally powerful technologies developers can harness to build the quality applications users expect.

As with any product development effort, the difference between good software applications and great software applications is the thoughtful blend of good design and appropriate use of the best technology available.

Integrating media into an application has the potential to create a truly compelling user experience and enable functionality simply not possible just a few years ago. The use of media and the technologies chosen to implement it will largely depend on the goal of the application and the intended target user.

We encourage you to experiment, be creative, and think through the experience you would like your users to have. Equally important, consider and plan for the experiences your users could have if the use of your application veers away from the ideal flow.

Thinking about your application in this way may have surprising results. While you may revisit the ”drawing board” more than once, you’re certain to finish with a product that hits all the right notes with your intended users.

at&t developer program · wml over cdpd and wap were the only options for delivering media...

Documents