XML Media Formats

Robin Burke

ECT 360

Outline

Multiple XML documents SMIL HTML+TIME Break WML XHTML MP Final Project Presentation

Multiple XML documents

ProblemXSLT needs to access multiple XML

documentsOnly has access to node-sets in

document being processed Example

curriculum designscourse information

Solution

document() function Retrieves document from URL

can be used as part of an XPath expression

<xsl:variable name="course-names"

select="document('course-list.xml')/course-list/course/@name" />

SMIL

Synchronized Multimedia Integration Language

Define Common Standards for multimedia documents.

SMIL 1.0 (1998) SMIL 2.0 (2001)

The Idea

General document format integrating different types of independent media objects.

Promote the notion of time and synchronization as a core functionality.

Bring together major industrial players to define a common standard.RealPlayer, Flash support

SMIL document

A Set of “components” accessible via URLs.

Components have different media types: audio, video, text, images and objects.

Intra- and Inter-object synchronization.

User interactions: media controls and hyper-links (random access).

Organization of a SMIL document

Two parts : Head : contains information of document level Body : contains the temporal scenario, animations,

transitions and the media objects

Structure of a document

toto.smi

head body

seq switchparLayout

Region 1

Media

Audio Channel

Media

Transition

Animation

Transition

Example

Document Head

META element: description of the document properties and metadata (RDF)Title, author, expiration date,

keywords, summary, …

Spatial Aspect

Layout elementHierarchical Model with hotspots

(regPoints)Layers instead of text flow (text flow

for text !)Simple Positioning close to the CSS

Model (x, y, z-index, fit)

Spatial Aspect

Region 1 Region 2

Region 3

b

c

a

Time flow

Regions and sub-regions for the spatial placement of media objects

Document Body: Synchronization

Contains the temporal scenario of the document

A scenario is defined recursively : Schedule elements

Schedule = Parallel | Seq | Switch

Basic Media Objects

Media Objects marked-up with:Audio, Video, Text, Img, Textstream,

Animation, Ref, Param, et … Prefetch

Attributes : Src : identifies the basic media object file (URL)

rtsp://rtsp.example.org/video.mpg

Type : mime type (eg. video/mpeg) Region : identifies the drawing surface Dur : duration of the media object

Synchronization Attributes

The Dur (duration) attribute: “intrinsic”: the duration corresponds to the duration of the

external file. “explicit”: the duration is specified in the document

(dur= “15 s”)

The repeat attribute:RepeatCount=“3” repeats the simple duration of the media.

RepeatDur=“12 s” :

Synchronization Attributes

The begin, end attributes: Value (begin= “13 s”) : offset relative to the parent element. Reference to another clock : (begin= “e2.end + 5 s ”) Reference to the absolute time reference:

(begin= “wallclock(2001-01-01Z)” Reference to an asynchronous event (interactivity):

(begin= “button.click”)

Media Clipping

Spatial Clipping using regions and sub-regions Temporal Clipping using clip-begin et clip-end

attributes (media objects are external files)

<video id="a" src=“video.mpg" clip-begin=“smpte=00:01:45" clip-end=“smpte=00:01:55" …/>

Media Slice

Semantics : play in sequence a set of media objects

Attributes Fill : used to make the object « persist » on the screen

• Remove : removes the object at end time• Freeze : keeps the last frame at end time

The sequential element

<seq> <image id="a" regionName=“x” src="wait.gif“ fill=“freeze”/> <video id="b" regionName=“x” src="video.au dur="20 s" /></seq>

The parallel element

Semantics : Play in parallel a set of media objectsEnd time : maximum duration of child

objects

Synchronization Arcs and Events

a

cb

d

...<par> <audio id="a" src="audio.au" begin="id(b)" /> <video id="b" src="video.au end="id(c)" /> <text id="c" src="text" begin="id(d)"

end=id(a)(end) /> <image id="d" src="image.gif" begin="id(b)(end)/></par> ...

Triggering of objects on events :

...<par> <img id="a" src=“image" /> <video id="b" src="video.mpg” begin=“a.activateEvent" end=“a.activateEvent /> <text id=“c" src=“text” end=“b.focusInEvent" /></par> ...

a

cb

Switch element An element to choose from a set of content

equivalent objects Choice is based on attribute values

language, screen size, depth, bitrate, systemRequired

…and user preferences

...<par> <text .../> <switch> <par bitrate="40000"> ... </par> <par bitrate="24000"> ... </par> ........ </switch> </par> ...

Animations

Definition : A set of attributes are target of the animation A function (calc mode) makes these attributes evolve A control on the instants where the changes are applied

Syntax animateMotion : graphical movements of

elements animate : generic animation applied to element

attributes from/to/by/calcMode set : discrete change of an attribute value at a

given instant animateColor : animation in the color space

Animations<img top="3" ...>    <animate begin= "5s" dur="10s" attributeName="top"

by="100" repeatCount="2.5" fill="freeze" calcMode="linear"/> </img>

Calc Mode : discrete, list of values with linear, log interpolation

TransitionsElement : transition

Type and Subtype (transition repository + variant) transIn and transOut attributes

... <transition id="wipe1" type=“zigZagWipWipe" subtype="leftToRight" dur="1s"/><transition id="wipe2" type=“veeWipe" subtype="leftToRight" dur="1s"/> ... <seq> <img src="butterfly.jpg" dur="5s"... /> <img src="eagle.jpg" dur="5s" fill="transition" transIn="wipe1" ... /> <img src="wolf.jpg" dur="5s" fill="transition" transIn="wipe2" transOut=“wipe1” ... /> </seq>

Transition Transition Transition

Hypermedia time-based links Compatible with (Xlink/Xpointer) Extension to the semantics of URLs

http://foo.com/path.smil#ancre(begin(id(anchor))

Two types (a: whole object, area: part of it) Jump over time and space !!

Attribute show Replace (default value) New (fork) Pause (procedure call)

Links with spatial and temporal anchors

Anchor on a sub-surface of an object

Anchor on a sub-duration of an object

<video src=“rtsp://www.w3.org/video.mpg”> <area href=“http://www.w3.org/AudioVideo” coords=“0%, 0%, 50%, 50%”/> <area href=“http://www.w3.org/Style” coords=“50%, 50%, 100 %, 100%”/></video>

<video src=“rtsp://www.w3.org/video.mpg”> <a href=“http://www.w3.org/AudioVideo” begin=“0 s” end=“5 s” /> <a href=“http://www.w3.org/AudioVideo” begin=“10 s” end=“15 s”/></video>

time

Example

HTML+TIME

A set of extensions to HTMLfor building in animation

Will eventually be replaced byXHTML + SMIL

Usable now

Example

Break

Wireless Web Content

WAP Wireless Application Protocol defines interaction style, not content

WML older standard based on "card" metaphor supported by most phones

XHTML MP MP = "mobile profile" regular HTML with restricted element set

cHTML "compact" HTML used for iMode (Japan) migrating to XHTML MP

WAP 1.0 Gateway

Binaryencoding

WMLContent

Webserver

WAPGateway

Mobiledevice

WAP 2.0 Gateway

XHTMLContent

XHTMLContent

Webserver

WAPGateway

Mobiledevice

Stylesheet

WML

Wireless Markup Languagecontent for cell phone display“microbrowser”

Special characteristicssmall sizemultiple “pages” per download

• cards

WML Elements

card a chunk of screen content

• title, id multiple cards in one WML document

• "deck" do

link to built-in "options" and "accept" softkeys p

text content a

links wtai protocols for phone functions

• dial number, save in phone book img

only wbmp supported

Example

Variable example

XHTML MP

Different

Differences

Multi-page vs deck can use multi-part message

Some functionality gone variables softkeys

Browser / device-dependence more important image formats table support CSS support

Example

Handling device differences

Basic problem want the best display for each device device capabilities vary widely CSS not widely implemented

• CSS not enough

Solution dynamic server-side page generation

• based on device profiles

can be done with XSLT

Device-specific information

WURFLWireless Universal Resource FileXML application

Webserver

Web requestw/ device type

XSLTstylesheet

Device typeparameter

XHTML MPpage

XHTML MPpage

WURFL

Page data

WURFL example

<device user_agent="SonyEricssonP800" fall_back="ericsson_generic" id="sonyericsson_p800_ver1">

<group id="product_info"> <capability name="brand_name" value="SonyEricsson"/> <capability name="model_name" value="P800"/></group><group id="markup"> <capability name="wml_13" value="true"/> <capability name="html32" value="true"/></group><group id="display"> <capability name="resolution_width" value="208"/> <capability name="resolution_height" value="144"/></group><group id="image_format"> <capability name="jpg" value="true"/> <capability name="gif" value="true"/> <capability name="bmp" value="true"/> <capability name="png" value="true"/> <capability name="colors" value="4096"/></group>...</device>

Final Project Presentation

10% Previous milestones 10% Presentation 20% Design 25% Schema 35% XSLT

Presentation

Whole team should be here Plan to stay to see all the projects Group assessment exercise