predictable processing of multimedia content, using mpeg-21 digital item processing

8/22/2019 Predictable Processing of Multimedia Content, Using MPEG-21 Digital Item Processing

http://slidepdf.com/reader/full/predictable-processing-of-multimedia-content-using-mpeg-21-digital-item-processing 1/10

Predictable Processing of Multimedia Content,

Using MPEG-21 Digital Item Processing

Chris Poppe, Frederik De Keukelaere, Saar De Zutter, Sarah De Bruyne,Wesley De Neve, and Rik Van de Walle

Ghent University - IBBTDepartment of Electronics and Information Systems - Multimedia Lab

Gaston Crommenlaan 8, B-9050 Ledeberg-Ghent, Belgium{chris.poppe, frederik.dekeukelaere, saar.dezutter, sarah.debruyne,

wesley.deneve, rik.vandewalle}@ugent.be

http://www.multimedialab.elis.ugent.be/

Abstract. Within an MPEG-21 architecture, the two key concepts arethe Digital Item, representing multimedia content, and Users, interactingwith this content. MPEG-21 introduced Digital Item Processing to allowcontent authors to describe suggested processing of their Digital Items.It standardizes ways to insert functionality into a Digital Item, as such,creating a dynamic and interactive multimedia format. Moreover, if aterminal wants to support Digital Item Processing, it needs to provide

an execution environment offering basic functionality. The semantics of this functionality have been standardized, however there is significantroom for interpretation. Consequently, a Digital Item author may notbe aware of the actual processing when using this functionality. In thispaper, a system is proposed, compliant with the Digital Item Processingspecification, to give content creators full control on the processing. Thisallows creating advanced predictable multimedia systems in an MPEG-21 environment.

1 Introduction

ISO/IEC 21000, better known as MPEG-21, is a standards suite developed byMPEG. It envisions to create a multimedia framework for the creation, delivery,and consumption of multimedia content across a wide range of networks anddevices [1]. An MPEG-21-compliant terminal is a terminal that provides thenecessary functionality to process MPEG-21 content and will further be calledMPEG-21 terminal. A content author will create a Digital Item (DI), containingreferences to multimedia content and metadata. Embedded functionality in theDI allows the author to define the way the content should be processed when anMPEG-21 terminal is used to consume it.

The work considered in this paper has been partially conducted while Frederik DeKeukelaere was a PhD student at Multimedia Lab. Since October 1, 2006 he isworking at IBM Japan Tokyo Research Laboratory.

H.H.S. Ip et al. (Eds.): PCM 2007, LNCS 4810, pp. 549–558, 2007.© Springer-Verlag Berlin Heidelberg 2007





550 C. Poppe et al.

This paper focuses on the processing part, defined in part 10 of the MPEG-21 Multimedia Framework, named Digital Item Processing (DIP)[2]. DIP allowsthe execution of script code inserted in the DI. Moreover, it provides basic func-tionality with standardized interfaces, available on any DIP-compliant terminal

(called DIP terminal). DIP has been succesfully applied in the Los Alamos Na-tional Library for the dissemination of digital objects and as means to createservices, linked to the digital objects, which can be executed by agents [3]. In thiscase the DIP terminals were all implemented by the same person and known tothe DI authors. Contrarily, in a more open environment, different DIP terminalsmight interprete the standardized interfaces differently. Regarding an MPEG-21environment, these possible implementations may lead to potential issues for thecreation of DIs containing DIP functionality. The paper shows how these differ-ent implementations prevent a DI creator to know how his content is processed.

Accordingly, a system is presented to allow the content author to take fullcontrol of the processing by using standardized DIP functionality. The proposedsystem allows an author to implement his own set of basic functionality whichcan be used on a client device when the content is consumed. As a result, theauthor exactly knows how his content will be processed, while still maintainingMPEG-21 compliance.

The outline of the paper is as follows. The next section elaborates on theMPEG-21 standard, specifically on DIP. Section 3 recapitulates the issues arisingwithin an ubiquitous MPEG-21 environment and in Section 4, we present our

solution. Section 5 elaborates on new use cases that become achievable by theproposed system. Finally, Section 6 formulates a number of conclusive remarks.

2 MPEG-21

The aim of MPEG-21, the so-called Multimedia Framework, is to enable trans-parent and augmented use of multimedia resources across a wide range of net-works, devices, and communities. In this framework, the fundamental unit of

transaction is a Digital Item (DI) as defined in part 2, named Digital Item Dec-laration (DID) [1]. This part defines the structure of a DI, which can contain(references to) multimedia content and metadata. The declaration of a DI uses anXML-based language, called Digital Item Declaration Language (DIDL), whichdefines the structure of the items. A DI can, for example, represent a musiccollection including audio files, descriptive information of every song, graphi-cal elements representing CD-covers, etc. This is a static presentation, meaningthere is no information available on how a DI should be processed by a consumer.

DIP has been created to permit the author of a DI to add explicit information

on how the item should be processed [2]. This way, an author can, for instance,add a method to the item which shows the cover of the CD, whilst playing asong and displaying a textual description. DIP allows the addition of interactionto the static declaration of a DI by means of Digital Item Methods (DIMs).These methods are written in the Digital Item Method Language (DIML), whichextends ECMAScript [5]. They are essentially code fragments inserted in the



Predictable Processing of Multimedia Content 551

XML representation of the DI. A DIP terminal will most likely contain a module,called a DIP engine, capable of executing these methods.

To extend the scripting functionality, DIP provides specific multimedia pro-cessing by defining a standardized set of functions. As such, similar behavior can

be obtained on different terminals. These functions, called Digital Item Base Op-erations (DIBOs), form a library, available on any DIP terminal, which can becalled from within a DIM. MPEG-21 has standardized the interfaces and se-mantics of this functionality and the developer of the DIP engine is responsiblefor providing an implementation. This has as advantage that different vendorscan compete in their implementation. The DIBOs are divided into different cat-egories, relating to different parts of the MPEG-21 framework. Moreover, theDOM Level 3 Core API and the DOM Level 3 Load and Save API [6] are in-cluded in DIML, allowing access, manipulation, loading, and serializing of the

DID at the XML level. For a detailled description of the DIBOs, the reader isreferred to [1].

An example of a DI containing DIP functionality is shown in Fig. 1. TheXML representation shows two Components. The first component (identified bythe id “movieResource”) defines a movie resource and a descriptor stating thiselement represents a Movie object. The second component (identified by the id“DIM”) contains a resource that represents a DIM. The first descriptor in thiscomponent is used to indicate the presence of a DIM. The second descriptor isused to denote the type of arguments the DIM takes. Consequently, the figure

shows a DIM which takes a Movie object as argument and then executes a DIBO(DIP.play() in the example) on this object. The play DIBO is one of the DIP-related DIBOs and renders the element, passed as an argument, into a transientand directly perceivable presentation. When the functionality provided by theDIBOs on a DIP terminal is not sufficient, a DI author can make use of DigitalItem eXtension Operations (DIXOs). A DIXO is externally generated code whichcan be included in the DI. DIP defines ways to invoke DIXOs from inside a DIMand a DIXO has access to the entire DIBO set through standardized bindings.So DIBOs are part of a DIP terminal, but DIXOs are typically externally created

by a DI author. The language of the DIXOs can be chosen freely, but currentlyonly DIXOs written in the Java language, called J-DIXOs, are standardized.The J-DIXO itself is a Java class (if necessary included in a Java archive) whichimplements a pre-defined J-DIXO interface. A specific DIBO has been defined,called runJDIXO, which invokes the J-DIXO.

Fig. 2 shows the different components of an MPEG-21 terminal from a DIPpoint of view. We can see that the DIP engine takes a central position; interactswith the User and is connected to additional modules related to the differentMPEG-21 parts. A DID engine parses item and forward the DIP elements to

the DIP engine. Through the DIBOs, an interface is created for a DI to utilize(part of) the underneath platform. However, there is room for interpretation of the semantics of the DIBOs and this vagueness can introduce several problemsin real life scenarios, as will be discussed in the next section.



552 C. Poppe et al.

Fig. 1. Example Digital Item

Fig. 2. MPEG-21 terminal




3 Problem Description

Given that every DIP terminal can have its own implementation of the DIBOs,a number of problems arise. Although the semantics of the function are deter-

mined, the actual implementation can vary considerably, making it hard for aDI author to compose his content without knowledge of the client application.

A short example is the alert DIBO, which takes a string as parameter andalerts the User. The semantics of this DIP-related DIBO are defined as “providesimple textual feedback to the User”[1]. It is obvious that the actual interpreta-tion of alerting a User is rather vague and can be synchronously showing a popupmessage on the screen, displaying a warning on the media player, or even justadding some information to a log file. This might issue a problem for a contentauthor if his application relies on the reaction of the user on this alert. For the

other DIBOs similar problems can be found.Several of the DIBOs can only be used to their full capacity if the creator of

the DI is aware of the actual implementation of the DIBOs at client side. Clearlya mechanism is needed that allows DI authors to control the processing of theircontent in a more detailled manner.

4 System for Predictable Digital Item Processing

The solution we propose makes use of the existing technology defined by DIP,therefore allowing full compliance with the standard. The basic idea is that aDI author provides an own implementation for a set of DIBOs (further calledauthorDIBOSet), encapsulated in a DIXO, which will then be used whenever aDIBO is called from within the DI.

To accomplish this, a DI author adds a specific method which can be calledby the DIP engine. By adding the attribute “autoRun” to the definition of theDIM and by setting its value to “true”, a DIP Engine will automatically executethis method when processing the DI. The method invokes a DIXO provided

by the content author, containing the authorDIBOSet. The DIXO itself can betransported along with the DI or can be made available online. The execution of the DIXO starts with changing the occurrences of the desired DIBO calls intocalls to the DIXO itself, as shown in Fig. 3. Since the DIXO can make use of the DIBOs and more specifically the DOM functionality, we can easily replacethe textual occurrence of the DIBO calls by calls to the DIXO. The first fourarguments of runJDIXO() are used to identify the element containing the Javaarchive or class, while the fifth argument (given the value “DIXOSet” in Fig. 3)is needed to define the appropriate class to be executed. The last argument is

an array of arguments which is passed to the class. In our system, we use thefirst element in this array to identify the DIBO call, which was replaced byusing the specific name of the DIBO (in this case the play DIBO, noticeable bythe “play” argument of the DIXO call). The rest of the array is used to passthe original arguments of the DIBO to the DIXO. This ends the initializationphase, corresponding to step 1, shown in the sequence diagram in Fig. 4. The



554 C. Poppe et al.

Fig.3. Conversion of DIBO calls to DIXO calls

Client Server

Load DI, execute autorunDIM

Get DIXO containing authorDIBOSet

DIXO

authorDIBOSet

DIBO call

Server side processing

ResultResult

1

Change DIBO calls to DIXO calls

2

3

DIXO cal

Fig. 4. Sequence diagram of the use of an authorDIBOSet

DIXO will then allow the User to choose a DIM in the updated document. If a DIBO is invoked within that DIM, a direct call to the DIXO containing theauthorDIBOSet is performed, with the appropriate arguments (step 2 in Fig. 4).At this point, the execution of the appropriate DIBO, provided by the DI author,starts and return values, if any, are passed back to the invoking DIM. As such,the content author can be sure that his implementation of the DIBO set is usedand can uniquely determine the outcome. This system is transparent to the user,

since he is not involved with the internal working of the methods, but only withthe perceivable outcome.

The DI author can deliver an implementation for every DIBO or restricthimself to the most relevant ones and reuse a number of the DIBOs available inthe client application. As shown in step 3 of Fig. 4, the author can even chooseto place the DIBO implementation on a centralized server, or provide the DIBO




functionality throug a web service, thereby reducing the processing effort on theclient device.

To increase the performance of the system, a simple check might be added inthe DIM which loads the DIXO, to see if the set is already present in the client

application. Since a content author typically produces numerous diverse DIs,there is a clear benefit to get the authorDIBOSet only once and consequentlyrefer to it in those DIs.

The described system gives an author full control of the processing, allowingto exploit his in-depth understanding of the content. Therefore, the possibilitiesfor different DIBOs are extended in the following ways.

The DIBO related to Digital Item Adaptation (DIA) allows to adapt elementsof the DI. According to the DIBO requests that an attempt is done to adapta specific element. This attempt might fail if the DIP engine has no adapta-

tion capabilities, or it does not know how to interpret the input arguments.Since the DI author is aware of the actual format of the input arguments, he ismore equiped to create the adaptation. Our system allows that the author canintroduce specific adaptation tools steered by associated metadata. MPEG-21DIA defines tools to adapt DIs based on context information. Resources canbe adapted according to descriptions of the usage environment, introducing anadvanced quality of service. For example, within DIA, means are defined to de-scribe the preferences of the user. This way, scenes of interest within a moviecan be identified and could be displayed at higher quality [7]. The rich variety

of adaptations introduced in DIA can only be used if appropriate software isavailable and if the format of the arguments is exactly known. Our system al-lows to execute this adaptation since the DI author himself will implement it. If high consuming adaptations cannot be run on the client device, the author canchoose to place them on a server.

The DIBOs related to part 2 of MPEG-21, Digital Item Declaration (DID),provide means to allow the end-user to make specific choices when dealing witha DI (for example the choice of a specific movie to be shown). The way thatthese choices are presented to the user is not defined. Through our system the

DI author can present the choices in a consistent and structured way, accordingto his own preferences.

The DIBOs related to part 3 of MPEG-21, Digital Item Identification (DII),specify means to retrieve elements from a DI according to a specific identification.The DI author is better suited to provide an implementation of these DIBOs,since he has a priori knowledge about the structure of the DI and the location of several identified elements. This way, high cost XML processing can be avoidedto increase the systems performance [8].

The DIP-related DIBOs, which mostly interact with the User, can be extended

with rich user interfaces allowing consistent presentation of different DIs from thesame content author. Playback of specific content can be achieved, by providingappropriate codecs and even entire players which offer advanced control to theUser.



556 C. Poppe et al.

The DIBOs related to part 5 of MPEG-21, Rights Expression Language(REL), can now be used according to the intention of the DI author. The authorcan set up his own license server and has, consequently, more control on theusage of his content.

A DI author can use an alternative to our proposed system. Upon creation of the DIMs within his DI, he might choose to use DIXOs instead of DIBOs. TheplayMovie DIM, in Fig. 3, will then directly contain a DIXO call instead of theplay DIBO call. This is similar to the DI formed after the initialization phase(step 1 in Fig. 4). The creation of DIs in such a manner has as an advantagethat there is no need to reconvert the DI at the Users’ side. However, when theDI author wants to change the inserted functionality this has to be done for allthe produced DIs. Our system collects the DIBO implementations in a set andallows easy updates. The author can also choose between DIBO implementations

on the DIP terminal or implementations from the authorDIBOSet, whereas thisis not possible in the alternative system, since this is hard-coded.

5 Discussions

Our system allows to accomplish new use cases and service delivery. In thissection, the use case of a museum equipped with an interactive multimedia in-frastructure is presented.

In the case of a closed environment, meaning a system in which the clientapplications and the provided content are known to the DI author, the authoris aware of the actual processing. This might be the case in an interactive mu-seum, where people get a museum-owned PDA containing an MPEG-21 terminal,which can be used to consume content provided by the museum (this was the usecase of the European project DANAE1). Since the client application is knownby the museum, several assumptions can be made on what the values are forthe different arguments of the DIBOs and what happens within the processing.If we want to achieve the goal of MPEG-21 and broaden the environment in a

way that the content authors do not need to have knowledge about the clientMPEG-21 terminals, we foresee difficulties for the content authors as mentionedabove.

Consider a museum with the infrastructure to present interactive multimediacontent. A central server stores MPEG-21 content and is responsible for deliv-ering this to available terminals. If a user enters the museum, carrying his ownPDA, cell phone or other multimedia device, containing an MPEG-21 termi-nal, he will be able to consume the museum content. Context about the con-sumers is collected and processed to generate advanced quality of service and user

experience.We can work out the use case through the system presented in this paper. The

museum creates an authorDIBOSet, implementing the relevant DIBOs, whichwill be used on all client devices. By using this authorDIBOSet, a specific user

1 http://danae.rd.francetelecom.com/




Fig.5. Multimedia application using DIP

interface can be created for communication with the user (see Fig. 5). Advertise-ments can be added to the user interface, without having to incorporate thesein the media content itself . A consistent uniform user interface is provided inany interaction and on any device. Control of the play DIBO allows to playproprietary content with a specific codec or player according to the needs of the museum (in the figure, the player is a multimedia player able to show rich

multimedia presentations including text, graphics, sound, and movies).A full implementation of the license related DIBOs, making use of internal

licensing and registration servers can be used to deliver specific content to ap-propriate users. As such, customers with a subscription to the museum can getaccess to additional content or services.

Heavy processing, like the adaptation of content is done on server side, allow-ing even the most constrained devices to consume the multimedia. Informationis gathered on the number of consuming clients, the maximum bandwidth, andthe capacity of the museum’s infrastructure. Consequently, this is taken into ac-

count when performing adaptations or delivering content. By using a centralizedserver, context can be collected and made available to each application. For ex-ample the figure shows the name of the consumer, “Katrien”, when starting aninteraction. The name is just a simple example of various contextual information,which can be collected when a consumer enters a museum. This use of contextcreates a more personalized approach to multimedia processing; it focuses on thekey player in an interactive multimedia application, namely the user.

The presented solution allows introduction of any MPEG-21 terminal into themultimedia infrastructure of the museum. Visitors can use their own multime-

dia devices resulting in reduced costs for the museum. The use case presented inthis section can be extended to other domains wherein a multimedia infrastruc-ture can be exploited, e.g., warehouses, educational environments, and culturalevents. By providing the appropriate DIBO sets and domain-specific content,the existing client application can be used in other settings.



558 C. Poppe et al.

6 Conclusions

The major contribution of our work is the development of an MPEG-21-compliantsystem to extend the way a DI author can define the actual processing of his DI.

The system gives the content authors full control over the actual processing. Anumber of advantages throughout the different parts of DIP were presented. Ause case has been presented, showing the applicability of the system in a real-lifescenario. Our system makes the processing of Digital Items more appealing andinteresting for industrial content providers, allowing advanced service delivery.

Acknowledgments. The research activities that have been described in thispaper were funded by Ghent University, the Interdisciplinary Institute for Broad-band Technology (IBBT), the Institute for the Promotion of Innovation by

Science and Technology in Flanders (IWT-Flanders), the Fund for ScientificResearch-Flanders (FWO-Flanders) and the European Union.

References

1. Burnett, I., Pereira, F., Van de Walle, R., Koenen, R.: The MPEG-21 Book, pp.195–204. Wiley, Chichester (2003)

2. De Keukelaere, F., De Zutter, S., Van de Walle, R.: MPEG-21 Digital Item Pro-cessing. IEEE Transactions on Multimedia 7, 427–434, 809–830(2005)

3. Bekaert, J., Balakireva, L., Hochstenbach, P., Van de Sompel, H.: Using MPEG-21 DIP and NISO OpenURL for the Dynamic Dissemination of Complex DigitalObjects in the Los Alamos National Laboratory Digital Library D-Lib Magazine,vol.10 (2004)

4. Poppe, C., De Keukelaere, F., De Zutter, S., Van de Walle, R.: Advanced MultimediaSystems Using MPEG-21 Digital Item Processing. In: Proceedings of Eighth IEEEInternational Symposium on Multimedia, pp. 785–786 (2006)

5. ECMA, Standard ECMA-262 ECMAScript Language Specification, 3rd edn.http://www.ecma-international.org/publications/standards/Ecma-262.htm

6. The W3C Document Object Model http://www.w3.org/DOM/

7. Devillers, S., Timmerer, C., Heuer, J., Hellwagner, H.: Bitstream SyntaxDescription-Based Adaptation in Streaming and Constrained Environments. IEEETransactions on Multimedia. 7, 463–470 (2005)

8. De Zutter, S., De Keukelaere, F., Poppe, C., Van de Walle, R.: Performance analysisof MPEG-21 technologies on mobile devices. In: Proceedings of Electronic Imaging,vol. 6074 (2006)

http://www.ecma-international.org/publications/standards/Ecma-262.htm

http://www.w3.org/DOM/

http://www.w3.org/DOM/

http://www.ecma-international.org/publications/standards/Ecma-262.htm

predictable processing of multimedia content, using mpeg-21 digital item processing

Documents