a java api interface for the integration of dtv services in embedded multimedia devices

7
M.Vidakovic et al.: A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices 1063 Contributed Paper Manuscript received 06/15/12 Current version published 09/25/12 Electronic version published 09/25/12. 0098 3063/12/$20.00 © 2012 IEEE A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices MilanVidakovic, Tomislav Maruna, Member, IEEE, Nikola Teslic, Member, IEEE, and Velibor Mihic, Member, IEEE Abstract Most of the modern DTV sets and set top boxes are embedded Linux devices. Apart from playing broadcast transport streams, some of these devices offer network connectivity and various applications. The development of software for these devices is not unified, so there is no compatibility between applications developed by different vendors. On the other hand, there are embedded systems developed on top of the embedded Linux kernel which offer network connectivity and a large number of applications that are available for download. Most of the applications developed for these platforms are developed in the Java programming language. However, those platforms do not support DTV services. This paper presents a proposal for the integration of DTV services into embedded systems which support the Java programming language. The proposed solution offers complete DTV functionality, along with the operation of regular applications. The proposal includes two API specifications: the Java API that is used to access DTV content from Java applications, and the DTV Middleware API that is used to connect the native code to the Java API 1 . Index Terms — Digital television integration, the mobile embedded platform, API specification. I. INTRODUCTION Consumer electronics is getting momentum by gaining more processor power, as well as by increasing the number of processor cores in embedded devices. The frequency of today's CE processors is already above 1 GHz, and increasing, and the number of cores is already two, and we expect this number to be four or more. These changes result in modern CE devices (and we focus on DTV devices) which are more capable of executing complex applications, written in programming languages other than C or C++. Most modern DTV devices (both TV sets and set top boxes) are based on the embedded Linux OS [1], [2]. Network and multimedia applications are constantly being developed to 1 This work was partially supported by the Ministry of Science and Technological Development of the Republic of Serbia under the projects No. 32014 and 44009, year 2011. Milan Vidakovic and Nikola Teslic are with the Computer Engineering and Computer Communications Department, Faculty of Technical Sciences, University of Novi Sad, Serbia (e-mail: [email protected], nikola.teslic@rt- rk.uns.ac.rs). Tomislav Maruna and Velibor Mihic are with the RT-RK Computer Based Systems LLC, Novi Sad, Serbia (e-mail: [email protected], [email protected]). provide some additional functionality, other than watching TV. One of the first attempts to integrate network and multimedia into DTV was the HbbTV [3]. This technology collects data from the Internet, while displaying the cable or antenna broadcast transport stream. The typical HbbTV implementation is based on the extension of a web browser. It enables the simultaneous display of the transport stream and CE-HTML [3]. While this technology is interactive, it is not a standard platform to develop and use applications. In order to increase interactivity, [4] uses a hybrid solution in which a dedicated server demodulates the DTV content and then broadcasts it using the IP streaming. Receivers are mobile devices, tablets, and personal computers, with a dedicated client application installed. The main purpose of this system is to be used in areas where DTV broadcast signals are not good enough, while the high speed mobile network or the Internet are available. Some vendors offer applications that can be installed on a TV set, but those applications are customized for a particular device. For example, it is possible to install a well-known video chat application into a TV set, but the list of supported TV sets is limited. Furthermore, if a new model of TV emerges, it will be necessary to adapt the program to work with that new device. Another example is the local network media player embedded in a TV set. This application is capable of finding and playing multimedia files in the local network. This application is also available for only a limited number of devices. All these solutions lead to an unnatural way of developing and using applications. Instead of making regular system applications to be used together with DTV applications, custom applications are being developed to fit into the DTV framework. Most multimedia and networking applications that are present in modern TV sets cannot be used with other DTV devices, especially from other vendors. Furthermore, these applications cannot be installed on any other devices than TV sets. On the other hand, there are embedded systems which support the Java programming language. Those systems have a number of multimedia and networking applications developed and available for download. However, those systems do not support DTV. In this paper we present our proposal to integrate DTV services in systems which support the Java programming language. Since all DTV vendors already have DTV services developed for native applications, we propose additional software layers to enable support for Java applications. Furthermore, we propose a procedure on how to adapt DTV

Upload: velibor

Post on 25-Mar-2017

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A java API interface for the integration of DTV services in embedded multimedia devices

M.Vidakovic et al.: A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices 1063

Contributed Paper Manuscript received 06/15/12 Current version published 09/25/12 Electronic version published 09/25/12. 0098 3063/12/$20.00 © 2012 IEEE

A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices

MilanVidakovic, Tomislav Maruna, Member, IEEE, Nikola Teslic, Member, IEEE, and Velibor Mihic, Member, IEEE

Abstract — Most of the modern DTV sets and set top boxes are

embedded Linux devices. Apart from playing broadcast transport streams, some of these devices offer network connectivity and various applications. The development of software for these devices is not unified, so there is no compatibility between applications developed by different vendors. On the other hand, there are embedded systems developed on top of the embedded Linux kernel which offer network connectivity and a large number of applications that are available for download. Most of the applications developed for these platforms are developed in the Java programming language. However, those platforms do not support DTV services.

This paper presents a proposal for the integration of DTV services into embedded systems which support the Java programming language. The proposed solution offers complete DTV functionality, along with the operation of regular applications. The proposal includes two API specifications: the Java API that is used to access DTV content from Java applications, and the DTV Middleware API that is used to connect the native code to the Java API1.

Index Terms — Digital television integration, the mobile embedded platform, API specification.

I. INTRODUCTION

Consumer electronics is getting momentum by gaining more processor power, as well as by increasing the number of processor cores in embedded devices. The frequency of today's CE processors is already above 1 GHz, and increasing, and the number of cores is already two, and we expect this number to be four or more. These changes result in modern CE devices (and we focus on DTV devices) which are more capable of executing complex applications, written in programming languages other than C or C++.

Most modern DTV devices (both TV sets and set top boxes) are based on the embedded Linux OS [1], [2]. Network and multimedia applications are constantly being developed to

1 This work was partially supported by the Ministry of Science and

Technological Development of the Republic of Serbia under the projects No. 32014 and 44009, year 2011.

Milan Vidakovic and Nikola Teslic are with the Computer Engineering and Computer Communications Department, Faculty of Technical Sciences, University of Novi Sad, Serbia (e-mail: [email protected], [email protected]).

Tomislav Maruna and Velibor Mihic are with the RT-RK Computer Based Systems LLC, Novi Sad, Serbia (e-mail: [email protected], [email protected]).

provide some additional functionality, other than watching TV. One of the first attempts to integrate network and multimedia into DTV was the HbbTV [3]. This technology collects data from the Internet, while displaying the cable or antenna broadcast transport stream. The typical HbbTV implementation is based on the extension of a web browser. It enables the simultaneous display of the transport stream and CE-HTML [3]. While this technology is interactive, it is not a standard platform to develop and use applications.

In order to increase interactivity, [4] uses a hybrid solution in which a dedicated server demodulates the DTV content and then broadcasts it using the IP streaming. Receivers are mobile devices, tablets, and personal computers, with a dedicated client application installed. The main purpose of this system is to be used in areas where DTV broadcast signals are not good enough, while the high speed mobile network or the Internet are available.

Some vendors offer applications that can be installed on a TV set, but those applications are customized for a particular device. For example, it is possible to install a well-known video chat application into a TV set, but the list of supported TV sets is limited. Furthermore, if a new model of TV emerges, it will be necessary to adapt the program to work with that new device.

Another example is the local network media player embedded in a TV set. This application is capable of finding and playing multimedia files in the local network. This application is also available for only a limited number of devices.

All these solutions lead to an unnatural way of developing and using applications. Instead of making regular system applications to be used together with DTV applications, custom applications are being developed to fit into the DTV framework. Most multimedia and networking applications that are present in modern TV sets cannot be used with other DTV devices, especially from other vendors. Furthermore, these applications cannot be installed on any other devices than TV sets.

On the other hand, there are embedded systems which support the Java programming language. Those systems have a number of multimedia and networking applications developed and available for download. However, those systems do not support DTV. In this paper we present our proposal to integrate DTV services in systems which support the Java programming language. Since all DTV vendors already have DTV services developed for native applications, we propose additional software layers to enable support for Java applications. Furthermore, we propose a procedure on how to adapt DTV

Page 2: A java API interface for the integration of DTV services in embedded multimedia devices

1064 IEEE Transactions on Consumer Electronics, Vol. 58, No. 3, August 2012

Middleware to fit the proposed software stack. We have also included our experience with this procedure.

There are DTV solutions which are implemented using the Java programming language. Solutions [5] and [6] provide the basic DTV playback features. Reference [5] is a customized solution for a specific hardware platform. Reference [6] is based on a modification of the frame buffer support. This solution enables the DTV playback to be displayed on a specific part of the screen. Both solutions do not offer standardized support for DTV features and are heavily customized implementations that are not universal enough. Also, neither implementation offers the level of abstraction that could be used for DTV standardization, or on other platforms or operating systems. On the other hand, [7] presents a different approach to DTV integration. The approach presented in the mentioned paper does not include heavy customization of the original OS, and works on a broad range of devices (embedded devices and personal computers). Besides implementing DTV services, [7] deals with setting standards.

This paper extends the idea presented in [7] in a way that it provides more details and proposes some improvements to the initial version. We propose a Java API and the native API which could be used to standardize DTV support in Java-based applications. The Java API proposal is not bound to any operating system, or to any platform. We also present an implementation of the DTV system, which is based on our proposal.

The rest of the paper is organized as follows. Section 2 considers a possibility of installing DTV services on the embedded devices which support the Java programming language. Section 3 presents the details of the DTV integration proposal. Section 4 presents our implementation which is based on the solution proposed in section 3. Section 5 describes the limitations of the current implementation. Section 6 outlines the conclusion of the presented work and proposes some future work.

II. DTV IMPLEMENTATION ON EMBEDDED DEVICES WITH

JAVA SUPPORT

There are three important elements in each DTV device implementation:

- a video playback subsystem, - a data structure handler, and - a DTV application. The video playback subsystem provides a broadcast

transport stream playback which can be performed using software or hardware solutions. The former sends the transport stream from one subsystem to another (e.g. a demultiplexer, a decoder, a filter, etc.) and finally to the video frame buffer, all using software. The latter simply routes the transport stream from one subsystem to another and finally to the video frame buffer, all using hardware.

The data structure handler deals with complex data structures which are present in the broadcast transport stream.

Some typical structures are: PES packets (Packetized Elementary Stream), the PSI table (Program Specific Information), subtitles, teletext, the EPG (Electronic Program Guide), etc. It is necessary to parse those structures, extract and use or display appropriate data from them. This feature is usually implemented in DTV middleware [3].

A DTV application is either a native application, or a modified browser [3] which uses the video player and DTV middleware to display a broadcast transport stream and display parsed data like EPG, teletext, subtitles, etc.

All the three elements are usually implemented in embedded Linux systems as customized solutions. On top of those implementations, multimedia and network programs are developed to provide users with social networking, file browsing, news reading, etc. Those applications are customized to the appropriate brand and model of DTV devices and usually cannot be shared among different vendors.

Additionally, there are embedded devices which have a large number of applications, but do not support DTV features. Most of them are just embedded devices, which are connected to a home TV set and network. Some of those devices are connected to existing set top boxes, rather than having their own tuner and DTV software stack. This means that at this moment there are only two types of devices for home entertainment:

- DTV devices (TV sets and set top boxes) which support DTV features, but do not have a large number of applications supported, and,

- various embedded devices which have a large number of applications available, but do not support the DTV. Most of them support the Java programming language.

We have identified the key elements for the implementation of Java-based DTV support in embedded devices. Our proposal depends on the following layers of the software stack:

- the DTV application, - the Java API layer, - the DTV JNI (Java Native Interface), - the DTV middleware API and DTV middleware, and - DTV hardware drivers.

The DTV application is written in the Java programming language and uses the DTV Java API that is proposed in this paper. It also uses standard GUI components to display DTV content.

The JNI layer connects the DTV Java API to the native middleware code. The DTV Java API encapsulates DTV service features so they can be used by Java applications.

The DTV middleware does all the complex data handling and is developed by DTV vendors. In order to have the middleware standardized, it should implement the DTV middleware API. The middleware is implemented in native programming languages like C++ or C, and is linked to the JNI layer as a library. In existing DTV devices, the DTV Middleware is the key component, since it does all the

Page 3: A java API interface for the integration of DTV services in embedded multimedia devices

M.Vidakovic et al.: A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices 1065

processing. When converting existing solutions to the one suggested in this paper, it is necessary to change existing DTV Middleware so it would be used from the JNI layer instead of the native code. More details will be presented in section 3.

At the bottom of the software stack, there are hardware drivers providing uniform access to DTV hardware. If the Linux kernel supports DVB core drivers, then there is a standardized way of having low-level support for DVB services, like demultiplexer, frontend, etc.

III. DTV INTEGRATION OUTLINE

The integration of DTV into Java-based embedded systems depends on multiple software layers. Each layer provides some specialized service for others. The system outline is presented in Fig. 1 and consists of the following layers:

- the DTV application, which is a Java application, and which is capable of performing video playback of the transport stream, as well as to display DTV-related data like EPG, teletext, etc.,

- the DTV Java API layer which encapsulates the DTV service into Java classes, interfaces and enumerations,

- the JNI layer which connects the DTV Java API and the DTV middleware,

- the DTV middleware API, which connects the JNI layer to the DTV middleware,

- the DTV middleware, which performs the DTV processing, and which is vendor-specific, and finally

- the DTV tuner driver and the DVB core driver (both in the kernel space).

Fig. 1. Proposed software structure.

The system outline starts with a regular Java application which works as a DTV application. The application uses regular GUI components which are used to play video clips. It also uses the DTV Java API to work with necessary DTV data structures like the Service list (i.e. the Channel list), EPG (Electronic Program Guide), etc. When a method of the DTV Java API is invoked, it is forwarded to the JNI layer which then invokes the appropriate DTV Middleware function. In the opposite direction, when a DTV event occurs, the DTV Middleware invokes the callback method in the JNI layer, which then invokes the appropriate Java method in the DTV Java API. If necessary, this layer then activates the appropriate listener in the Java application.

A. Video playback

Video playback is done by modifying the MediaPlayer subsystem of the underlying OS. There are two ways of implementing video playback: software and hardware.

Fig. 2 shows layers which are involved in the software playback. At the bottom of the stack, the DTV Middleware provides the transport stream. The new thread has been added to the existing MediaPlayer implementation. This thread reads the transport stream and fills the playback buffer. From this layer on, standard components are used and there are no more modifications to the OS.

Fig. 3 shows layers which are involved in the hardware playback. DTV Middleware sends the appropriate commands to the hardware to route the transport stream from and to various hardware devices and finally to the MediaPlayer component. From this layer on, standard components are used.

MediaPlayer

DTV Middleware

Video playbackcomponent

DTV Application

DVB Thread

Fig. 2. Software-based video playback modules.

Fig. 3. Hardware-based video playback modules.

Page 4: A java API interface for the integration of DTV services in embedded multimedia devices

1066 IEEE Transactions on Consumer Electronics, Vol. 58, No. 3, August 2012

An additional type of URI (Uniform Resource Identifier) is added to the list of supported URIs by the MediaPlayer component. The new URI type consists of three properties: frequency, video PID (Packet ID), and audio PID.

The frequency, video PID and audio PID are used to describe a service, and are embedded into the transport stream. The middleware extracts these properties and forwards them to the DTV Java API in a list of available services. Each service in that list holds those three properties (frequency, video PID, and audio PID). When a user selects a channel to watch, those three properties are used to compose a URI, which is then passed to the video playback GUI component. This component passes the URI to the MediaPlayer subsystem which uses three properties to set a new stream source and to start the playback. The playback is performed either by moving, or by routing the transport stream data from the demultiplexer component to the MediaPlayer component, depending on the type of the playback subsystem existing in the DTV device. In short, the code that starts the playback of the current channel would be: videoPlayback.setVideoUri(

dtvManager.getServiceListControl(). getCurrentService().toUri());

The videoPlayback variable is the standard video playback GUI component, and the dtvManager variable represents the main component of the DTV Java API. The latter is proposed in this paper and described in the next section.

B. DTV Java API

The DTV Java API consists of several components, the DTVManager component being the main one. This component holds all other components and is used to access them. It implements the IDTVManager interface, just as all other components implement their appropriate interfaces (e.g. the ServiceManager component implements the IServiceComponent interface), as shown in Fig. 4. When a Java application needs to work with the DTV service, it just gets the DTVManager component as a system service.

If there is a need to get a component that is a part of the DTV service, it is fetched from the DTVManager: ISetupControl ctrl =

dtvManager.getSetupControl();

The DTV manager consists of the following modules (as shown in Fig. 4): IDTVManager, IVideoControl, IAudioControl, IEpgControl, IMhegControl, ISubtitleControl, ITeletextControl, IConditionalAccessControl, ISetupControl, IReminderControl, IServiceListControl, IMasterServiceList, IServiceList, and IPvrControl.

The key components are: IDTVManager, IServiceListControl, IServiceList and IMasterServiceList. The IServiceListControl (as shown in Fig. 5) is the main channel list controller. It consists of a master service list (IMasterServiceList component) and a list of favorites (IServiceList component). The master service list is the main list of channels available to the client. The list of favorites is implemented as a list of IServiceList components, and is designed to store the user’s favorite channels.

IAudioControl

...

IConditionalAccessControl

...

IEpgControl

...

IMhegControl

...

IPvrControl

...

IReminderControl

...

IServiceList

...

IMasterServiceList

...

ISetupControl

...

ISubtitleControl

...

ITeletextControl

...

IVideoControl

...

IDTVManager

...

IServiceListControl

Fig. 4. Components of the DTV service.

serviceInfo

currentService

masterServiceList

favorites

MasterServiceList

...

ServiceList

...

...

IServiceList(os)

...

IMasterServiceList(os)

Service

...

...

ServiceInfo

...

...

ServiceListControl

...

...

IServiceListControl(os)

Fig. 5. ServiceListControl component structure.

Each channel is encapsulated in the Service class. This class

consists of the basic information (e.g. name), and additional information (e.g. frequency, audio PID, video PID, etc.) stored in the ServiceInfo class. The API is designed not to duplicate the data in the system. Since the middleware holds all the data, the Java API does not need to hold the copy of it. It rather fetches the data from the middleware and uses it in the Java layer. Only the restricted amount of data is duplicated. This is particularly important in embedded devices, since they have the limited amount of memory available. IPvrControl is a Personal Video Recorder control which is used to record DTV content to storage, and to reproduce the stored content (the media playback). IEpgControl is the Electronic Program Guide component which is used to display additional data about the current DTV content, as well as to manipulate the scheduled programs (also fetched from the EPG). ITeletextControl, ISubtitleControl and IMhegControl components are used to extract and manage appropriate data from the broadcast transport stream. IReminderControl manages reminders (calendar events linked to the DTV content). ISetupControl is used to store and retrieve various

Page 5: A java API interface for the integration of DTV services in embedded multimedia devices

M.Vidakovic et al.: A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices 1067

settings for the DTV device (contrast, brightness, timers, etc.). IVideoControl and IAudioControl manage video and audio parameters fetched from the transport stream. The IConditionalAccessControl manages scrambled channels and other protected content, like the Parental Control.

C. The JNI Layer

The JNI layer consists of native method declarations and implementations. Native methods are declared in the DTV Java API, and implemented as a library. There are two directions in method invocation: from the Java application to the middleware, and vice versa.

The first direction is rather straight-forward: the Java application invokes a method in the DTV Java API; the DTV Java API method invokes the method in the JNI layer, which then invokes the appropriate method in the middleware. All method invocations are in the same thread space, meaning that each method invocation is embedded in the method invocation of the layer above.

The Java-to-native direction is implemented as an invocation of the native method in the JNI layer. For example, getting additional information about a service is done by invoking the following method: JNIEXPORT jobject JNICALL Java_

_dtv_service_ServiceNative_getServiceInfo (JNIEnv *, jobject, jint);

The method above invokes the appropriate function of the middleware and returns service information back to the Java layer.

The other direction is a bit more complicated: when a DTV event occurs, the middleware invokes the callback method in the JNI layer (using pointers to functions), and this layer then invokes the appropriate Java method in the DTV Java API. The invocation of the Java method from the native layer is not straight-forward, and is not placed in the same thread space, but it is feasible to perform such invocations by attaching the current thread to the JVM (Java Virtual Machine) thread.

D. The DTV Middleware API and DTV Middleware

The DTV Middleware is a vendor-dependant component and is usually part of the existing native DTV system. It is used to provide a broadcast transport stream to the player, data extraction from the transport stream and the processing of that data. This is a rather complex piece of software which requires a lot of system resources. For example, the middleware we work with has approximately 500,000 lines of code, all done in the C programming language. When working, it consumes 30 or more threads. The complexity of the Middleware and the need for timely responses are the reasons why we haven’t converted it to pure Java. Instead, we have adapted the Middleware to be used by Java applications, as presented in this paper.

The DTV Middleware is the key component of any DTV implementation. It originates from the pure native environment which is still used by DTV vendors. When converting a DTV application from native to Java

environment, it is necessary to change the Middleware first. In native DTV implementations (“regular” DTV devices), the DTV Middleware is activated from the native code. All callbacks also end up in the native code. In order to change the DTV Middleware, it is necessary to decouple it from the native DTV application. All operations and callback functions need to end up in the JNI layer, so the Java applications would be able to invoke them, or to be notified using callbacks. To do so, all operations and callback declarations should fit the appropriate header files – DTV Middleware API. This way it is possible to fit any Middleware implementation to the proposed software architecture. For example, the function which gets the additional service information is declared as: bool cmMSLIST_GetServiceInfo(uint index,

tMLIST_ServiceInfo* pInfo); In the example above, it is necessary to use

tMLIST_ServiceInfo structure (from the DTV Middleware API) which describes a service.

If the Middleware is adapted to fit the DTV Middleware API, it is accessible by Java classes from the DTV Java API layer. We have already converted native structures from the DTV Middleware API to Java and vice versa. There are several ways of doing automated Java-to-native and native-to-Java translation[8], but we have chosen to do the manual conversion, since we had the exact specification of source and target Java classes and interfaces.

The proposed architecture in this paper also requires that the Middleware should be built as a library that will be linked to the JNI layer, as required by the Java specification.

IV. IMPLEMENTATION

We have implemented two test systems based on the proposal from the previous sections. The first system consists of an embedded board with the popular open source OS [7] installed, an USB DTV tuner, and a TV set connected to the HDMI output connector of a board, as depicted in Fig. 5. The second system uses a hardware tuner instead of the USB tuner, and uses the hardware-based playback.

Fig. 6. The structure of the software-based experimental system.

We have installed the OS on top of the embedded Linux

kernel. The tuner drivers had to be ported for this board, and that was one of two major adaptations. The second adaptation was to port the DTV Middleware to be used by the Java application, as required by the specification in section 3. The adaptation of the Middleware had to deal with the customized standard library, which is a replacement for the Linux standard libraries. We have experienced a number of problems while porting the native code with this library. Several libraries have not been implemented in the customized library.

Page 6: A java API interface for the integration of DTV services in embedded multimedia devices

1068 IEEE Transactions on Consumer Electronics, Vol. 58, No. 3, August 2012

Some of them have not been completely implemented, while others have not been implemented at all.

The Java DTV application has been developed as a regular application. Since it is a regular application, a user can switch from that application to another application (browser, chat, etc.). That could stop or obscure the video playback. This limitation (and our solution) is presented in section 5 (Limitations). In short, we have developed several plugins offering the most popular features that would enable the user to work with them without leaving the DTV application (as displayed in Fig. 7):

- read news headlines or weather data, - search a movie database; the search can be initiated using

the EPG data, so the user does not need to enter any search data like the name of the movie – it is already present in the EPG data,

- read social network messages, or even post messages, - get the stock exchange information, exchange rates, etc. The integration of social network features in the DTV

application offers a new way of measuring TV show rates and a new way of measuring the feedback from viewers. We have provided an instant feedback to TV shows via popular social networks. We have also developed several micro-games which can be started on top of the DTV application. The micro-game is a kind of game which is simple and can be used to entertain the user if the TV show is not interesting enough. The use of micro-games can be used to measure rates, too.

Fig. 7. TV application with plugins.

This solution is not bound to DTV devices only (TV sets and

set top boxes). It can also be applied to phones and tablets, providing that those devices have the USB host mode enabled, so the DTV tuner can be connected to the device. We have also tested our solution on personal computers (both desktops and laptops). We also use personal computers with a Linux distribution for the development and debugging of the DTV Middleware. This has been done because there was a good support for debugging, and a lot of tools. This has also proved that our proposal can be applied to a wide range of systems and devices.

This solution also offers another interesting feature: the existing tablet or mobile phone can be used as a remote control for the TV, as depicted in Fig. 8. The regular mobile phone can be used as a remote control, by using the remote control application also developed as a part of the project. The remote

control phone application is connected to the TV set and sends user inputs (touch, scroll, etc.). The connection is established using a wireless network. This type of remote control is not limited to the DTV application only. It can be used to play games, browse the web, and control all other applications.

Fig. 8. Tablet as a remote control.

V. LIMITATIONS

There is one significant limitation with the OS we have used – there can be only one active application window at a time. The user cannot watch the TV while browsing, sending email, etc. The user can, however, switch from the TV application to some other application, but then they will not be able to watch TV.

There are several solutions to this problem, of which we have tried two:

- to modify the OS to display TV content in a specific region of a window, or in the background [6], or

- the TV application is just another application and it has plenty of features integrated into the TV application as plugins, so the user does not have to leave the TV application [7].

The first solution offers continuous playback regardless of which application is started. However, it requires significant changes to the OS, as well as to the device vendor source code. It changes the frame buffer functionality so the TV content is always displayed in a certain region of a screen.

The second solution (that we propose in this paper) does not require significant changes to the OS. The user can watch TV while using built-in plugins, so that there is no need to exit the application if they want to use some of the most popular applications.

As described in section 3, it is still not possible to convert the complete DTV software stack to Java. The DTV Middleware still needs to be implemented as a native library, since it is necessary for it to respond in a timely manner to all DTV events. The amount of data exchanged between layers is also critical. At this moment, embedded systems still have the limited amount of memory available, so it is important to keep the memory use low.

VI. CONCLUSION

This paper presents one proposal of the Java API for the integration of DTV services into embedded devices. The proposal also defines the software stack necessary to be implemented in

Page 7: A java API interface for the integration of DTV services in embedded multimedia devices

M.Vidakovic et al.: A Java API Interface for the Integration of DTV Services in Embedded Multimedia Devices 1069

order to have DTV services integrated: the Java TV application, the DTV Java API, the JNI layer, the DTV middleware API, the DTV middleware, and hardware drivers.

In this proposal, the DTV application is implemented as a regular application capable of performing the broadcast transport stream playback. This application is also able to fetch and display various data existing in the transport stream (e.g. EPG, teletext, subtitles, etc.). The complete DTV functionality is encapsulated in the DTV Java API. In order to stay in the DTV application when using additional features, the application is enriched with various plugins, so the user does not have to leave the application in order to browse the internet, read news headlines, visit social networks, etc. Even if it were possible to have multiple windows at the same time, the plugins would still offer context-sensitive features, which would be related to the current TV show the user is watching.

The implementation based on our proposal has been shown, and the corresponding test system has been presented in this paper. Our experience with porting native software to this platform has been presented, too.

The presented proposal does not depend on any operating system. It has been tested on several embedded systems, as well as on a Linux distribution, on a personal computer. However, the key component – the DTV Middleware still needs to be implemented as a native library, and therefore needs to be compiled on a host operating system. As processor power and number of cores rise, this component will eventually be converted to the Java programming language and will not depend on the operating system.

The fact that the DTV application is just another application makes the DTV device a very powerful device with a large screen supporting HD resolution. This device is not a DTV device only, and can be used to run almost all applications existing on the Internet.

The device can be remotely controlled using the mobile phone or tablet. It is easier to control the device that way than using the IR remote control.

One of the advantages of our proposal is that it can also be applied to a large number of existing embedded devices which do not support DTV at this moment.

Our future work focuses on the further development of the applications connecting DTV to social networks, as well as on the integration to other technologies, such as the HbbTV.

REFERENCES [1] S. P. Moon, J. W. Kim, K. H. Bae, J. C. Lee, and D. W. Seo, “Embeded

Linux implementation on a commercial digital TV system”, IEEE Transactions on Consumer Electronics, vol. 49, no. 4, pp. 1402-1407, Nov. 2003.

[2] G. Miljkovic, V. Mihic, M. Ristic, and V. Kovacevic, “DTV linux device abstraction for embeded systems” 14th IEEE International Symposium on Consumer Electronics (ISCE), June 2010, pp. 1-6.

[3] Z. Lukac, M. Radonjic, B. Veris, T. Maruna, and N. Kuzmanovic, “The experience of implementing a Hybrid Broadcast Broadband Television on network enabled TV set”, 34th International Convention MIPRO, May 2011, pp. 840-844.

[4] Hendrawan, N. S. Sugiharto, D. Hermawan, “Design and Implementation of Interactive Mobile TV over Hybrid Network of DVB Broadcast Network and Unicast Network”, 6th International Conference on Telecommunication Systems, Services, and Applications (TSSA), Dec. 2011, pp. 288-291.

[5] Y. S. Lu, C. H. Lee, H. Y. Weng, and Y. M. Huang, “Design and implementation of digital TV widget for Android on multi-core platform”, International Computer Symposium (ICS), December 2010, pp. 576-580.

[6] N. Kuzmanovic, T. Maruna, M. Savic, G. Miljkovic, and D. Isailovic, “Google's Android as an application environment for DTV decoder system”, 14th IEEE International Symposium on Consumer Electronics (ISCE), June 2010, pp. 1-5.

[7] M. Vidakovic, N. Teslic, T. Maruna, and V. Mihic, “Android4TV: a proposal for integration of DTV in Android devices”, 30th IEEE International Conference on Consumer Electronics (ICCE), January 2012, pp. 441-442.

[8] J. Martin, and H. A. Muller, ”Strategies for migration from C to Java”, 5th European Conference on Software Maintenance and Reengineering, March 2001, pp. 200-209.

BIOGRAPHIES

Milan Vidakovic received the BSc, MSc and PhD degrees in electrical engineering from the Faculty of Technical Sciences, University of Novi Sad, in 1995, 1998 and 2003 respectively. He is a professor at the Computing and Control Department, University of Novi Sad. His research interest covers object oriented programming, network and distributed programming.

Tomislav Maruna (M’10) received the BSc and MSc degrees in electrical engineering from the Faculty of Technical Sciences, University of Novi Sad, in 1994 and 2001 respectively. He is leading the R&D process in RT-RK Computer Based Systems LLC. His main occupation is the system and multimedia software for embedded systems. His research interest is the deployment and

usability of embedded operating systems within consumer devices.

Nikola Teslic (M’06) received the BSc, MSc and PhD degrees in electrical engineering from the Faculty of Technical Sciences, University of Novi Sad, in 1995, 1997 and 1999 respectively. He is a professor at the Computing and Control Department, University of Novi Sad. He is CTO of RT-RK Computer Based Systems LLC, delivering development services and products in

the consumer area. His research interest covers the design of real time systems, audio and video processing and the design of testing systems.

Velibor Mihic (M’07) received the BSc and MSc degrees in electrical engineering from the Faculty of Technical Sciences, University of Novi Sad, in 1994, 2006 respectively. He holds the position of engineering manager at RT-RK Computer Based Systems LLC. He is also an assistant at the Computing and Control Department at the University of Novi Sad. His research

interest covers software programming in real time and embedded systems.