graphics subsystem in an embedded world - elinux...graphics in an embedded world • with trend to...
TRANSCRIPT
![Page 1: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/1.jpg)
Graphics Subsystem in an Embedded WorldIntegrating DirectFB into a UHAPI platform
Denis Oliver Kropp, DirectFBBas Engel, Philips Semiconductors
April 12 2006
![Page 2: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/2.jpg)
Contents
• Changing perspectives– Graphics and digital services in an embedded world– DirectFB and UHAPI– System partitioning
• Integrating DirectFB and UHAPI– FrameBuffer vs non FrameBuffer– Input handling– HW acceleration
• Summary and Q&A
![Page 3: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/3.jpg)
Graphics in an Embedded World
• With trend to digital content more emphasis on– The viewing experience– New services (VOD, content management, etc)
• Ease of use– Key factor in offering new services
• Increasing demand for high quality graphics– True Color with alpha channel (32 bit)– Slick animations and other eye-candy– HD resolutions
• Differentiation and scalability key aspect in future– Low end systems can have lower end graphics– CE companies require a versatile solution
![Page 4: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/4.jpg)
Ongoing trend: more features, more SW
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2009
1
10
100
1000
10000
100000
2
4
8
16
32
64
256
512
1024
2048
30004096
12000
32000
64000
100000
Code Size Evolution of High End TV Software
Year of Market Introduction
Kb
ytes
Predicted
Actual
So
urc
e: G
era
rd v
an
Loo
n
Consequently, we have an increasing amount of software in CE products
Increasing amount of digital services in future CE products
Integration and validation play a key role in fast TTM, so standardized APIs and building blocks are needed
![Page 5: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/5.jpg)
DirectFB
• It’s a subsystem that provides– Accelerated graphics operations
(blitting, scaling)– Multiple graphics and video layers– Input devices (remote control,
local keyboard, etc)– Fast anti-aliased text rendering– Many pixel formats (ARGB, YUV+Planar)– Video memory management (on/off-screen)
• Established technology for embedded appliances• Scalability allows state of the art desktop and embedded
environments• Adoption of OpenGL makes it competitive to the 3D market• See www.directfb.org
Java VM
AWT
XDirectFBGTK+
DirectFB - Multi Application Core
Application
X11 App
QtHAVi
MHP App
Kernel Hardware
NativeApplication
OpenGL
3DApplication
![Page 6: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/6.jpg)
UHAPI
• It’s an API specification for controlling for example– Tuner– Transport Stream Demultiplexer– ATSC Decoder– Video Mixer– PVR
• Hardware and implementation technology independent
• For home based media appliances
• Increasing industry adoption• The (proposed) CELF AVG
specification includes UHAPI• See www.uhapi.org
AV streaming platform
UHAPI
applications
middlewareOS
![Page 7: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/7.jpg)
Why DirectFB and UHAPI
• Use the best of both worlds– UHAPI focus: Audio/Video control– DirectFB focus: Graphics control
• UHAPI and DirectFB are complementary – Except where AV streaming “meets” graphics
• e.g. both have support for layer mixing and scaling– Roles have been defined and discussed at CELF June 2005 Yokohoma
• Optimize system – Performance
• Proper APIs leverage hardware capabilities– Portability
• Due to widely accepted standard interfaces• Simplicity in design
![Page 8: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/8.jpg)
Integrating DirectFB and UHAPI
Vmix
Vid Layer Gfx
Acc
eler
atio
n
Audio Video Hardware
Vid LayerGfx LayerGfx Layer
UHAPI
DirectFB implementation
DirectFB
+
Gfx Layer... LC
Linu
x O
S
CELF Application
![Page 9: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/9.jpg)
Main challenges
• Writing DirectFB „system module“• Implementing layer driver on top of uhIVmixLayer• Allocating and accessing graphics memory• Adding the accelerated driver
FBDev System Module UHAPI System Module
Graphics Memory Graphics Memory
/dev/fb /dev/memUHAPI
HW HW
ioctl()s mmap() mmap()API calls
![Page 10: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/10.jpg)
Put another layer in between?
Vmix
Vid Layer Gfx
Acc
eler
atio
n
Audio Video Hardware
Vid LayerGfx LayerGfx Layer
UHAPI
DirectFB implementation
DirectFB
+
Gfx Layer... LC
Linu
x O
S
CELF Application
?
![Page 11: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/11.jpg)
Frame Buffer Device System Module
FBDev System Module
Primary Layer Implementation
FB Device
Graphics Memory MMIO
mmap() ioctl() ioctl()ioctl() mmap()ioctl()
![Page 12: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/12.jpg)
UHAPI System Module
UHAPI System Module
Primary Layer ImplementationGraphics Memory
mmap()
Mem Device
uhIVmix
uhIVmixLayer
uhIVmixGfxLayer
SetB
uffer()
![Page 13: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/13.jpg)
Advantages of UHAPI System Module
Simplified architecture and implementation• Avoids going through very limited FBDev API layer
– Full feature set of DirectFB and HW without private extensions
• Flexible resource management– Easy match with the memory model of UHAPI and DirectFB
• Much improved interoperability– No kernel in between, notifications easily usable
• Modular approach– Different subsystems can be exchanged by others
![Page 14: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/14.jpg)
Accelerated Graphics Driver
• DirectFB focusses on hardware acceleration– However, optional due to software fallbacks
• Good reasons for acceleration– Performance boost– Parallel rendering and code execution– Keep CPU resources free for other work– Reduce latencies, e.g. in the user interface– Enable high end user interfaces
• Independent and reusable component– Could be tied to a system module though
• Well defined APIs for modules and drivers– Freedom of implementation– Can reuse existing driver libraries
Core Library
SystemModule
GraphicsDriver
Library
Hardware
Kernel Driver
![Page 15: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/15.jpg)
HW acceleration on embedded systems
• Most PC based DirectFB drivers have– Direct register programming via MMIO– Busy loop polling the state (FIFO, Idle)
• Embedded hardware– Busy loop requires too many cycles
• Use IRQs to signal finished HW operations– Command buffer mechanism
• User space puts packets into the ring buffer• Kernel driver does the actual programming• ISR processes next command packet• Trigger execution of first packet via ioctl when
the hardware is idle• Idle wait in user space when buffer is full• Can wait for finishing commands using ioctl
and serial number of packet
IRQ
Mini Driver
AcceleratedDriver
ioctl()
Shared M
emory (K
ernel/User)
wake up
put
get
Accelerator
Actual register programming
![Page 16: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/16.jpg)
Benchmarking
• Application benchmark– Realistic values for specific application scenarios, e.g. sprite animation
• Synthetic benchmark– Raw numbers, e.g. reflecting memory bandwidth or other bottle necks
100%
7%
27
9
Frames/sec
CPU Load
Frames/sec
CPU Load
With and without acceleration
![Page 17: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/17.jpg)
Input handling
InputCore
Input Driver Thread
Notification Handler Notification Handler Notification Handler
Input Device
Platform (Thread)
dispatchtime
![Page 18: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/18.jpg)
Summary
• DirectFB and UHAPI are a powerful combination– To leverage the value of state of the art digital processing and new
viewing experiences
• Not restricted by existing solutions – Simplified architecture and implementation
• Modular approach– Solution not specific to DirectFB and UHAPI
• Improved application integration– Due to standard infrastructure and proper hardware abstraction
![Page 19: Graphics Subsystem in an Embedded World - eLinux...Graphics in an Embedded World • With trend to digital content more emphasis on – The viewing experience – New services (VOD,](https://reader033.vdocuments.mx/reader033/viewer/2022060602/6056430a3cba4c37402f975c/html5/thumbnails/19.jpg)
Any questions