introduction to creating 3d ui with...
TRANSCRIPT
![Page 2: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/2.jpg)
Agenda
• Introduction to BeagleBoard multimedia architecture features.
• Review of Linux graphic stacks
• Introduction to clutter programming
• Hands on and exercises
• Review some example applications
• Questions section (also allowed during the class)
![Page 3: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/3.jpg)
Requirements
• BeagleBoard booting with ESC SD image + Patches.
• Install the SGX Drivers (accepting the SGX drivers license):
$ ./gfx_rel_ddk.sh (type ‘q’, then ‘yes’ (if you agree...)$ cd gfx_rel$ ./install.sh$ reboot(restart your board)
![Page 4: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/4.jpg)
Introduction
• Primary objective for this class is to introduce audience to the main concepts and technologies available to start developing 3D UIs with BeagleBoard.
• This doesn’t class wont cover extensively the APIs available and assumes basic knowledge on C programming, Linux and computer graphics.
![Page 5: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/5.jpg)
BeagleBoard and OMAP3 architecture
• Provides architecture with several multimedia features:
• C64x DSP
• Cortex A8 with Neon
• SGX graphics chip
• How to maximize the hardware features with the software stack?
![Page 6: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/6.jpg)
SGX Graphic Accelerator• Standards supported: OpenGL ES 1.1, 2.0 and
OpenVG.
• SDKs available for Linux and Windows from Imgtech[0]. Enables development and training on host machine.
• Drivers and libraries for BeagleBoard available early next year.
![Page 7: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/7.jpg)
OpenGL ES for the UI?
• OpenGL ES is designed for generic propose and doesn’t provide complete solution for UI creation:
• Input or event (picking) handling.
• Text rendering.
• EGL layer provides standard API for OGLES implementation.
![Page 8: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/8.jpg)
UI integration with the hardware features
• How to create user interface that integrates the best of the platform without having to write custom software stacks.
• DSP Codecs
• 2D / 3D hardware acceleration
• Input / Output
• Using standard open source technologies.
![Page 9: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/9.jpg)
Understanding Linux graphics stack
• Graphics stacks has several components:
• Graphic environment (windowing system)
• Support libraries
• Widget toolkits
![Page 10: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/10.jpg)
Popular embedded Linux graphic libraries
• X Server
• DirectFB
• Qtopia
• Cairo
• freetype / fontconfig
• pango
• gstreamer
![Page 11: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/11.jpg)
3D on Linux: X acceleration
• Several approaches:
• AIGLX is the main one.
• Xgl is abandonated
• Existing acceleration for Linux uses the GLX protocol.
![Page 12: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/12.jpg)
3D on Linux: GLX
• GLX is the protocol to bing X with OpenGL:
• Handle contexts
• Send OpenGL commands over network
• Can do direct or indirect rendering
![Page 13: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/13.jpg)
3D on Linux: GLX
X Client X Server
OpenGL Driver
Hardware
Accelerator
GLX commands
OpenGL Commands when
using indirect rendering
OpenGL Commands if DRI
is used and X server is on
the same machine of X client
![Page 14: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/14.jpg)
GLX and OpenGL ES
• GLX doesn’t support OpenGL ES
• Alternatives are software OpenGL implementation with OpenGL ES backend: experimental projects doing this with Mesa backends for OGLES
• Existing projects doesn’t support all the OpenGL functionality
• Gallium 3D project is doing OGLES backend for the new DRI interface. Not available yet.
![Page 15: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/15.jpg)
EGL
• EGL is an standard API for OpenGL ES and OpenVG to interact with the native windowing system.
• Depends from the vendor what the native windowing system is.
• Current SGX drivers on Beagle supports either the framebuffer or X window connection (but the former is not working well yet)
![Page 16: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/16.jpg)
3D without X: clutter project
• Library to create fast, rich, animated user interfaces.
• Based on standard open source technologies used in Gtk+
• Support different backends: OpenGL and OpenGL ES. Uses an abstraction layer: COGL.
![Page 17: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/17.jpg)
Clutter features
• Provides font rendering with pango.
• i18n
• Event handling and picking.
• Provides animation support.
• Simple basic widgets: labels, images. Complex widgets are not difficult to develop.
![Page 18: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/18.jpg)
Clutter features• Features integration with:
• Cairo: advance image rendering, door open for hw acceleration.
• Gstreamer: enables integration with hw accelerated codecs.
• Webkit
• box2d: 2D physics engine
• Support for several language bindings.
![Page 19: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/19.jpg)
Clutter features
• Supports an EGL backend: work with OMAP3 SGX driver.
• EGL backend supports tslib for input, generic kernel event interface in the future.
• Gtk+ integration allows to run a clutter scene inside a standard Gtk+ windows over X or DFB.
![Page 20: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/20.jpg)
Clutter disadvantages
• Needs more widgets.
• Lack on screen keyboard.
• Young project, need more documentation and examples. Getting there.
![Page 21: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/21.jpg)
Clutter usage scenarios
• Pure clutter solution. • Clutter over GTK+ solution.
![Page 22: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/22.jpg)
Visualizing the possible software stacks
Your application
Clutter
and support libraries
SGX
Hardware
Frame
buffer
OpenGL ES / OpenVG OpenGL ES / OpenVG
Clutter Framework:
Font rendering, animation, eventscairo: 2D render
Webkit: html render
box2d: 2D physics
Gstreamer: multimedia
framework
SGX
Hardware
Frame
buffer
OpenGL
ES
SGX
Hardware
Frame
buffer
X Server
Gtk +Your application
Your application
OpenGL ES based solution Pure clutter solution Mixed clutter and Gtk+ solution
![Page 23: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/23.jpg)
Hands on...
• Hello world
• Events
• Animation
• Images: textures and cairo
• 2D physics
• Gstreamer: video and audio
• Putting all together
![Page 24: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/24.jpg)
Examples:$ cd ESC341$ touch * # Prevents some timestamp warnings$ make$ ./hello$ ./hello2$ ./events$ ./bubbles$ ./clock$ ./videotest$ ./video /root/gst/bbb_320x180.mp4
![Page 25: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/25.jpg)
Gstreamer and OMAP 3
• Gstreamer video sink available (see ESC presentation on Gstreamer)
• Gstreamer can use NEON optimized Codecs on ARM side, or plugins for TI’s CodecEngine for DSP Codecs.
![Page 26: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/26.jpg)
Q&A
![Page 27: Introduction to creating 3D UI with BeagleBoardjkridner.s3.amazonaws.com/esc/ESC-341Slides_Dompe.pdf · UI integration with the hardware features •How to create user interface that](https://reader035.vdocuments.mx/reader035/viewer/2022081615/5fd40c3bcefb9163172318b0/html5/thumbnails/27.jpg)
References[0] http://www.imgtec.com/downloads.asp
[1] http://www.clutter-project.org/docs/clutter/0.8/
[2] http://www.openismus.com/documents/clutter_tutorial/0.8/docs/tutorial/html/
[3] http://focus.ti.com.cn/cn/lit/wp/spry110/spry110.pdf