track three - timesys... ©2011 timesys corp. track three building a rich ui based dual display...
TRANSCRIPT
www.timesys.com ©2011 Timesys Corp.
Track ThreeBuilding a Rich UI Based Dual Display Video Player
with the Freescale i.MX53 using LinuxLink
Session 3
How to leverage hardware accelerated video features to play back
720p/1080p video
Audio streaming is available for this event.
Turn on your speakers to listen.
www.timesys.com ©2011 Timesys Corp.
2 Tools You Can Use
Q&A and/or Chat
• Click on Q&A panel (?) or Chat panel icon in the bottom, right corner
• Type in your question in the space provided
• Click on ―Submit‖
www.timesys.com ©2011 Timesys Corp.
3 Tools You Can Use
Polling
• The poll will appear on your screen
• Select your answer for each question
• Click on ―Submit‖
www.timesys.com ©2011 Timesys Corp.
4 Session Information
You can download the slides for today’s session at
http://www.timesys.com/embedded-linux/training/timesys-
university/freescale_imx53
You can view a recording of today’s session at
http://www.timesys.com/embedded-linux/training/timesys-
university/freescale_imx53
Today’s speaker:
Maciej Halasz
Director, Product Management
Timesys
www.timesys.com ©2011 Timesys Corp.
5 Building a Rich UI Based Dual Display
Video Player Device
Session 1 – recording available
How to capture your initial Linux based product requirements and quickly build a custom BSP/SDK to jumpstart your development
Session 2 – recording available
How to build a media player control panel using Qt Embedded for Linux
Session 3 – Today
How to leverage hardware accelerated video features to play back 720p/1080p video
Session 4 – September 7 @ 1pm EDT
How to optimize, test and integrate the solution for fast boot and quick deployment
www.timesys.com ©2011 Timesys Corp.
6 Today’s Agenda
Recap of what we have done so far
Freescale i.MX53 VPU overview
GStreamer framework primer
Add support for dual display
Modify Video Player application
• Add code to play back video using gstreamer API
• Add scripts to autolaunch the application at boot time
Configure system for dual display on single LCD
• Discuss other options
Deploy on the target board
www.timesys.com ©2011 Timesys Corp.
7
Session 1 Recap
www.timesys.com ©2011 Timesys Corp.
8 What We Have Accomplished So Far
Learned about the Freescale i.MX53 LinuxLink and
the i.MX53QSB board – needed for all exercises
Reflected product requirements in Linux BSP
and SDK
Built a custom BSP with LinuxLink Web Edition
• Experiment on day one with a pre-built starting point
Setup a development environment (more today)
Setup a Qt based IDE development environment
• Developed a UI with Qt widgets
• Tested locally
Deployed the system on the target via NFS for
future development
• Transferred images
• Configured bootloader
www.timesys.com ©2011 Timesys Corp.
9 Project Requirements (Freescale i.MX53 QSB)
• Graphics
• Touchscreen
• Applications
− Screen calibration
− Control Panel
• VGA/Component
• Video Out
• 720p/1080p
• Serial port
communication
• SD/MMC Card
− Filesystem
• Ethernet
− Secure Connection
− Transfer (FTP/SCP)
− Console (Telnet/SSH)
• USB
• Storage (USB
stick)
• Extensions
• MicroSD
− Boot from
− Additional storage
• Audio (optional)
− Alsa Mixer
− Sound playback
www.timesys.com ©2011 Timesys Corp.
10 Dual Display Video Player (Blueprint)
U-Boot bootloader
Middleware
shell
LCD
calibration
i.MX53
QSB
USBGPIO VPUTouch
ScreenLCD
Linux kernel
Media Playback Application
AudioUser Interface, Buttons, etc Video Plaback
SDIO
Ethernet
NAND
Audio
Network
accessCPU
usage
Qt Embedded
Driver Driver
Framebuffer 2
Codecs
GStreamer
networking
openssh
DriverDriver DriverDriver
alsa
alsa-utils
Driver
setup
scripts
Driver Driver
Serial
Framebuffer 1
www.timesys.com ©2011 Timesys Corp.
11 Last Session’s Homework – Share Your
Experience
1. Design your own GUI with Qt Embedded
2. Adapt/design your own UI with desired set of
widgets
3. Create multiple windows. Connect buttons to
handling routines via signals
• How was your design with Qt experience?
• Did you try working with custom, third-party widgets?
• Any questions on a signal based communication between widgets?
www.timesys.com ©2011 Timesys Corp.
12 Giveaway
If you attend at least 3 out of 4 sessions in this
Timesys University track, we will automatically
enter you into a drawing for a chance to win a
Freescale i.MX53 Quick Start Development Board
www.timesys.com ©2011 Timesys Corp.
13
GStreamer Primer
www.timesys.com ©2011 Timesys Corp.
14 GStreamer Ingredients
Video
framework for
“stringing
together”
elements
• Video sources, syncs, multiplexing, codecs, etc.
Divided into
packages
• Baseline
• Plugins
• Third-party
Third-party
plugins can
take advantage
of hardware
specific
acceleration
www.timesys.com ©2011 Timesys Corp.
15 GStreamer: Linux Side
Delivered as a number of packages
• GStreamer
• OSS Plugins
– Base: Set of well-maintained plugins (coders/decoders for various formats, includes APIs for additional extensions)
– Good: All tested and available codecs/formats (LGPL)
– Bad: Not fully tested/verified codecs (GPLv2)
– Ugly: Good quality, tested codecs (Challenge = ability to distribute / patent license)
– FFMpeg
• Freescale Plugins
Auto-discovery of best plugins for the job
Code/Decode Audio/Video
Plugins are version sensitive
Can provide decoding features on non-accelerated
hardware
www.timesys.com ©2011 Timesys Corp.
16
i.MX53 VPU
www.timesys.com ©2011 Timesys Corp.
17 i.MX53 Hardware Block Diagram
www.timesys.com ©2011 Timesys Corp.
18 i.MX53 Hardware Block Diagram
www.timesys.com ©2011 Timesys Corp.
19 VPU + GStreamer = low CPU usage
Middleware
i.MX53 QSB IPU
GPU
VPU
Linux kernel
Applications
Qt Phonongst-launch TotemCustom
VPU IPU V4L inV4L out
imx-
lib
GStreamer core
GStreamer plugins
Freescale plugins Open Source plugins
VPU enc VPU dec
V4L sinc
AAC dec
V4L src
WMV dec
Hardware accl Software
www.timesys.com ©2011 Timesys Corp.
20 Multimedia codecs
Includes Freescale plugins
for codecs, sinks etc. used
in application to build pipes
Glue between codecs that
talk to drivers
Core functionality – modified
to accelerate codecs –
memory copying, etc.
Wrappers for various hardware
and software components, this
is where software leverages
VPU
Headers with appropriate
definitions needed by VPU
Package dependency
www.timesys.com ©2011 Timesys Corp.
21
Software Modifications
www.timesys.com ©2011 Timesys Corp.
22 Requirements Helper
Feature What do I need Where to select
Linux Kernel Latest, 2.6.35 kernel Linux kernel
Cross Toolchain GCC, Binutils, GDB Toolchains
C Library glibc Toolchains
Touch Screen tslib Packages
Qt qt-embedded-linux Packages
Dual Display Demo timesys-theatre-1080p Packages
GStreamer gstreamer
gst-plugins (base, good)
gst-fsl-plugin
Packages
Media Codecs fsl-mm-codelib
firmware-imx
Packages
Sound alsa-utils Packages
System Initialization busybox Packages
www.timesys.com ©2011 Timesys Corp.
23 Linux Kernel Adjustments
Add functionality to support dual display
Use the patch command
Leverage Desktop Factory to achieve repeatable
build
Adjust kernel command line:
console=ttymxc0,115200 video=mxcdi0fb:RGB24,SEIKO-WVGA root=b302 rw
Example 2
VGA Output of SXGA resolution (primary display), and SEIKO WVGA LCD Panel
video=mxcdi0fb:RGB24,SEIKO-WVGA video=mxcdi1fb:RGB24,VGA-SXGA vga di1_primary
www.timesys.com ©2011 Timesys Corp.
24
Video Player Control Panel –
GUI Modifications
www.timesys.com ©2011 Timesys Corp.
25
What We Have Accomplished
www.timesys.com ©2011 Timesys Corp.
26 What We Have Accomplished
Learned about hardware accelerated i.MX53
components
Discussed the GStreamer framework
Modified code
• Patched Linux kernel — added support for dual display
• Added code to our application
Cross-compiled and deployed on i.MX53QSB target
www.timesys.com ©2011 Timesys Corp.
27 Next Session
September 7 @ 1pm EDT
How to optimize, test and integrate the solution for
quick deployment
• How to achieve quick boot time
• U-Boot modifications
• Linux kernel optimizations
• Filesystem design changes
• SD Card deployment
• More fun to come…
www.timesys.com ©2011 Timesys Corp.
28 Homework
1. Work with the GStreamer framework
2. Setup playback pipelines from within your
application
3. Take a look at how Desktop Factory helps with
Linux kernel modifications
Let us know if you run into any challenges. We have setup a dedicated
communication channel to share questions, comments and replies.
Please subscribe at https://lists.timesys.com/listinfo/timesys-university
While your questions will be answered by Timesys expert engineers,
we encourage you to answer questions asked by others. Sharing
experiences is always the best way to learn.
www.timesys.com ©2011 Timesys Corp.
29 Glossary
LinuxLink (Web Edition) – Web-based version of LinuxLink
LinuxLink (Desktop Edition) – Local version with full customization and third-party tools integration
Workorder – Stores definition of your software – filenames, versions
Bootloader – Runs first, initializes necessary hardware, loads Linux
Linux kernel – Operating system that manages hardware access and other features for higher level software
Device Driver – Code that’s part of a Linux kernel, defines how software accesses specific hardware
File System – All files (libraries/utilities/scripts/etc.) combined on a single storage, e.g. NAND flash
Middleware – Complete frameworks including APIs, utilities that provide specific functionality, e.g. Qt
API (library) – Used by applications, provide functionality, abstract hardware access
Toolchain (cross) – The most important part of the development environment. Used to compile source code into binaries.
www.timesys.com ©2011 Timesys Corp.
30 About Timesys
Carnegie Mellon University spin-off in 1996
First real-time embedded Linux distribution
First to register carrier grade Linux (CGL)
First to market with an open source, commercial-
grade embedded Linux development framework
(LinuxLink)
First to develop and deliver an award-winning,
automated, intelligent, embedded Linux build
system (LinuxLink 3.0)
20012001
RealReal--timetime
LinuxLinux
LeadershipLeadership
ManualManual
CustomizedCustomized
BuildsBuilds
AutomatedAutomated
Build/TestBuild/Test
HostedHosted
Build Build
SystemSystem
Customized Customized
LinuxLinux
VersionsVersions
Integrate RealIntegrate Real--timetime
with OS Communitywith OS Community
First embeddedFirst embedded
EclipseEclipse--based based
ToolsTools
Semis adaptSemis adapt
Build systemBuild system
IntelligentIntelligent
Build systemBuild system100% Open Source100% Open Source
20022002 20032003 20042004 20052005 20072007 20092009
LinuxLinkLinuxLink 3.03.0
First to market First to market
open source, open source,
commercialcommercial--grade grade
Linux product Linux product ——
LinuxLinkLinuxLink 1.01.0 LinuxLinkLinuxLink 2.02.0
First to deliver an First to deliver an
automated, intelligent automated, intelligent
build systembuild system
First to register First to register
carriercarrier--grade Linux grade Linux
distributiondistribution
2011201120082008 20102010
EneaEnea Linux Linux
Platform BuilderPlatform Builder
PartnershipPartnership
Support for Support for
MulticoreMulticore
LinuxLinkLinuxLink
Factory Factory
adds 100adds 100thth
supported supported
boardboard
www.timesys.com ©2011 Timesys Corp.
31
You can download the slides for today’s session at
http://www.timesys.com/embedded-linux/training/timesys-university/freescale_imx53
You can view a recording of today’s session at
http://www.timesys.com/embedded-linux/training/timesys-university/freescale_imx53
More Info
Stay Online for Q&A!