2005/1/261igel co.,ltd. / renesas solution corp. our experiences on directfb in embedded application...

2005/1/26 1IGEL Co.,Ltd. / Renesas Solution Corp.

Our Experiences on DirectFB inOur Experiences on DirectFB inEmbedded Application DevelopmentEmbedded Application Development

IGEL Co., Ltd / Renesas Solution Corp.


Today’s TopicsToday’s Topics

DirectFB porting experiences on embedded platforms– SH7751 + SM501– SH7770

Missing pieces in DirectFB– Functionality missing to write specific applications


Porting DirectFBPorting DirectFB


DirectFB ArchitectureDirectFB Architecture

DirectFB works on a frame buffer device(/dev/fb) and provides the mechanism to use the hardware acceleration effectively.

DirectFB consists of the followings:– Core API Module– Generic GFX Driver– GFX Drivers for Specific Hardware

To bring out the best performance on a specific graphics hardware GFX Drivers for the hardware should be written.

– Generic GFX Driver checks whether the hardware acceleration by a GFX driver is available

• If yes, it handovers to the GFX driver• If not it uses software rendering engi

ne

DirectFBApplicationDirectFB

Application

DirectFB

DirectFB Core API ModuleDirectFB Core API Module

GFX drivers( )※GFX drivers( )※

Frame BufferDriver( )※

Frame BufferDriver( )※

Display UnitDisplay Unit 2D GraphicsHardware

2D GraphicsHardware

GenericGFX Driver

GenericGFX Driver

Use

r Level

Device

D

riversH

ardwar

e

※Modules that needs to be developed


Why do we need GFX drivers?Why do we need GFX drivers?

Embedded CPU and bus are slow compare to Desktop’s CPU– 200-400MHz CPU – 120MHz 32bit Bus

Therefore, handover the rendering tasks to specialized hardware is crucial!


Effects of Hardware AccelerationsEffects of Hardware Accelerations

　 A B C D

Fill Rectangles [MPixel/sec] 14.07 217.66 63.63 53.25

Fill Rectangles (blend) [MPixel/sec] 1.64 1.66 1.2 3.26

Fill Triangles [MPixel/sec] 12.25 93.69 62.26 50.51

Fill Triangles (blend) [MPixel/sec] 1.63 1.63 1.17 3.17

Draw Rectangles [KRects/sec] 1.81 15.45 10.67 8.57

Draw Rectangles (blend) [KRects/sec] 0.52 0.56 0.43 0.84

Draw Lines [KLines/sec] 7.1 67.09 61.33 48.84

Draw Lines (blend) [KLines/sec] 2.33 2.43 1.94 3.7

Blit [MPixel/sec] 8.12 102.47 38.68 32.56

Blit with format conversion [MPixel/sec] 4.04 4.12 3.59 17.79

CPU Bus RAM 2D Kernel H/W Accel

A SH7751 240MHz SH-Bus 64MB SMI SM501 2.4.19 Off

B SH7751 240MHz SH-Bus 64MB SMI SM501 2.4.20 On

C SH7751 240MHz PCI 64MB Matrox Millennium 2.4.20 On

D Celeron 450MHz PCI 128MB Matrox Millennium 2.4.20 On

The hardware acceleration shows remarkable results.

The performance depends on hardware acceleration engine

rather than CPU.


How to write GFX Drivers?How to write GFX Drivers?

Callback routines needs to be written– GFX Graphics Driver Functions– GFX Graphics Device Functions

Good starting point is gfxdrivers/i810/*.[ch]


GFX Graphics Driver FunctionsGFX Graphics Driver Functions

From core/gfxcard.h:typedef struct { int (*Probe) (GraphicsDevice *device); void (*GetDriverInfo) (GraphicsDevice *device, GraphicsDriverInfo *driver_info);

DFBResult (*InitDriver) (GraphicsDevice *device, GraphicsDeviceFuncs *funcs, void *driver_data, void *device_data);

DFBResult (*InitDevice) (GraphicsDevice *device, GraphicsDeviceInfo *device_info, void *driver_data, void *device_data);

void (*CloseDevice) (GraphicsDevice *device, void *driver_data, void *device_data); void (*CloseDriver) (GraphicsDevice *device, void *driver_data);} GraphicsDriverFuncs;


GFX Graphics Device FunctionsGFX Graphics Device Functions

From core/gfxcard.h:typedef struct _GraphicsDeviceFuncs { /* * function that is called after variable screeninfo is changed * (used for buggy fbdev drivers, that reinitialize something when * calling FBIO_PUT_VSCREENINFO) */ void (*AfterSetVar)( void *driver_data, void *device_data );

/* * Called after driver->InitDevice() and during dfb_gfxcard_unlock( true ). * The driver should do the one time initialization of the engine, * e.g. writing some registers that are supposed to have a fixed value. * * This happens after mode switching or after returning from * OpenGL state (e.g. DRI driver). */ void (*EngineReset)( void *driver_data, void *device_data );

/* * Makes sure that graphics hardware has finished all operations. * * This method is called before the CPU accesses a surface' buffer * that had been written to by the hardware after this method has been * called the last time. * * It's also called before entering the OpenGL state (e.g. DRI driver). */ void (*EngineSync)( void *driver_data, void *device_data );

/* * after the video memory has been written to by the CPU (e.g. modification * of a texture) make sure the accelerator won't use cached texture data */ void (*FlushTextureCache)( void *driver_data, void *device_data );

/* * Check if the function 'accel' can be accelerated with the 'state'. * If that's true, the function sets the 'accel' bit in 'state->accel'. * Otherwise the function just returns, no need to clear the bit. */ void (*CheckState)( void *driver_data, void *device_data, CardState *state, DFBAccelerationMask accel );

/* * Program card for execution of the function 'accel' with the 'state'. * 'state->modified' contains information about changed entries. * This function has to set at least 'accel' in 'state->set'. * The driver should remember 'state->modified' and clear it. * The driver may modify 'funcs' depending on 'state' settings. */ void (*SetState) ( void *driver_data, void *device_data, struct _GraphicsDeviceFuncs *funcs, CardState *state, DFBAccelerationMask accel );


GFX Graphics Device InterfaceGFX Graphics Device Interface(contd.)(contd.)

/* * drawing functions */ bool (*FillRectangle) ( void *driver_data, void *device_data, DFBRectangle *rect );

bool (*DrawRectangle) ( void *driver_data, void *device_data, DFBRectangle *rect );

bool (*DrawLine) ( void *driver_data, void *device_data, DFBRegion *line );

bool (*FillTriangle) ( void *driver_data, void *device_data, DFBTriangle *tri );

/* * blitting functions */ bool (*Blit) ( void *driver_data, void *device_data, DFBRectangle *rect, int dx, int dy );

bool (*StretchBlit) ( void *driver_data, void *device_data, DFBRectangle *srect, DFBRectangle *drect );

/* * emit any buffered commands, i.e. trigger processing */ void (*EmitCommands) ( void *driver_data, void *device_data );} GraphicsDeviceFuncs;


Porting DirectFB on Porting DirectFB on SH7751 + SM501SH7751 + SM501

Our first development GFX Driver for SM501 just

set registers to issue rendering commands– Issuing command is done

on the fly

– Callback functions immediately set registers to render

– Rendering comes on screen instantly

GFX Driver for SM501GFX Driver for SM501

SM501SM501

SM501 RegistersSM501 Registers


Porting DirectFB on SH7770Porting DirectFB on SH7770

Our second development GFX Driver for SH7770 creates list

of rendering commands, so called Display List

– The list is double buffered The driver fills the list until they’re

full, and then pass them to the 2D engine

– While the driver is filling one list, the 2D engine reads commands from another list

– Once the 2D engine is done with the list, it sends an interrupt, and get the next list

– Rendering doesn’t come on screen instantly

Sync mechanism is required to sync with software rendering done by generic GFX driver

GFX Driver for SH7770GFX Driver for SH7770

SH7770 2D EngineSH7770 2D Engine

DisplayList 1

DisplayList 1

DisplayList 2

DisplayList 2

RegistersRegisters


Missing Pieces in DirectFBMissing Pieces in DirectFB


What’s Missing?What’s Missing?

Access multiple layered frame buffer from a process– Recent graphics hardware has multiple frame buffers.

Using ‘scroll’ function on hardware– New feature not covered by DirectFB API

Synchronous rendering and display with VSYNC (QoS, delay handling)– Real-time motion graphics (e.g. game), car navigation, etc.

Synchronize 2D Engine and 3D Engine– Render with 2D and 3D Engine in a single layer– Synchronous display even 2D and 3D are on different layers


Access Multiple Layered Frame Access Multiple Layered Frame Buffer from a ProcessBuffer from a Process

Recent graphics hardware has multiple layered frame buffer.

To coordinate layers efficiently, an application process wants to issue rendering commands and switch on / off display of each layer.

Layer #1 (/dev/fb0)

Layer #1 (/dev/fb0)

Layer #2(/dev/fb1)

Layer #2(/dev/fb1)

Layer #3(/dev/fb2)

Layer #3(/dev/fb2)

Application process

DisplayUnit

DisplayUnit


Using ‘Scroll’ Function on Using ‘Scroll’ Function on Hardware Hardware

Use Case: car navigation system, web browser The ‘Scroll’ function reduces re-rendering cost.

Display start position

Simple scroll Wraparound scroll

A’

B’

C’

A

B

C


Synchronous Rendering and Synchronous Rendering and Display with VSYNC (QoS)Display with VSYNC (QoS)

In real-time motion graphics applications, the screen must be updated in sync with the VSYNC signal.

Under the standard (fairly) task scheduling in Linux, rendering might miss display timing (VSYNC) because of signal interrupts or other heavy tasks.

rendering rendering rendering

VSYNC

Ideal

y=16.6ms(for NTSC)

rendering rendering

Other tasks

Fact

A heavy task was joined. Interrupt signal handling

miss miss miss miss

Rendering task keeps needed resources in every VSYNC time slots.


Synchronous Rendering and Synchronous Rendering and Display with VSYNC (Delay Display with VSYNC (Delay

Handling)Handling) Real-time motion graphics applications could be

optimized for screen rendering, especially for delay handling.

Application needs VSYNC signal timing to notice the delay.


Delay Handling by ApplicationDelay Handling by Application

Example 1: Skipping the next frame rendering When an application noticed that display has been

delayed, it could skip the next frame and start rendering the frame after the next.

VSYNC

Rendering

Display

y=16.6ms(for NTSC)

Frame #1 Frame #2

Frame #1 Frame #1 Frame #2 Frame #4

Frame #4

missIdeal Frame #1 Frame #2 Frame #3 Frame #4

Catch up the delay

Skip the frame #3


Delay Handling by Application Delay Handling by Application (contd.)(contd.)

Example 2: Updating screen even if the rendering is not finished

Additionally, application could give priority to rendering operations in case of incomplete frame displaying.

VSYNC

Rendering

Display

y=16.6ms(for NTSC)

Frame #1 Frame #2

Frame #1Uncompleted

frame #2Frame #3 Frame #4

Frame #4

Ideal Frame #1 Frame #2 Frame #3 Frame #4

Enforce display #2

Terminate rendering #2

Frame #3


Synchronize 2D Engine and 3D Synchronize 2D Engine and 3D EngineEngine

Many 3D graphics applications combine 3D graphics and 2D graphics.

These 2D graphics must be synchronized with 3D graphics.

Some 3D acceleration hardware (nVIDIA, ATI, for example) are separated from 2D hardware.

Synchronization mechanism between 2D and 3D graphics (hardware) is needed.


Synchronization Problem in 2D Synchronization Problem in 2D Engine and 3D EngineEngine and 3D Engine

Situation 1: Rendering 2D graphics and 3D graphics into a single layer simultaneously

2D engine and 3D engine tries to draw into a frame without any synchronization. They should be serialized.

Issued rendering commands might be performed asynchronously.

LayerLayer

2D Engine2D Engine

3D Engine3D Engine Display

Application


Synchronization Problem in 2D Synchronization Problem in 2D Engine and 3D EngineEngine and 3D Engine

Situation 2: Displaying a 2D graphics layer and a 3D graphics layer synchronously

2D/3D engines can draw into each independent layer asynchronously if the multiple layered frame buffer is available.

Synchronous display function is still needed.

2D Layer2D Layer2D Engine2D Engine

3D Engine3D Engine 3D Layer3D Layer

Synchronous display

Application


ConclusionConclusion

We need to consider wherther DirectFB API should / could cover all application requirements or not.

We would like to submit proposals and distribute implementations against the issues.

2005/1/261igel co.,ltd. / renesas solution corp. our experiences on directfb in embedded application...

Documents

device void

renesas solution

directfb architecture

specific hardware

data void

porting directfb slide

directfb functionality

hardware acceleration