© copyright khronos group, 2003 - page 1 bringing 3d gaming to cell phones being developed by the

© Copyright Khronos Group, 2003 -

Bringing 3D Gaming Bringing 3D Gaming to Cell Phonesto Cell Phones

Being developed by theBeing developed by the


TopicsTopics

• Khronos and the OpenGL ES initiative- Neil Trevett, Secretary of the Khronos Group

• OpenGL ES API specification overview- David Blythe, OpenGL ES Specification Editor

• Demonstrations and industry support- Nokia, Hi, Motorola, PowerVR

• Call to action- Q&A

Neil TrevettNeil TrevettKhronos SecretaryKhronos [email protected]@3dlabs.com


Large Opportunity for Graphics SoftwareLarge Opportunity for Graphics Software3D capability is about to be everywhere3D capability is about to be everywhere

80’s3D Hardware

Acceleration in workstations

100Ks / annum

00’s3D Hardware

Acceleration in Handhelds

100Ms / annum

Beach head application is 3D games on cell phones which will drive

3D acceleration hardware into handsets in 2004

90’s3D Hardware

Acceleration in everyday PCs

10Ms / annum


Embedded Market has API ProblemEmbedded Market has API ProblemNo suitable low-level graphics APIs exist - yetNo suitable low-level graphics APIs exist - yet• These APIs are the foundation for all advanced graphics applications• It’s like having Desktop Windows without Direct3D or OpenGL

Embedded Hardware

Low-level Graphics APIs

High Level Graphics Libraries

Applications

No suitable APIs exist at this level

Games EnginesJSR-184

Games Productivity Apps

OpenGL is widely available across platforms – but is way

too big

?

Direct3D - not cross platform – so every platform other than Windows

CE has a real problem

?


Inverse Screen Size LawInverse Screen Size LawThe need for advanced graphicsThe need for advanced graphics• Smaller screens need more advanced graphics processing per pixel

- To make best use of restricted screen real-estate

• Advanced graphics techniques provided by state-of-the-art APIs- Anti-aliasing, multi-level compositing, rich media, 3D

Interactive3D Gaming Content

Video Processing and synchronization Perspective windows

maximize screen real-estate

Anti-aliased text to makesmall text extremely legible

Translucent Menus don’t obscure lower layers


Cross Platform Solution EssentialCross Platform Solution EssentialMany platforms and initiatives need these APIsMany platforms and initiatives need these APIs

WindML® – expanding multimedia capabilities of VxWorks®

Looking to further expand 3D and rich media delivery capabilities

Embedded Linux – need for low-footprint APIsNo standardized 2D API other than X Windows – not low footprint

Increasing use of OpenGL for 3D – but not low footprint

Java Community – Java2D, Java Media Framework, Java Advanced ImagingJava3D is a high-level API – need low-footprint 3D API for embedded applications

ARINC – standards for advanced cockpit displaysNeed standardized 2D & 3D APIs

Severe certification demands drive need for minimum API size

Web3D ConsortiumExtensible standards for 3D delivery over the internet, driving 3D in MPEG-4

BUT needs target small footprint 2D/3D API for higher-level standards

Symbian OS for mobile consumer devicesNeeds minimal API size and increasing functionality

Wireless Applications PlatformsNeed advanced graphics capability in small footprint for BREW and chipsets


Working to define full-featured 3D API.

Discussions with BroadcastCL group- to avoid divergence

Strong support from ARINC and aerospace companies …

At a meeting in April 2002, Symbian and the cell phone

industry selected OpenGL ES as the graphics API for cell

phones with accelerated 3D in 2004

Phones

Web pads

Cameras

PDAs

Hand-heldAppliances

Line-poweredAppliances

STBs

InternetTerminals

Consoles

OpenGL ES Industry MomentumOpenGL ES Industry Momentum

Safety Critical

Avionics

Automotive

New Visual Processing

APIs


Khronos – an Open Standards GroupKhronos – an Open Standards GroupOur Members are creating OpenGL-related APIs Our Members are creating OpenGL-related APIs • Working closely with the OpenGL ARB

Promoting Members

Contributing Members

ARB

Feedback and Ratification


OpenGL ESOpenGL ESA proposed embedded graphics APIA proposed embedded graphics API• Create well-defined subset profiles of OpenGL

- A series of increasingly compact profiles - Selecting functionality needed for key target markets

ARBFeedback and Ratification

Eliminate Redundancy

Eliminate Pure Workstation Functionality

EmbeddedFocus

WorkstationFocus


OpenGL and OpenGL ESOpenGL and OpenGL ESA close synergyA close synergy• OpenGL is already the most widely available cross-platform graphics API

- Workstations to embedded appliances, PCs to Macs, Windows to Linux

• Strong future roadmap with OpenGL 2.0- Enabling access to the new generation of programmable hardware

• Through Khronos – OpenGL ES be royalty free to the embedded industry- Agreed by SGI – MAJOR OPPORTUNITY

• Just two profiles - minimizing fragmentation- Common Profile

- Minimum footprint full function 3D with texture-mapping- Good gaming platform- Implementable on cell phones

- Safety Critical Profile- Absolute minimum 3D to ease safety certifications- Used in avionics and automotive displays

Common

Safety Critical

The focus for the rest of today’s presentation


API OverviewAPI OverviewDavid BlytheDavid Blythe

OpenGL ES Specification EditorOpenGL ES Specification [email protected]@bluevoid.com


OverviewOverview

• Where are we in OpenGL ES development?• Handheld problem space• Design goals• What’s In – What’s Out• Summary


Where Are We?Where Are We?

• Handouts are a snapshot from January• Presentation reflects specification as of March 4th• Specification undergoing public review • Specification still needs to be ratified

JulyKhronos Participation agreement available

at Siggraph

AugustAd hoc Working Group

Meetings

4Q02First draft

specification

1Q03“OpenGL ES 1.0”

ratifiable by Khronos

SeptemberOfficial Working Group

Meetings


Profile Design GoalsProfile Design Goals

• Preserve OpenGL’s Goodness- Don’t try to fix OpenGL, it isn’t broken

• Proper subsets of OpenGL 1.3- Match market space requirements

• Eliminate un-needed functionality- Redundancy- Expensive- Unused

• Leverage Workstation/PC experience- Extension mechanism, feature set

• Allow extensions to enable innovation- But should be the exception rather than the rule

• Defining conformance tests as well as the API- Minimum quality defined by conformance tests


Handheld Problem SpaceHandheld Problem Space

• Low power- Battery powered devices

• Low memory- 50KB code budget

• Low resolution- 120x120 display

• Rapidly evolving platforms- Cell phone, PDA, Set top, Wrist watch, ????


Low Power DevicesLow Power Devices

• Low power consumption• Modest processing power• 25-400MHz integer processing

- 400Mhz PDA, 50MHz Cell phone, 10MHz Wrist Watch, 100MHz Set top Box• Assume 32-bit integer• Floating-point not available• Battery power

- Maximize battery life minimize instruction/data traffic• Where is the sweet spot?


Memory FootprintMemory Footprint

• Platforms with limited storage- RAM 1-8-64MB, NAND FLASH 1-8-64MB

• Minimize library footprint- Reduce command footprint- Reduce number of ‘code paths’

• Minimize data storage- Vertex data and attributes- Texture storage


Floating-Point ArithmeticFloating-Point Arithmetic

• No HW floating-point in most of the target space- Today!

• Be both:- Integer friendly- Floating-point ready

• Key Issue• Lots of Options

- Multiple profiles- Fixed-point arithmetic- Floating-point emulation

• Lots of Tradeoffs- Robustness/Stability- Performance- Memory Footprint- API Complexity- Ease of Use


Evolving Device SpaceEvolving Device Space

• Regular review of profiles- Target is yearly revision

• Profiles don’t become obsolete- Use with emerging devices

• Use OpenGL extension model- Enable HW vendors to ship extra features- Add to profile when demand is there

• Avoid creating fragmentation!


Floating- and Fixed-pointFloating- and Fixed-point

• Critical Issue• Want:

- Efficient vertex data representation- Dynamic range for transformations- Accommodate platforms with and without floating-point

• Don’t want:- Redundant or seldom used code paths- API bloat- Confused application writers


What’s In/What’s OutWhat’s In/What’s Out

• A Trip Down the OpenGL Pipeline• Vertex & Geometry Processing• Rasterization• Texture Mapping• Fragment Processing• Buffer Operations• Miscellaneous Operations• Profile Structure• Platform Embedding• Utility Libraries


Vertex and Geometry ProcessingVertex and Geometry Processing

In Out

Vertex Arrays Begin/End, Edge Flags

All Primitives TexGen

Matrix Stack User Clip Planes

Viewport, DepthRange

Vertex Lighting Back MaterialsLocal Viewer, Secondary Color

ShadeModel


RasterizationRasterization

In Out

Multisampling (optional)

Points and AA Points

Lines & AA Lines Line Stipple

Polygons Polygon Stipple, Polygon Smooth


RasterizationRasterization

In Out

Face Culling Polygon Mode

PolygonOffset – fill mode PolygonOffset – line, point mode

Bitmap

DrawPixels


Texture MappingTexture Mapping

In Out

2D Textures 1D, 3D, Cube Maps, ProxyWrap repeat, edge_clamp clamp, border_clamp

Texture BordersCompressed Texture GetCompressedTexImage

TexSubImage, CopyTexImage

Multitexture Combine Environment

BGRA / packed formats, L, LA All other image formatsAll Filters


Fragment ProcessingFragment Processing

In Out

Fog

Scissor Test

Alpha Test

Stencil Test (optional)

Depth Test (required)

Blending

Logic Op

Dither


Framebuffer OpsFramebuffer Ops

In Out

Clear Accum

ReadPixels CopyPixels

Draw/ReadBuffer


MiscellaneousMiscellaneous

In Out

Flush/Finish Evaluators

Hint Feedback

Selection

Display Lists

Get – static state (constants) Get – dynamic state Push/PopAttrib


Profile AssumptionsProfile Assumptions

• Lots of profiles doesn’t help- Creates support problems for developers- Tuned profiles not fully exploited

• Optional extensions are similar to multiple profiles• Floating-point a medium-term trend

- Eventually want both floating-point transforms and vertex attributes- Some platforms support it well now- Need to understand migration path now

• Want to cover today’s platforms well- Important for adoption

• Fixed-point is practical- Not ideal, but …- Successfully used in the past


Profile StructureProfile Structure

• Two profiles for the entertainment and related spaces:- Mobile- PDA- Set top box- Consoles- Specialty devices, kiosks, terminals

• Start with profile that covers as much of this space as possible- Common profile

• Address less-capable platforms with subset- Common-light profile

• Vernacular- Integer = GLbyte, GLubyte, GLshort- Fixed-point = 16.16 representation (GLfixed)- Floating-point = GLfloat (single-precision), GLdouble (double-precision)


Common ProfileCommon Profile

• Includes entire feature set just described- Integer + single-precision floating-point vertex attributes- Integer + single-precision commands as described in the spec- Double-precision floating-point commands where no pre-existing single-precision

• Adds fixed-point data type extension- Vertex attributes, command parameters- Pipeline implements floating-point dynamic range, even with fixed-point input

• Adds signed byte vertex data type extension- Vertex and texture attributes (normal, color already present)

• Adds extension to determine ‘implementation-specific’ read format- Minimize number of formats- Allow efficient read


Common-Lite ProfileCommon-Lite Profile

• Proper subset of common profile• Currently, ‘one’ difference from common profile• No floating-point (single- or double-precision) anywhere• Command parameters, vertex data are fixed-point and integer• Support contemporary and trailing-envelope platforms• Support only one form of commands• Nominally 16.16 dynamic range in all computations (including transforms)

- Subject to discussion• No overflow checking or reporting

- Subject to discussion• Compile & run-time (dynamic libraries) with common profile


Common Vs Common-LiteCommon Vs Common-Lite

Common-LiteCommon

PDA

Smart-phone

Set Top

FootprintPerformancePower Consumption

GameDevice

Kiosk

ModestHandset


ExtensionsExtensions

• Definition- Commands/features that change/extend the behavior of the OpenGL state machine- In the ‘gl’ name space

• Leverage current OpenGL extension mechanism- Extension string, function pointer query- ARB extensions- OpenGL ES extensions (OES)- Other OpenGL extensions- Vendor extensions

• Rules1. Can’t re-add commands a profile removed (no begin/end)2. Can’t create negative extensions (contractions) to remove features3. Can add existing OpenGL extensions (e.G., Texture_cube_map)4. Can add OpenGL ES working group (ESWG) exceptions to 1

- Candidates: texgen5. Can add vendor-specific exceptions

- Use to violate rule 1


Platform EmbeddingPlatform Embedding

• Interface OpenGL renderer to platform window system- Similar to GLX/AGL/WGL

• Issues:• Platform specific or not

- Is there significant value in a cross-platform API• Feature set

- Minimum platform capabilities- Video, p-buffers, render-to-texture


PackagingPackaging

• Retain ‘GL’ namespace• Single C header file

- #defines for versions, profile variants- GL_ES_PROFILE_XX- GL_ES_PROFILE_XX_1_0- XX = CM, CL, SC

• Runtime queries- GetString(GL_VERSION) OpenGL ES -XX 1.0

- XX = CM, CL, SC, …- GetString(GL_EXTENSIONS) “foo_OES bar_OES …”

• Proprosal for link/runtime library packaging- libGLESXX.{a,so,lib}- Versioning uses platform native versioning mechanism


Utility LibrariesUtility Libraries

• No GLU subset- Project, Unproject, perspective useful, not hard to implement- Quadrics – questionable value (demo programs)- Tessellator, NURBS too large

• GUI libraries- Run away!


SummarySummary

• Embedded market space is very diverse- Focusing on handheld and safety-critical markets

• Creating OpenGL profile that enable handheld devices- Small footprint- Software renderers- Integer/fixed-point

• Evolving standard- Yearly revisions

• Draft spec undergoing public review- Now is the time to comment


DemonstrationsDemonstrations


DemonstratorsDemonstratorsProto OpenGL ES ImplementationsProto OpenGL ES Implementations• Kari Pulli – Nokia• Mark Callow – Hi Corporation• Mike Olivarez – Motorola• Mitsubishi• MBX and PowerVR Technology overview – Khronos Booth


OpenGL ES Hardware SupportOpenGL ES Hardware Support• The technology is here now!

- New hardware generation for mobile platforms.• MBX and PowerVR Technology overview

- Tile Based Rendering and Deferred Rendering

• More information at: www.imgtec.com www.powervr.com www.pvrdev.com

Advanced Graphic Hardware

Embedded Software Developers

OpenGL ESAdvanced Consumer Content


OpenGL 2.0“Pure” OpenGL 2.0 subsetOpenGL 2.0“Pure” OpenGL 2.0 subset

OpenGL ES 2.0OpenGL ES 2.0

OpenGL Standards UniverseOpenGL Standards UniverseHow things COULD developHow things COULD develop

OpenGL 1.3OpenGL 1.3

Aiming for Ratification by Siggraph 2003OpenGL 2.0 includes full 1.4 functionality – applications run unmodified

OpenGL 1.4GL2 extensionsOpenGL 1.4GL2 extensions

Applications can incrementally can use shading language extensions

OpenGL ES 1.0OpenGL ES 1.0

“Pure OpenGL 2.0” subset uses programmability to form a simple, compact, efficient and flexible API

Full shader programmability

for embedded devices


Call to ActionCall to ActionYour chance to influence OpenGL ES!Your chance to influence OpenGL ES!• Join Khronos – as a Promoter or Contributor

- Have a voice in OpenGL’s development- Reasonable annual fee plus participation agreement- Talk to Elizabeth Riegel, the Khronos Group Manager here at the session,

on our booth, or via email at [email protected]

• Review the draft specification – we are looking for ISV volunteers- Simple review agreement required to participate- Email [email protected]

• Information at www.khronos.org- Or email the Secretary at [email protected]


Any Questions?Any Questions?

OpenGL is a registered trademarks of Silicon Graphics, Inc., and OpenML is a trademark of Silicon Graphics, Inc., used with permission by the Khronos Group. All other product names, trademarks,

and/or company names are used solely for identification and belong to their respective owners.

www.khronos.org

© copyright khronos group, 2003 - page 1 bringing 3d gaming to cell phones being developed by the

Documents