building markets for advanced devices through open standards · 2014-04-08 · khronos - forum for...

60
© Copyright Khronos Group, 2010 - Page 1 Building Markets for Advanced Devices through Open Standards Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group

Upload: others

Post on 13-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 1

Building Markets for Advanced Devices through Open Standards

Neil TrevettVice President Mobile Content, NVIDIA

President, The Khronos Group

Page 2: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 2

Thank you for coming!

• Overview of Khronos

- The vision, mission and organization

• Technology overview

- A brief review of Khronos standards

- Desktop and mobile

Page 3: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 3

Morning Agenda

• Welcome

- Jeff Li, Vice President & General Manager, Vivante

• Khronos Overview

- Neil Trevett, President Khronos, VP Mobile Content, NVIDIA

• OpenGL

- Ed Plowman, Technical Marketing Manager, Media Processing Division, ARM

• Break - 11:15 AM

• OpenCL

- Neil Trevett Chair OpenCL Working Group

• COLLADA and 3D UI with Kanzi Engine

- Cesar Qui, Rightware

Page 4: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 4

Afternoon Agenda

• Walk to Lunch - 12:30PM – restart at 2PM

• OpenVG overview : implementations and applications

- Hwanyong Lee, Chief Technical Officer HUONE

• OpenSL ES and OpenMAX AL | IL | DL

- Erik Noreke, OpenSL ES Work Group Chairperson

• Achieving Graphics Performance - Android Integration with OpenGL ES

- Cemil Azizoglu, Architect, Android Solution

Page 5: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 5

Khronos Vision

• Billions of devices needs advanced

imaging, graphics, media and compute

functionality to deliver advanced user

experiences

• Developers need consistent access to

silicon through Application Programming

Interfaces (APIs)

• Khronos enables companies to cooperate

to create open API standards that build

markets for silicon, software and devices

“The best way to predict the future is to invent it.”Alan Kay

Page 6: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 6

Annual Market Size

Device Power Consumption

1W10W100W1,000W

1M

10M

100M

1B

1000x Lower Power Changes the World

The next 10 billion Computers will be deeply integrated into the

fabric of our lives

Page 7: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 7

Modern Mobile Silicon Functionality

• Rich Experiential Processing

- Parallel Computing

- GPU Computing

- 3D Graphics

- 2D Graphics

- Camera control

- Image processing

- Video encode

- Video decode

- Audio encode

- Audio decode

Page 8: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 8

The Critical Role of APIs

Khronos APIs are low-level – just above raw silicon - to create the

“foundation” functionality needed on every platform

APIs enable software developers to turn

silicon functionality intorich end user experiences

Page 9: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 9

Khronos - Forum for Industry Cooperation

Open International MembershipAny company is welcome to join to suggest working groups and technology directions

Open StandardsPublicly available onweb-site to be used

by anyone

Royalty FreeNo per-device royalty

to ship APIimplementations

Industry CooperationHundreds of man years invested by industry experts to create state-of-the APIs

Opportunity to compete to create chips, software and systems that deliver amazing end user experiences in international markets

International Market Growth= ¥ $ € !

Page 11: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 11

Khronos Family of Standards

Embedded 3DCross platform

desktop 3D

3D Digital Asset Exchange format

Advanced Audio Vector 2D

Context, Sync and Surface Management

Mobile OSAbstraction

Au

tho

rin

g a

nd

a

cce

ssib

ilit

yA

pp

lica

tio

n

Acce

lera

tio

n

Parallel Computing

Plugin-free 3D Web Content

Steaming MediaSafety Critical 3D

Syste

m

Inte

gra

tio

n

Window SystemAcceleration

A coordinated ecosystem of compute, graphics and media

standards and APIs

Video, Audio andImage Acceleration

Codec Creation

Page 12: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 12

Khronos IP Framework

IP “Safety Zone”Khronos members agree to not sue

other members over Khronos standards

Industry ProtectionKhronos membership and hence mutual

IP non-assert from other members is open to any international company

Member ProtectionNo IP disclosure is necessaryVery restricted license grant:

- No implementation IP- Only essential IP (i.e. no alternative)- Only for conformant implementations

Health WarningKhronos makes no warranties

about non-members IP status - though Khronos working groups attempt to

not infringe third party IP

Implementation IPIs not licensed – enabling market

competition over cost, performance, power etc.

Page 13: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 13

Cooperation versus Competition

100%COMPETITION

100% COOPERATION

Competing technologies confuse and fragment market: e.g. video codecs ,

VHS vs. Betamax, BluRay vs. HD DVD

Lack of competition stifles innovation and slows creation of new user-valued functionality

Productive cooperation to enable market competition - maximizing

end-user value and market growth

$ CommercialOpportunity

Every industry needs foundational standards to enable sufficient value to be delivered

economically by competitive vendors to enough end users to build a market

Page 14: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 14

How Does Khronos Make Money?

• It doesn’t! Khronos is a non-profit organization

- Funded by member dues and conformance fees

- Used for conformance tests, samples implementations, documentation etc.

$ $Taxing an API with royalties

would generate relatively small amounts of revenue

An open, royalty free API standard creates much larger market opportunities for

products using that API

Page 15: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 15

Khronos Process

Working GroupsOne working per API

Promoter Members- Participation and vote in working groups- Board seat for strategy, budget and spec ratification

Contributor Members- Participation and vote in working groups

Academic Members- Participation in working groups

Any member can propose a new working group and can contribute technology they want to see adopted by the industry

Ratified Specifications

Conformance Tests

SDKs, Sample Implementations, Documentation and Man Pages

Adopters Build conformant implementation and products

DevelopersDevelop applications using the APIs

Khronos Adopters Agreement enables API trademark to be used on conformant products

Page 16: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 16

Khronos Conformance Process

• A standard without strong tests for conformance is not a standard at all

- Need multiple vendor consistency to create reliable development platform

• Khronos Administers an Adopters Program for each API

- Khronos APIs have exhaustive set of conformance tests

- Full source access to tests for small fee

- Peer review of uploaded results by working group

- Conformant products listed on web-site

Adopters Agreement and

Fee to enter Adopters program

Port Test Source to

product and generate test

results

Upload test results to Khronos private

web-site. Peer review by

members/Adopters

Successful Review enables products to use

Khronos trademarks

Company implementing Khronos spec

wishes to use the trademark

E.g. “We have implemented OpenGL ES”

Page 17: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 17

Ecosystem Growth

Tools and SDKs

Conformance Tests

Open Authoring Standards

High-quality platforms

Open Acceleration API Standards

Market demand for rich applications

Great Applications

IndustryCooperation

Building a upward spiral of commercial opportunity

Page 18: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 18

OpenGL ES Penetration

0

100

200

300

400

500

600

700

800

900

2006 2007 2008 2009 2010 2011 2012 2013

3D phones

Phones with OGL ES

Source: Jon Peddie Research, March 2009

OpenGL ES has become the most widely deployed 3D APISmartphones, games consoles, GPS, media players, automotive, tablets…

Page 19: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 19

Driving API Adoption into Device Platforms

Plug-in

Native support

SurfaceFlinger

Page 20: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 20

Desktop and Authoring Standards Overview

Page 21: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 21

OpenGL for Each Hardware Generation

Geometry Shaders

1.X

Fixed Function

2.X

Vertex and Fragment Shaders

3.X

Tessellation and Compute

4.X

Page 22: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 22

Accelerating OpenGL Innovation

• OpenGL 4.1 and GLSL 4.10 specifications available!

• Support for the latest generation of programmable hardware

• Superset of DX11 functionality

• OpenGL increased pace of innovation

- Six new spec versions in two years

- Actual implementations following specifications closely

DirectX 10.1

2004 2006 2008

OpenGL 2.0 OpenGL 2.1 OpenGL 3.0

OpenGL 3.1

2009

DirectX 9.0c DirectX 10.0 DirectX 11

20102005 2007

OpenGL 3.2

OpenGL 3.3+

OpenGL 4.0

2011

OpenGL 4.1

SIGGRAPH2010

Page 23: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 23

OpenGL 4 = DX11 Superset

• Interop with a complete compute solution

- OpenGL is for graphics – OpenCL is for compute

• Get_program_binary

- Ability to query a binary, and save it for reuse later

• Flow of content between desktop and mobile

- All of OpenGL ES 2.0 capabilities available on desktop

- EGL on Desktop in the works

- WebGL bridging desktop and mobile

• Cross platform

- Mac, Windows, Linux, Android, Solaris, FreeBSD

- Result of being an open standard

Buffer andEvent

Interop

Page 24: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 24

Processor Parallelism

CPUsMultiple cores driving performance increases

GPUsIncreasingly general purpose data-parallel

computing

Graphics APIs and Shading

Languages

Multi-processor

programming – e.g. OpenMP

EmergingIntersection

HeterogeneousComputing

OpenCL is a programming framework for heterogeneous compute resources

Page 25: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 25

OpenCL Timeline

• Six months from proposal to released OpenCL 1.0 specification

- Due to a strong initial proposal and a shared commercial incentive

• Multiple conformant implementations shipping

- For CPUs and GPUs on multiple OS

• 18 month cadence between OpenCL 1.0 and OpenCL 1.1

- Backwards compatibility protect software investment

Apple proposes OpenCL working group and contributes draft specification to Khronos

Khronos publicly releases OpenCL 1.0 as royalty-free specification

Khronos releases OpenCL 1.0 conformance tests to ensure high-quality implementations

Jun08

Dec08

May09

2H09

Multiple conformant implementations ship across diverse OS and platforms

Jun10

OpenCL 1.1 Specification released and first implementations ship

Page 26: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 26

Anatomy of OpenCL

• Language Specification

- C-based cross-platform programming interface

- Subset of ISO C99 with language extensions - familiar to developers

- Well-defined numerical accuracy - IEEE 754 rounding behavior with specified

maximum error

- Online or offline compilation and build of compute kernel executables

- Includes a rich set of built-in functions

• Platform Layer API

- A hardware abstraction layer over diverse computational resources

- Query, select and initialize compute devices

- Create compute contexts and work-queues

• Runtime API

- Execute compute kernels

- Manage scheduling, compute, and memory resources

Page 27: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 27

COLLADA = XML Interchange

• COLLADA is a XML database schema for 3D assets

- Can hold everything to do with a scene: geometry with full skinning, advanced

material and visual effects, animation, physical properties and collisions

• COLLADA can be used to transport 3D assets between applications

- Enables binding of diverse DCC and 3D processing tools into a production pipeline

• COLLADA is an open, archive-grade format that retains meta information

- Retains all information - even multiple versions of the same asset

• Supported import format for KML

- Lots of joint work- Integration of CityGML and Collada for

High-Quality Geographic Data Visualization- http://www.springerlink.com/content/242771575gj3x237/

Page 28: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 28

COLLADA

• COLLADA is an interchange format

- NOT a delivery format or a scene graph

• Targets use the same asset database

through a conditioning pipeline

- 1. Strips out authoring-only information

- 2. Data optimization to suit the target

platform: geometry optimization, normal

map computation, data streaming

optimization etc..

- 3. Compresses and formats binary data for

the target platform run-time engine

Multiple tools create assets and scenes in a COLLADA

Database

Conditioning Pipeline

Conditioning Pipeline

Page 29: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 29

COLLADA Conformance Framework

Page 30: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 30

Khronos Mobile API Ecosystem

Page 31: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 31

Mobile OS Fragmentation

• Every handset is unique from the programmers perspective

- Differences in OS functions, Java implementations and media functionality

Symbian^3, PocketPC / Windows Mobile / WinCE

Linux variants – Android, Limo, WebOSJava MIDP-1, MIDP-2, JSR fragmentation

RTOS – Nucleus, SynergyBrew, WIPI, iPhone OS

Severe platform fragmentationISVs traditionally need to port to and support 100's (even 1000's) of

source variants of each application/title

We need cross platform standard

APIs to de-fragment access to graphics, media and

compute acceleration if the

mobile market is to be fully realized

Page 32: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 32

Other Needs for Mobile Acceleration APIs

0.36 0.14

1.85

17.8

0

2

4

6

8

10

12

14

16

18

ARM9 32-Bit RISC at

400MHz

DSP at

175MHz

APA 512

MiMagic 6

POWER EFFICIENCYLess Power!

Hardware accelerators exploit media pipeline parallelism and caching for a x10 increase in power efficiency over software

Better User ExperienceSmall screens need advanced graphics processing per pixel

Faster Performance at Higher QualityHardware delivers smoother interaction with much

better looking visuals

Software 3D Accelerated 3D

Page 33: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 33

The Khronos Mobile API Family

Mobile 3D

Advanced Audio

Vector 2D

Parallel Computing

Video, Audio and ImageSilicon Acceleration

Steaming Media

POSIX-like Mobile OS

Abstraction:User Input, files,

threads, math libraries

Context, Sync and Surface Management

Complete set of APIs for rich, portable mobile application development

Page 34: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 34

OpenGL ES Versions

• OpenGL ES 1.1 – fixed-function pipeline

- Based on OpenGL 1.5

- Vertex Arrays / Buffer Objects

- Transform & Lighting

- Multi-texturing (min 2 units)

- Fixed-point & Floating-point profiles

• OpenGL ES 2.0 – programmable pipeline

- Based on OpenGL 2.0

- Adds vertex and fragment shader programming

- Removes fixed function pipeline

- Super-compact, efficient API

- High level language (GLSL ES)

- On-line or off-line compilation

Page 35: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 35

OpenVG - Accelerated Vector Graphics

• OpenVG is the industry’s first native Bezier acceleration API

- Enables a new class of hardware acceleration – Bezier primitives – not polygons

• Primarily used to accelerate higher-level engines such as SVG

- OpenVG is an OpenGL-style, low-level API

Applications and UI

Hardware Acceleration

SVG, Vector andFont Packages etc..

OpenGL ES accurately represents

PERSPECTIVE and LIGHTING using 3D polygons

OpenVG accurately represents

SHAPE and COLOR using 2D Bezier Curves

Page 36: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 36Copyright Khronos 2009

OpenCL 1.0 Embedded Profile

• OpenCL – parallel programming of

heterogeneous processors

• OpenCL 1.0 has Embedded profile - no

need for a separate “ES” spec

• Almost identical functionality – some

reduced precision requirements

• An always-on, connected, mobile device

with multiple sensors, graphics and

imaging PLUS a supercomputer – all in

the palm of your hand will create a new

wave of application opportunities…A concept GPS phone processes

images to recognize buildings and landmarks and uses the internet to

supply relevant data

Page 37: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 37

OpenMAX IL Media Acceleration

Low-level Acceleration Media Acceleration

OpenMAX IL enables diverse high-level media frameworks and applications to portably

tap into silicon media acceleration

StageFright

Page 38: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 38

OpenMAX IL – Video, Audio and Imaging

• Enables arbitrary multimedia pipelines by plugging blocks together

- Componentized architecture abstracts multimedia functionality block interfaces

• Wide variety of building blocks for imaging, video and audio functions

- Encode, decode, apply an effect, capture, render, split, mix, etc

• Enables blocks from different sources to work together

- Blocks can be implemented in software or hardware

Portable & reusable media processing

building blocks

Page 39: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 39

OpenMAX IL - Video, Audio and Imaging

• Standardized component interfaces enable flexible media graphs

- Including tunneling between components for execution efficiency

• Wide variety of components for imaging, video and audio functions

- Encode, decode, apply an effect, capture, render, split, mix, etc

Example: MPEG-4 video synchronized with AAC audio decode

*.mp4 / *.3gpFile Reader

Clock for AV Sync

AudioDecoder

AudioRenderer

VideoScheduler

VideoRenderer

Speakers

Display

AAC Audio

MPEG4/H.264 Video

TimeData

VideoDecoder Decompressed

Video

Page 40: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 40

OpenMAX AL – Object Oriented Media

• OpenMAX AL - simplified object-oriented streaming media

- Built to enable common use cases – but also extensible

- Can be implemented over OpenMAX IL

• OpenMAX AL Media Objects enable PLAY and RECORD of media

- Media Objects input and output to Devices

- Perform some operation on an input and emit the result as output

- Can handle audio, images, video with synchronized audio

• Objects have control interfaces

- Play, Seek, Rate, Audio, Display Region, Metadata Extraction

- Record, Camera, Video Encoder, Audio Encoder, Metadata Insertion, Radio, MIDI

OpenMAX ALMedia Object

URI

Memory

Content pipe

Camera

Audio Input

URI

Memory

Content pipe

Audio Mix

Display Window

DSrc DSnk

Page 41: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 41

Embedded Audio Fragmentation

• Modern mobile devices have advanced audio capabilities

- Including high-quality music and 3D gaming

• No standard way to access audio hardware acceleration

- Even playing a simple sound on different platform requires different code

• What about ALSA, OSS, GStreamer, OpenAL?

- OpenAL is targeted for desktop PCs

- OSS is obsolete, replaced by ALSA

- ALSA is Linux specific

- GStreamer is not designed to be optimally hardware accelerated

- Are all released under variations of GNU Public License

Page 42: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 42

OpenSL ES Profiles

Basic mobile phonesRing tone and alert tone playback (basic MIDI functionality), basic audio playback and record

functionality, simple 2D audio games

Game-centric mobile devicesAdvanced MIDI functionality, sophisticated audio

capabilities such as 3D audio, audio effects, ability to handle buffers of audio, etc.

Music-centric mobile devicesHigh quality audio, ability to support

multiple music audio codecs, audio streaming support

A device can implement any combination of profiles

Page 43: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 43

OpenSL ES and OpenMAX AL

3D Audio

AudioEffects

AdvancedMIDI

AudioPlayback

AudioRecording

BasicMIDI

Videoplayback

Camera

Videorecording

Image capture& displayBuffer

queues

Radio and RDS

(Enhanced audio API) (Multimedia API)

Both working groups collaborate to define common API functionality

Page 44: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 44

EGL Becoming an Interoperability Hub

Buffers, textures and video streams flow efficiently

between any combination of client APIs

Inter-API Synchronization events enable efficient

resource sharing

EGL Interoperability Conformance Tests being

created to ensure that client APIs can cleanly communicate

Enabling the Khronos individual APIs to interoperate as a

coherent ecosystem

Page 45: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 45

Mobile Visual ComputingCompute, graphics and AV APIs

interoperate through EGL

Visual Computing Ecosystem

Desktop Visual ComputingOpenGL and OpenCL have direct

interoperability. OpenCL objects can be created from OpenGL Textures, Buffer

Objects and Renderbuffers

Page 46: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 46

Mobile Application Portability

CompositingUser Interfaces

Games and Game EnginesGames and

Game EnginesGames and Game Engines

Flash PlayersFlash PlayersFlash and SVG

Players

TV and Video StacksTV and Video

StacksTV and Video Stacks

Applications are portable toKhronos-enabled platforms

OS resource abstraction

Browsers

Silicon and platform

independent graphics, media

and compute acceleration

Page 47: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 47

Raising 2D and 3D Visual Quality

Older generation APIs Provide rudimentary graphics functionality and quality

State-of-the-art APIs enable compelling consumer displays

Advanced functionality, fast interactivity and extremely high quality

High-quality 2D graphics and test using OpenVG

High-quality 3D displays using OpenGL ES

Video and image processing with

OpenMAX IL

Page 48: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 48

WebGL

Page 49: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 49

WebGL – 3D on the Web – No Plug-in!

• Historic opportunity to bring accelerated 3D graphics to web

- WebGL defines JavaScript binding to OpenGL ES 2.0

• Leveraging HTML 5 and uses <canvas> element

- Enables a 3D context for the canvas

• Very different dynamic to VRML

- Browser vendors want 3D this time

- No plug-in

Availability of OpenGL ES 2.0 on almost every

web-capable device

JavaScriptbinding to

OpenGL ES 2.0Increasing JavaScript performance.HTML 5 Canvas Tag

Page 50: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 50

WebGL Implementations

• Khronos working group launched in

March 2009

- Mozilla, Apple, Google and Opera

working with OpenGL GPU vendors

- Provisional spec – November 2009

- Implementations being developed in

public daily builds

- WebGL 1.0 expected spring 2011

- WebGL on Windows uses Google Angle

to create ES2 over DX9

• WebGL JS middleware already

- GLGE, SpiderGL, X3DOM, EnergizeGL,

SceneJS, O3D, CopperLicht, CubicVR

ContentJavaScript, HTML, CSS, ...

WebGL HTML5

JavaScript CSS

BrowserProvided

OSProvided

DownloadFrom Web

Plug-in

Production support expected in 1Q11

OpenGL ES 2.0OpenGL

DX9/Angle

Page 51: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 51

WebGL Compositing

• WebGL HTML fully participates in canvas/browser compositing

- Can overlay and underlay HTML content - including SVG

- Easy to make HUDs or user interfaces

• WebGL TexImage2D accepts

- Raw buffer

- HTML image element

- HTML video element (takes current frame)

- HTML canvas element

E.g. draw text to 2D canvas, use as bump-map texture in WebGL

then composite 3D model over HTML text

Page 52: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 52

OpenGL Ecosystem

Leading-edge functionality developed first on desktop

OpenGL ES 2.0 on desktop as subset of OpenGL 4.1 for mobile

content flexibility – including native support for WebGL

WebGL will drive new-generation security into

OpenGL family

Mobile functionality subset that is deployed on billions of devices

Pervasive OpenGL ES 2.0 availability enables Browser vendors to build 3D

directly into HTML5

Page 53: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 53

Khronos Summary

• Consortium for cooperation over open, royalty-free API standards

- ‘By the industry, for the industry’

• Strong commercial focus

- Enabling members and the wider industry to maximize market opportunities

• Any company welcome to join and influence activities and directions

- Non-profit founded ten years ago – over 100 members

• IP Framework enables open and royalty-free standards

- Conformance tests provide cross-platform reliability

• “Foundation” APIs to access graphics, imaging and compute silicon

- Drives widespread adoption into multiple platforms

• Hundreds of man years invested

- Beyond any single company to produce specifications of this breadth and depth

Page 54: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 54

How to Get Involved!

Page 55: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 55

Khronos Membership

Academic Contributor Promoter

Member Fee $1,000 $10,000 $25,000

Marketing Benefits

Full WG Participation

WG Vote

Board Seat

Page 56: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 56

How Do Khronos Working Groups Work?

• All working groups use online Khronos resources

- Mail reflector, Wiki, SVN, Bugzilla etc…

• Most working groups teleconference 90 minutes per week

- Meetings recorded and posted to member web-site

• 120 members attend a 5 day meeting three times a year

- All Khronos members welcome, all working groups are covered

Page 57: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 57

Key F2F San Jose Discussions

• Progressing/Finalizing key specifications

- OpenGL ES 3.0, OpenCL 1.2, OpenGL 4.2, OpenMAX IL 1.2, EGL 1.5, WebGL 1.0

• New working groups

- OpenRT (ray-tracing), OpenCV (computer vision)?

• Progressing COLLADA Conformance Program – inc COLLADA 1.5 tests

• Finalizing EGL synchs and stream specs for graphics/video interop

• 3D Convergence - does OpenGL ES become an OpenGL profile over time?

• Need EGL and OpenMAX IL on desktop – increased application portability?

• Bringing vector graphics to OpenGL ES while evolving OpenVG

• W3C liaison for web-based augmented reality

- Advanced camera control, system-wide synch, WebCL

Page 58: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 58

Thank you for coming!

• Khronos Chinese language website

- http://www.khronoschina.org/

- All the slides from today will be posted there

• More questions?

- Walter Wen [[email protected]]

Page 59: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 59

An Invitation…If you want to know more, join us tomorrow for a fast overview, more demos

and a 10th anniversary luncheon. At this event you will have more time to

talk 1-1 with the Khronos speakers!

Beijing Hilton Wangfujing Hotel

No.8 Wangfujing East Street, Beijing

10:00am to 1:00pm – Huton room

Fast Overviews on a wide variety of Khronos APIs by Khronos founding members

1:00pm – 2:00pm – Lantern room

10th Anniversary Luncheon and numerous exciting interactive demos

For detailed agenda, maps, and registration visit:

http://www.khronos.org/news/events/detail/khronos-beijing-devu/

Page 60: Building Markets for Advanced Devices through Open Standards · 2014-04-08 · Khronos - Forum for Industry Cooperation Open International Membership Any company is welcome to join

© Copyright Khronos Group, 2010 - Page 60

邀 请如果您对今天的会议内容感兴趣,并希望贵公司今天未能参会的同事,朋友了解

Khronos Group 相关技术, 我们在此诚邀您及您的朋友参加明天在希尔顿王府井酒店

的简短会议,届时会有简洁的技术介绍演讲和相关产品演示。

我们并诚邀您参加明天的 Khronos Group 成立十周年庆祝午宴,届时您可有更多与

会员和行业伙伴面对面交流的机会。

北京希尔顿王府井酒店

王府井东路8号

10:00am - 1:00pm – 酒店五层 Hutong 会议厅

由 Khronos 创始会员为您介绍 Khronos 广为采纳的 API 标准技术

1:00pm – 2:00pm – 酒店五层 Lantern 餐厅

Khronos Group 十周年庆祝午宴和产品演示

详细日程,地图及相关信息: http://www.khronos.org/news/events/detail/khronos-beijing-devu/