![Page 1: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/1.jpg)
Understanding M3G 2.0 and its Effect on Producing Exceptional 3D Java-Based Graphics
Sean EllisConsultant Graphics Engineer
ARM, Maidenhead
![Page 2: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/2.jpg)
222Confidential
IntroductionM3G 1.x RecapARM M3G IntegrationM3G 2.0 UpdateM3G 2.0 Hints and Tips
![Page 3: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/3.jpg)
333Confidential
AudienceWho were we addressing with M3G 1.x?
Java design skill
3D design skill
“TechnicalArtists”
ExperiencedSoftware Engineers
The same people need to be able to use M3G 2.0.
![Page 4: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/4.jpg)
444Confidential
Basic MIDlet ArchitectureLoad assets, find commonly used objects, set up display, initialize game state
paint()Canvas calls Graphics3D to perform rendering
initialize
user input
scene update
request redraw
wait
shut down
Get any user input, network play, etc.
Game logic, animate, align if necessary
Wait to ensure consistent frame rate
Release assets, tidy up
Exit request
Main loop.
Don’t create/destroy objects if possible
Throttle to consistent frame rate
Keep paint() as simple as possible
Be careful with threads
![Page 5: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/5.jpg)
555Confidential
Typical RelativePerformance
Java Virtual MachineOther
Library(OEM specific)
OtherClasses
(OEM specific)
Java with Native Library
Hardware
OS
NetworkLibrary
(J2ME)
InputLibrary
(J2ME)
SoundLibrary
(OEM specific)
NetworkClasses
(J2ME)
InputClasses
(J2ME)
SoundClasses
(OEM specific)
3D Application using SwV classes2D UI
Classes(J2ME)
2D UILibrary
(J2ME)
M3GClasses
M3GLibrary
Scene managementAnimations
3D transformation Lighting
Triangle drawingTexturing
Transfer to 2D
Interaction from user3D object control
FASTMost work donehere - at high performance
![Page 6: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/6.jpg)
666Confidential
Class HierarchyObject3D Node
Mesh
Camera
Light
Group
VertexBuffer VertexArray IndexBuffer TriangleStripArray
Appearance
Material
Fog
PolygonMode
CompositingMode
Texture2D
Background
Image2D
World
SkinnedMesh
MorphingMeshSprite3D
Graphics3D
Transformable
Transform
AnimationTrack AnimationController
KeyframeSequence
Loader RayIntersection
![Page 7: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/7.jpg)
777Confidential
M3G Integration – Top End
C++ Implementation
C Interface Layer
Some C++ libraries pre-linked
JSR184/297 API
C API
Java Interface Layer
Supplied as a linkable library
Interface Description
Example API implementations
shipped as source
Java Application
API interface codeis auto-generated
for KNI, JNI, C
![Page 8: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/8.jpg)
888Confidential
M3G Integration – Bottom End
C++ Implementation
C Interface Layer
Some C++ libraries pre-linked
Supplied as a linkable library
OSAbstraction
GLAbstraction
SurfaceAbstraction
GLES EGL
OS
Example implementations
shipped as source
Abstraction APIsSurface abstraction
needs access to Java VM internals (pixel buffers for
images and Graphics objects)
![Page 9: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/9.jpg)
999Confidential
M3G 2.0 Integration TipGood news, everyone – almost exactly the same as M3G 1.1Some software-renderer-only integration APIs removedNew abstraction interface for DynamicImage2D
![Page 10: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/10.jpg)
101010Confidential
M3G 2.0 New FeaturesCore block
Fixed function (Open GL ES 1.1)M3G 1.x compatibleMali55™ or Mali200™
Advanced blockProgrammable hardwareMali200Can mix modes in API, with some restrictionsCan implement both blocks on OpenGL ES 2.0Must be completely present or absent – less fragmentationJavax.microedition.m3g.shader package is major new addition
DisclaimerSpec still in draft – details subject to change
M3G2.0
Advanced
M3G 2.0 Core
M3G 1.1
M3G 2.0 Core
![Page 11: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/11.jpg)
111111Confidential
NativeVideo Source
New Image Types
Fast pathRenderer
Slow path
Java Video Control
ImageBaseImageCube
DynamicImage
Image2D
![Page 12: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/12.jpg)
121212Confidential
Mesh and its componentsMesh
VertexBuffer
IndexBufferIndexBufferIndexBuffer
AppearanceAppearanceAppearance
VertexArray Vertex positions
VertexArray Vertex normals
VertexArray Vertex colors
VertexArray Texture coords
VertexArray Point sizes
VertexArray User attributes
VertexArray User attributes
Vertex arrays now can contain values of type byte, short, 16.16 fixed point,
float, or half-float
![Page 13: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/13.jpg)
131313Confidential
Mesh and its componentsMesh
VertexBuffer
IndexBufferIndexBufferIndexBuffer
AppearanceAppearanceAppearance
TriangleStripArrayExplicit or sequential vertex indices
Index into…
TriangleArrayExplicit or sequential vertex indices
LineArrayExplicit or sequential vertex indices
PointArrayExplicit or sequential vertex indices
![Page 14: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/14.jpg)
141414Confidential
ShaderAppearance
Appearance and its components
AppearanceBase
Material Diffuse, emissive, specular, ambient colors, shininess
PrimitiveMode
Perspective correction, winding, culling
CompositingMode
Screen buffer write mode, thresholds, depth offset
Fog Fog density, color
Texture2DTexture2DTexture application mode, coordinate transform, wrapping
Image2D
Appearance PointModeLineMode
PolygonMode
![Page 15: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/15.jpg)
151515Confidential
ShaderAppearance
Appearance and its componentsShader
Program
ShaderUniforms
VertexShader
Texture2D TextureCube
Image2D
FragmentShader
ImageCube
![Page 16: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/16.jpg)
161616Confidential
Vertex ShaderFirst, vertex shader transforms the position of each vertex in an object.Often also calculates other parameters
InputObject
TransformedObject
Transformed PositionLit Color
Transformed Texture Coords
PositionNormalColor
Texture Coords
![Page 17: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/17.jpg)
171717Confidential
Vertex ShaderSame vertex shader runs on all vertices in the object.Can assign different shaders to other objects.Results are interpolated across primitives.
InputObject
TransformedObject
TransformedObject
![Page 18: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/18.jpg)
181818Confidential
Fragment ShaderFragment shader takes interpolated (“varying”) variables, and calculates the actual color at each pixel.
TransformedObject
RenderedObject
Texture
![Page 19: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/19.jpg)
191919Confidential
RenderedObject
Fragment ShaderOr it abandons the pixel, rendering nothing, and leaving a hole.
TransformedObject
Texture
![Page 20: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/20.jpg)
202020Confidential
Fragment ShaderSame fragment shader runs on all pixels in the object.Can assign different shaders to other objects.
TransformedObject
RenderedObject
You getthe idea
![Page 21: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/21.jpg)
212121Confidential
M3G 2 Shader Model
Appearance
uniform mat4 modelViewMatrix;attribute vec4 vertexPos;void main(){
gl_Position =modelViewMatrix * vertexPos;
}
uniform mat4 modelViewMatrix;attribute vec4 vertexPos;void main(){
gl_Position =modelViewMatrix * vertexPos;
}
CompleteAppearance
VertexBuffer
UniformVariables
CompiledVertex Shader
CompiledFrag Shader
MeshDisclaimer: Specification
subject to change
CompleteShader Program
![Page 22: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/22.jpg)
222222Confidential
Why keep Appearance?ShaderAppearance can do anything than Appearance canSo why keep it?
Easy transitionMore artists know fixed function pipeline than shaders.Perfectly adequate for a lot of content.Familiarity means faster development.Can use all the legacy content out there.Can mix and match both in one scene.Therefore, shader designers can concentrate effort where it’s going to be noticed.
![Page 23: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/23.jpg)
232323Confidential
Shader Example
![Page 24: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/24.jpg)
242424Confidential
Shader WalkthroughSo, what does a shader look like?
// Astoundingly dull fragment shaderuniform vec4 flatColor;void main(){gl_FragColor = flatColor;
}
Isn’t that just C?
![Page 25: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/25.jpg)
252525Confidential
Same Geometry, Dull Shader
![Page 26: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/26.jpg)
262626Confidential
C-like languageSyntax familiar to most programmersSimplified: no pointers, no charactersNew types: vectors, matrices, samplersNew qualifiers: attribute, uniform, varyingLimited size95% common between Vertex and Fragment shaders
![Page 27: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/27.jpg)
272727Confidential
RenderPassMesh
VertexBuffer
IndexBufferIndexBufferIndexBuffer
AppearanceAppearanceAppearance
Texture2D
Image2D
RenderPass
Renders to…
World
Meshes
Any Node can have a…
![Page 28: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/28.jpg)
282828Confidential
M3G2 Shader TipsTANSTAAFLShaders are not free – they take time, even on hardwareComplex shaders take more timeSpecialize shaders for common parameter values
![Page 29: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/29.jpg)
292929Confidential
The #1 Important Shader Optimization TipDON’T DO IT…yet
Premature optimisation is the root of all evil – C.A.R.Hoare
Profile!Is it really the shaders at all?How fast does my shader run?What’s really holding it up?Then you can really improve things
![Page 30: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/30.jpg)
303030Confidential
M3G2 Shader TipsCompilation time is fast, but not insignificantCompile once, use many times(This is why compilation happens at construction)Don’t create shaders on the fly
It’s a balancing actMore specialized shaders = faster executionMore specialized shaders = slower startup
![Page 31: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/31.jpg)
313131Confidential
M3G2 Shader TipsTools support often at the low levelRequires cooperation from the hardwareMali hardware has debugging/profiling countersLow-level performance analysis tools
FramerateThroughputCache usage
Higher level analysis tools under development:Shader debuggerShader profiler
![Page 32: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/32.jpg)
323232Confidential
Animation ChangesContent developers requested changes to animation system.Designed for ease-of-use.Details currently under discussion in expert group.
![Page 33: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/33.jpg)
333333Confidential
You can contributeEarly Draft Review Spec available at:http://jcp.org/aboutJava/communityprocess/edr/jsr297Or just go to jcp.org and enter 297 in the “Go to JSR” box. The download link is on the resulting page.
We want to hear feedback, especially from content developers
We want you to understand itWe want you to use itWe want you to make good-looking applications
exceptional
![Page 34: Sean Ellis Consultant Graphics Engineer ARM, Maidenheadrtcgroup.com/arm/2007/presentations/121 - Understanding M3G 2.0 and Its Effect on... · Understanding M3G 2.0 and its Effect](https://reader030.vdocuments.mx/reader030/viewer/2022040622/5d31f91d88c9937a3b8ccf5d/html5/thumbnails/34.jpg)
343434Confidential
Q&A
http://jcp.org/aboutJava/communityprocess/edr/jsr297