gltf: efficient 3d models...3d graphics and virtual reality with vulkan and openxr neil trevett,...
TRANSCRIPT
© The Khronos® Group Inc. 2018 - Page 1This work is licensed under a Creative Commons Attribution 4.0 International License
glTF: Efficient 3D Models
Tokyo, December 2018
© The Khronos® Group Inc. 2018 - Page 2This work is licensed under a Creative Commons Attribution 4.0 International License
Welcome to Khronos BOF Sessions!
Time Session Speakers
9:30AM-12:30PM 3D Graphics and Virtual Reality with Vulkan and OpenXR
Neil Trevett, NVIDIA - Introduction to Vulkan and OpenXRHai Nguyen, Google - Vulkan: Getting Started, Best Practices and using HLSL
Ashley Smith, AMD - Vulkan Memory and Heap Types – and When to Use Which!Nuno Subtil, NVIDIA - Using Subgroups to Bring More Parallelism to your Application
Teemu Virolainen, Basemark - Vulkan in the Rocksolid Engine
2:30PM-4:30PM WebGL: Latest Techniques
Xavier Ho, CSIRO’s Data61 and Juan Miguel de Joya, UN ITU - Welcome!Neil Trevett, NVIDIA - A Brief History of WebGL
Xavier Ho and Juan Miguel de Joya - Optimizing your WebGL Xavier Ho and Juan Miguel de Joya - Survey of WebGL Applications
Mike Alger and Germain Ruffle, Google - User Experiences for Digital/Augmented Spaces
4:30PM-5:30PM glTF: Efficient 3D Models Neil Trevett, NVIDIA - glTF Overview and Latest Updates Mark Callow, glTF Working Group - Universal Compressed Texture Transmission Format Update
5:30PM-5:35PM Japanese Greeting and Invitation to Khronos! Hitoshi Kasai, Khronos - Learn how your company can benefit from joining Khronos!
5:35PM-6PMStandards for machine learning,
inferencing and vision acceleration: NNEF, OpenVX and OpenCL
Neil Trevett, NVIDIA - Overview of NNEF, OpenVX and OpenCL and how they relate to each other to deliver effective acceleration for inferencing, machine learning and vision processing
All slides will be posted at www.khronos.org
© The Khronos® Group Inc. 2018 - Page 3This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos Mission
Software
Silicon
Khronos is an International Industry Consortium creating royalty-free, open standards to enable software to access hardware acceleration for
3D Graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing
Asian Members
© The Khronos® Group Inc. 2018 - Page 4This work is licensed under a Creative Commons Attribution 4.0 International License
Khronos Standards
High-performance, low-latency 3D Graphics
Download 3D object and scene data
Vision and sensor processing -including neural network
inferencing for machine learning
Portable interaction with VR/AR sensor, haptic and
display devices
© The Khronos® Group Inc. 2018 - Page 5This work is licensed under a Creative Commons Attribution 4.0 International License
glTF – The JPEG of 3D!
Efficient, reliable transmissionBring 3D assets into 1000s of
apps and engines
glTF 2.0 – June 2017Native AND Web APIsPhysically Based Rendering
Metallic-Roughness and Specular-Glossiness
glTF 1.0 – December 2015Primarily for WebGL
Uses GLSL for materials
glTF spec development on open GitHub – get involved!
https://github.com/KhronosGroup/glTF
Compact to TransmitSimple and Fast to LoadDescribes Full Scenes
Runtime NeutralOpen and Extensible
© The Khronos® Group Inc. 2018 - Page 6This work is licensed under a Creative Commons Attribution 4.0 International License
glTF 2.0• Graphics API neutral
- Proven by engine implementations using WebGL, Vulkan and Direct3D
• Physically Based Rendering (PBR) material definitions- Material information stored in textures
• Deployment as single file- Binary glTF (.glb) in core spec
• Morph Targets- Enhanced animation system
• Download the glTF 2.0 Reference Guide!- https://www.khronos.org/files/gltf20-reference-guide.pdf
© The Khronos® Group Inc. 2018 - Page 7This work is licensed under a Creative Commons Attribution 4.0 International License
glTF 2.0 Scene Description Structure
.gltf (JSON)Node hierarchy, PBR material textures, cameras
.binGeometry: vertices and indices
Animation: key-framesSkins: inverse-bind matrices
.png.jpg...
Textures
Texture based PBR materials
Geometry
Mandatory Metallic-Roughness Materials
Optional Specular-Glossiness Materials
© The Khronos® Group Inc. 2018 - Page 8This work is licensed under a Creative Commons Attribution 4.0 International License
glTF 2.0 Scene Description Structure
.gltf (JSON)Node hierarchy, PBR material textures, cameras
.binGeometry: vertices and indices
Animation: key-framesSkins: inverse-bind matrices
.png.jpg...
Textures
PBR materialsGeometry
© The Khronos® Group Inc. 2018 - Page 9This work is licensed under a Creative Commons Attribution 4.0 International License
PBR Materials
© The Khronos® Group Inc. 2018 - Page 10This work is licensed under a Creative Commons Attribution 4.0 International License
PBR Materials
© The Khronos® Group Inc. 2018 - Page 11This work is licensed under a Creative Commons Attribution 4.0 International License
PBR Materials
© The Khronos® Group Inc. 2018 - Page 12This work is licensed under a Creative Commons Attribution 4.0 International License
PBR Sample Applications
https://github.com/Microsoft/glTF-DXViewerhttp://github.khronos.org/glTF-WebGL-PBR/
© The Khronos® Group Inc. 2018 - Page 13This work is licensed under a Creative Commons Attribution 4.0 International License
glTF Binary - .glb• Binary version of glTF• Single file that encapsulates all geometry and textures• Ideal for end users to interact with a single file
© The Khronos® Group Inc. 2018 - Page 14This work is licensed under a Creative Commons Attribution 4.0 International License
Industry Support for glTF
https://www.khronos.org/gltf/#gltf_industry_support
© The Khronos® Group Inc. 2018 - Page 15This work is licensed under a Creative Commons Attribution 4.0 International License
glTF Ecosystem
Sony 3D Creator
Oculus
Paint 3D
Collada2gltf FBX2glTF
gltf-vscode
glTF-validator glTF-asset-generator
ExperienceApps and Engines
DiscoverRepositories
Drive DemandUsers
CreateTools
Windows Mixed Reality Home
Modo
Mixed Reality Viewer
3D BuilderPrep for 3D printing
OBJ2GLTF
glTF-Toolkit
Titania
HTC
glTF Reference Viewer
© The Khronos® Group Inc. 2018 - Page 16This work is licensed under a Creative Commons Attribution 4.0 International License
glTF Recent Highlights
Adds glTF to StemCell - 60K+ 3D artists and 700K 3D modelshttps://www.khronos.org/blog/turbosquid-adds-gltf-to-supported-formats-for-its-stemcell-initiative
Supports drag and drop of glTF models to your feedhttps://developers.facebook.com/docs/sharing/3d-posts/
Over 150K royalty-free glTF modelshttps://sketchfab.com/features/gltf
Dimension generates glTF for delivery of 3D marketing assetshttps://theblog.adobe.com/behind-the-scenes-with-adobe-dimension-engineers-how-we-built-the-3d-publish-feature/
Integrating glTF into ‘HUBS’ Web VR Meeting Space and ‘Spoke’ VR Authoring Toolhttps://www.roadtovr.com/mozillas-hubs-one-click-vr-meeting-space-ive-waiting/
OGC liaison and 3D Tiles OGC Community Standard candidate https://github.com/AnalyticalGraphicsInc/3d-tiles/
Microsoft makes glTF files as pervasively usable as JPGs in Windows 10Open sources entire glTF SDK! https://github.com/Microsoft/glTF-SDK
Open source WebComponent 3D model viewer Import of glTF into AR Core apps via the Google Sceneform Tools plugin
https://github.com/GoogleWebComponents/model-viewer
© The Khronos® Group Inc. 2018 - Page 17This work is licensed under a Creative Commons Attribution 4.0 International License
Blender glTF 2.0 Exporter
glTF IO shipping with Blender 2.8 Beta! https://developer.blender.org/rBAb1f2133fa2849da272e9d8404f371220226ddaf1
© The Khronos® Group Inc. 2018 - Page 18This work is licensed under a Creative Commons Attribution 4.0 International License
Focus on glTF Ecosystem Robustness• Khronos constantly working on improving ecosystem's consistency
- Rendering (reference viewer, reference environment) - Technical low-level issues (validator & asset generator)
• If you are CREATING glTF Files- Ensure generated files are validator clean
- https://github.com/KhronosGroup/glTF-Validator- Help the community understand what your exporter supports
- https://github.com/KhronosGroup/glTF/issues/1271
• If you are LOADING glTF files- Ensure loader can correctly load all sample models (integration tests)
- https://github.com/KhronosGroup/glTF-Sample-Models- Ensure loader can correctly load all asset generator models (unit tests)
- https://github.com/bghgary/glTF-Asset-Generator
Users of glTF can help to keep glTF reliable and consistent!
© The Khronos® Group Inc. 2018 - Page 19This work is licensed under a Creative Commons Attribution 4.0 International License
Rendering quality output
glTF Reference Viewer• Generates accurate Ground Truth renderings of glTF Models
- https://github.com/KhronosGroup/glTF-WebGL-PBR/tree/reference-viewer
• Not all glTF apps and engines need visual consistency- But is critical to key use cases such as online retail
• Headless mode, generates images - Compare against offline path-traced renderings - For regression testing
• Can be embedded in Visual Studio for live model previews
© The Khronos® Group Inc. 2018 - Page 20This work is licensed under a Creative Commons Attribution 4.0 International License
glTF Roadmap and Extensions• glTF manages its roadmap very carefully – complexity is the enemy
- Mission #1: ensure widespread, consistent, reliable usage
- Extension for your own use – create a Vendor Extension- Register your PREFIX by submitting a GitHub Issue
- Multiple applications from multiple vendors need an extension- Create a multi-vendor “EXT” extension
- Broadly applicable across all apps/platforms - propose a “KHR” extension- Need at least two implementations- Discussed and agreed by glTF working group - Covered by Khronos Intellectual Property Framework
- Always have a fallback to core spec- Avoid breaking compatibility with
broader ecosystem- If you choose to not have a fallback
list your extension in extensionsRequired
Stable Core Spec
Domain-specificextensions stay as
extensions
New widely needed functionality ships first as
extensions
Integrate extensions into new core spec only when:1) Widespread need is confirmed by the industry
2) Widespread reliable implementation is enabled (e.g. open source)
© The Khronos® Group Inc. 2018 - Page 21This work is licensed under a Creative Commons Attribution 4.0 International License
Draco glTF Mesh Compression Extension• Library for compressing and decompressing 3D geometric meshes and point clouds
- https://github.com/google/draco
• Google has released Draco encoders and decoders in open source - C++ source code encoder to compress 3D data- C++ and JavaScript decoders for the encoded data- https://github.com/google/draco/tree/gltf_2.0_draco_extension
• Draco glTF extension launched in February 2018- Draco designed and built for compression efficiency and speed - great fit with glTF!- https://github.com/KhronosGroup/glTF/blob/master/extensions/2.0/Khronos/KHR_draco_mesh_compression/README.md
Mesh Compression
Ratios
© The Khronos® Group Inc. 2018 - Page 22This work is licensed under a Creative Commons Attribution 4.0 International License
glTF Roadmap Discussions• Many of these are being discussed on GitHub
- Come and give your views! https://github.com/KhronosGroup/glTF
• Universal Compressed Texture Transmission Format CTTF)- https://github.com/KhronosGroup/glTF-Texture-Transmission-Tools/milestone/1
• PBR 2.0 – advanced materials- Call for key use cases: https://github.com/KhronosGroup/glTF/issues/1442
• Animation 2.0- Including Avatars and Face emoji, with compression
• Metadata• LOD and Streaming• Point Clouds (with compression)
NVIDIA MDL Physically Based Rendering
glTF Roadmap is Driven by Developer Feedback and Requirements
© The Khronos® Group Inc. 2018 - Page 23This work is licensed under a Creative Commons Attribution 4.0 International License
3D Descriptor Database
With Consumer Capture – 3D Will Go Social!
ObjectCapture
Upload, View, Share and Comment
Inspire and Motivate
3D Printing (e.g. Shapeways.com)
Social Loop Photos -> FacebookVideos -> YouTube3D -> glTF!
Manufacturers provide 3D Object Descriptors - much
more information than Amazon-style 2D search
Search
© The Khronos® Group Inc. 2018 - Page 24This work is licensed under a Creative Commons Attribution 4.0 International License
Thank You!• Khronos is creating cutting-edge royalty-free open standards
- For 3D, VR/AR, Compute, Vision and Machine Learning
• Please join Khronos!- We welcome members from Japan and Asia- Influence the direction of industry standards- Get early access to draft specifications- Network with industry-leading companies
• More Information- www.khronos.org- [email protected] @neilt3d
Khronos Group Japan Local Member MeetingOpen to current and prospective members!
Thursday, 6 December 20185:00 PM – 8:00 PM JST
Yurakucho Cafe & DiningB1F, Tokyo International ForumChiyoda-ku, Tōkyō-to 100-0005Please Register for Free!