matteo valoriani - how augment your reality: different perspective on the reality-virtuality...
TRANSCRIPT
How Augment your Reality: Different perspective on the Reality / Virtuality Continuum
Matteo Valoriani
CODEMOTION MILAN - SPECIAL EDITION
10 – 11 NOVEMBER 2017
How Augment your Reality: Different perspective on the Reality / Virtuality Continuum
Nice to Meet You
Matteo Valoriani, PhD
CEO of FifthIngenium
mvaloriani at gmail.com@MatteoValorianiSlideshare: www.slideshare.net/MatteoValorianiLinkedin: https://it.linkedin.com/in/matteovalorianiBlog: http://fifthingenium.com/blogGitHub: https://github.com/mvaloriani
How we are arrived here
Virtual Continuum
VR
• Outside-in vs Inside-out
AR
• Visual Inertial Odometry (VIO)
• ARKit/ARCore
• HoloLens
Agenda
2010 - 2017
1968, Apollo Guidance
Computer
2000 Transistor
4k Memory
CPU evolution (Transistors)
5M 160M 1B7B / 2B on mobile
10B / 3.3B on mobile
230 4.400 32.000 90.000 (1996) (2003) (2008) (2016)
https://www.youtube.com/watch?time_continue=257&v=z6wvFg-c0Rs
RECAP
Virtual Reality
CAVE vs HMD (Head Mounted Display)
Mobile VR
Console VR
Desktop VR (Outside-in)
Pros
• Accuracy (you can add more trackers around the room to increase accuracy)
• Latency very low (reduce VR sickness).
Cons
• Setup
• Occlusion
• Limitation of the cameras (field of view)
Desktop VR (Inside-out)
• .
Pros• Simplicity / No Setup
• Less hardware needed (no external camera)
• No limit of space (potentially)
Cons• Reduced accuracy and
latency
• Only the object in front to the user can be tracked
• Powerful hardware
VR addons
VR Scenarios and Limitations
Augmented Reality
Optical vs Video see - through
Visual Inertial Odometry (VIO)
Visual Inertial Odometry (VIO)
6dof in real time : • 3D of xyz motion (translation)• 3D of pitch/yaw/roll (rotation)
Pose is tracked:• via the Camera system, by matching a point in the real world to a pixel on the camera sensor each frame. • by the Inertial system (accelerometer & gyroscope — Inertial Measurement Unit or IMU)The output of both of those systems are then combined via a Kalman Filter which determines which of the two systems is providing the best estimate of your “real” position.
b
𝑑 = 𝑥𝑙 − 𝑥𝑟
𝑏+𝑥𝑙 − 𝑥𝑟
𝑍−𝑓=
𝑏
𝑍
Z =𝑏∗𝑓
𝑑
Depth Sensing (Multi camera)
Single Camera?
The secret here is to have *really* good IMU error removal: Dead-reckoning
• .
TrueDepth CameraFace tracking.
Scene Understanding Detection of horizontal planes (tables, floors, …) and tracking of object.
Lighting EstimationEstimation of light available in a scene in order to apply the correct amount of lighting to virtual objects.
Hardware and Rendering OptimizationsApple A9, A10, and A11 processors.
• .
Motion trackingEstimates a phone’s relative location based on internal sensors and video footage
Scene Understanding Uses the camera to detect flat surfaces
Lighting EstimationHelps virtual props cast accurate shadows and otherwise fit in with their surroundings
Anchoring objects
• .
ARCore vs Tango
Tango’s dedicated infrared depth-sensing camera to create detailed 3D meshes.
ARCore has to estimate scale based on the camera feed, while Tango directly measures distance.
Nikhil Chandhok (AR director of product): “ARCore is for all the apps that we think that users want”
HoloLens
What is HoloLens?HoloLens is the first, fully wireless holographic computer that redefine personal computing and empowers people in new ways.
How to use
DEMO
Hololens Hadware Blocks
Holographic Processing Unit v 1.0
TSMC-fabricated 28 nm co-processor.
24 Tensilica DSP cores (12 clusters)
65 million logic gates (used 50%)
8 MB of SRAM
1GB DRAM
1 Trillion Operation per second
Sensor aggregator with gesture and environment processing
200x over software implementation
Low Power (<10 Watts)
12 mm
12
mm
AR Devices
Altri framework: Vuforia
• .I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die.
Developer Resources
Preview Pricing
Documentation
Client SDKs
Example Code
Join Our Community
Thank you!
Developer Resources
Preview Pricing
Documentation
Client SDKs
Example Code
Join Our Community
Thank you!How Augment your Reality Matteo Valoriani
https://medium.com/super-ventures-blog/how-is-arcore-better-than-arkit-5223e6b3e79d
https://www.theverge.com/2017/8/29/16219696/google-arcore-augmented-reality-platform-announce-release-pixel-samsung
https://www.youtube.com/watch?v=dNXBvDKRg1M
https://www.marxentlabs.com/deep-dive-apple-arkit-game-changer-augmented-reality/
https://developer.apple.com/arkit/
https://developers.google.com/ar/discover/concepts
https://kotaku.com/how-lara-croft-has-changed-over-the-last-16-years-1505856265
http://www.ign.com/articles/2016/07/11/23-perfectly-timed-pokemon-go-screenshots
https://www.wareable.com/trending/trending-vr-add-ons-are-next-big-thing
https://www.howtogeek.com/246333/oculus-rift-vs.-htc-vive-which-vr-headset-is-right-for-you/
https://www.slideshare.net/AugmentedWorldExpo/hiren-bhinde-qualcomm-ondevice-motion-tracking-for-immersive-vr
references