gesture and facial tracking for html5 using intel ... · pdf filegesture and facial tracking...
TRANSCRIPT
Gesture and Facial Tracking for HTML5 using Intel® RealSense Technology
Shwetha DossSr. Application Engineer
Software and Services Group
Intel Corporation
Agenda
• Intel® RealSense™ Technology
• Exploring the Intel® RealSense SDK
• Integration of Intel® RealSense SDK with
HTML5
• Summary
The Intel® RealSense™ SDK
6
Free Tools and APIs for building natural user interfaces.
Public Beta for Windows available
Accessible for beginners. Open and extensible for experts.
Support for Windows 8.1 today. Android support coming.
Reach a broad-base of customers by writing apps for the Intel RealSense 3D Camera.
Stand out in a crowded market. Delight your customers.
Works with languages, frameworks, and game engines you already use
INTEL CONFIDENTIALSoftware and Services GroupSoftware and Services GroupSoftware and Services GroupSoftware and Services Group
Front-Facing Camera Usage
Learning and Edutainment
Immersive Collaboration,Creation
Capture and Share
Interact NaturallyGaming and Play
Broad Enabling
INTEL CONFIDENTIALSoftware and Services GroupSoftware and Services GroupSoftware and Services GroupSoftware and Services Group
Rear-Facing Camera Usage
Learning and Edutainment
Capture the world in 3D
Immersive Gaming
Enhanced Photo and Video
Collaboration
INTEL CONFIDENTIALSoftware and Services GroupSoftware and Services GroupSoftware and Services GroupSoftware and Services Group12
Snapshot Camera Usage
13
SDK Supported FeaturesRequired Hardware A system with a minimum of a 4th generation Intel® Core™
processor
(codenamed Haswell), either IA-32 or Intel® 64, with
integrated depth camera
Required OS Microsoft* Windows 8.1 OS (32- and 64-bit)
Microsoft* Windows 8.1 Metro (coming soon)
Supported Programming
Languages
C++, C#, JavaScript, Java (JDK 1.7.0_11 or later for face,
voice, and gesture)
Supported IDE Microsoft* Visual Studio C++ 2010-2013 with service pack
1 or newer
Eclipse.org IDE
Supported Development Tools -Microsoft* .NET 4.0 Framework for C# development
-Unity* PRO 4.0.0 or later for Unity game development
-Processing* 2.0.7 or later for Processing framework
development
-OpenFrameworks* v0071 or later for OpenFrameworks
development
-Havok* Vision SDK 2012.2.1 or later for Havok Vision
SDK development
-Intel® Media SDK 2014
Hands
Face
Speech
Environment
15
The Intel® RealSense™ SDK
Modules
Use standalone or in combination
Categories of
Input
Capabilities Features
Hands • Hand and Finger
Tracking
• Gesture Recognition
• 22-point Hand and Finger Tracking
• 9 static and dynamic mid-air gestures
Face • Face Detection and
Tracking
• Multiple Face Detection and tracking
• 78-point Landmark Detection (facial features)
• Emotion Recognition (7 emotions, coming post-Beta)
• Pulse Estimation
• Face Recognition (Coming post-beta)
Speech • Speech Recognition • Command and Control
• Dictation
• Text to Speech
Environment • Segmentation
• 3D Scanning
• Augmented Reality
• Background Removal
• 3D Object / Face / Room Scanning (Coming post-beta)
• 2D/3D Object Tracking
• Scene Perception (coming post-beta)
SDK Runtime
Hand
Tracking
Microsoft® USB Video Class Camera Driver
Background
Segmentatio
n
Core SpeechFace
Analytics
Aug-
mented
Reality
Application
Intel® RealSense SDK Architecture
RealSense SDK Package
Developer Tools
Libs
Docs & Samples
…
Power Mgmt FW UpdateIOModule
Camera Module Camera
Firmware
Depth Camera
Management
19
19
Gestures and Facial Analysis
Unity* Web Player games
HTML5 / JavaScript (interactive websites)
Unity Web Player App
HTML5 / JavaScript
SDK Local
Web-Socket Server
Intel® RealSense™
SDK runtime
Operating System
Web Socket Communication
127.0.0.1:4181
* Other names and brands may be claimed as the property of others
Web/ HTML5
Alerts
– HandDetected:
– a hand was identified
– HandTracked:
– full skeleton available for detected hand
– HandCalibrated:
– hand measurements ready
– HandOutOfLeftBorder:
– hand touching the tracking borders
– Same for: HandOutOfRightBorder, HandOutOfTopBorder,
HandOutOfBottomBorder
– HandTooClose / HandTooFar: hand reaching closest/furthest range
25
Output
Face Detection
Bounding rectangle
Average face depth
Face landmark
Points
Face Pose
Facial Expressions
31
Alerts
- Face in scene
- New face detected
- Face lost
- FOV
- Face out of FOV
- Face Back to FOV
- Occlusion
- Face Occluded
- Face not occluded
- Face attached object
32
Create A New World of
Interactivity• Intel RealSense offers an opportunity to
enhance how users interact with compute
devices
• The ecosystem is gaining momentum and Intel
is making a significant investment
• Depth camera integration in 2014 paves the
way to volume
41
More @ https://software.intel.com/realsense
Connect with us
• Visit us at the Intel booth
• Attend the Intel sessions at MODS
• Building your First Internet Of Things Solution - a Software Developer Perspective:
Oct 9, Hall B,11.10 – 12.10
• Accelerating Your Software Development for Android* on Intel® Platforms : Oct 9,
SD Hall,15.20 – 16.20
• Gesture and Facial Tracking for HTML5 apps using Intel® RealSense™ Technology
Oct 10, Hall B,12.20 – 13.20
• Join us at Intel® Developer Zone: software.intel.com
• Look for us on:
facebook.com/inteldeveloperzone
twitter.com/intelsoftwareinFacebook#IntelSoftwa
re