phase messaging method for time-of-flight cameras€¦ · softkinetic ds311 . time-of-flight...
TRANSCRIPT
![Page 1: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/1.jpg)
Phase Messaging Method for Time-of-flight Cameras
Wenjia Yuan † Richard Howard † Kristin Dana † Ramesh Raskar ‡
Ashwin Ashok † Marco Gruteser † Narayan Mandayam †
† Rutgers University
‡ Massachusetts Institute of Technology
CNS-1065463
![Page 2: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/2.jpg)
5/3/2014 2
Camera-Display Communication
Displays Cameras
![Page 3: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/3.jpg)
Related work: QR codes
5/3/2014 3
[1] Kamijo et al. ICIP. 2008.
![Page 4: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/4.jpg)
Related work: Visual MIMO
5/3/2014 4
[1] Ashok et al. MobiCom. 2010. [2] Yuan et al. WACV. 2012. [3] Yuan et al. GlobalSip.2013
![Page 5: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/5.jpg)
Other related work
5/3/2014 5
[1] Schmid et al. MobiHoc. 2013. [2] Perli et al. MobiCom. 2010. [3] Mondal et al. APCC. 2012.
![Page 6: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/6.jpg)
Our work
5/3/2014 6
Time-of-flight Cameras
Phase Messaging Array (PMA)
![Page 7: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/7.jpg)
Time-of-Flight Cameras
5/3/2014 7
PMD CamBoard Nano Swiss Ranger SR4050
ToF camera sensor
SoftKinetic DS311
![Page 8: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/8.jpg)
Time-of-Flight Cameras
5/3/2014 8
PMD CamBoard Nano Swiss Ranger SR4050
ToF camera sensor
SoftKinetic DS311
Infrared LEDs
![Page 9: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/9.jpg)
Time-of-Flight Cameras
5/3/2014 9
phase difference
depth
SoftKinetic DS311
delay 𝜏
![Page 10: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/10.jpg)
5/3/2014 10
![Page 11: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/11.jpg)
System Overview
5/3/2014 11
Extracted Message Stream
Depth Image
LEDs of the ToF camera
Depth Recovery
Sensor of the ToF camera
Photo Detector
Amplifier
LED
Phase Shifting
Multiplexer
Message Stream
FIFO
Data Format
Converter
Shift Register
Light Detection Phase Modulation
……
p
p0 pn-1
A0
Am-1
Data Processing
Transmitter PMA
Synchronized
Receiver ToF Camera
![Page 12: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/12.jpg)
Phase Messaging Array (PMA)
5/3/2014 12
Light Detection
LED
Ph
ase
Mo
du
lati
on
Data Processing
![Page 13: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/13.jpg)
Intuitive Solution
5/3/2014 13
LED
Light Detection
LED
Ph
ase
M
od
ula
tio
n
Data Processing
Time-of-flight Camera
![Page 14: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/14.jpg)
Our Solution
5/3/2014 14
Light Detection
LED
Ph
ase
M
od
ula
tio
n
Data Processing
Photo Detector
LED
![Page 15: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/15.jpg)
Phase Modulation
• Phase Shifting
5/3/2014 15
Light Detection
LED
Ph
ase
M
od
ula
tio
n
Data Processing
185o 280o 250o 320o
![Page 16: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/16.jpg)
Phase Modulation
• Messaging and Signal Selection
5/3/2014 16
Light Detection
LED
Ph
ase
M
od
ula
tio
n
Data Processing
![Page 17: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/17.jpg)
Phase Modulation
• Message Recovery
5/3/2014 17
Light Detection
LED
Ph
ase
M
od
ula
tio
n
Data Processing
p2 p0 p1 p3 p2 p1 p0 p3 p1
10 00 01 11 10 01 00 11 01
![Page 18: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/18.jpg)
Experiments
5/3/2014 18
PMA SoftKinetic
DS311
Five independent experiments 8000 random bits/experiment Average accuracy 97.8%
![Page 19: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/19.jpg)
Experiments
• Data rate analysis
– 9 LEDs
• One for synchronization indicator
• Another 8 for messaging
– 4 candidate phases => 2 bits/LED
– ToF camera
• 60 fps => 8x2x60 = 960bps
5/3/2014 19
![Page 20: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/20.jpg)
Applications
• Dual use of cameras
– Imaging (original use)
– Communication (added channel)
• Applications
– Object recognition: ID beacon
– Navigation: path marker
– Robotics: line of sight communication
5/3/2014 20
![Page 21: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/21.jpg)
Conclusions
• Propose a camera-display communication methodology based on a low-cost continuous-wave ToF camera
• Implement Phase Messaging Array (PMA)
• Demonstrate high messaging accuracy in experiments
5/3/2014 21
![Page 22: Phase Messaging Method for Time-of-flight Cameras€¦ · SoftKinetic DS311 . Time-of-Flight Cameras 5/3/2014 8 PMD CamBoard Nano Swiss Ranger SR4050 ToF camera sensor SoftKinetic](https://reader034.vdocuments.mx/reader034/viewer/2022050217/5f638778284c0060890b3527/html5/thumbnails/22.jpg)
5/3/2014 22