touchscreen everywhere: on transferring a normal planar … · 2017-08-11 · skinput (bio-acoustic...
TRANSCRIPT
Touchscreen Everywhere:
On Transferring a Normal Planar
Surface to a Touch-Sensitive Display
Jingwen Dai
Postdoctoral Research Associate
University of North Carolina at Chapel Hill
Motivation
04/11/2013
Bigger Display Portability
VS.
NTU IMI PhD Seminar 2/30
Motivation
04/11/2013 3/30
DLP Pico Projector
DC DV Mobile Phone
NTU IMI PhD Seminar
04/11/2013 NTU IMI PhD Seminar 4/30
~400mm Single Image 3D Action
Previews Works
Additional Sensors Light Touch (IR optical sensors)
Diamondtouch (capacitive sensor array)
Smartskin (mesh-shaped antenna)
Skinput (bio-acoustic sensing array)
LightSpace, Omnitouch (Kinect)
Computer Vision [Letessier2004] -- Fingertip tracking, not touching detection
[Kjeldsen2002, Hardenberg2001] -- Delay-based scheme
[Marshall2008] – Color change of the fingernail
[Song2007, PlayAnywhere2005] -- Shadow casted by finger
[Fitriani2007] -- Deformation on soft surface
04/11/2013 NTU IMI PhD Seminar 5/30
Overview
04/11/2013 NTU IMI PhD Seminar 6/30
Priors in Projector-Camera System
Geometric (Homography)
All these geometric priors can be derived through 2 projection-capture cycles in initialization stage.
04/11/2013 NTU IMI PhD Seminar 7/30
Embedding Codes into Video Projection
04/11/2013 8/30 NTU IMI PhD Seminar
R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs, “The office of the future”, SIGGRAPH98.
Embedded Pattern Design Strategy
Method Array Size Win. Size Alph. Length
[Morita 1988] 24 * 24 3 * 4 2
[Kiyasu 1995] 18 * 18 4 * 2 2
[Salvi 1998] 29 * 29 3 * 3 3
[Spoelder 2000] 65 * 63 2 * 3 2
[Albitar 2007] 27 * 29 3 * 3 3
[Desjardins 2007] 53 * 38 3 * 3 3
[Chen 2008] 82 * 82 3 * 3 7
04/11/2013 9/30
Summary of typical spatial coding methods
Constraints of Pattern Generation Code Uniqueness
Large Hamming Distance
NTU IMI PhD Seminar
Hand Segmentation
04/11/2013 10/30
A
A
B
C
C
NTU IMI PhD Seminar
Contrast Saliency
Region Discontinuity
Histogram-based Contrast Saliency
M. Cheng et al. Global contrast based salient region detection, CVPR2011.
04/11/2013 11/30 NTU IMI PhD Seminar
Mean-Shift Region Smoothing
04/11/2013 12/30 NTU IMI PhD Seminar
D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5):603–619, 2002.
Precise Segmentation by Fusing
04/11/2013 13/30
CF > Δ
NTU IMI PhD Seminar
Fingertip Detection
04/11/2013 NTU IMI PhD Seminar 14/30
Fingertip Candidates Detected Fingertips
Hand Contour Binary Hand Image
Touch Detection Through
Homography and Embedded Codes
04/11/2013 NTU IMI PhD Seminar 15/30
From Resistive Touch to Capacitive
Touch or Floating Touch
04/11/2013 16/30 NTU IMI PhD Seminar
Homology
Experiments – System Setup
04/11/2013 17/30 NTU IMI PhD Seminar
Experiments – System Initialization
Camera-Projector Homography Estimation
04/11/2013 18/30
Projected chessboard Captured image
NTU IMI PhD Seminar
Experiments – System Initialization
Camera-Table Homography Estimation
Four Correspondences: C1 ~ O, C2 ~ Px, C3 ~ Py, C4 ~ Pxy
04/11/2013 19/30
Credit Card Captured image
NTU IMI PhD Seminar
Experiments – Display Quality Evaluation
04/11/2013 20/30 NTU IMI PhD Seminar
Experiments – Hand Segmentation Accuracy
Evaluation
(a) Original Image
(b) Ground-truth
(c) Our Method
(d) SCM
(e) BkSub
(f) GB
04/11/2013 21/30 NTU IMI PhD Seminar
Experiments – Hand Segmentation Accuracy
Evaluation
04/11/2013 22/30 NTU IMI PhD Seminar
04/11/2013 NTU IMI PhD Seminar 23/30
Experiments – Touch Accuracy Evaluation
More HD videos http://cs.unc.edu/~dai/Research/TouchEverywhere/TouchEverywhere.htm
Experiments – Touch Accuracy Evaluation
04/11/2013
Comparison with recent depth-camera sensing based methods
In [Wilson 2010], the informal observed spatial error of finger detection on planar surface was between 3-6 pixels In [Omni-Touch 2011], the FRR and FAR of finger click detection on four different surfaces were 0.8% and 3.3%.
NTU IMI PhD Seminar 24/30
Experiments – Trajectory Tracking
Evaluation
04/11/2013 NTU IMI PhD Seminar 25/30
Experiments – Multi-Touch Evaluation
04/11/2013 NTU IMI PhD Seminar 26/30
Experiments – Efficiency Evaluation
04/11/2013 27/30
Subroutine Hand Seg.
FTip Loc.
Touch Det.
Total
Time (ms/frame) 14.63 1.32 1.74 17.69
NTU IMI PhD Seminar
Summary
Using only off-the-shelf devices
Precise hand segmentation in PROCAMS
Achieving 3D sensing without explicit 3D reconstruction
04/11/2013 NTU IMI PhD Seminar 28/30
04/11/2013 NTU IMI PhD Seminar
Related Publications Conference paper [1] J. Dai and R. Chung, Making Any Planar Surface into a Touch-sensitive Display by a Mere Projector and Camera, In Proc. of 9th IEEE International Workshop on Projector–Camera Systems (PROCAMS2012), pages 35-42, 2012.
[2] J. Dai and R. Chung, Combining Contrast Saliency and Region Discontinuity for Precise Hand Segmentation in Projector-Camera System, In Proc. of The 21st International Conference on Pattern Recognition (ICPR’12), pages 2161-2164, 2012.
Journal paper [3] J. Dai and R. Chung, Touchscreen Everywhere: On Transferring a Normal Planar Surface to a Touch-Sensitive Display, Submitted to IEEE Trans. on System, Man and Cybernetics (TSMC-B), November 2012. (under 2nd round review)
29/30
http://cs.unc.edu/~dai/Publications/Publications.htm
THANK YOU!!
04/11/2013 30/30 NTU IMI PhD Seminar