cvpr 2008 june 24 – 26, 2008 infrared camera: mid wave: 3.0-5.0 microns resolution: 640*512...
Post on 15-Jan-2016
214 views
TRANSCRIPT
CVPR 2008 June 24 – 26, 2008
Infrared camera: Mid wave: 3.0-5.0 microns Resolution: 640*512 pixels with 14bits Frame rate: 30/60/115 fps Sensitivity: about 25mKTo explore contact-free heart rate and
respiratory rate detection through measuring infrared light modulation emitted near superficial blood vessels or a nasal area.
Ming Yang2, Qiong Liu1, Thea Turner1, Ying Wu2
1 FX Palo Alto Laboratory, Inc., 3400 Hillview Ave., Palo Alto, CA 94304 2 Dept. of EECS, Northwestern Univ., 2145 Sheridan Rd., Evanston, IL 60208
Vital Sign Estimation from Passive Thermal Video
Experiments
Overview of our approach
Ground truth: ADI PowerLab 4/30
Test dataset: Age 20-60, F:8 and M:12
20 subjects for heart rate estimation
7 subjects for respiratory rate estimation
Accurate subject alignment for temporal signal extraction, e.g. involuntary muscular movements are inevitable.
Robust harmonic analysis with low signal-to-noise ratio (SNR) temperature modulation signal, e.g. modulation magnitude 0.1K vs. camera sensitivity 0.025K.
A novel contact-free vital sign measurement method.
Low risk of harm & convenience for quick deployment.
Potential applications: airport heath screening, long-term elder care, workplace preventive care, etc.
N. Sun, M.Garbey, A. Merla, I. Pavlidis. Imaging the cardiovascular pulse. CVPR 2005. (S)
S.Y. Chekmenev, A.A. Farag, E.A. Essock. Multiresolution approach for non-contact measurements of arterial pulse using thermal imaging. CVPR 2006 Workshop.
Goal
Motivations
Challenges
Reference frame
t = 1/ 60 s t = 25/ 60 s t = 26/ 60 s t = 27/ 60 s t = 58/ 60 s t = 59/ 60 st = 0/ 60 s
Frame differences
Pioneering work
Automatic ROI segmentation and alignment
Signal enhancement and outlier removal
Robust harmonic analysis
Vote
Dominant frequency componentRough manualinitialization
Isothermextraction
Optimal thresholdselection
3Dvisualization
Region-of-interests segmentation by thresholding the isotherms and alignment by contour tracking.
Signal enhancement using a non-linear filter, and outlier removal by pixels-of-interests clustering.
Robust harmonic analysis by dominant frequency voting. Subject alignment and
motion compensation
Thermal video captured byan infrared camera
Signal enhancementand denoising
Robust harmonicanalysis
ROIsegmentation
Contourtracking
Pixel ofinterests
clustering
Medianfiltering
Non-linearfiltering
Dominantfrequency voting
time
bpm
( , ) [ ( ) ( , )]j jH f F W t s tx x
Perform N-point (N=1024/2048/4096) FFT of all temperature signals of all pixels using a sliding window:
Non-linear filtering by taking the point-by-point minimum of a rectangle window Wr(t) and a Hamming window Wh(t)
Cluster H(xj, f ) in the band of interest (40-100 bpm for heart rates, and 6-30 bpm for respiratory rates) using K-means, then select the largest cluster to estimate.
( , ) min( [ ( ) ( , )], [ ( ) ( , )])j r j h jH f F W t s t F W t s tx x x
Subject #
fps # of frames
GT bpm
Est. bpm
Diff.
4 60 3000 18 15.8 -2.27 60 3000 17 15.1 -1.9
10 115 5000 11 11.8 +0.811 115 5000 17 16.8 -0.214 115 5000 16 13.9 -2.115 115 5000 15 13.1 -1.917 115 5000 20 18.5 -1.519 115 5000 16 15.2 -0.8
Segment the initial ROI by selecting the isotherm with the sharpest gradient.
Align the ROI by tracking the contour
Extract the temporal signals for individual pixels inside the ROI and denote by
2* 2
( ) ( ( ) ( ))
|| ( ) || || || , 0.001
t
t
t image t ext t
jt j j
E E E
I I
x x x
Respiratory rate estimation results
Heart rate estimation results
Point-by-point comparisonsSubject
#fps # of
framesGT
bpmEst. bpm
Diff. RMSE
1 30 2000 65.3 65.8 +0.5 1.92 30 2000 66.6 63.9 -2.7 3.93 30 1750 65.7 64.7 -1.0 3.34 60 3000 59.8 60.7 +0.9 2.55 60 3500 60.7 60.3 -0.4 3.36 60 2500 66.3 53.0 -3.3 3.97 60 3000 61.1 60.9 -0.2 2.38 115 5000 64.0 65.0 +1.0 3.89 115 5000 78.9 80.1 +1.2 1.9
10 115 5000 65.2 64.4 -0.8 1.711 115 5000 62.8 66.2 +3.4 4.212 115 5000 63.5 62.4 -1.1 3.213 115 5000 73.3 72.6 -0.7 1.814 115 5000 86.6 87.9 +1.3 4.915 115 5000 78.7 76.5 -2.2 3.116 115 5000 75.3 74.7 -0.7 1.917 115 5000 83.1 83.2 +0.1 2.118 115 5000 67.2 68.2 -1.0 1.319 115 5000 67.6 69.3 +1.7 2.820 115 5000 68.7 70.1 +1.4 2.9
The initial ROI segmentation results
( , )js tx
Insensitive to initialization and robust to gentle subject movement and facial expressions.
More stable estimation results compared with the state-of-the-art methods.
Conclusions
t