depth from structured light ii: error analysis michael h. rosenthal april 19th, 2000

Depth from Structured Light II:Error Analysis

Michael H. Rosenthal

April 19th, 2000

4-19-00 Rosenthal - 2

Outline

• Review of Structured Light Projection

• Error in Ideal Model

• Lighting and Surface Property Errors

• Stripe Assignment Errors

• Motion Errors

• Structured Light in Practice


Structured Light Projection

• Camera and projector are calibrated

• Light stripes are projected onto scene

• Depth for each pixel is measured by intersecting pixel ray with stripe plane


Binary Stripe Encoding

• Pixels are “labeled” using binary encoding: light = 1, dark = 0

• High-order bits encode broad regions, low-order bits encode fine location


Error in Ideal Model



• Pixels and stripes have finite size

• All visible surfaces within the pixel/stripe intersection volume are sampled and lumped together



• Size of sampled volume depends upon stripe width, pixel width and projector orientation

• Corollary: depth resolution is controlled by the same factors


Error from Lighting and Reflection


Lighting and Reflection

• Prior assumptions:– Bright Lambertian

surface

– Path from projector to surface to camera

• Real world– Dark or shiny materials

(apples, metal, cloth)

– Things in your way Specular highlights interfere with structured light patterns



Preventive Methods:

• Capture images with projector fully on and fully off

• Use difference to measure intensity range for each pixel

• Threshold difference to find to find trouble spots

Images from structured light projected into a simple scene



•Minimum intensity image may have bright spots - specular highlights

•Maximum intensity image may have dark spots - shadows or dark objects



•Difference image shows range of intensity between on and off

•Narrow range corresponds to non-ideal behavior

•Threshold to identify trouble spots

•Exclude trouble spots from final depth image



• Other practical solutions:– Lambertian coatings

and sprays (talc)

– High dynamic range cameras

– Variable exposure times


Error in Stripe Assignment


Stripe Assignment Errors

• Stripe encoding requires binary decision for each pixel

• What happens if a pixel is misassigned to the wrong stripe (i.e. gets a 0 rather than a 1 when illuminated)?

Images from structured light projected into a simple scene



• Caused by random noise, low contrast, motion, edge uncertainty, etc.

• Magnitude depends upon significance of bit!

• 1st bit yields error N/2, 2nd bit yields error N/4, kth bit yields error N/2k

Binary stripe encoding (phase shifted method)



• Assume that probability of misassignment is p• Expected error from kth bit is p*N/2k • Total expected error E = p*N/2 + p * N/4 +… + p

* N/2k + … + p*N/2logN

• E = p*N*(1- 1/2logN)• For large N, E ~ p*N• For < 1% expected error, we must have < 1%

chance of misassignment


Error from Motion


Motion Errors

• What errors can motion cause in a structured light system?

• Two components of motion:– Along a pixel ray

– Pixel to pixel Moving object in a structured light system


Pixel to Pixel Motion

• Similar to stripe assignment error - “random” bits are flipped in affected pixels

Moving object in a structured light system


Motion on a Pixel Ray

• Motion along a pixel ray causes different stripes to be reflected over time

• Final pixel code is composed of bits from different positions

Motion along a pixel ray traverses projected stripes over time



• Magnitude of error depends upon initial position and velocity

• Very large errors near edges in broad stripes (high order bits)



Absolute error relative to final object position - note the chaotic dependence upon position and velocity


Structured Light in Practice

• Office of the Future uses it for surface measurement

• Augmented Reality Biopsy project has tested it for skin surface measurement and for internal surgery

• Videos!


Conclusions

• Four major classes of error in structured light:– Error intrinsic to ideal system - affects spatial resolution

– Lighting and reflection error - some regions of the scene will be unmeasurable

– Stripe assignment error - need reliable methods for binary categorization

– Motion error - yields large, unpredictable errors

• Despite these problems, structured light is useful and widely available

depth from structured light ii: error analysis michael h. rosenthal april 19th, 2000

Documents

structured light systempixel

structured light systemmotion

pixel width

pixel raymotion

pixel motionsimilar

pixel raypixel

error n2kbinary stripe

error n4