range imaging from calibration to modeling - tu wien · range imaging from calibration to modeling...
TRANSCRIPT
Range Imaging
From Calibration to Modeling
Norbert Pfeifer, Sajid Ghuffar, Willi Karel,
Camillo Ressl, Stefan Niedermayr
Institute of Photogrammetry and Remote Sensing
Vienna University of Technology
Why range cameras ?
Background 1 is Photogrammetry
• Passive brightness images, stills
• Object reconstruction by intersection of multiple rays to corresponding points
• Accuracy 1:10.000 – 1:100.000 (++)
• Resolution: diffraction limited
Background 2 is Laser Scanning
• Active ranging, hemispherical, sequential
• Object reconstruction from multiple scans for complex objects
• Accuracy 1:10.000 – 1:100.000
• Beam divergence near diffraction limit, sampling distance function of time
Range Imaging
• Active range imaging, in comparatively narrow FoV
• Accuracy 1:100 – 1:1.000
• High frame rate
Geodata collection
• Geometric parameters of urban furniture/vegetation
e.g.: tree height, DBH, height of first living branch
• Enrichment of existing GeoDB
• Object static, camera dynamic
• Advantage: Ease of use, automating different manual procedures
See also: B. Jutzi, KIT, Karlsruhe
Building interiors, 3D BIM
• „As used“ documentation of buildings: 3D models of rooms
• Ontop floorplan backbone
• Object static, camera dynamic
• Advantage: Very high automation possible, cost-effective
See also: J. Böhn, UCL, London
GEO domain applications 1/2
Fast geomorphological processes
• Landslide, mudslide, embankment failures
• Boundary values for better process understanding
• Object dynamic, camera static
• Advantage: range imaging as enabling technology
Traffic monitoring
• Cars, trains, etc.
• Objects dynamic, camera static (or on moving platform)
• Advntage: high reliability
See also: many others
GEO domain applications 2/2
From calibration to modeling – Presentation outline
Scattering compensation Laboratory
Calibration Laboratory
Orientation Real scenes under advantageous conditions
Modeling Real scenes under advantageous conditions
Overall aim (medium term goal)
Scattering compensation Lab measurements/not required(?)
Calibration model identification Lab measurements
Calibration parameters + orientation Real scenes, on-the-job
Modeling Real scenes
Scattering
Echo of the emitted optical signal is scattered to some extent over the sensor due to multiple reflections within the camera:
• i.e. between the lens, the optical filter, and the sensor
• “Lens Flare” in conventional photogrammetry
Distance distortions for far away objects can be up to 1m or greater
Ongoing instrumental developments, but effects also reported in new generation (Lichti, UCalgary)
Modelling the Point Spread Function
Experimental Setup: point like target in foreground scatters light over
darker background
Investigating the depedence of PSF on the parameters:
• Target size
• Target distance
• Target position
• Integration time
PSF from deconvolution
Spatially variant Point Spread Function
The intensity of scattering distortion changes with angle and distance
from the principal point
Shape of the scattering halo remains constant
[m]
Evaluating the PSF
Applying the correction model to real test scenes
Compensating for distance and amplitude distortion in a combine
deconvolution algorithm
Overall error reduced, but compensation partly overshootes
Calibration in Lab, handheld movies
Reference plane with control points
• Avoids scattering and object space multi path
• Very simple object model
Spatial resection only from amplitude image
Exterior and interior orienation estimated
for each frame
Target tracking fully automated
Significant intrinsic parameters
• Principal point coordinates
• Focal length
• Radial distortion of 3rd order
• Principle point depends on camera attitude
Calibration in Lab, handheld movies
Reference plane with control points
Spatial resection
Significant intrinsic parameters
Systematic error = range observation – reference range
Advantage: very large data volume (e.g. 6000 frames)
Disadvantage
• Low amplitudes for large distances
• Motion blur for large distances
Calibration in Lab – stills
Larger distances due to stills (max. range)
Experimental environment similar
• Control points cover larger volume
Target identification automated
• But computationally more intensive
Number of frames smaller (e.g. 850)
• But better SNR
amplitudes – observed range – range std.dev. – systematic range error
Calibration results
Harmonic range errors according to modulation wavelength (10cm span)
Hyperbolic-type range error for amplitudes (20cm span)
Range error as function of position in sensor plane (50cm span)
Range error increases with integration time (3cm span)
0 1 2 3 4 5 6 7 8-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
Observed dist. [m]
Der.
min
us o
bs.
dis
t. [
m]
Der. minus obs. dist. corr. 4 all but obs. dist. , corr. model
offset,d1,d2s,d2c,d3s,d3c,A1,iT1,Row1,Row2,Col1,Col2,RowCol2 / origObs
0 1 2 3 4 5 6 7 80
2
4
6
8
10
12x 10
4
count
0 2000 4000 6000 8000 10000 12000 14000 16000 18000-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
Amplitude []
Der.
min
us o
bs.
dis
t. [
m]
Der. minus obs. dist. corr. 4 all but amplitude , corr. model
offset,d1,d2s,d2c,d3s,d3c,A1,iT1,Row1,Row2,Col1,Col2,RowCol2 / origObs
0 2000 4000 6000 8000 10000 12000 14000 16000 180000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5x 10
5
count
10 30 60 90 120 150 180 210 255-0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
int. time []
der.
min
us o
bs.
dis
t. [
m]
Der. minus obs. dist. corr. 4 all but int. time , corr. model
offset,d1,d2s,d2c,d3s,d3c,A1,iT1,Row1,Row2,Col1,Col2,RowCol2 / origObs
Orientation and Modeling
Example 1
• Handheld movie
• Focus on orienation
• Weak object modeling to support orientation
• Scene requirements: containing planes (for noise reduction)
Example 2
• Simulation of range cameras with improved accuracy
• Focus on modeling
• Orienatition solved as necessary task for modeling extended scenes
• Scene requirements: brightness differences in object
Orientation (and Modeling)
Scence contains large objects with simple geometric description (planes)
Objects found automatically by (video) segmentation
Subsequent scenes of movie transformed onto each other
• Scene n: described as set of planes
• Scene n+1: select „plane segment points“ close to planes of scene n
• Euclidean transformation of (n+1) to (n) [ ICP-type orientation with filters ]
Orientation (and Modeling)
Video Segmentation
Orientation (and Modeling)
Transformation of new to previous scene(s)
(Orientation and) Modeling
Video data acquisition including still scenes (tripod)
• Similarity between stills ~ 75% (rotation by ¾ FoV)
Noise reduction in still sequences by averaging
Point correspondence between images throught SIFT
Orientation
• Using direction and range observations
• Tracking points through multiple images
• Global orientation in the final stage
Modeling as subsequent step
(Orientation and) Modeling
Orientation fully automated
Amplitudes Object space in camera view
(Orientation and) Modeling
Modeling
Filter erroneous points (low intensity, very short distances, corona points)
Triangulate (Geomagic), smooth triangulation, automated hole filling
(Orientation and) Modeling
Residuals of
selected points
vs. smoothed
triangulation
up to 5cm
Conclusions 1/2
Scattering
• Instrumental developments reduce problems
• Because range observations are used directly,
complete removal remains a research topic
On the job calibration
• Feasible
• Lab environment
• Reduction of systematic errors by 50-90%
Orienation
• Feasible for hand held movies
• Scene requirements in examples (planes or markers or noise reduction)
Modeling
• Random noise reduction necessary
• Tight integration with calibration and orientation seems possible
EO device may become necessary (MEMS IMU, etc.)
Outlook 2/2
Ongoing projects
• Acclimatization (MESA and PMDTec)
• Warm-Up and other influcences (MESA, together with Lichti)
• Orientation without scence requirements
Dreams and Wishes
1. Increase device stability (weight!)
2. Increase insensitivty to background light: outdoor applications
3. Increase resolution
4. Increase maximum range
5. Increase accuracy