automatic registration of color images to 3d geometry computer graphics international 2009 yunzhen...
TRANSCRIPT
Automatic Registration of Color Images to 3D Geometry
Computer Graphics International 2009
Yunzhen Li and Kok-Lim Low
School of ComputingNational University of Singapore
* Presented by Binh-Son Hua
Problem StatementRange images
Color images from untracked camera
. . .
3D model Colored 3D model
Automatically register color images to 3D model
2
MotivationsApplications of active range sensing
Manufacturing, cultural heritage modeling, etc.Photometric properties needed for visually-
realistic modelsOnly some range scanners can capture colorColor may not have required resolution
E.g. for close-up or zoomed-in views of paintingsView-dependent reflection requires many color
images from different directionsTherefore, better to capture color separately
However, impractical to manually register color images to 3D geometry
3
Previous WorkFeature-based approaches
Match corresponding features in both color images and 3D model
Can be fully automatedRestricted to certain types of objects[Stamos & Allen, ICCV 2001], [Liu & Stamos, CVPR 2005]
Statistics-based approachesUsed only if reflected intensities of range sensing light
were recorded with range dataSensing light often not in visible light spectrum
Compute statistical dependence between color images and sensing light intensitiesMutual information, chi-square, cross-correlation
Camera calibrated & tracked, or co-locate with scanner[Williams et al, 2004], [Hantak & Lastra, 3DPVT 2006]
4
Our ApproachColor images
. . .
Detailed scanned 3D model
Colored 3D model
Color mapping
Registration
Multiview geometry
reconstruction
Sparse 3D model
5
Steps
1. Data acquisition
2. Multiview geometry reconstruction
3. Approximate registration of sparse model to detailed model
4. Registration refinement
5. Color mapping
6
1. Data AcquisitionRange data
Laser range scanner
Color images Uncalibrated and untracked
digital cameraProject special light pattern
on large textureless surfacesImprove image feature
detection and MVG reconstruction
7
2. MVG ReconstructionDetect and match features in color images
Use SIFT
Compute MVGStructure-from-motionIncrementally add a new image and apply
sparse bundle adjustment (SBA)
Result is a sparse 3D model3D point cloudCamera parameters
8
2. MVG ReconstructionExample sparse 3D model
9
3. Approximate RegistrationTo align sparse model with detailed model
Unknown relative scale and poseRegister one image in MVG to 3D model
User input 6 point correspondencesEstimated transformation propagated to other
views and 3D points in MVGSparse model only approximately aligned to
detailed modelError in user inputsError in MVGGeometric distortion in detailed model
10
4. Registration RefinementNeed non-rigid alignment of MVG with detailed model
To overcome geometric distortion in range images
Registration refinementAutomatically detect planes in detailed modelIdentify 3D points in MVG near the planesRefine MVG to minimize distance
between 3D points and planesEasily incorporated into
sparse bundle adjustment
Better than using ICP algorithmTwo models are treated as rigid shapesCannot refine MVG
11
4. Registration RefinementExample result
12
Before registration refinement
Afterregistration refinement
5. Color MappingColors from different views can be used for
view-dependent renderingView-dependent texture mappingSurface light field
We simply want to assign a single color to each surface point, butSimple averaging blurs out detailsDifferent exposuresOcclusionsDepth boundariesVignetting and view-dependent reflection
13
5. Color MappingUse weighted blending
Use lower weights near image and depth boundaries
Preserve fine details
14
With details
preservation
Without details
preservation
5. Color MappingSmooth color and intensity transitions
With weighted blending
Without weighted blending
15
ResultOffice scene
30 color images (7 with projected pattern)
16
ConclusionAchieve accuracies within 3–5 pixels
everywhere on each imageNot reliant on detection of any specific type of
features in both color images and geometric model
Project light pattern to improve robustness of MVG
Better registration accuracy in face of geometric distortion
Effective color mapping method
17
AcknowledgementsThe Photo Tourism team
For sharing part of their code on MVGPrashast Khandelwal
For contribution to preliminary workSingapore Ministry of Education
For the funding
18