robust feature point matching ohne.ppt [kompatibilitätsmodus]

Robust Feature Point MatchingRobust Feature Point MatchingRobust Feature Point Matching Robust Feature Point Matching in General Multiin General Multi--Image SetupsImage Setups

WSCG 2011

A i S ll M i Ei M MAnita Sellent, Martin Eisemann, Marcus MagnorTU Braunschweig, Germany

GeneralGeneral Sets of ImagesSets of ImagesUnsynchronized => no epipolar geometry

GeneralGeneral Sets of ImagesSets of Images

Unordered => no trackingCommon field of view

2Robust Feature Point Matching Anita Sellent

Consistent Feature MatchingConsistent Feature MatchingConsistent Feature MatchingConsistent Feature MatchingSparse featuresFeatures correspond to 3D points => consistency


Three Image Optical FlowThree Image Optical FlowDense correspondences

Three Image Optical FlowThree Image Optical Flow

High accuracyBrightness constancySmall displacements


[Sellent et al.: Consistent Optical Flow for Stereo Video, ICIP 2010]

OutlineOutlineOutlineOutlineThree Image Feature Matchingg gCombination of Feature Matching and Optical Flow


OutlineOutlineOutlineOutlineThree Image Feature MatchingCombination of Feature Matching and Optical Flow


Three Image MatchingThree Image MatchingThree Image MatchingThree Image MatchingUnordered, unsynchronized imagesCommon field of viewAny features (SIFT, SURF, FAST, Harris,…)


Three Dimensional Assignment Three Dimensional Assignment P blP blProblemProblemSets of equal, finite cardinalityNP-hardApproximation algorithms exist[Crama, Spieksma:Approximation algorithms for the 3 dimensional assignment problem, 1992]


Step 1Step 1Step 1Step 1(Robust) nearest neighbor matchingSymmetry

3I

1I 2I


Step 2+3Step 2+3Step 2+3Step 2+3Combine distance function

S

),(),(),(ˆ323131 ffdffdffd +=

Symmetry3I

1I 2I


Step 4Step 4Step 4Step 4Restart from all directions


Step 5Step 5Step 5Step 5Only accept consistent solutions


Results: Scene Art with SIFT Results: Scene Art with SIFT FFFeaturesFeatures

Two Image Matching Three Image MatchingTwo Image Matching1444 Matches53 % wrong matches

Three Image Matching603 Matches11 % wrong matches


Removed 703 wrong matches at the cost of 138 correct matches

Results: Scene Laundry with Results: Scene Laundry with SURF FSURF FSURF FeaturesSURF Features





Results: Scene Stonemill with Results: Scene Stonemill with H i CH i CHarris CornersHarris Corners





Results: Real World Scenes Results: Real World Scenes i h SIFT Fi h SIFT Fwith SIFT Featureswith SIFT Features

Two Image Matching Three Image MatchingTwo Image Matching Three Image Matching


Three Image Feature MatchingThree Image Feature MatchingThree Image Feature MatchingThree Image Feature MatchingNo prior knowledge requiredMatch features consistently on three imagesTrade off number of matches for increased quality of the matchesSparse


OutlineOutlineOutlineOutlineThree Image Feature MatchingCombination of Feature Matching and Optical Flow


Combination of Feature Combination of Feature M hi d O i l FlM hi d O i l FlMatching and Optical FlowMatching and Optical Flow

Features Optical Flow◦ Sparse◦ Low accuracy

Optical Flow◦ Dense◦ High accuracyy

◦ Large motion◦ Invariant to color

g y◦ Small motion◦ Based on brightness

changes◦ …

gconstancy ◦ …


Feature Matches as Motion Feature Matches as Motion P iP iPriorPriorDefine motion prior only where )(xPmatches are foundAssign weight proportional to )(xμmatching costAdd weighted motion prior to optical flow d t tdata-term


Results: Results: A E d i EA E d i EAverage Endpoint ErrorAverage Endpoint Error

2 image OF 2 image OF &2 image features


Stonemill 4 53 23 16 3 81 3 52Stonemill 4.53 23.16 3.81 3.52Waving 1.03 31.39 0.97 0.92Art 10.62 84.82 9.34 8.70Books 14.60 55.37 6.43 4.85


[Sellent et al.: Consistent Optical Flow for Stereo Video, ICIP 2010]

Flow Fields: Scene StonemillFlow Fields: Scene StonemillFlow Fields: Scene StonemillFlow Fields: Scene Stonemill

Ground-truth


3 image OF &3 image features

3 image OF


Flow Fields: Scene WavingFlow Fields: Scene WavingFlow Fields: Scene WavingFlow Fields: Scene WavingGround-truth


3 image OF &3 image features

3 image OF


SummarySummarySummarySummaryConsistency on three images can improve sparse and dense correspondences without further knowledge

N lib i◦ No calibration◦ No synchronization

M lti l di i◦ Multiple dimensionsFor high quality feature matches the accuracy of optical flow can be improved byaccuracy of optical flow can be improved by the inclusion of features


Future WorkFuture WorkFuture WorkFuture WorkWhat is the best number of images to be used?◦ Number of visible/ detected features◦ Quality of the matching


Thank You for Your Attention!Thank You for Your Attention!Thank You for Your Attention!Thank You for Your Attention!

http://graphics.tu-bs.de

This work has been funded by the German Science Foundation


This work has been funded by the German Science Foundation, DFG MA2555/4-1

Consistent Optical FlowConsistent Optical FlowConsistent Optical FlowConsistent Optical Flow

Brightness ConstancyBrightness Constancy)()( 2,121 wxIxI +≈

0ˆ22,12,1 ≈−ww

Smoothness Assumption

0ˆ 2,1

r≈∇w

Estimate dense flow between 3 images simultaneously


[Sellent et al.: Consistent Optical Flow for Stereo Video, 2010]

Estimate dense flow between 3 images simultaneously

Synchronized CamerasSynchronized CamerasSynchronized CamerasSynchronized Camerassp

ace

time


[Hartley, Zisserman: Multiple View Geometry in Computer Vision, 2003]

Unsynchronized CamerasUnsynchronized CamerasUnsynchronized CamerasUnsynchronized Camerassp

ace

time

[Veenman, Reinders, Backer: Establishing Motion Correspondence


Using Extended Temporal Scope, 2003]

robust feature point matching ohne.ppt [kompatibilitätsmodus]

Documents