3d multi-view reconstruction young min kim karen zhu cs 223b march 17, 2008
Post on 19-Dec-2015
222 views
TRANSCRIPT
![Page 1: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/1.jpg)
3D Multi-view 3D Multi-view ReconstructionReconstruction
Young Min KimYoung Min KimKaren ZhuKaren ZhuCS 223BCS 223B
March 17, 2008March 17, 2008
![Page 2: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/2.jpg)
OutlineOutline
ProblemProblem
Data SetData Set
MRFMRF
Noise ReductionNoise Reduction
Multi-view ReconstructionMulti-view Reconstruction
ConclusionConclusion
![Page 3: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/3.jpg)
ProblemProblem
Create broad-view high-resolution 3D viewCreate broad-view high-resolution 3D view
3D View
Normal Camera
Depth Camera
![Page 4: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/4.jpg)
Data SetData Set
![Page 5: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/5.jpg)
MRFMRF
Single view super-resolution reconstructionSingle view super-resolution reconstructionObjective function: E=Ed+EcObjective function: E=Ed+Ec Ed: Similarity between the up-sampled depth and the Ed: Similarity between the up-sampled depth and the
depth sensor measurementdepth sensor measurement Ec: Regions with similar color have similar depthEc: Regions with similar color have similar depth
mrfDepthSmooth code from Stephen Gould mrfDepthSmooth code from Stephen Gould
[1] James Diebel, Sebastian Thrun, “An Application of Markov Random Fields [1] James Diebel, Sebastian Thrun, “An Application of Markov Random Fields to Range Sensing”, to Range Sensing”, Proceedings of Conference on Neural Information Proceedings of Conference on Neural Information
Processing Systems (NIPS)Processing Systems (NIPS), MIT Press, Cambridge, MA, 2005., MIT Press, Cambridge, MA, 2005.
![Page 6: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/6.jpg)
MRF: ResultMRF: Result
Original depth mapOriginal depth map MRF resultMRF result
![Page 7: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/7.jpg)
Noise ReductionNoise Reduction
Single-view ImprovementSingle-view Improvement Median filteringMedian filtering Occlusion boundary removalOcclusion boundary removal
Original depth image Median filtered depth image
![Page 8: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/8.jpg)
Original MRF
Median filtered Occlusion boundary removed
![Page 9: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/9.jpg)
Multi-view ReconstructionMulti-view Reconstruction
Problem: misalignment
![Page 10: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/10.jpg)
Multi-view ReconstructionMulti-view Reconstruction
New Objective New Objective function using multi-function using multi-view information: view information: E=Ed+Ec+EmE=Ed+Ec+Em
Em: similarity Em: similarity between depth in two between depth in two different viewdifferent view
![Page 11: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/11.jpg)
Multi-view ReconstructionMulti-view Reconstruction
![Page 12: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/12.jpg)
Multi-view Reconstruction: ResultMulti-view Reconstruction: Result
![Page 13: 3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008](https://reader030.vdocuments.mx/reader030/viewer/2022033103/56649d3a5503460f94a1482e/html5/thumbnails/13.jpg)
ConclusionConclusion
Median filter is effective in removing Median filter is effective in removing sensor noisesensor noise
Removing occlusion boundary reduce Removing occlusion boundary reduce noise due to motionnoise due to motion
By using information from multi-view, By using information from multi-view, depth images are better aligneddepth images are better aligned