multi-view thin structure reconstruction via 3d lines and points...

1
Multi-View Thin Structure Reconstruction via 3D Lines and Points (#12) Montiel Abello [mabello] | Sudharshan Suresh [sudhars1] Introduction and Motivation Results Future Work Line-based 3D Model Global SfM Dense Reconstruction From the SfM output, we get the ordered image sequence sparse point cloud and visibility information. We use Line3D++, a pipeline to abstract 3D scene information using line segments. We run a line segment detector on every image, establish correspondences across images with supporting views. Generate 3D hypothesis based on plausible correspondences; cluster the 2D segments based on them. Refine estimate to get final 3D line model. Scenes with thin structures and edges are poorly reconstructed by multi-view stereo algorithms, due to appearance variation. We propose the integration of line segments into the SfM point-cloud via a radial sampling technique. We then perform dense reconstruction and mesh texturing on this combined point-cloud. Results highlight the recovery of thin structures, along with enhanced visibility of edges. We use openMVS to convert our sparse point cloud to a dense 3D mesh. The components of the pipeline are: Densify the sparse point cloud with patch match algorithm Use visibility information in a graph-cut algorithm to reconstruct a mesh from the densified points Refine the mesh by subdividing triangles based on size and visibility information Texture the mesh using the original images Fitting splines to clustered line segments will better model curved surfaces Separate treatment of thin and regular structures will allow for optimum parameter tuning during dense reconstruction Evaluation on scenes with objects of varying shape and size To generate a sparse model of the 3D structure of the scene, we use the SfM library from openMVG. Given an unordered set of images with camera intrinsics specified, openMVG estimates the global pose of each camera, and global position of features points openMVG optimizes the following cost function: where is the image coordinate of the j th feature point in the i th camera. Thin Structure Sampling Strategy Pipeline [1] Moulon, Pierre, Pascal Monasse, and Renaud Marlet. "Global fusion of relative motions for robust, accurate and scalable structure from motion." Proceedings of the IEEE ICCV. 2013. [2] Manuel Hofer, Michael Maurer, and Horst Bischof. Line3d: Efficient 3d scene abstraction for the built environment. In German Conference on Pattern Recognition, pages 237–248. Springer, 2015. [3] Jancosek, Michal, and Tomas Pajdla. "Exploiting visibility information in surface reconstruction to preserve weakly supported surfaces." International scholarly research notices 2014 (2014). Figure 2. Image of scene and optimized sparse 3D points We evaluate our method on our nsh_a_floor dataset. The reconstructions recover many structures and edges that are lost in the standard pipeline. Figure 1. Incorporating thin structure modeling into a multi-view reconstruction pipeline Figure 3. Optimized 3D line segments For each line segment represented by points --- and--- , we sample------ round-------- points on the cylinder of radius-- around the segment, where----------------------- and--- is a scaling factor. We do this by sampling-----angles------------------ and---- lengths----- and computing the corresponding 3D points.------- Figure 4. Points sampled around line segment Acknowledgement: We thank Zimo Li for his advice with the project. Figure 6. Final textured meshes from the (top) standard pipeline and (bottom) our pipeline. We improve reconstruction of thin structures - we recover the end of the object, along with a better reconstructed frame. Figure 6. Line features are matched more robustly, leading to improved coverage and alignment in the final reconstruction

Upload: others

Post on 26-Jan-2021

1 views

Category:

Documents


0 download

TRANSCRIPT

  • Poster Print Size:This poster template is 36” high by 48” wide. It can be used to print any poster with a 3:4 aspect ratio.

    Placeholders:The various elements included in this poster are ones we often see in medical, research, and scientific posters. Feel free to edit, move, add, and delete items, or change the layout to suit your needs. Always check with your conference organizer for specific requirements.

    Image Quality:You can place digital photos or logo art in your poster file by selecting the Insert, Picture command, or by using standard copy & paste. For best results, all graphic elements should be at least 150-200 pixels per inch in their final printed size. For instance, a 1600 x 1200 pixel photo will usually look fine up to 8“-10” wide on your printed poster.

    To preview the print quality of images, select a magnification of 100% when previewing your poster. This will give you a good idea of what it will look like in print. If you are laying out a large poster and using half-scale dimensions, be sure to preview your graphics at 200% to see them at their final printed size.

    Please note that graphics from websites (such as the logo on your hospital's or university's home page) will only be 72dpi and not suitable for printing.

    [This sidebar area does not print.]

    Change Color Theme:This template is designed to use the built-in color themes in the newer versions of PowerPoint.

    To change the color theme, select the Design tab, then select the Colors drop-down list.

    The default color theme for this template is “Office”, so you can always return to that after trying some of the alternatives.

    Printing Your Poster:Once your poster file is ready, visit www.genigraphics.com to order a high-quality, affordable poster print. Every order receives a free design review and we can deliver as fast as next business day within the US and Canada.

    Genigraphics® has been producing output from PowerPoint® longer than anyone in the industry; dating back to when we helped Microsoft® design the PowerPoint® software.

    US and Canada: 1-800-790-4001

    Email: [email protected]

    [This sidebar area does not print.]

    Multi-View Thin Structure Reconstruction via 3D Lines and Points (#12)

    Montiel Abello [mabello] | Sudharshan Suresh [sudhars1]

    Introduction and Motivation Results

    Future Work

    Line-based 3D Model

    Global SfM

    Dense Reconstruction

    ● From the SfM output, we get the ordered image sequence sparse point cloud and visibility information.

    ● We use Line3D++, a pipeline to abstract 3D scene information using line segments.

    ● We run a line segment detector on every image, establish correspondences across images with supporting views.

    ● Generate 3D hypothesis based on plausible correspondences; cluster the 2D segments based on them.

    ● Refine estimate to get final 3D line model.

    ● Scenes with thin structures and edges are poorly reconstructed by multi-view stereo algorithms, due to appearance variation.

    ● We propose the integration of line segments into the SfM point-cloud via a radial sampling technique.

    ● We then perform dense reconstruction and mesh texturing on this combined point-cloud.

    ● Results highlight the recovery of thin structures, along with enhanced visibility of edges.

    We use openMVS to convert our sparse point cloud to a dense 3D mesh. The components of the pipeline are:● Densify the sparse point cloud with patch match algorithm● Use visibility information in a graph-cut algorithm to

    reconstruct a mesh from the densified points● Refine the mesh by subdividing triangles based on size and

    visibility information● Texture the mesh using the original images

    ● Fitting splines to clustered line segments will better model curved surfaces

    ● Separate treatment of thin and regular structures will allow for optimum parameter tuning during dense reconstruction

    ● Evaluation on scenes with objects of varying shape and size

    ● To generate a sparse model of the 3D structure of the scene, we use the SfM library from openMVG.

    ● Given an unordered set of images with camera intrinsics specified, openMVG estimates the global pose of each camera, and global position of features points

    ● openMVG optimizes the following cost function:

    where is the image coordinate of the jth feature point in the ith camera.

    Thin Structure Sampling Strategy

    Pipeline

    [1] Moulon, Pierre, Pascal Monasse, and Renaud Marlet. "Global fusion of relative motions for robust, accurate and scalable structure from motion." Proceedings of the IEEE ICCV. 2013.[2] Manuel Hofer, Michael Maurer, and Horst Bischof. Line3d: Efficient 3d scene abstraction for the built environment. In German Conference on Pattern Recognition, pages 237–248. Springer, 2015.[3] Jancosek, Michal, and Tomas Pajdla. "Exploiting visibility information in surface reconstruction to preserve weakly supported surfaces." International scholarly research notices 2014 (2014).

    Figure 2. Image of scene and optimized sparse 3D points

    We evaluate our method on our nsh_a_floor dataset. The reconstructions recover many structures and edges that are lost in the standard pipeline.

    Figure 1. Incorporating thin structure modeling into a multi-view reconstruction pipeline

    Figure 3. Optimized 3D line segments

    ● For each line segment represented by points--- and--- , we sample------ round-------- points on the cylinder of radius-- around the segment, where----------------------- and--- is a scaling factor.

    ● We do this by sampling-----angles------------------ and---- lengths----- and computing the corresponding 3D points.-------

    Figure 4. Points sampled around line segment

    Acknowledgement: We thank Zimo Li for his advice with the project.

    Figure 6. Final textured meshes from the (top) standard pipeline and (bottom) our pipeline. We improve reconstruction of thin structures - we recover the end of the object, along with a better reconstructed frame.

    Figure 6. Line features are matched more robustly, leading to improved coverage and alignment in the final reconstruction