texture fields - cvlibs · autonomous vision group max planck institute for intelligent systems...
TRANSCRIPT
Autonomous Vision Group
Max Planck Institutefor Intelligent Systems
Texture FieldsLearning Texture Representations in Function Space
Michael Oechsle1,2,3 Lars Mescheder1,2 Michael Niemeyer1,2
Thilo Strauss3 Andreas Geiger1,2
¹MPI for Intelligent Systems 2University of Tübingen 3ETAS GmbH, Stuttgart
1. Motivation
Existing 3D Representations
Deep learning has achieved impressive results •
for image and texture generation in the 2D domain
However, texture generation in the 3D domain lacks far behind.
Major problem: Representation of texture in 3D
for learning-based reconstruction of 3D geometry.•
Colored Voxels
Grid structure
Low-frequency texture
Memory requirement
Texture Atlas
Texture from a Single Rendered Image
Texture from a Single Real Image
Quantitative Evaluation
2. Existing Texture Representations
No discretization
Independent ofshape representation
No template required
For learning texture reconstruction, we condition the Texture Field on an image and an untextured 3D model.
Idea: Represent texture as continuous 3D field
3. Our Representation
2D Image Ours ProjectionNVS baseline
2D Image 2D ImageOurs Ours
2D Image ONet ONet + Texture Fields
5. Experiments - Texture Reconstruction
4. Our Model
High-frequency texture
Discontinuities
Template required
Textured 3D Model
3D Model
2D Image
Neural Network
6. Experiments - Generative Model
Full Pipeline
Shape
Texture
VAE - Texture TransferGAN - SamplesVAE - Samples
VAE - Latent Space Interpolations
Texture from a Single Rendered Image
Full Pipeline with ONet
tiny.cc/texfield
with slides, videos and more
Blog
KL Divergence
Shape Encoder
Texture Field
Sampling
Image Encoder
3D Shape
2D Image
Recon.Loss
Depth Map 3D Point Predicted Image True Image
Point Cloud
Rendering
Unprojection
VAE Encoder
Adver-sarialLossConditional Model
GAN ModelVAE Model
Color Legend:
Color
Discriminator