model-driven 3d content creation as variation hao (richard) zhang – 张皓 gruvi lab, simon fraser...
TRANSCRIPT
![Page 1: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/1.jpg)
Model-Driven 3D Content Creation as
Variation
Model-Driven 3D Content Creation as
VariationHao (Richard) Zhang – 张皓
GrUVi Lab, Simon Fraser University (SFU)
Talk @ HKUST, 04/21/11
Hao (Richard) Zhang – 张皓GrUVi Lab, Simon Fraser University (SFU)
Talk @ HKUST, 04/21/11
TAUTAU ZJUZJUNUDTNUDTSFUSFU
![Page 2: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/2.jpg)
3D content creation
Inspiration a readily usable digital 3D modelInspiration a readily usable digital 3D model
Inspiration?Inspiration?
![Page 3: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/3.jpg)
Realistic reconstruction
• Inspiration = real-world data
[Nan et al., SIGGRAPH 2010][Nan et al., SIGGRAPH 2010]
![Page 4: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/4.jpg)
Creative inspiration
•Creation of novel 3D shapes
• Inspiration = design concept, mental
picture, …
sketchsketch
High demand in VFX, games, simulation, VR, …
High demand in VFX, games, simulation, VR, …
![Page 5: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/5.jpg)
3D content creation is hard
•2D-to-3D: an ill-posed problem
▫Shape from shading, sketch-based modeling, …
•Creation from scratch is hard: job for skilled
artistsOne of the most central problems in
graphics; One of the most discussed at SIG’10 panel
One of the most central problems in graphics; One of the most discussed at
SIG’10 panel
![Page 6: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/6.jpg)
Usable 3D content even harder
•Models created are meant for subsequent
use
•Creation of readily usable 3D models
![Page 7: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/7.jpg)
Usable 3D content even harder
•Models created are meant for subsequent use
•Creation of readily usable 3D models
•Higher-level information beyond low-level mesh
▫Part or segmentation information
▫Structural relations between parts
▫Correspondence to relevant models, etc.Hard shape analysis problems!Hard shape analysis problems!
![Page 8: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/8.jpg)
Key: model reuse
•Reuse existing 3D models and associated
info
•Model-driven approach: creation is driven
by or based on existing (pre-analyzed)
models
![Page 9: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/9.jpg)
Key: model reuse
•Reuse existing 3D models and associated
info
•Model-driven approach: creation is driven
by or based on existing (pre-analyzed)
models
•Two primary modes of reuse:
▫New creation via part composition
![Page 10: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/10.jpg)
Key: model reuse
•Reuse existing 3D models and associated info
•Model-driven approach: creation is driven by
or based on existing (pre-analyzed) models
•Two primary modes of reuse:
▫New creation via part composition
▫New creation as variation or modification of
existing model(s), e.g., a warp or a
deformation
![Page 11: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/11.jpg)
Modeling by example
•New models composed by parts retrieved
from an existing data repository
•Key: retrieve relevant parts
•Many variants …
[Funkhouser et al., SIGGRAPH 2004][Funkhouser et al., SIGGRAPH 2004]
![Page 12: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/12.jpg)
Pros and cons
•Pros:▫Significant deviation from existing models
▫Exploratory modeling via part suggestions
[Chaudhuri & Koltun., SIG Asia 2010][Chaudhuri & Koltun., SIG Asia 2010]
![Page 13: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/13.jpg)
Pros and cons
•Pros:▫Significant deviation from existing models
▫Exploratory modeling with part suggestions
•Cons:▫Are models composed by parts readily
usable?
![Page 14: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/14.jpg)
Pros and cons
•Pros:▫Significant deviation from existing models
▫Exploratory modeling with part suggestions
•Cons:▫Are models composed by parts readily
usable? structure lost by part composition; how to stitch?
![Page 15: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/15.jpg)
Pros and cons
•Pros:▫Significant deviation from existing models
▫Exploratory modeling with part suggestions
•Cons:▫Are models composed by parts readily
usable? structure lost by part composition; how to stitch?
▫Does part exploration always reflect user design intent?
![Page 16: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/16.jpg)
Model-driven creation as variation
•New creation as variation of existing model(s)
Enrich a set; generate “more of
the same” …
Enrich a set; generate “more of
the same” …
Photo-inspired 3D model creation
Photo-inspired 3D model creation
Inspiration = photographsInspiration = photographs
Inspiration = a model set
Inspiration = a model set
![Page 17: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/17.jpg)
Model-driven creation as variation
•New creation as variation of existing model(s)
Enrich a set; generate “more of
the same” …
Enrich a set; generate “more of
the same” …
Inspiration = a model set
Inspiration = a model set
![Page 18: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/18.jpg)
Style-Content Separation by Anisotropic Part Scales
Kai Xu1,2, Honghua Li2, Hao Zhang2, Daniel Cohen-Or3
Yueshan Xiong2, and Zhi-Quan Cheng2
1Simon Fraser Universtiy 2National Univ. of Defense Tech. 3Tel-Aviv University
![Page 19: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/19.jpg)
Motivation
•Enrich a set of 3D models with their derivatives
Set belongs to the same family or class
Set belongs to the same family or class
![Page 20: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/20.jpg)
Variations in shape parts in the set
Geometric or content difference
Part proportion (= style) difference
![Page 21: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/21.jpg)
?
Style transfer as a derivative
Part proportion style
![Page 22: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/22.jpg)
?
Style transfer as a derivative
Part proportion style
![Page 23: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/23.jpg)
Difficulty with style transfer
•Style transfer needs part correspondence
•Part correspondence is difficult
▫Unsupervised problem
▫Both content and style variations
Variations can be significant!
![Page 24: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/24.jpg)
Work at part and OBB level
Parts enclosed and characterized by tight oriented bounding boxes (OBBs)Parts enclosed and characterized by tight oriented bounding boxes (OBBs)
![Page 25: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/25.jpg)
Style content separation
•To address both shape variations in the set▫Separate treatment of “style” and “content”
Style 1
Style 2
Style 3
ContentContent
Sty
leS
tyle
![Page 26: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/26.jpg)
Style transfer as a derivative
•Creation = filling in the style-content table
![Page 27: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/27.jpg)
Style vs. content
•Fundamental to human perception
Content Style
Language Words Accents
Text Letters Fonts
Human face Identities Expressions
![Page 28: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/28.jpg)
Style content separation
•Previous works on faces, motion, etc.
▫Prerequisite: data correspondence
▫Correspondence dealt with independently
▫Correspondence itself is the very challenge!
![Page 29: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/29.jpg)
Our approach
• One particular style:
Anisotropic part scales or part proportions
![Page 30: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/30.jpg)
Our approach
• One particular style:
Anisotropic part scales or part proportions
• The approach:
Style-content separation with style
clustering in a correspondence-free way
![Page 31: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/31.jpg)
Algorithm overview
•Pipeline
Style clustering Co-segmentation Inter-style part correspondence
Contentclassification
![Page 32: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/32.jpg)
Anisotropic part scales
•Measure style distance between two shapes
![Page 33: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/33.jpg)
Anisotropic part scales
•Measure style distance between two shapes
Part OBB graphs of
given segmentatio
n
![Page 34: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/34.jpg)
Anisotropic part scales
•Measure style distance between two shapes
Computestyle
signatures
……
Part OBB graphs of
given segmentatio
n
![Page 35: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/35.jpg)
Anisotropic part scales
•Measure style distance between two shapes
……
Part OBB graphs of
given segmentatio
n
Euclideandistance
Computestyle
signatures
![Page 36: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/36.jpg)
Style distance issues
•Unknown segmentation
•Unknown correspondence
?
?
![Page 37: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/37.jpg)
Style distance
•Search over all part compositions and part counts
……
……
![Page 38: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/38.jpg)
Style distance
•For each part count, find minimal distance
……
……
A good signature will return min distance across all part counts to reflect corresponding part decompositions …
![Page 39: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/39.jpg)
Correspondence-free style signature
Binary relations: difference of part scales between adjacent OBBs
Use Laplacian graph spectra:
OBB graph
![Page 40: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/40.jpg)
Correspondence-free style signature
Unary attributes: anisotropy of parts
Use Laplacian graph spectra:
OBB graph linear planar spherical
Graph spectra is permutation-free
![Page 41: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/41.jpg)
Style clustering
•Spectral clustering using style distances
![Page 42: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/42.jpg)
Pipeline
Style clustering Co-segmentation Inter-style part correspondence
Contentclassification
![Page 43: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/43.jpg)
Co-segmentation
•Approach:▫Consistent segmentation [Golovinskiy & Funkhouser,
SMI 09]
▫ Initial guess: global alignment (ICP)
[Golovinskiy & Funkhouser 09]
![Page 44: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/44.jpg)
Co-segmentation
•Approach:▫Consistent segmentation [Golovinskiy & Funkhouser, SMI 09]
▫ Initial guess: global alignment (ICP)
•We co-segment within a style cluster▫Removing non-homogeneous part scaling from analysis
[Golovinskiy & Funkhouser 09]
![Page 45: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/45.jpg)
Co-segmentation
•Approach:▫Consistent segmentation [Golovinskiy & Funkhouser, SMI 09]
▫ Initial guess: global alignment (ICP)
•We co-segment within a style cluster▫Removing non-homogeneous part scaling from analysis
[Golovinskiy & Funkhouser 09] After style separation
![Page 46: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/46.jpg)
Pipeline
Style clustering Co-segmentation Inter-style part correspondence
Contentclassification
![Page 47: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/47.jpg)
Inter-style part correspondence
•Approach: deform-to-fit
▫Deformation-driven correspondence [Zhang et al., SGP 08]
▫Consider common interactions between OBBs
1D-to-1D 1D-to-2D 2D-to-2D 2D-to-3D
![Page 48: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/48.jpg)
Inter-style part correspondence
•Deform-to-fit: appropriate deformation energy
Pruned priority-driven search
![Page 49: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/49.jpg)
Pipeline
Style clustering Co-segmentation Inter-style part correspondence
Contentclassification
![Page 50: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/50.jpg)
Content classification
•Use Light Field Descriptor [Chen et al. 2003]
•Compare corresponding parts
Part-level LFD Global LFD
![Page 51: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/51.jpg)
Synthesis by style transfer
•OBBs are scaled
•Underlying geometry via space
deformationcontent
style style transfer
![Page 52: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/52.jpg)
Results: hammers
![Page 53: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/53.jpg)
Results: goblets
![Page 54: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/54.jpg)
Results: humanoids
![Page 55: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/55.jpg)
Results: chairs
![Page 56: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/56.jpg)
Pros and cons
•Pros:▫Automatic generation of many variations
▫Unsupervised
▫Deals with anisotropic part scales
▫Variation = part scaling: structure preservation
![Page 57: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/57.jpg)
Pros and cons
•Pros:▫Automatic generation of many variations
▫Unsupervised
▫Deals with anisotropic part scales
▫Variation = part scaling: structure preservation
•Cons:▫Rely on sufficiently good initial segmentations
▫Variation does not create new content
![Page 58: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/58.jpg)
Interesting future work
•Learn and synthesize with generic styles
![Page 59: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/59.jpg)
Model-driven creation as variation
•New creation as variation of existing model(s)
Photo-inspired 3D model creation
Photo-inspired 3D model creation
Inspiration = photographsInspiration = photographs
![Page 60: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/60.jpg)
Photo-inspired 3D modeling
Photo-Inspired Model-Driven 3D Object Modeling
Kai Xu1,2, Hanlin Zheng4, Hao Zhang2, Daniel Cohen-Or3
Ligang Liu4, and Yueshan Xiong2
1NUDT 2SFU 3TAU 4ZJU Conditionally acceptedConditionally accepted
![Page 61: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/61.jpg)
Overview
Input: single photograph + pre-analyzed datasetInput: single photograph + pre-analyzed dataset
![Page 62: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/62.jpg)
Overview
1. Model-driven labelled segmentation of photographed object
1. Model-driven labelled segmentation of photographed object
![Page 63: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/63.jpg)
Overview
2. Choosing of a candidate model from the database
2. Choosing of a candidate model from the database
![Page 64: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/64.jpg)
Overview
3. Silhouette-constrained deform-to-fit of candidate
3. Silhouette-constrained deform-to-fit of candidate
![Page 65: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/65.jpg)
Overview
OutputOutput
![Page 66: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/66.jpg)
Structure preservation
•Any higher-level structural info in the candidate
models is preserved during deform-to-fit
▫Symmetry relations
▫Part-level correspondence in the set
▫Controller structures [Zheng et al. @ HKUST, EG 11]
![Page 67: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/67.jpg)
Structure preservation
•Any higher-level structural info in the candidate
models is preserved during deform-to-fit
▫Symmetry relations
▫Part-level correspondence in the set
▫Controller structures [Zheng et al. @ HKUST, EG 11]
•Structures also serve to constrain deformation of
candidate model
![Page 68: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/68.jpg)
Controller representations
•Controllers: cuboids and generalized cylinders
•Relations: symmetry, proximity, etc.
Fitting primitivesFitting primitives
![Page 69: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/69.jpg)
Controller representations
•Controllers: cuboids and generalized cylinders
•Relations: symmetry, proximity, etc.
Fitting primitivesFitting primitives
![Page 70: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/70.jpg)
Deformation of controllers
photophoto
![Page 71: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/71.jpg)
Controller primitivesController primitives
Deformation of controllers
photophoto candidate modelcandidate model
![Page 72: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/72.jpg)
Controller primitivesController primitives
Deformation of controllers
Result of silhouette-driven deform-to-fit
Result of silhouette-driven deform-to-fit
photophoto candidate modelcandidate model
![Page 73: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/73.jpg)
Structure preservation at work
symmetrysymmetry
![Page 74: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/74.jpg)
Structure preservation at work
symmetrysymmetry
proximityproximity
![Page 75: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/75.jpg)
Structure preservation at work
symmetrysymmetry
proximityproximity
optimizationoptimization
![Page 76: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/76.jpg)
Structure preservation at work
symmetrysymmetry
proximityproximity
optimizationoptimization
outputoutput
Short videoShort video
![Page 77: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/77.jpg)
Results
•Guidance in single view but coherent 3D results
![Page 78: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/78.jpg)
Results
![Page 79: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/79.jpg)
The Google chair challenge
![Page 80: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/80.jpg)
Not just chairs …
![Page 81: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/81.jpg)
Pros and cons
•Pros:▫Photos: immensely rich source of inspiration
▫Silhouette-driven deformation
▫Variation is less “intrusive” to retain high-level info of source model readily usable
![Page 82: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/82.jpg)
Pros and cons
•Pros:▫Photos: immensely rich source of inspiration
▫Silhouette-driven deformation
▫Variation is less “intrusive” to retain high-level info of source model more readily usable
•Cons▫Variation does not create new structures
![Page 83: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/83.jpg)
Future work
•Photo-inspired model deformation only a start
•Further model refinement, e.g., via sketches
![Page 84: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/84.jpg)
Future work
•Photo-inspired model deformation only a start
•Further model refinement, e.g., via sketches
•Model-driven structure modification
![Page 85: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/85.jpg)
Future work
•Photo-inspired model deformation only a start
•Further model refinement, e.g., via sketches
•Model-driven structure modification
•Other inspirations for 3D content creation
▫Sketch-inspired model variation
![Page 86: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/86.jpg)
Future work
•Photo-inspired model deformation only a start
•Further model refinement, e.g., via sketches
•Model-driven structure modification
•Other inspirations for 3D content creation
▫Sketch-inspired model variation
•Style transfer with unknown style in a set
![Page 87: Model-Driven 3D Content Creation as Variation Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11 TAUZJUNUDT SFU](https://reader038.vdocuments.mx/reader038/viewer/2022102500/56649d135503460f949e729d/html5/thumbnails/87.jpg)
Thank you, 谢谢
TAUTAU ZJUZJUNUDTNUDTSFUSFU