neural puppet: generative layered cartoon characters · working in the lower dimensional space of...
TRANSCRIPT
Neural Puppet: Generative Layered Cartoon Characters
Vladimir KimOmid Poursaeed Eli Shechtman Serge BelongieJun Saito
MotivationGenerating motion in image / video is difficult
Controllable Video Generation, Hao et al., CVPR 2018MoCoGAN, Tulyakov et al., CVPR 2018
2
Main IdeaUsing an underlying mesh representation for deformation / interpolation
Image
Mesh
DeformationDifferentiable
Rendering
3Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Main IdeaAdvantages:
● Well-defined regularization energies on meshes
● Working in the lower dimensional space of vertex positions
● Alleviating blurry outputs
4Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Template Mesh
Architecture
5Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Architecture
6Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Results
Rendered (Ours)
Generated (Ours)
Input
PWC-Net (Baseline)
DAE (Baseline)
Train Test Train Test
7Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Results
Rendered (Ours)
Generated (Ours)
Input
PWC-Net (Baseline)
DAE (Baseline)
Train Test Train Test
8Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Results
Average L2 distance to ground truth
9Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Interpolation between posesLinearly interpolating between encoded latent vectors
10Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
User-constrained DeformationThe user specifies a desired location for a point on the character
11Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019
Applications: Face manipulation
12Neural Puppet: Generative Layered Cartoon Characters, Poursaeed et al., WACV 2019