disrupting the creative industry with ai - nvidia€¦ · challenge: the client selected a sample...

® 2017 - Confidential

Disrupting The Creative Industry with AIMarco Marchesi - Head Of Tech

Daniel Cheetham - Global Chief Interactive Officer

® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL® 2017 - CONFIDENTIAL


Transparently Immersive

Experiences



Greater Engagement

& Relevance


Use Artificial Intelligence to create an image of a ‘perfect mum’, based on a data set of unrealistic depictions of motherhood in the media and on social networks.

Make sure the output resolution holds up on a massive digital out of home screen in London

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Goals

Dataset of faces (1796 images)

?

1024px

1024px

• Generate a face from a limited dataset of faces • Achieve photorealism, industry quality (and Megapixel size)


Generative Models


• Variational Autoencoders (VAE)

• PixelRNNs

• Generative Adversarial Networks

VAE BEGAN

Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel recurrent neural networks." arXiv preprint arXiv:1601.06759 (2016).

Berthelot, David, Tom Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017).

Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).


Z Generator Fake Sample

Discriminator Real Sample DatasetScore

Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014

GAN


GAN

Probability that the image came from the dataset

Probability that the image came from the Generator


Deep Convolutional GAN

Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).

“Traditional” GANs are difficult to train DCGAN

DCGAN characteristics:• Pooling Layers replaced with strided (D) and fractional-strided (G) convolutions

• Removed the FC Layers • Batch Normalization on D and G Layers• ReLU for G and LeakyReLU for D

Generator Architecture [Radford]


Our Implementation

Characteristics:

• D and G trained alternatively twice (every 50 epochs)• Batch size from 128 (192px) down to 6 (1024px)• Epochs 100 to 500• LR 0.0002• GPUs 8GB & 12GB


Training Process

A training sequence (from a batch size = 128)

video sequence here

Samples generated at 192x192px


First Results

A special case in testing

z = np.random.uniform(-1, 1, size=(sample_num, z_dim))

z is usually a vector in a random uniform (or gaussian) distribution

z = np.zeros((sample_num, z_dim))

z = [0,0,0,…,0]


Optimizing the Latent Space

Idea: z as a hyperparameterto minimize the testing loss

Our best choice at 1024px

z = np.random.uniform(-0.5, 0.5, size=(sample_num, z_dim))

Recent ideas:Generative Latent Optimization

(GLO)

Bojanowski, Piotr, et al. "Optimizing the Latent Space of Generative Networks." arXiv preprint arXiv:1707.05776 (2017).


Optimizing the Latent Space

Samples generated at 192x192px with reduced latent space random distribution

22


New Results

82px

189px 2810px

326px


New Challenge: upscaling

Challenge: the client selected a sample of 256px

Idea: upscale to 1024px and find z from G(z) BiGAN

Sample selected at 256px

Donahue, Jeff, Philipp Krähenbühl, and Trevor Darrell. "Adversarial feature learning." arXiv preprint arXiv:1605.09782 (2016).

Super-resolution?


New Challenge: upscaling

Problem: P(G(z)) changes when we scale up (new

training and different model)

Samples generated at 1024px

BiGAN needs further investigation


New Challenge: Z mapping

Solution: Semi-automatic z mapping

G(z)


New Challenge: Z mapping

The “Perfect Mum” image The final ad campaign


Final Result


Some Stats & Campaign Results

● Sales increase 160% above projected launch target.

● 4.2 million social impressions achieved double Dove’s usual engagement levels.

We generated the first Megapixel image using GANs with some relevant limitations:

Time Dataset Hardware


Conclusions

The Turing Test of Creative Retouching


What’s next

• Marchesi, Marco. "Megapixel Size Image Creation using Generative Adversarial Networks." arXiv preprint arXiv:1706.00082 (2017).

• Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014• Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).

• Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).• Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel recurrent neural networks." arXiv preprint arXiv:1601.06759 (2016).

• Berthelot, David, Tom Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017).

• Bojanowski, Piotr, et al. "Optimizing the Latent Space of Generative Networks." arXiv preprint arXiv:1707.05776 (2017).• Donahue, Jeff, Philipp Krähenbühl, and Trevor Darrell. "Adversarial feature learning." arXiv preprint arXiv:1605.09782 (2016).


References

www.happyfinish.com Everything is possible.

[email protected]

[email protected]

http://www.happyfinish.com

disrupting the creative industry with ai - nvidia€¦ · challenge: the client selected a sample...

Documents