disrupting the creative industry with ai - nvidia€¦ · challenge: the client selected a sample...

® 2017 - Confidential

Disrupting The Creative Industry with AIMarco Marchesi - Head Of Tech

Daniel Cheetham - Global Chief Interactive Officer

® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL

Transparently Immersive

Experiences

® 2017 - CONFIDENTIAL

® 2017 - CONFIDENTIAL

Greater Engagement

& Relevance

® 2017 - CONFIDENTIAL

Use Artificial Intelligence to create an image of a ‘perfect mum’, based on a data set of unrealistic depictions of motherhood in the media and on social networks.

Make sure the output resolution holds up on a massive digital out of home screen in London

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Goals

Dataset of faces (1796 images)

?

1024px

1024px

• Generate a face from a limited dataset of faces • Achieve photorealism, industry quality (and Megapixel size)

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Generative Models

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

• Variational Autoencoders (VAE)

• PixelRNNs

• Generative Adversarial Networks

VAE BEGAN

Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel recurrent neural networks." arXiv preprint arXiv:1601.06759 (2016).

Berthelot, David, Tom Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017).

Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Z Generator Fake Sample

Discriminator Real Sample DatasetScore

Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014

GAN

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

GAN

Probability that the image came from the dataset

Probability that the image came from the Generator

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Deep Convolutional GAN

Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).

“Traditional” GANs are difficult to train DCGAN

DCGAN characteristics:• Pooling Layers replaced with strided (D) and fractional-strided (G) convolutions

• Removed the FC Layers • Batch Normalization on D and G Layers• ReLU for G and LeakyReLU for D

Generator Architecture [Radford]

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Our Implementation

Characteristics:

• D and G trained alternatively twice (every 50 epochs)• Batch size from 128 (192px) down to 6 (1024px)• Epochs 100 to 500• LR 0.0002• GPUs 8GB & 12GB

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Training Process

A training sequence (from a batch size = 128)

video sequence here

Samples generated at 192x192px

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

First Results

A special case in testing

z = np.random.uniform(-1, 1, size=(sample_num, z_dim))

z is usually a vector in a random uniform (or gaussian) distribution

z = np.zeros((sample_num, z_dim))

z = [0,0,0,…,0]

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Optimizing the Latent Space

Idea: z as a hyperparameterto minimize the testing loss

Our best choice at 1024px

z = np.random.uniform(-0.5, 0.5, size=(sample_num, z_dim))

Recent ideas:Generative Latent Optimization

(GLO)

Bojanowski, Piotr, et al. "Optimizing the Latent Space of Generative Networks." arXiv preprint arXiv:1707.05776 (2017).

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Optimizing the Latent Space

Samples generated at 192x192px with reduced latent space random distribution

22

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

New Results

82px

189px 2810px

326px

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

New Challenge: upscaling

Challenge: the client selected a sample of 256px

Idea: upscale to 1024px and find z from G(z) BiGAN

Sample selected at 256px

Donahue, Jeff, Philipp Krähenbühl, and Trevor Darrell. "Adversarial feature learning." arXiv preprint arXiv:1605.09782 (2016).

Super-resolution?

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

New Challenge: upscaling

Problem: P(G(z)) changes when we scale up (new

training and different model)

Samples generated at 1024px

BiGAN needs further investigation

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

New Challenge: Z mapping

Solution: Semi-automatic z mapping

G(z)

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

New Challenge: Z mapping

The “Perfect Mum” image The final ad campaign

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Final Result

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Some Stats & Campaign Results

● Sales increase 160% above projected launch target.

● 4.2 million social impressions achieved double Dove’s usual engagement levels.

We generated the first Megapixel image using GANs with some relevant limitations:

Time Dataset Hardware

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

Conclusions

The Turing Test of Creative Retouching

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

What’s next

The Turing Test of Creative Retouching

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

What’s next

The Turing Test of Creative Retouching

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

What’s next

• Marchesi, Marco. "Megapixel Size Image Creation using Generative Adversarial Networks." arXiv preprint arXiv:1706.00082 (2017).

• Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014• Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).

• Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).• Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel recurrent neural networks." arXiv preprint arXiv:1601.06759 (2016).

• Berthelot, David, Tom Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017).

• Bojanowski, Piotr, et al. "Optimizing the Latent Space of Generative Networks." arXiv preprint arXiv:1707.05776 (2017).• Donahue, Jeff, Philipp Krähenbühl, and Trevor Darrell. "Adversarial feature learning." arXiv preprint arXiv:1605.09782 (2016).

DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.

References

www.happyfinish.com Everything is possible.

marco.marchesi@happyfinish.com

daniel.cheetham@happyfinish.com