aws re:invent 2016: deep learning at cloud scale: improving video discoverability by scaling up...

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

November 2016

MAC205

Deep Learning at Cloud ScaleImproving Video Discoverability by Scaling Up Caffe on AWS

Andres Rodriguez, PhD, Solutions Architect, Intel Corporation

Juan Carlos Riverio, CEO, Vilynx

Content Outline

• Deep learning overview and usages

• Worked example for fine-tuning a NN

• Some theory behind deep learning

• Vilynx – videos discoverability

2

Deep Learning

• A branch of machine learning

• Data is passed through multiple non-linear

transformations

• Goal: Learn the parameters of the transformation that

minimize a cost function

3

Bigger Data Better Hardware Smarter Algorithms

Why Now?

Image: 1000 KB / picture

Audio: 5000 KB / song

Video: 5,000,000 KB / movie

Transistor density doubles

every 18 months

Cost / GB in 1995: $1000.00

Cost / GB in 2015: $0.03

Advances in algorithm innovation, including neural networks, leading to better accuracy in training models

4

Types of Deep Learning

• Supervised learning

• Data -> Labels

• Unsupervised learning

• No labels; Clustering; Reducing dimensionality

• Reinforcement learning

• Reward actions (e.g., robotics)

http://ode.engin.umich.edu/presentations/idetc2014/img/image_feature_learning_clear.png

5

data

output expected

…

0.10 0.15 0.20 …0.05

person cat dog bike

0 1 0 … 0

person cat dog bike

penalty(error or cost)

…

Forward

Propagation

Back

Propagation

Training

6

data

output expected

…

person cat dog bike

0 1 0 … 0

person cat dog bike

inference

Training

0.10 0.15 0.20 0.05

penalty(error or cost)

7

… …

Forward

Propagation

Back

Propagation

Deep Learning Use Cases

• Fraud / face detection

• Gaming, check processing

• Computer server

monitoring

• Financial forecasting and

prediction

• Network intrusion

detection

• Recommender systems

• Personal assistant

• Automatic Speech

recognition

• Natural language

processing

• Image & Video

recognition/tagging

• Targeted Ads

Cloud Service Providers

Financial

Services

Healthcare

Automotive

8

Optimized Deep Learning Environment

Fuel the development of vertical solutions

Deliver excellent deep learning environment

Develop deep networks across frameworks

Maximum performance on Intel architecture

EC2

Intel® Math Kernel Library (Intel® MKL)

9

Elastic Compute Cloud (EC2)

C4 Instances

• “Highest performing processors and the lowest price/compute

performance in EC2”1

• Vilynx

• Deep learning for video content extraction

• Supports various companies: CBS, TBS, etc.

•

1https://aws.amazon.com/ec2/instance-types/https://www.stlmag.com/news/st-louis-app-pikazo-will-turn-your-profile-picture/

• Pikazo app

• Transforms photos into artistic render

10

Elastic Compute Cloud (EC2)

C4 Instances

c4.8xlarge On-Demand:

• $1.675/hr

GoogleNet inference:

• batch size 32

• 237 ims/sec = 4.2 ms/im

• 1 million images costs

$1.96

Spot prices are cheaper

OS: Linux version 3.13.0-86-generic (buildd@lgw01-51) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #131-Ubuntu SMP Thu May 12 23:33:13

UTC 2016. MxNet Tip of tree: commit de41c736422d730e7cfad72dd6afc229ce08cf90, Tue Nov 1 11:43:04 2016 +0800. MKL 2017 Gold update 1

11

6.1 2.4 1.2 0.8

679.5

262.5

79.7 73.9

0

200

400

600

800

AlexNet GoogLeNet v1 ResNet-50 GoogLeNet v3

Imag

es/S

ec

c4.8xlarge MXNet Inference

No MKL MKL

Intel® Math Kernel Library 2017 (Intel® MKL 2017)

• Optimized for EC2 instances with Intel® Xeon® CPUs

• Optimized for common deep learning operations

• GEMM (useful in RNNs and fully connected layers)

• Convolutions

• Pooling

• ReLU

• Batch normalization

Recurrent NN Convolutional NN12

Naïve Convolution

https://en.wikipedia.org/wiki/Convolutional_neural_network

13

Cache Friendly Convolution

arxiv.org/pdf/1602.06709v1.pdf

14

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

15

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

𝑑𝐽 𝒘(0)

𝑑𝒘

16

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

𝒘(1) = 𝒘(0) −𝑑𝐽 𝒘(0)

𝑑𝒘

17

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

𝒘(1) = 𝒘(0) − 𝛼𝑑𝐽 𝒘(0)

𝑑𝒘

learning rate

18

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

𝒘(1) = 𝒘(0) − 𝛼𝑑𝐽 𝒘(0)

𝑑𝒘

𝒘(1)

too small

19

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

𝒘(1) = 𝒘(0) − 𝛼𝑑𝐽 𝒘(0)

𝑑𝒘

𝒘(1)

too large

20

Gradient Descent

𝐽 𝒘(0) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(0), 𝒙𝑖)

𝒘𝒘(0)

𝒘(1) = 𝒘(0) − 𝛼𝑑𝐽 𝒘(0)

𝑑𝒘

𝒘(1)

good enough

21

Gradient Descent

𝐽 𝒘(1) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(1), 𝒙𝑖)

𝒘𝒘(2)

𝒘(2) = 𝒘(1) − 𝛼𝑑𝐽 𝒘(1)

𝑑𝒘

𝒘(1)

22

Gradient Descent

𝐽 𝒘(2) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(2), 𝒙𝑖)

𝒘

𝒘(3) = 𝒘(2) − 𝛼𝑑𝐽 𝒘(2)

𝑑𝒘

𝒘(2)

𝒘(3)

23

Gradient Descent

𝐽 𝒘(3) =

𝑖=1

𝑁

𝑐𝑜𝑠𝑡(𝒘(3), 𝒙𝑖)

𝒘

𝒘(4) = 𝒘(3) − 𝛼𝑑𝐽 𝒘(3)

𝑑𝒘

𝒘(4)

𝒘(3)

24

Transfer learning via fine-tuning

• First few layers are usually very similar within a domain

• Last layers are task specific

• Take a trained model and fine-tune it for a particular task

http://vision.stanford.edu/Datasets/collage_s.png

https://www.kaggle.com/c/dogs-vs-cats

http://adas.cvc.uab.es/task-cv2016/papers/0026.pdf

25

• Install Intel-Optimized Caffe (or your favorite framework)

• https://software.intel.com/en-us/articles/training-and-deploying-deep-

learning-networks-with-caffe-optimized-for-intel-architecture

• Download a pre-trained model

• http://dl.caffe.berkeleyvision.org/bvlc_reference_caffenet.caffemodel

• Modify the training model (next slide)

Fine-tuning steps

26

Fine-tuning: ILSVRC -> DogsVsCatslayer {

name: "data"

type: "Data"

data_param {

source: "ilsvrc12_train_lmdb"

...

}

...

}

...

layer {

name: "fc8"

type: "InnerProduct"

inner_product_param {

num_output: 1000

...

}

}

layer {

name: "data"

type: "Data"

data_param {

source: “dogs_cats_train_lmdb"

...

}

...

}

...

layer {

name: "fc8-ft"

type: "InnerProduct"

inner_product_param {

num_output: 2

...

}

}>> # From the command line

>> caffe train -solver solver.prototxt -weights trainedModel.caffemodel

27

Fine-tuning guidelines

• Freeze all but the last layer (or more if new dataset is very different)

• lr_mult=0 in local learning rates

• Earlier layer weights won't change very much

• Drop the initial learning rate (in the solver.prototxt) by 10x

Replace 1000 with 2 unit layer

Train the 4096+1 x 2 weights

http://www.mdpi.com/remotesensing/remotesensing-07-14680/article_deploy/html/images/remotesensing-07-14680-g002-1024.png

28

Demo

• Fine-tune trained model for dog vs cats

http://vision.stanford.edu/Datasets/collage_s.png

https://www.kaggle.com/c/dogs-vs-cats

29

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Juan Carlos Riveiro: CEO and Cofounder

30

How?. . .Building the biggest dataset for video deep learning by auto tagging selected video

scenes in real-time and leveraging web and social media to continues update the tags

Hello. We're Vilynx, the video personalization company

We select the relevant contents targeted to individual needs

solving the content discovery problem.

Benefit?..Increase views, time spent watching videos and in video search.

Markets: Media, Smart Phones, Drones, Security, Robots, Smart Cities.

31

Outstanding Tech Team: Experienced and Very Successful

Juan Carlos Riveiro, CEO

More than 100 patents in Signal Processing, Data

statistics/algorithms and Machine Learning.

Founder and CEO of Gigle Networks (Acquired by

Broadcom),

CTO & VP of R&D at DS2 (Acquired by Marvell).

Elisenda Bou, CTO

PhD from UPC and MIT and expert on Machine Learning

and Complex SW Architectures. Worked on adaptive

satellite control systems and recipient of the 2013 Google

Faculty Research Awards.

José Cordero Rama

MS for Deep Learning at UPC/BSC

Data Scientist at King, Bdigital and Gen-Med

Joan Capdevila, PhD

MS and PhD for Machine Learning

At Georgia Tech and UPC/BSC

Data Scientist at AIS and Accenture

Jordi Pont-Tuset, PhD

PostDoc on Machine Learning at ETH Zurich

PhD on Image Segmentation at UPC

Disney Research

Asier Aduriz

Computer Science and Telecom Engineering

degree at UPC (Top 1% of class)

Engineer at CERN.

Dèlia Fernàndez

MS on Deep Learning at Columbia University

Signal Processing Researcher at Northeastern University

Data Scientist at InnoTech

David Varas, PhD

PhD for Video Object Tracking at UPC

Adjunct Professor on Computer Vision &

Statistical Signal Processing at UPC

32

Vilynx: Indexing Visual Knowledge

8 cameras/car

Smart Cities

Connecting Everything

VR/AR Changing Everything A camera at every

corner in London

Drones everywhere (Amazon)

How is all this visual content going to be indexed?

Just like the internet before Google

+1000 hours of video uploaded

every minute in internet

33

The Vilynx Knowledge Graph

The average vocabulary of a 5-year

old is 5000 words

• 4800 words/concepts

• 1.8 tags per video

• 8M videos

The average vocabulary of an adult

is 30,000 words

• 2M words/concepts

• 50 tags per video

• 10M videos 34

First Market driven by Video Content Producers

Media companies need content personalization to drive audience

through multiple channels

35

Some Customer Examples:

http://www.cbs.com/shows/the-late-

show-with-stephen-colbert/

https://www.americasgreatestmakers.com/

http://www.vanitatis.elconfidencial.com/36

Vilynx Products

Inputs:

Outputs:

Applications:

37

Videos Audience DataContextual Data:

Social Networks, YouTube, Web

Key 5 sec clips Intelligent Auto Tagging

• Better video

discovery

• Native Ad

integration

• Programmatic

Ad matching

• More video

views and

longer

engagement

times

• VOD & Live

Events

• Drive branding

• Amplification

with keyword

recommendation

• Drive Click

through rates

• Better user

experience

Video Thumbnails Social Sharing Recommendations Video Search Ad Market

Vilynx | Workflow

Machine Learning or Deep Learning

4

3

12

98% accuracy to find the relevant parts of the video

CTR increase between 50% to 500% (customer validated)38

1. We ingest customer videos and the contextual information around it.

2. We then take cues from around the Web and social networks.

3. This combined input is fed to the most advanced convolutional deep neural network in the industry.

4. Output are video previews optimized to engage your audience and rich metadata that can further drive your video content.

A data training set of video moments that includes:

10M (and growing) tagged 5 sec video moments,

ImageNet for video has only 4000 moments

2M Contextual tags (and growing)

Continuously updated training set of new tags by

crawling of social media/the web

Real time unsupervised training of the network to

autonomously learn and identify new patterns

Advancing Deep Learning Networks:

Move from simple classification to indexing all visual content

39

Demo Results

• Fine-tune dogs vs cats classifier results

http://vision.stanford.edu/Datasets/collage_s.png

https://www.kaggle.com/c/dogs-vs-cats

40

Call to action

• Use Intel Optimized Frameworks for workloads

• https://github.com/intel/caffe

• https://github.com/dmlc/mxnet

• https://github.com/intel/theano

• https://github.com/intel/torch

• other frameworks coming soon…

• Deep learning tutorial

• https://software.intel.com/en-us/articles/training-and-deploying-deep-learning-networks-with-caffe-

optimized-for-intel-architecture

• Distributed training of deep networks on AWS

• https://software.intel.com/en-us/articles/distributed-training-of-deep-networks-on-amazon-web-

services-aws

41

Legal Notices & Disclaimers

This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice.

Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at

intel.com, or from the OEM or retailer. No computer system can be absolutely secure.

Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual

performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance

and benchmark results, visit http://www.intel.com/performance.

Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may

affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.

Statements in this document that refer to Intel’s plans and expectations for the quarter, the year, and the future, are forward-looking statements that involve a

number of risks and uncertainties. A detailed discussion of the factors that could affect Intel’s results and plans is included in Intel’s SEC filings, including the

annual report on Form 10-K.

The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current

characterized errata are available on request.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm

whether referenced data are accurate.

Intel, the Intel logo, Pentium, Celeron, Atom, Core, Xeon and others are trademarks of Intel Corporation in the U.S. and/or other countries.

*Other names and brands may be claimed as the property of others.

© 2016 Intel Corporation.

42

Thank you!

(huge) contributions from:

Joseph Spisak, Elisenda Bou, Hendrik Van der Meer, Zhenlin Luo, Ravi Panchumarthy,

Ryan Saffores, Niv Sundaram, and many more..

Remember to complete

your evaluations!