1© 2015 The MathWorks, Inc.

Deep Learning in

MATLAB

성 호 현 부장hhsung@mathworks.com

2

Deep Learning beats Go champion!

3

AI, Machine Learning, and Deep Learning

Any technique

that enables

machines to

mimic human

intelligence

Statistical methods

enable machines to

“learn” tasks from data

without explicitly

programming

Neural networks with many layers that

learn representations and tasks

“directly” from data

Artificial

IntelligenceMachine

LearningDeep Learning

1950s 1980s 2015

FLOPS MillionThousand Quadrillion

4

What is can Deep Learning do for us?(An example)

5

Example 1: Object recognition using deep learning

6

Object recognition using deep learning

Training (GPU)

Millions of images from 1000

different categories

PredictionReal-time object recognition using

a webcam connected to a laptop

7

What is Machine Learning?What is Deep Learning?

8

Machine Learning vs Deep Learning

Machine Learning

We specify the nature

of the features we want

to extract… …and the type of model

we want to build.

9

Deep Learning

Machine Learning vs Deep Learning

We need only specify

the architecture of the

model…

10

▪ Deep learning is a type of machine learning in which a model learns to perform tasks like classification – directly from images, texts, or signals.

▪ Deep learning performs end-to-end learning, and is usually implemented using a neural network architecture.

▪ Deep learning algorithms also scale with data – traditional machine

learning saturates.

11

Why is Deep Learning So Popular Now?

Source: ILSVRC Top-5 Error on ImageNet

Human

Accuracy

AlexNet

12

Two Approaches for Deep Learning

2. Fine-tune a pre-trained model (transfer learning)

1. Train a Deep Neural Network from Scratch

13

Pains In Deep Learning

Expertise Time to Train Data

14

Example: Vehicle recognition using deep transfer learning

5 Category

Classifier

Cars

Trucks

SUVs

Big Trucks

Vans

15

Import the Latest Models for Transfer Learning

Pretrained Models*

▪ AlexNet

▪ VGG-16

▪ VGG-19

▪ GoogLeNet

▪ Inception-v3

▪ ResNet50

▪ ResNet-101

▪ Inception-resnet-v2

▪ SqueezeNet

▪ MobileNet(coming soon)

* single line of code to access model

Import Models from Frameworks

▪ Caffe Model Importer

▪ TensorFlow-Keras Model Importer

▪ Onnx - Importer/ Exporter (Coming Soon)

AlexNetPRETRAINED

MODEL

CaffeI M P O R T E R

ResNet-50PRETRAINED MODEL

TensorFlow-

KerasI M P O R T E R

VGG-16PRETRAINED

MODEL

GoogLeNet PRETRAINED

MODEL

ResNet-101PRETRAINED MODEL

Inception-v3M O D E L S

16

Detection and localization using deep learning

Regions with Convolutional Neural Network Features (R-CNN)

17

What is semantic segmentation?

18

Original Image

ROI detection

Pixel classification

Localization using deep learning

19

Semantic Segmentation Network

Boat

Airplane

Other classes

20

Semantic Segmentation Network

21

Semantic Segmentation Demo

CamVid Dataset1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008

2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters

22

Semantic Segmentation

CamVid Dataset1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008

2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters

23

“I love to label and

preprocess my data”

~ Said no engineer, ever.

24

Data

Ground truth Labeling

“How do I label

my data?”

New App for

Ground Truth

Labeling

Label pixels

and regions for

semantic

segmentation

25

Attributes and Sublabels NEW in

26

Types of Datasets

Numeric

Data

Time Series/

Text Data

Image

Data

ML or LSTMLSTM or CNN

CNN

27

Analyzing signal data using deep learning

Signal Classification using LSTMs Speech Recognition using CNNs

28

Deep learning features overview

▪ Classification

▪ Regression

▪ Semantic segmentation

▪ Object detection

▪ Scalability – Multiple GPUs

– Cluster or cloud

▪ Custom network layers

▪ Import models – Caffe

– Keras/TensorFlow

▪ Data augmentation

▪ Hyperparameter tuning – Bayesian optimization

▪ Python MATLAB interface

▪ LSTM networks – Time series, signals, audio

▪ Custom labeling – API for ground-truth labeling

automation

– Superpixels

▪ Data validation – Training and testing

29

Prediction Performance: Fast with GPU Coder

Why is GPU Coder so fast?

– Analyzes and optimizes

network architecture

– Invested 15 years in code

generation

AlexNet ResNet-50 VGG-16

TensorFlow

MATLAB

MXNet

GPU Coder

Images/Sec

Using CUDA v9

and cuDNN v7

30

Overview of deep learning deployment options

“How do I deploy

my model?”

Deploy / Share

▪ Create Desktop Apps

▪ Run Enterprise Solution

▪ Generate C and C++ Code

GPU Coder

▪ Target GPUs

Introducing:

GPU Coder-

Convert to

NVIDIA CUDA

code

▪ Generate C and C++ Code

31

GPU Coder Fills a Gap in Our Deep Learning Solution

Access Data Preprocess Select Network Train

Image Acq.Image

Processing

Computer

Vision

Neural

NetworkPCT

GPU

Coder

Deploy

InferenceTraining

32

Deploying to CPUs

GPU

Coder

Deep Learning

Networks

NVIDIA

TensorRT &

cuDNN

Libraries

ARM

Compute

Library

Intel

MKL-DNN

Library

33

MATLAB products for deep learning

▪ Neural Network Toolbox

▪ Parallel Computing Toolbox

▪ Image Processing Toolbox

▪ Computer Vision System Toolbox

▪ Statistics and Machine Learning Toolbox

▪ MATLAB Coder

▪ GPU Coder

▪ Automated Driving System Toolbox

Required products Recommended products

34

Deep learning features overview

▪ Classification

▪ Regression *

▪ Semantic segmentation

▪ Object detection *

▪ Scalability *– Multiple GPUs

– Cluster or cloud

▪ Custom network layers *

▪ Import models *– Caffe

– Keras/TensorFlow

▪ Data augmentation *

▪ Hyperparameter tuning *– Bayesian optimization

▪ Python MATLAB interface *

▪ LSTM networks *– Time series, signals, audio

▪ Custom labeling *– API for ground-truth labeling

automation

– Superpixels

▪ Data validation *– Training and testing

* We can cover in more detail outside this presentation

35

Thank you!

Deep Learning Onramp – MATLAB

36

Deep learning in

automated driving…

37

Deep Learning Onramp

▪ Get started using deep learning

methods to perform image recognition.

▪ Free access for everyone

▪ Interactive exercises and short video

demonstrations

▪ Work on real-life image recognition

problems

▪ Topics include:

– Convolutional neural networks

– Working with pre-trained networks

– Transfer learning

– Evaluating network performance

38

Convolutional Neural Networks (CNN)

ObjectsShapes

Edges

39

Deep Reinforcement Learning (E.g. Deep Q Network)

▪ Policy is a sequence of actions to

observations to get maximum reward

▪ Reinforcement Learning finds the optimal

policy maximizing the reward

▪ Reinforcement Learning adapts to

changes in environment by improving the

policy

▪ No need for explicit model (model-free)

AGENT

OBSERVATION

SACTIONS

REWARD

POLICY

40

Google Deepmind’s Deep Q Learning playing Atari Breakout

41