deep learning and reinforcement learning workflows in a.i. · increased productivity with...
TRANSCRIPT
![Page 1: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
Deep Learning and Reinforcement
Learning Workflows in A.I.
Emelie Andersson
![Page 2: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/2.jpg)
2
Why MATLAB for Artificial Intelligence?
![Page 3: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/3.jpg)
3
Artificial Intelligence
Development of computer systems to perform tasks that normally
require human intelligence
![Page 4: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/4.jpg)
4
Speech RecognitionObject Classification
Signal Classification
Predictive Maintenance
Automated Driving Stock Market Prediction
A.I. Applications
![Page 5: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/5.jpg)
5
Unsupervised
Learning[No Labeled Data]
Clustering
Machine Learning
Machine Learning and Deep Learning
![Page 6: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/6.jpg)
6
Unsupervised
Learning[No Labeled Data]
Supervised Learning
[Labeled Data]
Clustering Classification Regression
Machine Learning
Machine Learning and Deep Learning
![Page 7: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/7.jpg)
7
Unsupervised
Learning[No Labeled Data]
Supervised Learning
[Labeled Data]
Clustering Classification Regression
Machine Learning
Machine Learning and Deep Learning
Deep Learning
Supervised learning typically involves
feature extraction
Deep learning typically does not
involve feature extraction
![Page 8: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/8.jpg)
8
Machine Learning
Deep Learning
Deep Learning
▪ Subset of machine learning with automatic feature extraction
– Learns features and tasks directly from data
– More Data = better model
![Page 9: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/9.jpg)
9
Deep Learning Uses a Neural Network Architecture
Input
Layer Hidden Layers (n)
Output
Layer
![Page 10: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/10.jpg)
10
Deep Learning Datatypes
SignalImage
TextNumeric
![Page 11: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/11.jpg)
11
Deep Learning Workflow
Ground truth labeling
Train ModelPrepare Data
Model exchange
across frameworks
Model design,
Hyperparameter
tuning
Data access and
preprocessing
Deploy
Enterprise
Deployment
Edge deployment
Multiplatform code
generation (CPU, GPU)
Hardware-
accelerated training
![Page 12: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/12.jpg)
12
Why MATLAB for A.I. Tasks?
Increased productivity with interactive tools
Generate simulation data for complex models and systems
Ease of deployment and scaling to various platforms
Full A.I. workflows that cannot be easily
replicated by other toolchains
![Page 13: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/13.jpg)
13
Why MATLAB for A.I. Tasks?
Increased productivity with interactive tools
Full A.I. workflows that cannot be easily
replicated by other toolchains
Labeling TrainingModel
Exchange
![Page 14: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/14.jpg)
14
Labeling for deep learning is repetitive,
tedious, and time-consuming…
but necessary
![Page 15: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/15.jpg)
15
![Page 16: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/16.jpg)
16
![Page 17: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/17.jpg)
17
![Page 18: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/18.jpg)
18
Signal Labeler – annotate signals with labels/sublabels, export to workspace for training
Define
Labels
Interactively
Label Signals
View
properties
of labels
![Page 19: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/19.jpg)
19
User Story – Veoneer (Autoliv)
▪ Automotive
– Software and hardware for active safety,
autonomous driving, occupant protection,
and brake control
▪ Building radar sensor – check
accuracy using LiDAR-based
verification
▪ Human analyzes hours of recorded
data
▪ Used MATLAB to semi-automate
labeling and tracking of 3D LiDAR
point clouds.
![Page 20: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/20.jpg)
20
Manual Labeling for 25 events took over 20 minutes.
After full automation with MATLAB’s tools, it took 5 minutes
![Page 21: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/21.jpg)
21
Use Deep Network Designer to Create Networks
![Page 22: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/22.jpg)
22
Transfer Learning with Pre-trained Models
Inception-v3Inception-
ResNet-v2
ResNet-18
SqueezeNet ResNet-50
DenseNet-201
VGG-16
AlexNet
GoogLeNet
ResNet-101
VGG-19
![Page 23: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/23.jpg)
23
Model Exchange with MATLAB
PyTorch
Caffe2
MXNet
Core ML
CNTK
Keras-
Tensorflow
Caffe
MATLABONNX
Open Neural Network Exchange
(…)
![Page 24: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/24.jpg)
24
Why MATLAB for A.I. Tasks?
Increased productivity with interactive tools
Generate simulation data for complex models and systems
Ease of deployment and scaling to various platforms
Full A.I. workflows that cannot be easily
replicated by other toolchains
![Page 25: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/25.jpg)
25
Why MATLAB for A.I. Tasks?
Generate simulation data for complex models and systems
Full A.I. workflows that cannot be easily
replicated by other toolchains
Reinforcement
Learning
![Page 26: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/26.jpg)
26
Unsupervised
Learning[No Labeled Data]
Supervised Learning
[Labeled Data]
Clustering Classification Regression
Machine Learning
Reinforcement Learning vs Machine Learning vs Deep Learning
Deep Learning
Supervised learning typically involves
feature extraction
Deep learning typically simplifies
feature extraction
![Page 27: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/27.jpg)
27
Unsupervised
Learning[No Labeled Data]
Supervised Learning
[Labeled Data]
Clustering Classification Regression
Deep Learning
Machine Learning
Reinforcement
Learning
[Interaction Data]
Decision
MakingControls
Reinforcement Learning vs Machine Learning vs Deep Learning
Reinforcement learning:
▪ Learning through trial & error
[interaction]
▪ It’s about learning a
behavior or accomplishing a
task
![Page 28: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/28.jpg)
28
What is Reinforcement Learning?
▪ What is Reinforcement
Learning?
– Type of machine learning
that trains an ‘agent’
through repeated
interactions with an
environment
▪ How does it work?
– Through a trial & error
process that uses a reward
system to maximize
success
![Page 29: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/29.jpg)
29
Reinforcement Learning enables the use of Deep Learning for
Controls and Decision Making Applications
A.I. Gameplay
Controls
Robotics
Autonomous driving
![Page 30: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/30.jpg)
30
How Does Reinforcement Learning Work?
AGENT
ENVIRONMENT
ACTION
REWARD
STATE
The goal of Reinforcement learning is for the agent to find an optimal algorithm for
performing a task
![Page 31: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/31.jpg)
32
Deep Networks are commonly found in the agent, because
they can model complex problems.
AGENT
• Turn left
• Turn right
• Brake
• Accelerate
![Page 32: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/32.jpg)
33
Reinforcement Learning Workflow
Train ModelGenerate Data Deployment
Developing reward
system to optimize
performance
Training agent to
perform task
Scenario Design
Simulation-based
data generation
Enterprise
Deployment
Edge deployment
Reinforcement learningMultiplatform code
generation (CPU,
GPU)
Simulink –
generate data for
dynamic systems
(planes, cars,
robots, etc.)
Ground truth labeling
Prepare Data
Data access and
preprocessing
![Page 33: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/33.jpg)
34
Why MATLAB and Simulink for Reinforcement Learning?
Virtual models allow you to simulate conditions
hard to emulate in the real world.
![Page 34: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/34.jpg)
36
Why MATLAB for A.I. Tasks?
Increased productivity with interactive tools
Ease of deployment and scaling to various platforms
Generate simulation data for complex models and systems
Full A.I. workflows that cannot be easily
replicated by other toolchains
![Page 35: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/35.jpg)
37
Why MATLAB for A.I. Tasks?
Full A.I. workflows that cannot be easily
replicated by other toolchains
Code
GenerationEmbedded
Devices
Enterprise
Scaling
Increased productivity with interactive tools
Ease of deployment and scaling to various platforms
Generate simulation data for complex models and systems
![Page 36: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/36.jpg)
38
Deployment and Scaling for A.I.
Embedded Devices Enterprise
MATLAB
![Page 37: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/37.jpg)
39
Enterprise Scaling
Run thousands of simulations in parallel with MATLAB Parallel Server to save hours of
training time.
![Page 38: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/38.jpg)
40
Embedded Devices – Automatic Code Generation
MATLAB Code Deployment
Target
Auto-generated
Code
(C/C++/CUDA)
![Page 39: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/39.jpg)
41
Deploying Deep Learning Models for Inference
Coder
Products
Deep Learning
Networks
NVIDIA
TensorRT &
cuDNN
Libraries
ARM
Compute
Library
Intel
MKL-DNN
Library
![Page 40: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/40.jpg)
42
GPU Coder is fast
Intel® Xeon® CPU 3.6 GHz - NVIDIA libraries: CUDA10 - cuDNN 7 - Frameworks: TensorFlow 1.13.0, MXNet 1.4.0 PyTorch 1.0.0
GPU Coder is faster
than TensorFlow,
MXNet and Pytorch
TensorFlow
MXNet
GPU Coder
PyTorch
![Page 41: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/41.jpg)
44
Musashi Seimitsu Industry Co.,Ltd.Detect Abnormalities in Automotive Parts
MATLAB use in project:
▪ Preprocessing of captured images
▪ Image annotation for training
▪ Deep learning based analysis
– Various transfer learning methods
(Combinations of CNN models, Classifiers)
– Estimation of defect area using Class Activation Map
(CAM)
– Abnormality/defect classification
▪ Deployment to NVIDIA Jetson using GPU CoderAutomated visual inspection of 1.3 million
bevel gear per month
![Page 42: Deep Learning and Reinforcement Learning Workflows in A.I. · Increased productivity with interactive tools Generate simulation data for complex models and systems Ease of deployment](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5ebaee7e2c82607890704d1e/html5/thumbnails/42.jpg)
45
Why MATLAB for A.I. Tasks?
Increased productivity with interactive tools
Generate simulation data for complex models and systems
Ease of deployment and scaling to various platforms
Full A.I. workflows that cannot be easily
replicated by other toolchains