embedded deep learning with nvidia jetson
TRANSCRIPT
![Page 1: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/1.jpg)
![Page 2: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/2.jpg)
Webinar Agenda
Topic:
• Demystifying Deep Learning
• NVIDIA Tools & SDKs
• Deploying with Jetson
• Deep Vision Primitives
• 2 Days To A Demo
• Reinforcement Learning
• Simulation
• Conclusion / Q&A
![Page 3: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/3.jpg)
“ Billions of intelligent devices will take advantage of DNNs to provide personalization and localization as GPUs become faster and faster over the next several years.”
— Tractica
INTELLIGENT MACHINES
![Page 4: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/4.jpg)
Train Model
Robust Validation
Configure Network
Turing-Complete
Runtime Inference
Onboard Intelligence
A NEW COMPUTING MODEL
![Page 5: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/5.jpg)
ERA of LEARNING
Deep Learning
Power Efficiency
Krizhevsky et al., NIPS 2012. ImageNet Classification with Deep Convolutional Neural Networks.
Image Classification Rate
![Page 6: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/6.jpg)
Microsoft “Super Deep Network”
Berkeley’s Brett End-to-End
Reinforcement Learning
Deep Speech 2 One network, 2 languages
DeepMind AlphaGo Rivals a World Champion TU Delft Deep-Learning
Amazon Picking Champion
MILESTONES IN AI
Microsoft & Google “Superhuman” Image Recognition (ResNet)
WEpod Autonomous People Shuttle On the Road Starship Delivery
Robot Enters Service
2014-2016
![Page 7: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/7.jpg)
PC GAMING
ONE ARCHITECTURE — END-TO-END AI
CUDA + Linux throughout the stack.
ONE ARCHITECTURE — END-TO-END AI
![Page 8: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/8.jpg)
8
POWERING THE DEEP LEARNING ECOSYSTEM NVIDIA SDK accelerates every major framework
COMPUTER VISION
OBJECT DETECTION IMAGE CLASSIFICATION
SPEECH & AUDIO
VOICE RECOGNITION LANGUAGE TRANSLATION
NATURAL LANGUAGE PROCESSING
RECOMMENDATION ENGINES SENTIMENT ANALYSIS
DEEP LEARNING FRAMEWORKS
Mocha.jl
NVIDIA DEEP LEARNING SDK
developer.nvidia.com/deep-learning-software
DIGITS
![Page 9: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/9.jpg)
Test Image
Monitor Progress Configure DNN Process Data Visualize Layers
Interactive Training with DIGITS
github.com/NVIDIA/DIGITS
github.com/NVIDIA/DIGITS
![Page 10: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/10.jpg)
Classified Object!
Trained Deep Neural Net Model
Camera Inputs
X
Training: NVIDIA DIGITS Inference: NVIDIA Jetson TX1
TRAINING
DATA MANAGEMENT
MODEL VALIDATION
DGX-1
TESLA
TITAN-X
cuDNN TensorRT
DIGITS Workflow
![Page 11: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/11.jpg)
JETSON TX1 EMBEDDED AI SUPERCOMPUTER
10W | 1 TF FP16 | >20 images/sec/W
![Page 12: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/12.jpg)
JetPack 2.3
Software Components
Linux4Tegra R24.2 Ubuntu 16.04 LTS aarch64
Multimedia API SDK Low-level camera/codec
Runtime Inference TensorRT RC1 + FP16
CUDA Toolkit 8 cuDNN v5.1, cuGRAPH
VisionWorks 1.5.2.14 OpenCV4Tegra 2.4.13.17
OpenGL 4.5.0 EGL 1.4 / OpenGL ES 3.1
Tegra System Profiler Tegra Graphics Debugger
Double Performance, 20x Efficiency
developer.nvidia.com/jetpack
![Page 13: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/13.jpg)
DIGITS OPTIMIZATION ENGINE
EXECUTION ENGINE
PLAN NEURAL NETWORK
TensorRT
• Fuse network layers
• Eliminate concatenation layers
• Kernel specialization
• Auto-tuning for target platform
• Select optimal tensor layout
• Batch size tuning TRAINED
NEURAL NETWORK
OPTIMIZED INFERENCE RUNTIME
developer.nvidia.com/TensorRT
![Page 14: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/14.jpg)
Object Detection Localization
Image Recognition Classification
Segmentation Free Space
DEEP VISION
Registration SLAM
3D ShapeNet
![Page 15: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/15.jpg)
GETTING STARTED JETSON COMMUNITY
Developer Forums devtalk.nvidia.com eLinux Wiki eLinux.org/Jetson_TX1
![Page 16: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/16.jpg)
16
developer.nvidia.com/embedded
COMPREHENSIVE DEVELOPER S ITE
• JetPack SDK
• Libraries
• Developer tools
• Design collateral
• Developer Forum
• Training and Tutorials
• Ecosystem Partners
![Page 17: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/17.jpg)
Hands-On Labs
nvidia.com/DLI
nvidia.com/DLI
Self-Paced Courses Nanodegree Programs
![Page 18: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/18.jpg)
PARTNERSHIP Latest Technology GPU/DL Experts Global Sales & Marketing Network Training from Deep Learning Institute
INCEPTION PROGRAM
CONNECTING WITH AI STARTUPS
www.nvidia.com/inception
![Page 19: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/19.jpg)
2 DAYS TO A DEMO
Ten Steps to Deep Learning
1. Install DIGITS on PC
2. Get JetPack 2.3 & TensorRT
3. Build the Vision Source
4. Dig Into the Example Code
5. Classify Images with ImageNet
6. Run the Camera Recognition Demo
7. Re-train the Network with DIGITS
8. Locate Object Coordinates - DetectNet
9. Run the Live Camera Detection Demo
10. Re-train DetectNet with DIGITS
github.com/dusty-nv/jetson-inference makezine.com/projects/jetson
Project includes: 1000 types of objects
pedestrian DetectNet
Robot OS (ROS) nodes
![Page 20: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/20.jpg)
HORUS SOCIAL INNOVATION
AWARD WINNER
BLIND
ASSISTANCE
DEVICE
![Page 21: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/21.jpg)
AERIALTRONICS
![Page 22: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/22.jpg)
CNN
CNN
Sensory Inputs
CNN
Online L
earn
ing
MOTION CONTROL | BEHAVIOR IMITATION
PATH PLANNING | AUTONOMOUS NAVIGATION
Actions
feature maps
confidence values
Pixels to Actions
backpro
pagati
on
Experience Actuator
CNN RNN/LSTM
Loss/Reward
Solver
REINFORCEMENT LEARNING
![Page 23: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/23.jpg)
OpenAI Gym
Gazebo
GTA-V
Unreal4Torch
OpenRAVE
TORCS
Physical Intuition
SIMULATION
![Page 24: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/24.jpg)
LSTM ACCELERATION
cuDNN v5 - GPU-accelerated RNNs/LSTM
Forms internal working memory cells
Partially-observable inputs/environments
Launch a 2D grid of RNN cells
Supports:
Uni/Bidirectional RNNs
Non-uniform length minibatches
Dropout between layers
![Page 25: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/25.jpg)
Jetson TX1 Developer Kit
Jetson EDU Program available
www.nvidia.com/embedded
![Page 26: Embedded Deep Learning with NVIDIA Jetson](https://reader034.vdocuments.mx/reader034/viewer/2022052514/587ce90e1a28ab564b8b4c2b/html5/thumbnails/26.jpg)
THANK YOU! Q&A: WHAT CAN I HELP YOU BUILD?
2 Days To a Demo github.com/dusty-nv/jetson-inference
Reinforcement Learning github.com/dusty-nv/jetson-reinforcement
Access these Slides github.com/dusty-nv/jetson-presentations
Post Discussion devtalk.nvidia.com/default/topic/969035
eLinux TX1 Wiki eLinux.org/Jetson_TX1