adam grzywaczewski | nvidia · 2019-04-26 · about me - deep learning solution architect @ nvidia...
TRANSCRIPT
![Page 1: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/1.jpg)
1
Developing Perception Algorithms for Self-Driving CarsADAM GRZYWACZEWSKI | NVIDIA
Original content by Clement Farabet
![Page 2: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/2.jpg)
ABOUT ME
- Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions
- 10 years expereince deliverying Machine Learning of all scale (from embedded, mobile to Big Data)
- My past experience:
- Capgemini: https://goo.gl/MzgGbq
- Jaguar Land Rover Research: https://goo.gl/ar7LuU
Adam Grzywaczewski – [email protected]
![Page 3: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/3.jpg)
3
CONTEXT
![Page 4: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/4.jpg)
![Page 5: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/5.jpg)
NVIDIA AGXEMBEDDED AI HPC
High-speed SerDes — 109 Gbps + 320 Gbps I/O
Up to 320 TOPS Tensor Ops
Up to 25 TFLOPS FP32
Up to 16 GIGA Rays
Starting from 15W
![Page 6: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/6.jpg)
End to End System for AV
Sources: NVIDIA, RAND Corporation
SIMULATETRAIN MODELSCOLLECT DATA RESIMULATE
Lanes Lights
Path
Signs
PedestriansCars
1 PB+ per car per year10+ DNNs
1M+ images per DNN
10B+ miles to
ensure safe driving
Validate and verify
for self-driving cars
by 2020
MAPPING
5+ DNNs
Create and update
HD maps
![Page 7: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/7.jpg)
7
DRIVE AR
DRIVE IX
DRIVE AV
DRIVE OS
Lidar
Localization
Surround Perception
RADAR LIDAR
Egomotion
LIDAR Localization Path Perception
Path PlanningCamera LocalizationLanes Signs Lights
Trunk Opening Eye Gaze Distracted Driver Drowsy Driver Cyclist Alert
CGTrackDetect
NVIDIA DRIVE: SOFTWARE-DEFINED CARPowerful and Efficient AI, CV, AR, HPC | Rich Software Development Platform
Functional Safety | Open Platform | 370+ partners developing on DRIVE
DRIVE AGX XAVIERDRIVE AGX PEGASUS
![Page 8: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/8.jpg)
NVIDIA DRIVE PLATFORM ADOPTION ACROSS TRANSPORTATION
CARS
MOBILITY SERVICES
TRUCKS
TIER ONES
MAPPING
SPECIALTY
![Page 9: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/9.jpg)
9
Artificial Intelligence Computer Vision Accelerated Computing Multimedia
Gesture recObj detectPath planningDepth est
NVIDIA JETSONSOFTWARE-DEFINED AUTONOMOUS MACHINES
Powerful and efficient AI, CV, HPC | Rich Software Development Platform Open Platform | 200K Developers
Pose est Speech rec
Sensors
JETSON COMPUTER
JETPACK SDK
ACCELERATED MODULES
ECOSYSTEM
BSP • Linux • Security Architecture
AI/Systems Software Design Services
![Page 10: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/10.jpg)
Industrial Aerospace/Defense Healthcare Construction Agriculture Smart City
Inspection ServiceRetail Logistics Inventory Mgmt Delivery
BILLIONS OF AUTONOMOUS MACHINESTaking advantage of the progress in self driving cars
![Page 11: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/11.jpg)
DEEP LEARNING
![Page 12: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/12.jpg)
NEURAL NETWORKS ARE NOT NEWAnd are surprisingly simple as an algorithm
![Page 13: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/13.jpg)
NEURAL NETWORKS ARE NOT NEWThey just historically never worked well
0 5 10 15 20 25
Algorithm performance in small data regime
ML1Dataset Size
Accura
cy
Andrew Ng, “Nuts and Bolts of Applying Deep Learning”, https://www.youtube.com/watch?v=F1ka6a13S9I
![Page 14: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/14.jpg)
NEURAL NETWORKS ARE NOT NEWThey just historically never worked well
Dataset Size
Accura
cy
0 5 10 15 20 25
Algorithm performance in small data regime
ML1 ML2 ML3
Andrew Ng, “Nuts and Bolts of Applying Deep Learning”, https://www.youtube.com/watch?v=F1ka6a13S9I
![Page 15: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/15.jpg)
NEURAL NETWORKS ARE NOT NEWThey just historically never worked well
Dataset Size
Accura
cy
0 5 10 15 20 25
Algorithm performance in small data regime
Small NN ML1
ML2 ML3
Andrew Ng, “Nuts and Bolts of Applying Deep Learning”, https://www.youtube.com/watch?v=F1ka6a13S9I
![Page 16: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/16.jpg)
NEURAL NETWORKS ARE NOT NEWHistorically we never had large datasets or computers
Dataset Size
Accura
cy
0 5 10 15 20 25
Algorithm performance in small data regime
Small NN ML1
ML2 ML3
The MNIST (1999) database
contains 60,000 training images
and 10,000 testing images.
Andrew Ng, “Nuts and Bolts of Applying Deep Learning”, https://www.youtube.com/watch?v=F1ka6a13S9I
![Page 17: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/17.jpg)
COMPUTEHistorically we never had large datasets or compute
1980 1990 2000 2010 2020
102
103
104
105
106
107
Single-threaded perf
1.5X per year
1.1X per yearTransistors
(thousands)
GPU-Computing perf
1.5X per year 1000X
By 2025
![Page 18: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/18.jpg)
CONTEXT1.759 petaFLOPs in November 2009
![Page 19: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/19.jpg)
CONTEXT2 petaFLOPs - today
![Page 20: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/20.jpg)
NEURAL NETWORKS ARE NOT NEWBut that changed and transformed the way we do machine learning
Dataset Size
Accura
cy
0 50 100 150 200 250
Algorithm performance in big data regime
Small NN ML1
ML2 ML3
Andrew Ng, “Nuts and Bolts of Applying Deep Learning”, https://www.youtube.com/watch?v=F1ka6a13S9I
![Page 21: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/21.jpg)
NEURAL NETWORKS ARE NOT NEWData and model size the key to accuracy
Dataset Size
Accura
cy
0 50 100 150 200 250 300 350 400 450
Algorithm performance in big data regime
Small NN ML1 ML2 ML3 Big NN
![Page 22: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/22.jpg)
2016 – Baidu Deep Speech 2Superhuman Voice Recognition
2015 – Microsoft ResNetSuperhuman Image Recognition
2017 – Google Neural Machine TranslationNear Human Language Translation
100 ExaFLOPS8700 Million Parameters
20 ExaFLOPS300 Million Parameters
7 ExaFLOPS60 Million Parameters
To Tackle Increasingly Complex Challenges
NEURAL NETWORK COMPLEXITY IS EXPLODING
![Page 23: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/23.jpg)
100 EXAFLOPS=
2 YEARS ON A DUAL CPU SERVER
![Page 24: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/24.jpg)
NEURAL NETWORKS ARE NOT NEWExceeding human level performance
Dataset Size
Accura
cy
0 500 1000 1500 2000 2500
Algorithm performance in large data regime
Small NN ML1 ML2 ML3 Big NN Bigger NN
Andrew Ng, “Nuts and Bolts of Applying Deep Learning”, https://www.youtube.com/watch?v=F1ka6a13S9I
![Page 25: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/25.jpg)
EXPLODING DATASETSLogarithmic relationship between the dataset size and accuracy
Sun, Chen, et al. "Revisiting Unreasonable Effectiveness of Data in Deep Learning Era." arXiv preprint arXiv:1707.02968 (2017).
Shazeer, Noam, et al. "Outrageously large neural networks: The sparsely-gated mixture-of-experts layer." arXiv preprint arXiv:1701.06538 (2017).
Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014).
![Page 26: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/26.jpg)
EXPLODING DATASETS
Hestness, J., Narang, S., Ardalani, N., Diamos, G., Jun, H., Kianinejad, H., ... & Zhou, Y. (2017). Deep Learning Scaling is Predictable, Empirically. arXiv preprint arXiv:1712.00409.
Logarithmic relationship between the dataset size and accuracy
• Translation
• Language Models
• Character Language Models
• Image Classification
• Attention Speech Models
![Page 27: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/27.jpg)
EXPLODING DATASETS
Hestness, J., Narang, S., Ardalani, N., Diamos, G., Jun, H., Kianinejad, H., ... & Zhou, Y. (2017). Deep Learning Scaling is Predictable, Empirically. arXiv preprint arXiv:1712.00409.
Logarithmic relationship between the dataset size and accuracy
![Page 28: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/28.jpg)
Making complex problems easy
![Page 29: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/29.jpg)
Making unsolvable problems
expensive
![Page 30: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/30.jpg)
![Page 31: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/31.jpg)
PERCEPTION ALGORITHMS
![Page 32: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/32.jpg)
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
BUILDING AI FOR SDC IS HARD
Every neural net in our DRIVE
Software stack needs to
handle 1000s of conditions
and geolocations
HazardsAnimalsBicyclesPedestrians
Backlit
Snow
Vehicles
Day
Clear FogRainCloudy
Street
LampsNightTwilight
![Page 33: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/33.jpg)
— Target robustness per model(miles)
–– Test dataset size required(miles)
— NVIDIA’s ongoing data collection(miles)
DATA AND INFERENCE TO GET THERE?
30PB
60PB
120PB
180PB
Real-time test runs in 24h
on 400 Nodes*
24h test
on 1,600 Nodes*
24h test
on 3,200 Nodes*
* DRIVE PEGASUS Nodes
WHAT TESTING SCALE ARE WE TALKING ABOUT?We’re on our way to 100s PB of real test data = millions of real miles
+ 1,000s DRIVE Constellation nodes for offline testing alone
& billions of simulated miles
15PB
![Page 34: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/34.jpg)
SDC SCALE TODAY AT NVIDIA
1PB collected/week
12-camera+Radar+Lidar
RIG mounted on 30 cars
4,000 GPUs in cluster
= 500 PFLOPs1,500 labelers
20M objects labeled/mo100 DRIVE
Pegasus in cluster
(Constellations)
15PB active training+test
dataset
20 unique models
50 labeling tasks
1PB of in-rack object
cache per 72 GPUs,
30PB provisioned
![Page 35: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/35.jpg)
TRAINING
![Page 36: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/36.jpg)
“For any size of the data it’s a
good idea to always make the
data look small by using a huge
model.”
Geoffrey Hinton
![Page 37: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/37.jpg)
EXPLODING MODEL COMPLEXITYGood news – model size scales sublinearly
Hestness, J., Narang, S., Ardalani, N., Diamos, G., Jun, H., Kianinejad, H., ... & Zhou, Y. (2017). Deep Learning Scaling is Predictable, Empirically. arXiv preprint arXiv:1712.00409.
Bes
t M
od
el S
ize
Amount of Data
Model Size Scales Sublinearly
![Page 38: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/38.jpg)
EVIDENCE FROM IMAGE PROCESSINGGood news – model size scales sublinearly
Zoph, Barret, et al. "Learning transferable architectures for scalable image recognition." arXiv preprint arXiv:1707.07012 (2017).
![Page 39: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/39.jpg)
IMPLICATIONSExperimental Nature of Deep Learning - Unacceptable training time
Idea
Code
Experiment
![Page 40: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/40.jpg)
IMPLICATIONSAutomotive example
Majority of useful problems are too complex for a single GPU training
VERY CONSERVATIVE CONSERVATIVE
Fleet size (data capture per hour) 100 cars / 1TB/hour 125 cars / 1.5TB/hour
Duration of data collection 260 days * 8 hours 325 days * 10 hours
Data Compression factor 0.0005 0.0008
Total training set 104 TB 487.5 TB
InceptionV3 training time (with 1 Pascal GPU)
9.1 years 42.6 years
AlexNet training time (with 1 Pascal GPU)
1.1 years 5.4 years
![Page 41: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/41.jpg)
CONCLUSIONSWhat does your team do in the mean time
Xkcd.com
![Page 42: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/42.jpg)
CONCLUSIONSWhat does your team do in the meantime?
Xkcd.com
![Page 43: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/43.jpg)
CONCLUSIONSNeed to scale the training process for a single job
VERY CONSERVATIVE
CONSERVATIVE
Total training set 104 TB 487.5 TB
InceptionV3 (one DGX-1V)
166 days
(5+ months)778 days
(2+ years)
AlexNet(one DGX-1V)
21 days
(3 weeks)
98 days
(3 months)
InceptionV3 (10 DGX-1V’s)
16 days
(2+ weeks)
77 days
(11 weeks)
AlexNet(10 DGX-1V’s)
2.1 days 9.8 days
10 NVIDIA DGX-1’s
1 NVIDIA DGX-1
Training From
Months or Years
ToWeeks or Days
![Page 44: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/44.jpg)
BALANCED HARDWAREDGX-1 as a reference point for solution design
![Page 45: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/45.jpg)
DGX PODReference architectures
- Hiding the complexity of hardware
- Hiding complexity of devops- Hiding the complexity of software
toolkit management- Partner reference architectures
with every major storage provider
![Page 46: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/46.jpg)
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
TRAINING
![Page 47: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/47.jpg)
SUMMIT27360 Tesla V100
Kurth, T., Treichler, S., Romero, J., Mudigonda, M., Luehr, N., Phillips, E., ... & Houston, M. (2018, November). Exascale deep learning for climate
analytics. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis (p. 51). IEEE Press.
![Page 48: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/48.jpg)
ITERATION TIME
1740
264
60 48 4024
15
3.7
1
10
100
1000
10000
Microsoft(Dec 2015)
PreferedNetoworks
(Feb 17)
Facebook(Jun 17)
IBM (Aug 17) SURFsara(Sep 17)
UCB (Oct 17) PreferredNetoworks
(Nov 17)
Sony (Nov18)
ResNet 50 Training Time in minutes
Short iteration time is fundamental for success
![Page 49: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/49.jpg)
VALIDATION
![Page 50: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/50.jpg)
50
SAFE AV REQUIRES A COMPREHENSIVE VALIDATION APPROACH
Large Scale | Millions of Miles
Diverse Vehicle and World Conditions
Data Driven | Scenario based
Repeatable and Reproducible
End-to-End System Level Test
![Page 51: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/51.jpg)
51
COMPONENT LEVEL SILLow Fidelity | Scalable
ON ROAD TESTINGHigh Fidelity| Doesn’t Scale
THE AV VALIDATION GAP
No Coverage for Extreme & Dangerous Scenarios
![Page 52: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/52.jpg)
52
NVIDIA DRIVE VALIDATION METHODOLOGYThree Pronged Approach
FID
ELIT
Y
Large Scale System LevelHIL
On Road Testing
Component LevelSIL
MILES
![Page 53: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/53.jpg)
53
HARDWARE IN THE LOOP SIMULATIONBit Accurate & Timing Accurate
CONTROLSteering | Throttle | Brake
PERCEPTIONCamera | Radar | Lidar | IMU
![Page 54: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/54.jpg)
54
DRIVE CONSTELLATION ARCHITECTURE
DRIVE Constellation Simulator
DRIVE Sim API
DRIVE Constellation Vehicle
Environment
Model
DRIVE Constellation OS
DRIVE Sim
Traffic
Model
Vehicle
Model
Sensor
Model
Scenario
Model
DRIVE OS
DRIVE CORE | DRIVE NETWORKS
DRIVE AV
DRIVE IXPerception Mapping Planning
![Page 55: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/55.jpg)
55
ENVIRONMENT MODELHighly Detailed Environments
![Page 56: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/56.jpg)
56
ENVIRONMENT MODELChange Time of Day
![Page 57: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/57.jpg)
57
ENVIRONMENT MODELAdd Traffic Scenarios
![Page 58: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/58.jpg)
58
ENVIRONMENT MODELChange Weather
![Page 59: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/59.jpg)
59
DRIVE CONSTELLATIONFOR 3RD PARTY AV VALIDATION
Open Platform | Wide Ecosystem Support
Cloud Based Solution | Scalable
Accessible to OEMs and Researchers
Demonstrate Best Practice for AV Validation
Open | Accessible | Available at Scale
![Page 60: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/60.jpg)
60
THE DRIVE SIM ECOSYSTEM
ENVIRONMENT MODELMAPPING VEHICLE DYNAMICS SYSTEM INTEGRATORS
TRAFFIC MODELSENSOR MODEL SCENARIO MODEL 3rd PARTY CERTIFICATION
![Page 61: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/61.jpg)
61
Partnerships with leading safety organizations
Public | Private | Worldwide
Creating best practices and standards
BEST PRACTICES FOR AV VALIDATIONPlanning the Path to Safety
![Page 62: ADAM GRZYWACZEWSKI | NVIDIA · 2019-04-26 · ABOUT ME - Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions - 10 years expereince deliverying](https://reader033.vdocuments.mx/reader033/viewer/2022053014/5f132859170c590ba2223629/html5/thumbnails/62.jpg)
62
THANK YOUtwitter.com/nvidiaAI