an overview of nvidia’s autonomous vehicles …...“part of overall safety relating to the...
TRANSCRIPT
![Page 1: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/1.jpg)
Pradeep Kumar Gupta | August 2017
Global Head – Solutions Architecture and Engineering-Autonomous Driving
AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES PLATFORM
![Page 2: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/2.jpg)
2
AGENDA
Autonomous Vehicles
Functional Safety
Levels of Automation
DRIVE PX Platform
Development Workflow
![Page 3: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/3.jpg)
3
NVIDIA
#1 PC Gaming #1 Pro Graphics #1 Accelerated Computing #1 AI Computing
Founded in 1993 — 11,000 Employees Invented the GPU Invented GPU-accelerated Computing
Fastest Supercomputers in Japan, U.S., Europe Pioneering AI Car Computer Nintendo Switch
![Page 4: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/4.jpg)
4
AUTONOMOUS VEHICLE
![Page 5: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/5.jpg)
6
AI REVOLUTIONIZING TRANSPORTATION
280B Miles per year 800M parking spots for 250M cars in the U.S.
Domino’s: 1M Pizzas delivered per Day
![Page 6: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/6.jpg)
7
COLLISIONS
CRITICAL REASONATTRIBUTED TO
ESTIMATED
NUMBERPERCENTAGE*
±95% CONF. LIMITS
Drivers 2,046,000 94% ±2.2%
Vehicles 44,000 2% ±0.7%
Environment 52,000 2% ±1.3%
Unknown Critical Reasons
47,000 2% ±1.4%
Total 2,189,000 100%
* Percentages are based on unrounded estimated frequencies (Data Source: NMVCCS 2005-2007)
CRITICAL REASON
ESTIMATED (Based on 94% of the NMVCCS Crashes)
NUMBERPERCENTAGE*
±95% CONF. LIMITS
Recognition Error 845,000 41% ±2.2%
Decision Error 684,000 33% ±3.7%
Performance Error 210,000 11% ±2.7%
Non-Performance Error (sleep, etc.)
145,000 7% ±1.0%
Other 162,000 8% ±1.9%
Total 2,046,000 100%
* Percentages are based on unrounded estimated frequencies (Data Source: NMVCCS 2005-2007)
![Page 7: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/7.jpg)
8
32
34
36
38
40
42
44
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
MOTOR VEHICLE FATALITY ESTIMATESNumber of deaths in thousands per year
Year
Source: National Safety Council: http://www.nsc.org/NewsDocuments/2017/12-month-estimates.pdf
![Page 8: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/8.jpg)
9
FUNCTIONAL SAFETY
“Part of overall safety relating to the equipment under control and the
control system that depends on the correct functioning of the
electrical, electronic, and programmable electronic safety-related
systems and other risk reduction measures”
–IEC 61508-4:2010; 3.1.12
“Absence of unreasonable risk due to hazards caused by malfunctioning
behavior of electrical/electronic systems”
–ISO 26262-1:2011; 1.51
![Page 9: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/9.jpg)
10
FUNCTIONAL SAFETY
![Page 10: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/10.jpg)
11
FUNCTIONAL SAFETY
ASILs identified by Standard - ASIL A, ASIL B, ASIL C, ASIL D.
ASIL D dictates the highest integrity requirements on the product and ASIL A the lowest.
ASIL’s are 3D with 3 variables: severity, probability of exposure, and controllability.
ISO 26262-3, section 7 “Hazard analysis and risk assessment” provides tables that break these three variables down into classes.
Probability of exposure has five classes: “Incredible” to “High probability” (E0-E4).
Severity has four classes: “No injuries” to “Life-threatening injuries (survival uncertain), fatal injuries” (S0-S3).
ASIL Standard
![Page 11: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/11.jpg)
12
FUNCTIONAL SAFETY
Controllability, which means controllability by the driver, not by the vehicle electronic systems, has four classes: “Controllable in general” to “Difficult to control or uncontrollable.”
http://www.electronicdesign.com/embedded/understanding-iso-26262-asils
http://www.eetimes.com/document.asp?doc_id=1331459&page_number=4
Why ASIL D now?
If your car meets the ISO 26262 specification for ASIL D, it means that the machine [car] is “making a decision for you” when it comes across a safety-critical path, explained Raucher. At ASIL B, the car would warn a driver of imminent danger, but at ASIL D, the car — faced with a hazard — brakes and stops or pulls into a safe space.”
ASIL Standard
![Page 12: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/12.jpg)
13
LEVELS OF AUTOMATION
Source: SAE International – Levels of Driving Automation
![Page 13: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/13.jpg)
15
150M AUTONOMOUS VEHICLES BY 2025
Source: ABI
2017 2018 2019 2020 2021 2022 2023 2024 2025
—130M L2
—15M L3
—5M L4
![Page 14: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/14.jpg)
17
Artificial IntelligenceAutonomous Vehicles
NVIDIA AND DEEP LEARNINGAt the Center of AV Revolution
![Page 15: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/15.jpg)
18
DEEP LEARNING AUTOMOTIVE PARTNERS
![Page 16: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/16.jpg)
19
AUTONOMOUS VEHICLES
It is quite clear that putting Autonomous Vehicles on the road is possible only with Artificial Intelligence. AV = AI.
The key challenge with Autonomous Vehicles is that we need an AI super computer in the car.
And, we brought the DRIVE Computer to address the challenge of Autonomous Vehicles
Key summary points
![Page 17: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/17.jpg)
20
DRIVE PLATFORMLevel 2 -> Level 5
1 TOPS
10 TOPS
100 TOPS
DRIVE PX 2 ParkerLevel 2/3
DRIVE PX XavierLevel 4/5
DRIVE PX 2
2 PARKER + 2 PASCAL GPU | 20 TOPS DL | 120 SPECINT | 80W
DRIVE PX (Xavier)
30 TOPS DL | 160 SPECINT | 30W
ONE ARCHITECTURE
![Page 18: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/18.jpg)
22
2 Complete AI SystemsPascal Discrete GPU
1,280 CUDA Cores4 GB GDDR5 RAM
Parker SOC Complex256 CUDA Cores4 Cortex A57 Cores2 NVIDIA Denver2 Cores8 GB LPDDR4 RAM64 GB Flash
Safety MicroprocessorAURIX Safety Microprocessor
DRIVE PX 2 COMPUTE COMPLEXES
14
![Page 19: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/19.jpg)
23
IS IT THAT SIMPLE?
![Page 20: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/20.jpg)
24
NVIDIA DRIVE – SOFTWARE
Computer Vision Libraries
OS
Perception AI
CUDA, cuDNN, TensorRT
Localization Path Planning
![Page 21: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/21.jpg)
25
DRIVE PLATFORM
DriveWorks
CUDA, TensorRT
DRIVE PX 2 — System Software
DRIVE PX 2 — System Hardware Vehicle Control
DNNs and Autonomous Driving Applications
Sensors
HD Maps
![Page 22: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/22.jpg)
26
Modular
Scalable
Optimized for GPU
Rapid prototyping & production
DESIGN PHILOSOPHY
APIs(> 450 AND GROWING)
USE CASES
DRIVEWORKS
Autonomous Driving
Data
Data Acquisition
![Page 23: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/23.jpg)
27
DRIVE PX 2
Linux PC
DEVELOP BENCHMARK/DEPLOY
Cross-Compile
Feedback & Iterate
GETTING STARTED…Workflow
![Page 24: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/24.jpg)
28
Sensors
Sensors
Trained Neural
Network Model
(DNN)
HD Maps
PC Based Algorithm /
App Development
Real / Simulated
Sensors
Real / Simulated
Vehicle
Data Acquisition &
Sensor Fusion
Data Acquisition &
Sensor Fusion
Neural Network
Training
Trained Neural
Network Model (DNN)
Trained Neural
Network Model
(DNN)
Hi Def Maps
Data Acquisition &
Sensor Fusion
Algorithms,
Applications
In-Vehicle Computing Platform
DIFFERENT WORKFLOWS
![Page 25: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/25.jpg)
29
APPLICATION DEVELOPMENT
Compute libraries
NN Inferencing
System level applications
DATA CURATION, LABELING
Manage data lifecycle
DNN TRAINING
Train NN with data
MAP CREATION
Create HD map
DATA ACQUISITION
Sensor integration
Sensor calibration
Image capture
Storage
SIMULATE, DRIVE, TEST
HIL/SIL simulation
Vehicle integration
Test
DEVELOPMENT FLOWSelf Driving Vehicle Development
DRIVE PX PC
![Page 26: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/26.jpg)
30
APPLICATION DEVELOPMENT
Compute libraries
NN Inferencing
System level applications
DATA CURATION, LABELING
Manage data lifecycle
DNN TRAINING
Train NN with data
MAP CREATION
Create HD map
DATA ACQUISITION
Sensor integration
Sensor calibration
Image capture
Storage
SIMULATE, DRIVE, TEST
HIL/SIL simulation
Vehicle integration
Test
DEVELOPMENT FLOWSelf Driving Vehicle Development
DRIVE PX PC
![Page 27: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/27.jpg)
31
DATA ACQUISITION
Sensor integration
Sensor calibration
Image capture
Storage
SIMULATE, DRIVE, TEST
HIL/SIL simulation
Vehicle integration
Test
DATA CURATION, LABELING
Manage data lifecycle
DNN TRAINING
Train NN with data
MAP CREATION
Create HD map
APPLICATION DEVELOPMENT
Compute libraries
NN Inferencing
System level applications
DEVELOPMENT FLOWSelf Driving Vehicle Development
DRIVE PX PC
![Page 28: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/28.jpg)
32
HOW MUCH DATA?
![Page 29: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/29.jpg)
34
TARGETED DATA COLLECTION
Learning to drive
Lane keeping, emergency braking, intersections
Perceiving your surroundings
Detect lane boundaries, drivable paths, cars, pedestrians
Summarizing your world-view to support others
Provide my understanding of this location to other vehicles
Updating the “base map”
Use Cases
Cam -60FOV
Cam- 120FOV
LIDAR
IMU/GPS
![Page 30: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/30.jpg)
35
Sensor fusion interfaces:GMSL camera, CAN, GbE, BroadR-Reach, FlexRay, LIN, GPIO
Displays interfacesHDMI, FPDlink III and GMSL
Storage interfaces10GbE, USB3
INTERFACES70 Gigabits per second of I/O
DRIVE PX 2
Tegra A (CPU+ integrated
GPU)
Pascal A (discrete GPU)
Tegra B(CPU+ integrated
GPU)
Pascal B (discrete GPU)
ASIL-D Safety
Microcontroller
CANCamera
GPIOsGigabit Ethernet
DisplayBroadR-Reach
LINUSB 3.0
FlexRayUSB 2.0
10 Gigabit EthernetGigabit Ethernet
Dispay (HDMI)JTAG
Gigabit Ethernet
Auto Grade Connectors Debug/Lab Interfaces
![Page 31: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/31.jpg)
36
FPDLinkVideo Out
GMSLVideo Out
Vehicle Harness
• 6 CAN• 4 LIN• 2 FlexRay• GbE• AURIX Debug
HDMI Parker AUSB Debug
10 GbE
HDMI Parker B
GbEUSB 3.0 Parker AUSB 3.0 Parker B
BroadR-Reach
Ethernet+ 8 GPIO
12 Cameras
• GMSL• Power over Coax
DRIVE PX 2 HARDWARE CONNECTIVITY
![Page 32: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/32.jpg)
37
DISTRIBUTED RECORDING
Multiple sensors; multiple DRIVE PX devices log synchronized data
Each DRIVE PX unit collecting data with a timestamp recording time (ms)
Data between units sync’d based on timestamps
DriveWorks Tools
![Page 33: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/33.jpg)
38
RECORDING TOOLDriveWorks Tools
Calibration & Sensor Registration
1. Route setting
2. Configuration switching
3. Storage info
4. Camera preview
5. Start Recordingcontrol button
6. Settings button7. GPS info8. Sensor Status
![Page 34: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/34.jpg)
39
CAMERA CALIBRATION & SENSOR REPLAY DriveWorks Tools
CALIBRATION & SENSOR REGISTRATION DATA REPLAY TOOL
Calibrate N
cameras
Cameras can be
pinhole or
fisheye
No restriction on
field of views
No manual
measurements
involved
Replays the sensor
data captured
Displays the data for
each sensor type
Provides a simple UI
for quick sanity checks
![Page 35: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/35.jpg)
40
DATA ACQUISITION
Sensor integration
Sensor calibration
Image capture
Storage
SIMULATE, DRIVE, TEST
HIL/SIL simulation
Vehicle integration
Test
DATA CURATION, LABELING
Manage data lifecycle
DNN TRAINING
Train NN with data
MAP CREATION
Create HD map
APPLICATION DEVELOPMENT
Compute libraries
NN Inferencing
System level applications
DRIVE PX PC
DEVELOPMENT FLOWSelf Driving Vehicle Development
![Page 36: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/36.jpg)
41
![Page 37: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/37.jpg)
42
DATA ACQUISITION
Sensor integration
Sensor calibration
Image capture
Storage
SIMULATE, DRIVE, TEST
HIL/SIL simulation
Vehicle integration
Test
DATA CURATION, LABELING
Manage data lifecycle
DNN TRAINING
Train NN with data
MAP CREATION
Create HD map
APPLICATION DEVELOPMENT
Compute libraries
NN Inferencing
System level applications
DRIVE PX PC
DEVELOPMENT FLOWSelf Driving Vehicle Development
![Page 38: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/38.jpg)
43
DRIVEWORKS MODULES
Sensing Maps
Vehicle IO*
Egomotion
Image Pipeline
Image Processing
Computer Vision
Renderer
Detection
Multi-Camera
Lidar
Radar
Map Birds-Eye View
SfM Stereo
Rectifier Color Correction
TrajectoryDNN Inference
![Page 39: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/39.jpg)
44
![Page 40: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/40.jpg)
45
AI CO-PILOT
Autonomous driving modes are OFF
Vehicle continues to be aware of its surroundings
Driver facing camera
Driver notification via:
Audio (sounds and language)
Visual (LEDs, icons, text, HUD)
Vibration (steering wheel, seat, seatbelt)
Adding Value When Humans Choose to Drive
![Page 41: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/41.jpg)
46
AI CO-PILOTConvenience
Driver Recognition
Route Memory
Driver Habit Customizations
![Page 42: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/42.jpg)
47
AI CO-PILOTCommunication
Lip Reading
Natural Language Understanding
Conversational Interaction
![Page 43: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/43.jpg)
48
AI CO-PILOTSafety
Meaningful Alerts Filter
Tracks Where Driver is Looking
Notifies on Unseen Situations
![Page 44: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/44.jpg)
49
DATA ACQUISITION
Sensor integration
Sensor calibration
Image capture
Storage
SIMULATE, DRIVE, TEST
HIL/SIL simulation
Vehicle integration
Test
DATA CURATION, LABELING
Manage data lifecycle
DNN TRAINING
Train NN with data
MAP CREATION
Create HD map
APPLICATION DEVELOPMENT
Compute libraries
NN Inferencing
System level applications
DRIVE PX PC
DEVELOPMENT FLOWSelf Driving Vehicle Development
![Page 45: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/45.jpg)
50
PX2: ON WHEELS
HARDWARE
DRIVE PX 2 nicely mounted in the trunk of a car, pre-wired for cameras, and other sensors
Sensors — Your choice or NVIDIA’s configuration
SOFTWARE
All of what comes with DRIVE PX2
![Page 46: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/46.jpg)
51
DRIVE PX + SENSORS CONFIGURED TO GO
With Ford Fusion + DRIVE PX + cameras, LIDAR, radar, navigation sensors and storage options
Photo courtesy of AutonomouStuff
![Page 47: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/47.jpg)
52
DRIVE PLATFORM
QNX — Common arch across Linux and QNX
Common APIs across CUDA, TensorRT, cuDNN, NvMedia + Support for POSIX APIs
For Production
ZF Pro AI — built on DRIVE PX 2 Auto Cruise
Bosch — building on DRIVE PX using Xavier
Xavier: 30 DL TOPS + ASIL-C for Level 4
SOFTWAREHARDWARE
![Page 48: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/48.jpg)
53
NVIDIA AI CARS
Baidu
nuTonomyVolvo WEpods
Mercedes-BenzTeslaAudi
RoboRace
![Page 49: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/49.jpg)
54
![Page 50: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/50.jpg)
55
![Page 51: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/51.jpg)
56
![Page 52: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/52.jpg)
57
SUMMARY
What are autonomous vehicles
Introduced the concept of functional safety
Discussed the levels of automation
AV & Co-Pilot
DRIVE PX
Development workflow
![Page 53: AN OVERVIEW OF NVIDIA’S AUTONOMOUS VEHICLES …...“Part of overall safety relating to the equipment under control and the control system that depends on the correct functioning](https://reader034.vdocuments.mx/reader034/viewer/2022042921/5f6a45e92669331ca35628fa/html5/thumbnails/53.jpg)
QUESTIONS?