developing deploying autonomous driving ......multicore extension to classic autosar safety up to...
TRANSCRIPT
![Page 1: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/1.jpg)
WITH THE NVIDIA DRIVE PX PLATFORM
DEVELOPING & DEPLOYING AUTONOMOUS DRIVING APPLICATIONS
Shri Sundaram, Product Manager, DRIVE PX Platform
![Page 2: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/2.jpg)
2
DRIVE PX: AV Development Platform
AV Developers: DRIVE PX as your tool
AV HW/SW Ecosystem: DRIVE PX as your platform to reach developers
![Page 3: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/3.jpg)
3
NVIDIA DRIVE PXOpen AV Computing Platform for the Transportation Industry
Powerful and scalable AV computer
Deep Neural Network, Sensor Fusion and Computing
Extensive I/O to interface with wide range of sensors and vehicle networks
An open SW stack
Level 3 to Level 5; ASIL-D functional safety
![Page 4: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/4.jpg)
4
0
50
100
150
200
250
300
350
400
FY18 Q1 FY18 Q3
DRIVE PX DRIVING AV AI
Launched CES 2015
Spike in AV AI engagements after we powered on discrete GPU
More than doubled in last 6 months
Plus >145 AV Startups on NVIDIA DRIVE
DRIVE Platform Engagements
Source: NVIDIA statistics
![Page 5: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/5.jpg)
5
AV DEVELOPMENTPath from Idea to Production
PRODUCTION
Deploy
Safety hardening
Performance tuning
SW upgrades
PROTOTYPE
Test
Feature development
Validation
Develop
Perception
Mapping/Localization
Path Planning
Combination/More…
IDEA DEVELOPMENT
OBJECTIVE
TOOL
PC PC DRIVE Platform
Automotive Sensors
Production SW
OTA framework
Scalable compute with discrete GPUsEcosystem of sensors + other HW/SW peripheralsTensorRT, CUDA, Open Source Frameworks
![Page 6: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/6.jpg)
6
DEVELOPMENT FLOWUsing DRIVE PX Platform
Data Acquisition to train DNN
Curated/AnnotatedTraining Data
Data Acquired From Sensors
Deep Neural Network
Autonomous VehicleApplications
Autonomous VehicleApplication Development
Test/Drive
Simulation
HD Map
Data Acquisition to Generate Map
Autonomous Application Development
Testing In-Vehicle or With Simulation
Neural NetworkTraining
1
2
3
4
1
2
3
4
![Page 7: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/7.jpg)
7
DATA ACQUISITION
Sensor fusion interfaces:GMSL Camera, CAN, GbE, BroadR-Reach,
FlexRay, LIN, GPIO
Displays interfacesHDMI, FPDLink III and GMSL
Storage Interfaces10GbE, USB3 (also HDMI)
Rapidly Growing Ecosystem of Supported Sensors & Storage Solutions
![Page 8: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/8.jpg)
8
Sekonix
Camera Module based on latest AR0231 Automotive sensor from On Semi
IP69K waterproof packaging
Ultra high resolution lens
Leopard
Wide range of sensors from OnSemi, SONY and Omnivision
Proof of concept & quick turn
VISION CAMERAS – SEKONIX & LEOPARD
![Page 9: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/9.jpg)
9
THERMAL CAMERAS
Thermal sensors create images from heat, not light, so they can detect pedestrians and oncoming vehicles regardless of lighting conditions
Start collecting thermal data in minutes
Easy set-up, operation, and integration
Most importantly available on NVIDIA DRIVE™ PX 2 support available
FLIR Thermal Sensor — Automotive Development Kit (ADK)
![Page 10: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/10.jpg)
10
GPS/GNSS SOLUTION FROM NOVATEL
Under development
![Page 11: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/11.jpg)
11
STORAGE SOLUTIONSfrom Quantum and AutonomouStuff
![Page 12: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/12.jpg)
12
DRIVE PX + SENSORS CONFIGURED TO GO
With Ford Fusion + DRIVE PX + cameras, LIDAR, radar, navigation sensors and storage options
Photo courtesy of AutonomouStuff
![Page 13: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/13.jpg)
13
DATA ACQUISITIONTools to acquire, time stamp, sync and store data
1. Route setting
2. Configuration switching
3. Storage info
4. Camera preview
5. Start Recording control button
6. Settings button
7. GPS info
8. Sensor Status
Set Route ID; File Config (RAW, Lossless RAW, Compressed), start/stop recording!
Get useful information about storage left, camera preview, sensor status
![Page 14: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/14.jpg)
14
DEVELOPMENT FLOWUsing DRIVE PX Platform
Data Acquisition to train DNN
Curated/AnnotatedTraining Data
Data Acquired From Sensors
Deep Neural Network
Autonomous VehicleApplications
Autonomous VehicleApplication Development
Test/Drive
Simulation
HD Map
Data Acquisition to Generate Map
Autonomous Application Development
Testing In-Vehicle or With Simulation
Neural NetworkTraining
1
2
3
4
1
2
3
4
![Page 15: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/15.jpg)
15
PUTTING IT ALL TOGETHERAV App Development
DEVELOP BENCHMARK / DEPLOY
LINUX PC
DRIVE PX 2
Cross-
Compile
Feedback
& Iterate
![Page 16: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/16.jpg)
16
EMBEDDED SOFTWARE DEVELOPMENT WORKFLOW
SOFTWARE DEVELOPMENT
TOOLCHAIN SETUPCROSS-COMPILATION
PORTING
DEBUGCPU/GPU
REMOTE DEBUGGING
PROFILESYSTEM/CPU/GPU/IO/…
REMOTE PROFILING
RUN
SHIP IT!
DriveInstall
Nsight EE
Eclipse
Tegra/LinuxGraphics Debugger
Tegra/LinuxGraphics Debugger
CUDA Visual Profiler
TegraSystem Profiler
Cuda-gdb
PerfWorks
nvprof
CUPTI
Cuda-memcheck
Nsight EE
Desktop Tools
![Page 17: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/17.jpg)
17
DRIVE SOFTWARE STACK
DRIVE PX
Cameras
Sensors &Maps
NVMEDIA DRIVE OS, CUDA
CUDA libraries & engines(including CuDNN/TensorRT)
DriveWorksSAL
DriveWorks Algorithm Modules
Autonomous Driving Applications
DriveWorksTools DNNs
![Page 18: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/18.jpg)
18
TEST / DRIVEWith PX on Wheels Research Vehicles
![Page 19: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/19.jpg)
19
DRIVE PLATFORM TODAY FOR LEVEL 3/4
DRIVE Platform – Open AI Autonomous Vehicle platform
DRIVE PX 2 DRIVE PX XavierOne Architecture, Increased Performance per Watt
![Page 20: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/20.jpg)
20
DRIVE PLATFORM For Production
DRIVE PX
DRIVEWORKS SDK
DRIVE AV
PlanningLocalizationPerception
DRIVE OS
Level 3 to Level 5 | ASIL-D Functional Safety
![Page 21: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/21.jpg)
21
Placeholder to amplify whatever gets revealed in JHH GTC Keynote
![Page 22: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/22.jpg)
22
OTHER RELATED SESSIONSBy NVIDIA team
TUE (10/10) WED (10/11) THU (10/12)
INTRODUCTION AND INTEGRATION WITH
DRIVEWORKS ON DRIVE PX2By Aaraadhya Narra & Alessandro Ferrari (11)
X X X
CUDA PROGRAMMING ON DRIVE PX2By Siva Rama & Chethan Ningaraju
X X
EGLSTREAMS: INTEROPERABILITY FOR CAMERA, CUDA AND OPENGLBy Yogesh Kini, Anshuman et. al (11)
X X
DEPLOYMENT OF SEMANTIC SEGMENTATION NETWORK USING TENSORRTBy Joohoon Lee & Chethan Ningaraju
X
![Page 23: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/23.jpg)
23
OTHER RELATED SESSIONSBy NVIDIA partners
TUE (10/10) WED (10/11) THU (10/12)
AI DRIVEN ENVIRONMENT MODELING FOR AUTONOMOUS DRIVING ON NVIDIA DRIVE PX 2By Alexey Abromov, Continental (14c) X
ARCCORE'S PLATFORM SOLUTIONS FOR SELF-DRIVING CARSBy Eric Envall & Hakan Berglund (Arccore) (14c) X
OPEN FUSION PLATFORM FOR AUTOMATED DRIVING CARS BASED ON NVIDIA DPX2By Paulin Fouopi and Mohsen Sefati
X
DESIGNING A SOFTWARE FRAMEWORK FOR AUTOMATED DRIVINGBy Sebastian Ohl, Elektrobit X
COMBINING AI, RGB, AND 3D FOR SELF-DRIVING COGNITION SYSTEMSBy Yaron Tanne/Doron Elinav, Vayavision
X
VISUAL PERCEPTION FOR AUTONOMOUS DRIVING ON THE NVIDIA DRIVEPX2 AND USING SYNTHIABy Juan Moure/Antonio Espinosa, Barcelona
X
![Page 24: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/24.jpg)
24
OTHER RESOURCES
TOOLS
Developer Zone (developer.nvidia.com/DRIVE)
Developer Forums (devtalk.nvidia.com)
SCHOOLS
Deep Learning Institute – Autonomous Driving
Udacity Nanodegree
OTHER PRESENTATIONS
![Page 25: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/25.jpg)
![Page 26: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/26.jpg)
26
AUTOSAR STACK FROM ELEKTROBITIncluded & Preinstalled with DRIVE PX 2
![Page 27: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/27.jpg)
27
AUTOSAR REFERENCE STACK FROM ARCCOREAutomotive Software Platform for DRIVE PX2
![Page 28: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/28.jpg)
28
AUTOSAR REFERENCE STACK FROM ARCCOREAutomotive Software Platform for DRIVE PX2
Embedded ECUsClassic AUTOSAR platform
Multicore extension to classic AUTOSAR
Safety up to ASIL-D
Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA)
High performance embedded ECUs (Linux/QNX)Adaptive AUTOSAR
Virtual AUTOSAR ECUs
Ethernet communicationSOME/IP – Service Discovery Ethernet protocol
gPTP TimeSynchronization
Development ToolsAUTOSAR Authoring and BSW configuration
System Replay for Virtual Verification and Function Development
Realtime Analysis Tool
Logging
3D/360 Sensor analysis viewer
![Page 29: DEVELOPING DEPLOYING AUTONOMOUS DRIVING ......Multicore extension to classic AUTOSAR Safety up to ASIL-D Support for DrivePX2 3LSS Safety SW Architecture (NVIDIA) High performance](https://reader036.vdocuments.mx/reader036/viewer/2022063021/5fe3bbb6db2e946f0a442ad0/html5/thumbnails/29.jpg)
29
V2X – COHDA WIRELESS