![Page 1: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/1.jpg)
Real-Time Object Detection
and Semantic Segmentation
NavInfo Europe
Wei LiCorporate Development manager
Presented at GTC 2019
Session S9351
Geetank RaipuriaComputer Vision Engineer
![Page 2: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/2.jpg)
NavInfo - our growth benefits from AI
2015-Future
2002-
2015
• Surveying and Mapping Knowledge • Image processing
Smart Mobility Provider
Chips
Autonomous
Driving
Connected
Vehicle Service
(CVS)
Navigation
Map & Navigation Provider
AI developed by us provides:• Perception for our autonomous solutions • Can also benefit & apply to other industries• Support related companies/partners, like
NVIDIA
AI
![Page 3: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/3.jpg)
NavInfo’s business growth path
2002 2004 2006 2007 2009 2010 2011 2012 2015 2016 20182014
1st ObtainedGovernment License
1st R&D NavigationMap
1st CommercialNavigation Map
1st On Mobile Devices & Internet
1st Commercial Dynamic TrafficInformation
1st Pedestrian Navigation Map
1st Navi Data Supplier on SZ Stock Exchange
1st ADAS Map of China to BMW
BMW L3 carapplies NavInfo HD Map in China
Provide NVIDIA HAD map for its localization service
Provide Autonomous driving solution to several Chinese OEMs
Provide Autopilot solution based on ourown localization product
1st Commercial NDS Map Product to Market
Tencent New Shareholder
Advanced ADAS Map of China to BMW, Daimler & VW
Expand the HD Map development
20212019
More HAD map solutions
Working with more OEM autonomous solutions
JV’s with NAVTEQ & TOYOTA Tsusho
![Page 4: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/4.jpg)
China
• 21 Localization Bases for Data Collection and Technology Services
• 6 R&D Centers( Shanghai, Xi’an,
Shenyang, Wuhan, Hefei, Shenzhen )
• Beijing Headquarters
Netherlands
1) NavInfo EU• Advanced AI
Research Lab• EU Business
Expansion2) AIIM• Autonomous
driving & Robotics solution provider
3) Mapscape
America
• International Business Expansion
Singapore
• Southeast Asia Business Expansion
NavInfo’s Footprint
![Page 5: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/5.jpg)
Cooperation between NavInfo & NVIDIA
NavInfo’s Training Model
NavInfo trains and optimizes its models on NVIDIA DGX-1servers
NavInfo’s Perception Technology
NVIDIA DRIVE Software use NavInfo’s HD Map to give customer a way of localizing their autonomous driving car.
NVIDIA’s new system - DRIVE Localization
NavInfo’s research lab developed a vision based system that can detect and classify objects in real time on NVIDIA Xavier
Artificial Intelligence
We support and benefit each other’s achievements, driven by AI, to generate better products and services to our customers and end users
![Page 6: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/6.jpg)
NavInfo Service Offerings in Europe
Autonomous driving and robotic solutions
AI based algorithms
AI based solutions for different industries
![Page 7: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/7.jpg)
Results from TU/e sponsored by NavInfo Europe
Results with the highest speed and lowest error
in Stereo Odometry (not Lidar, not SLAM)
No. 4 in the world in the Semantic Segmentation
Reference: [1] Panagiotis Meletis and Gijs Dubbelman, "Training of Convolutional Networks on Multiple Heterogeneous Datasets for Street Scene Semantic Segmentation", In IEEE IV 2018. June 2018, 0–8.
![Page 8: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/8.jpg)
Deep Learning for HD Mapping
![Page 9: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/9.jpg)
HD Mapping
Includes highly accurate lane and road features.
![Page 10: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/10.jpg)
Deep Learning for HD Mapping Feature Extraction
• Deep learning provides automating feature extraction from video feed of collection vehicles
Cameras,LIDARs
Real TimeFeature
Extraction
(Semi) Automated
MapUpdating
Localization
![Page 11: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/11.jpg)
Real Time Object Detection
Geetank Raipuria
Computer Vision Engineer
Advanced Research Lab
NavInfo Europe
Andrei Pata
Software Engineer
Advanced Research Lab
NavInfo Europe
![Page 12: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/12.jpg)
Object Detector based on Deep Convolutional Neural Network architecture, to localize and
classify road signs and traffic lights from a real-time camera feed
Real-Time Object Detection System
Two Stage System: Best of both worlds
High Accuracy
Low Inference time
![Page 13: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/13.jpg)
Features Supported
350+ supported classes including
Traffic Signs
• Regulatory Signs
• Warning Signs
• Guide Signs
• Information Signs
• Road Work Signs
Signboards
Traffic Lights
Digital Traffic Signs
Real-Time Object Detection System
Real Time Performance
2-3x Speedup using Tensor RT
About : 35 fps on INT8 on NVIDIA Xavier SoC
: 110 fps on Titan XP
Inference at Full HD resolution (1920x1080)
Able to extract and classify object as small as 25x25 pixels
Robust to extreme lighting conditions
![Page 14: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/14.jpg)
Sample Detections
![Page 15: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/15.jpg)
Sample Detections
![Page 16: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/16.jpg)
Sample Detections
![Page 17: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/17.jpg)
Demo Realtime Object DetectionAdvanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China.We comply with all Chinese regulations regarding the processing of China data.
Online video available at
https://youtu.be/-QtYF0XUZh0
![Page 18: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/18.jpg)
Real Time Semantic Segmentation
Matti Jukola
Software Engineer
Advanced Research Lab
NavInfo Europe
Ahmed Badar
Computer Vision Engineer
Advanced Research Lab
NavInfo Europe
![Page 19: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/19.jpg)
Deep learning architecture to segment and extract road markings at pixel level
Camera input
Segmentation Network
Segmented road markings
Fully Convolutional
Neural Network
Real-Time Semantic Segmentation
![Page 20: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/20.jpg)
Multi-branch segmentation network
Our model is based on a multi-branch
convolutional neural network architecture
Top-view transformation
Camera input
Top-down view Top-down predictions
Front-view predictions
Front-view transformation
Real-Time Semantic Segmentation
![Page 21: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/21.jpg)
Real-Time Semantic Segmentation
Real Time Performance
3x Speedup using Tensor RT
About : 90 fps on NVIDIA Xavier SoC
: 300 fps on Titan XP
Inference at 1024 × 384 image sizes
This includes image transformations (top and front view)
Currently supports 40 Road Marking Classes, including:
• Lane lines
• Arrows
• Text
![Page 22: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/22.jpg)
Sample Segmentations
Input Image PredictionGround Truth
![Page 23: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/23.jpg)
Sample Segmentations
Input Image PredictionGround Truth
![Page 24: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/24.jpg)
Demo Realtime Semantic SegmentationAdvanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China.We comply with all Chinese regulations regarding the processing of China data.
Online video available at
https://youtu.be/E4hU-COkHDo
![Page 25: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/25.jpg)
Real Time Scene Understanding
Elahe Arani
Senior AI Researcher
Advanced Research Lab
NavInfo Europe
Mahmoud Gamal
Computer Vision Engineer
Advanced Research Lab
NavInfo Europe
![Page 26: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/26.jpg)
Real-Time Scene Understanding
A real time unified object detection and semantic segmentation for autonomous driving
cars/HD mapping.
DNN
![Page 27: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/27.jpg)
Joint Object Detection and Segmentation
Currently supports 40 Road Marking Classes
and 350+ Road Sign classes including:
– Traffic Signs
– Gantry Signboards
– Traffic Lights
– Digital Traffic Signs
– Lane Markings
– Text
– Arrows
Performance
Inference at 512x512 image sizes
About 45 FPS on Titan XP
Other Features Supported
– Guard Rails
– Curbs
– Speed Limits on Road
![Page 28: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/28.jpg)
Sample Detections
![Page 29: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/29.jpg)
Input Image Ground Truth Prediction
Sample Segmentation
![Page 30: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/30.jpg)
Prediction (Dec) Ground Truth Prediction (Seg)
Joint Detections and Segmentation
![Page 31: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/31.jpg)
Occlusion Handling
Input Image Prediction (Dec) Ground Truth Prediction (Seg)
![Page 32: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/32.jpg)
Occlusion Handling
Input Image Prediction (Dec) Ground Truth Prediction (Seg)
![Page 33: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/33.jpg)
Demo Realtime Scene UnderstandingAdvanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China.We comply with all Chinese regulations regarding the processing of China data.
Online video available at
https://youtu.be/NJVNFfueKb4
![Page 34: Real-Time Object Detection and Semantic …...Real-Time Object Detection and Semantic Segmentation NavInfo Europe Wei Li Corporate Development manager Presented at GTC 2019 Session](https://reader030.vdocuments.mx/reader030/viewer/2022041115/5f255f49f65bb1247f384f01/html5/thumbnails/34.jpg)
Real-Time Object Detection
and Semantic Segmentation
NavInfo Europe
Wei LiCorporate development manager
Presented at GTC 2019
Session S9351
Geetank RaipuriaComputer Vision Engineer