ai and it's application in intelligent & connected vehicle ... · ai and it's...

AI and It's Application in Intelligent Connected Vehicle

同济大学汽车学院School of Automotive Studies, Tongji University

122June 26, 2018© 2015 The MathWorks, Inc.

AI and It's Application in Intelligent Connected VehicleHui CHEN


Introduction of AI and ICV

Government Policies and Projects

Recent R&D Activities

Summary

同济大学汽车学院School of Automotive Studies, Tongji University Hui CHEN

222

234

1

June 26, 2018

1 Introduction of AI and ICV

1 : Introduction of AI and ICV

AI and It's Application in Intelligent Connected Vehicle 322

June 26, 2018


AI ICV

What’s Artificial Intelligence (AI)[1]

Intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and other animals.

“General AI”“Narrow AI”• Works in different individual

domains;• Not capable of experiencing

consciousness

[1]https://en.wikipedia.org/wiki/Artificial_intelligence

• Perform any intellectual task that a human being can;

• Capable of experiencing consciousness

Why AI brings so much attention ?



June 26, 2018


AI ICV

[1] https://en.wikipedia.org/wiki/Artificial_intelligence[2] https://en.wikipedia.org/wiki/Supervised_learning[3] https://en.wikipedia.org/wiki/Unsupervised_learning[4] https://en.wikipedia.org/wiki/Reinforcement_learning

Artificial Intelligence

Expert Systems Fuzzy Logic and Rough Set Heuristic Search …

Machine LearningSupervised Learning[2]

Unsupervised Learning[3]

Reinforcement Learning[4]

Deep learning

CNN RNN …

Learning a function that maps an input to an output based on

example input-output pairs

Inferring a function to describe hidden structure from

"unlabeled" data

Concerned with how to take actions in an environment

to maximize the cumulative reward

Use a cascade of multiple layers of nonlinear

processing units for feature extraction &

transformation

Study and construction of algorithms that can learn from and make predictions on data

What’s Artificial Intelligence (AI)[1] Why AI brings so much attention ?



June 26, 2018


AI ICV

The relationship of AV、CV and ICV

ü Using vehicle sensorsü infrastructure-independent automatic driving

ü Communicating with nearby vehicles and roadside facilities

ü Infrastructure/traffic-dependent automatic driving

• Connected vehicle (CV)

ü Using vehicle sensors ü Communicating with surroundingsü Infrastructure/traffic-aided automatic drivingü Safe, efficient, comfortable, energy-saving,

ultimately autonomous

• Intelligent connected vehicle (ICV)

What’s intelligent connected vehicle (ICV) [1]?

[1]李克强, 戴一凡, 李升波,等. 智能网联汽车(ICV)技术的发展现状及趋势[J]. 汽车安全与节能学报, 2017, 8(1):1-14.

• Autonomous vehicle (AV)

Why ICV is an inevitable trend?

622

2 : Government Policies and Projects

AI and It's Application in Intelligent Connected Vehicle同济大学汽车学院School of Automotive Studies, Tongji University Hui CHEN

June 26, 2018

• Americaü Led by enterprise. Google, Facebook, Microsoft have

increased investment in the research of AI;ü Valued by government, to ensure the leading position

in the field of AI;ü Good at standards making, leading the development

of global AV.

• Europeü Academic institutions in Britain have

made outstanding achievement on AI;ü The motor producer over Europe will

help to the realization of the internationalization of future traffic.

• Japanü The Japanese government and automobile

companies are cautious about automated-driving vehicles, and slow to develop technical standards;

ü Japan is focusing on making automated driving vehicles popular as soon as possible.

• Chinaü China still has a relatively short board in the basic

theoretical research of AI, key common technologies, basic platforms, and talent teams;

ü the government has repeatedly stressed on the establishment of a new generation of AI basic theoretical system and key common technology system.

Features of AI/ICV Policies of Different Areas

722

3 : Recent R&D Activities


June 26, 2018

Up & Down stream Application in ICV Progress of CEC

Algorithm

Machine Learning AlgorithmØ Regression, Decision tree, SVM

Deep Neural NetworkØ Image Recognition：CNNØ Machine Translation：RNN，LSTM

Reinforcement LearningØ Model is known: Dynamic programmingØ Model is unknown: Value function, Policy search

2012Hinton,AlexNet2016Kaiming He,ResNet

2016AlphaGo2017AlphaGo Zero

Development Community

• GitHub

• Apollo by Baidu

822



June 26, 2018


AI Chips• Development of chip design technology

• Examples of hardware solutions for automated driving

Drive PX Xavier NVIDIA Drive-AI CAR PLATFORM

ü Brings the cost and power consumption characteristics of machine learning-related parallel processing to an acceptable level;

ü Making it possible to use large amounts of data to train deep neural networks.

ü NVIDIA Drive PX Xavier

922



June 26, 2018


Big data• Features of big data:

Ø VolumeØ VelocityØ Variety

• Big Data Industry:Ø America has advantages in information and software/hardware technology;Ø Europe, Japan and Australia focus on the fundamental research;Ø China is still in a primary stage, but market grows rapidly.

Data Collection Data Storage Data Analysis Data Application

Big Data Industry Chain

Web Spider

Internet of Things (IoT)

Big

Dat

a Te

chno

logy

Distributed File System

Quantum Computer

Machine Learning

Deep Learning

Data Visualization

Text Visualization

1022



June 26, 2018


Network Information Security

• AI-based information security solution example

ü Wide variety of loT devices with no unified operating system makes it impossible to develop specialized security software for each type of device

ü Limited computing capacity of IoT device terminals makes it impossible to apply general-purpose systems with powerful functions

ü Internet of Things devices are connected in a variety of ways

Traditional security solutions may not be competent in IoT security

ü The MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) and the company PatternEx have jointly developed an artificial intelligence-based cybersecurity platform AI2 that can accurately predict, detect, and block 85% of network attacks.

• Network virus attacks spreading from IT to Internet of Things (IoT)

1122



June 26, 2018


Environment Perception

ü processing data from multiple sensorsü Perceiving & understanding the surroundingsü obtaining key information related to automatic

driving tasks from the surroundings

Intelligent connected vehicle(ICV)

ü Environment perceptionü Decision makingü Motion planningü Motion control

• The function of environment perception

• The general architecture of ICV

1222



June 26, 2018



Traditional vision algorithms

DNN-based vision algorithms

Feature extraction Manually Self-learning

Accuracy Usually with an upper limit

Inclined to get more accurateresults with more data

Robustness Robust under simpler scenarios

Inclined to get more robust results under complex scenarios

ü Using video information captured by a camera as input to a trained Convolutional Neural Network

ü Multi-target detection and classification results can be obtained

[1]余贵珍，深度学习在自动驾驶环境感知中的应用，2016.

• Comparison of these two types of algorithms

• DNN-based solution example in target perception[1]

1322



June 26, 2018


ü Assess the environmental informationü Determine the goal of safety decisionü Local path planning for vehicle movement

ü Posing autonomous driving as a supervised learning problem is difficult due to strong interactions with the environment

ü Valeo proposed a framework for autonomous driving using deep reinforcement learning

ü The vehicle learned autonomous maneuvering in a scenario of complex road curvatures and interaction of other vehicles

• AI-based solution example in decision & planning[1]

Decision & Planning

More intelligent learning from situation-action interactions

• The function of decision and planning

[1] Sallab A E L, Abdou M, Perot E, et al. Deep reinforcement learning framework for autonomous driving[J]. Electronic Imaging, 2017, 2017(19): 70-76.

1422



June 26, 2018


ü Calculate the steering and speed control commands for vehicle, based on the results of local path planning

• AI-based solution example in motion control[1]

ü Nvidia trained a CNN to map raw pixels from a single front-facing camera directly to steering commands

Motion Control• The function of motion control

Training the neural network

[1] Bojarski M, Del Testa D, Dworakowski D, et al. End to end learning for self-driving cars[J]. arXiv preprint arXiv:1604.07316, 2016.

1522



June 26, 2018


Human-Machine Interaction• Definition

The kind of language used to accomplish information exchange tasks between human and machine.

• Traditional HMI：Interface Design • HMI of ICV: Perception Intelligence + Cognition Intelligence

BMW I Drive

Media：Screen and buttonsFunction：Vehicle state information, communication, entertainment, navigation, safety function.

Baidu DeepSpeechGoogle Tacotron

Megvii MegBrain Ford ECG

Voice ：front-end processing, voice recognition, speech synthesis, speaker distinguishing.Vision：face recognition, gesture recognition, driver state perception, object detection.Tactility: force/torque, movement, temperature.Bioinformation：physiological index like blood pressure, heart rate and galvanic skin response.

Deep neural network& end-to-end system

152 layers, ResNet Driver mental work load index definition

Ø Higher reliability and ensured safety of ICV;Ø More excellent practicability and user experience;

Ø Improved maneuverability and flexibility;Ø Assistance to help perform driving tasks better.

• Function

1622



June 26, 2018


Experiment and Testing

• Hardware Test • Software Test

Simulation

Closed-Course

Real-WorldDriving

FunctionalSafety

• Base Vehicle Test

ReliabilityandDurability

1722



June 26, 2018


Service for Travel• Intelligent order assignment

• Intelligent Traffic Light

• Autonomous Parking System

Only consider the distance from driver to passenger

Consider the arrival time based on route planning and evaluation of driver’s service

Consider the supply and demand over the area and optimize the matching

Traffic light interval is fixed

Interval is dynamically adjusted according to nearby traffic

Difficult for parking

Automatically complete parking

The product has been push out in Marvel X

1822



June 26, 2018


Local path planning

A stereogram of an obstacle’s potential field A stereogram of the potential field representing a road with two lanes

The contour map of the corresponding potential field

Desired Acceleration Generation

Environment Info (Road&Obstacles)

Vehicle State Info

Local Path Planning

Low-level Control

Behavior Decision

Vehicle State

Environmental Perception

Target Lane

PF Model

Road PF Model

Obstacle PF Model

Lateral Planning

Desired Steering Wheel Angle Generation

Longitudinal Planning

Local Path Planning Algorithm Diagram

Heuristic Search

1922



June 26, 2018


Validation scenarios：：Local path planning

Papers：：1. "A Potential Field

Based Lateral Planning Method for Autonomous Vehicles", SAE Int.J.Passeng.Cars–Electron. Electr. Syst.10(1):2017.

2. "Longitudinal Planning and Control Method for Autonomous Vehicles Based on A New Potential Field Model", SAE Technical Paper 2017-01-1955, 2017.

Patents:1. 一种统一的自动驾

驶横向规划方法与系统（申请号：201610822955.4）.

2. 一种自动驾驶纵向统一规划方法及系统（申请号：201710811802.4）.

2022



June 26, 2018



Planning & Tracking

HIL Test

Co-Simulation

Method

Ø Learn from parking experience to Improve algorithm efficiency

Ø Uniform framework fordifferent scenarios

Feature

Automatic Parking: AI-based Method

Perception AI-basedAlgorithm

relative posture

historical information

v

δ

• paper ü Z. Fan, H. Chen. (2016), Study on Path Following Control Method for Automatic

Parking System Based on LQR.ü M. Fan, Z. Hu, K. Hamada, H. Chen. (2014), Line Filter Based Parking Slot Detection for

Intelligent Parking Assistance Systemü Kunpeng Cheng, Ye Zhang, Hui Chen. (2013), Planning and Control for a Fully-automatic

Parallel Parking Assist System in Narrow Parking Spaces.• Patentü 201210429726.8, 201310545995.5, 201410557306.7

Automatic Parking : Traditional Method

2122

4 : Summary


June 26, 2018

In the era of big data, with the development of computing chips, algorithms, and the increased market demands, AI ushered in a golden period of rapid development

As a key enabling technology, AI is gaining more and more government investment and policy supports, and will profoundly influence the international competitions in industrial s and nations

As the strategic commanding point for the development of the global automotive industry, intelligent connected vehicles have huge market prospects and will become an important area for application of AI


同济大学汽车学院School of Automotive Studies, Tongji University

2222June 26, 2018

Thank you for your attention!

Hui CHENSchool of Automotive Studies Tongji University, Shanghai, [email protected]

ai and it's application in intelligent & connected vehicle ... · ai and it's...

Documents