self-driving cars: technologies & challenges · rtca do 178c software considerations in...
TRANSCRIPT
![Page 1: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/1.jpg)
Self-Driving Cars:Technologies & ChallengesDeep Conversations on Deep LearningA technical series hosted by IEEE Maine Section
W.D. Rawle, PhDSenior Member IEEEChair, IEEE Maine SectionOctober 21, 2020
![Page 2: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/2.jpg)
2… Deep Conversations on Deep Learning Series
Advanced Driver Assistance SystemsLevel 2 Autonomy
ELE Times
![Page 3: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/3.jpg)
3… Deep Conversations on Deep Learning Series
ADAS Examples
ToyotaCamry
TeslaModel 3
![Page 4: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/4.jpg)
4… Deep Conversations on Deep Learning Series
Self Driving Cars !! Level 4 Autonomy
Waymo Aurora Voyage
AptivAurora/Peterbilt
![Page 5: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/5.jpg)
Autonomy
2… Deep Conversations on Deep Learning Series
Level 0: full control from the driver. Vehicle has no support systems.
Level 1: involves basis assistance features. Level 1 cars are equipped with anti-
lock breaking (ABS) and cruise control. The driver is in full control
Level 2: semi-autonomous driving. Vehicle can drive straight, maintain lane,
and maintain control over the distance to vehicles in front of it..
Level 3: defines the moment when the on-board systems can take over all
driving functions, but only in certain situations. Driver must remain behind steering wheel all the time and be ready to take over
Level 4: a fully autonomous experience with driver behind the steering wheel.
Most of the time the vehicle can drive on its own and will handle even complicated situations on highway and city traffic. No need for driver to constantly observe traffic. At Level 4, vehicles will communicate and inform each other about maneuvers such as changing lanes
Level 5: truly self-driving cars. Operating autonomously in all conditions. There
is completely no need for people in the car to take any action. Such vehicles will not be equipped with a steering wheel Wikipedia
![Page 6: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/6.jpg)
6… Deep Conversations on Deep Learning Series
Level 4 Autonomy Technology Landscape
AUTONOMY
INFRASTRUCTURE
SYSTEMS
PERCEPTION PREDICTION PLANNING CONTROL
SAFETY REDUNDANCY DIAGNOSTICS PERFORMANCE
MAPPING SIMULATION REMOTEOPERATIONS
RELIABLENETWORKS
Oliver Cameron, CEO VoyageMIT S.0694 Guest Lecture
![Page 7: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/7.jpg)
7… Deep Conversations on Deep Learning Series
Challenges with Perception: FMVCONVOLUTIONAL
NEURAL NETWORK
CL
AS
SIF
ICA
TIO
N
IMA
GE
IN
PU
TO
NE
FR
AM
E P
ER
CO
LO
R
SIZE/STRIDE
MAX POOL
Fooled by a little distortionMIT 6.S094 Lex Friedman Deep LearningLecture 1
CNN Approximations• Size/stride- convolution approx.• Max Pool – data loss• Training – insufficient samples
![Page 8: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/8.jpg)
8… Deep Conversations on Deep Learning Series
Perception, CNNs, and Scene Segmentation
Mapping every voxel to an identified object for prediction and path planning
SegNet: University of Cambridge
![Page 9: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/9.jpg)
9… Deep Conversations on Deep Learning Series
Perception, CNNs, and Scene Segmentation Questions??
• Is the image sufficiently sampled to capture “high frequency” effects- Nyquist criteria
• Does the discretization of the convolution function compromise the output
• How much data is lost when using max pool compression
• Is fidelity of training data sufficient• Would alternate approaches (DCT, for
example) provide sufficient compression and maintain fidelity
• What would be the difference in compute resource requirements
![Page 10: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/10.jpg)
10… Deep Conversations on Deep Learning Series
LIDAR & It’s Challenges
![Page 11: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/11.jpg)
11… Deep Conversations on Deep Learning Series
LIDAR & It’s Challenges
Elastic Backscatter LIDAR Extinction Ambiguity
where ��� � is the LIDAR elastic backscattered power (W) from range (R), ��� is the
LIDAR system constant incorporating the transmitted optical pulse energy, the
effective telescope receiving area, and the optical path losses in the system. ������ and
������ comprise the two elements of the extinction coefficient: aerosol backscatter and
molecular absorption. ���� (R) represents the two-way admittance from the instrument
to the backscatter target. ��� (R) represents to so called overlap function,
incorporating the unit normalized cross over function between the laser illuminated
atmosphere at range R and the telescope’s field of view.
![Page 12: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/12.jpg)
12… Deep Conversations on Deep Learning Series
LIDAR & It’s Challenges
A potential solution
![Page 13: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/13.jpg)
13… Deep Conversations on Deep Learning Series
Level 4Autonomy & Safety
Twelve Principles of Automated Driving
VSFail safe (FS), Fail Degraded (FD)Capabilities Derived from Dependability Domains
Safety First for Automated Driving
Aptiv Services 2019
![Page 14: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/14.jpg)
14… Deep Conversations on Deep Learning Series
Level 4 Autonomy and SafetyISO/PAS 21446 Safety of the Intended Function (SOTIF)
Area 1: Known safe behaviorMaximize safe function of system
Area 2: Known behavior that could bepotentially dangerous or possiblyunintended behavior in certaincircumstancesMinimize known potential unintendedscenarios
Area 3: unknown and potentially dangerousbehavior Minimize unknown unintended scenarios
![Page 15: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/15.jpg)
15… Deep Conversations on Deep Learning Series
Level 4 Autonomy and SafetyA
RP
47
54
G
uid
elin
es f
or
the
Dev
elo
pm
ent
of
Civ
il A
ircr
aft
an
d S
yste
ms
![Page 16: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/16.jpg)
16… Deep Conversations on Deep Learning Series
Level 4 Autonomy and Safety
ARP 4754 Guidelines for the Development of Civil Aircraft and Systems
![Page 17: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/17.jpg)
17… Deep Conversations on Deep Learning Series
Level 4 Autonomy and SafetyARP 4761Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment
Fault Tree Analysis Failure Modes and Effects Analysis Common Cause Analysis Time Limited Dispatch Dependence Diagrams Markov Analysis
![Page 18: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/18.jpg)
1818… Deep Conversations on Deep Learning Series
RTCA DO254 Design Assurance Guidance for Airborne Electronic HardwareRTCA DO 178CSoftware Considerations in Airborne Systems and Equipment Certification
Level 4 Autonomy and Safety
![Page 19: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/19.jpg)
1919… Deep Conversations on Deep Learning Series
RTCA DO 178CSoftware Considerations in Airborne Systems and Equipment Certification
Level 4 Autonomy and Safety
![Page 20: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/20.jpg)
2020… Deep Conversations on Deep Learning Series
Level 4 Autonomy and Safety
RTCA DO 178CSoftware Considerations in Airborne Systems and Equipment Certification
![Page 21: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/21.jpg)
2121… Deep Conversations on Deep Learning Series
Level 4 Infrastructure: 5G Networks
![Page 22: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/22.jpg)
2222… Deep Conversations on Deep Learning Series
RAN – Radio Access Network
Level 4 Infrastructure: 5G Networks
Enabling Intelligent Transport in 5G NetworksEricsson Technology Review #9 2017
![Page 23: Self-Driving Cars: Technologies & Challenges · RTCA DO 178C Software Considerations in Airborne Systems and Equipment Certification Level 4 Autonomy and Safety. 19… Deep Conversations](https://reader035.vdocuments.mx/reader035/viewer/2022062611/613508c6dfd10f4dd73c1ce7/html5/thumbnails/23.jpg)
Thank you
23
W.D Rawle PhDSenior Member, IEEEChair, IEEE Maine [email protected]
Many thanks to Dr. Ali Abedi, NE Area Chair, IEEE Region 1 for providing Webex Resources