deterministic ethernet: addressing the challenges of ......3rd workshop on safety and security of...
TRANSCRIPT
![Page 1: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/1.jpg)
3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017)
Deterministic Ethernet: Addressing the Challenges of Asynchronous Sensing in Sensor Fusion Systems
Date: 2017.06.25
Ayhan Mehmed
Sasikumar PunnekkatMDH Västerås, Sweden
Wilfried SteinerTTTech Computertechnik AG
[email protected] Computertechnik AG
![Page 2: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/2.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
• Understanding the OOSM problem
• Methods for measurement timestamping
• The use of deterministic Ethernet networks
for precise timestamping
• Results
• Conclusions and future work2
![Page 3: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/3.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
3
![Page 4: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/4.jpg)
ADAS Introduction
4
Informing Warning
Controlling
![Page 5: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/5.jpg)
Sensor fusion
5
Sensing:• Long and Short Range Radars
• Ultrasonic sensors
• LiDARs
• Mono and Stereo Cameras
Ego-vehicle sensors• Gyro, accelerometer
• Wheel speed sensors
• Steering angle sensor
• GPS
Virtual sensors:• Digital maps
• Wireless communication
Se
nso
r fusio
n
![Page 6: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/6.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
7
![Page 7: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/7.jpg)
Performance of sensor fusion
systems
8
Quality of the input data
Efficiency of the fusion algorithm
• Average number of missed targets,
• Average number of extra targets and etc..
Improvement and quality assessment of the input data:
• Specifically the degree in confidence in terms of attributes
such as reliability and credibility
Rate at which the data is provided to the fusion system:
• Rate of the sensors (measurement cycle)
• Quality of the communication link (latency, jitter)
Neglect OOSM Perf. Decrease
![Page 8: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/8.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
• Understanding OOSM problem
9
![Page 9: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/9.jpg)
Open questions
10
HOW DOES THEOOSM
EFFECTS THE SENSOR FUSION SYSTEM?
Kalman filter
HOW DOES THEOOSM OCCUR?
![Page 10: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/10.jpg)
The Kalman Filter
11
THE BEST ESTIMATEWE CAN MAKE OF
THE LOCATION OF AN OBJECTIS PROVIDED BY COMBINING OUR
KNOWLEDGE FROM THE PREDICTIONAND THE MEASUREMENT
An optimal recursive data processing algorithm, that computes the best
estimate of the current target state, based on the preceding target state
estimate, the current measurement and the control input (ut).
![Page 11: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/11.jpg)
The Kalman Filter
12
1
2
3
![Page 12: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/12.jpg)
Application of Kalman filter for
multiple sensors
13
![Page 13: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/13.jpg)
The occurrence of OOSM
14
• Differing measurement latency times
• Asynchronous sensing
![Page 14: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/14.jpg)
The effect of OOSM
15
NTMO
![Page 15: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/15.jpg)
The amount of the error
introduced by NTMU
16
Size of NTMU The Kalman gain Velocity
Constant speedConstant K=1
Error introduced by the NTMU
![Page 16: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/16.jpg)
Handling the OOSM
17
Single-lag and multi-lag OOSM
Sophisticated algorithms Simple algorithms
Buffering algorithm
Assumption
Precise timestamp
Not valid
Lack of SWRC
Chronological order
Not guaranteed
NTMU
InevitableOOSM
![Page 17: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/17.jpg)
The amount of the error introduced
by imprecise measurement
18
130 Km/h
Vehicle speed
Error <= 20cm
Objective
Size of NTMU
Error of the timestamp
Error introduced by
![Page 18: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/18.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
• Understanding the OOSM problem
• Methods for measurement timestamping
19
![Page 19: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/19.jpg)
Timestamping data at arrival
(Centralized method)
20
Measurement cycle time
Measurement transf. time
Measurement latency. time Error of the timestamp
Precision of the cycle time
Communication jitter
![Page 20: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/20.jpg)
Triggering method
21
Trigger latency. time
Trigger transfer latency
Activation latency
Error of the timestamp
Communication jitter
Constant trigger lat. time
![Page 21: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/21.jpg)
Timestamping at the time of
acquisition (distributed method)
22
Measurement with
timestamp
Sensors
No cycle times
Benefit
No transfer times
Global time
Difficulty
Needed
Global sync. mechanism
Precision of the sensor
internal clock with respect
to UTC time.
Precision of the timestamp
Error of the timestamp
![Page 22: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/22.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
• Understanding the OOSM problem
• Methods for measurement timestamping
• The use of deterministic Ethernet networks
for precise timestamping
23
![Page 23: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/23.jpg)
What do we target
24
Distributed app.Centralized approach Triggering app.
• Ensure a system-wide synchronized time
• Low jitter data transmission
Communication standards
Standard Ethernet
IEEE 802.1AS
+
SAE AS6802Sync accuracy 1 µs
IEEE 802.1Qav Jitter ~10 – 100 µs
IEEE 802.1Qbv Jitter 1 µs
AV
BT
SN
Jitter 1 µs +
Sync accuracy 1 µs
![Page 24: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/24.jpg)
Suitability of the standards for
achieving precise timestamps
25
• Traffic class
• The data that is transmitted
• Maximum communication jitter(Numbers taken from simulation based
performance comparisons)
Standards and their spec.
Assumed to be achieved with:
• Centralized, triggering or
distributed approach.
• If one or more comm. Standards
are used.
Timestamp precision
![Page 25: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/25.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
• Understanding the OOSM problem
• Methods for measurement timestamping
• The use of deterministic Ethernet networks
for precise timestamping
• Results
26
![Page 26: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/26.jpg)
Error introduced by the
timestamp precision
27
![Page 27: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/27.jpg)
Outline
• Advanced Driver Assistance Systems (ADAS)
• Problem definition
• Understanding the OOSM problem
• Methods for measurement timestamping
• The use of deterministic Ethernet networks
for precise timestamping
• Results
• Conclusions and future work28
![Page 28: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/28.jpg)
Conclusions
29
Objectives of our research
Study the benefits of different communications standards for the
application area of sensor fusion systems..
Prerequisites
1. Understanding the Kalman filter
2. The cause of OOSM and how they affects the Kalman filter
3. How the absence of precise measurement timestamp leads to
NTMU, same as the caused by the OOSM,
4. And finally investigate the methods for sensor measurement
timestamping and formulate their precision
Based on this knowledge
• We were able to show that communication standards can
contribute for solving the problem of NTMU.
• By minimizing the error introduced to the optimal state estimate
down to the range from 0 to less than 0,5cm depending on the
timestamping methods and the communication standards used.
![Page 29: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/29.jpg)
Future work
30
Optimistic results
consequence of idealized conditions and communication
Specifications.
Verification
Simulation based studies to verify the correctness of the theoretical
assumptions made in this paper.
![Page 30: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/30.jpg)
Copyright © TTTech Computertechnik AG. All rights reserved.
Vienna, Austria (Headquarters)
Phone +43 1 585 34 34-0
USA
Phone +1 978 933 7979
Japan
Phone +81 52 485 5898
China
Phone +86 21 5015 2925-0
www.tttech.com
Thank You for
Your Attention!
![Page 31: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/31.jpg)
Timestamping data at arrival
(Centralized method)
32
Measurement cycle time
Measurement transf. time
Measurement latency. time
![Page 32: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/32.jpg)
Drawback of current solutions
33
• Assume a timestamp with precise timestamp
• Not valid: due to the lack of system wide-reference
clock.
![Page 33: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/33.jpg)
Deterministic sensor data acquisition
34
• Deterministic Behavior
A system exhibits a deterministic behavior when the performance measures of its services are
predicable under a number of conditions and characterized by specific nonrandom equations
• Context of in-vehicle networks
The most important performance measure is message latency
• Context of multi-sensor data
Result of non-deterministic message latency are multi-sensor data fusion systems are the out-of-
sequence measurements.
• Solution
• Exploring the features of different real-time network technologies:
• TTEtthernet,
• AVB,
• TSM
• Explore their ability of achieving deterministic behavior in the process of acquisition of sensory
data from different ADAS sensors
• Abilities:
• Tightly synchronize the data acquisition from multiple cameras, LiDARs, LRR and SRR
• Provide a “time-stamp” for each measurement
Summary Timeline Sensor Fusion LayerStudies Deterministic SDA
![Page 34: Deterministic Ethernet: Addressing the Challenges of ......3rd Workshop on Safety and Security of Intelligent Vehicles (SSIV 2017) Deterministic Ethernet: Addressing the Challenges](https://reader035.vdocuments.mx/reader035/viewer/2022081619/60f76535894ab96a63187b68/html5/thumbnails/34.jpg)
The Kalman Filter
35
1
2
3