haveit joint system · the future of driving. final event haveit joint system first hotel grand,...

The future of driving.

Final Event

HAVEit Joint System First Hotel Grand, Borås, Sweden

June 21- 22, 2011

Frank Flemisch,

Anna Schieben, Gerald Temme, Christian Löper, Julian Schindler, Johann Kelsch (DLR)

Fazwi Nashashibi, Paulo Resende (INRIA),

George Thomaidis (ICCS), Serge Boverie (CAF),

Sebastien Glaser, Benoit Vanholme, Salim Hima (LCPC/IFFSTAR)

Nadja Schoemig, Armin Kaussner (WIVW)

Before looking into the future: A look back in history

1900

1908Champs-Élysées 1900

Flemisch & Schieben / 2011 DLR

Joint System in assistance and automation

??


Before HAVEit: Example SPARC (Secure Propulsion using Advanced Redundant Control)

Inform Support InterveneWarn

tw tu tcteti

Strategy of Assistant Systems


Before HAVEit: Example H-Metaphor, H-Mode (DLR, TU-M, DFG, NASA)

Before HAVEit: Example H-Metaphor, H-Mode (DLR, TU-M, DFG, NASA)

Assistance & Automation Scale

Shared & Cooperative Control

Joint Cognitive System (Hollnagel)

Tight

Rein

Loose

Rein

analogous to

„rider and horse―

From H-Mode to HAVEit

Assistance & Automation Scale

Shared & Cooperative Control

Tight

Rein

Loose

Rein

analogous to

„rider and horse―

Co-system state

Driver state

Command

Driver intention

Recommendation

System intentionDRIVER Co-System

Environment Driving Task

Manual switches

Automation Mode

TRANSITIONS

Dynamic task repartition

HAVEit Joint System Demonstrator


Co-system state

Driver state

Command

Driver intention

Recommendation

System intentionDRIVER Co-System

Environment Driving Task

Manual switches

Automation Mode

TRANSITIONS

Dynamic task repartition

HAVEit Joint System Demonstrator


Generic concepts

Iterative refinement

Initial demonstrator concepts

Theater-technique

Development steps: From initial to generic concepts


Theater technique: Designing the behavior and interaction


HAVEit Joint System development steps: Iterative waterfall

Generic Concepts

Validation by simulationValidation in trail blazer

Implementation

Migration to

simulators

Migration to

vehicle(s)



Design with theater system


Building up the Joint System: Sensors

Environment sensors

SICK

Vehicle sensors

Sensor data fusion

ICCS

Drivetrain control

Steering Brakes GearboxEngine

Driver

monitoring

WIVW, CAG

Driver

HMI

DLR Driver state

assessment

WIVW

Mode selection arbitration unit

DLR

Co-Pilot

INRIA, IFSTTAR, DLR

automation level

maneuvers,

trajectory

Command and haptic feedback generation

IFSTTAR, DLR

motion control vector

Building up the Joint System: Sensor Data Fusion

Da

ta A

cq

uis

itio

n

Laser

scanner

Track

States

SRR

Camera Tracking

Track-

Track

associatio

n

Track

Fusion

Global

Tracker

Short Range Radars

Laser Scanner

Camera

Driver monitoring Camera

Track

States

Track

States

Track

States

Tracking

Tracking

Building up the Joint System: Co-Pilot

Environment sensors

SICK

Vehicle sensors

Sensor data fusion

ICCS

Drivetrain control


Driver

monitoring

WIVW, CAG

Driver

HMI

DLR Driver state

assessment

WIVW


DLR

Co-Pilot

INRIA, IFSTTAR, DLR

automation level

maneuvers,

trajectory


IFSTTAR, DLR


HAVEit Joint System: Co-System algorithms (examples)

Manoeuvres: Decision on the safety of the manoeuvres based on 3 algorithms that have veto-

right

Trajectories: Generation of trajectory and speed profile for each of the selected target lanes

1. Fuzzy logic based

manoeuvre tree algorithm

2. Sampling-based

manoeuvre grid algorithm

3. Partial Motion Planning

(PMP) or quintic polynomial

trajectory algorithm


HAVEit Joint System: Command and Haptic Feedback Generation

Generation of direct commands on motor and front wheel

Generation of haptic feedback commands on pedals and steering wheel

Switch from optimal to minimum risk trajectory (MRS) in the case of system

failure and driver absence

Haptic CommandDirect Command

Inceptor Input

translation

Command

Generation

Haptic

Feedback

Generation

Coupling valve MSUMSU

Driver Co-pilot


Building up the Joint System: Driver State Assessement

Environment sensors

SICK

Vehicle sensors

Sensor data fusion

ICCS

Drivetrain control


Driver

monitoring

WIVW, CAG

Driver

HMI

DLR Driver state

assessment

WIVW


DLR

Co-Pilot

INRIA, IFSTTAR, DLR

automation level

maneuvers,

trajectory


IFSTTAR, DLR


2323

HAVEit Joint System: Driver in the loop assessment

Development of SW component for online Driver

State Assessment

- Distraction detection

- Drowsiness detection

Combination of techniques

- Direct driver monitoring (CAF camera)

- Indirect driver monitoring (driver’s behaviour

and performance, WIVW)

Driver State Assessment

drowsiness detection

distraction detection

Driver monitoring Driver monitoring














Building up the Joint System: HMI & MSU

Environment sensors

SICK

Vehicle sensors

Sensor data fusion

ICCS

Drivetrain control


Driver

monitoring

WIVW, CAG

Driver

HMI

DLR Driver state

assessment

WIVW


DLR

Co-Pilot

INRIA, IFSTTAR, DLR

automation level

maneuvers,

trajectory


IFSTTAR, DLR


HAVEit Joint System: Mode Selection & Arbitration Unit (MSU)

Ai [failure corrected]

Driver Assisted Semi Automated Highly Automated

Emergency

Brake

MRM

Failure

Start

point

Ai

[distracted]

|| [drowsy]

Ai

[preconditions not

longer fullfilled]Ai

[preconditions not

longer fullfilled]

Ai

[distracted]

|| [drowsy]

Normal driving

Ai [failure]

Ai [standstill?]

Ai [emergency]

Ai [No response

by driver]

Ai [No response

by driver]

Ai [No response by driver]



MRM / Emergency

Failure

End point

to MRS

End point

to MRS

End point

to MRS

End point

to MRS

Ai [emergency]

Ai [distracted]

|| [drowsy]

Ai [No response

by driver]

power off

power on

Di [switch DA]

[driver grasps steering wheel]

[driver

grasps

steering

wheel]

Di [switch DA] || [steering over threshold] || [accelerating over threshold] || [breaking over threshold]

Di

[switch SA]

[driver grasps

steering wheel]

[driver

attentive

again][driver

attentive

again]

Di [steering over threshold]

|| [braking over threshold]

|| [accelerating over threshold]

Di [switch DA]

|| [braking over threshold]

|| [accelerating over threshold]

Di [switch SA]

Di [switch HA]

[driver

attentive

again]

Activation impossible

Di [any switch]


Di [any switch]


Di [any switch]


HAVEit Joint System: Interaction & display schemes


Joint System Demonstrator

VW TAPConti ARC VTEC AQuA

Validation of Joint System principles in simulator and vehicles studies at WIVW, DLR and VW (results documented in D33.1, D33.3, D33.6)

HAVEit Joint System: Evaluation of principles


Subjective evaluation of a highly automated vehicle

(32 participants „Transition study― see D33.6)

HAVEit Joint System: Evaluation of principles


Publications (selection)Flemisch, F. Nashashibi, N. Rauch, A. Schieben, S. Glaser, G. Temme, P. Resende, B. Vanholme, C. Löper, G. Thomaidis: ―Towards highly automated driving: Intermediate report on the HAVEit Joint

System,‖ in European Transport Research Arena Conference (TRA), 2010, pp. 7–10.

Flemisch, F.; Schieben, A.(Ed.): Validation of preliminary design of HAVEit systems by simulation (Del. 33.3). Public deliverable to the EU-commission; Brussels; 2010

Flemisch, F., Schieben, A., Schoemig, N., Strauss, M., Lueke, S. & Heyden, A. Design of human-computer interfaces for highly automated vehicles in the EU-project HAVEit. To be presented at the

International Conference on Human-Computer-Interaction, Orlando, Florida, 9-14. July 2011.

Flemisch, F.O.; Adams, C. A.; Conway S. R.; Goodrich K. H.; Palmer M. T. ; Schutte P. C.: The H-Metaphor as a guideline for vehicle automation and interaction; NASA/TM—2003-212672; NASA

Langley Research Center; Hampton, Va, USA; 2003

Glaser, S., Vanholme, B., Mammar, S., Gruyer, D., & Nouveliere, L., ―Maneuver-based trajectory planning for highly autonomous vehicles on real road with traffic and driver interaction,‖ IEEE Trans.

Intell. Transp. Syst., vol. 11, no. 3, pp. 589 –606, Sep. 2010.

Hima, S., Lusetti, B., Vanholme, B., Glaser, S., and Mammar, S., ―Trajectory tracking for highly automated passenger vehicles,‖ in IFAC World Congress, 2011, submitted for publication.

Hoeger, R.; Amditis A. , Kunert M.; Hoess, A.; Flemisch, F.; Krueger, H.-P.; Bartels, A.; Beutner, A.: HIGHLY AUTOMATED VEHICLES FOR INTELLIGENT TRANSPORT: HAVEit APPROACH; ITS

World Congress, NY, USA, 2008Holzmann, F; Flemisch, F.; Siegwart, R.; Bubb, H.: From Aviation down to Vehicles – Integration of a Motions-Envelope as Safety Technology: SAE 2006

Automotive Dynamics Stability and Controls Conference; Novi, Michigan, USA, 2006

Rauch, N., Kaussner A., Krüger, H.-P., Boverie, S. & Flemisch, F. (2010). Measures and countermeasures for impaired driver’s state within highly automated driving. Paper presented at the Transport

Research Arena Europe 2010, Brussels, Belgium, 07.-10. June 2010.

Rauch, N., Kaussner, A., Krueger, H.-P., Boverie, S. & Flemisch, F. (2009). The importance of driver state assessment within highly automated vehicles. Paper presented at the 16th ITS World

Congress, Stockholm, Sweden, 21.-25. September, 2009.

Resende, P., Nashashibi, F., ―Real-time Dynamic Trajectory Planning for Highly Automated Driving in Highways‖, 13th International IEEE Conference on Intelligent Transportation Systems, ITSC 2010

(2010).

Schieben, A., Temme, G., Köster, F. & Flemisch, F. (in press). How to interact with a highly automated vehicle – Generic interaction design schemes and test results of an usabilty assessment. In D.

de Waard, N. Gerárd, L. Onnasch, R. Wiczorek, and D. Manzey. Human Centred Automation. Maastricht: Shaker Publishing.

Schieben, A.; Flemisch, F. (Ed.): HAVEit Del. 33.6. Validation of concept on optimumtask repartition, 2010

Schömig, N., Kaussner, A. & Totzke, I. (2011). Das Out-of-the-Loop Problem beim automatisierten Fahren - Lösungsansätze im EU-Projekt HAVEit. Presentation at the SafetyWeek 2011- die Zukunft

der Fahrzeugsicherheit. Kooperationsforum Fahrerassistenzsysteme, 26.05.2011, Aschaffenburg, Germany.

Schömig, N. & Kaussner, A. (2010). Assessing the driver state within HAVEit. Invited speaker within the Special Session ―Highly Automated Vehicles and Applications for Intelligent Transport: Tools,

Systems and Application‖ at the 17th ITS World Congress, 25-29 October 2010, Busan, Korea.

Spiegelberg, G. (2002). Ein Beitrag zur Erhöhung der Verkehrssicherheit und Funktionalität von Fahrzeugen unter Einbindung des Antriebstrangmoduls MOTIONX-ACT®. Göttingen: Cuvillier Verlag.

Thomaidis, G., Amditis, A. , Temme, G., Mosebach, H., Flemisch, F. (2011). Integrated perception in automated vehicles, Paper presented at the18th World Congress on Intelligent Transport Systems,

October 16-20, Orlando, Florida.

Vanholme, B., Glaser, S., Mammar, S., & Gruyer, D., ―Manoeuvre based trajectory planning for highly autonomous vehicles on real road with traffic,‖ in European Control Conference (ECC), vol. 9,

2009, pp. 23–26.

Vanholme, B., Gruyer, D., Glaser, S., & Mammar, S., ―Fast prototyping of a highly autonomous cooperative driving system for public roads,‖ in Intelligent Vehicles Symposium (IV), IEEE, Jun. 2010, pp.

135 –142.

Vanholme, B., Gruyer, D., Glaser, S., & Mammar, S, ―A legal safety concept for highly automated driving on highways,‖ in Intelligent Vehicles Symposium (IV), IEEE, Jun. 2011, submitted for

publication.


Migration to HAVEit demonstrators


Thank you, Joint System team and HAVEit partners, for the extremely fruit- and joyful cooperation!

Thank you, EU officers, experts and reviewers, for enabling and supporting this fascinating project!

Generic Concepts

Validation by simulation Validation in trail blazer

Implementation

Migration to

simulators

Migration to

vehicle(s)



Design with theater system

Initial concepts

HAVEit Joint System

3333

Thank you

for your kind

attention !


haveit joint system · the future of driving. final event haveit joint system first hotel grand,...

Documents