multimodel hmi for advanced driver assistance

Patrick Nebout, Director Advanced TechnologiesSeptember 20th 2016

Multimodal HMI for Advanced Driver Assistance

Content

Introduction

HMI & Multimodal

Technical Concept

Proof of Concept

Customer Research Findings

Conclusion

Visteon Fast Facts

+11,000EMPLOYEES

9GLOBALTECH CENTERS

14 REGIONAL TECHCENTERS

MANUFACTURINGLOCATIONS22 3.0B

CONSOLIDATEDANNUAL SALES

$

21OPERATING INCOUNTRIES

ARGENTINA, BRAZIL, BULGARIA, CHINA, FRANCE, GERMANY, HUNGARY, INDIA, JAPAN, MACEDONIA, MEXICO, PORTUGAL, RUSSIA, SLOVAKIA, SOUTH AFRICA, SOUTH KOREA, SPAIN, THAILAND, TUNISIA, UNITED KINGDOM, UNITED STATES

Increased Focus on User Experience via Innovative Displays and Gesture

Market Technology Trends

ADAS

Forward Collision WarningLane Departure Warning

Surround ViewPark Assist

Autonomous Driving

V2XDriver MonitorNight VisionPlatooning

High Definition MapsPrecision Positioning

2010-2015 2015-2020

Roadmap for Autonomous Car Technology

Technology Trends Driving Opportunity in Adjacent Products

Visteon Core Business

Complete cockpit electronics portfolio positions Visteon well for continued growth

Multimodal Perception

Environment

Visual Acoustic Tactile Smell Taste

Brain

Environment

Multi Sensor Processing

Voice, Handy/Body Movement, Facial Expression, Gaze etc.

Why visual HMI?

Head-Up Display is Visual Assistance System to increase Driving Safety and User Experience

Goa

ls

Security - driver can keep the eyes on the road while reading an information Relaxed vision - no/low accommodation, excellent perceptibility Low distraction - relevant and intuitive information about driving and ADAS Fast information flow - direct relation between displayed info and environment.

Why Audio HMI?

The human ear is always active.

The ear is permanently listening in all directions.

So far the acoustic channel is hardly used for automotive HMI.

Goa

ls

Bikes

TrafficForward Collision Warning

Pedestrians

Using the capability of the human ear to perceive spatial sounds for intuitive information transfer.

Integrating the driver into his environment through acoustic cues.

Proof of Concept – development pillars

Vehicle & Sound

Vehicle & Sound Scenarios & Simulation Multi modal HMI & Tools

Technical Concept

Interface

3D Virtual Environmentor vehicle sensor frontend

Proximity & Audiocontrol algorithms

Goa

ls

Provide a full development tool chain to simulate, parameterize and test Audio HMI for Advanced Driver Assistance user scenarios

Use production near Software and frameworks for fast conversion to a product

Interface

Multi modal HMI

Parameters Parameters

Proof of Concept – Vehicle & Sound

Multi modal vehicle HMI Demo Scenarios and media

Audio System development Cockpit integration

Proof of Concept – Scenarios & Simulation

Customer specific scenarios in virtual environment

Simulator Experience design

Proof of Concept – Multi modal HMI & Tools

Tool chain for parameterization

GUIs for multi modality

Customer Research – Test group

69%

31%Female

Male

GenderVehicle Segments Age Range

26%

49%

26%18-32

33-45

46-65

2%

71%

27%

D

B

E

Sam

ples

Focus on D and E-segment vehicles Gender mixed population Majority of participants in mid-age range

Customer Research – General Findings

• Distracting in some situations/ too much (23%)

• Could be irritating (14%)

• Unsettling (7%)

• Needs adaptation (5%)

Initial Reaction to the TechnologyInitial Reaction to the Technology

Very Negative

Neutral

Very Positive

*Top 2 Box Score

84%*

• Increased safety, avoids accidents (30%)

• Useful, supportive (21%)

• Shows hints from every direction /for all risks(14%)

• Sounds raise attention (14%)

• Good in general (7%)

• Quicker reaction (5%)

• Especially pedestrian alert very good (5%)

“I don’t want to have signal alerts

all the time, it should be tuned”

Res

ults

Immersive Audio HMI received very positive feedback. Especially the safety factor and its usefulness was considered a benefit

Audio is generally a good addition to visual assistance support but need to be carefully applied

Res

ults

Customer Research – Multi Modality

43% 23% 34%

*Top 2 Box Score

Added value of visual/ audio support

Very Negative

Neutral

Very Positive

91%*

Evaluation of using vision and sound

80%*

56% 25% 19%

Audio Both equally Visual

Benefit of audio and visual warnings**

Importance of audio and visual support

Low

Neutral

High

Using visual and audio support is perceived as a high value add. Overall, an audio signal is perceived more important. Especially the

benefit of audio warnings is rated more important than visual warnings.

Res

ults

Customer Research – Safety perception

Only 1 out of 5 is concerned about potential distraction. The immersive audio HMI technology especially stands for safety. Vast majority imagine having this technology and would be happy to use it.

Safe

Imaginable for next car

Happy to get this support

Enjoyable

Real help

Distracting

82%

80%

75%

73%

64%

20%

Res

ults

Customer Research – Dynamic Sound Evaluation

Informative aspect of warning is significantly improved Dynamic sounds created by Object Based Audio are easier to locate. ADAS use cases can benefit from this technology.

Semantic evaluation of soundsDynamic (moving) sound objects evaluation

Conclusion

Immersive audio HMI is a new dimension of augmented reality in addition to head-up display HMI

Multimodal HMI is an intuitive way for relevant information about driving environment, especially for dynamic objects

Visteon innovation lab framework enables this functionality in the vehicle

www.visteon.com