ICT work programme 2014-2015

ICT 22

Multimodal and naturalcomputer interaction

Aleksandra Wesolowska (Unit G.3 - Data Value Chain)

Juan Pelegrin (Unit G.4 - Inclusion, skills & youth)

Philippe Gelin (Unit G.2 – Creativity)

Multimodal & Natural human-computer interaction ICT 22 – 2014 – Call 1

Rationale• systems and devices are becoming more and more powerful

• the human-computer interface is lagging behind

• bottleneck for efficient and seamless use

Objective• achieve transparency and invisibility of technology

• effortless, effective human-machine dialogue

• easy use of complex and powerful systems, easy access to information

Multidisciplinary research: communication technologies, language/speech processing, cognitive & behavioural analysis, creative industries…

Various uses: search, information retrieval, elderly, people with special needs, designers/artists... 2

Multimodal & Natural human-computer interaction ICT 22 – R&I action (a)

Several research projects (budget: 7,5 MEUR) on advanced human machine interaction

• interactive, intuitive, multimodal systems: natural ways of delivering answers to user queries

• human-like social agents: communicative, conversational, affective and social capabilities

• autonomous: able to learn and react proactively to new situations• conversational/spoken interaction: able to cope with spontaneous

dialogue, in multiple languages• knowledge-based agents: able to find, retrieve,

combine/integrate information from many sources • multiple delivery platforms for developed technologies• research based on and/or producing freely available & reusable

resources

3

Multimodal & Natural ICT 22 – R&I action (a)human-computer interaction Expected impact

4

ICT 22 – R&I action (a)

• Improve multilingual speech processing and bridge the gap between recognition and synthesis, exploiting metadata and other contextual data

• Increase the automatic inferences capacities on rich context thanks to improved language understanding, sensed environments/objects, use of social media and agent’s experience

Multimodal & Natural ICT 22 – R&I action (b)human-computer interaction Expected impact

R&I projects on Multimodal and Natural interaction(Budget: 16 MEUR)

• Develop multi modal & adaptive interfaces (inc. BNCI) to assist people with disabilities

• Research:

• User interaction with intelligent systems

• user modelling

• How to detect behaviours

• Interoperability standards

• Interaction and cooperation between machine and human intelligence

• Advance human machine interaction to enable disabled/ elderly to participate fully in society 5

Multimodal & Natural ICT 22 – Innovation action (c)human-computer interaction

Several Innovation projects on multimodal interfaces (7,5 MEUR)

What is exected• more efficient and natural ways of interacting with computers • improve users' experience.

How• Create non-intrusive interaction with humans where real and virtual content

are blended.

Starting From• one or multiple smart devices and sensors (scene analysis, voice recognition, human position, gestures and body language detection,…)

Thanks to • Built with a user centric approach

Result:• cost effective; • address clear market needs • be validated (in domains such as those of the creative industries fields.)

6

Multimodal & Natural ICT 22 – Innovation action (c)human-computer interaction Expected impact

7

ICT 22 – R&I action (a)

• Enable better uses of ICT technologies within the creative industries by providing directly usable solutions addressing their specific needs.

• Provide a large spill over of the knowledge acquired to a maximum of European industries.

• Improve the competitive position of the European industries through the provision of cost effective, innovative and high-value products and services.

8

Thank you!

ICT 22 action (a)Aleksandra.Wesolowska@ec.europa.eu

ICT 22 action (b)Juan.Pelegrin@ec.europa.eu

ICT 22 action (c)Philippe.Gelin@ec.europa.eu