situated interaction (insitu) - lri.fracl/aeres2013/presentations/... · 27-28 nov. 2013 comité...

$: Situated Interaction (InSitu) - lri.fracl/AERES2013/Presentations/... · 27-28 Nov. 2013 Comité d'évaluation du LRI 1 Head: Wendy Mackay Situated Interaction (InSitu) ... Comité$
27-28 Nov. 2013� Comité d'évaluation du LRI� 1�

Head: Wendy Mackay�

�

�

Situated Interaction (InSitu)�

27-28 Nov. 2013! Comité d'évaluation du LRI! 2!

InSitu Group members!

Permanent faculty!!Caroline Appert, CR1 CNRS !Michel Beaudouin-Lafon, PRCE P-11!!Anastasia Bezerianos, MC P-11 !Olivier Chapuis, CR1 CNRS!!Cédric Fleury, MC P-11 !Stéphane Huot, MC P-11!!Wendy Mackay, DR1 INRIA !Theophanis Tsandilas, CR1 INRIA!

Ph.D. students!!David Bonnet !Jérémie Garcia!!Ghita Jalal !Alexandre Kouyoumdjian!!Can Liu !Justin Mathew!!Cyprien Pindat !Daniel Strazzula !!!Oleksandr Zinenko!

Post-docs !!Lora Oehlberg !Halla Olafsdottir!

Engineers!!Gabriel Tezier !Monireh Sanaei!!Jonathan Thorpe !Amani Kooli!


Computers are everywhere!


Computer!Capa

city!

Computer capacity is increasing…!

Time!


Computer!Capa

city!

Computer capacity is increasing…!

but human cognitive and sensorimotor skills are not.!

Time!

Human!


Artificial Intelligence!

2001 A Space Odyssey

Creating Smart Computers


Human-Computer Interaction�

Iron Man

Augmenting Human Capabilities


Human-�Computer�Interaction�

Artificial�Intelligence�

Social media�Multimedia�

Three interaction paradigms�

Computer as tool�First person interfaces�Empower users�

�Computer as servant�

Second person interfaces�

Delegate tasks��Computer as medium�

Third person interfaces�Communicate�

�


Powe

r!

Trade-off!

How can we keep simple things simple …!but enable complex tasks?!

Simplicity!


Powe

r!

Research Challenge!

How can we move the curve?!

Simplicity!


Scientific Themes & Objectives!

InSitu emphasizes situated interaction !!We develop theory, methods and technology that! !• generate new forms of interaction, !!• significantly empower users, and!!• inspire the next generation of interactive systems.!


Positioning!

France!!Closest groups: Grenoble, Toulouse, Lille!

Europe!!UK Labs (Lancaster, RCA, Glasgow)!!Scandinavian labs (KTH, Aarhus)!!German labs (Aachen, Hasso Platner)!

North American Universities!!Stanford, MIT, Carnegie Mellon,!!U. Toronto, UCSD, Georgia Tech!

!Corporate labs!!Microsoft (Redmond & Cambridge), Google!


InSitu Research Themes!

Interaction and visualization paradigms!!Manage the trade-off between power and simplicity!

Mediated communication!!Establish peripheral and focused awareness at a distance!

Research methods!!Engage users throughout the design process!

Engineering of interactive systems!!Create tools and platforms for advancing the state of the art!


design experiment!

build system!

develop working !prototypes!

no standard benchmarks!

operationalize!

run experiment!

extract key features!

controlled lab studies,!longitudinal field studies!

develop theory!

analyze results!

Fitts’ law, human !perception!

quantitative and qualitative!

get an idea!

frame paper!

participatory design to!avoid toy problems!

key insights to generalize!

HCI Research Process!


Research Process!

evaluation!observation! new!evaluation!

model! new !model!

new!model!

Real world!

Theory!


Research Process!

evaluation!observation! new!evaluation!

model! new !model!

new!model!

Artifact! prototype!prototype! system!

Real world!

Theory!


Research Contributions!

Fundamental research!!investigate details of human behavior!

Invention!!design & benchmark novel interaction techniques!

Real-world exploration!!participatory design with users in real-world settings!

Innovation!!transfer our technology!

!


Interaction and visualization paradigms!

Interaction on the desktop: Manage the trade-off between power and simplicity!

DynaSpot! JellyLens! Octopocus! Dwell-and-Spring!

Combine area and point cursor to

improve performance!

Add undo to!direct manipulation,!e.g., drag ‘n drop!!

Learn complex gestures with a dynamic guide !

Adapt dynamically to object’s shape for!better focus+context!


Example: JellyLenses!

Contribution: Invention!Take advantage of an underlying representation!to improve the trade-off between focus and context !!JellyLenses !• adapt dynamically! to the object’s shape!• optimize placement of ! focus, context & transition ! regions!• consistently outperform ! fisheye lenses!!! !!!


Interaction and visualization paradigms!

Interaction off the desktop Manage the trade-off between power and simplicity!

Musink*! WILD pan & zoom! BiPad! EWE!

Augment paper scores with user-

defined annotations!

Eyes-free entry of commands with rhythmic input !

Bimanual tablet interaction: taps,

chords & gestures!

Compare 12 wall-size interaction techniques across 3 dimensions!

*Zoom on interactive paper!


WILD Pan & Zoom!

Contribution: Invention!Create and evaluate 12 multi-scale navigation techniques!3 Factors: uni/bimanual, guidance & movement type!Identify individual and interaction effects!

Nancel, Wagner, Pietriga, Chapuis & Mackay (2011) CHI ’11 (Best Paper)!


Mediated Communication!

Establish peripheral and focused awareness !!at a distance!

WeMe! Marker Clock! CHI’13 Scheduler!

Multi-scale mediated

communication !

Collaborative constraint solving with crowd-sourced

data, 1000+ events!

Elderly peer-care via peripheral

awareness !

Ambient & intentional

sharing!

Pêle-Mêle!


Example: ICI-TV!

Contribution: Innovation!Research from the EU InterLiving project on !!mediated communication led to the ICI-TV OMTE!!project with Presto, a CEA startup!

!Praesto provides services!to the elderly via a !television set-top box !

Emphasizes !bilateral communication!to help the elderly!age in place!!! !!!


Research Methods!

Engage users throughout the design process!!with participatory design!

Generative Walks! Touchstone! Inksplorer! Prism!

Structured walk- throughs of socio-technical principles !

7-month field study!of a hybrid paper-

electronic notebook!

Support transition between paper and

electronic exploration!

Evaluate trade-offs and run controlled!

experiments!


Example: Prism�

Contribution: Real-world exploration�Participatory design with biologists (Institut Pasteur, INSERM, INRA) �Design of a hybrid interactive paper/electronic laboratory notebook�Field study with INRA biologists (7 months)��

Tabard, Mackay & Eastmond (2008) ACM CSCW'08�


Engineering of Interactive Systems!

Create tools and platforms that support our research !!and advance the state of the art!

!

FlowStates! JBricks! Substance!

Toolkit that combines data flows

& state machines!

Middleware for multi-surface applications with diverse input!

Platform for zooming through massive datasets

on a wall display!


FlowStates!

Contribution: Fundamental research!!Combine two computational models to increase power of expression!

!!Data Flow: ! ! ! !State Machines:!!Low-level event streams ! !Higher-level interactions!

!!!

!!!!

Appert, Huot, Dragicevic & Beaudouin-Lafon, IHM’09 (Best paper)!

Tracking!Menu


Demos

Gimlense: a multi-view, detail-in-context visualization technique that lets users navigate complex 3D models by interactively drilling holes into their outer layers to reveal objects that are buried, possibly deep, into the scene.

Gliimpse: a quick preview technique that smoothly transitions

between document markup code (HTML, wiki markup or LATEX) and its visual rendering. It supports rapid overviews of code-to-document mappings, saving screen real-estate.

WILD interactive visualization of CHI 2013 schedule:

(500 presentations in 200 sessions, with 13 parallel tracks over four days). Abstracted into a simpler classification task for an experiment to compare desktop and wall-sized interaction.


Scientific Production

Publications!!Journals !22 major + 3 minor !!Conferences !77 !(41 major** + 36 mid-level) !!!Book Chapters !7!!Edited Proceedings !5 !

!Contracts !6,343 M€!

!!Keynotes !5!Invited Talks !81!Software & Toolkits !13!!

** !A+ ACM/SIGCHI !23 ACM/CHI, 7 ACM/UIST, 1 ACM/CSCW!


Teaching!

Theses !11 !defended!Habilitations ! 2 defended!!Created 2 International HCI Masters programs

Master Research HCID Professional (ICT Labs)

Teaching in 16 Master-level courses Master’s Research seminar (CS department) Ph.D. Seminar Post-thesis research careers

MIT, Disney/CMU, Microsoft Research, U. Munich, U. Copenhagen, Alcatel Lucent, Samsung research Telecom Paris-Tech, U. Lyon, U. Grenoble

!!

!


Honors and Visibility

Honors CHI academy Gilles Kahn thesis prize 9 Best Paper or Honorable Mention Awards

ERC advanced grant IUF Senior Marie Curie grant (with MIT)

Academic exchanges

Muse: Equipe Associée with U. Toronto SIRIUS: Equipe Associée with Stanford, UC San Diego, Berkeley Chile ALMA radio-telescope


Diffusion & Professional Service

Conference organization !ACM/CHI’13 in Paris!! !6 days, 1000+ events, 3500 attendees!

Editorial Boards !7 (all international)!Program Chairs !12 (10 international)!Program Committees !52 (46 international)!!Interviews !France Culture, France Inter, France Bleu,

!La Recherche, Science & Vie,!! !Forum Le Monde La Recherche!

!Fêtes de la Science !3 organizers!Demos !uncountable…!!Transfer to startups !2 (ICI-TV, CEA | Buena Vista, SDS)!


�

�

Head: Wendy Mackay�Situated Interaction (InSitu)�


Highlights

Prizes and Honors 9 paper awards, 1 thesis prize 2 ACM/CHI academy members, I IUF member

Major Grants

ERC Advanced, Digiscope EquipEx, Microsoft-Inria Marie Curie, ANR Jeunes Chercheurs, Digiteo

Organization of CHI 2013 in Paris

Teaching 2 International HCI Masters Master’s Research seminar

Interactive paper for �music composition�

Jérémie Garcia Theophanis Tsandilas

Wendy Mackay

InSitu �

�

Motivation�

27-28 Nov. 2013 Comité d'évaluation du LRI 2

Musical score by K. Penderecki � OpenMusic, |RCAM�

Composition process�


Paper

Software

Expression Exploration Execution

Time

Finale OpenMusic Max/MSP

final scores early sketches

Garcia J., Tsandilas T., Agon C., and Mackay W. Inksplorer: Exploring Musical Ideas on Paper and Computer, NIME 2011

Goal�

Design interactive tools that support this process from the early most creative phase


Challenge�

Each composer has a very personal composition style

How to address composers’ diverse strategies?


Research methodology�


experiments quasi-experiments

public demonstrations field studies

Design

Understand users

Evaluate participatory design

prototyping toolkits

interviews technology probes exploratory studies

Musink [CHI�09, Best Paper Award]�


�Writing over a paper score with a personal music vocabulary �

�Interactive tool for defining the syntax and recognition of new gesture vocabularies �

Paper Substrates [CHI�12]�


�Paper-based modules representing composition rules and functions �

�Layered paper components that define rhythms, pitch and spatialization�

Participatory design with composers

Paper Substrates [CHI�12]�


�Substrates Toolkit�

Interactive, modular paper components specialized to specific types of musical data

�Interactive prototypes�

Applications�


�Substrates for specialized notation�

�PaperTonnetz [Demonstrated at the Palais de la Découverte and CHI�13]�

�Fête de la Science, 2011 �

Evaluation - Polyphony�


Controlled study of composition with 12 composers

�Condition A: Graphics Tablet � �Condition B: Interactive Paper�

Research & creation project�

With composer Philippe Leroux to compose a commissioned piece

concert scheduled for June 27, 2014 at IRCAM




Original score



Redrawing shapes over the original score

Main publications�


Musink [Tsandilas et al., CHI�09] � InkSplorer [Garcia et al., NIME�11] �

Paper Substrates [Garcia et al., CHI�12] � PaperTonnetz [Garcia et al., SMC�12, EA CHI�13] �


From in|situ| to HCC

Michel Beaudouin-Lafon

�

�


In|situ| assessment

Created in 2002, rapid growth

International group, outstanding visibility

Spun off three groups

Achieved or surpassed most goals

Beyond the scientific results, contributed to the visibility of HCI in France


What’s next?

InSitu will reach the 12-year limit for Inria groups in 2014

Opportunities:

Evolution of the field of HCI Paris-Saclay Campus

Strategy: more focused research within a strong ecosystem


Evolution of HCI

Towards physical computing Digital fabrication, Mixed reality, Internet of Things, �

Towards social computing Social networks, Crowdsourcing, Collective intelligence, �

More than ever, understand how people

adopt, adapt and adapt to technology


Paris-Saclay

Unique opportunity to be the epicenter of HCI research in Paris-Saclay and in Île-de-France

Digiscope / Digipods projects Interaction specialty in the Paris-Saclay

Master in Computer Science European HCID Masters (EIT ICT Labs)


Human-Centered Computing

The HCC group will be composed of two more focused research groups (and maybe more in the future)

One around the CREATIV ERC Grant

Towards Human-Computer Partnerships One tentatively called ILDA

Interacting with Large DatA


CREATIV – Wendy Mackay

Creating Human-Computer Partnerships Faculty:

Michel Beaudouin-Lafon, Cédric Fleury, Stéphane Huot, Fanis Tsandilas

Co-adaptive instruments Generative design of co-adaptive systems Participatory design w/ creative professionals Real-time interaction and interaction streams Personal language creation Reusable activity patterns


ILDA – Emmanuel Pietriga

Interaction with Large DatA Faculty: Caroline Appert,

Anastasia Bezerianos, Olivier Chapuis Data-centric interaction

Interaction with structured data Multiscale navigation Collaborative interaction Visual perception Application domains: Critical systems & Scientific data


Digiscope

Equipex – 22M� total, 6.7M� from ANR Interaction “in-the-large”: 9 interconnected

interactive rooms for collaborative visualization of complex data


Digipods

Funded by Région Île-de-France Add interactive capabilities to Digiscope

Digicarts “Interaction cloud”

Interactivepaper & pen

3D motion capture

Webcam

Haptic input

Dock for laptop

Cart on wheels

Loudspeakers

High-resolution multitouch screen

Keyboard and mouse

Documents

Multitouch overlay screen


HCC over-arching goals

Understanding fundamental aspects of human behavior in interactive situations

Designing novel interaction and visualization methods

Developing design methods that harness user innovation

Creating advanced tools for creating interactive systems


Other goals for HCC

Develop HCI in teaching programs Develop partnerships with design schools

and design research groups Industry transfer Paris-Saclay HCI Institute?