touchless interaction from an embodied perspective

Touchless

from

an

Embodied

Interaction

Perspective

Touchless

from

an

Embodied

Interaction

Perspective

Exploring Perceptual and Motor

Gestalt in Touchless Interactions with

Distant Displays

Debaleena ChattopadhyayAdvisor: Davide Bolchni

January 16, 2015

In our everyday life..

we are constantly using our

hands to interact with objects..

Opening a door Closing a door

Holding an objectReaching for

an objectPushing a button

For example..

Wiping

These “natural” gestural

interactions have two typical

properties

Haptic Feedback

A 3D Worldview

But touchless interactions

with distant displays lack

both

Haptic Feedback

A 3D Worldview

Everyday

gestural

interactions

Touchless

interactions

with distant

displays

Public

Spaces

Sterile

Operating

Rooms

Interactive

TVs

Distant

Displays

Touchless

Interaction

Usage contexts

Large-scale

InfoVis

Walter, et al., 2013

O’Hara, et al., 2014

Vatavu, et al., 2014

Dostal, et al., 2014

Haptic FeedbackA 3D Worldview

Touchless

Explore touchless from an

embodied interaction

perspective

Heidegger’s phenomenological view

Gibson’s ecological psychology

Gestalt Psychology

Heidegger’s

phenomenological view

Device – less

interaction

Device – based

interaction

there is no embodied conception of a tool—no

transition of an input device from present-at-

hand (an object of activity) to ready-to-hand

(absorbed in the fabric of the activity)

But touchless interactions

with distant displays lack

both

Haptic Feedback

A 3D Worldview

Gibson’s ecological

psychology

cognition as a construct involving the ability of

an entity, action, and the environment

3D Worldview 2D Display

Violation of how seeing and acting is related in

our familiar 3D environment

Gestalt Psychology

the psychological principles

affecting visual perception, such

as Gestalt principles, will have a

significant effect on touchless

motor action

Preliminary Findings

Perceptual grouping by Similarity of

Orientation principle affected the efficiency of

touchless interaction: Expert users were faster

when crossing-to-select a rectangular menu

option than a circular menu option with a circular

cursor. 1

Perceptual

grouping by

Continuity affected

the effectiveness of

touchless

interactions: Users

were less precise

when aiming an

isolated rectangular

target (A) compared

with a rectangle as

part of a semi-

circular

arrangement (B).2

When intending to make mid-air movements

perpendicular to a vertical display, such as a

pull gesture, users repeatedly made oblique

motions toward the center of their torso—to

optimally reach static equilibrium, thus

minimizing their body’s energy expenditure.

Law of Prägnanz

affected gesture intuitiveness.

3

Our findings are posteriori

arguments …Chattopadhyay, D., & Bolchini, D. (2014). Touchless circular menus: toward an

intuitive UI for touchless interactions with large displays. Proc. AVI, 33 – 40.

Chattopadhyay, D., & Bolchini, D. (2014). Motor-Intuitive Interactions Based on

Image Schemas: Aligning Touchless Interaction Primitives with Human

Sensorimotor Abilities, [Special Issue on Intuitive Interaction] Interacting with

Computers (in press).

Chattopadhyay, D., & Bolchini, D. (2014). Understanding Visual Feedback in

Large-Display Touchless Interactions: An Exploratory Study. [Research Report]

IUPUI Scholar Works, Indiana University.

Proposed Research

Aim 1: Identify how Gestalt principles of visual

perception and motor action affect touchless

user experience (UX).

Aim 2: Design and evaluate Gestalt-informed

touchless interaction techniques for large-

display information visualizations (InfoVis).

controlled, quantitative studies

iteration, formative evaluation

At a Glance

Thank you.We thank the study participants and UITS AVL. This research is partially supported by an IUPUI Research Support Funds Grant.

1

2

3

4

Touchless interaction with distant, 2D displays lack haptic

feedback and a 3D world view.

I explore the mismatch between gestural interaction in

everyday life and touchless interaction with distant

displays from an embodied interaction perspective.

I argue that a lack of haptic feedback and a 3D worldview

violates how seeing and acting is related in a familiar

environment.

My preliminary findings suggest that Gestalt principles of

proximity, similarity and Prägnanz may affect touchless.