personal interaction via multi-touch gestures
DESCRIPTION
TRANSCRIPT
Personal interaction via multi-touch gesturesA step toward NUI
Alessio Malizia, Associate ProfessorAndrea Bellucci, Ph.D. candidate
Dept. of Computer Science, Universidad Carlos III de Madrid, Spain
SynopsisResearch areasBackgroundProblem(s)Question(s)HypothesisContributionExpected result
Research areas
Human Computer InteractionNatural User InterfacesInteraction/UX Design
Human Factors
1960s –CLICommand Line InterfaceMemorization/Recall
1980s –GUIGraphic User InterfaceRecognition (recall = shortcuts)
2000s –NUINatural User InterfaceRecall? Gestures are like shortcuts
Exploration
TESIS: Turn Every Surface into an Interactive SurfaceDemo @ACM Int. Conference ITS 2011
Don’t touch me!Demo @ACM Int. Conference AVI 2010
Problem(s)
Question(s)
• How can I create an interface in which users may quickly gain expertise using actions that feel natural to them? (research question 1)
• How can I be sure that I succeeded in creating an interface that feels natural to the user?(research question 2)
Hypothesis
• Natural properties are related to universals
• The concept of affordance is crucial for defining a widget-level architecture for NUI– Interface as a Service– Widgets expose their affordances
• Personalization and adaptation/evolution as a step toward natural interaction
I’d like to convince you(thesis)
• Gestures (as signs) are something alive, which change depending on cultural and human aspects, time and context. Therefore users’ personalization can be a step to address natural interaction.
• Refined research question 1: to what extend personalization of multi-touch gestures can be valuable for enabling natural interaction?
Natural properties and Universals
• Concept of perfect natural properties from metaphysics– Perfect natural properties are shared among all the elements
of a set
• For we humans, perfect natural properties are our perception faculties (hearing, feeling by touch, seeing, tasting, smelling) – There are also acquired skills...
• Could there exist such a thing as a natural gesture? – Take into account users innate and learned abilities
Which one is natural?
Zooming in real life: bring the newspaper closer
iPhone: pinch for zooming
Samsung Galaxy s2: holding down on the screen with two fingers and moving your phone back and forth
Affordance
• Why affordance is important for natural interaction?
– Gibson (1977) defined affordances as all action possibilities latent in the environment
– Affordances as perceived action possibilities (Norman, 1988)
– Affordance is the visual clue to the function of the object (an opportunity for action)
Design without affordance
You can move an object in single-pixel increments by nudging it.
Touch and hold the object with one finger, and then use another finger to swipe across the page in the direction you want the object to move.
From iPad’s Pages gestures list
A good affordance
What happens if you attempt to make users interact in a way which is not natural to them?
Beyond affordance languages
Personal/Private
Public/Shared
Small
Medium/High
Personalization
Adaptation/Evolution
Form factor
Display
Expected result