mixed reality: a model of mixed interaction céline coutrix, laurence nigay user interface...
TRANSCRIPT
Mixed Reality: A Model of Mixed InteractionCéline Coutrix, Laurence Nigay
User Interface Engineering TeamCLIPS-IMAG Laboratory, University of Grenoble
Introduction: Mixed Systems
•Merging Physical and Digital Worlds
•In many application fields
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
1
Introduction: Designers Needs
A framework for :
Understanding and capitalizing on existing Mixed Systems
Descriptive power
Creating designs Generative power
Assessing multiple design alternatives and increasing usability
Comparative power
2
•Mixed Interaction Model
•Descriptive Power
•Generative and Comparative Powers
•Conclusion and Future Work
Outline
3
•Mixed Interaction Model
• Mixed object
• Mixed interaction
•Descriptive Power
•Generative and Comparative Powers
•Conclusion and Future Work
Outline
4
ExampleQuickTime™ et un
décompresseur codec YUV420sont requis pour visionner cette image.
5
Mixed Interaction Model: Mixed Object
mixed object
tool task object
6
Mixed Interaction Model: Mixed Object
physical object
physical properties
digital object
digital properties
mixed object
physical properties
digital properties?
• Interaction modality m = (d, l)
• speech = (microphone, natural language)
7
Mixed Interaction Model: Mixed Object
•Linking modality
mixed object
physical properties
digital properties
inputdevice
outputdevice
output language
acquiredphysical
data
inputlanguage
generatedphysical
data
8
Mixed Interaction Model: Mixed Object
mixed drawing
physical properties shapes
camera
projector selection
image computervision
image
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
9
Mixed Interaction Model: Mixed Object
a music bottle
physical properties
musicpart
loudspeakers identityaudio
sequence
screentable
lightingcolored
sequence
pitch andvolumeanalysis
QuickTime™ et undécompresseur codec YUV420
sont requis pour visionner cette image.
10
Mixed Interaction Model: Mixed Object
mixed object
physical properties
digital properties
Linkingmodalities
tool task object
11
tool
task object
action
reaction
elementarytask
response
feedback
Mixed Interaction Model: Mixed Interaction
12
• Interaction modality m = (d, l)
Mixed Interaction Model: Mixed Interaction
13
mixed tool
physical properties
digital properties
inputdevice
outputdevice
output language
acquiredphysical
data
inputlanguage
generatedphysical
data
interactionoutput
language
interactioninput
language
task object
Mixed Interaction Model: Mixed Interaction
14
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
Mixed Interaction Model: Mixed Interaction
15
mixed tool
paperbutton
buttonpressedword,place
on drawing
camera imagecomputer
vision
translation
drawing
mixed tool
Mixed Interaction Model: Mixed Interaction
16
•Mixed Interaction Model
•Descriptive Power
•Generative and Comparative Powers
•Conclusion and Future Work
Outline
17
•Schemes for classification and better understanding of mixed systems
Descriptive Power
18
•Mixed Interaction Model
•Descriptive Power
•Generative and Comparative Powers
•Conclusion and Future Work
Outline
19
RAZZLE: Case Study
20
RAZZLE: Case Study
mixed piece
physical properties
IdentifierLocation
Orientation
locationsensor
head mounteddisplay
3Dprojection
numericvalues
geometrictransforma°
image
orienta°sensor
numericvalues
geometrictransforma°
21
Generative Powermixed tool
? ?
? ?
?
22
RAZZLE: Gestural Modality
QuickTime™ et undécompresseur DV - PAL
sont requis pour visionner cette image.
23
RAZZLE: Gestural Modalitymixed tool
physicalproperties
recognizedgestures
camera image computervision
translation
24
RAZZLE: Tactile Modality
QuickTime™ et undécompresseur DV - PAL
sont requis pour visionner cette image.
25
RAZZLE: Tactile Modalitymixed tool
physicalproperties
click
touchpad data inputlanguage
translation
26
•Mixed Interaction Model
•Descriptive Power
•Generative and Comparative Powers
•Conclusion and Future Work
Outline
27
Comparative Power
• Criteria within the model for assessing multiple design alternatives
• Observability
• Spatial continuity
28
Comparative Power: Observability
task object
physical properties
digital properties
inputdevice
outputdevice
output language
acquiredphysical
data
inputlanguage
generatedphysical
data
mixed tool
physical properties
digital properties
inputdevice
outputdevice
output language
acquiredphysical
data
inputlanguage
generatedphysical
data
interactionoutput
language
interactioninput
language
29
Comparative power: Observabilitymixed tool
physicalproperties
recognizedgestures
camera image inputlanguage
translation
30
Comparative Power: Observabilitymixed tool
physicalproperties
click
touchpad data inputlanguage
translation
31
task object
physical properties
digital properties
inputdevice
outputdevice
output language
acquiredphysical
data
inputlanguage
generatedphysical
data
Comparative Power: Spatial Continuity
mixed tool
physical properties
digital properties
inputdevice
outputdevice
output language
acquiredphysical
data
inputlanguage
generatedphysical
data
interactionoutput
language
interactioninput
language
32
Comparative Power: Spatial Continuitymixed tool
physicalproperties
recognizedgestures
camera image computervision
translation
physical properties
IdentifierLocation
Orientation
head mounteddisplay
3Dprojectionimage
mixed piece
locationsensor
numericvalues
geometrictransforma°
orienta°sensor
numericvalues
geometrictransforma°
33
Comparative Power: Spatial Continuitymixed tool
physicalproperties
click
touchpad data inputlanguage
translation
physical properties
IdentifierLocation
Orientation
head mounteddisplay
3Dprojectionimage
mixed piece
locationsensor
numericvalues
geometrictransforma°
orienta°sensor
numericvalues
geometrictransforma°
34
•Mixed Interaction Model
•Descriptive Power
•Generative and Comparative Powers
•Conclusion and Future Work
Outline
35
•Definition of a Mixed Interaction Model which
•Capitalizes existing approaches to describe mixed systems
•Allows designers to create new interaction modalities
•Allows them to choose between multiple design alternatives
Conclusion
36
•Improving comparative power by identifying more criteria
Future Work
37
•Providing a development environment based on ICARE
Future Work
38
Mixed Reality: A Model of Mixed Interaction
User Interface Engineering TeamCLIPS-IMAG Laboratory, University of Grenoble
Céline Coutrix, Laurence Nigay