4 november 2000bridging the gap workshop 1 control of avatar gestures francesca barrientos...
TRANSCRIPT
![Page 1: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/1.jpg)
4 November 2000 Bridging the Gap Workshop 1
Control of avatar
gestures
Francesca [email protected]
Computer Science DivisionUC Berkeley
![Page 2: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/2.jpg)
4 November 2000 Bridging the Gap Workshop 2
Communicating via desktop VR
Communicate verbally using speech Communicate visually using avatars
Body poseBody movement
![Page 3: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/3.jpg)
4 November 2000 Bridging the Gap Workshop 3
Project goals
Build system for controlling avatar gestures
Simple input device such as a pen and tablet
Understand expression using avatar body
How do we express ourselves using avatar bodies?How do we evaluate a user’s sense of expressiveness?
![Page 4: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/4.jpg)
4 November 2000 Bridging the Gap Workshop 4
Control problems
Limited input and complex outputControl interface divides user’s attention
![Page 5: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/5.jpg)
4 November 2000 Bridging the Gap Workshop 5
Nonverbal behaviorsDisplays
GestureFacial expressionGaze
Posture
FunctionsAugment/replace speechRegulate conversationIndicate emotion
CognitionAwarenessConscious control
![Page 6: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/6.jpg)
4 November 2000 Bridging the Gap Workshop 6
Other approaches
Expression selection
Automaticanimation
Naturallanguageinference
Nivi waves to Harry.)
![Page 7: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/7.jpg)
4 November 2000 Bridging the Gap Workshop 7
Gesticulation
Co-occurs with speech
Free form and made up in the moment
Meaning derived from combination of words and gestures
![Page 8: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/8.jpg)
4 November 2000 Bridging the Gap Workshop 8
Demo of Doodler
![Page 9: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/9.jpg)
4 November 2000 Bridging the Gap Workshop 9
Doodler: things that worked
Within a small range, tracking is intuitiveCan produce free form gesturesMovement seems expressiveControl is transparentNo hunting for controls
![Page 10: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/10.jpg)
4 November 2000 Bridging the Gap Workshop 10
Doodler: limitations
Hard to form specific shapesNeed a way to switch between different mappingsHard to do gestures which involve different body parts seriallyLack of avatar proprioception
![Page 11: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/11.jpg)
4 November 2000 Bridging the Gap Workshop 11
Speech independent behaviors
Emblematic gestureReplace wordsSymbolicArbitrary
Affective expressionEmotionState of being
![Page 12: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/12.jpg)
4 November 2000 Bridging the Gap Workshop 12
Demo with writing
Using alphabet
![Page 13: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/13.jpg)
4 November 2000 Bridging the Gap Workshop 13
Writing as control
Discrete symbols+
Continuously valued variables
Intentional and unconscious
![Page 14: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/14.jpg)
4 November 2000 Bridging the Gap Workshop 14
Writing as control
Elements of discrete and continuous control
Letter selects a discrete choiceWay it is written selects values for
Specify multiple variablesExplicit control?Implicit control?Don’t have to look
![Page 15: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/15.jpg)
4 November 2000 Bridging the Gap Workshop 15
Summary
SimpleContinuousImplicitFlowsAbstractUnwittingTransparent
ComplexDiscreteExplicitConsciousIntentionalSymbolic
![Page 16: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/16.jpg)
4 November 2000 Bridging the Gap Workshop 16
Future work
Design and record gesture vocabularyMap variations in writing to variations in gesture animationFigure out how to analyze expressive aspects of control
![Page 17: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/17.jpg)
4 November 2000 Bridging the Gap Workshop 17
Facing the Gap
How to think about expressionComputer sciencePsychologyHuman-computer interactionPuppetryNew culture
How to study it
![Page 18: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/18.jpg)
4 November 2000 Bridging the Gap Workshop 18
Explore design space
![Page 19: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/19.jpg)
4 November 2000 Bridging the Gap Workshop 19
Two control schemes
Continuous control with free form motionDiscrete control with more complex motion and multiple parameters
![Page 20: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/20.jpg)
4 November 2000 Bridging the Gap Workshop 20
Other solutions continued
Analysis of textComicChat uses keywords, acronyms, punctuation, etc.
Semi-autonomous behaviorsBodyChat by Vilhjálmsson
Simple kinematic controlsSliders and similar widgets (e.g.. Slater)
Full body motion capture
![Page 21: 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley](https://reader036.vdocuments.mx/reader036/viewer/2022062803/56649f485503460f94c69f32/html5/thumbnails/21.jpg)
4 November 2000 Bridging the Gap Workshop 21
Other solutions for nonverbal communication
Discrete choices (menus) of expressions
Usually affective (happy, sad, angry…)
Usually facialUsually used with chat environmentsExamples:
Emotion wheel in ComicChatPalaceGesture/Mimic panel in Vlnet