aiming computing technology at enhancing the quality of life of people with als some sketches on...
TRANSCRIPT
Aiming Computing Technology at Enhancing the Quality of Life of
People with ALS
Some Sketches on Directions in Minimal Signaling Communication
Communication Working GroupMicrosoft Research &
ALS Society of British Columbia
Directions• Tools and method for planning and creating long-term
corpora and strategies for communication
• Creating effective input strategies and systems for different phases of the progression of ALS
• Exploration of novel signaling strategies, such as eye gaze, as an effective input modality
• Coupling contextual information as source of evidence in reasoning about intentions
• Potential for integrated robotic systems for motion
• Multiple directions
Core Challeges & Opportunities with Gaze-Centric Systems
• Move beyond potentially frustrating point and dwell approaches
• Toward new UI designs, metaphors & control methodologies that can ease user effort and enhance accuracy of selection, control.
• Adaptive techniques, use of context can ease user effort and error.
• Promise of creating multiple rewarding and valuable applications, for communication, creative expression, information access, enjoyable experiences, control of real-world sensors and effectors.
Sample Applications
Example
• Gaze-centric interaction with (previously authored) databases of images and audiovisual snippets for enhanced communication
Memories library from video and image photolibrary
Social discourse library of videos and stills for sharing emotions, moods, gestures.
Example
• Gaze-centric control of confirmation, negation, selection of alphabetic controls for writing, editing.
Yes No
[ask] [tell] [have]
Q W E R T Y U I O P A S D F G H J K L Z X C V B N M . ?[ I want to …
Predictive language model with correction
Example
• Gaze-centric control of integrated, supportive robotic systems– Head motion– Head gestures (nods, yes; shakes no)– Arms (and hand?) motion …etc.
Turn head left or right (e.g., in increments)
Conversation
Confirmation
Nod “yes”; shake head “no”(Beyond confirmation: nod as a natural cue for understanding in stream with conversation)
Toward more general application of “integrated robotics,” e.g., arms controls, etc., for atomic and patterned moves (e.g., wave hello).
Example
• Gaze-centric control of navigation through cached real-world, or 3D virtual-reality animation sequences, potentially with other appropriate sensory stimulation.
Example
• Gaze-centric communication and linking for single and multiplayer games, including chess, card playing, virtual worlds (Second Life, etc.)
Example
• Gaze-centric control of telepresence applications for real-time viewing from different viewpoints.
Example
• Gaze-centric control of painting, CAD, sculpting machinery, machine lathes, other artistry and crafts.
Longer-Term: Addressing Totally Locked in State (TLS)
• Challenge of use of EEG for preference assessment, acknowledgment, confirmation, selection
• E.g., Rotating selection rings, with “other” option; navigation among hierarchy of rings, use of simple input for halting, selecting, confirmation.
