empathic tele-existence
TRANSCRIPT
![Page 1: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/1.jpg)
EMPATHIC TELE-EXISTENCE: COLLABORATION ACROSS TIME AND SPACE
Mark BillinghurstUniversity of South Australia
January 16th 2018
Boston
![Page 2: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/2.jpg)
![Page 3: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/3.jpg)
Travelled 10,855 miles in 34.5 hours for a 20 minute talk …
![Page 4: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/4.jpg)
Adelaide - 35o C/95o F
Boston - 1o C/35o F
![Page 5: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/5.jpg)
![Page 6: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/6.jpg)
“Communication is not only the essence of being human, but a vital property of life.”
- John A. Piece
![Page 7: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/7.jpg)
A wide variety of communication cues used.
SpeechParalinguisticPara-verbalsProsodicsIntonation
AudioGazeGestureFace ExpressionBody Position
Visual
Object ManipulationWriting/DrawingSpatial RelationshipObject Presence
Environmental
Face to Face Communication
![Page 8: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/8.jpg)
Teleconferencing Today
![Page 9: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/9.jpg)
Limitations with Current Technology
• Lack of spatial cues• Person blends with background
• Poor communication cues• Limited gaze, gesture, non-verbal communication
• Introduction of artificial seams between physical/digital• Separation of task/communication space
![Page 10: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/10.jpg)
Augmented Reality Teleconferencing
• Natural spatial cues• Blend physical and virtual• Rich communication cues
![Page 11: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/11.jpg)
To Make This Science Fiction Fact..• Display
• Unobtrusive AR display• Tracking
• Robust AR tracking• Capture
• Real time volumetric capture of people• Networking
• High bandwidth, wireless networking
![Page 12: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/12.jpg)
Nearly 20 Years of Research ...
1999Live video texture
2003Volumetric Video
2008Live Broadcast
2016Live Volumetric Capture
![Page 13: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/13.jpg)
![Page 14: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/14.jpg)
2016
https://www.youtube.com/watch?v=7d59O6cfaM0
![Page 15: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/15.jpg)
Are We There Yet?• Display
• Vuzix, Lumus• Tracking
• SLAM, ARKit, ARCore• Capture
• Microsoft, Mimesys, DoubleMe• Networking
• Gb/s, 5G
Vuzix Blade
Mimesys
![Page 16: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/16.jpg)
Going Beyond Being There • Better than Face to Face conferencing
• Changing body scale • Moving inside another’s body• Interacting with virtual content• Capture/replay of conferencing• Making multiple copies of yourself• Sharing enhanced communication cues
![Page 17: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/17.jpg)
Mixed Space Collaboration (2017)
• Make 3D copy of real space• AR user in real world, VR user in 3D copy of real space• Share virtual body cues (head, hands, gaze information)
Real World Virtual World
![Page 18: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/18.jpg)
Virtual Communication Cues
• AR/VR displays• Gesture input (Leap Motion)• Room scale tracking
![Page 19: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/19.jpg)
![Page 20: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/20.jpg)
Multi-Scale Collaboration
• Changing the user’s virtual body scale
![Page 21: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/21.jpg)
![Page 22: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/22.jpg)
From Tele-Presence to Tele-Existence• Tele-Presence
• Enabling you to feel that remote people are part of your space
• Tele-Existence• Enabling you to feel like you are in a remote place
![Page 23: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/23.jpg)
Handheld Tele-Existence Examples
![Page 24: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/24.jpg)
Head Worn Camera Views
• Camera + Processing + Display + Connectivity• Ego-Vision Collaboration (But with Fixed View)
![Page 25: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/25.jpg)
Shared Sphere – 360 Video SharingShared
Live 360 Video
Host User Guest User
Theta S360 Camera
Hi-res Camera
Epson BT-200See-through HMD
Oculus RiftHMD
Leap Motion
![Page 26: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/26.jpg)
![Page 27: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/27.jpg)
Empathic Computing
Systems that allow us to share what we are seeing, hearingand feeling with others..
![Page 28: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/28.jpg)
Empathy Glasses
• Combine together eye-tracking, display, face expression• Implicit cues – eye gaze, face expression
++
Pupil Labs Epson BT-200 AffectiveWear
![Page 29: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/29.jpg)
AffectiveWear – Emotion Glasses
• Photo sensors to recognize expression• User calibration• Machine learning• Recognizing 8 face expressions
![Page 30: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/30.jpg)
Empathy Glasses in Use
• Eye gaze pointer and remote pointing• Face expression display
![Page 31: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/31.jpg)
https://www.youtube.com/watch?v=CdgWVDbMwp4
![Page 32: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/32.jpg)
Technology Trends• Advanced displays
• Wide FOV, high resolution
• Real time space capture• 3D scanning, stitching, segmentation
• Natural gesture interaction• Hand tracking, pose recognition
• Robust eye-tracking• Gaze points, focus depth
• Emotion sensing/sharing• Physiological sensing, emotion mapping
![Page 33: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/33.jpg)
Emotion Sensing HMDs are Coming• Looxid
• EEG/Emotion sensing• Eye tracking
• Emteq• EMG sensing
• Neurable• EEG/BCI for VR
• PhysioHMD (MIT Media Lab)• GSR, PPG, Emotion Sensing
![Page 34: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/34.jpg)
Technology Trends
• Advanced displays• Real time space capture• Natural gesture interaction• Robust eye-tracking• Emotion sensing/sharing
Empathic Tele-Existence
![Page 35: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/35.jpg)
Empathic Tele-Existence
• Move from Observer to Participant• Explicit to Implicit communication• Experiential collaboration – doing together
![Page 36: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/36.jpg)
Future Research• Many directions for future research
• User evaluation• Communication models • Novel interactions• Scaling up• Emotion capture/sharing• Space/body capture• Possible applications
![Page 37: Empathic Tele-Existence](https://reader036.vdocuments.mx/reader036/viewer/2022062504/5a64ed1e7f8b9afd398b45cf/html5/thumbnails/37.jpg)
Conclusions• AR Reality Conferencing
• New types of seamless remote collaboration• From Tele-Presence to Tele-Existence
• Trend toward Empathic Computing• Sharing what you see, hear and feel
• Many directions for future research• Capture/sharing emotion, interface design, etc..