20021028-videoconferencing-chen.ppt
TRANSCRIPT
![Page 1: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/1.jpg)
Challenging 5 Common Assumptions about
Videoconferencing
Milton ChenComputer Systems Lab
Stanford University
Presented at Internet2 Advanced Applications Track 10/28/2002
![Page 2: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/2.jpg)
Copyright 2002 Milton Chen
The Stanford Video Auditorium
desktop interface
15’ x 5’ video wall
![Page 3: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/3.jpg)
Copyright 2002 Milton Chen
Video Auditorium publicity/usersIntel president Paul Otellini’s Intel Developer Forum keynoteInvited demo to NASA headquarters for Paul G. Pastorek
CANARIE, CanadaCUDI, MexicoComdex, BrazilIBM Almaden LabManhattan College Hopkins Marine Station Stanford Medical SchoolStanford Learning LabStanford Center for Design ResearchBerkeley Bioengineering LabUniversidade Federal do Rio Grande do Sul, Brazil
![Page 4: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/4.jpg)
OutlineCommon assumptions
– Technology1. High-fidelity AV requires dedicated hardware2. Difficult to install and use
– Human factors3. Life size displays are ideal4. Floor control requires interactive frame rate5. Eye contact is difficult
Beyond MCU and H323– Peer-to-peer– Stanford’s Port Bootstrap Protocol– Personal directory
An evaluation of distance learning at StanfordWhy videoconferencing is not ubiquitous
![Page 5: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/5.jpg)
1. High-fidelity low-latency AV requires
dedicated hardware
![Page 6: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/6.jpg)
Copyright 2002 Milton Chen
$700 Pentium 4 computer $7000 systemsoutperforms
Your PC outperforms all dedicated systems
![Page 7: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/7.jpg)
Comparison of videoconferencing solutions
Max number of links
Max video resolution
BW required at 352x288 15fps
NetMeeting 1 352x288 200 Kbps
WIDE DVTS 1 720x480 3000 Kbps
Vbrick 1 720x480 2000 Kbps
Polycom, Sony, … 4 352x288 200 Kbps
AccessGrid, VRVS many 720x480 400 Kbps
Stanford Video Auditorium
16 to more than 100
720x480 100 Kbps
* CUSeeME, iVisit, Yahoo messenger have unacceptable latency
![Page 8: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/8.jpg)
demo
![Page 9: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/9.jpg)
* TrueSpeech 8.5* MPEG-4* Encrypted, AES (Rijndael), streaming* Simultaneous AV recording* Perceptual streaming adapts to network conditions
A scalable AV streaming architecture
audiocapture
audiocompress
audiosend
audioreceive
audiodecompress
audiorender
videocapture
videocompress
videosend
videoreceive
videodecompress
videorender
Copyright 2002 Milton Chen
![Page 10: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/10.jpg)
Copyright 2002 Milton Chen
Beyond MCU and H323
MCU vs. peer-to-peer– Scalability– Ease of deployment
H323 vs. Stanford’s Port-Bootstrap Protocol– Firewall– Ease of deployment
Personal directory
![Page 11: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/11.jpg)
2. Videoconferencing systems are difficult to install and use
![Page 12: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/12.jpg)
Copyright 2002 Milton Chen
One click operationTo use the Video Auditorium
– “Nothing” to install– One click on the html speed dial
<OBJECTCLASSID="CLSID:E80F7B8F-7906-4A89-B59E-B19871F474A9"
CODEBASE="runtime/VA_Start.ocx#Version=-1,-1,-1,-1"> <PARAM NAME="addr" VALUE="stanford -client_only"></OBJECT>
Makes conferencing as simple as surfing the web
![Page 13: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/13.jpg)
3. Life size displays are ideal
![Page 14: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/14.jpg)
Copyright 2002 Milton Chen
Each video should be between 6° and 14° wide
smile recognition time
0
350
700
0 10 20 30
video size (deg of visual angle)
time
(mse
c)
* 12 people sat 10’ from the display Subjectively, people reported 6° as minimum and 14° as ideal. Life size is 12°.
![Page 15: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/15.jpg)
Copyright 2002 Milton Chen
Balance between size and head movements
* 12 people viewed 9 and 36 students on a large and immersive display. Immersive display requires head movements to see all the students.
0%
50%
100%
9 students 36 students
class size
pre
fere
nce
immersive (64°)
large (27°)
9°
14°
7°
4°
![Page 16: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/16.jpg)
4. Effective floor control requires
interactive frame rate
![Page 17: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/17.jpg)
Copyright 2002 Milton Chen
Minimum required frame rate
Interactive 10 fps
Tolerable 5 fps– [Tang and Isaac ’93]
Lip synchronization 5 fps– [Watson and Sasse ’96]
Content understanding 5 fps– [Ghinea and Thomas ’98]
Sign language recognition 1 fps– [Johnson and Caird ’96]
![Page 18: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/18.jpg)
Copyright 2002 Milton Chen
Gesture Detection Algorithm
input image frame difference after erosion
Visualization of algorithm
![Page 19: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/19.jpg)
Copyright 2002 Milton Chen
Requires 10% of full motion bandwidth
0
25
50
75
100
0 100 200 300
time (frame number)
fram
e s
ize (
kb
its)
0
25
50
75
100
0 100 200 300
time (frame number)
fram
e s
ize (
kb
its)
full-motion (10 fps)
gesture-sensitive (0.2 fps)
* MPEG4 encoded at 320x240
![Page 20: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/20.jpg)
Copyright 2002 Milton Chen
Gesture sensitive allows dynamic discussion
15 fps ~0.2 fps 0.2 fps
0
1
2
3
4
5
full motion gesture sensitive low update
spea
ker c
hang
e per
min
ute )
* 8 groups of 4 people during a discussion
![Page 21: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/21.jpg)
5. Eye contact is difficult
![Page 22: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/22.jpg)
Copyright 2002 Milton Chen
Eye contact fires up our brain
[Kampe et al. ’01]
![Page 23: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/23.jpg)
Copyright 2002 Milton Chen
Eye contact is difficult
Looking into the camera Attempting eye contact
![Page 24: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/24.jpg)
Copyright 2002 Milton Chen
Solutions to eye contact
Half-silvered mirror [Rosenthal ’47] MAJIC [Okada, et al. ’94]
ClearBoard [Ishii, et al. ’92]GazeMaster [Gemmell, et al. ’00]
![Page 25: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/25.jpg)
Copyright 2002 Milton Chen
A simple solution
Hydra [Sellen, Buxton, and Arnott ’92]
![Page 26: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/26.jpg)
Copyright 2002 Milton Chen
Eye contact sensitivity is high
Spatial perception task
As good as Snellen acuity[Gibson and Pick ’63]
2 m
0 8.5-8.50
100stdev = 2.8°
Eye
con
tact
(%
)
Angle (deg)
* 6 observers judged 1 looker
looker observer
![Page 27: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/27.jpg)
Copyright 2002 Milton Chen
Sensitivity is symmetricCline ’67
Kruger and Huckstedt ‘69
Anstis, et al. ’69
Stokes ’69
Ellgring ’70
PicturePhonecamera above display
Hydracamera below display
![Page 28: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/28.jpg)
Copyright 2002 Milton Chen
Methodology
* Two rooms can be linked in a videoconferencing session
Observers watch videos of looker and judge eye contact
large display with camera at the center
Record lookers gazing at different targets
![Page 29: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/29.jpg)
Copyright 2002 Milton Chen
Sensitivity is asymmetric
* 16 observers judged recorded videos of 1 looker
![Page 30: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/30.jpg)
Copyright 2002 Milton Chen
An anatomical explanation
looking at you looking sideways
looking up
looking down eye closing
Illustrations from The Artist’s Guide to Facial Expression[Faigin ’90]
![Page 31: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/31.jpg)
Copyright 2002 Milton Chen
Sensitivity is less in conversation
0
25
50
75
100
0 5 10 15visual angle (deg)
eye
cont
act (
%)
* 16 observers judged videos of 1 looker
(down)
recorded
conversation
![Page 32: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/32.jpg)
Copyright 2002 Milton Chen
Sensitivity is less in video
0
25
50
75
100
0 5 10 15visual angle (deg)
eye
cont
act (
%)
* 16 observers judged 1 looker in conversation
(down)
face-to-face
video
![Page 33: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/33.jpg)
Copyright 2002 Milton Chen
We are biased to perceive contact
angle
eye
cont
act (
%)
sideway,up down
down &video
down &video &conversation
Snellen Acuity Conferencing Acuity
0
100
![Page 34: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/34.jpg)
Copyright 2002 Milton Chen
Maximum camera to eyes distance
* Assuming a sensitivity of 7°
device minimum viewing distance
camera to rendered eyes distance
Palm held 1’ 1.5”
Desktop 2’ 3”
Wall size 8’ 12”
![Page 35: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/35.jpg)
Copyright 2002 Milton Chen
Eye contact in the Video Auditorium
![Page 36: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/36.jpg)
Why is videoconferencing essential to distance learning:
An evaluation of distance learning at Stanford
![Page 37: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/37.jpg)
Copyright 2002 Milton Chen
Distance learning at Stanford
Remote students can call in during class
Instructor cannot see the remote students
a 1969 classroom
a 2002 operator console
a 2002 lecture viewer
![Page 38: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/38.jpg)
Copyright 2002 Milton Chen
Students like distance learning
Attitude toward distance learning
0%
50%
100%
students TAs faculty
enjoy
does not matter
dislike
other
* 120 students, 15 TAs, and 41 faculty
![Page 39: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/39.jpg)
Copyright 2002 Milton Chen
Learning is less effective
Learning outcome
0%
50%
100%
students TAs faculty
increasegreatly
increasesomewhat
does notchange
decreasesomewhat
decreasegreatly
* 120 students, 15 TAs, and 41 faculty
![Page 40: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/40.jpg)
Copyright 2002 Milton Chen
F2F interaction is important
Importance of f2f interaction
0%
50%
100%
students TAs faculty
extremely
very
moderately
somewhat
not
F2F is important for lecturing and crucial for discussions
![Page 41: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/41.jpg)
Copyright 2002 Milton Chen
No interaction with remote students
Classroom observation of 4 CS classes– Instructor on average asked 9 questions per
session– Local students on average asked/made 3
questions/comments per session
– Remote students spoke once in 6 month
![Page 42: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/42.jpg)
Copyright 2002 Milton Chen
Value of video beyond audio
Cues only transmitted by the visual channel– Negative feedbacks, …
Emotional bond– Establishing and maintaining relationships
Can you imagine it?– A new face, …
![Page 43: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/43.jpg)
A proposal
![Page 44: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/44.jpg)
The world’s largest video wall:link all Internet2 members for Spring 03
Developed technologyOne Mouse
AV stream migration
Bandwidth: 2 x 300 x (100 Kbps + 10 Kbps) 60Mbps
Cost: 10 P4 laptops + 10 portable projectors $30K
![Page 45: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/45.jpg)
A prediction
![Page 46: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/46.jpg)
Copyright 2002 Milton Chen
A plane that does not fly is not a plane
First flight, Wrights 1903
A videophone that limits communication is not a videophone• poor audio fidelity• poor video fidelity• excessive latency• no eye contact• poor lip synchronization
Why all videoconferencing products has failed
![Page 47: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/47.jpg)
Copyright 2002 Milton Chen
Threshold of quality for the 2nd revolution
first mobile phone, 1924 first handheld phone, 1973
1st Revolution: Possible 2nd Revolution: Practical
first videoconferencing system, 1927
![Page 48: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/48.jpg)
Copyright 2002 Milton Chen
Conclusion
Common assumptions1. High-fidelity AV requires dedicated hardware higher on a PC
2. Difficult to install/use one click
3. Life size displays are ideal 6° to 14°
4. Floor control requires at least 10fps 0.2 fps avg
5. Eye contact is difficult 7° down
Videoconferencing is essential to distance learning
A MCU-less and H323-less future
![Page 49: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/49.jpg)
You already have a one-click high-fidelity multiparty
videoconferencing system
We are at the dawn of a videoconferencing revolution that will fuel the demand for a 1000X increase in available bandwidth
![Page 50: 20021028-Videoconferencing-Chen.ppt](https://reader035.vdocuments.mx/reader035/viewer/2022081603/558d3d1dd8b42a0b318b45fd/html5/thumbnails/50.jpg)
Acknowledgement– NASA– Intel– Sony– Interval Research– Wallenberg Global Learning Network– Department of Defense
Future work– Gold release for Feb 2003– SDK– The Wall