september 29, 2002ubicomp 021 nist meeting data collection jean scholtz national institute of...
TRANSCRIPT
![Page 1: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/1.jpg)
September 29, 2002 Ubicomp 02 1
NIST Meeting Data CollectionNIST Meeting Data Collection
Jean Scholtz
National Institute of Standards and Technology
Gaithersburg, MD 20899-8940
USA
![Page 2: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/2.jpg)
September 29, 2002 Ubicomp 02 2
Goals / ApproachGoals / ApproachGoals• (Intermediate) Support evaluation efforts for speech
community• (Future) Support evaluation efforts for
context-aware/ubicomp communityApproach• Provide rich/diverse pool of audio video corpora for
advanced recognition research • Provide infrastructure for integration and evaluation• Provide flexible architecture for incorporating and
annotating activities in sensor –rich environment
![Page 3: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/3.jpg)
September 29, 2002 Ubicomp 02 3
NIST Data Collection LaboratoryNIST Data Collection Laboratory• Typical meeting space equipped with standard meeting
equipment• Instrumented with
– 200 microphones, 5 cameras – all synchronized with NIST SmartFlow system across 13 processors
– several disk arrays handle huge storage requirements
• Monitor/Review workstation in adjoining room– operator can start/stop data streams, select video views,
audio channels, and manipulate cameras– participants can review meeting recordings and deselect
excerpts from public distribution
• Relational database log with user interface
![Page 4: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/4.jpg)
September 29, 2002 Ubicomp 02 4
EntranceRamp
Whiteboard
Modular ConferenceTable 13’ x 5’
NIST ‘SmartFlow’ synchronization
NIST Meeting Data Collection LaboratoryNIST Meeting Data Collection Laboratory
7’ x
7’ s
cree
n
CeilingProjector
E-White
board
mic micmic
mic
mic
mic mic mic
omnimic
quadmic
omnimic
COTS MicData Collection
System
Wireless Mic
Receivers
omnimic
3 Mic ArrayData Collection
Systems
Mic
Arr
ay
Mic Array
Mic Array
5 VideoData Collection
Systems
![Page 5: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/5.jpg)
September 29, 2002 Ubicomp 02 5
Meeting Review/Monitor Meeting Review/Monitor WorkstationWorkstation •Used to both collect
and review recordings
•LINUX-based, interfaces with Smartflow live or archived data
•User can select video views and audio channels
•User can control camera view/movement with joystick
![Page 6: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/6.jpg)
September 29, 2002 Ubicomp 02 6
Scenario OntologyScenario Ontology
Scenario Type
Scenario Example
DimensionsTasksTime Participants
Dur
atio
n (m
inut
es)
Loc
al v
s. R
emot
e
Sam
e pe
ople
Dif
fere
nt p
eopl
e
Org
. pre
vs.
adh
oc
#of
part
icip
ants
Dom
ain
expe
rtis
e
Mod
erat
or
plan
ning
Bra
inst
orm
ing
Neg
otia
tion
Dec
isio
n m
akin
gC
ompe
titi
ve
perf
orm
ance
Dis
sem
inat
ion
of I
nfor
mat
ion
Different
Focus group discussion
Place
Local 60 5-8 none adhoc
Round Table discussion: May include expert to present info and moderate discussion
Staff meeting Local 45 5-8 adhocParticipants discuss real technical issues, brainstorm ideas and make decisions.
Planning Local 45 5-8 none adhocEvent planning: Plan the different tasks that each member will carry out, timelines and coordination.
Interactive game playing Local 60 3-5none pre
Information exchange and decision making Information gathering and decision making
Local
Local
45
45
5-8
3-5 adhoc
prenone
none
Office Furnishing: An expert will help participants to choose office furniture, carpet, etc. for an office.
Participants engage in a storytelling games, e.g. Once upon a time, Settlers of catan.
Shop on line: Participants search the internet and collaborate to purchase computer hardware.
![Page 7: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/7.jpg)
September 29, 2002 Ubicomp 02 7
Pilot Corpus Design• 20 hours of meetings are being collected
– ~60 GB per hour uncompressed data rate– data stored/distributed on large hard disks
• Varied forums– focus groups, game playing, interacting with experts, real
working group meetings, event planning
• Varied meeting lengths– 15 minutes to 1 hour
• Varied number of participants– 3 to 8 participants
![Page 8: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/8.jpg)
September 29, 2002 Ubicomp 02 8
Sample of Meetings CollectedSample of Meetings CollectedTopic Meeting Type Date Length
(min)
Native
M / F
Non-native
M / F
Participants
IAD New Year Party Planning
Planning 12/12/01 60 0 / 2 0 / 1 3
IAD Open House Planning 3/5/02 56 1 / 4 1 / 1 7
ITL Party Planning Planning 1/11/02 28 3 / 2 0 / 1 6
Bioterrorism Focus group 11/15/01 120 3 / 1 0 / 0 4
Office Furniture Layout
Interaction with expert
2/14/02 55 1 / 3 0 / 2 6
Settlers of Catan Game playing 1/25/02 60 0 / 0 3 / 1 4
Once Upon a Time Game playing 3/4/02 58 1 / 1 2 / 0 4
Real Working Group Staff meeting 2/13/02 69 3 / 2 1 / 0 6
Shop On-line Collaborative design
3/19/02 60 1 / 0 3 / 0 4
![Page 9: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/9.jpg)
September 29, 2002 Ubicomp 02 9
Use of Architecture
Streaming Sensor Input:video, audio, images, location tracks, application interactions
Distributed Services: Smartflow, Device management sw, Distributed data management, Archival
Storage, Service/ device discovery
MetaData Architecture: ATLAS, Annotation/review stations, Annotation ontology, Visualization,
Browsing, Annotation generation, Annotation tools, Logging tools for COTS products, sensor data annotation
Context Architecture: Annotation integration and abstraction, Realtime annotation maintenance,
Context entities (people, activities, artifacts)
Component Technologies: Context abstraction components, simulations for context-aware apps
Corpora
Component test bed
Context-Aware apps
Non-aware apps
Componenttest bed
Test bed
Test bed
Architecture
Component Technologies: Recognition components (ASR, Speaker ID, Face Recognition), Location
recognition, Visualization of sensor data
Architecture
Architecture
Can incorporate component technologies, log interactions, evaluate
Can incorporate context widgets, log interactions, evaluate
![Page 10: September 29, 2002Ubicomp 021 NIST Meeting Data Collection Jean Scholtz National Institute of Standards and Technology Gaithersburg, MD 20899-8940 USA](https://reader035.vdocuments.mx/reader035/viewer/2022062801/56649e485503460f94b3b8b0/html5/thumbnails/10.jpg)
September 29, 2002 Ubicomp 02 10
Future PlansFuture Plans•Extend to distributed meetings/activities
•Provide test bed capabilities to allow developers of applications to incorporate new component technologies and capture user interactions for user-centered evaluations.
• Develop annotation ontology for user interactions to facilitate user-centered evaluations of meeting room applications.
• Extend this capability to context-aware applications allowing developers access to sensor inputs without having to install those sensors in their environment.
• Create community of researchers interested in using this technology