pervasive pixels

24
Pervasive Pixels Henning Schulzrinne (PI) Henning Schulzrinne (PI) Steven K. Feiner Steven K. Feiner Gail Kaiser Gail Kaiser John Kender John Kender Kathleen McKeown Kathleen McKeown Department of Computer Department of Computer Science Science Columbia University Columbia University

Upload: rogan-rich

Post on 31-Dec-2015

49 views

Category:

Documents


0 download

DESCRIPTION

Pervasive Pixels. Henning Schulzrinne (PI) Steven K. Feiner Gail Kaiser John Kender Kathleen McKeown Department of Computer Science Columbia University. Proposed Research. Goal: seamless mobile multimedia collaboration across distance Integrate advances across fields Collaborative work - PowerPoint PPT Presentation

TRANSCRIPT

Pervasive PixelsPervasive Pixels

Henning Schulzrinne (PI)Henning Schulzrinne (PI)Steven K. FeinerSteven K. Feiner

Gail KaiserGail KaiserJohn KenderJohn Kender

Kathleen McKeownKathleen McKeown

Department of Computer ScienceDepartment of Computer ScienceColumbia UniversityColumbia University

Proposed ResearchProposed Research

Goal: seamless mobile multimedia collaboration Goal: seamless mobile multimedia collaboration across distanceacross distance

Integrate advances across fieldsIntegrate advances across fields Collaborative workCollaborative work Graphical and visual interfacesGraphical and visual interfaces Spoken language understanding and generationSpoken language understanding and generation Vision sensing and understandingVision sensing and understanding Networking and securityNetworking and security

ContributionsContributions

Contextual information managementContextual information management use workflow to determine display contentuse workflow to determine display content multimedia summaries of past and present sessionsmultimedia summaries of past and present sessions

Harmonizing physical and virtual environmentsHarmonizing physical and virtual environments map changing virtual information onto physical displaysmap changing virtual information onto physical displays map layout of physical environment onto virtual spacemap layout of physical environment onto virtual space

Network servicesNetwork services clear, flexible interface to common servicesclear, flexible interface to common services authentication and privacy supportauthentication and privacy support operating system infrastructure for large displaysoperating system infrastructure for large displays

Features of Research InfrastructureFeatures of Research Infrastructure

Large, instrumented multi-display Large, instrumented multi-display workspacesworkspaces

Networked mobile devices of various Networked mobile devices of various capabilitiescapabilities

Transparent and automatic adaptability to Transparent and automatic adaptability to changes of place, platform or groupchanges of place, platform or group

Support for a wide range of hardware and Support for a wide range of hardware and software, from commercial to novelsoftware, from commercial to novel

Pervasive Pixels Teams

HarmonizingPhysical and VirtualEnvironments

NetworkServices

Project Leader

Schulzrinne

ContextualizedInformationManagement

Enabling communicationsSchulzrinne

DatabaseRoss

Summarization

Kender, McKeown

Environment ManagementFeiner

Capture and DisplayNayar, Kender

3D ModelingAllen

WorkflowKaiser

Virtual DisplayNieh

Access and PrivacyKeromytis

Pervasive Pixels Teams

HarmonizingPhysical and VirtualEnvironments

NetworkServices

Project Leader

Schulzrinne

ContextualizedInformationManagement

Enabling communicationsSchulzrinne

DatabaseRoss

Summarization

Kender, McKeown

Environment ManagementFeiner

Capture and DisplayNayar, Kender

3D ModelingAllen

WorkflowKaiser

Virtual DisplayNieh

Access and PrivacyKeromytis

Pervasive Pixels Teams

HarmonizingPhysical and VirtualEnvironments

NetworkServices

Project Leader

Schulzrinne

ContextualizedInformationManagement

Enabling communicationsSchulzrinne

DatabaseRoss

Summarization

Kender, McKeown

Environment ManagementFeiner

Capture and DisplayNayar, Kender

3D ModelingAllen

WorkflowKaiser

Virtual DisplayNieh

Access and PrivacyKeromytis

Pervasive Pixels Teams

HarmonizingPhysical and VirtualEnvironments

NetworkServices

Project Leader

Schulzrinne

ContextualizedInformationManagement

Enabling communicationsSchulzrinne

DatabaseRoss

Summarization

Kender, McKeown

Environment ManagementFeiner

Capture and DisplayNayar, Kender

3D ModelingAllen

WorkflowKaiser

Virtual DisplayNieh

Access and PrivacyKeromytis

Pervasive Pixels Teams

HarmonizingPhysical and VirtualEnvironments

NetworkServices

Project Leader

Schulzrinne

ContextualizedInformationManagement

Enabling communicationsSchulzrinne

DatabaseRoss

Summarization

Kender, McKeown

Environment ManagementFeiner

Capture and DisplayNayar, Kender

3D ModelingAllen

WorkflowKaiser

Virtual DisplayNieh

Access and PrivacyKeromytis

The Department of Computer ScienceThe Department of Computer Science

A growing facultyA growing faculty 5 new assistant professors (pictures and areas here)5 new assistant professors (pictures and areas here) 1 new full professor: Peter Belhumeur1 new full professor: Peter Belhumeur 1 new full professor 9/02: Julia Hirschberg1 new full professor 9/02: Julia Hirschberg

A large student populationA large student population 80 PhDs80 PhDs 155 MS155 MS 380 undergraduate majors380 undergraduate majors

Internal, fluid collaborationInternal, fluid collaboration Interdisciplinary connectionsInterdisciplinary connections

Building Critical MassBuilding Critical Mass

NetworkingNetworking Schulzrinne, Misra, YeminiSchulzrinne, Misra, Yemini links to EElinks to EE

VisionVision Allen, Belhumeur, Kender, NayarAllen, Belhumeur, Kender, Nayar links to computer graphicslinks to computer graphics

Human Computer InteractionHuman Computer Interaction graphical and visual user interfaces: Feiner, Kendergraphical and visual user interfaces: Feiner, Kender spoken and written language: Hirschberg, McKeownspoken and written language: Hirschberg, McKeown links to speech and multimedia in EElinks to speech and multimedia in EE

Areas of GrowthAreas of Growth

Security and systemsSecurity and systems Computer engineeringComputer engineering Computer graphicsComputer graphics TheoryTheory Sofware systems (e.g., programming Sofware systems (e.g., programming

languages)languages)

Administrative SupportAdministrative Support

Institutional cost share on this proposal: $1 Institutional cost share on this proposal: $1 million in totalmillion in total

$100,000/year for 5 years (Provost)$100,000/year for 5 years (Provost) $100,000/year for 5 years (Dean)$100,000/year for 5 years (Dean)

Support for growth: slots, spaceSupport for growth: slots, space Financial supportFinancial support

return in revenue (e.g., Columbia Video Network)return in revenue (e.g., Columbia Video Network)

Technical transfer supportTechnical transfer support dedication of Columbia Innovative Enterprise servicesdedication of Columbia Innovative Enterprise services

Current Departmental FacilitiesCurrent Departmental Facilities

Shared infrastructure of over 20 Sun and Pentium Shared infrastructure of over 20 Sun and Pentium serversservers

Two teaching laboratoriesTwo teaching laboratories 40 seat student lab (CLIC) of Intel Pentium PCs, dual 40 seat student lab (CLIC) of Intel Pentium PCs, dual

projection capability, wireless microphone, video mixer projection capability, wireless microphone, video mixer and multiple camerasand multiple cameras

30 seat Microsoft Research Lab30 seat Microsoft Research Lab

Nortel Meridian telephone switchNortel Meridian telephone switch Research infrastructure of > 500 nodes plus serversResearch infrastructure of > 500 nodes plus servers Staffed by 4 professional system administratorsStaffed by 4 professional system administrators

Research InfrastructureResearch Infrastructure

User-based personalization: user locationUser-based personalization: user location magnetic card readersmagnetic card readers active badge systemactive badge system triangulation on mobile devicestriangulation on mobile devices visual trackingvisual tracking

Stationary setupsStationary setups Seminar roomSeminar room 3 meeting rooms3 meeting rooms 12 faculty offices12 faculty offices 5 public areas5 public areas portable unitsportable units

Seminar roomSeminar room

High-resolution DV video High-resolution DV video camera, 2 pan-tilt speaker camera, 2 pan-tilt speaker cameras, Omnicam cameras, Omnicam omnidrectional audience omnidrectional audience camera, video switcher, 16 camera, video switcher, 16 ceiling mounted ceiling mounted microphones, electronic microphones, electronic whiteboard, 3 XGA high-whiteboard, 3 XGA high-brightness video projectorsbrightness video projectors

Meeting RoomMeeting Room

2 remote-2 remote-controlled pan-tilt controlled pan-tilt video cameras, video cameras, Omnicam, Omnicam, conference table conference table PZM boundary PZM boundary microphones, microphones, automatic audio automatic audio mixer, ceiling mixer, ceiling speakers, 3 high-speakers, 3 high-lumen XGA lumen XGA

video projectorsvideo projectors

Faculty OfficeFaculty Office

Mimio electronic whiteboard, XGA video projector, Mimio electronic whiteboard, XGA video projector, Ethernet speaker phone, wal-mounted pan-tilt video Ethernet speaker phone, wal-mounted pan-tilt video camera, PocketPCscamera, PocketPCs

Public areasPublic areas

Walk-by stations and laboratoriesWalk-by stations and laboratories XGA video projector, steerable ceiling-mounted dome XGA video projector, steerable ceiling-mounted dome

video camera, embedded ceiling mounted netowrked video camera, embedded ceiling mounted netowrked Pentium computer, celing speaker, Andrea DA-400 array Pentium computer, celing speaker, Andrea DA-400 array microphone, Microtouch IBID 600 whiteboard.microphone, Microtouch IBID 600 whiteboard.

Picture herePicture here

Design for walk-by collaborationDesign for walk-by collaboration

Figure 1 hereFigure 1 here

FunctionalitiesFunctionalities

ConferencingConferencing Internet conferencing server to mis IP and PSTN audio streamsInternet conferencing server to mis IP and PSTN audio streams

Interconnection with analog phoneInterconnection with analog phone Digital hybrid connects digital or analog sound to existing telephone Digital hybrid connects digital or analog sound to existing telephone

system in classroomsystem in classroom Netowrk voice-over-IP interface attached to Nortel Meridian PBX for 20 Netowrk voice-over-IP interface attached to Nortel Meridian PBX for 20

simultaneous conversationssimultaneous conversations

Multi-processor servers and IA64 compute and database server, Multi-processor servers and IA64 compute and database server, plus othersplus others

File storageFile storage Face, speaker and fingerprint recognitionFace, speaker and fingerprint recognition 2 printers and tape library system in year 4 for backup2 printers and tape library system in year 4 for backup

Remaining AgendaRemaining Agenda

Research OverviewResearch Overview Contextualized Information Management (Kaiser)Contextualized Information Management (Kaiser) Harmonizing Physical and Virtual (Feiner)Harmonizing Physical and Virtual (Feiner) Network Services (Schulzrinne)Network Services (Schulzrinne)

Genomics testbed (Noble)Genomics testbed (Noble) Responding to reviews (Kender)Responding to reviews (Kender) Telephony demo (Schulzrinne)Telephony demo (Schulzrinne) LunchLunch DemosDemos Visit with administratorsVisit with administrators DemosDemos Small group meetingsSmall group meetings