virtual storytelling 2001, avignon - e. badique - under construction in europe - virtual and mixed...
TRANSCRIPT
Virtual Storytelling 2001, Avignon - E. Badique’
- Under construction in Europe -Virtual and Mixed Reality
for a Rich Media Experience
Eric Badique’
Virtual Storytelling 2001, Avignon - E. Badique’
On behalf of….
INRIA
UCL
U.LEEDS
IST
ADETTI
KULUCL
CSELT
UPCUPM
PHILIPSHHIGMD
FHG
U.Munchen
P.MilanoREALVIZ
U.GENEVAEPFL
INTRACOM
RSU
Tampere U.VTT
MATRA
Symah
BT
BBC
SONY
Siemens
NOKIA
Erricson
TU. Wien
~ 15 projects
~60 organisations
15 countries
Virtual Storytelling 2001, Avignon - E. Badique’
Reality-Virtuality Continuum
Mixed RealityAugmented Reality Augmented Virtuality
Real environment Virtual environment
Virtual Storytelling 2001, Avignon - E. Badique’
Why Mixed Reality ?
• Most attractive and modern media exploit a form of mixed reality
• Enabling technology for the future of media
• Important investment in AV representation and coding over past 10 years (MPEG…)
• Significant activities in CG, visualisation, VR
• Some activities in computer vision
• Efforts to promote synthetic/natural coding for 5 years + (MPEG-SNHC..)
• Combines realism of real world pictures with flexibility of CG
• Multi-disciplinary - good framework for collaborative projects
• P.Milgram, MR Labs and ISMR’99 did provide some inspiration and a framework...
• … but MR picture was refined over several years
MR as main theme for IST WP2000-2001
Virtual Storytelling 2001, Avignon - E. Badique’
Projects portfolio
Analyzing and modeling the world
Merging Real & Virtual
ContentInteractivity
Audio-VisualInteractiveServices
Virtual Storytelling 2001, Avignon - E. Badique’
RE
VE
Communication
TelepresenceVIRTUE
ARARTHUR
ARIS
EnhancedNarrative
Space ALIVE
EmotionalInterface
INTERFACESAFIRAMEGAHybrid IB/3D
renderingINVIEW
3D modeling(uncalibrated)
VISIREEVENTS
Medical Imaging
VR-SUR
2nd GeniTVPISTE MR
for trainingSTAR
Programmeproduction
ORIGAMICrowd
SimulationCROSSES
ExcavatorSimulation
SHE
Consumption
Current IST MR Portfolio
interactivity Next gen.
Portals
SONG
Virtual Storytelling 2001, Avignon - E. Badique’
The full story...
ResearchStandards
Content, authoringtools & stories
Mixed Reality technologies
ApplicationsiTV
Culture &art
Industry Mobile services
IST/KAIII
Virtual Storytelling 2001, Avignon - E. Badique’
Research
• Computer graphics and animation
• Emotional interfaces
• Components and systems• Human factors and usability -> Guidelines and fitness-for-purpose measures
CalibrationBackground segmentation, reference image managementHandling illumination changes and shadow detectionSegmentation and object extractionMotion estimation, motion segmentation , structure from motion Tracking (Head-tracking)Disparity and stereo, stereo vision3D pose estimation and 3D tracking
3D reconstruction, 3D geometry from video.High-level features extractionData fusionMosaickingView synthesis and renderingReal-time aspectsBenchmarking
• Image Processing & Computer vision -> Clustering and Bench-marking
Virtual Storytelling 2001, Avignon - E. Badique’
Standards
• Bridging the gap between coding and CG world (MPEG-4-SNHC/VRML/Web3D)
• Rich media standards for next gen. iTV (DVB/MHP-MPEG-4)
• Agents (FIPA)
• Indexing, search and retrieval (MPEG-7, TV-Anytime..) From low level (shape, size, texture, color..) to semantic information
This area is at the forefront of technological developments and should play its ‘ standard-pulling ’ role
Virtual Storytelling 2001, Avignon - E. Badique’
MR opportunities
• Completely new markets ?
• Provides bandwidth hungry applications: Distributed mixed reality environments are the essence of future communication services - provide requirements for future networks
• “Revaluation” of manual work through AR
• Next wave of innovation in industry following CAD and robotics
• Next generation of interactive TV
• AR as new telescope and microscope of the 21st century ?
• Mixed Reality also to be seen in a wider context: Integration of visualisation and interfaces with mechanics, robotics and appliances
Virtual Storytelling 2001, Avignon - E. Badique’
Upcoming FP5 calls
• IST 7th call (deadline October 17, 2001)
– IV.4.2 Mixed realities and new imaging frontiers
– III.1.1 Publishing digital content
– III.5.1 X-content future
– FET Open
• IST 8th call (November 15, deadline February 21, 2002) – IV.5.1 Networked audio-visual systems and services
– CPA 15 Technology platforms for cultural and arts creative expressions
Virtual Storytelling 2001, Avignon - E. Badique’
WP2002 - CPA15: Technology platforms for cultural and artistic creative expressions
• Generic platforms for artistic and cultural content creation..
.. in media production, cultural assets exploitation, artistic design, contemporary arts and performance.
• Exploit technologies for virtual, augmented and mixed realities, new displays, multisensory and multimodal interfaces.
• Involve independent artists, young talents, cultural designers and creators.
• Invent new forms of content and applications.
Virtual Storytelling 2001, Avignon - E. Badique’
FP6: Media Futures - From information to experience?
Context and enhanced interactivity, storytelling Multi-sensor(y) data collection (including real-time) Visualisation (including 3D, VR and Mixed Reality
etc..) Delivery over heterogeneous networks (including
broadcasting) Databases, knowledge management & statistics Metadata Personalisation (including multi-linguality)
Virtual Storytelling 2001, Avignon - E. Badique’
ImTV
CDTV
Interactive TV
VoD
VCR
Rock concertFootball
Theatre
Cinema
IMAX
Consumer electronics Real live events
Immersive TV: Closing the Gap to Live Sensation
Sensation
Costs
[Source: ITC]
Virtual Storytelling 2001, Avignon - E. Badique’
We need
• High fidelity representation of the (MR) world
• Use our five senses
• Continuous acquisition of data about the world around us
• Assimilation of data with information and knowledge from the past
• Organising information in a way that people experience, by time and space and around concepts, not keywords.
Storytelling to play a central role ?
Virtual Storytelling 2001, Avignon - E. Badique’
First experiences Portal for live sport events / tracking and alarm – integration with database for past and
future events (Praja.com) 24 hours boats tracking - race around the world - stats and AV coms Cars tracking and inclusion in a game in real-time – merging broadcasting and gaming
(Kalisto) Multi-camera recording - possibility to choose viewing angle (Superball 2000, Kanade) Augmentation of sport (other) TV broadcasting with interactive elements (3D,
animation, statistics etc…- eg. IST Piste project) Enhanced narrative spaces (IST Art.Live) ...
Emphasis on integration
Powerful concept for new media
FP6 Contributions welcome
Virtual Storytelling 2001, Avignon - E. Badique’
FocusFocusAdded-value Added-value Clear objectivesClear objectivesMulti-disciplinarityMulti-disciplinarityAddress fundamental issuesAddress fundamental issuesStandards and interoperability Standards and interoperability Concentrate on generic aspects Concentrate on generic aspects Enabling technology as opposed to technological gadgetryEnabling technology as opposed to technological gadgetry
If you want to be part of IST...If you want to be part of IST...
Virtual Storytelling 2001, Avignon - E. Badique’
Contacts Information Society Technologies
http://www.cordis.lu/ist
Mixed Reality http://www.cordis.lu/ist/ka4/vision/
- Project descriptions
- Reference documents
- Info on events