co-me-di-a cooperation and mediation in digital arts 2007 - 2010 andrew gerzso - ircam co-me-di-a...
TRANSCRIPT
![Page 1: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/1.jpg)
CO-ME-DI-A
COoperation and MEdiation in DIgital Arts2007 - 2010
Andrew Gerzso - IRCAMCO-ME-DI-A coordinator
![Page 2: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/2.jpg)
CO-ME-DI-A partners
• IRCAM (F) - Institut de Recherche et Coordination Acoustique Musique, Paris.
• UNIGE (IT) - Università degli Studi di Genova, Genoa.
• CIANT (CZ) - Center for Art and New Technologies, Prague.
• SARC (UK) - Queen's University, Belfast.
• IEM (A) - University of Music and Dramatic Arts, Graz.
• HFMT (DE) - Hochschule für Musik und Theater, Hambourg.
• HCMF (HU) - Hungarian Computer Music Foundation, Budapest.
![Page 3: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/3.jpg)
Previous Experience(some examples)
![Page 4: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/4.jpg)
Network Performance at SARC
Geographically Displaced Ensembles
[2005]
- three-site Irish Traditional Music Session (with CCRMA, Stanford/USA, SARC and the
University of Washington)
- two-site Christmas laptop concert using Frequencyliator
[2006]
- three-site Christmas concert in 2006 (with CCRMA, Stanford/USA and UEA Norwich, UK)
[2007]
- Strings Apart Piano Duo (with CCRMA, Stanford.USA)
- Disparate Bodies v. 1.: three-site acoustic piece written specifically for the network (Nime
2007)
- Disparate Bodies v. 2.: three-site distributed ensemble piece (with IEM Graz and Hoschule
für Musik und Kunst Hamburg)
[2008]
- Laptop Ensemble (BLISS (Belfast Legion for Improvised Sights and Sounds) and Göteborg’s
Academy of Music and Drama laptop ensemble, Sweden
![Page 5: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/5.jpg)
Network Performance at SARC
Performance in Virtual Environments
[2007]- Second Life as a performance venue (Disparate Bodies v.2)
[2008]
- SL Performance with Gestural Interaction between performer and
avatars
Future Plans
- Use of binaural streaming for spatial music
- Avatar development based on multimodal performance capture
![Page 6: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/6.jpg)
Network Performance at SARC
Distributed Installations
[2008]
2 Places Exhibition (Belfast - Limerick)Audio Feedback loop through two vibrating metal
plates
![Page 7: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/7.jpg)
Unreal bridge
CIANT server
CIANTClient A CIANTClient B CIANTClient X
CIANT SERVER
TCP, UDP multicast
GAME ENGINE
![Page 8: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/8.jpg)
Transatlantic Rehearsal
•Feb. ‘07 - New World Symphony
•Reinbert de Leeuw conductor
•“Correspondances” by Henri Dutilleux
•“Color” by Marc-André Dalbavie
![Page 9: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/9.jpg)
View from Ircam
![Page 10: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/10.jpg)
View from NWS
![Page 11: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/11.jpg)
“Ius Lucis” of Valerio
Sannicandro•Work for two ensembles in two
halls
•Centre Pompidou and IRCAM
•Premiere June 6, 2007
•Ensemble Intercontemporain / Xavier Roth
![Page 12: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/12.jpg)
Important Ideas
•Remote listening of “the other” hall
•Instruments are cross spatialized
•DSP cross synthesis
•Spatialization composition
•Acoustic “signature” of each hall
![Page 13: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/13.jpg)
Hall One : CNAC-GP
![Page 14: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/14.jpg)
Hall Two : IRCAM
![Page 15: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/15.jpg)
EyesWeb - Casa Paganini
![Page 16: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/16.jpg)
QuickTime™ et undécompresseur Video
sont requis pour visionner cette image.
Test for “Rew(ind)”choreography : Hervé Robbecomposition : Andrea Cera
![Page 17: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/17.jpg)
QuickTime™ et undécompresseur Animation
sont requis pour visionner cette image.
![Page 18: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/18.jpg)
Comedia Main Goals
• doing and learning networking through public events, workshops and residencies.
• creation of a common platform for efficient communication and expression.
• organization of residencies in order develop the technical and artistic content.
![Page 19: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/19.jpg)
First Step : Scenarios
•plain description of event or situation
•topology : configuration and function of nodes
•data : audio, video, control, motion capture
•dramaturgy : what should be perceived by audience, participants and external observers
![Page 20: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/20.jpg)
“Comedia bus”
•common communication infrastructure
•continuous flow of data (audio, video, data).
•eases activating network events
![Page 21: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/21.jpg)
“Lecture or Training Network
Event”•can be a simple oral lecture or
demonstration of technology involving audio and video
•the remote audience can express feedback via a local host or spokesperson for the audience
![Page 22: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/22.jpg)
“Robotic Ensemble Remote
Installation”•robotic instruments placed on different nodes on the network
•each robot collects data provided by the other nodes
•each robot reacts to the collected data
•each place has a specific character
![Page 23: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/23.jpg)
“Multi-way Performance with Remote Avatars”•2 sites with 2 musicians.
•avatar : replication of relevant aspects of “the original”.
•avatar data used in interactive processes
•audio, visual, (biometric ?) data.
•audience in each site has a specific perspective of the performance.
![Page 24: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/24.jpg)
“Provokalia Choir Audience”
•one choir with conductor on one node
•two other choirs in remote nodes
•video and gesture data used for interaction
• “etude” to identify most effective and relevant expressive cues
![Page 25: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/25.jpg)
“Remote Performance”
•a concert is transmitted to remote place where audience is
•concert audio “signature” or “image” is rendered to the audience - how to do this ?
•concert performers get feedback from audience
•awareness in audience that their feedback is perceived by the performers - how to do this ?
![Page 26: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/26.jpg)
“Distributed Ensemble and Real Time Composition”• each node has a performer playing scores appearing in
real time on a screen
• conductor controls central computer generates the scores
• performer’s playing is analyzed locally and the derived data acts on central computer and thereby on the composition itself
• uses concept of virtual score
• audience at each node receives a “unified” audio image
• audience sees scores as they are generated
![Page 27: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/27.jpg)
“Bi-directional Interactive
Performance”•two musicians at two nodes
•audio, video, motion capture data used for interaction
•two nodes differentiated by spatialisation and/or dramaturgical aspects
•each audience aware of the other
![Page 28: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/28.jpg)
“Motion Enabled Live Electronics”
• spatial interface placed on stage on each of two nodes
•musicians and dancers act on collaborative virtual instruments controlled via motion capture
•a virtual audio image is mapped out over the network
•audio image is rendered to real and virtual audience spaces
![Page 29: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/29.jpg)
“Network as Acoustic Medium”•network inherently contains delays
(speed of light !)
•delays, reverberation phenomena emerge
•dynamic real time connections to create sound “netscape”
![Page 30: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/30.jpg)
“Network Mediated Social Interaction”• two nodes - each with an audience
• how to build up awareness of the other audience ?
• interactivity via sound and music to increase mutual influence
• work on the expression of a sense of community, “entrainement”, competition, shared goals ...
• scenario can be built up starting with individuals
![Page 31: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/31.jpg)
“Multi-way Performance in Virtual
Environment”•3 sites with 3 musicians
•audio, visual and motion capture data
•content of 3 sites rendered to Internet audience in a virtual environment.
![Page 32: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/32.jpg)
“Playing Together One Computer Music
Application”•one real time computer music program
on which several musicians act
•related to phenomena of “live coding”
•can be improvised or directed
•sound played in each location with same spatialisation and/or acoustic character
•parallel performer communication (chat ...)
![Page 33: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/33.jpg)
SAME Project
![Page 34: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/34.jpg)
“Network Active Listening”
![Page 35: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/35.jpg)
Important Concepts
•local & global rendering
•external observers
•avatars
•virtual performance spaces
•audience awareness
•dramaturgy
![Page 36: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/36.jpg)
Types of Dramaturgy
• projected dramaturgy : one node retains authorship of dramaturgy using other nodes as contributors with their specific expertise.
• directed dramaturgy : each node contributes specific expertise and supports the dramaturgy developed by an invited artist.
• distributed dramaturgy : each node contributes specific expertise and retains authorship of its own dramaturgy.
![Page 37: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/37.jpg)
Context of Comedia
![Page 38: CO-ME-DI-A COoperation and MEdiation in DIgital Arts 2007 - 2010 Andrew Gerzso - IRCAM CO-ME-DI-A coordinator](https://reader035.vdocuments.mx/reader035/viewer/2022081603/56649ef05503460f94bffff7/html5/thumbnails/38.jpg)
That’s where we are !
Questions ?