e-science collides with the performing arts

e-Science Collides with thePerforming Arts

e-Science Collides with thePerforming Arts

Dr R P Fletcher

University of York

Dr R P Fletcher

University of York

Thursday, 20 July 2007 Digital Representations of Performing Arts 2

So what is this e-Science??So what is this e-Science??

At the inception of the UK e-Science programme John Taylor said that:

“e-Science is about global collaboration in key areas of science and the next generation of infrastructure that will enable it”

Another source says it is:“science increasingly done through distributed global collaborations enabled by the Internet, using very large data collections, terascale computing resources and high performance visualisation “

Malcolm Atkinson has said that:“e-Science is the development and exploitation of advanced informatics methods to support collaborative research in all disciplines “

At the inception of the UK e-Science programme John Taylor said that:

“e-Science is about global collaboration in key areas of science and the next generation of infrastructure that will enable it”

Another source says it is:“science increasingly done through distributed global collaborations enabled by the Internet, using very large data collections, terascale computing resources and high performance visualisation “

Malcolm Atkinson has said that:“e-Science is the development and exploitation of advanced informatics methods to support collaborative research in all disciplines “


OVERVIEWOVERVIEW

Academic Background – am I a real scientist?

Involvement in Performing Arts – am I a fraud?

Involvement with technology – am I a technologist?

TechnologyBespoke hardware and software

Computers

Networks

Software

Engaging with this technologyRight people in the right place

Future – some ideas

Questions – whatever you want to throw at me!

Academic Background – am I a real scientist?

Involvement in Performing Arts – am I a fraud?

Involvement with technology – am I a technologist?

TechnologyBespoke hardware and software

Computers

Networks

Software


Future – some ideas

Questions – whatever you want to throw at me!


My Academic backgroundMy Academic background

1968 Degree in BiologyGeneral

1971 D.Phil in NeurophysiologyInformation processing in single sensory nerve cells

1975 Research Fellow in PsychologyRisk of schizophrenia

Psychophysiology

1981 Computing ServiceGraphics

Database systems

Programming

Mathematical and statistical systems and libraries

1968 Degree in BiologyGeneral

1971 D.Phil in NeurophysiologyInformation processing in single sensory nerve cells

1975 Research Fellow in PsychologyRisk of schizophrenia

Psychophysiology

1981 Computing ServiceGraphics

Database systems

Programming

Mathematical and statistical systems and libraries


Academic Background

Involvement in Performing Arts

My involvement with technology

TechnologyBespoke hardware

Computers

Networks

Software


Future

Questions

Academic Background




Computers

Networks

Software


Future

Questions


My “Performing Arts” backgroundMy “Performing Arts” background

1965 Staged school bands1968 Staged University bands1971 President of the Electronic Arts Society1972 Recording engineer York Musicians Cooperative1972 Worked with Trevor Wishart1973 Recording/mixing engineer campus band1974 Built own synthesiser system1976 Programmed FX on PDP-111978 Recording/mixing engineer campus band (again)1991 SGI Indigos bootstrap Music Technology1996 Chaotic Constructions1999 RIMM2005 UKLight for Collaborative Composition/Performance 2006 PEACHES committee of TERENA2007 Content Creation tools for Digital Cinema?

1965 Staged school bands1968 Staged University bands1971 President of the Electronic Arts Society1972 Recording engineer York Musicians Cooperative1972 Worked with Trevor Wishart1973 Recording/mixing engineer campus band1974 Built own synthesiser system1976 Programmed FX on PDP-111978 Recording/mixing engineer campus band (again)1991 SGI Indigos bootstrap Music Technology1996 Chaotic Constructions1999 RIMM2005 UKLight for Collaborative Composition/Performance 2006 PEACHES committee of TERENA2007 Content Creation tools for Digital Cinema?


Technology Input and OutputTechnology Input and Output

Specialised software to edit synthetic speech

User interfaces for Music Technology

Courses in graphics for Music Technology

External Examiner MA Digital Arts (Middlesex)

Midi I/O systems for Sun Workstations

Developer for GIMP (video plugins for Unix)

Developer for XMX (collaborative X-Windows)

Developer for .ogg audio formats Still ambisonic developments

Tracking technology for Chaotic Constructions

Graphics system designer/developer for RIMMReal-time Interactive Multiple Media

PEACHES committee at TERENAPanel of Experts for Arts, Culture and Humanities E-Science

Specialised software to edit synthetic speech

User interfaces for Music Technology

Courses in graphics for Music Technology

External Examiner MA Digital Arts (Middlesex)

Midi I/O systems for Sun Workstations

Developer for GIMP (video plugins for Unix)

Developer for XMX (collaborative X-Windows)

Developer for .ogg audio formats Still ambisonic developments

Tracking technology for Chaotic Constructions

Graphics system designer/developer for RIMMReal-time Interactive Multiple Media

PEACHES committee at TERENAPanel of Experts for Arts, Culture and Humanities E-Science


Academic Background




Computers

Networks

Software


Future

Questions

Academic Background




Computers

Networks

Software


Future

Questions


Technologies Technologies

Aim to lead you through some technologies

Ones I have worked on over the last 15 years (or so!)

At some point technology overlaps e-Science

Need to engage with the new technologies

Need to help drive their development

Performing Arts need new/different facilitieseither from the existing infgrastructures

or via new facilities

But, unless we know what’s possible

How do we know what to ask for?

Aim to lead you through some technologies

Ones I have worked on over the last 15 years (or so!)

At some point technology overlaps e-Science

Need to engage with the new technologies

Need to help drive their development

Performing Arts need new/different facilitieseither from the existing infgrastructures

or via new facilities

But, unless we know what’s possible

How do we know what to ask for?


YORKTALKYORKTALK

Worked with the Language and Linguistics staff

Their technology allowed about six synthesis runs PER DAY!

Data transferred via Kermit file transfer program!

Real informed trial and error to fix problems

I created a graphical interface to the data

Ran on a then high performance system

Could interact with the data

Synthesis was up to near real time

Combined visualisation to “see” errors

Used interactive graphics to “fix” the problems

Worked with the Language and Linguistics staff

Their technology allowed about six synthesis runs PER DAY!

Data transferred via Kermit file transfer program!

Real informed trial and error to fix problems

I created a graphical interface to the data

Ran on a then high performance system

Could interact with the data

Synthesis was up to near real time

Combined visualisation to “see” errors

Used interactive graphics to “fix” the problems


YORK TALKYORK TALK

Research vehicle for speech synthesis

DecTalk example (may recognise the voice)

YorkTalk examplesSame words as DecTalk

A sentence (with a cold!)

Research vehicle for speech synthesis

DecTalk example (may recognise the voice)

YorkTalk examplesSame words as DecTalk

A sentence (with a cold!)


Engaging with “artistsEngaging with “artists

Next project was with staff in Music

Needed help to create interfaces for a system

Needed some lateral thinking

Definitely moving into the Arts domain

High performance computing was needed

Multiple tasks by same computerVideo

Audio

Graphics

Midi

Next project was with staff in Music

Needed help to create interfaces for a system

Needed some lateral thinking

Definitely moving into the Arts domain

High performance computing was needed

Multiple tasks by same computerVideo

Audio

Graphics

Midi


Chaotic Constructions - 1994Chaotic Constructions - 1994

Certain amount of serendipity

Peter Fluck talking on Radio 4 about Chaotic motion in kinetic scuplture

Tony Myatt talked a few weeks later about Chaos in Music

Phoned each other classic “WE SHOULD TALK” moment

By 1996 developed a projectto use the chaotic motion of a kinetic sculpture to drive a music composition system

PROBLEM – how to capture the motion

ANSWER – ask me

Certain amount of serendipity

Peter Fluck talking on Radio 4 about Chaotic motion in kinetic scuplture

Tony Myatt talked a few weeks later about Chaos in Music

Phoned each other classic “WE SHOULD TALK” moment

By 1996 developed a projectto use the chaotic motion of a kinetic sculpture to drive a music composition system

PROBLEM – how to capture the motion

ANSWER – ask me


Chaotic Constructions – the Tracking SystemChaotic Constructions – the Tracking System

Request came to meHow can we do this?and …It will be installed in Tate Modern in March 1997and …It was now January 1997!

First thoughts were to use shape tracking systemsProblems:

most systems were proprietary and expensiveshapes to be tracked changed because sculpture was 3 dimensionalcould not use trajectory prediction due to the chaotic motion

ANSWER – simply track unique coloured items

Request came to meHow can we do this?and …It will be installed in Tate Modern in March 1997and …It was now January 1997!

First thoughts were to use shape tracking systemsProblems:

most systems were proprietary and expensiveshapes to be tracked changed because sculpture was 3 dimensionalcould not use trajectory prediction due to the chaotic motion

ANSWER – simply track unique coloured items


VIDEO EXAMPLEVIDEO EXAMPLE

The Making

of

CHAOTIC

CONSTRUCTIONS

The Making

of

CHAOTIC

CONSTRUCTIONS


Stage 1 – Define a Colour “Fingerprint”Stage 1 – Define a Colour “Fingerprint”

Program creates a set of “unique fingerprints”Program creates a set of “unique fingerprints”


Tuning the “Fingerprints”Tuning the “Fingerprints”

Check motion does not lose sync with colour Check motion does not lose sync with colour


Peter Fluck Hand Cranking the SculpturePeter Fluck Hand Cranking the Sculpture


Running with Tracking and Music ProgramRunning with Tracking and Music Program

“Fingerprint” used by second program

Motion tracking is just one element

“Fingerprint” used by second program

Motion tracking is just one element


Changing a Tracking PointChanging a Tracking Point

What are the best points to track?

Due to chaotic motion some trial and error!

What are the best points to track?

Due to chaotic motion some trial and error!


And finally …the prototypeAnd finally …the prototype

… we get there … a small excerpt … we get there … a small excerpt


And the real thingAnd the real thing


Credits and AwardsCredits and Awards

Exhibited to great acclaim in Tate Modern

Other UK Art Galleries

Travelled to Hong Kong and Corsica

Herald Angel Award by the Glasgow Herald for the best piece of visual art at the 1997 Edinburgh Festival

International Critics Award as part of the 1997 Demarco European Art Foundation exhibition

Exhibited to great acclaim in Tate Modern

Other UK Art Galleries

Travelled to Hong Kong and Corsica

Herald Angel Award by the Glasgow Herald for the best piece of visual art at the 1997 Edinburgh Festival

International Critics Award as part of the 1997 Demarco European Art Foundation exhibition


Not really “Performing Arts” I hear you sayNot really “Performing Arts” I hear you say

In St. Ives staff gave children coloured “bats”

Switched off the sculpture

Children waved the bats around

And, of course, “made music”

Security was called once to “remove” a visitor

Would not leave at closing time

Transfixed by the whole thing

Some people came 2 or 3 times a week

For a “fix”

In St. Ives staff gave children coloured “bats”

Switched off the sculpture

Children waved the bats around

And, of course, “made music”

Security was called once to “remove” a visitor

Would not leave at closing time

Transfixed by the whole thing

Some people came 2 or 3 times a week

For a “fix”


How can this Technology be used …How can this Technology be used …

… for Performing Arts

Mount a camera above a stage

Give actors coloured hats

Now you have their position on stage

Link their movement around the stage

To …

Orchestrate lighting changes

HotspotsMove to one place on stage and make a sound

Detect “collisions”

Allow actors to fully interact with their environment

… for Performing Arts

Mount a camera above a stage

Give actors coloured hats

Now you have their position on stage

Link their movement around the stage

To …

Orchestrate lighting changes

HotspotsMove to one place on stage and make a sound

Detect “collisions”

Allow actors to fully interact with their environment


…or ……or …

Point the camera at a tropical fish tank

Let the fish make music

Point the camera at a tropical fish tank

Let the fish make music


Bringing e-Science into the FrameBringing e-Science into the Frame

So far York work done on a single modest computer

Scale upTo multiple specialist computers

Networks

Wireless devices

Complex tracking

3D sound

Interactive graphics

All controlled by …

The PERFORMER

So far York work done on a single modest computer

Scale upTo multiple specialist computers

Networks

Wireless devices

Complex tracking

3D sound

Interactive graphics

All controlled by …

The PERFORMER


Real Time Interactive Multiple Media - RIMMReal Time Interactive Multiple Media - RIMM

“Construction 3”

Realised by the Real-time Interactive Multiple Media Project

University of York Music Department (UK), IRCAM (France) and SIMPK (Germany)

European Commission Information Society Technologies Programme

Incorporates a live performer on soprano saxophone

Three-dimensional sculpture

Computer generated graphics

Ambisonic surround sound

“Construction 3”

Realised by the Real-time Interactive Multiple Media Project

University of York Music Department (UK), IRCAM (France) and SIMPK (Germany)

European Commission Information Society Technologies Programme

Incorporates a live performer on soprano saxophone

Three-dimensional sculpture

Computer generated graphics

Ambisonic surround sound


construction 3 is a synergetic compositionconstruction 3 is a synergetic composition

The musical and graphical elements of the piece have been composed concurrently

Both the computer graphics and music are triggered by the saxophonists natural gestures

Through infra-red video tracking

Use i/r device and i/r filter on camera

Electronic sensors

Footswitches, accelerometers

Machine listening

Score following

Gesture recognition using neural networks

e.g. “throw” sound into the audience

The musical and graphical elements of the piece have been composed concurrently

Both the computer graphics and music are triggered by the saxophonists natural gestures

Through infra-red video tracking

Use i/r device and i/r filter on camera

Electronic sensors

Footswitches, accelerometers

Machine listening

Score following

Gesture recognition using neural networks

e.g. “throw” sound into the audience


Block Diagram of SystemBlock Diagram of System


System DiagramSystem Diagram


The “Team”The “Team”

Lots of people needed to make this happenProject manager to keep us in lineThe Composer/Sound stage designerThe Graphic DesignerThe Graphics ProgrammerThe Audio ProgrammerThe Electronics technical wizardThe Local site organiserThe commercial suppliersAnd …

THE PERFORMER

Lots of people needed to make this happenProject manager to keep us in lineThe Composer/Sound stage designerThe Graphic DesignerThe Graphics ProgrammerThe Audio ProgrammerThe Electronics technical wizardThe Local site organiserThe commercial suppliersAnd …

THE PERFORMER


Team PhotoTeam Photo


Screens and PerformerScreens and Performer


Examples of the RIMM Computer GraphicsExamples of the RIMM Computer Graphics


Technology becomes e-ScienceTechnology becomes e-Science

Clearly a lot of technology to make it work

Some is bespoke electronics

Majority of tasks were programmed

The SGI Origin 20008 fast processors

Highly parallel

jMax

24-bit optical output for audio

High speed network interconnect to graphics computer

SGI O2 graphicsOptimised for OpenGL

Texture memory drawn from main memory

Clearly a lot of technology to make it work

Some is bespoke electronics

Majority of tasks were programmed

The SGI Origin 20008 fast processors

Highly parallel

jMax

24-bit optical output for audio

High speed network interconnect to graphics computer

SGI O2 graphicsOptimised for OpenGL

Texture memory drawn from main memory


Academic Background




Computers

Networks

Software


Future

Questions

Academic Background




Computers

Networks

Software


Future

Questions


Technologies for AllTechnologies for All

Many of these technologies are availableMany of the ideas are now embodied in softwareMany are free

Pure Data – aka PD (with GEM add-in)Eyesweb

VJ-ing is growingMany VJ products around

Grid Computing is availableNot easily accessibleLacks some functionality for Performing Arts

Photonic networks are availableHigh speed links point to pointUp to 10 gb/s possible now1 gb/s across the globe

Many of these technologies are availableMany of the ideas are now embodied in softwareMany are free

Pure Data – aka PD (with GEM add-in)Eyesweb

VJ-ing is growingMany VJ products around

Grid Computing is availableNot easily accessibleLacks some functionality for Performing Arts

Photonic networks are availableHigh speed links point to pointUp to 10 gb/s possible now1 gb/s across the globe


jMax, Pure Data/GEMjMax, Pure Data/GEM

Graphics Programming

Lots of modules

Connect up inputs and outputs

Uses the modular synthesiser metaphor

GEMGraphics Environment for Musicians

Link audio to graphics

Limited by modules unless you write your own

Graphics Programming

Lots of modules

Connect up inputs and outputs

Uses the modular synthesiser metaphor

GEMGraphics Environment for Musicians

Link audio to graphics

Limited by modules unless you write your own


eyeswebeyesweb

Video programming with modules

Huge number of modules

Includes “man” tracking capability

Can link to rudimentary audio

But …

Can also link to PDusing Open Sound Control protocols - OSC

Video programming with modules

Huge number of modules

Includes “man” tracking capability

Can link to rudimentary audio

But …

Can also link to PDusing Open Sound Control protocols - OSC


REACTABLEREACTABLE

Collaborative electronic music instrument

Tabletop tangible multi-touch interface

Several simultaneous performers

Share complete control over the instrumentby moving and rotating physical objects

on a luminous round table surface.

Objects represent components of a classic modular synthesizer

can create complex and dynamic sonic topologies

a kind of tangible modular synthesizer or

graspable flow-controlled programming language

Collaborative electronic music instrument

Tabletop tangible multi-touch interface

Several simultaneous performers

Share complete control over the instrumentby moving and rotating physical objects

on a luminous round table surface.

Objects represent components of a classic modular synthesizer

can create complex and dynamic sonic topologies

a kind of tangible modular synthesizer or

graspable flow-controlled programming language


REACTABLEREACTABLE


REACTABLE - demoREACTABLE - demo

Click on Image to go to site with Videos

[will open a browser]


More Video Tracking and PerformanceMore Video Tracking and Performance

Messa di Voce

Tmema,Blonk,La Barbara

http://www.tmema.org/messa/messa.html

Messa di Voce

Tmema,Blonk,La Barbara

http://www.tmema.org/messa/messa.html


Messa di Voce - demoMessa di Voce - demo

Click image to go to the Host site

[will try to play Quicktime in a browser]


And just for fun … another demoAnd just for fun … another demo


Academic Background




Computers

Networks

Software


Future

Questions

Academic Background




Computers

Networks

Software


Future

Questions


So what is this Grid Computing?So what is this Grid Computing?

The term originated in the early 1990’s

Metaphor for making computer power as easy to access as an electric power grid

Many years on and still mainly in the domain of “BIG SCIENCE”

Most Grids are run like 80’s batch style computingSubmit a job description to a scheduler

It decides when and where the job runs – more or less

This is a computational grid

Many computers and clusters are interconnected

Can be globally distributed via photonic networks

The term originated in the early 1990’s

Metaphor for making computer power as easy to access as an electric power grid

Many years on and still mainly in the domain of “BIG SCIENCE”

Most Grids are run like 80’s batch style computingSubmit a job description to a scheduler

It decides when and where the job runs – more or less

This is a computational grid

Many computers and clusters are interconnected

Can be globally distributed via photonic networks


What is this Data Grid?What is this Data Grid?

A grid computing system that deals with data

The controlled sharing and management of large amounts of distributed data

These are often, but not always, combined with computational grid computing systems

Finding data is not easy

Accessing it is not easy

Mainly in the domain of “BIG SCIENCE”

Some shift to Arts and HumanitiesWe have lots of data too!

More later in context of National Grid Service

A grid computing system that deals with data

The controlled sharing and management of large amounts of distributed data

These are often, but not always, combined with computational grid computing systems

Finding data is not easy

Accessing it is not easy

Mainly in the domain of “BIG SCIENCE”

Some shift to Arts and HumanitiesWe have lots of data too!

More later in context of National Grid Service


What are these Photonics NetworksWhat are these Photonics Networks

A worldwide network of optical fibres

Speeds are 1gbit/s to 40 gbit/s

Run by different organisationsUK - Janet

Europe - GEANT2

USA - StarLight

USA - National Lambda Rail

Canada – CANARIE

etc …

A connection is usually point-to-pointVery exclusive

Lots of bandwidth

Low latency and jitter (more of this later)

A worldwide network of optical fibres

Speeds are 1gbit/s to 40 gbit/s

Run by different organisationsUK - Janet

Europe - GEANT2

USA - StarLight

USA - National Lambda Rail

Canada – CANARIE

etc …

A connection is usually point-to-pointVery exclusive

Lots of bandwidth

Low latency and jitter (more of this later)


What might I use such links?What might I use such links?

Many examples

Interactive danceUse HDTV rather then Lo-res AccessGrid technology

Collaborative audio and videoMaster classes are more commonplaceCollaborative tools for distributed, remote teams

Digital cinema (4k x 2k) is used for:Entertainment, media, art and cultureScience, medicine, education and researchand also …Military, intelligence, security and police

Many examples

Interactive danceUse HDTV rather then Lo-res AccessGrid technology

Collaborative audio and videoMaster classes are more commonplaceCollaborative tools for distributed, remote teams

Digital cinema (4k x 2k) is used for:Entertainment, media, art and cultureScience, medicine, education and researchand also …Military, intelligence, security and police


Nagano Winter Olympics 1998Nagano Winter Olympics 1998

Massive endeavour with live networked music

Opening ceremony had a chorus in:Berlin

Cape Town

Beijing

New York

Sydney

Performed the final movement of Beethoven's 9th

Together with the full symphony orchestra in:Nagano Kenmin Cultural Centre Hall.

Also …

2000-person chorus in the Olympic stadium

Massive endeavour with live networked music

Opening ceremony had a chorus in:Berlin

Cape Town

Beijing

New York

Sydney

Performed the final movement of Beethoven's 9th

Together with the full symphony orchestra in:Nagano Kenmin Cultural Centre Hall.

Also …

2000-person chorus in the Olympic stadium


Nagano Winter OlympicsNagano Winter Olympics

The final result was mixed and broadcast worldwide

This setup required:7 satellites

12 uplinks with encoders and decoders

Multiple-delay compensation

The longest delay was to and from BerlinAround 4 seconds!

Incredibly …

It all worked and the result was orchestras and choirs all in time

The final result was mixed and broadcast worldwide

This setup required:7 satellites

12 uplinks with encoders and decoders

Multiple-delay compensation

The longest delay was to and from BerlinAround 4 seconds!

Incredibly …

It all worked and the result was orchestras and choirs all in time


Orchestra RehearsalsOrchestra Rehearsals

New World Symphony in Miami rehearsed pieces by French composers

The composers participated live from IRCAM in Paris

This transatlantic rehearsal enabled live exchanges between musicians in Miami and the composers in Paris

Via RENATER-4 in France, GEANT2 in Europe, and Internet2

These advanced research and education networks provided the bandwidth and quality of service to ensure high quality audio and video

New World Symphony in Miami rehearsed pieces by French composers

The composers participated live from IRCAM in Paris

This transatlantic rehearsal enabled live exchanges between musicians in Miami and the composers in Paris

Via RENATER-4 in France, GEANT2 in Europe, and Internet2

These advanced research and education networks provided the bandwidth and quality of service to ensure high quality audio and video


Using High Speed Networks for ArtsUsing High Speed Networks for Arts

Cinegrid is a good Example

Cinegrid’s Mission Statement:

To build an interdisciplinary community that is focused on the research, development, and demonstration of networked collaborative tools to enable the production, use and exchange of very-high-quality digital media over photonic networks

See www.cinegrid.org for more details

Cinegrid is a good Example

Cinegrid’s Mission Statement:

To build an interdisciplinary community that is focused on the research, development, and demonstration of networked collaborative tools to enable the production, use and exchange of very-high-quality digital media over photonic networks

See www.cinegrid.org for more details


CinegridCinegrid

Converging on the digitalFast networking

Access to shared instruments, computers, storage

Collaboration tools

Robust security for intellectual property

High visual quality

Greater speed

More distributed applications

and, importantly …

Next generation of trained professionals

Converging on the digitalFast networking

Access to shared instruments, computers, storage

Collaboration tools

Robust security for intellectual property

High visual quality

Greater speed

More distributed applications

and, importantly …

Next generation of trained professionals


Cinegrid@AES October 25th 2006Cinegrid@AES October 25th 2006

This event was structured in four acts

Each demonstrating a different facet of the CineGrid philosophy of networked extreme media

In Act 1, a sequence of 4K “digital shorts” at 24 frames per second (fps), together with fully mixed synchronized audio, were pulled in real time from network-connected servers in Los Angeles and San Diego.

This event was structured in four acts

Each demonstrating a different facet of the CineGrid philosophy of networked extreme media

In Act 1, a sequence of 4K “digital shorts” at 24 frames per second (fps), together with fully mixed synchronized audio, were pulled in real time from network-connected servers in Los Angeles and San Diego.



In Act 2, 4K tele-presence was used for interactive video-conferencing and ultra-realistic reproduction of a classical music performance from Tokyo

In Act 2, 4K tele-presence was used for interactive video-conferencing and ultra-realistic reproduction of a classical music performance from Tokyo



Acts 3 and 4 were designed to prove the concept:networked, remote audio post-production for digital cinema

by creative teams spread around the world

who demand the highest-quality production values

In Act 34K motion pictures were sent compressed from Tokyo

24-channel non-compressed digital audio was streamed from San Diego

In Act 4The performance system was re-configured

to use uncompressed 2K motion pictures coming from ILM servers in the LDAC facility, synchronized to 24-channel, non-compressed digital audio streaming from San Diego

Acts 3 and 4 were designed to prove the concept:networked, remote audio post-production for digital cinema

by creative teams spread around the world

who demand the highest-quality production values

In Act 34K motion pictures were sent compressed from Tokyo

24-channel non-compressed digital audio was streamed from San Diego

In Act 4The performance system was re-configured

to use uncompressed 2K motion pictures coming from ILM servers in the LDAC facility, synchronized to 24-channel, non-compressed digital audio streaming from San Diego



Was a good test of the CineGrid concept of:

Using 4K cameras

Multi-channel, non-compressed audio

Ultra-realistic “live” experiences of music concerts

and …

other kinds of performing arts

To distant audiences in theatres

Connected by high-speed networks

Was a good test of the CineGrid concept of:

Using 4K cameras

Multi-channel, non-compressed audio

Ultra-realistic “live” experiences of music concerts

and …

other kinds of performing arts

To distant audiences in theatres

Connected by high-speed networks



"I believe the very high-speed optical networks such as now being deployed by research organizations will become an essential infrastructure for digital cinema production and distribution”

"But, we still have to learn how to integrate systems that creative people can use to make beautiful 4K content - picture and sound - in new ways appropriate for the 21st century. Demonstrations such as CineGrid@AES force us, in a good way, to learn by doing."

"I believe the very high-speed optical networks such as now being deployed by research organizations will become an essential infrastructure for digital cinema production and distribution”

"But, we still have to learn how to integrate systems that creative people can use to make beautiful 4K content - picture and sound - in new ways appropriate for the 21st century. Demonstrations such as CineGrid@AES force us, in a good way, to learn by doing."

Tomonori Aoyama, Professor at Keio/DMC and Chairman of the Digital Cinema Consortium of Japan (DCCJ) said:Tomonori Aoyama, Professor at Keio/DMC and Chairman of the Digital Cinema Consortium of Japan (DCCJ) said:


UK National Academic NetworksUK National Academic Networks

UKERNAUK Education and Research Networking Association

Now renamed to JANET

Operate the standard IP serviceWhat most of use use daily

With the rollout of SUPERJANET-5 we now have:

JANET Lightpath Service

Evolved from UKLightSee www.uklight.ac.uk for details

Various research projects

Many coordinated from NeSC and the eSI

UKERNAUK Education and Research Networking Association

Now renamed to JANET

Operate the standard IP serviceWhat most of use use daily

With the rollout of SUPERJANET-5 we now have:

JANET Lightpath Service

Evolved from UKLightSee www.uklight.ac.uk for details

Various research projects

Many coordinated from NeSC and the eSI


What is a lightpath?What is a lightpath?

End-to-end Network CapacityPoint-to-point circuits

Capacities from about 50Mbit/s to 10Gbit/s

Presentations

• Ethernet – 100/1G/10G (WAN/LAN)• SDH – STM-1/4/16/64

Implemented over JANET optical transmission infrastructure

including SDH equipment re-deployed from UKLight

End-to-end Network CapacityPoint-to-point circuits

Capacities from about 50Mbit/s to 10Gbit/s

Presentations

• Ethernet – 100/1G/10G (WAN/LAN)• SDH – STM-1/4/16/64

Implemented over JANET optical transmission infrastructure

including SDH equipment re-deployed from UKLight


Janet NetworksJanet Networks

JANET Optical Transmisson

JANET IP Research capacityUKLight

AdditionalWavelengths


SUPERJANET-5 Integrated with UKLIGHTSUPERJANET-5 Integrated with UKLIGHT

EuropeGEANTUSA

StarLight

YHMAN

C&NLMAN

Clydenet

EaStMAN

NorMAN

NNW

TVN LMN

EMMAN

Bristol

Glasgow

Reading

Warrington

Leeds

LondonT-House

T-City

NIRAN

Dublin (HEANET)

SWERN

LeNSE

Kentish MAN

WREN

EastNet

AbMAN

FaTMANUHI

MidMAN


Getting a Lightpath - JANET PolicyGetting a Lightpath - JANET Policy

Subject to capacity and budget availability, lightpaths will be free at point of use up to 1Gbit/s

Tariffs available for 10Gbit/s

Delivered to Institutions at the JANET Access Point

Institutions must commit to extending the lightpath across their networks

See www.ja.net/lightpath for details

Documents shortlyFirst public version in final editorial stage

Service description and application lightpath application form

Subject to capacity and budget availability, lightpaths will be free at point of use up to 1Gbit/s

Tariffs available for 10Gbit/s

Delivered to Institutions at the JANET Access Point

Institutions must commit to extending the lightpath across their networks

See www.ja.net/lightpath for details

Documents shortlyFirst public version in final editorial stage

Service description and application lightpath application form


High Performance ComputingHigh Performance Computing

The more power we need the less we can afford it

Not sensible to have massive systems duplicated

Look towards the grid for power

But, what if we need it in real time?

Current technology geared towards batch computing

Some notable exceptionsInteractive visualisation

Computational steering

VLBI

Some CERN LHA experiments

Data reduction in real time

The more power we need the less we can afford it

Not sensible to have massive systems duplicated

Look towards the grid for power

But, what if we need it in real time?

Current technology geared towards batch computing

Some notable exceptionsInteractive visualisation

Computational steering

VLBI

Some CERN LHA experiments

Data reduction in real time


Grid Computing in the UKGrid Computing in the UK

National Grid Service (NGS)

NGS-1 4 core sites with compute and TB storage

NGS-2 Officially in pre-production as of 20th June

Sevices …

Job Submission via:Globus, GridSAM, Condor, gLite

or via portals and applications repositories“to simplify job submission for users who do not wish to delve into the technical details of grid computing “

Grid Support Centre to help users

National Grid Service (NGS)

NGS-1 4 core sites with compute and TB storage

NGS-2 Officially in pre-production as of 20th June

Sevices …

Job Submission via:Globus, GridSAM, Condor, gLite

or via portals and applications repositories“to simplify job submission for users who do not wish to delve into the technical details of grid computing “

Grid Support Centre to help users


Data Grid in the UKData Grid in the UK

NGS offers:Oracle 10g

Storage Resource Broker (SRB)

OGSA-DAI

VOMS

Oracle 10g – high performance database system

SRB - data storage servicestore your data in geographically distributed locations

access them all using only the logical filename

Reliabilty and efficiency are improved dramatically

OGSA-DAIThe Open Grid Services Architecture Data Access and Integration

middleware that helps users access and integrate data

from separate structured data sources e.g. XML or Oracle

NGS offers:Oracle 10g

Storage Resource Broker (SRB)

OGSA-DAI

VOMS

Oracle 10g – high performance database system

SRB - data storage servicestore your data in geographically distributed locations

access them all using only the logical filename

Reliabilty and efficiency are improved dramatically

OGSA-DAIThe Open Grid Services Architecture Data Access and Integration

middleware that helps users access and integrate data

from separate structured data sources e.g. XML or Oracle


VOMSVOMS

Virtual Organisation Membership Service

To help grids manage the authorization of their users

Delegates the approval of users to the to the VO itself

Thus removing the onus upon the end user to register with each resource they might use as part of the VO

NGS resources are in the process of adopting VOMS to authorize NGS members

In the future the number of VOMS-aware resources is likely to increase

Virtual Organisation Membership Service

To help grids manage the authorization of their users

Delegates the approval of users to the to the VO itself

Thus removing the onus upon the end user to register with each resource they might use as part of the VO

NGS resources are in the process of adopting VOMS to authorize NGS members

In the future the number of VOMS-aware resources is likely to increase


Academic Background




Computers

Networks

Software


Future

Questions

Academic Background




Computers

Networks

Software


Future

Questions


High Performance ComputingHigh Performance Computing

Wavefield Synthesis is a good example

Computationally demanding

Desktop machines and small clusters not capable

Can we have real time grid computing please?

If not, will someone fund a big lorry:With lots of computers

Lots of aircon

Plug in 3-phase power at one end

Gigabit ethernet comes out the other end!

Wavefield Synthesis is a good example

Computationally demanding

Desktop machines and small clusters not capable

Can we have real time grid computing please?

If not, will someone fund a big lorry:With lots of computers

Lots of aircon

Plug in 3-phase power at one end

Gigabit ethernet comes out the other end!


Wavefield Synthesis – what, why, when?Wavefield Synthesis – what, why, when?

The limitation of stereo or ambisonic techniquesit only works perfectly well for one listener

positioned on the so-called sweet spot

Thus in common concert environmentsthe intended effect of movement of the sound will in these cases not be heard by a majority of the listeners

Wave field synthesis can overcome this limitation

Can provide a good perceptual localisation in a relatively large listening area

This makes the technique ideal for concert environments

The limitation of stereo or ambisonic techniquesit only works perfectly well for one listener

positioned on the so-called sweet spot

Thus in common concert environmentsthe intended effect of movement of the sound will in these cases not be heard by a majority of the listeners

Wave field synthesis can overcome this limitation

Can provide a good perceptual localisation in a relatively large listening area

This makes the technique ideal for concert environments


Wavefield Synthesis – Simplistic overviewWavefield Synthesis – Simplistic overview

Sound is wavesNext wave can be synthesised by a infinite number of small speakersNeed to approximate “infinite”

Sound is wavesNext wave can be synthesised by a infinite number of small speakersNeed to approximate “infinite”


Wavefield Synthesis – the realityWavefield Synthesis – the reality

Various experimental speaker arraysVarious experimental speaker arrays


Exemplar VenueExemplar Venue

RECOMBINANT MEDIA LABS, SF

Founded to create, research, and portray Spatial Media Synthesis

The science of projecting image and sound objects in 3-dimensional space

RML cultivates radical methodologies for experiential engineering

Exploring performative processes which expand the formal aesthetic and technological boundaries of immersive installation and surround cinema

RECOMBINANT MEDIA LABS, SF

Founded to create, research, and portray Spatial Media Synthesis

The science of projecting image and sound objects in 3-dimensional space

RML cultivates radical methodologies for experiential engineering

Exploring performative processes which expand the formal aesthetic and technological boundaries of immersive installation and surround cinema


RMLRML

The San Francisco facility contains:

A flexible black box environment that houses:

A high definition multi-channel audio-visual systemknown as Surround Traffic Control

Full fidelity array consists of a design specification for:10 screens in 360 degrees

supported by an ultra impact 16.8.2 horizontal and vertical sound diffusion system

scalable for all types of rectangular rooms

High Speed Networking

The San Francisco facility contains:

A flexible black box environment that houses:

A high definition multi-channel audio-visual systemknown as Surround Traffic Control

Full fidelity array consists of a design specification for:10 screens in 360 degrees

supported by an ultra impact 16.8.2 horizontal and vertical sound diffusion system

scalable for all types of rectangular rooms

High Speed Networking


SummarySummary

Seen the evolution of some technology

The newer technologies require:High performance computing

Very high speed networks

Massive storage

New breeds of creative artists

Many different experts in the technologies

Networking of experts

New theatres

Investment

and much much more

Seen the evolution of some technology

The newer technologies require:High performance computing

Very high speed networks

Massive storage

New breeds of creative artists

Many different experts in the technologies

Networking of experts

New theatres

Investment

and much much more


ANY QUESTIONS?

ANY QUESTIONS?

e-science collides with the performing arts

Documents

digital cinema

key areas of science

collaborative research

music technologycourses

technologyright people

technology input

artsmy involvement

statistical systems