cs6022 principles of interactive media - may 4th

CS6022 Principles of Interactive Media - May 4th

Virtual Marimba

Creating a novel musical instrument that produces visual representations of sounds.

An Interactive Media Thesis by Chris Hackett & David Delaney

• Business Studies Graduate• Multi-Instrumental Musician• Collaborative Musician

(Palestinian/South African,etc)• Sound Effects/Soundtracking• Focus: Interactive Media in

Tradtional Musical Instruments.

Sound Synthesis & Manipulation

profiles

David Delaney


profiles

• Computer Systems Graduate• Specialism: Web/Graphic Design• Works with: Dreamweaver,

Photoshop, Flash, Sketch-Up• Interests: Graphics,

Animation,Music, Audio-visual Art, Film, Photography

• Focus: Graphics, Visual Representations of Sound.

Chris Hackett


• Interest in Music• Similar Thesis Proposals• Existing artefacts e.g. Marimba Lumina• Ailbhe’s ‘Rhythm Manifesto’ • Composer/Designers e.g. Todd Winkler• Audio - Visual Art Culture

inspiration


design process

• User-Centred Design• Research• Requirements Gathering• Design• Prototyping• Evaluation


visual music • ‘A means of expressing music in visual form requiring the

active involvement of an artist, designer, or director to interpret the music and find the means to express it visually.’ (Friedlander, ‘98)

• ‘A visualization of music which is the translation of a specific musical composition(or sound) into a visual language, with the original syntax being emulated in the new visual rendition.’ (Ox, J and Keefer, C, ‘06)

• Origins: Pythagoras, Aristotle, Sir Isaac NewtonCorrespondences of the colour spectrum and sound waves, music and colour, sound and light.

• 18th Century - Castel’s Clavecin Oculaire• Late 19th/Early 20th - Experimentation, abstract film• 20th Century - Art e.g. Scott Drave’s Electric Sheep• Present - Artists, composers, designer, virtual djs


• VJ: Audio-visual Art and VJ Culture (Faulkner, M)• On Curating Recent Digital Abstract Visual Music (Ox, J

& Keefer C) • Amplifying Reflective Thinking in Musical Performance

(Johnson, A et al)• Visual Music-Searching for an Aesthetic (DeWitt, T)• Exploration of the correspondence between visual and

acoustic parameter spaces (Gerhard et al)• Issues for Designing a flexible expressive audiovisual

system for real-time performance and composition (Franco et al)

• Rhythm Manifesto – Ailbhe Keogan

gbackground reading


• malletKat• Marimba Lumina• ‘Music Plays Images X Images Play Music’ MPI X IPM

Ryuichi Sakamoto • ‘Falling Up’ - Todd Winkler• ‘Rhythm Manifesto’ - Ailbhe Keogan

benchmarks


• NIME www.nime.org• The MEGA Project www.megaproject.org• InfoMus Lab infomus.dist.unige.it• SARC www.sarc.qub.ac.uk• The s2S2 www.s2s2.org• ConGas www.cost287.org• www.centerforvisualmusic.org• www.howstuffworks.com• www.acm.org

websites


empirical work undertaken

• Benchmark Research e.g. malletKat• Informal discussions with classmates• Interview with Ailbhe Keogan, 27.02• Interview with David Merrill, MIT, 16.03• Early sketches, prototyping, scenarios


further empirical work

• Interviews• Cormac Byrne, Marimba player• Evelyn Glennie, Killarney 12.07• Micheal O Suilleabhain, Musician• Music Academics, e.g. Jurgen Simpson• Further sketches, prototyping, scenarios

artefact

layout


projection screens


visuals

• Mapping Techniques• Colour - 7 colours to 7 units of the scale• Vibrato – sine wave• Interrupted Cadence – expectation• Relative Minor – sinister, dark, gothic• Timbre of Sound• Mikael’s Manifesto• PD & GEM• 3Ds Max, Photoshop, Flash


Sound Parameters can be manipulatedDevice Sensors PC Hardware (USB, Firewire or Wireless)

OS (Windows)

BIOS

DeviceDrivers

Max/MSP, Pure Data, VVVV, Director

Electronic Marimba App

Sound (Speakers) Images (Projector)

Camera

architecture


hardware required

• Arduino board – prototyping• Piezo sensors• Pico device• Projector – possibly 3• Webcamera• Computer (PC/Mac)• Under discussion – whether to build device

from scratch or use existing commercial product


software required

• Pure Data and GEM - probably• MAX/MSP - not so likely• VVVV - not so likely• Director & Flash (Lo-Fi Prototyping)• Photoshop (Graphics)• 3D studio Max (Modelling, Graphics)• Google Sketch-Up (sketches)• Logic Pro (Synthetic Sounds for lo-fi’s)


extra scope for instrument

• Multi Instrumental• Display Dynamic Visuals• Pitch bend, vibrato• Velocity Sensitivity• Pressure Sensitivity (Aftertouch)• Recording and Playback of loops.


schedule


May June July August September

Brainstorming

Presentation & Report

Developing Concept

Users & Requirements

Research Theory

Research Technology

Designing

Prototyping

Building

Evaluation

Documentation

Thesis Write up

Assessment & Dawn

documenting the process

http://0030511.blogspot.com


concerns

• Finalising concept• Building device Vs Existing product• Real-time Vs Pre-Rendered• Latency Problems• Pressure/Velocity Sensitivity• Number of Projectors Needed• Software Skills• Cost• Time


issues for discussion

• Suitable & novel interface?• Existing products• Ideas for graphics• Thesis Title• Anyone know PD, GEM?• Have a good weekend!


Q&A


cs6022 principles of interactive media - may 4th

Documents

interactive media thesis

thvisual music

visualization of music

visual language

visual form

et alvisual music

new visual rendition

sound waves