video game music overview

82
Game audio Karen Collins

Upload: karen-collins

Post on 17-Jan-2015

4.183 views

Category:

Entertainment & Humor


3 download

DESCRIPTION

Video Game Music overview for Universite de Montreal class on music and the moving image.

TRANSCRIPT

Page 1: Video Game Music Overview

Game audio

Karen Collins

Page 2: Video Game Music Overview

OUTLINE

– Defining Games Audio– History– Functions of Games Audio – Approaches to composition– diegesis and dynamic audio

Page 3: Video Game Music Overview

Who cares about games?

• Majority of people in western world play games (75% of heads of households (in the United States) play computer or video games)

• average age for a game player is thirty• nearly half of game players are female (55%

male, 43% female). • “Women over the age of 18 represent a greater

portion of the gameplaying population (28%) than boys from ages 6 to 17 (21%).

Page 4: Video Game Music Overview

Who cares about game audio?

• 83% of Adult Gamers listed sound as one of the most important video game console elements

• 47% of Game Console owners (18-25 yr) hook up their game console to a home theatre system

• 48% of Hardcore Gamers said surround sound is a purchase driver for next generation consoles

Page 5: Video Game Music Overview

What is game audio?

1. Dialogue/speech sounds

2. Sound effects

3. Music/score

4. Ambience

5. Interface sounds (menu, etc.)

Page 6: Video Game Music Overview

What’s wrong?

Page 7: Video Game Music Overview

How games are not movies:The problems of games audio

• Linearity Vs. Non-Linearity/ unpredictability, mixing

• Interactivity with player

• Temporality (length)

• Budgets

• Delivery methods/technology

Page 8: Video Game Music Overview

What is Interactive Audio?

• sound events occur in direct reaction to a player’s movements. The player triggers the cue, and can repeatedly activate it, such as by making a character jump up and down.

Page 9: Video Game Music Overview

Interactive Audio:

FootstepsEnemy music cue

Page 10: Video Game Music Overview

What is adaptive audio?

• “adaptive” audio is generally referred to as sound that reacts to transformations in the gameplay environment—such as a change from day to night set by a game’s timer mechanism. Adaptive audio is not directly triggered by a player

Page 11: Video Game Music Overview

Adaptive Audio

Change in music day to nightWolf, crow

Page 12: Video Game Music Overview

What is Dynamic Audio

• Dynamic audio is a broad term to refer to both interactive and adaptive audio

Page 13: Video Game Music Overview

A brief history of game audio

(in 4 minutes!)

Page 14: Video Game Music Overview

History

Page 15: Video Game Music Overview

Functions of Games Sound

1. Commercial functions2. Structural reinforcement3. Immersion/suspension of disbelief4. Semiotics functions5. Narrative functions6. Emotional functions7. Aesthetic functions8. Kinetic functions

Page 16: Video Game Music Overview

1. Commercial functions

– Sell games– Sell artists– Sell soundtracks– Sell movies– Sell other games– Branding: familiarization

Page 17: Video Game Music Overview

Sales, marketing and branding

Page 18: Video Game Music Overview

2. Structural reinforcement

• Mask structure and editing

• Enhance/reinforce structure of game

• Create structure

• Situate player in game matrix

• Indicate change in narrative

• Enhance continuity

Page 19: Video Game Music Overview

Structural reinforcement

Page 20: Video Game Music Overview

3. Immersion/suspension of disbelief

• Add realism: – Emphasize real sounds– Represent location/spatialization

• Create illusion

• Cover external noise

Page 21: Video Game Music Overview

Immersion/suspension of disbelief

Page 22: Video Game Music Overview

4. Semiotic functions

• Sonic symbols, leitmotifs: identify objects and decrease learning curve

• Preparatory function

• Acousmatic sound: Focus attention, identify goals

• Convey meaning, emotion

• Comment

• Add slant or bias

Page 23: Video Game Music Overview

Semiotic Preparatory Function

Page 24: Video Game Music Overview

A test of the semiotics of space/place in Zelda

1. Ranch (farm)2. Windmill3. Fire temple4. Evil Castle underground5. Valley of gypsy thieves6. Flying fairy7. Dawn8. Temple of time9. Ice cavern

ABCDEFGHI

Page 25: Video Game Music Overview

5. Narrative functions

• Advancement of plots• Locate player in storyline• Anchor player in terms of game matrix/level,

etc. • anticipate/foreshadow• Dialogue can be a major event in plot• Access character’s thoughts/reveal character

details• Reveal goals

Page 26: Video Game Music Overview

Narrative: correct response

Page 27: Video Game Music Overview

6. Emotional functions

• Emotional and physiological: manipulate player

• Disturb or enhance gameplay: battle scenes

• Mood induction

• Create intimacy (immersive)

Page 28: Video Game Music Overview

Emotional

Page 29: Video Game Music Overview

7. Aesthetic functions

• Improve our appreciation of the game

• Make the game better/more enjoyable

• Make it “cool”

• Indicate genre/style

• Inter-textual referencing

Page 30: Video Game Music Overview

8. Kinetic functions

• Aerobic exercise: fitness

• Music as motivating factor for movement

• Edutainment for toddlers, etc.

Page 31: Video Game Music Overview

Kinetic Functions

• Dance Dance Revolution, etc.

Page 32: Video Game Music Overview

Problems (and some solutions) of Interactive Audio in Games

Page 33: Video Game Music Overview

Mixing: Problems:-too many sounds

-no “post production” mixing

-unpredictable timings: results in “muddy”,

clash of sounds

Page 34: Video Game Music Overview

Mixing Solutions!

• Reduce amount of sounds

• Control frequencies of sounds

• Assign priorities in code– E.g. dialogue = priority “1”

• Ambience = priority “4”• Gunshot = priority “2”• Music = priority “3”• Footsteps = priority “5”

Page 35: Video Game Music Overview

Problem: technological constraints

• Trying to write songs in a few kB for constrained technology such as mobile phones

Page 36: Video Game Music Overview

Solutions: technological constraints:

• Playing a single sample at different pitches to produce variation.

• The "pitch it up, play it down" technique for saving space.

Page 37: Video Game Music Overview

Sampling to save size

• This game contains only a single "trumpet fall" sound effect

• modify the playback sample rate so that each time you pick up a piece of cheese, the sound is played at a different pitch

• randomly vary the pitch during gameplay, so there's no repeating pattern to it

• The sound tends to mask the transition from one mix to another, helping to create a more seamless audio experience.

• same kind of pitch-shifting effect applied to car horn beeps: (fig 1)

Page 38: Video Game Music Overview

Sampling to save space• Another way to save space is using the "pitch it up, play

it down" technique. • take your original, high-resolution sound effect and

transpose it up an octave, halving the length. • Then convert it to a low-resolution compressed format,

like this: (cat growl, octave up, car start, octave up. Then in the game, play it down an octave: (octave down).

• Although the game sound might be a little crunchy, you've just cut the size of your file in half without losing too much audio fidelity. Obviously, the higher you pitch the sound, the "crunchier" the playback will become, but the technique can be used for custom instruments as well as sound effects, and is particularly effective on the tiny speakers in mobile devices.

Page 39: Video Game Music Overview

• Problem: transitions from cue to cue– No “post” =

unpredictable timings– Abrupt jumps are

jarring

Page 40: Video Game Music Overview

(1) Solution: transitions: Cross fade

Page 41: Video Game Music Overview

• Problem: Multi-player interactivity

• Unscripted events!

• parameter-based music is difficult:– Music cannot be tied to specific events or

locations– Music cannot be tied to specific

parameters

Page 42: Video Game Music Overview

Solutions: multi-player interactivity

• Work with multi-players!

• Asheron’s Call II, Lord of the Rings: players can bring instruments and jam out with each other

Page 43: Video Game Music Overview

Multi-player music

Page 44: Video Game Music Overview

Problem: listener fatigue

• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.• Loops are boring.

Page 45: Video Game Music Overview

Solutions: Listener fatigue

• Learn how to use silence!

• “boredom switch”: timer for set levels to stop

• VARIABILITY!

Page 46: Video Game Music Overview

Dynamic Composition: Variability

1. variable tempo2. variable pitch3. variable volume4. variable DSP5. variable melodies (algorithmic generation)6. Variable harmony (chord arrangements)7. Variable harmony (keys)8. variable mixing 9. Variable sequencing10. variable structure

Page 47: Video Game Music Overview

Variable tempo

Page 48: Video Game Music Overview

Variable tempo

Page 49: Video Game Music Overview

Variable pitch

Page 50: Video Game Music Overview

Variable Pitch

Page 51: Video Game Music Overview

Variable volume

Page 52: Video Game Music Overview

Variable DSP

• Change DSP effects to change music:– E.g. reverb to make it softer, dreamier– Phasing to create a “dazed” effect– Overdrive to make it more aggressive, etc.

Page 53: Video Game Music Overview

Variable melodies

• Ballblazer (1984): the music plays forever, without repeating itself but without straying too far from the original theme. The bassline doesn't vary at all, only the lead line in the higher register is algorithmically varied.

Page 54: Video Game Music Overview

Variable harmony

• Asheron’s Call 2: changes in mode indicate changes in gameplay:– A player standing alone in an area would

hear the background score in Lydian, but as creatures fill the area, the music transitioned to Phrygian. If monsters outnumber player, music transition to diminished mode and quickened drum groove

Page 55: Video Game Music Overview

Variable mixing

• Adjust mix by– Bringing instruments up in mix that were

quiet or silent before– Drop elements from mix

Page 56: Video Game Music Overview

Cheese Racer

• Muting and unmuting multiple tracks in a MIDI file to produce various mixes.– 2 drum loop samples,– three tones: bass, melody and chords

• In the code for the first level, ten different combinations of tracks: percussion + bass, percussion + bass + melody, bass + melody + chords, etc.

Page 57: Video Game Music Overview

(cont..)

Notice how the mix changes every time the mouse gets a piece of cheese. (Fig 2 : exit) (Fig 3)

• Music loop 40 seconds. 10 different combinations: therefore, each level has more than 6 minutes of different music mixes. “Because the mix changes depending on gameplay, the music will never play exactly the same way twice, thereby increasing variation and decreasing "ear fatigue."”

• In fact, there are two more sets of tracks playing two more styles of music, using the same tempo and percussion tracks as the first level.

• Entire game contains almost 20 minutes of various music mixes, using only 68K of compressed sample and MIDI data.

Page 58: Video Game Music Overview

Dynamic Music- example

• Layer of instrumentation added when Mario rides Yoshi in Super Mario World.

Page 59: Video Game Music Overview

Russian Squares• Using layers, each cue

increment represents a puzzle row being cleared by the player.

Page 60: Video Game Music Overview

Variable sequencing• Alter

sequences of music

• Zelda Ocarina: Hyrule field

Page 61: Video Game Music Overview

Variable structure

• Create “transition matrix”: chart out all possible directions for each sequence, and create transitions for marker points/jumps

Page 62: Video Game Music Overview

Transition Matrix: No One Lives Forever

• Music and Arranging by Guy Whitmore and Tobin Buttram

• This music clip moves through all six of the music states for the Ambush theme, demonstrating transitions called from a transition matrix. Time      Music State0:00      6 (combat 2)1:22      transition 1:30      2 (sub-ambient)2:14      transition2:16      4 (suspense)3:10      transition3:12      3 (ambient)4:08      transition4:12      5 (combat 1)5:33      transition     1 (silence)

Page 63: Video Game Music Overview

No One Lives Forever

• EarthOrbit:

• The Ambush theme starts in music state 5 (combat 1), transitions to music state 2 (ambient), then transitions to music state 6 (combat 2) with motifs.

Page 64: Video Game Music Overview

What else can be done?

• Dynamic music offers all kinds of interesting possibilities to have music as a driving force in a game!

Page 65: Video Game Music Overview

Super Mario DS

• New Super Mario Bros DS– Enemies jump and fly

in time to music

Page 66: Video Game Music Overview

Electroplankton

Page 67: Video Game Music Overview

Rez

Page 68: Video Game Music Overview

Vib Ribbon

Page 69: Video Game Music Overview

Levels of dynamic sound

1. non-dynamic, non-diegetic sound2. Adaptive non-diegetic sound3. Interactive non-diegetic sound4. Non-dynamic diegetic sound5. Adaptive diegetic sound6. Interactive diegetic sound7. Kinetic diegetic and non-diegetic

(reminder: Non-Diegetic: Outside the character’s space. Diegetic: “real sounds”, inside character’s space)

Page 70: Video Game Music Overview

Non-dynamic, non-diegetic sound

• non-dynamic, linear music or sound found most frequently in the cinematic sequences (movie sequences in which the player’s functions are stopped while a short clip to advance the plot is played). In other words, the player has no control over the possibility of interrupting the music (short of resetting/turning the game off).

Page 71: Video Game Music Overview

Non-dynamic, non-diegetic sound

Page 72: Video Game Music Overview

Adaptive non-diegetic sound

• Sound events occurring in reaction to gameplay, but which are unaffected by the player’s direct movements

• The player cannot re-trigger these events.

Page 73: Video Game Music Overview

Adaptive non-diegetic sound

Page 74: Video Game Music Overview

Interactive non-diegetic sound

• Sound events occurring in the character’s space

• The player can re-trigger these events.

Page 75: Video Game Music Overview

Interactive non-diegetic sound

Page 76: Video Game Music Overview

Non-dynamic diegetic sound

• In non-dynamic diegetic audio, the sound event occurs in the character’s space, but the character does not directly interact with it.

Page 77: Video Game Music Overview

Non-dynamic diegetic sound

Page 78: Video Game Music Overview

Adaptive diegetic sound

• Diegetic sounds adapt to gameplay environment (e.g. night to day)

Page 79: Video Game Music Overview

Adaptive Diegetic Sounds

Page 80: Video Game Music Overview

Interactive diegetic sounds

• occur in the character’s space and with which the player’s character can interact. The player instigates the audio cue, but does not necessarily affect the sound of the event once the cue is triggered.

Page 81: Video Game Music Overview

Interactive Diegetic Sounds

Page 82: Video Game Music Overview

Kinetic interactive Sound

• Player interacts with the sound by mimicking the sound/image kinetically– Guitar Hero– Donkey Konga