School of Computer Science & Information Technology

School of Computer Science & Information Technology

G6DPMM - Lecture 11G6DPMM - Lecture 11

Media Design II – SoundMedia Design II – Sound

Sound & MultimediaSound & Multimedia

Do not underestimate its importanceDo not underestimate its importance Consider the five sensesConsider the five senses

Smell & TasteSmell & Taste Never implemented, except in researchNever implemented, except in research

TouchTouch Rarely implemented in mainstream multimedia – starting to Rarely implemented in mainstream multimedia – starting to

become used in games and for specialised applicationsbecome used in games and for specialised applications

Sight & HearingSight & Hearing The basis of most multimediaThe basis of most multimedia

Sound is both underused and badly usedSound is both underused and badly used

Sound vs Still ImagesSound vs Still Images

Sound is active, images are passiveSound is active, images are passive Users are not usually in immediate control of soundUsers are not usually in immediate control of sound Sound interrupts other activitiesSound interrupts other activities

Sound is time-basedSound is time-based May be finite or loopingMay be finite or looping Images remain until erasedImages remain until erased

Sound is non-localisedSound is non-localised Hard to annotateHard to annotate Stereo effects can give only a rough idea of locationStereo effects can give only a rough idea of location

Notation is difficultNotation is difficult

Purposes of SoundPurposes of Sound Most ideas developed for early radioMost ideas developed for early radio Background soundBackground sound

““Noise” - usually ignored, but can create a strong Noise” - usually ignored, but can create a strong impression of an environmentimpression of an environment

““Atmos” – term from radio f/xAtmos” – term from radio f/x Pleasant soundsPleasant sounds

Communicate emotions rather than meaningCommunicate emotions rather than meaning Natural (e.g. birdsong, wind, sea, rain, waterfall)Natural (e.g. birdsong, wind, sea, rain, waterfall) Artificial (e.g. music)Artificial (e.g. music)

Unpleasant soundsUnpleasant sounds Harsh, grating soundsHarsh, grating sounds Effects are often personal, and hard to reproduceEffects are often personal, and hard to reproduce

Purposes of Sound (cont.)Purposes of Sound (cont.) Emotive soundsEmotive sounds

Human voices (e.g. anger, pain, child crying)Human voices (e.g. anger, pain, child crying) Danger (e.g. animal growling, brakes squealing)Danger (e.g. animal growling, brakes squealing)

SpeechSpeech Meaning as in textMeaning as in text More subtleties than text (tone of voice, accent etc.)More subtleties than text (tone of voice, accent etc.) Subtext can be conveyed (e.g. emotions, state of Subtext can be conveyed (e.g. emotions, state of

health etc.)health etc.)

SilenceSilence The absence of sound – can be very powerfulThe absence of sound – can be very powerful

Designing with SoundDesigning with Sound Sound may have meaningSound may have meaning

Multimedia designers can develop complex vocabularies Multimedia designers can develop complex vocabularies of soundsof sounds

Sounds may be coupled with various actions and fulfil Sounds may be coupled with various actions and fulfil various purposes in a multimedia system.various purposes in a multimedia system.

Changing statesChanging states Provision of user-feedback about a stateProvision of user-feedback about a state Not generally good design (states are continuous)Not generally good design (states are continuous)

Reinforcement of eventsReinforcement of events User events – e.g. clicking on a buttonUser events – e.g. clicking on a button Mirrors real-world association between action and soundMirrors real-world association between action and sound Should be subtle and appropriateShould be subtle and appropriate

Designing with Sound (cont.)Designing with Sound (cont.) Sound as a navigation aidSound as a navigation aid

Most common in virtual environments – especially gamesMost common in virtual environments – especially games Various sound-cues can provide navigational aids (e.g. Various sound-cues can provide navigational aids (e.g.

mimicking acoustics of a building)mimicking acoustics of a building)

Sound to reinforce design metaphorSound to reinforce design metaphor Sounds taken from an object used as a metaphorSounds taken from an object used as a metaphor Ringing phone, clicking of a camera etcRinging phone, clicking of a camera etc Can be extremely effective, but avoid overuseCan be extremely effective, but avoid overuse

Sound for emotion and realismSound for emotion and realism Music / atmos etc.Music / atmos etc.

Physical aspects of soundPhysical aspects of sound VolumeVolume

Loud sounds are shocking (especially if emotive)Loud sounds are shocking (especially if emotive) Very occasionally, this is usefulVery occasionally, this is useful More often this is confusing and distracting – so they are usually to More often this is confusing and distracting – so they are usually to

be avoidedbe avoided

Lead-inLead-in A period of build-up makes a loud sound far less shockingA period of build-up makes a loud sound far less shocking Usually it is good practice to provide warning of loud soundsUsually it is good practice to provide warning of loud sounds

LengthLength Sound is extremely distractingSound is extremely distracting Hence – sounds should be as short as possibleHence – sounds should be as short as possible If they accompany visual cues they should be no longer than thisIf they accompany visual cues they should be no longer than this

Sound SetsSound Sets

Sound sets should be consistentSound sets should be consistent Same issues as icon setsSame issues as icon sets

Physical characteristicsPhysical characteristics Pitch, length, volume etcPitch, length, volume etc NB exceptions (e.g. warnings, or a long animation)NB exceptions (e.g. warnings, or a long animation)

AssociationsAssociations Sound “themes” are commonly usedSound “themes” are commonly used Physical source (e.g. water, metallic etc.)Physical source (e.g. water, metallic etc.) Environment (e.g. factory, forest etc.)Environment (e.g. factory, forest etc.)

Combinations of soundsCombinations of sounds Atmos will probably co-exist with feedback soundsAtmos will probably co-exist with feedback sounds Must be distinguishableMust be distinguishable

SpeechSpeech Parallel channels of communicationParallel channels of communication

e.g. images illustrated by speeche.g. images illustrated by speech can be very effective or confusing, depending upon can be very effective or confusing, depending upon

how well it is designed. Beware distraction!how well it is designed. Beware distraction! Slows interactionSlows interaction

Users often wait until the end of speechUsers often wait until the end of speech This can be frustrating – make sure it is easy to This can be frustrating – make sure it is easy to

interrupt (and make sure the user knows this!)interrupt (and make sure the user knows this!) Interactions can take a long timeInteractions can take a long time

ContextContext Speech is very contextual – the system must provide Speech is very contextual – the system must provide

the contextthe context

Speech (cont.)Speech (cont.) Text rulesText rules

Most of the writing guidelines also apply for speechMost of the writing guidelines also apply for speech ChoicesChoices

In exclusive speech systems, choice puts a higher demand In exclusive speech systems, choice puts a higher demand onto short-term memory than with graphics.onto short-term memory than with graphics.

Hence choice should be limited (e.g. consider voicemail)Hence choice should be limited (e.g. consider voicemail) Voice FragmentsVoice Fragments

Dynamic systems often assemble words (or even Dynamic systems often assemble words (or even phonemes)phonemes)

This always sounds very artificial (machine-like!)This always sounds very artificial (machine-like!) Scripted speech sounds far better, but is much harder to Scripted speech sounds far better, but is much harder to

make dynamicmake dynamic

Using sound in multimediaUsing sound in multimedia Sound is powerfulSound is powerful

Emotionally Emotionally InformativeInformative

Sound may not be available or desirableSound may not be available or desirable Accessibility issues – consider deaf usersAccessibility issues – consider deaf users Hardware (e.g. may be no speakers for public computers)Hardware (e.g. may be no speakers for public computers) Extraneous sound (noisy environments)Extraneous sound (noisy environments) Privacy issues (for public computers)Privacy issues (for public computers)

SummarySummary Make good use of soundMake good use of sound Design interfaces that have an alternative to soundDesign interfaces that have an alternative to sound