ComputingMultimedia Technology

A multimedia application might be defined as an interactive piece of software communicating to the user using several media, for example text, images (photographs,illustrations), audio (music, sounds), video and animation.

What is multimedia?

Multimedia data is either

• captured from real life (digitised) or created in the computer (synthesised)

• a bitmap (discrete) type of data or a vector (object oriented) type of data

• static (like an image) or time-dependent (like a video)

• You should be able to apply 3 of these terms to every data type.

The Software Development Process and Multimedia Applications

• Analysis

• Design

• Implementation

• Testing

• Documentation

• Evaluation

• Maintenance

The Software Development Process and Multimedia Applications

• Analysis: The main purpose of this stage is to be absolutely clear about what the multimedia project is supposed to do.

The Software Development Process and Multimedia Applications

• Design: Producing a detailed plan which defines what the different parts of the project are and how they are linked together.

The Software Development Process and Multimedia Applications

• Implementation: This is the point at which the multimedia authoring, or web-page authoring, package is used to turn the design into a working project.

• Testing: A series of practical tests are carried out to check that the multimedia project functions properly

• Documentation: User Guide and Technical Guide.

The Software Development Process and Multimedia Applications

The Software Development Process and Multimedia Applications

• Evaluation: Checks that the finished application meets the user’s requirements. Includes an assessment of the HCI design

• Maintenance: Fixing bugs and adapting the design to suit client needs and the demands of new technology.

Methodologies used in creation of multimedia applications

• Text editor

• WYSIWIG editors

• Multimedia authoring packages

• Presentation software

Methodologies used in creation of multimedia applications

• Text editor: Creating simple web pages with HTML using Notepad or TextPad

WYSIWYG editors

HTML – HyperText Markup Language

Methodologies used in creation of multimedia applications

• WYSIWIG editors: Web page/site development using applications such Dreamweaver and Freeway

Dreamweaver

Dreamweaver

Freeway

Methodologies used in creation of multimedia applications

• Multimedia authoring packages, e.g. Mediator 9 and Blender

Mediator 9 Blender Blender Example video

Methodologies used in creation of multimedia applications

• Presentation software, e.g. PowerPoint and Keynote

KeynotePowerPoint

Methodologies used to view multimedia applications

• Displaying multimedia applications in a browser.

• Displaying multimedia applications using a file player or viewer.

Methodologies used to view multimedia applications

Multimedia application as an executable file

Bit-mapped Graphic Data• Capturing still images

using a digital camera • Using an array of

Charged Coupled Devices to capture light coming in through a lens

• CCD: a series of sensors to capture light

• Capturing images using a scanner: also uses CCD

Bit-mapped Graphic Data

• Storing graphics as a bit map: each pixel in the image is represented by a binary number.

• Uncompressed bit-map format: a file which holds a binary number for each pixel in an image.

• Large file size: main limitation of bit-map format.• Need for compression: to relieve demands on

storage and transmission times.

Bit-mapped Graphic DataCompression using GIF format

• Lossless compression: GIF format compresses graphic data without losing any information about the image. It compresses by encoding repeated patterns of data.

• Limited number of colours: limited to 28, 256 colours• Transparency: colours set as transparent let the

background colours and patterns show through.• Used for storing cartoons, and line drawings.• Replacing GIF is PNG

Bit-mapped Graphic Data JPEG • Joint Photographic Experts Group: a

commonly used method of lossy compression for digital photography (image).

• parts of the graphic are cut out, e.g. shades of colour. At low rates of compression this is not noticeable.

JPEG

Bit-mapped Graphic Data

JPEG• JPEG format suitable for

storing photographs and paintings.• JPEG – 24 bit colour

know as True Colour giving 16,777,216 colours

RGB colour codes

• The RGB colour code is a system for representing colours in a computer system using a combination of the three primary colours red, green and blue

• RGB typically uses 24 bits to represent a colour by specifying the amount of each colour in 8 bits

• RGB(255,255,0) is yellow (red and green)• RGB(255,105,180) is “hot pink”- mainly red with

some green and blue

RGB colour coder Colour chart

Bit-mapped Graphic Data

Trading quality for file size• Resolution: increasing resolution increases the

number of pixels, can improve the quality of a graphic but increases the file size.

• Colour depth: increasing colour depth increases the number of colours or shades of grey, can improve the quality of a graphic but increases the file size.

• Lossy compression: reduces file size and, providing the rate of compression is not too high, does not affect the quality of the graphic.

Bit-mapped Graphic Data

Simple bit-map editing and creation software

Painting programs

Fill tool: a feature for pouring colour into a graphic.

Paintbrush tool: for more precise application of colour.

Bit-mapped Graphic Data

Main features of image editing programs

• Decrease resolution

• Alter colour depth

• Crop

• Alter brightness and contrast

• Insert graphic

• Re-size.

Technical terms

• Dithering

• Anti-aliasing

• Re-sampling

Bit-mapped Graphic Data

Hardware for displaying 2D graphics

• CRT monitor: bulky, heavy and consume more power the LCD/TFT screens

• LCD and TFT screens: flat, light, and need less power than CRT monitors

CRT in specialised markets LCD TFT

Bit-mapped Graphic Data

Hardware for displaying graphic data

• Need for graphics cards to store and process graphic data, relieving the main processor of the task.

Digitised Sound DataHardware for capturing sound

• Microphone

• Sound cards: to sample, store and process audio data.

Digitised Sound DataUncompressed sound data

• RAW: Uncompressed samples of sound waves

0101010101010101 1111111101010101 0011001100101011 1100101000110001

• RIFF: Resource Interchange File Format• WAV: Microsoft's format for sound files,

part of RIFF

Digitised Sound Data

• Lossy compression: reduces file sizes by cutting out some of the data.

• MP3: uses lossy compression without noticeable loss of sound quality.

Digitised Sound Data

Balancing file size and sound quality• Sampling depth: increased sound depth =

greater range of values = better sound quality and greater file size.

• Sampling frequency: The higher the sampling frequency, the better the sound quality, the greater the file size.

• Sound time: affects file size but not quality.

Digitised Sound Data

Simple sound editing software• Reducing sample frequency, e.g. from 44.1

KHz to 22.05 KHz, reduces file size and audio quality.

• Reducing sample depth, e.g. from 16 bits to 8 bits per sample, reduces file size and audio quality.

Digitised Sound Data

Editing sound file features:

Volume

Effects

Echo

Reverse

Digitised Sound Data

Sound cards and playback:

Sound cards needed to change the digital audio data into analogue signals to control output from speakers.

Video Data

Hardware for capturing images• Digital video camera

• Web cam

Video Data

Storing video data

• Uncompressed format

• Uncompressed video data = Large file sizes

• 1 second of uncompressed wide-screen video can take up 53 Megabytes of storage

• AVI: Audio Visual Interleave, an uncompressed format.

Video Data

Compressing video data

• MPEG

Lossy compression: cuts out unnecessary parts of a video clip

Video Data

Compressing video dataUsing MPEG compression:• Reduces file sizes

• With no noticeable loss of video quality

• Only key frames are stored in full as a JPEG

• Then only the differences between key frames are stored

Calculating the file size of video

• Find the file size of a single frame– Use resolution and colour depth

• Multiply the size of a single frame by the frame rate and number of seconds in the clip

• (R*CD) * (fps*L)

Calculating the file size of video

• 20 minute video • A single frame • 768*576 = 442368 *24 = 10616832/8 = 1327104

(bytes)• 1327104/1024 = 1296/1024 = 1.265625 MB

• Frame rate and length• 1.265625 *25= 31.640625 *20*60 =

37968.75/1024 = 37.08 Gigabytes

Video Data

Video quality and file size• Colour depth: increasing colour depth improves

quality and file sizes.• Resolution: increasing resolution improves quality

and increases file sizes.• Frame rate: measured in frames per second, fps.

30fps is the rate for a video clip. Increasing frame rate increases file size. Lower frame rates reduce file size but make video clip ‘jerky’.

Video Data

Video quality and file size• Video time: increasing or reducing the time of a

video is the obvious way to affect the file size. Quality of the display of the clip is not affected.

• Lossy compression: Using MPEG compression reduces file sizes without affecting quality.

Video Data

Video editing software features and applications used with single video clips

• Cropping: cutting unwanted data from the beginning and end of a clip.

• Add effects, titles, sound tracks.

• The need for graphics cards to process and output video data.

Vector Graphics Data

• VG store an image as a list of individual layered objects with each object being described by its attributes

3D objects in vector graphics

• The attributes will be the same as flat (2D) objects but with a few extra ones

• angle of rotation, the x, y and z axis need to be defined

• surface texture, the surface appearance of the 3D object

• shadow, the position and intensity of the light source

Vector Graphics Data

Standard formats for vector graphics

Scalable Vector Graphics (SVG) format

Scalable: resolution independent

Vector: represents objects by defining a series of attributes

Vector Graphics Data

Standard formats for vector graphics

Virtual Reality Markup Language

A standard language used to model and animate geometric shapes

Used to define 3D environments for the WWW.

A comparison of Vector and Bitmapped Graphics

Synthesised Sound Data

Musical Instrument Digital Interface: MIDI

Common attributes of notes stored as MIDI data:– Instrument: defines which instrument is playing– Pitch: defines the height of the note– Volume: determines the amplitude– Duration: determines the length of the note.

Synthesised Sound Data

Common attributes of notes stored as MIDI data

Duration: determines the length of the note.

Tempo: the speed at which a piece of music is played.

MIDI hardware/software

• MIDI keyboards, or other instruments, are used to create sound data in MIDI format

• Or just use virtual instruments through MIDI software

Advantages of MIDI

• Music can be produced at the computer with an easy to use interface no instruments required

• Individual instruments and notes can be easily edited.

• MIDI format has much smaller file size than digitised sound data as it is the attributes of the sound that is stored not every sound sample.

• The MIDI format is a common standard which allows the musical data to be passed between devices made by different manufactures of musical instruments and a computer

Disadvantages of MIDI

• Synthesised sound, poor representation of human voice and other naturally occurring non uniform sounds

• Limited realism, sound can be “electronic” sounding

• Sound quality, dependent on quality of soundcard

Converging technologies

• Converging technologies means two or more technologies being combined, some examples are smartphones, Pocket PCs, digital television and tablet computers.

Implications: Multimedia Technologies

Converging contemporary technologies• Smartphone: merging technologies of a

mobile phone and a laptop.• Pocket PC: merging technologies of a

laptop, mobile phone and desktop operating system and application software.

Implications of Multimedia Technologies

Contemporary technologies

• Digital television: an interactive multimedia device which, because of the digital nature of its signals, is easily integrated into your digital computer and home networks.

Implications of Multimedia Technologies

Contemporary technologies

Virtual reality

The ultimate multimedia experience where the user is immersed in the world of the computer and can journey through, and interact with, a computer generated 3-dimensional multimedia world.

Virtual reality

• Used for

• For training e.g. pilots

• Creating and inspecting and testing a 3D CAD model

• Simulating scientific processes

• Forecasting e.g. weather

• Gaming

Implications of Multimedia Technologies

• Immersive VR

• Output :Using speakers, stereo screens, headsets

• Input: sensors in gloves, headsets and suits