1 cs 558 : multimedia systems lecturer: don adjeroh office: rm937, esb e-mail: [email protected]...
TRANSCRIPT
![Page 1: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/1.jpg)
1
CS 558 : CS 558 : Multimedia Systems Multimedia Systems
Lecturer: Don Adjeroh
Office: Rm937, ESB
e-mail: [email protected]
Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)
Thursday 9:30 - 10:55pm (Rm G449, ESB)
![Page 2: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/2.jpg)
2
Important Dates
Assignment-1 Week of Feb. 11, 2008 (Due 2 weeks after)
Assignment-2 Week of Mar 3, 2008 (Due 2 weeks after)
Assignment-3 Week of Mar 17, 2008 (Due 2 weeks after)
Student Projects Week of Mar 3, 2008 (due Apr. 28, 2008)
Student Seminar From Apr 21, 2008
Assessment
Weekly Quiz 20%
3 Assignments 30%
Student Projects 20%
Student Seminar 20%
Class Participation 5% extra on Assignments
![Page 3: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/3.jpg)
3
References:
Lu G., Communication and Computing for Distributed Multimedia Systems, Artech House, Norwood, Massachusetts, 1996.
Ze Nian Li , Mark S. Drew, Fundamentals of Multimedia, Prentice Hall, 2004
P. A. Chou and M. van der Schaar, (eds.), Multimedia over IP and Wireless Networks: Compression, Networking, and Systems, Elsevier, 2007.
4. Gonzalez R. C. and Woods R. E., Digital Image Processing. Addison-Wesley Publishing Co., Reading, Massachusetts, 2001.
Others Raghavan, S. V. and Tripathi, S. K., Networked Multimedia Systems:
Concepts, Architecture and Design, Prentice Hall, 1997 Papers and other reference materials to be provided on need basis
![Page 4: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/4.jpg)
4
Course OutlineCourse Outline
Introduction
Multimedia Communications and Transport
Multimedia Data Compression
Multimedia Information Systems
Information Networks
![Page 5: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/5.jpg)
5
Elements of MM Computing SystemsElements of MM Computing Systems
MM data acquisition/generation MM storage MM information processing MM communication MM presentation MM QoS specification and performance measurement.
![Page 6: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/6.jpg)
6
Data Acquisition/generationData Acquisition/generation
Multimedia data can be computer generated - e.g. graphic objects or animation sequences, or images.
Most MM data are however externally acquired. Externally acquired MM data requires two basic units: A transducer – a physical sensing device that transforms the
physical data such as sound or light into electrical signals. A digitizer – a device that converts the electrical signal into a
digital form, example the scanner.
Some devices perform the two basic functions at the same time, example digital cameras.
![Page 7: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/7.jpg)
7
Analogue to Digital ConversionAnalogue to Digital Conversion
Stages in ADC : sampling, quantization, and coding.
Sampling – converts the continuous-time signal into a discrete-time signal.
Quantization - converts the discrete-time continuous values into discrete-amplitude values. A sample value that falls into a given quantization level is assigned that quantization level.
Coding - represents the quantized amplitude values in the form of digital codes. Facilitates the transmission and storage of the digital signal.
![Page 8: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/8.jpg)
8
![Page 9: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/9.jpg)
9
MM Data StorageMM Data StorageText: (One page)
Plain text at 60 lines/page, with 80 characters 48KB
Formatted text: (2 bytes/character) 96KB
Images: (e.g. a 1024x1024 image at 8-bits/colour level)
Grey-scale image requires about 1MB of storage.
Colour image will require 3MB of storage.
Video:
A 5-minute video clip at 30fps will require:
5*60*30 *3MB=27000 MB or 27GB of storage
Audio - Storage depends on: the sampling rate used, and the number of bits used to represent each sample value
Generally, Bit rate =sampling rate * no of bits per sample
![Page 10: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/10.jpg)
10
Example:
For CD audio, the frequency range is 20 – 20,000Hz, or a bandwidth of about 20kHz. For accurate reconstruction, the minimum sampling rate is thus 40kHz. The industry standard is 44.1kHz. Using 16 bit quantization, and for the 2 audio (i.e. stereo) channels, we will have 44100x2x16 or 1410kbps. Thus, to store a 5 minute CD audio, we will need 1410*5*60 kb (or > 50MB) of storage.
Four types of storage used for multimedia data: Short-term storage: main memory On-line storage: magnetic disks, optical disks, CDs, MOs Near-line storage: juke-boxes, Off-line (archival) storage: magnetic tapes, WORM optical
disks
![Page 11: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/11.jpg)
11
MM Information ProcessingMultimedia information usually involves
media-specific processing and analysis, and considerations across different media.
E.g: MM indexing, search and retrieval, MM data compression, MM information summarization, etc.
MM Communication limited network bandwidth large data sizes
A typical voice-phone line can transmit at rates up to 9600bps. To transmit the 8-bit 1024x1024 grey-scale image over this line will require about 14 minutes.
Approaches: compression, network & transport protocols, network scheduling and multimedia shaping algorithms
![Page 12: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/12.jpg)
12
MM PresentationHow is the multimedia data finally presented to the user?
Key issues : Human perception of MM information Play-out or media consumption rate Interactivity – what level of control does the user have over the
presentation, Synchronization – both inter- and intra-media Consideration of spatio-temporal constraints and Media integration.
Performance and QoSPerformance of a MM system is often checked with respect to the
acceptable QoS specified by the user.
The overall performance is typically affected by the performance of the different functional elements.
![Page 13: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/13.jpg)
13
Nature of MM DataNature of MM Data
MM Data type - form in which the MM data is presented
Example MM data types Text – plain text, formatted text Images – colour, black & white photos, maps, paintings Graphic objects – drawings, sketches, illustrations, 2D or
3D objects Animation sequences – moving images or graphic objects,
usually independently generated Video – sequence of images showing a record of real-life
event, usually produced by a video recorder Audio – speech and non-speech audio, music, etc Composites – formed from a combination of two or more
media types.
![Page 14: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/14.jpg)
14
Classification of MM Data Static Vs Continuous media
Static media – are time independent. Meaning remains the same independent of presentation time.
Continuous media – are time-dependent. Their contents and meaning depend on the time and rate at which they are presented.
Stored Vs live, -for continuous media.
Multimedia system usually refers to systems a that is capable of handling at least one continuous-media data type, and one static media type.
Multimedia document - a document that contains at least one type of each media class.
![Page 15: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/15.jpg)
15
General characteristics of MM dataGeneral characteristics of MM data
Huge volumes of dataMM data typically involves huge volumes of data.
Compression and storage techniques to the rescue! Temporal constraints –
Continuous MM data have temporal dependencies. These must be played-out at a given standard rate and in a certain order .
Implications for storage, processing, communication and presentation of these data types.
Inter/Intra media dependence Intra-media dependence - dependence among different
components of data belonging to the same media type. Inter-media dependence involves multimedia data across
different media types.
![Page 16: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/16.jpg)
16
Semantic structure
Users have the final say on information content. MM data items may be described differently by different users. It is often difficult to precisely describe the information content of a multimedia data. Querying and searching MM data is thus performed in terms of similarity or approximate matches.
Error tolerance
Some level of error is often tolerated in considering the presented MM data. MM system designers usually make use of the limitations of human perception in providing acceptable levels of service to the user
Criterion: the presented results should not degrade below the user’s stated acceptable level, as indicated in the QoS specification.
![Page 17: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/17.jpg)
17
Characteristics of MM SystemsCharacteristics of MM Systems Integration - aims at minimizing the number of
different systems that need to be connected.
Interactivity - how much control the user has over the presentation.
Degree of interactivity or user customization: time to start order of presentation (linear or non-linear) form of presentation speed of presentation automation of the presentation
Pro: could provide statistical info about user interaction
Con: Added level of complexity.
![Page 18: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/18.jpg)
18
Computer control - MM systems require computer control of the presentation of information to the users.
Digital Representation - Some of the advantages: Uniformity in representation and information handling Error-free transformations and reliable transmission Ease of encryption Ease of regeneration or replication.
Two problems with digitization: Digitization is generally a not completely reversible. some
errors may be introduces. Digitized data often results in large data sizes, which in turn
requires more storage. Performance
QoS Traditional performance measures not adequate !
![Page 19: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/19.jpg)
19
Classification of MM systems/applicationsClassification of MM systems/applications
Stored Vs Live (usually applicable to continuous media)
Interactive Vs non-interactiveWhat level of control does the user have over the presentation
of the MM data? Networked Vs non-networked (local)
Local – involves only a stand-alone system. Examples applications are authoring, training, etc.
Networked – distributed, usually involves clients and one or more servers communicating over a network.
![Page 20: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/20.jpg)
20
Some application areasSome application areas
Entertainment - VoD, games, tour, museums, Education- tele-teaching, authoring tools, tutoring Medicine- telemedicine Business - video conferencing, video phone,
training, presentation Information Kiosks (IoD) Digital libraries Digital TV broadcasting Personal communication - video phone Others.
![Page 21: 1 CS 558 : Multimedia Systems Lecturer: Don Adjeroh Office: Rm937, ESB e-mail: don@csee.wvu.edu Course Schedule: Tuesday 9:30 - 10:55am (Rm G449, ESB)](https://reader034.vdocuments.mx/reader034/viewer/2022051620/56649ee85503460f94bf9329/html5/thumbnails/21.jpg)
21
Other related issues MM, Hypermedia, and Hypertext
MM and the Internet
MM and Virtual Reality
Course Structure General approach taken in this
course - see diagram
DistributedMM
MMCompression
MM InfoSystems