![Page 1: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/1.jpg)
NaresuanUniversity
Multimedia
Paisarn Muneesawang
http://www.ecpe.nu.ac.th/paisarn/multimedia
![Page 2: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/2.jpg)
NaresuanUniversity
What is Multimedia?
• Apps that involve more than conventional data types (e.g., text, drawing and images)
• Best examples are continuous media (e.g., animations, audio, and video)– Called continuous media because of time basis
• <text document, graphic, image, audio, video>
![Page 3: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/3.jpg)
NaresuanUniversity
Why is Multimedia?
• Human brain is much more efficient at processing and interpreting visual and audio information– than text, text+graphics
• Human beings and technology developments– image: 2-dimensional data– video: 3-dimensional data+audio– virtual reality
![Page 4: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/4.jpg)
NaresuanUniversity
Challenges
• Multimedia data size
• Real-time nature of multimedia
• Why is the semantic nature of multimedia data a problem?
![Page 5: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/5.jpg)
NaresuanUniversity
Multimedia Data Size
![Page 6: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/6.jpg)
NaresuanUniversity
The First Challenge is Size
• A single good quality colored image could require 6 Mb.
• A video object consist of a sequence of such images (called frames) will be very large.– 30 frames per second, a five minute video clip would require 54 Gb.
• A typical sequence of audio will occupy 8 Kb for each second.
• Data size will effect the storage, retrieval and transmission of multimedia. Thus data compression techniques are crucial.
![Page 7: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/7.jpg)
NaresuanUniversity
Multimedia Data Acquisition
DataCapture LossSampling Predict/
Model Transform Send
DelayReorderReceiveRestoreDisplay
![Page 8: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/8.jpg)
NaresuanUniversity
Digitization of Data
• First Step: capturing analogue signal.
• Second Step: sampling the signal and convert it to digital value,– Theory of sampling by Nyquist: sampling rate >= (2)*(BW ) to accurately
reproduce an analog signal.
• Sampling rate data size
![Page 9: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/9.jpg)
NaresuanUniversity
Multimedia System Design
• Should be designed based on human visual and audio sensory systems:– The human voice is in the frequency range 0-4kHz requiring 8kHz rate
for digitization.
– But, the intelligible part of human speech is carried in the voice band (300-3400 Hz). Thus, we can use smaller bandwidth.
• Should be designed based on applications:– Image in medical application may require higher resolution than images
transmitted on the web application.
![Page 10: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/10.jpg)
NaresuanUniversity
Reducing the Media Object’s Size
• Compression is the technique to remove redundant data.– For example, by abbreviating any repeated information in an image and
eliminating information that is difficult for the human eye to see.
• Compression/decompression process– Lossy compression some information is loss by the process.– Lossless compression non of the original information is lost.
• Many compression methods attempt to address the same objectives:
– Reduced BW and/or storage;– Decode signal should be as close as possible to the original;– Lowest possible implementation strategy;– Application to as many signal types as possible;– Robustness;– Scalability;– extensibility
![Page 11: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/11.jpg)
NaresuanUniversity
Media Data and Format Standards
Media data Special nature Typical digital file formats
Text data ASCII, RTF, HTML, SGML, XML
Audio data Time dependence with digitized data size can affect quality
WAV, AU, MPEG-MP3
Image data Results from pictures, drawings and photographs
JPEG, GIF, TIFF, BMP, PNG
Video data Time-dependent sequence of video frames
AVI, FLI, GIF, JPEG, MPEG
![Page 12: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/12.jpg)
NaresuanUniversity
How to Choose the Formats
• Multimedia data is available in a range of file formats. Although this variety could appear confusing, this gives the designer great flexibility.
• We may select a format that:– Suit the application and the requirements of different users.
This computer-generated graphic image can be stored in several different file formats.
bit map: 1088 KbJPEG: 748 KbGIF: 109 Kb.
Note: JPEG is designed for photographs so it is not necessarily the best format for computer graphic files.
![Page 13: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/13.jpg)
NaresuanUniversity
Digital File Formats, Applications and Features
Digital video format
Application Features
MPEG Multimedia Key frame plus motion encoding
INDEO Web pages Wavelet progression
CENEPAK Computer Video efficient
SORENSON Computer/videophone Video efficient
QUICKTIME Computer Incorporates compression
AVI Computer Incorporates compression
![Page 14: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/14.jpg)
NaresuanUniversity
Important File Formats
• Image: JPEG is a standard of the ISO and can operate in four modes: sequential, progressive, lossless, and hierarchical.
• High quality compression for audio and video is the MPEG standards of the Moving Picture Expert Group.– MPEG-1: Interactive CDROM, video CDROM, Audio-MP3
– MPEG-2: Digital TV, DVD, Audio-Music
– MPEG-4: Multimedia for fixed and mobile web. (for low bandwidth applications)
– MPEG-7: Search video and audio content
– MPEG-21: Multimedia framework for interoperability
![Page 15: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/15.jpg)
NaresuanUniversity
How Large is a Video Object
Time 640x480 320x240 160x120
Compressed
JPEG 25:1
640x480
Compressed MPEG 100:1
640x480
1 s 27 Mb 6.75 Mb 1.68 1.1 Mb 270 Kb
1 min 1.6 Mb 400 Mb 100 65 Mb 16 Mb
1 h 97 Gb 24 Gb 6 Gb 3.9 Gb 970 Mb
1000 h 97 Tb 24 Tb 6 Tb
![Page 16: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/16.jpg)
NaresuanUniversity
Real-Time Nature of Multimedia
![Page 17: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/17.jpg)
NaresuanUniversity
The Real-Time Nature of Multimedia
• Video and audio are continuous periodic media:– The frames of the video must run in the correct sequence and at an
acceptable rate, otherwise it becomes meaningless.
• The effect of time to the media objects is referred to as the real-time nature of multimedia.
• This relationship with time will have significant for the way the media objects are stored, retrieved, transmitted and synchronized.
![Page 18: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/18.jpg)
NaresuanUniversity
Video Data
• Video is a continuous media but for database storage and manipulation such as random access it is important to be able to deal with portions of video object.
• Video Segmentation―cutting long video into portions:shot, scene, and clip
– Shot define a low level syntactic building blocks of video sequence.
– Scene is the logical grouping of shots into semantic unit.
– Clip is not clearly defined so it can last from a few seconds to several hours.
![Page 19: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/19.jpg)
NaresuanUniversity
Video Segmentation
![Page 20: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/20.jpg)
NaresuanUniversity
Story Clip
Scene
Shot a Shot b
![Page 21: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/21.jpg)
NaresuanUniversity
Organization of Video Data
![Page 22: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/22.jpg)
NaresuanUniversity
Why is the Semantic Nature of Multimedia Data a Problem?
![Page 23: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/23.jpg)
NaresuanUniversity
Why is the Semantic Nature of Multimedia Data a Problem?
• “A picture is worth a thousand words”
Subjectivity:Different people have difference opinions.
Multimedia data, instead of having an explicit nature, have an implicit semantic nature.
![Page 24: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/24.jpg)
NaresuanUniversity
Describing Multimedia Content
• Traditional database systems:
Metadata is know as:– Schema definitions, indexes, users, integrity constraints, security constraints etc.
.......Information that is of interest to the system itself.
• But, in multimedia database management systems (MMDBMS) metadata can refer to information about individual objects.
Metadata = Characteristics of the media data:
– Texture for images;– Frequencies for audio;– Speech ― keywords such as identification of the speakers, place, time;– Video clip ― camera motion and lighting
![Page 25: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/25.jpg)
NaresuanUniversity
Techniques for Describing Multimedia
• Image annotation: a simple text description (keywords)– Time consuming for a large image database
– Difficult to describe an image precisely in text
• Content-based technique: An image is described by specific features in the image, such as color or texture.– Description is generated automatically
– Description is the content itself (low-level feature)
![Page 26: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/26.jpg)
NaresuanUniversityImage Content with Low-Level Description
![Page 27: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/27.jpg)
NaresuanUniversity
Metadata for Multimedia
• Text metadata + content descriptors
• A new standard MPEG-7 will have a major impact on the issue of metadata. MPEG-7 multimedia contents include:– Low-level descriptions of each individual object in a scene, such as
shape, color, and movement;
– High-level abstract descriptions of the scene, the objects it contains and the events taking place;
– Audio information such as key, mood and tempo.
• MPEG-7 makes the process of searching and retrieving
an image or a video clip much easier.
![Page 28: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/28.jpg)
NaresuanUniversityKey-Word Search
![Page 29: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/29.jpg)
NaresuanUniversity
Content-Based Retrieval
![Page 30: Naresuan University Multimedia Paisarn Muneesawang](https://reader036.vdocuments.mx/reader036/viewer/2022062321/56649e365503460f94b25efb/html5/thumbnails/30.jpg)
NaresuanUniversity
Content-Based Retrieval