Image and Video Image and Video

Processing –Processing –

An IntroductionAn Introduction

Image and Video Image and Video

Processing –Processing –

An IntroductionAn Introduction

Fall Fall ‘‘13 Instructor: 13 Instructor: Dr. Engr. Junaid Dr. Engr. Junaid Zafar Zafar

Electrical Engineering Department Electrical Engineering Department

Government College University, LahoreGovernment College University, Lahore

Scope of EE-7105Scope of EE-7105Scope of EE-7105Scope of EE-7105

First graduate course on image/video processingFirst graduate course on image/video processing

Not assume you have much exposure on image Not assume you have much exposure on image processing at undergraduate levelprocessing at undergraduate level

Require and build on background in random process Require and build on background in random process and DSPand DSP

Emphasis on fundamental conceptsEmphasis on fundamental concepts Provide theoretical foundations on multi-dimensional Provide theoretical foundations on multi-dimensional

signal processing built upon pre-requisitessignal processing built upon pre-requisites Coupled with assignments and projects for hands-on Coupled with assignments and projects for hands-on

experience and reinforcement of the conceptsexperience and reinforcement of the concepts

Follow-up courses Follow-up courses image analysis, computer vision, pattern recognitionimage analysis, computer vision, pattern recognition multimedia communications and securitymultimedia communications and security

Textbooks and ReferencesTextbooks and ReferencesTextbooks and ReferencesTextbooks and References

R.C. Gonzalez and R.E. Woods: R.C. Gonzalez and R.E. Woods: Digital Image ProcessingDigital Image Processing, , Prentice Hall, 3Prentice Hall, 3rdrd Edition, 2008. Edition, 2008.

J. G. Proakis and D.G Manolakis: J. G. Proakis and D.G Manolakis: Digital Signal Processing, Digital Signal Processing, Principles, Algorithms & ApplicationsPrinciples, Algorithms & Applications, 4, 4thth Edition. Edition.

Related technical publications (will be announced in class).Related technical publications (will be announced in class).

Other related textbooks Other related textbooks Y. Wang, J. Ostermann, Y-Q. Zhang: Y. Wang, J. Ostermann, Y-Q. Zhang: Digital Video Digital Video

Processing and CommunicationsProcessing and Communications, Prentice Hall, 2001. , Prentice Hall, 2001. A. K. Jain: A. K. Jain: Fundamentals of Digital Image ProcessingFundamentals of Digital Image Processing, ,

Prentice Hall, 1989. Prentice Hall, 1989. John W. Woods: John W. Woods: Multidimensional Signal, Image, and Multidimensional Signal, Image, and

Video Processing and CodingVideo Processing and Coding, Academic Press, 2006. , Academic Press, 2006. A. Bovik: A. Bovik: Handbook Of Image & Video ProcessingHandbook Of Image & Video Processing, 2, 2ndnd

Edition, Academic Press, 2005.Edition, Academic Press, 2005.

Image and Video Processing: Image and Video Processing:

An Introduction and An Introduction and

Overview Overview

Image and Video Processing: Image and Video Processing:

An Introduction and An Introduction and

Overview Overview

Why Do We Process Images?Why Do We Process Images?Why Do We Process Images?Why Do We Process Images?

Enhancement and restorationEnhancement and restoration Remove artifacts and scratches from an old photo/movieRemove artifacts and scratches from an old photo/movie Improve contrast and correct blurred imagesImprove contrast and correct blurred images

Composition Composition (for magazines and movies),(for magazines and movies), Display, Display, Printing …Printing …

Transmission and storageTransmission and storage images from oversea via Internet, or from a remote planetimages from oversea via Internet, or from a remote planet

Information analysis and automated recognitionInformation analysis and automated recognition Providing Providing ““human visionhuman vision”” to machines to machines

Medical imaging Medical imaging for diagnosis and explorationfor diagnosis and exploration Security, forensics and rights protectionSecurity, forensics and rights protection

Encryption, hashing, digital watermarking, digital fingerprinting Encryption, hashing, digital watermarking, digital fingerprinting ……

Why Digital?Why Digital?Why Digital?Why Digital?

““ExactnessExactness”” Perfect reproduction without degradationPerfect reproduction without degradation Perfect duplication of processing resultPerfect duplication of processing result

Convenient & powerful computer-aided Convenient & powerful computer-aided processingprocessing Can perform sophisticated processing through Can perform sophisticated processing through

computer hardware or softwarecomputer hardware or software Even kindergartners can do some!Even kindergartners can do some!

Easy storage and transmissionEasy storage and transmission 1 CD can store hundreds of family photos!1 CD can store hundreds of family photos! Paperless transmission of high quality photos Paperless transmission of high quality photos

through network within secondsthrough network within seconds

Examples of Digital Image & Examples of Digital Image & Video ProcessingVideo ProcessingExamples of Digital Image & Examples of Digital Image & Video ProcessingVideo Processing

Compression Compression Manipulation and RestorationManipulation and Restoration

Restoration of blurred and damaged imagesRestoration of blurred and damaged images Noise removal and reductionNoise removal and reduction MorphingMorphing

ApplicationsApplications Visual mosaicing and virtual viewsVisual mosaicing and virtual views Face detectionFace detection Visible and invisible watermarkingVisible and invisible watermarking Error concealment and resilience in video Error concealment and resilience in video

transmissiontransmission

CompressionCompressionCompressionCompression

Color image of 600x800 pixelsColor image of 600x800 pixels Without compressionWithout compression

600*800 * 24 bits/pixel600*800 * 24 bits/pixel = 11.52K bits = 1.44M bytes = 11.52K bits = 1.44M bytes

After JPEG compression After JPEG compression (popularly used on web)(popularly used on web)

only 89K bytesonly 89K bytes compression ratio ~ 16:1compression ratio ~ 16:1

Movie ~ Movie ~ Image SequenceImage Sequence 720x 480 per frame, 30 720x 480 per frame, 30

frames/sec, 24 bits/pixelframes/sec, 24 bits/pixel Raw video ~ 243M bits/secRaw video ~ 243M bits/sec DVD ~ about 5M bits/secDVD ~ about 5M bits/sec Compression ratio ~ 48:1Compression ratio ~ 48:1

DenoisingDenoisingDenoisingDenoising

DeblurringDeblurringDeblurringDeblurring

Visual MosaicingVisual MosaicingVisual MosaicingVisual Mosaicing

Image enhancement, feature extractions, and statistical Image enhancement, feature extractions, and statistical modeling are often important steps in computer vision tasksmodeling are often important steps in computer vision tasks

Face DetectionFace Detection Face DetectionFace Detection

Visible Digital Visible Digital WatermarksWatermarks

Visible Digital Visible Digital WatermarksWatermarks

Invisible Watermark Invisible Watermark Invisible Watermark Invisible Watermark

Original, marked, and their amplified luminance differenceOriginal, marked, and their amplified luminance difference human visual model for imperceptibility: protect smooth areas and sharp edgeshuman visual model for imperceptibility: protect smooth areas and sharp edges

Error ConcealmentError ConcealmentError ConcealmentError Concealment

25% blocks in a checkerboard pattern are corrupted

corrupted blocks are concealed via edge-directed interpolation

(a) original lenna image (c) concealed lenna image(b) corrupted lenna image

Different Ways to View an Different Ways to View an Image Image Different Ways to View an Different Ways to View an Image Image

In 3-D (x,y, z) plot with z=I(x,y);red color for high value and blue for low

Equal value contour in (x,y) plane

Intensity visualization over 2-D (x,y) plane

Sampling and QuantizationSampling and QuantizationSampling and QuantizationSampling and Quantization

Computer handles Computer handles ““discretediscrete”” data. data. SamplingSampling

Sample the value of the image at the nodes of a Sample the value of the image at the nodes of a regular grid on the image plane.regular grid on the image plane.

A pixel (picture element) at (i, j) is the image A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer intensity value at grid point indexed by the integer coordinate (i, j).coordinate (i, j).

QuantizationQuantization Is a process of transforming a real valued sampled Is a process of transforming a real valued sampled

image to one taking only a finite number of image to one taking only a finite number of distinct values.distinct values.

Each sampled value in a 256-level grayscale image Each sampled value in a 256-level grayscale image is represented by 8 bits.is represented by 8 bits.

0 (black)

255 (white)

Examples of SamplingExamples of SamplingExamples of SamplingExamples of Sampling

256x256 64x64 16x16

Examples of Examples of

QuantizaionQuantizaion

Examples of Examples of

QuantizaionQuantizaion

8 bits / pixel 4 bits / pixel 2 bits / pixel