Digital Image ProcessingDigital Image ProcessingIt makes all the difference whether one It makes all the difference whether one sees darkness through the light or sees darkness through the light or brightness through the shadows.brightness through the shadows.

David LindsayDavid Lindsay

OverviewOverview

Early days of computing, data was numerical.Later, textual data became more common.Today, many other forms of data: voice, music, speech, images, computer graphics, etc.Each of these types of data are signals.Loosely defined, a signal is a function that conveys information.

Relationship of SignalProcessing to other fields

As long as people have tried to send or receive through electronic media : telegraphs, telephones, television, radar, etc. there has been the realization that these signals may be affected by the system used to acquire, transmit, or process them.Sometimes, these systems are imperfect and introduce noise, distortion, or other artifacts.

Relationship of SignalProcessing to other fields (1)

Understanding the effects these systems have and finding ways to correct them is the fundamental of signal processing.Sometimes, these signals are specific messages that we create and send to someone else (e.g., telegraph, telephone, television, digital networking, etc.).That is, we specifically introduce the information content into the signal and hope to extract it out later.

Relationship of SignalProcessing to other fields (2)

Sometimes, these man-made signals are encoding of natural phenomena (audio signal, acquired image, etc.),Other times we can create them from scratch (speech generation, computer generated music, computer graphics).Finally, we can sometimes merge these technologies together by acquiring a natural signal, processing it, and then transmitting it in some fashion.

History: The Bartlane History: The Bartlane TransmitterTransmitter

History: The Bartlane History: The Bartlane Transatlantic CableTransatlantic Cable

A digital picture A digital picture produced in 1921 produced in 1921 from a coded tape from a coded tape by a telegraph by a telegraph printer with special printer with special type faces. type faces. (McFarlane.†)(McFarlane.†)

History: The Bartlane History: The Bartlane Transatlantic CableTransatlantic Cable

A digital picture A digital picture made in 1922 from made in 1922 from a tape punched a tape punched after the signals after the signals had crossed the had crossed the Atlantic twice. Atlantic twice. Some errors are Some errors are visible. visible. (McFarlane.)(McFarlane.)

History: The Bartlane History: The Bartlane Transatlantic CableTransatlantic Cable

Unretouched cable Unretouched cable picture of General picture of General Pershing and Pershing and Foch, transmitted Foch, transmitted in 1929 from in 1929 from London to New London to New York by 15-tone York by 15-tone equipment. equipment. (McFarlane.)(McFarlane.)

First official image of the moon First official image of the moon taken from spacetaken from space

Image Processing in medicineImage Processing in medicineX-RAY, 1895. Wilhelm X-RAY, 1895. Wilhelm

Conrad RoentgenConrad RoentgenNobel Prize in PhysicsNobel Prize in Physics

Image Processing in MedicineImage Processing in MedicineInventors of the Computerized Axial Tomography (CAT or CT), early Inventors of the Computerized Axial Tomography (CAT or CT), early 1970s. Sir Godfrey N. Hounsfield (left) and Prof. Allan M. Cormack 1970s. Sir Godfrey N. Hounsfield (left) and Prof. Allan M. Cormack

(right). Shared Nobel prize in medicine.(right). Shared Nobel prize in medicine.

From acquisition to interpretationFrom acquisition to interpretation

Acquirenaturalimage

Enhancethe

Picture

Compressfor

transmission

Encode andtransmit over

digital network

Transmittedcodes ofimage

Decoded Decompressed

Displayed tocreate another

signal (visible lightof the display)

Sender:

Recipient:

Computer

Receivedby eyes

Interpreted in somefashion by our brain

Concerned fieldsConcerned fields

Digital CommunicationDigital CommunicationCompressionCompressionSpeech Synthesis and RecognitionSpeech Synthesis and RecognitionComputer GraphicsComputer GraphicsImage ProcessingImage ProcessingComputer VisionComputer Vision

What is Image Processing?What is Image Processing?

Image processing is a subclass of signal processing concerned specifically with pictures.Improve image quality for human perception and/or computer interpretation.

Image ImageProcessing

BetterImage

Several fields deal with images

Computer Graphics : the creation of images.Image Processing : the enhancement or other manipulation of the image – the result of which is usually another image.Computer Vision: the analysis of image content.

Several fields deal with images

Computer Vision, Image Processing and Computer Graphics often work together to produce amazing results.

Input / OutputInput / Output ImageImage DescriptionDescription

ImageImage Image Image ProcessingProcessing

Computer Computer VisionVision

DescriptionDescription Computer Computer GraphicsGraphics

AIAI

2 Principal application areas

Improvement of pictorial information for human interpretation.Processing of image data for storage, transmission, and representation for autonomous machine perception.

Examples of fields that use Digital Image Processing (DIP)

Categorize by image sources Radiation from the Electromagnetic spectrum Acoustic Ultrasonic Electronic (in the form of electron beams used

in electron microscopy) Computer (synthetic images used for

modeling and visualization)

Radiation from EM spectrum

EM waves = a stream of massless (proton) particles, each traveling in a wavelike pattern and moving at the speed of light.Spectral bands are grouped by energy per photon

Gamma rays, X-rays, Ultraviolet, Visible, Infrared, Microwaves, Radio waves

Electromagnetic SpectrumElectromagnetic Spectrum

Gamma-Ray ImagingGamma-Ray ImagingNuclear Image

(a) Bone scan (b) PET (Positron

emission tomography) image

Astronomical Observations.

(c) Cygnus LoopNuclear Reaction

(d) Gamma radiation from a reactor valve

aa bbcc dd

PET ScanPET ScanPositron Emission TomographyPositron Emission Tomography

X-ray Imaging

Medical diagnostics (a) chest X-ray (familiar) (b) aortic angiogram (c) head CT

Industrial imaging (d) Circuit board

Astronomy (e) Cygnus Loop

aadd

bbcc ee

Computerized Axial Computerized Axial TomographyTomography

AngiogramAngiogram

Imaging in Ultraviolet Band

LithographyIndustrial inspectionMicroscopy (fluorescence)

(a) Normal corn (b) Infected with smut corn

LasersBiological imagingAstronomical observations

(c) Cygnus Loop

aa bbcc

Fluorescence MicroscopyFluorescence Microscopy(Fluorspar mineral)(Fluorspar mineral)

Imaging in Visible andInfrared Bands

AstronomyLight microscopy

pharmaceuticals(a) taxol (anticancer agent) - 250x(b) cholesterol – 40x

Micro inspection for materials characterization

(c) Microprocessor – 60x (d) Nickel oxide thin film – 600x (e) Surface of audio CD – 1750x (f) Organic superconductor –

450x

aa bb ccdd ee ff

License Plate RecognitionLicense Plate Recognition

Thermal InfraredThermal Infrared

Imaging in Microwave BandImaging in Microwave BandImaging radar : the only way to explore inaccessible regions of the Earth’s surface (Tibet)Radar image of mountains in southeast TibetNote the clarity and detail of the image, unencumbered by clouds or other atmospheric conditions that normally interfere with images in the visual band.

Imaging in Radio BandImaging in Radio Band

MedicineMagnetic resonance image (MRI) : 2D picture of a section of the patient (any plane)

(a) knee (b) spine

Astronomy

Acoustic ImagingAcoustic Imaging

Geological applications : use sound in the low end of the sound spectrum (hundred of Hz)Mineral and oil exploration

Cross-sectional image of a seismic model.The arrow points to a hydrocarbon (oil and/or gas)

trap (bright spots)

Ultrasound ImagingUltrasound Imaging

ManufacturingMedicine (1540 m/s)

(a) Baby (b) Another view of

baby (c) Thyroids (d) Muscle layers

showing lesion

aa bbcc dd

Images generated by computerImages generated by computer

Fractals : an iterative reproduction of a basic pattern according to some mathematical rules

(a) and (b)

3-D computer modeling

(c) and (d)

aa bbcc dd

3 types of computerizedprocesses

Low-level : input, output are images Primitive operations such as image preprocessing to

reduce noise, contrast enhancement, and image sharpening

Mid-level : inputs may be images, outputs are attributes extracted from those images

Segmentation Description of objects Classification of individual objects

High-level : Image analysis

Fundamental stepsFundamental steps

Image Acquisition

An image is captured by a sensor (such as a monochrome or color TV camera) and digitized.If the output of the camera or sensor is not already in digital form, an analog-to digital converter digitizes it.

CameraCameraCamera consists of 2 parts

A lens that collects the appropriate type of radiation emitted from the object of interest and that forms an image of the real object

a semiconductor device –so called charged coupled device or CCD which converts the irradiance at the image plane into an electrical signal

Frame GrabberFrame Grabber

Frame grabber only needs circuits to digitize the electrical signal from the imaging sensor to store the image in the memory (RAM) of the computer.

Image EnhancementImage Enhancement

To bring out detail is obscured, or simply to highlight certain features of interest in an image

Image RestorationImage Restoration

Improving the appearance of an imageTend to be based on mathematical or probabilistic models of image degradation

Color Image ProcessingColor Image Processing

Gaining in importance because of the significant increase in the use of digital images over the Internet.Nowadays hardware is capable of realizing image processing in color.

WaveletsWavelets

Foundation for representing images in various degrees of resolution.Used in image data compression and pyramidal representation (images are subdivided successively into smaller regions)

CompressionCompression

Reducing the storage required to save an image or the bandwidth required to transmit it.Ex. JPEG (Joint Photographic Experts Group) image compression standard.

Morphological processingMorphological processing

Tools for extracting image components that are useful in the representation and description of shape.

Image SegmentationImage SegmentationComputer tries to separate objects from the image foreground from the image background.It is one of the most difficult tasks in DIP.A rugged segmentation procedure brings the process a long way toward successful solution of an image problem.Output of the segmentation stage is raw pixel data, constituting either the boundary of a region or all the points in the region itself.

Representation & Description (1)

Representation make a decision whether the data should be represented as a boundary or as a complete region. Boundary representation => focus on external

shape characteristics, such as corners and inflections.

Region representation => focus on internal properties, such as texture or skeleton shape.

Representation & Description (2)

Recognition & Interpretation

Recognition => the process that assigns a label to an object based on the information provided by its descriptors.Interpretation => assigning meaning to an ensemble of recognized objects.

Knowledge base

A problem domain detailing regions of an image where the information of interest is known to be located.Help to limit the search

Not all the processes areneeded. Ex. Postal Code Problem

Components of a General-Components of a General-Purpose Image Proc. SystemPurpose Image Proc. System

SensingSensing

Two elements are requiredTwo elements are required Physical DevicePhysical Device DigitizerDigitizer

Specialized Image Processing Specialized Image Processing HardwareHardware

DigitizerDigitizerDedicated hardware based on a ALU that Dedicated hardware based on a ALU that can perform pre-processing (e.g. can perform pre-processing (e.g. averaging for noise reduction)averaging for noise reduction)Acts as a front-end subsystemActs as a front-end subsystemDigitize at 30 frames/secondDigitize at 30 frames/second

ComputerComputer

Image processing systemImage processing systemCould range from a general purpose PC to Could range from a general purpose PC to a supercomputera supercomputer

Image Processing SoftwareImage Processing Software

Specialized modules that perform specific Specialized modules that perform specific taskstasksWell designed packages permit the user to Well designed packages permit the user to write code as well as integrating moduleswrite code as well as integrating modules

Mass StorageMass Storage

This is a must, a 1024 x 1024 grayscale This is a must, a 1024 x 1024 grayscale image requires 1Mbyte of storage alone image requires 1Mbyte of storage alone (uncompressed)(uncompressed)3 storage categories for image processing 3 storage categories for image processing applications:applications: Short-term storage (during processing)Short-term storage (during processing) On-line storage (fast recall)On-line storage (fast recall) Archival storage (infrequent access)Archival storage (infrequent access)

• Frame BuffersFrame Buffers

Image DisplaysImage Displays

Color flat screensColor flat screensIn many applications touch sensitive In many applications touch sensitive monitors are usedmonitors are used

HardcopyHardcopy

Laser printersLaser printersFilm camerasFilm camerasOptical Storage MediaOptical Storage MediaMagnetic Storage MediaMagnetic Storage Media