Digital Image Processing

and Machine Vision

Fundamentals

By Dr. Rajeev Srivastava Associate Professor

Dept. of Computer Sc. & Engineering, IIT(BHU), Varanasi

Overview

• In early days of computing, data was numerical.

• Later, textual data became more common.

• Today, many other forms of data such as voice, music, speech, images, computer graphics, etc.

• Each of these types of data are signals.

• Loosely defined, a signal is a function that

conveys information.

8/3/2014 2

Relationship of Signal Processing

to other fields

• As long as people have tried to send or receive through electronic media : telegraphs, radar, telephones, television etc there has been the realization that these signals may be affected by the system used to acquire, transmit, or process them.

• Sometimes, these systems are imperfect and introduce noise, distortion, or other artifacts.

8/3/2014 3

• Understanding the effects these systems have and finding ways to correct them is the fundamental of signal processing.

• Sometimes, these signals are specific messages that we create and send to someone else (e.g., telegraph, telephone, television, digital networking, etc.).

• That is, we specifically introduce the information content into the signal and hope to extract it out later.

8/3/2014 4

• Sometimes, these man-made signals are encoding of natural phenomena (audio signal, acquired image, etc.),but sometimes we can create them from scratch (speech generation, computer generated music, computer graphics).

• Finally, we can sometimes merge these technologies together by acquiring a natural signal, processing it, and then transmitting it in some fashion.

8/3/2014 5

Concerned fields of Signal

Processing

• Digital Communication

• Compression

• Speech Synthesis and Recognition

• Computer Graphics

• Image Processing

• Computer Vision

8/3/2014 6

What is an image?

• A representation, likeness, or imitation of an object or thing

• A vivid or graphic description

8/3/2014 7

Why do we need images?

8/3/2014 8

• Various imaging modalities help us to see invisible objects due to

-Opaqueness (e.g., see through human body) -Far distance (e.g., remote sensing) -Small size (e.g., light microscopy)

• Other signals (e.g., seismic) can also be translated into images to facilitate the analysis • Images are important to convey information and support reasoning

A picture is worth a thousand words!

Fields related to images

• Computer Graphics

• Image Processing

• Computer Vision

8/3/2014 9

Input /Output Image Description

Image

Image Processing

Computer Vision

Description

Computer Graphics

AI

Computer Graphics

Computer Graphics deals with generation of 2D computer images from the descriptions of real time 3D object or data.

Computer graphics deals with designing suitable 2D scene images to simulate our 3D world.

8/3/2014 10

Image Processing

Image processing is the manipulation of an image for the purpose of either extracting information from the image or producing an alternative representation of the image.

Image processing is a subclass of signal processing concerned specifically with pictures.

Improve image quality for human perception and/or computer interpretation.

8/3/2014 11

Computer Vision

Computer vision is related to the construction of the 3D world from the observed 2D images.

Computer vision deals with the analysis of image content.

Computer graphics pursues the opposite direction in designing suitable 2D scene images to simulate our 3D world.

Image processing can be considered as the crucial middle way connecting the vision and graphics fields.

8/3/2014 12

Computer Vision

• Computer Vision components:

Image Processing

Image Analysis

Image Understanding

8/3/2014 13

Process Input Output

Image Processing Image Analysis Image Understanding

Image Image Image

Image Measurements High-level description

Related Fields

8/3/2014 14

Signal Processing

Physics

Imaging

NeuroBiology Mathematics

Machine Learning

Artificial Intelligence

Control Robotics

Computer Vision

Image Processing

Machine Vision

DIGITAL IMAGE

8/3/2014 15

Satellite image

Volcano Kamchatka Peninsula, Russia

8/3/2014 16

Satellite image

Volcano in Alaska

8/3/2014 17

Medical Images:

MRI of normal brain

8/3/2014 18

Medical Images:

X-ray knee

8/3/2014 19

Ultrasound: Five-month Foetus

8/3/2014 20

Astronomical images

8/3/2014 21

Radiation from the Electromagnetic spectrum

Acoustic

Ultrasonic

Electronic (in the form of electron beams used in electron microscopy)

Computer (synthetic images used for modelling and visualization)

8/3/2014 22

Categorization by Image

Sources

Radiation from EM Spectrum

8/3/2014 23

Electromagnetic Spectrum

8/3/2014 24

X-ray Imaging

Imaging in Ultraviolet Band

Imaging in Visible and

Infrared Bands

Imaging in Microwave Band

Imaging in Radio Band

Other Imaging Modalities that uses neither of energy bands from EM radiation

Acoustic Imaging :Use of sound waves to capture images of an object.

Examples: Geographical Explorations, Ultrasound Imaging.

Computer generated images

8/3/2014 30

Acoustic Imaging

Ultrasound Imaging

Generated Images by

Computer

Application

8/3/2014 37

Image Acquisition

• An image is captured by a sensor such as

a monochrome or color TV camera and

digitized.

• If the output of the camera or sensor is

not already in digital form, an analog-to-digital converter digitizes it.

8/3/2014 40

Digital Image Acquisition Process (Ref: R.C. Gonzalez)

8/3/2014 41

FROM ANALOG TO DIGITAL

8/3/2014 42

Record

output Display

Object Observe

Imaging

systems

Digitize

Sample and

quantize

Store

Digital

storage

(disk)

Process

Digital

computer

Refresh

/store

On-line

buffer

Digital Image Representation

8/3/2014 43

•Images are 2D signals represented in 2D matrix form.

•Each element of matrix is known as pixels.

•Each pixel is associated with some integer/real value that represents intensity or color at that point.

Types of images

Binary Images

Gray scale images

Indexed color images

RGB color images

Multispectral images 8/3/2014 44

Binary Image

8/3/2014 45

Each Pixel have either 0 or 1 value. 0white 1black

Intensity (Gray-Level) Image

8/3/2014 46

•Each pixel is associated with 8-bits of Intensity. •A pixel may attain any value in between 0 to 255.

Coloured Image

8/3/2014 47

Indexed Color Images

8/3/2014 48

Typically 256 colors (GIF-format)

RGB Color Images

8/3/2014 49

Red, green and blue channels, typically 256 levels each: 2(3*8) = 16777216 colors. (e.g. TIF and JPEG formats)

General Purpose Image Processing

System

8/3/2014 50

Computer

Mass Storage

Image Processing Software

Image Sensors

Specialized Image

Processing Hardware

Hardcopy

Image Displays

Problem Domain

Hardware Required

Camera

Hardware Required

Frame Grabber

Image Processing

• Manipulation of multidimensional signals

image (photo)

video

CT, MRI

),( yxf

),,( tyxf

),,,( tzyxf

Image Processing

Why it is needed?

For:

– Coding/compression

– Enhancement, restoration, reconstruction

– Analysis, detection, recognition, understanding

– Visualization

8/3/2014 54

Image Processing Tasks

Enhancement

Restoration

Edge Detection

Segmentation

Compression

Object Description etc.

8/3/2014 55

Image Enhancement

Another example : MRI Power Law Transformation: CRγ (Ref: R.C. Gonzalez)

i. A magnetic resonance image of an upper thoracic human spine with a fracture dislocation and spinal cord impingement – The picture is predominately dark – An expansion of gray levels are

desirable needs < 1 ii. Result after power-law transformation with = 0.6, c=1 iii. Transformation with = 0.4 (best result) iv. Transformation with = 0.3 (under acceptable level)

Image Restoration

Distorted Image Restored Image

Types of Distortion in Images: Distortion due to Camera Misfocus

8/3/2014 59

Original image Distorted image

Distortion due to motion

8/3/2014 60

Camera lens

Distortion due to Random

Noise

8/3/2014 61

Distorted image Original image

Types of noises in an image

8/3/2014 62

Applications: Image

Inpainting

8/3/2014 63

Image Inpainting

8/3/2014 64

Applications: Reduction of Speckle Noise From Ultrasound Images

8/3/2014 65

Applications: Reduction of Speckle Noise From Remote Sensing SAR Images

8/3/2014 66

Applications: Restoration and Enhancement of Microscopic Images

8/3/2014 67

Image Segmentation

Image Segmentation

8/3/2014 69

Applications of Segmentation

8/3/2014 70

Where do we require it ?

Extraction of the desired object/constituent from an image

For volumetric analysis in MRI images for early detection of diseases like Alzheimer’s, Parkinson’s disease and Schizophrenia.

Motion tracking : e.g. finding the speed of an aeroplane.

Detection or identification of objects in an image in the field of Robotics

Automatic car assembly in robotic vision

Various Techniques used

Edge Detection

Active Contour Snake model

Registration and Masking in images

Color Image Processing

Compression

Image Compression

Signal-Processing Based:

Encoder

8/3/2014 73

),( yxf

H ),( yxg Compressed

Representation

Decoder

1H

),(ˆ yxf),( yxg

Morphological Processing

Representation and

Description

Representation and

Description

Recognition and Description

Knowledge Base

Ex: Postal Code Problem

Not all the processes are needed

Computer Vision System:

Framework

8/3/2014 80

Image Preproce

-ssing

Input Image

Feature Extraction

Segmentn

Feature Extraction

Symbolic Reprstn

Interpretation and

description

Classification and

description

Image Analysis System

Image Understanding System

Data Analysis Conclusion from Analysis

8/3/2014 81

Application Areas of

Computer Vision

• Controlling Processes : An industrial robot or an autonomous vehicle

• Detecting Events: For Visual Surveillance or people counting

• Organizing Information: For indexing databases of images and image sequences

• Modeling objects or environments: Industrial Inspection ( Detection in circuit of PCB layout) and Medical Image

Analysis • Interaction : As the input to a device for computer human

interaction • 3D- shape modeling

8/3/2014 82

Digital Image processing and Machine Vision: Applications

8/3/2014 83

RADAR imaging

Gray level image

processing

Medical Imaging

Biometrics (Forensic science)

Remote sensing

Copy right protection-

Digital water marking

Digital holography

Image Compression

DIP and Machine

Vision

And many more: object recognition, 3D vision, Robotics, Industrial Inspection etc.

Areas of applications

Other application Areas of Image

Processing and Vision

Image Registration

Optical Coherence Tomography

Remote Sensing and Agriculture

Astronomy

Digital Watermarking

Microscopic image processing of biological samples

Scientific Visualization

8/3/2014 84

Applications: Medical Image Registration Initial Condition of MR-PET Registration

8/3/2014 85

Final Configuration for MR-PET Registration

8/3/2014 86

Applications of Registration

• Register 2 MRIs of brain to visualize anatomy and tumor

8/3/2014 87

Applications of Registration…

• Create at 3-D model for surgical planning and visualization

Tumor(green), Vessels(red), Ventricles(blue), Edema (orange)

8/3/2014 88

Segmentation Applications in Medical Imaging: Semi-Automated Detection of

Alzheimer’s Disease using segmentation (Active-Contour Model)

8/3/2014 89

Coronal view

Axial view

Saggital view

Framework for snakes

• A higher level process or a user initializes any curve close to the

object boundary.

• The snake then starts deforming and moving towards the

desired object boundary.

• In the end it completely “shrink-wraps” around the object.

8/3/2014 90

ALZHEIMER’S DISEASE (AD)

Most Common cause of Dementia

Characterized by progressive cognitive deterioration

Can be diagnosed accurately only after Biopsy

Latest research : AD leads to atrophy of Hippocamus and Corpus Callosum of the Brain

Corpus Callosum

8/3/2014 91

SEGMENTED HIPPOCAMPUS

HIPPOCAMPUS IN CORONAL SLICE 8/3/2014 92

SEGMENTED CORPUS CALLOSUM

CORPUS CALLOSUM IN SAGGITAL SLICE

8/3/2014 93

Digital Watermarking

Host Image (Top Left), Watermark (Top right), Visible Watermarked image (Bottom Left), and Invisible watermark (Bottom Right)

8/3/2014 94

Applications of Computer Vision in Space

8/3/2014 95

Remote Sensing and Vision in Agriculture

8/3/2014 96

Remote Sensing and Vision in Agriculture

8/3/2014 97

References:

[BOOK]Digital Image Processing By R.C. Gonzalez