image processing, 2012

Sanjay Goel, JIIT, 2012

JIIT, Noida

Image processing (10B1NCI831) BTech, 4th year, 2012, JIIT

Lecture Notes and Assignments

1. Lect#1 (14.01.12) 1. Computational processing on data?

a. Store/Recovery b. Search/Retrieve/Pattern Matching/Traversal/Path finding c. Calculate/Process/Modify/Recoding/Transform/Compress/Translate/Estimate/

Optimise d. Measure/Control/Send/Receive e. Sort/Arrange/Rearrange/Classify/Cluster f. Generate//Scheduling/Layout g. Simulate/Synthesis/Render h. Generalize/Dissolve/Summarize/Merge

2. Image: Variety of data for processing a. Image content

i. Visual a) Bitonal b) Grey c) Colour

ii. X-ray iii. IR iv. Range v. UV

vi. MRI vii. …

b. Image set i. Single image

ii. Database of single images (same or different scenes) iii. Sequence of single images of same scene

a) Time sequence b) Progressive Panning sequence c) Progressive Zooming sequence

iv. Video v. Multiple co-scenic (fully/partially) single images (homogenous image

content) vi. Multiple co-scenic (fully/partially) image sequences (homogenous image

content) vii. Multiple co-scenic (fully/partially) videos (homogenous image content)

viii. Multiple co-scenic (fully/partially) images/video (heterogeneous image content)

3. Assignment: a. Explore the possibilities of Image Processing in the domain of Cultural Heritage and

Entertainment


JIIT, Noida

2. Lect#2,3 2hr. (18.01.12)

1. Possibilities of Image Processing in the domain of Cultural Heritage and Entertainment 2. Colour perception 3. RGB and CMYK colour models 4. Assignment:

a. WAP to convert a gray label image as text file. b. Explore the colour vision power of different species. c. Learn to use Kinect to capture range images

3. Lect #4 (21.01.12)

1. RGB and CMYK colour models 2. HLS, HSV, YIQ, YUV, YCrCb colour models 3. Assignment:

• WAP to convert RGB image into any two other colour models.

4. Lect #5,6 2 hrs (25.01.12) 1. ppi, dpi, 2. Continuous tone, half tone 3. Screening, Threshold, Tiled threshold, Random modulation, 4. Error diffusion

a. Floyd Steinberg and other diffusion filters 5. Assignment:

a. Create a database of range images of simple objects. b. WAP to display the three channels of images stored in different colour models. c. WAP to convert continuous tone images into halftone images.

5. Lect #7 1 hrs (28.01.12)

1. Clustered dot, Dispersed dot, Beyer’s recursive approach for dispersed dot 2. Colour quantization

6. Lect #8 1 hrs (31.01.12) (extra class)

1. Colour Quantisation a. Using clustering algorithms b. Octree method

2. Geometric Transformations on raster images - Translation, Scaling, Reflection, Shear, Rotation - Non-linear transformations - Splitting- Shooting Algorithm

3. Image Warping a. User inputs for control points

4. Assignment: i. WAP to show the effect of some geometric transformations on images.


JIIT, Noida

7. Lect #9,10 2 hrs (01.02.12)

1. Computational Models (perspectives) of image for

- input image - desired output image - object of interest within a given image

a. Image as a histogram b. Image as collection of primitive structures (morphological Image processing) c. Image as a multi-dimensional signal

i. Image as a matrix ii. Image as a frequency spectrum over a set of 2d basis functions/signals

♦ Fourier ♦ Cosine ♦ Sine… ♦ Wavelets

d. Image as a discrete surface (applying the tools of partial differential equations, differential geometry)

e. Image as a Markov random field (applying the tools of stochastic modeling and analysis).

2. Histogram based image processing techniques

a. Grey level Transfer Functions i. Thresholding

ii. Grey level slicing iii. Inverse iv. Power-law transformations v. Log transformations

vi. Piecewise linear Contrast stretching

8. Lect #11 (04.02.12)

1. Histogram based image processing techniques a. Grey level Transfer Functions

ii. Power-law transformations iii. Log transformations iv. Histogram equalization v. Histogram matching/specification

2. Assignment: i. Consider extending the Grey level Transfer Functions to 3d colour

images.

9. Lect #12 (07.02.12)

1. Histogram based image processing techniques a) Adaptive Thresholding b) Extension of Grey level Transfer Functions to 3d colour images.

i. Color slicing ii. Colorization


JIIT, Noida

10. Lect #13,14 2 hrs (08.02.12)

1. Histogram based image processing techniques

a) Local Histogram Equalisation b) Pseudo coloring c) Colorization of Gray level images, Color transfer of color images

1. For every pixel in the source image: Map the L value to target image’s L* value, search for best match of L* in target image and import chroma (a and b values) to source image’s pixel.

a. Map through histogram matching/specification b. Map through statistical matching

2. Matrix based image processing techniques a) Matrix Addition

1. Superimposition/blending of two or more images 2. Noise removal by averaging multiple images of same scene.

b) Matrix Subtraction 1. Background removal, 2. Object detection, surveillance

c) Matrix Division 1. Image Ratioing (in remote sensing)

d) Eigen vector and Eigen values 1. Eigen images (e.g. Eigen faces)

e) Image Composition 1. Luma Keying 2. Chroma Keying

3. Assignment:

i. WAP for colorization of Gray image/video wrt supplied reference image.

11. Lect #15 (14.02.12)

1. Spatial signal filtering based image processing techniques: i. Linear and nonlinear spatial filters

a) Convolution a. 1d signal b. 2d signal

ii. Noise removal by averaging filter (Linear low pass filter): 1d 2d a) Constant weight ( 1 1 1, 1 1 1, 1 1 1) b) Gaussian weight

iii. Edge detection by first difference operators (Linear high pass filter): 1d 2d a) Delta f (x) = f(x+1) – f (x)

a. Horizontal edge 1 -1

b. Vertical edge [-1 1] c. Diagonal edge (Roberts Cross operator)

1 0 and 0 1 0 -1 -1 0


JIIT, Noida

12. Lect#16,17 2hrs. (15.02.12)

1. Spatial signal filtering based image processing techniques:

i. Edge detection by difference operators (Linear high pass filter): 1d 2d a) Roberts edge detector using Roberts Cross operator

a. G = Sqrt (Gx^2 + Gv^2) b) Prewitt filter

a. Delta f (x) = [f(x+1) – f (x-1)]/2 b. [-1 0 1] (-1 0 1, -1 0 1, -1 0 1)

c) Sobel filter a. Higher weights assigned to central row in Prewitt filter

i. (-1 0 1, -2 0 2, -1 0 1) and its transpose d) Kirsh operator

a. (3 3 3, 3 0 3, -5 -5 -5) and other its 7 rotations. ii. Line detection (single pixel wide)

a) (-1 2 -1, -1 2 -1, -1 2 -1) iii. High pass filtered image = image – low pass filtered image

a) High pass filter = all pass filter – low pass filter a. All pass filter = (0 0 0, 0 1 0, 0 0 0) b. Using Constant weight (1 1 1, 1 1 1, 1 1 1) for low pass filter

i. -1 -1 -1 -1 8 -1 -1 -1 -1 {This is known as Laplacian filter}

c. Using Gaussian weight for low pass filter iv. Band pass filtered image =

Low pass filtered image [small window] – Low pass filtered image [large window]

v. Applications of Correlation a) Pattern matching applications

2. Assignment:

i. Program above filters and test with some real images. ii. WAP a simple OCR for any Indian language using correlation.

13. Lect#18 (21.02.12)

1. Spatial signal filtering based image processing techniques:

i. Applications of Amplitude modulation a) Stagnography b) Spatial Watermarking

a. Visible Watermarking i. Weighted superimposition

b. Invisible Watermarking i. LSB method

ii. Pixel Surrounding method

14. Lect#19,20 2 hrs (22.02.12)

1. Spatial signal filtering based image processing techniques: i. Median filter for noise removal


JIIT, Noida

ii. Image compressions techniques in spatial domain: a) DPCM b) Truncation

a. Spatial resolution truncation (down sampling) b. Colour resolution truncation

c) Huffman encoding d) RLE

iii. Covariance and normalized correlation for pattern matching.

3. Assignment: i. Define and get approval on the deliverables of your mini project in this course.

15. Lect#21 (13.03.12)

1. Set (Structure) based image processing techniques: (Morphological Image Processing)

i. Dilation ii. Erosion

iii. Opening iv. Closing

- Noise removal 2. Assignment: WAP to demonstrate the four basic Morphological operations

16. Lect#22,23 2 hrs (14.03.12)

1. Set (Structure) based image processing techniques:

(Morphological Image Processing) • Mathematical definitions

i. Dilation ii. Erosion

• Applications iii. Boundary detection iv. Gradient v. Region filling

17. Lect#24 (20.03.12)

1. Project problem presentations by groups 2. Set (Structure) based image processing techniques:

(Morphological Image Processing) • Some more discussion on Opening and Closing

- Open as a filter - Open as union of translated B’s - Open(Open (A, B), D) = Open(Open (A, D), B) /*associative property - Idempotent property of Open and close - Open(A,B) < Erosion (A.B) < A< Close(A,B) < Dilation(A,B) (if pivot

is within B) - If D is B open, i.e., Open (D,B) = D

then Open(Open (A, B), D) = Open (A,D)


JIIT, Noida

18. Lect#25 (27.03.12)

1. Set (Structure) based image processing techniques: (Morphological Image Processing) • Duality property wrt Erosion, Dilation, Open and Close

2. Assignment: • Rewrite your programs for Erosion, Dilation, Open and Close using and verify the

Duality property.

19. Lect#26-27 (28.03.12)


- Properties of Dilation, Erosion, • Associativity • Distributivity/antidistributivity • Translation invariant

- Additional properties of open/close • Extensivity/antiextensivity • Idemopotent

- Connected component labeling (check the impact of SE) - τ - opening (union of Image openings with multiple SE’s)

• Key issue – designing SE’s - Hit or Miss Transform

• Key issue – designing SE’s • Pattern matching • OCR

20. Lect#28 (03.04.12) 1. Set (Structure) based image processing techniques:

(Morphological Image Processing) - Erosion with SE’s with pivot = 0 - Hit or Miss Transform

• OCR (retaining selected 1’s) • Corner filling (Converting 0 1)

2. Assignment: WAP a simple OCR for any language using HTM

21. Lect#29-30 (04.04.12) 1. Set (Structure) based image processing techniques:

(Morphological Image Processing) - Hit or Miss Transform

• Converting 0 1 a. Single pixel corner/single pixel intrusion filling

i. Sequential HTM with different SE’s + Union with earlier Image

b. Convex Hull i. Iterated HTM + Union with earlier Image

ii. Application in Computer vision


JIIT, Noida

• Converting 1 0 a. Thinning

22. Lect#31 (17.04.12)

1. General discussion about Image processing difficulties and challenges faced by students

in their major, minor, or mini projects or any other work. e.g. internship etc.

23. Lect#32-33 (18.04.12)

1. Project review (next review scheduled in 2nd week of May) 2. Set (Structure) based image processing techniques:

(Morphological Image Processing) - Hit or Miss Transform

• Converting 1 0 a. Thinning b. Pruning

3. WAP to thin + prune the given binary images. Test with text images etc.


(Morphological Image Processing) - Lantuejoul’s formula for Skeletonisation

• S(A) = Union (S(A,k)) : k = 1..M /* union of Sub-skeletons • S(A,k) = Erosion (A,kB) – Open (Erosion (A,kB), B) • M = max{k | Erosion(A,kB) is not φ} • S(A) with this method may be disconnected

25. Lect#35-36 (25.04.12)


- Reconstruction using sub-skeletons • A = Union (Dilate (S(A,k), kB)) : k = 1..M • Compression

- Segmentation • Top Hat Transformations

a. White Top Hat (= A-Open(A,B)) b. Black Top Hat (= Close (A,B) – A)

• Texture segmentation and classification a. Granulometrics (Ref: Luc Vincent and Edward R.

Dougherty, Morphological Segmentation for Textures and Particles, 1994)


(Morphological Image Processing) - Distance function

• Dist (A,p) = min {k | p is not in Erosion (A, kB} - Ultimate Erosion - Connected skeleton extraction using local maxima of distance function


JIIT, Noida

- Skeleton by Influence Zone (SKIZ)

27. Lect#38-39 (02.05.12) 1. Set (Structure) based image processing techniques:

(Morphological Image Processing) - Recursive Transforms

• Ref: Haralick et al, Recursive Opening Transform, 1991, IEEE • Ref: Chen and Haralick, Recursive Erosion, Dilation, Opening, and

Closing Transforms, 1995, IEEE • Recursive Erosion Transform (RET) • Recursive Dilation Transform (RDT) • Recursive Opening Transform (ROT) • Recursive Closing Transform (RCT) • Application in Document Layout Analysis

a. Automated Skew Detection o Ref: Chen and Haralick, Automated Skew Estmation

in Document Images, 1996, IEEE; o Ref: Najman, Using Mathematical Morphology for

Document Skew Estimation

28. Lect#40 (08.05.12) 1. Student project presentations

i. Crack filling in painting images (Shreya, Akshay) ii. Sketch transformation of sculpture images (Jasmeet, Soniya, Pranjul)

iii. Sketch transformation of monument images (Saransh, Abhijit) iv. Animating paintings (Shivani, Ujjwal, Abhinandan) v. 3d Model from multiple Kinect’s range images (Siddharth, Sameer, Abhinav,

Vikrant) vi. Finger counter (Saurabh, Varun, Himanshu, Sachin)

2. Project deliverables i. Demonstration

ii. Video record of demonstration iii. Report (hard copy and soft copy)

29. Lect#41 (15.05.12)

1. Computer Vision i. Shape measure: Feature vector (signature)

a) Major axis: length, angle b) Minor axis: length, angle c) Ratio of major axis length to minor axis length d) Perimeter e) Area f) Ratio of area to perimeter

a. Roundedness = 4pi x area/(perimeter)2 g) Bounding box area h) Number of holes i) Hole area j) Ratio of hole area to total object area k) Number of corners l) Relative position of corners


JIIT, Noida

ii. Evaluation criteria of shape measures a) Distinguishing (identification) power b) Computation speed c) Invariance /Tolerance to

a. Translation b. Rotation c. Scale d. Minor defect / variations in the boundary e. Illumination f. Partial occlusion

2. Student project presentations i. Triangulation (Khushboo, Anshika)

3. Assignment: Propose opportunities for Web based computer vision applications.

30. Lect#42-43 (16.05.12)

1. Computer Vision i. Shape measure: Feature vector (signature)

a) Boundary based signatures and their evaluation wrt above criteria a. Spatial domain vectors (for curve matching)

i. Explicit list of points ii. Chain code

iii. Relative chain code iv. Fixed length line segments (relative angles) v. Variable length line segments (length, relative angles)

vi. Radial scan (angle, distance of boundary point from object’s centre)

vii. Curvature b. Frequency domain vectors

i. First five Fourier/Cosine coefficient of any of the above spatial domain vectors

ii. Normalised Fourier/Cosine coefficient of any of the above spatial domain vectors

b) Projections a. Horizontal b. Vertical c. Radial

c) Moments (Grey level dependant signatures) (Refer Hu’s work) ----------- Good Luck -----------

image processing, 2012

Technology