image processing - ucl computer science · gv12/3072 image processing. 57 dark current •ccds...
TRANSCRIPT
GV12/3072
Image Processing.
1
Image Processing
Gabriel Brostow
GV12/3072
Image Processing.
2
GV12/3072
Image Processing.
3
Motivation and Goals
• Grounding in image processing techniques
• Concentrate on algorithms useful for
machine vision
• Best sensors ever!
GV12/3072
Image Processing.
4
Motivation and Goals
• Grounding in image processing techniques
• Concentrate on algorithms useful for
machine vision
• Best sensors ever! With a few problems…
Transmission interference
5
Compression artefacts
6
Spilling
7
Scratches, Sensor noise
8
Bad contrast
9
Resolution Super resolution?
10
Removing motion blur
Original image
Cropped subwindow
After motion blur removal
[Images from Amit Agrawal]12
Removing motion blur
13
14
Removing motion blur
15
GV12/3072
Image Processing.
16
Syllabus
1. The digital image
2. Image segmentation
3. Image transformations (Simon Prince)
4. Morphological operations
5. Feature characterization
GV12/3072
Image Processing.
17
Features and Object Recognition
GV12/3072
Image Processing.
18
Syllabus (GV12/3072)
6. Image Filtering
7. Edge detection
8. Corner detection
9. Color images
10.Template matching
GV12/3072
Image Processing.
19
Why Now?
• Why are these sectors paying more attention?
•Medicine•Automatic or assisted diagnosis
•Image-guided surgery
•Agriculture
•Film and television
•Surveillance and police work
•Military
•http://www.cs.ubc.ca/spider/lowe/vision.html
GV12/3072
Image Processing.
20
Course content
• Lots of material!
• Some mathematics• Calculus
• Geometry and matrix algebra
• Probability and statistics
• Some programming – Matlab.
GV12/3072
Image Processing.
21
Assessment
• Four Courseworks 20%.
• Implement and test algorithms in matlab
• Exam 80%
• Honor System
GV12/3072
Image Processing.
22
Lectures and notes• Monday 09:00-10:00 (1.20)
• Monday 11:00-13:00 (Rockefeller 337 David Sacks)
• Lab sessions
– Mon 10-11 (1.05)
– Tue 14-15 (4.06)
– Wed 11-12 (1.05)
• Web page coming soon!
• subject: “join” to [email protected]
GV12/3072
Image Processing.
23
GV12/3072
Image Processing.
26
Unassessed CW Assignment
• Matlab introduction
• Start matlab:
% matlab or
% /opt/matlab7/bin/matlab
• Download any simple image
• Load it into matlab:
>> I = imread(‘foo.jpg’);
GV12/3072
Image Processing.
27
Unassessed CW Assignment
• Display the image in Matlab:
>> imshow(I);
• Print the image data array:
>> I
• Print the size of the image array and create a subimage:
>> size( I )
>> Isubwindow = I(72:92, 62:82);
>> imshow(Isubwindow);
GV12/3072
Image Processing.
28
Unassessed CW Assignment
• Start the Matlab help tool (Help menu).
• In the “Contents” pane to the left of the
window. Click on MATLAB.
• Go through the “Getting Started” section.
• Continue to the “Using MATLAB” section
when you have time.
GV12/3072
Image Processing.
29
GV12/3072
Image Processing.
30
The Digital Image
GV12/3072
Image Processing.
31
Outline
• What is an image?
• What is a pixel?
• How do we store them?
GV12/3072
Image Processing.
32
What is an image?
Image as 2D signal
• Signal: function depending on some variable with physical meaning
• Image: continuous function2 variables: xy - coordinates3 variables: xy + time (video)
• Brightness is usually the value of the function
• But can be other physical values too: temperature, pressure, depth …
33
Example 2d images
ultrasound temperature
camera image CT34
GV12/3072
Image Processing.
39
Image?
>> t=rand(256,256);
>> imshow(t)
What is a pix-el?
x
y
(x,y)
f(x,y)
(0,0)
42
Not a little square!
• A Pixel Is Not A Little
Square, A Pixel Is Not A
Little Square, A Pixel Is
Not A Little Square! (And
a Voxel is Not a Little
Cube),
– Alvy Ray Smith,
MS Tech Memo 6, Jul 17, 1995
GV12/3072
Image Processing.
43
Not a little square!
Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995
GV12/3072
Image Processing.
44
Gaussian reconstruction filter
Not a little square!
Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995
GV12/3072
Image Processing.
45
Cubic reconstruction filter
Not a little square!
Graphics: Dick Lyon, 2006
GV12/3072
Image Processing.
46
GV12/3072
Image Processing.
47
Where do images come from?
• Digital cameras
• MRI scanners
• Computer graphics packages
• Body scanners
• Laser range finders
• Many more…
GV12/3072
Image Processing.
48
Where do images come from?
• Digital cameras
• MRI scanners
• Computer graphics packages
• Body scanners
• Laser range finders
• Many more…
GV12/3072
Image Processing.
49
The digital camera
• A Charge Coupled Device (CCD).
Lens
Sensor
array
ADC
Image array
GV12/3072
Image Processing.
50From: Lecture Notes – EAAE
and/or Science “Nuggets” 2000
Capturing photons
GV12/3072
Image Processing.
51
http://www.astro.virginia.edu/class/oconnell/astr121/im/CCD-fullframearc-FSU.jpg
GV12/3072
Image Processing.
53
The sensor array
• Can be < 1cm2.
• An array of photosites.
• Each photosite is a bucket of electrical charge.
• They contain charge proportional to the incident light intensity during exposure.
GV12/3072
Image Processing.
54
Analog to Digital Conversion
• The ADC measures the
charge and digitizes the
result.
• Conversion happens line
by line.
• The charges in each
photosite move down
through the sensor array. ADC
RAM
GV12/3072
Image Processing.
55
ADC
RAM
GV12/3072
Image Processing.
56
Blooming
• The buckets have finite capacity
• Photosite saturation causes blooming
GV12/3072
Image Processing.
57
Dark Current
• CCDs produce thermally-generated charge.
• They give non-zero output even in darkness.
• Partly, this is the dark current.
• Fluctuates randomly.
• How can we reduce dark current? From: Lecture Notes - EAAE
Yohkoh satellite, 9 years apart ..
GV12/3072
Image Processing.
58
Dark Current
• CCDs produce thermally-generated charge.
• They give non-zero output even in darkness.
• Partly, this is the dark current.
• Fluctuates randomly.
• How can we reduce dark current? From: Lecture Notes - EAAE
GV12/3072
Image Processing.
59
GV12/3072
Image Processing.
62
Sampling 1D
Sampling in 1D takes a function, and returns a vector whose elements are
values of that function at the sample points.
63
Sampling 2D
Sampling in 2D takes a function and
returns an array; we allow the
array to be infinite dimensional and to
have negative as well as positive indices.
64
GV12/3072
Image Processing.
65
Greyscale digital image
Nyquist Frequency
• Half the sampling frequency of a discrete
signal processing system
GV12/3072
Image Processing.
66
Sampling grids
67
Retina-like sensors
68
Quantization
• Real valued function will get digital values –
integer values
• Quantization is lossy!!
– After quantization, the original signal cannot be
reconstructed anymore
• This is in contrast to sampling, as a sampled but
not quantized signal can be reconstructed.
• Simple quantization uses equally spaced levels
with k intervals
bk 269
Quantization
00
01
10
11
70
Quantization
00
01
10
11
71
Usual quantization intervals
• Grayvalue image
8 bit = 2^8 = 256 grayvalues
• Color image RGB (3 channels)
8 bit/channel = 2^24 = 16.7Mio colors
• 12bit or 16bit from some sensors
• Nonlinear, for example log-scale
72
GV12/3072
Image Processing.
73Photo: Paulo Barcellos Jr.
Winding Down
GV12/3072
Image Processing.
74
Properties
• Image resolution
• Geometric resolution: How many pixel per area
• Radiometric resolution: How many bits per pixel
75
Image resolution
1024x1024
512x512
512x1024
76
GV12/3072
Image Processing.
77
Geometric resolution
144x144 72x72 36x36
18x18 9x9 4x4
GV12/3072
Image Processing.
78
Radiometric resolution
256 128 64 32
16 8 4 2
Lossless vs. Lossy
• Name some formats?
GV12/3072
Image Processing.
79
GV12/3072
Image Processing.
80
Aliasing and SNR
• What is the disadvantage of low sampling
resolution?
• What is the disadvantage of high sampling
resolution?
Finish
Next week:
Image segmentation
GV12/3072
Image Processing.
81