anatomy skills image processing fundamentals - … 1 aaron ponti anatomy skills image processing...

Click here to load reader

Post on 09-Apr-2018

219 views

Category:

Documents

2 download

Embed Size (px)

TRANSCRIPT

  • 2/24/2012

    1

    Aaron Ponti

    Anatomy SkillsImage processing fundamentals

    Image processing and analysis circle

    Image processing fundamentals-- Definitions

    Definitions Digital image

    Image resolution Grayscale resolution Spatial resolution

    Image data types Binary (1 bit) Intensity (8 bit, gray) Indexed (8 bit, color) RGB (24 bit, color)

    Color spaces

  • 2/24/2012

    2

    Definitions Digital image

    Image resolution Grayscale resolution Spatial resolution

    Image data types Binary (1 bit) Intensity (8 bit, gray) Indexed (8 bit, color) RGB (24 bit, color)

    Color spaces

    Digital image A digital image is a discrete function defined over a rectangular grid

    (lattice) representing the characteristics of the objects being imaged.

    Sampling (digitization): example in 1D.

    sample

    continuous (analog) signal

    sampling interval

    Digital image Digital image

    In 2D images, each grid element, or pixel(picture element), is defined as a location and a value representing the characteristic of the object in that location.

    In 3D images, the pixel is called voxel(volume element).

    A 3D image is just a stack of 2D images.

  • 2/24/2012

    3

    Definitions Digital image

    Image resolution Grayscale resolution Spatial resolution

    Image data types Binary (1 bit) Intensity (8 bit, gray) Indexed (8 bit, color) RGB (24 bit, color)

    Color spaces

    Image resolution The resolution of an image is a measure of the fidelity of the representation

    of the original scene.

    Resolution is related firstly to the characteristics of the imaging system and secondly to the number of pixels (i.e. spatial resolution) and the range of brightness values (i.e. grayscale resolution) that are used for digitization.

    The grayscale resolution of an image is expressed as its bit depth.

    The maximum number of brightness (i.e. gray) levels in an n-bit image is 2n.

    Image resolution ::Grayscale resolution

    Image resolution ::Grayscale resolution

  • 2/24/2012

    4

    Image resolution ::Spatial resolution Image resolution summary

    Definitions Digital image

    Image resolution Grayscale resolution Spatial resolution

    Image data types Binary (1 bit) Intensity (8 bit, gray) Indexed (8 bit, color) RGB (24 bit, color)

    Color spaces

    Image data types ::Binary

    2 gray values: black (0) and white (1)

    Binary values (0 or 1)

    1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 1 1 1 11 1 1 1 0 0 0 1 1 1 11 1 1 1 1 1 0 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1

  • 2/24/2012

    5

    178 181 182 182 183 183 186 181 185 185 186 185 186 178 182 182 183 184 183 183 182 182 182 188 184 186 181 182 184 185 184 181 183 185 187 184 183 189 186 186 186 184 187 185 183 189 185 184 186 190 187 186 185 182 183 189 185 187 187 186 187 187 188 190 185 184 183 185 185 187 185 189 186 189 189 189 193 193 183 184 187 182 184 187 189 185 190 188 193 199 198 183 182 181 186 190 189 195 191 187 162 156 116 30 185 184 191 192 209 195 198 168 102 49 22 15 11 187 187 187 194 197 173 77 25 19 19 26 17 11 190 187 190 191 158 37 15 14 20 20 15 12 11 187 182 180 127 32 16 13 16 14 12 13 11 14 184 186 172 100 20 13 15 18 13 18 14 15 10 186 190 187 127 18 14 15 14 12 10 11 11 10 189 192 192 148 16 15 11 10 10 9 9 9 9 192 195 181 37 13 10 10 10 10 10 9 9 9 189 194 54 14 11 10 10 10 9 8 9 8 9 194 140 19 16 11 11 10 10 9 9 9 9 9 192 88 12 11 11 10 10 10 9 9 9 9 9 194 37 10 10 9 9 9 9 9 9 9 8 8

    Image data types :: Intensity

    256 gray values

    Intensity (brightness)

    Image data types :: Indexed

    256 colors

    colormap = 1.0000 1.0000 1.0000 11.0000 1.0000 0 21.0000 0 1.0000 31.0000 0 0 4

    0 1.0000 1.0000 50 1.0000 0 60 0 1.0000 70 0 0 8

    0.5176 0 0 90.4510 0 0 100.3882 0 0 110.5490 0.0627 0.0627 120.4824 0.0627 0.0627 130.4196 0.0627 0.0627 140.5490 0.0941 0.0941 15... ... ... ...

    145 140 121 89 78 42 117 87 117 116 ...140 140 140 86 86 157 114 137 135 117 ...105 140 145 150 137 135 135 135 126 137 ...86 149 140 126 135 114 150 157 135 114 ...137 114 117 137 137 135 114 114 114 117 ...140 121 117 137 137 117 142 142 157 142 ...129 137 161 135 135 117 161 149 157 142 ...140 121 89 156 142 87 14 86 137 114 ...114 86 78 138 138 138 156 117 137 114 ...150 157 157 156 156 142 156 117 135 157 ...... ... ... ... ... ... ... ... ... ... ...

    Indices

    Image data types ::RGB (24 bit)

    3 x 256 gray values: 3 x 8 bit = 24 bit image

    8 bit gray ( red)

    8 bit gray ( blue)8 bit gray ( green)

    3 channels RGB, 32 bit color

    Definitions Digital image

    Image resolution Grayscale resolution Spatial resolution

    Image data types Binary (1 bit) Intensity (8 bit, gray) Indexed (8 bit, color) RGB (24 bit, color)

    Color spaces

  • 2/24/2012

    6

    The Lab color space isdesigned to approximatehuman vision. It aspiresto perceptual uniformity,and its L componentclosely matches humanperception of lightness.

    The RGB color model isan additive color model inwhich red, green, and bluelight are added together invarious ways to reproducea broad array of colors.

    HSL and HSV are the twomost common cylindrical-coordinate representations ofpoints in an RGB colormodel, which rearrange thegeometry of RGB in anattempt to be moreperceptually relevant thanthe Cartesian representation.

    Color spaces

    Image processing fundamentals-- Grayscale image processing basics

    Grayscale image processing basics Color lookup tables (LUT)

    Image contrast

    Image histogram

    Pixel statistics Profile statistics Region-of-interest statistics

    Fundamental spatial image processing tools Neighbors and connections (Matrix) transformations: scaling, rotation, other transforms Interpolation

    Grayscale image processing basics Color lookup tables (LUT)

    Image contrast

    Image histogram

    Pixel statistics Profile statistics Region-of-interest statistics

    Fundamental spatial image processing tools Neighbors and connections (Matrix) transformations: scaling, rotation, other transforms Interpolation

  • 2/24/2012

    7

    Color lookup tables (LUT) In biomedical imaging it is common to apply false colors while retaining

    the original 8 bit scale.

    Colored SEM image of soybean cyst nematode and egg. The color makes the image easier for non-specialists to view and understand the structures and surfaces revealed in micrographs.

    Color can also be used to convey a visual meaning to the values or measurements in the image.

    Color lookup tables (LUT)

    Heat map generated from DNA microarray data reflecting gene expression values in several conditions

    Differences in blood and oxygen levels in the brain shown by fMRI maps.

    Grayscale image processing basics Color lookup tables (LUT)

    Image contrast

    Image histogram

    Pixel statistics Profile statistics Region-of-interest statistics

    Fundamental spatial image processing tools Neighbors and connections (Matrix) transformations: scaling, rotation, other transforms Interpolation

    Image contrast Contrast is a measure of brightness difference, both globally (for the whole

    image) and locally (in neighboring regions). One possible way to express it is:

    C = ( IS IB ) / IB

  • 2/24/2012

    8

    Grayscale image processing basics Color lookup tables (LUT)

    Image contrast

    Image histogram

    Pixel statistics Profile statistics Region-of-interest statistics

    Fundamental spatial image processing tools Neighbors and connections (Matrix) transformations: scaling, rotation, other transforms Interpolation

    Image histogram The grayscale image histogram is a way of illustrating the distribution of

    gray levels in an image: it shows how many pixels have particular gray values (or gray value ranges: histogram bins).

    The histogram is a very important tool for performing image enhancement.

    Dynamic range ( 0 255 for 8 bit)

    Image histogram Color (RGB) images have three independent channels, displayed in Red,

    Green and Blue, respectively. Each channel has its own histogram.

    Grayscale image processing basics Color lookup tables (LUT)

    Image contrast

    Image histogram

    Pixel statistics Profile statistics Region-of-interest statistics

    Fundamental spatial image processing tools Neighbors and connections (Matrix) transformations: scaling, rotation, other transforms Interpolation

  • 2/24/2012

    9

    Pixel statistics ::Profile statistics (plots)

    Profile

    Pixel statistics :: ROI statistics

    Region of interest (ROI)

    SquareCircularFree-formManually drawn(Semi-)automatically extracted

    Grayscale image processing basics Color lookup tables (LUT)

    Image contrast

    Image histogram

    Pixel statistics Profile statistics Region-of-interest statistics

    Fundamental spatial image processing tools Neighbors and connections (Matrix) transformations: scaling, rotation, other transforms Interpolation

    Common pixel neighborhoods

    4-connected(2D)

    6-connected(3D)

    8-connected(2D)

    18-connected(3D)

    6-connected(2D)

    26-connected(3D)

  • 2/24/2012

    10

    Matrix transformations

    scaling

    rotationfree transform

    Interpolation Interpolation is a method of constructing new data points within the

    range of a discrete set of known data points.

    Used e.g. when scaling or generally transforming an image. There are several interpolation methods: linear, polynomial, spline,

    92 72 54

    92 72 5484 63Simplest linear interpolation.

    Image processing fundamentals-- The five classes of image processing

    The five classes of image processing Image enhancement

    Image histogram operations

    Spatial domain filtering

    Frequency domain filtering

    Image restoration Deconvolution

    Correction of geometrical distortions

    Correction of gray-level inhomogeneities

    Image analysis Segmentation

    Classification

    Image compression

    Image synthesis Registration

    Visualization

    3D rendering

  • 2/24/2012

    11

    The five classes of image processing-- Image enhancement

    Image enhancement Used to improve some aspects of the quality of an image:

    Increase contrast or and/or brightness

    Sharpen details

    Remove noise

    Image enhancement operations

    Contrast enhancement

    Spatial filtering

    Frequency filtering

    Often performed interactively

    Often performed as preprocessing step in an automated image analysis operation

    Result is often subjective, but quantitative measures of image quality do exist (i.e. contrast to noise ratio)

    Image histogram operations :: Contrast enhancement (gray)

    Low contrast image

    Histogram

    Datatype (8 bit) dynamic range: 0 255

    Image histogram operations ::Contrast enhancement (gray)

    Actual dynamic rangeLow contrast image

    Histogram

    Datatype (8 bit) dynamic range: 0 255

  • 2/24/2012

    12

    Image histogram operations ::Contrast enhancement (gray)

    Actual dynamic rangeLow contrast image

    Histogram

    Datatype (8 bit) dynamic range: 0 255

    Histogram stretching (linear mapping)

    Image histogram operations ::Contrast enhancement (gray)

    Actual dynamic rangeLow contrast image

    Histogram

    Datatype (8 bit) dynamic range: 0 255

    Histogram stretching (linear mapping)

    Adaptive histogram equalization

    (local)

    Histogram stretching

    (global)

    Image histogram operations :: Contrast enhancement (gray)

    Histogram equalization

    (global)

    Original

    Image histogram operations ::Contrast enhancement (color) Contrast enhancement of color images is typically done by transforming an

    image to a color space that has image intensity as one of its components (e.g. Lab) and then work on the luminosity layer L of the image.

    Manipulating luminosity affects the intensity of the pixels, while preserving the original colors.

  • 2/24/2012

    13

    Image histogram operations ::Contrast enhancement (color)

    Original Histogram stretching

    Histogram equalization Adaptivehistogram equalization

    (global)

    (global) (local)

    Spatial filtering operations ::Rank filters (nonlinear filters) Spatial filters are operations applied to a pixel using information taken from

    the neighborhood of that pixel.

    In rank (or ordered) filtering, the gray values of pixels within a defined neighborhood around the pixel of interest are arranged in a list in ascending order. The new value of for the pixel of interest is the value at the required rank position in the list.

    Common rank filters: maximum ( rank = n ) minimum ( rank = 1 ) median ( rank = n/2 ) range ( maximum minimum )

    The outcome of the filtering depends on the size and shape of the neighborhood.

    Rank filters ::Median filter

    Original image Added salt-and-pepper noise Median filtered image

    Rank filters ::Median filter

    Original image Added salt-and-pepper noise Median filtered image

    The rank filter causes some (limited) blurriness in the image.

  • 2/24/2012

    14

    Spatial filtering operations ::Convolution filters (linear filters) Like rank filtering, convolution filter also replaces a pixel value with a new

    value obtained from the pixel neighborhood. But there are a few more calculations involved.

    A square kernel is defined to represent the neighborhood around the pixel. Each location in the kernel is associated a numerical value, called weight.

    a b c

    d e f

    g h i

    The kernel is moved across the image, and the pixel value under the center of the kernel is replaced by the weighted sum of the surrounding pixels.

    Kernel size can vary a lot,depending on the application.This is a 3x3 kernel.

    Spatial filtering operations

    Convolution filters

    a b c

    d e f

    g h i

    A B C

    F G H

    K L M

    D E

    I J

    N O

    P Q R

    U V W

    S T

    X Y

    A B C

    F cG bH

    K fL eM

    D EaI JdN O

    P iQ hR

    U V W

    gS T

    X Y

    A B C

    F aG bH

    K dL eM

    D EcI JfN O

    P gQ hR

    U V W

    iS T

    X Y

    kernel image

    convolution

    correlation

    Mirrored in case of convolution!

    Used to find similarities between two images.

    Spatial filtering operations The convolution kernel is mirrored. For most applications, the kernel is symmetric, and mirroring does not

    change it. Many (lazy) convolution implementations indeed use correlation.

    Convolution: M = iG + hH + gI + fL + eM + dN + cQ + bR + aS

    Correlation: M = aG + bH + cI + dL + eM + fN + gQ + hR + iS

    Spatial filtering operations

    919191919191919191

    h

    3x3 average filter

    0.07510.12380.07510.12380.20420.12380.07510.12380.0751

    h

    3x3 Gaussian filter

    111181111

    h

    High-pass filter

    Examples of convolution filters

  • 2/24/2012

    15

    Spatial filtering operations

    121/1...121/1121/1121/1...............121/1...121/1121/1121/1121/1...121/1121/1121/1121/1...121/1121/1121/1

    h

    11x11 average filter

    101-101-101-

    h

    PrewittVertical edge

    111000111

    h

    PrewittHorizontal edge

    Examples of convolution filters

    Given the same kernel size, convolution filters introduce more blurring than rank filters.

    Spatial filtering operations

    Low-pass filterGaussian kernel

    Band-pass filterDifference of Gaussians kernel

    High-pass filter1 - Gaussian kernel

    Hybrid filters may include both rank and convolution steps, or involve other image processing operations.

    Adaptive filters perform a different operation depending on the image content in the region in which they are being applied. For instance, an adaptive filter could be designed to blur only those regions of an

    image that do not contain an edge (e.g. anisotropic filtering).

    Spatial filtering operations :: Hybrid filters and adaptive filters

    Frequency filtering has an effect equivalent to convolution filtering, but the approach is different.

    Shapes in images are made up of changes in gray levels across the image, from dark to light and back to dark. The rate of this change is called spatial frequency.

    A real image will be much more complex and contain many more frequencies. However, we can say that: high spatial frequencies correspond to fine detail, such as noise and edges

    low spatial frequencies correspond to larger objects with fairly uniform gray values.

    Frequency domain filtering ::Spatial and frequency domains

    Increasing spatial frequency

  • 2/24/2012

    16

    A high-pass filter preserves high frequencies.

    A low-pass filter preserves low frequencies.

    A band-pass (or notch) filter preserves a particular band, or range, or frequencies.

    Frequency domain filtering ::Types of frequency filters

    Frequency domain filtering ::Approach to frequency filtering Generate a representation to show which spatial frequencies are present in

    an image: i.e. generate a frequency spectrum using Fourier transformation.

    Remove selected frequencies from this representation.

    Reverse the process to get back to the image (spatial representation): i.e. apply the inverse Fourier transformation.

    Frequency domain filtering ::Advantages of frequency domain filtering It is possible to perform operations that are difficult in the spatial domain,

    such as removing or enhancing only specific frequencies in the image.

    Periodic patterns can be selectively removed or enhanced.

    For operations that require larger kernel sizes in the spatial domain (especially in 3D), frequency domain filtering is computationally faster.

    Frequency domain filtering

    Increasing frequency

    Increasingfrequency

  • 2/24/2012

    17

    Frequency domain filtering

    Fourier transform

    Frequency domain filtering ::Low-pass filtering

    InverseFourier transform

    Set high frequencies to 0.

    RemarkIn practice, one does not create sharp cut-offs in frequency domain, since this creates ringing artifacts that appear as spurious signals near sharp transitions in a signal, i.e. they appear as "rings" near edges.

    Frequency domain filtering ::High-pass filtering

    InverseFourier transform

    Set low frequencies to 0.

    Frequency domain filtering :: Band-pass filtering

    InverseFourier transform

    Preserve frequencies only in this band.

  • 2/24/2012

    18

    Frequency domain filtering

    Fourier transform ?

    Frequency domain filtering

    Frequency domain filteringFrequency domain filtering ::Convolution theorem

    Convolution in direct space=

    multiplication in frequency (Fourier) space

    Inverse filtering

    Fourier transform

    Inverse Fourier transform

  • 2/24/2012

    19

    The five classes of image processing-- Image analysis

    Image analysis The goal of image analysis is to extract and assign identities to objects in

    images, and then extract some form of (numerical) measurement from them.

    Operations:

    Image segmentation

    Object classification

    Image arithmetic

    Binary operations

    Examples of measurements:

    Length, area, volume, density,

    Usually statistical analyses are then performed on the obtained measurements

    Image analysis ::Segmentation Image segmentation refers to the process of partitioning a digital image

    into multiple sets of pixels with the goal of simplifying and/or changing the representation of an image into something that is more meaningful and easier to analyze.

    Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images.

    More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics.

    Image analysis ::Classification Classification assigns objects obtained from segmentation into categories

    based on quantitative information on one or more characteristics inherent in the items (referred to as traits, variables, characters, etc).

    There are many classification algorithms divided in several categories: Linear classifiers (Fisher's linear discriminant, Logistic regression, Naive Bayes classifier, Perceptron,

    Support vector machines)

    Quadratic classifiers k-nearest neighbor Boosting Decision trees (Random forests) Neural networks Bayesian networks Hidden Markov models

    The separation between segmentation and classification is quite blurred in all but the simplest segmentation approaches!

  • 2/24/2012

    20

    Segmentation There are three approaches to segmentation:

    Manual

    Semiautomatic

    Automatic

    Manual segmentation Advantages:

    Simple, provided there are suitable tools

    Disadvantages:

    Very time-consuming

    Subject to human error

    Subjective

    Poor (intra-observer) reproducibility

    Manual segmentation Manual segmentation involves an expert observer outlining the object of

    interest in the image.

    The computer simply provides tools to help with drawing.

    Manual segmentation ::Simple thresholding The user chooses an intensity threshold level in the histogram.

    The software shows a preview to facilitate the selection of the threshold.

  • 2/24/2012

    21

    Manual segmentation :: Simple thresholding After the threshold is set, all values below the threshold are set to 0.

    The values above the threshold are either left unchanged, or set to white (e.g. 255).

    Semi-automatic segmentation In semi-automatic segmentation, the observer helps the segmentation by

    providing the software with a rough selection to refine.

    Region growing

    Active contours, snakes, and deformable surfaces

    Semi-automatic segmentation ::Region growing The operator defines a range of gray levels that represent the region, and

    chooses a pixel (the seed point) that is known to be part of the region.

    All the pixels that are in the intensity range and are connected to the seed point are selected as being part of the region.

    Alternatively, the software returns the boundary of the region as an editable line.

    Semi-automatic segmentation ::Region growing

    Range = 10 gray values

  • 2/24/2012

    22

    Semi-automatic segmentation ::Active contours, deformable surfaces Active contour model, also called snakes ( = deformable splines), is a

    framework for delineating an object outline from a possibly noisy 2D image.

    This framework attempts to minimize an energy associated to the current contour as a sum of an internal and external energy:

    The external energy is supposed to be minimal when the snake is at the object boundary position (e.g. low energy values for high gradient values);

    The internal energy is supposed to be minimal when the snake has a shape which is supposed to be relevant considering the shape of the sought object (e.g. high energy to elongated contours (elastic force) and to bended/high curvature contours (rigid force), considering the shape should be as regular and smooth as possible).

    The framework also allows for training.

    Semi-automatic segmentation ::Active contours / snakes

    Range = 10 gray values

    Semi-automatic segmentation ::Deformable surfaces

    Range = 10 gray values

    Automatic segmentation Automatic (computational) segmentation methods are areas of

    active research.

    We will only look into a very simple automatic segmentation algorithm: Otsus method.

  • 2/24/2012

    23

    Automatic segmentation ::Otsus method Otsu's method is used to automatically perform histogram shape-based

    image thresholding.

    The algorithm assumes that the image to be thresholded contains two classes of pixels (e.g. foreground and background) then calculates the optimum threshold separating those two classes so that their combined spread (intra-class variance) is minimal.

    Range = 10 gray values

    threshold

    Ideal case

    Automatic segmentation ::Otsus method

    Range = 10 gray values

    This image probably contains more than two classes of pixels.

    There are extensions of Otsus algorithm that find multiple thresholds.

    Image arithmetic Image arithmetic is the implementation of standard arithmetic operations,

    such as addition, subtraction, multiplication, and division, on images.

    Image arithmetic has many uses in image processing both as a preliminary step in more complex operations and by itself. Examples: Image subtraction can be used to detect differences between two or more images of the

    same scene or object.

    Image subtraction or division are used for shading corrections. Image multiplication is used for masking. Image addition is used for visualizing composite images (where each channel has a

    color which is not exclusively either Red, Green, or Blue).

    Binary and morphological operations Binary and morphological operations can be used to perform common

    image processing tasks, such as contrast enhancement, noise removal, thinning, skeletonization, filling, and segmentation.

    Morphology is a broad set of image processing operations that process images based on shapes.

    Morphological operations apply a structuring element to an input image, creating an output image of the same size.

    In a morphological operation, the value of each pixel in the output image is based on a comparison of the corresponding pixel in the input image with its neighbors.

    By choosing the size and shape of the neighborhood, you can construct a morphological operation that is sensitive to specific shapes in the input image.

  • 2/24/2012

    24

    Binary and morphological operations ::Example: Image dilation Dilation is one of the two basic operators in the area of morphology, the

    other being erosion. It is typically applied to binary images, but there is a version for grayscale

    images. Dilation gradually enlarges the boundary of regions of foreground pixels

    (i.e. white pixels): thus area of foreground pixels grow in size, while holes within these regions become smaller.

    The dilation operator takes two pieces of data as inputs. The first is the image which is to be dilated. The second is a (usually small) set of coordinate points known as a structuring element, for example:

    1 1 1

    1 1 1

    1 1 1

    0 1 0

    1 1 1

    0 1 0

    Binary and morphological operations ::Example: Image dilation Mathematically, dilation is defined as:

    This basically means, place the structuring element B on some position on the image A. If the pixel value of A under the center of B is 1, replace the n x n pixels in A with the structuring element B.

    Example:

    Binary and morphological operations

    Problem: detecting (i.e. segmenting) a cell

    Binary and morphological operations

    A first rough segmentation is given by a threshold of gradient of the image, to give the strongest edges.

    Binary gradient mask

  • 2/24/2012

    25

    Binary and morphological operations

    Image dilation: the binary gradient mask is dilated.

    Dilated gradient mask

    Binary and morphological operations

    Image flood-fill: the dilated gradient mask shows the outline of the cell quite nicely, but there are still holes in the interior of the cell that we flood-fill.

    Dilated gradient mask with filled holes

    Binary and morphological operations

    Here we got rid of the cell at the border. Admittedly, this is not strictly a morphological operation

    Cleared border image

    Binary and morphological operations

    Image erosion: finally, in order to make the segmented object look natural, we smoothen the object by eroding the image twice with a diamond structuring element.

    Segmented image

  • 2/24/2012

    26

    Binary and morphological operations

    An alternate method for displaying the segmented object would be to place an outline around the segmented cell.

    Segmented image with outline

    The five classes of image processing-- Image restoration

    Image restoration Required if the acquisition method (e.g. the microscope) causes:

    geometric distortion

    blurring

    gray-level inhomogeneities

    Geometric distortionAlthough distortion can be irregular or follow many patterns, the most commonly encountered distortions are radially symmetric, arising from the symmetry of a lens. The radial distortion can usually be classified as one of two main types: Barrel distortion and Pincushion distortion.

    In barrel distortion, image magnification decreases with distance from the optical axis. The apparent effect is that of an image which has been mapped around a sphere (or barrel).

    In pincushion distortion, image magnification increases with the distance from the optical axis. The visible effect is that lines that do not go through the centre of the image are bowed inwards, like a pincushion.

  • 2/24/2012

    27

    Blurring Any imaging system has a blurring effect.

    If a point object is imaged, the resulting image is not a perfect point, but a spread-out version of that point.

    Mathematically, this blurring effect is described using the convolution operation:

    Every imaging system has a characteristic blurring function called point spread function (PSF): h(x)

    Point object, whose dimensions are below the resolution limit.

    Point spread function, lateral view.

    Axial view.

    Removing blur Deconvolution Knowledge of the PSF of a system is useful because of the potential for removing

    blurring by applying the inverse of convolution, deconvolution.

    We saw that convolution can be performed in frequency space. Can we deconvolve by inverse filtering, then?

    Point spread function, lateral view.

    Axial view.

    f

    h

    F

    H

    G g

    f

    F -1(G/H)?

    F -1(G)

    F (f)

    F (h)

    Deconvolution :: Cookie cutter

    XZ

    Missing frequencies

    Sub-resolution trainNoise-free convolution and deconvolution

    Deconvolution :: Deconvolving trains

  • 2/24/2012

    28

    Sub-resolution trainNoise-free convolution and deconvolution

    Confocal

    Widefield

    Deconvolution :: Deconvolving trains

    Confocal

    Widefield

    Restored confocal

    Restored widefield

    Sub-resolution trainNoise-free convolution and deconvolution

    Deconvolution :: Deconvolving trains

    Deconvolution :: MLE

    Extreme noise amplification!

    Inverse filtering will never allow us to recover the true object function f.

    Artifacts!

    H = 0 at many places!

    Future promising algorithms are based on wavelets.

    Deconvolution is performed in practice with iterative algorithms like the maximum likelihood estimation (MLE) algorithm.

    Correction of gray-level inhomogeneities(shading correction)

    Background estimation

    Rolling ball Morphological Opening

    Subtraction

  • 2/24/2012

    29

    Correction of gray-level inhomogeneities(shading correction)

    A correction is important, for instance, for stitching applications to prevent border effects.

    Correction of gray-level inhomogeneities(shading correction)

    A correction is important, for instance, for stitching applications to prevent border effects.

    The five classes of image processing-- Image compression

    Image compression Image data files are often very large in terms of number of bytes

    Large disk space usage

    Slow transfer over networks

    Compression algorithms exists

    Lossless

    Lossy

  • 2/24/2012

    30

    Image compression :: Lossy compression

    100%, 64kb

    Image compression ::Lossy compression

    Jpeg, 80%, 12kb

    Image compression ::Lossy compression

    Jpeg, 50%, 7kb

    Image compression ::Lossy compression

    Jpeg, 10%, 3kb

  • 2/24/2012

    31

    Image compression ::Lossy compression

    Jpeg, 1%, 1kb

    Image compression ::Lossless compression

    100%, 64kb

    Image compression ::Lossless compression

    PNG, best compression, 48kb

    The five classes of image processing-- Image synthesis

  • 2/24/2012

    32

    Image synthesis Image synthesis is the general term for bringing together information from

    more than one image.

    Synthesis can be separated into two parts:

    Image registration, covering processes required to bring images into spatial alignment

    Visualization, which allows information (from the aligned datasets) to be viewed

    Reasons for performing image synthesis include:

    Assessment of disease progression or growth using series of images

    Combination of structural and functional information from different imaging modalities

    Comparison of corresponding regions in different individuals by matching both to a standard coordinate system

    Generation and analysis of atlases or templates representing the typical appearance in health or disease

    Arithmetical or statistical operations which require registered images

    Image registration How to map one onto the other?

    Image 1

    Image 2

    Smaller, rotated version of image 1.

    Image registration The four steps of image registration:

    Feature extraction

    One needs to identify features that appear in both images. This can be done manually, by placing markers on the images, or automatically. All pixels intensities could also be used.

    Pairing (also called: identifying correspondences)

    Once features have been identified in both images, they need to be analyzed to determine which feature in one image matches which feature in the other image.

    The features can be fiducial landmarks or pixel intensities (or related values).

    The term similarity metric is the general way of describing a measure that is used in the process of pairing points with similar properties. Examples: correlation coefficient, mutual information, entropy

    Calculation of transformation

    The transformation is the mathematical operation that will give the best alignment (mapping) of all pairs of features. There arefour different types of transformations:

    Rigid: in a rigid transformation all parts of the object are assumed to move as a whole, e.g. translation, rotation

    Affine: in affine transformations, straight lines remain straight and parallel lines remain parallel: e.g. scaling, shearing

    Projective: projective transformation include further deformation. Straight lines remain straight, but parallel lines do not remain parallel.

    Curved or elastic (non-rigid): this is the most general case of transformation. Straight lines need not remain straight. It should be used with care.

    Application of transformation

    This step is often called matrix transformation or matrix operation.

    Applying the transformation will very often require interpolation.

    Image registrationExample: rigid transformation

    1. The user chooses some fiduciary marks by clicking on them2. The positions are refined by cross-correlation3. The (rigid) transformation matrix is obtained by Least-Square fitting4. Image 1 is transformed (mapped) onto image 2 by matrix transformation and interpolation.

    11

    22

    33

    4455

    12

    3

    4

    5

    111222

    443

    334

    555

  • 2/24/2012

    33

    Image registration End result of registration

    Image registration example ::Atlas generation

    Mouse hippocampus

    A user draws the outline of the hippocampus overmany aligned slices to reconstruct the 3-D shape.

    The same operation is performed on 3 mice, and the resulting 3D objects are registered on top of each other to generate an atlas (i.e. a model of the hippocampus).

    1)2)

    3)

    Image registration example ::Correlative microscopy

    Biocytin labelled cell, Z projection of 80 confocal planes. (200nm X-Y pixel size)

    Same cell imaged in SBFSEM (98x98x30 microns)

    Visualization Visualization is the display of image data.

    Visualization in biomedical imaging needs specialized approaches:

    Visualized objects can have a huge range of scales, from single molecules and cells to body parts

    Different attributes of these objects can be visualized, including biophysical, biomechanical and physiological properties.

    The result of visualization can be used for diagnosis, treatment planning, rehearsal, assessment, and intra-operative guidance.

    There are several approaches to visualization:

    Slice mode

    Projection mode

    Volume mode

    Surface mode

  • 2/24/2012

    34

    Visualization ::Slice mode

    Fluorescence image of the mouse retina. Red: collagen IV, Green: GFAP (source Imaris demos)

    Shown: one slice (focal plane) in the middle of the 3D acquisition.

    Visualization ::Slice mode

    Shown: all slices in one view.

    Visualization ::Maximum intensity projection

    Visualization ::Volume rendering

  • 2/24/2012

    35

    Visualization ::Surface rendering

    Visualization ::Surface rendering (fancy)

    Last remarks

    Last remarks Every image processing action taken must make sense:

    Do not apply any filter just because like this the image looks better.

    Remember that applying the wrong filter can destroy real features in the image and also introduce spurious ones that actually DO NOT EXIST.

    Keep a complete protocol of the actions taken: The result (i.e. the figure in a paper) must be reproducible by you and by others.

    Make sure you know the characteristics of your acquisition system (e.g. a microscope), because this will define:

    The parameters for acquiring the images.

    The parameters for processing the images.

    Image processing was born to process different kinds of pictures than biomedical ones (which are diffraction-limited, with extremely low SNR, 3D or more, )

    There is a lot of work going on extending standard image processing techniques for these new fields.

    Image processing of microscopy data requires a lot of statistical modeling, learning algorithms, and the like (which we havent touched in this introductory course).