1

ITK Lecture 8 - NeighborhoodsITK Lecture 8 - Neighborhoods

Methods in Image AnalysisCMU Robotics Institute 16-725U. Pitt Bioengineering 2630

Spring Term, 2006

Methods in Image AnalysisCMU Robotics Institute 16-725U. Pitt Bioengineering 2630

Spring Term, 2006

Damion Shelton

2

Goals for this lectureGoals for this lecture

Understand what a neighborhood is and and the different ways of accessing pixels using one

Use neighborhoods to implement a convolution/correlation filter

Understand what a neighborhood is and and the different ways of accessing pixels using one

Use neighborhoods to implement a convolution/correlation filter

3

What is a neighborhood?What is a neighborhood? You may already be familiar with the concept of pixels having neighbors

Standard terminology in 2D image processing will refer to the 4 neighborhood (N,E,S,W) and the 8 neighborhood (4 neighborhood + NE, SE, SW, NW)

You may already be familiar with the concept of pixels having neighbors

Standard terminology in 2D image processing will refer to the 4 neighborhood (N,E,S,W) and the 8 neighborhood (4 neighborhood + NE, SE, SW, NW)

4

Neighborhoods in ITKNeighborhoods in ITK

ITK carries this concept a bit further

A neighborhood can be any collection of pixels that have a fixed relationship to the “center” based on offsets in data space

See 11.4 in the ITK Software Guide

ITK carries this concept a bit further

A neighborhood can be any collection of pixels that have a fixed relationship to the “center” based on offsets in data space

See 11.4 in the ITK Software Guide

5

Neighborhoods in ITK, cont.Neighborhoods in ITK, cont. In general, the neighborhood is not completely arbitrary Neighborhoods are rectangular, defined by a “radius” in N-dimensions

ShapedNeighborhoods are arbitrary, defined by a list of offsets from the center

The first form is most useful for mathematical morphology kinds of operations, convolution, etc.

In general, the neighborhood is not completely arbitrary Neighborhoods are rectangular, defined by a “radius” in N-dimensions

ShapedNeighborhoods are arbitrary, defined by a list of offsets from the center

The first form is most useful for mathematical morphology kinds of operations, convolution, etc.

6

Neighborhood iteratorsNeighborhood iterators

The cool & useful thing about neighborhoods is that they can be used with neighborhood iterators to allow efficient access to pixels “around” a target pixel in an image

The cool & useful thing about neighborhoods is that they can be used with neighborhood iterators to allow efficient access to pixels “around” a target pixel in an image

7

Neighborhood iteratorsNeighborhood iterators

Remember that I said access via pixel indices was slow? Get current index = I Upper left pixel index IUL = I - (1,1)

Get pixel at index IUL

Neighborhood iterators solve this problem by doing pointer arithmetic based on offsets

Remember that I said access via pixel indices was slow? Get current index = I Upper left pixel index IUL = I - (1,1)

Get pixel at index IUL

Neighborhood iterators solve this problem by doing pointer arithmetic based on offsets

8

Neighborhood layoutNeighborhood layout

Neighborhoods have one primary parameter, their “radius” in N-dimensions

The side length along a particular dimension i is 2*radiusi + 1

Note that the side length is always odd because the center pixel always exists

Neighborhoods have one primary parameter, their “radius” in N-dimensions

The side length along a particular dimension i is 2*radiusi + 1

Note that the side length is always odd because the center pixel always exists

9

A 2x1 neighborhood in 2DA 2x1 neighborhood in 2D

0 1 2 3 4

5 6 7 8 9

10 11 12 13 14

10

StrideStride

Neighborhoods have another parameter called stride which is the spacing (in data space) along a particular axis between adjacent pixels in the neighborhood

In the previous numbering scheme, stride in Y is amount then index value changes when you move in Y

In our example, Stridex = 1, Stridey = 5

Neighborhoods have another parameter called stride which is the spacing (in data space) along a particular axis between adjacent pixels in the neighborhood

In the previous numbering scheme, stride in Y is amount then index value changes when you move in Y

In our example, Stridex = 1, Stridey = 5

11

Neighborhood pixel accessNeighborhood pixel access The numbering on the previous page is important! It’s how you access that particular pixel when using a neighborhood iterator

This will be clarified in a few slides...

The numbering on the previous page is important! It’s how you access that particular pixel when using a neighborhood iterator

This will be clarified in a few slides...

12

NeighborhoodIterator accessNeighborhoodIterator access Neighborhood iterators are created using: The radius of the neighborhood The image that will be traversed The region of the image to be traversed

Their syntax largely follows that of other iterators (++, IsAtEnd(), etc.)

Neighborhood iterators are created using: The radius of the neighborhood The image that will be traversed The region of the image to be traversed

Their syntax largely follows that of other iterators (++, IsAtEnd(), etc.)

13

Neighborhood pixel access, cont.Neighborhood pixel access, cont.

1.2 1.3 1.8 1.4 1.1

1.8 1.1 0.7 1.0 1.0

2.1 1.9 1.7 1.4 2.0

Let’s say there’s some region of an image that hasthe following pixel values

14

Pixel access, cont.Pixel access, cont.

Now assume that we place the neighborhood iterator over this region and start accessing pixels

What happens?

Now assume that we place the neighborhood iterator over this region and start accessing pixels

What happens?

15

Pixel access, cont.Pixel access, cont.

1.20

1.31

1.82

1.43

1.14

1.85

1.16

0.77

1.08

1.09

2.110

1.911

1.712

1.413

2.014

myNeigh.GetPixel(7) returns 0.7so does myNeigh.GetCenterPixel()

16

Pixel access, cont.Pixel access, cont.

Next, let’s get the length of the iterator and the stride length

Size() returns the #pixels in the neighborhood

unsigned int c = iterator. Size () / 2;

GetStride returns the stride of dimension N

unsigned int s = iterator. GetStride(1);

Next, let’s get the length of the iterator and the stride length

Size() returns the #pixels in the neighborhood

unsigned int c = iterator. Size () / 2;

GetStride returns the stride of dimension N

unsigned int s = iterator. GetStride(1);

17

Pixel access, cont.

1.20

1.31

1.82

1.43

1.14

1.85

1.16

0.77

1.08

1.09

2.110

1.911

1.712

1.413

2.014

myNeigh.GetPixel(c) returns 0.7myNeigh.GetPixel(c-1) returns 1.1

18

Pixel access, cont.

1.20

1.31

1.82

1.43

1.14

1.85

1.16

0.77

1.08

1.09

2.110

1.911

1.712

1.413

2.014

myNeigh.GetPixel(c-s) returns 1.8myNeigh.GetPixel(c-s-1) returns 1.3

19

The ++ methodThe ++ method

In ImageRegionIterators, the ++ method moves the focus of the iterator on a per pixel basis

In NeighborhoodIterators, the ++ method moves the center pixel of the neighborhood and therefore implicitly shifts the entire neighborhood

In ImageRegionIterators, the ++ method moves the focus of the iterator on a per pixel basis

In NeighborhoodIterators, the ++ method moves the center pixel of the neighborhood and therefore implicitly shifts the entire neighborhood

20

Does this sound familiar?Does this sound familiar? If I say:

I have a region of interest defined by a certain radius around a center pixel

The ROI is symmetric I move it around an image

What does this sound like?

If I say: I have a region of interest defined by a certain radius around a center pixel

The ROI is symmetric I move it around an image

What does this sound like?

21

Convolution (ahem, correlation)!Convolution (ahem, correlation)!To do convolution we need 3

things:1. A kernel2. A way to access a region of an

image the same size as the kernel

3. A way to compute the inner product between the kernel and the image region

To do convolution we need 3 things:1. A kernel2. A way to access a region of an

image the same size as the kernel

3. A way to compute the inner product between the kernel and the image region

22

Item 1 - The kernelItem 1 - The kernel

A NeighborhoodOperator is a set of pixel values that can be applied to a Neighborhood to perform a user-defined operation (i.e. convolution kernel, morphological structuring element)

NeighborhoodOperator is derived from Neighborhood

A NeighborhoodOperator is a set of pixel values that can be applied to a Neighborhood to perform a user-defined operation (i.e. convolution kernel, morphological structuring element)

NeighborhoodOperator is derived from Neighborhood

23

Item 2 - Image access methodItem 2 - Image access method We already showed that this is possible using the neighborhood iterator

Just be careful setting it up so that it’s the same size as your kernel

We already showed that this is possible using the neighborhood iterator

Just be careful setting it up so that it’s the same size as your kernel

24

Item 3 - Inner product methodItem 3 - Inner product method The NeighborhoodInnerProduct computes the inner product between two neighborhoods

Since NeighborhoodOperator is derived from Neighborhood, we can compute the IP of the kernel and the image region

The NeighborhoodInnerProduct computes the inner product between two neighborhoods

Since NeighborhoodOperator is derived from Neighborhood, we can compute the IP of the kernel and the image region

25

Good to go?Good to go?

1. Create an interesting operator to form a kernel

2. Move a neighborhood through an image

3. Compute the IP of the operator and the neighborhood at each pixel in the image

Voila - convolution in N-dimensions

1. Create an interesting operator to form a kernel

2. Move a neighborhood through an image

3. Compute the IP of the operator and the neighborhood at each pixel in the image

Voila - convolution in N-dimensions

26

Inner product exampleInner product example

itk::NeighborhoodInnerProduct<ImageType> IP; itk::DerivativeOperator<ImageType> operator ;

operator->SetOrder(1); operator->SetDirection(0); operator->CreateDirectional();

itk::NeighborhoodIterator<ImageType> iterator(operator->GetRadius(), myImage, myImage->GetRequestedRegion());

itk::NeighborhoodInnerProduct<ImageType> IP; itk::DerivativeOperator<ImageType> operator ;

operator->SetOrder(1); operator->SetDirection(0); operator->CreateDirectional();

itk::NeighborhoodIterator<ImageType> iterator(operator->GetRadius(), myImage, myImage->GetRequestedRegion());

27

Inner product example, cont.Inner product example, cont.iterator.SetToBegin();while ( ! iterator. IsAtEnd () ) { std::cout << "Derivative at index " <<

iterator.GetIndex () << is << IP(iterator, operator) << std::endl; ++iterator; }

iterator.SetToBegin();while ( ! iterator. IsAtEnd () ) { std::cout << "Derivative at index " <<

iterator.GetIndex () << is << IP(iterator, operator) << std::endl; ++iterator; }

28

NoteNote

No explicit reference to dimensionality in neighborhood iterator

easy to make N-d

No explicit reference to dimensionality in neighborhood iterator

easy to make N-d

29

This suggests a filter...This suggests a filter... NeighborhoodOperatorImageFilter wraps this procedure into a filter that operates on an input image

So, if the main challenge is coming up with an interesting neighborhood operator, ITK can do the rest

NeighborhoodOperatorImageFilter wraps this procedure into a filter that operates on an input image

So, if the main challenge is coming up with an interesting neighborhood operator, ITK can do the rest

30

Your arch-nemesis... image boundariesYour arch-nemesis... image boundaries One obvious problem with inner product techniques is what to do when you reach the edge of your image

Is the operation undefined? Does the image wrap? Should we assume the rest of the world is empty/full/something else?

One obvious problem with inner product techniques is what to do when you reach the edge of your image

Is the operation undefined? Does the image wrap? Should we assume the rest of the world is empty/full/something else?

31

ImageBoundaryConditionImageBoundaryCondition

Subclasses of itk::ImageBoundaryCondition can be used to tell neighborhood iterators what to do if part of the neighborhood is not in the image

Subclasses of itk::ImageBoundaryCondition can be used to tell neighborhood iterators what to do if part of the neighborhood is not in the image

32

ConstantBoundaryConditionConstantBoundaryCondition The rest of the world is filled with some constant value of your choice

The default is 0 Be careful with the value you choose - you can (for example) detect edges that aren’t really there

The rest of the world is filled with some constant value of your choice

The default is 0 Be careful with the value you choose - you can (for example) detect edges that aren’t really there

33

PeriodicBoundaryConditionPeriodicBoundaryCondition The image wraps, so that if I exceed the length of a particular axis, I wrap back to 0 and start over again

If you enjoy headaches, imagine this in 3D

This isn’t a bad idea, but most medical images are not actually periodic

The image wraps, so that if I exceed the length of a particular axis, I wrap back to 0 and start over again

If you enjoy headaches, imagine this in 3D

This isn’t a bad idea, but most medical images are not actually periodic

34

ZeroFluxNeumannBoundaryConditionZeroFluxNeumannBoundaryCondition I am not familiar with how this functions

The documentation states that it’s useful for solving certain classes of differential equations

A quick look online suggests a thermodynamic motivation

I am not familiar with how this functions

The documentation states that it’s useful for solving certain classes of differential equations

A quick look online suggests a thermodynamic motivation

35

Using boundary conditionsUsing boundary conditions With NeighborhoodOperatorImageFilter, you can call OverrideBoundaryCondition

With NeighborhoodOperatorImageFilter, you can call OverrideBoundaryCondition

36

SmartNeighborhoodIteratorSmartNeighborhoodIterator This is the iterator that’s being used internally by the previous filter; you can specify its boundary behavior using OverrideBoundaryCondition too

In general, I would suggest using the “smart” version - bounds checking is good!

This is the iterator that’s being used internally by the previous filter; you can specify its boundary behavior using OverrideBoundaryCondition too

In general, I would suggest using the “smart” version - bounds checking is good!

37

An aside: numeric traitsAn aside: numeric traits This has nothing to do with Neighborhoods but is good to know

Question: given some arbitrary pixel type, what do we know about it from a numerics perspective?

This has nothing to do with Neighborhoods but is good to know

Question: given some arbitrary pixel type, what do we know about it from a numerics perspective?

38

itk::NumericTraitsitk::NumericTraits

NumericTraits is class that’s specialized to provide information about pixel types

Examples include: min and max values IsPositive(), IsNegative() Definitions of Zero and One

NumericTraits is class that’s specialized to provide information about pixel types

Examples include: min and max values IsPositive(), IsNegative() Definitions of Zero and One

39

Using traitsUsing traits

What’s the maximum value that can be represented by an unsigned char?

itk::NumericTraits<unsigned char>::max()

Look at vnl_numeric_limits for more data that can be provided

What’s the maximum value that can be represented by an unsigned char?

itk::NumericTraits<unsigned char>::max()

Look at vnl_numeric_limits for more data that can be provided