robust statistical method for background extraction in image segmentation

Robust statistical method for background extraction in image segmentation

Doug KeenMarch 29, 2001

Source Paper

• Rodriguez, Arturo A., Mitchell, O. Robert. “Robust statistical method for background extraction in image segmentation” Stochastic and Neural Methods in Signal Processing, Image

Processing, and Computer Vision. Vol. 1569, 1991

Problem• Given a

digital image, how can we differentiate objects of interest from the background?

Problem

One simple and fast method is thresholding– Create a graytone histogram of a sample of the

background– Find threshold values for the right and left

shoulders of the background histogram– Compare graytone values of all other pixels of the

image to the background histogram• If pixel falls between right and left background

thresholds, that pixel belongs to the background

Problems with thresholding• Local background variations may be small, but

background variations across the entire image may be substantial

• Thresholding also assumes just a single background

Problems with thresholding

• In applications with a fixed background, one could do an empirical analysis of a background only image– Once an object is introduced, graytone properties

of the background could change (due to reflectance properties of the object, scene illumination, automatic gain control of the camera, and/or shadows)

Alternate solution

• A background extraction method that is based on local statistical measurements performed on log-transformed image data– Works despite smooth changes in background– Doesn’t matter if objects are darker or brighter

than the background– Works with multiple backgrounds– Works without a priori knowledge of the

background of the image

Log-Transformation

• Log-transformation can help reduce various illumination effects in the image

g(x, y) = i(x, y) * f(x, y) g(x,y) is the grayscale value of a pixel

i(x,y) is the illumination componentf(x,y) is the reflectance component

• A log-transformation would cause multiplicative illumination effects to become additive

Log-Transformation

In an image with graytone values:g [gL, gR]

The log-transformation is expressed as:

)1log()1log()1log()1log(]1)1([)(

LR

LLR

ggggggPggL L

Where P is a fraction of the original gray scale resolution

Background Extraction Segmentation Method

Step 1:• Decompose the image into a grid of non-overlapping

blocks

Blocks along the periphery are boundary blocks, and blocks that are not boundary blocks are interior blocks.

Step 2:• For each block in the image, calculate the mean

graytone, the left and right standard deviations of the log-transformed histogram (denoted by , Tleft, and Tright respectively)


Step 2 (cont.):• A block can be considered homogenous if Q percent

or more of its pixels belong to the same class. Specifically, a block can be considered homogenous if:

andWhere S is the standard deviation of the image

• Homogenous blocks can be object-homogenous or background-homogenous. If the block is not homogenous, it is considered uncertain.


)1( QQSTleft )1( QQSTright

Step 2 (cont.):• Homogenous boundary blocks are assumed to be

background blocks, and their measured statistical parameters are considered their background distributions


Step 3:• Non-homogenous boundary blocks are examined by

starting from an arbitrary homogenous boundary block and proceeding [counter]clockwise around the boundary blocks

• The background parameters of a non-homogenous boundary block are estimated from the two nearest background-homogenous blocks

• Once the background parameters of the non-homogenous boundary block have been estimated, that block is marked background-homogenous


Step 4:• Interior blocks are then examined in a certain sequence

that assures that the block being examined has 3 adjacent blocks that have already had their background parameters estimated. One of these blocks must be horizontally adjacent, one must be vertically adjacent, and the third must be diagonally adjacent between the two other adjacent blocks.


X XX Y

X X XY

XX Y X

Step 4 (cont.):• A homogenous interior block is considered object-

homogenous if its measured graytone mean and the background mean of a vertically or horizontally adjacent block are significantly different. Otherwise it is background-homogenous.

• Background parameters of background-homogenous blocks are measured, while background parameters of object-homogenous or uncertain blocks are estimated.


Step 5:• Once the background parameters of each block have

been measured or estimated, calculate the left and right shoulder thresholds of the background distribution using the following formulas:

Where is a prespecified constant (in the case of this study’s experiments, 2.5)

• Assign these threshold values to the center of each block


leftleft Tvt rightright Tvt v

Step 5:

(dots indicate positioning of threshold values)


Step 6:• Left and right shoulder thresholds for a pixel (x, y) are

obtained by bilinear interpolation of the left and right shoulder thresholds assigned to the four block centers surrounding that pixel

• Let L[g(x,y)] be the log-transformed graytone of pixel (x,y)• Pixel (x,y) is darker than the background if L[g(x,y)] < tleft

• Pixel (x,y) is lighter than the background if L[g(x,y)] > tright


Step 6 (cont.):• In order to preserve local brightness relationships between

object pixels and background pixels, keep two floating histograms as pixels are being classified: one of bright pixels (Fb) and the other of dark pixels (Fd).• The abscissa of these floating histograms represents

the difference between the log-transformed data value of a pixel and the log-transformed data value of the corresponding background shoulder threshold at that pixel


Step 7:• Once every pixel has been classified by the background

extraction procedure, calculate the mean and the left and right standard deviations of Fb and Fd.

• Calculate left and right shoulder thresholds of Fb and Fd:


dleft

ddleftt

bleft

bbleftt

dright

ddrightt

bright

bbrightt

Final pixel classification can then be obtained from:


0),( bright; saturated is ),( pixel :else1),( bright;han brighter t is ),( pixel : then}255)],([{ elseif

2),( bright; is ),( pixel : then)}),(()],([{ elseif

3),( bright;-darker is ),( pixel : then)}),(()],([{ elseif

5),( ;background is ),( pixel : then)},()],([{ elseif7),( dark;-brighter is ),( pixel : then)},()],([{ elseif

8),( dark; is ),( pixel : then)}),(()],([{ elseif

9),( dark;n darker tha is ),( pixel : then)}),(()],([{ if

yxlyxyxlyxyxgL

yxlyxtyxtyxgL

yxlyxtyxtyxgL

yxlyxyxtyxgLyxlyxyxtyxgL

yxlyxtyxtyxgL

yxlyxtyxtyxgL

brightright

bleftright

right

left

drightleft

dleftleft

(Where represents the label associated with each of the descriptive categories)

),( yxl

Experimental Results

• See figures in handout

AssessmentPros Cons

•No a priori knowledge req’d•Works with non-uniform backgrounds•Performs successfully without parameter adjustments or human interaction•Works under different lighting conditions•Doesn’t matter if the object is darker or brighter than the background… there doesn’t even have to be an object

•Only tested on industrial scenes and moderately complex outdoor scenes… unproven on natural scenes•Estimation of background parameters may be error-prone•Only works on grayscale images

Possible Improvements

• Color image support• Processing spatial and local information in

addition to brightness information to reduce misclassified pixels

• Robust and real-time performance on natural scenes

robust statistical method for background extraction in image segmentation

Documents