G e o - s p a t i a l I n f o r m a t i o n

Sc ience (Quar te r ly )

Volume 8,Issue 1

March 2005

Article ID:1009-5020(2005)01-039-06 Document code: A

Deblurring Texture Extraction from Digital Aerial Image by Reforming "Steep Edge" Curve

W U J u n C H E N Danqing

ABSTRACT Texture extract from digital aerial image is widely used in three-dimensional city modeling to

generate "photo-realistic" views. In this paper, a method based on reforming "Steep edge" curve, which clear-

ly explains how the diffraction of the sunlight makes digital aerial image blurring, is proposed to deblur the

texture extraction from digital aerial image, and the experiment shows a good result in visualization and auto-

mation.

K E Y W O R D S "Steep edge" curve; image deblurring; city modeling

CLC NUMBER P231

Introduction

Texture extract from digital aerial image is

widely used for three dimensional city modeling

to generate "photo-realistic" views. Image de-

blurring is one of main problems in image pro-

cessing, and two classical methods, image en-

hancement and image restoration, are usually in-

volved. Both image restoration and image en-

hancement can upgrade the input blurred image

to some extent. Usually, the image restoration

is specialized in deblurring image caused mainly

by point spreading or linear image motion with

constant speed, though its implementation is

complicated in algorithm and huge in calcula-

tion. As for the image enhancement, the result

is often not satisfactory when only applying gen-

eral linear or non-linear transformation, which

may be implemented comparatively simply, of

gray-level to whole image to gain improved con-

trast or adjusting gray-levels. This paper focuses

on the basic structure--line feature in digital

aerial image and by reasonably relating image

blurring to " steep edge" curve, which deter-

mines how each line feature appears in digital

aerial image and clearly explains how diffraction

of sunlight makes digital aerial image blurring.

1 Blurring model in digital aerial

image

Think of two adjacent parallel thin lines drawn

with black ink on a piece of dried white paper.

At first, even at a glance, the two lines are easi-

ly distinguished because of the distinct border-

line. However, once dried white paper is affect-

ed with damp the two blank lines are obviously

becoming blurred because of ink-spreading,

which makes the original blank line scale-up in

width and scale-down in contrast. Compared to

the ink-spreading on white paper, the diffraction

of sunlight at shooting is the main reason for

blurring in digital aerial images, which leads to

"wider" edge and lower contrast for each line

feature in digital images. The mathematical

analysis on how the diffraction of sunlight makes

digital aerial image blurred can be defined in

three steps: point spread function, line spread

function, "steep edge" curve.

Received on July 16, 2004.

WU Jun, associate professor, School of Geo-information Science and Engineering, Shandong University o{ Science and Technology, Qingdao 266510, China.

E-mail= wujun93161@163, corn

40 Geo-spatial Information Science (Quarterly)

1) Point spread function(PSF)

It is well known that the projection in image for

any bright space point is not a single point but circu-

lar facula because of the diffraction of sunlight at

shooting. Usually, we use the point spread function

(PSF) to describe the intensity distribution of the

facula and mathematically express it with two di-

mensional coordinates P ( x , y ) , where ( x , y )

stands for the location for any point within the

range of facula~ P stands for the intensity value

for point located at (x ,y ) . Actually, when tak-

ing the intensity I as Z axis,the PSF P ( x , y ) can

also be in three dimensional form. Fig. 1 shows

its front view and elevation view.

f~ i i

X 0

Fig. 1

Y

O

Point spread function

II ;r ~ �9 x

2) Line spread function(LSF)

Furthermore, one straight line can be consid-

ered as the collection of infinite single points and

certainly its projection in image is the combina-

tion of infinite image "point". Then, according

to PSF, the projection in image for one bright

space line is no longer one straight line but ex-

tending in both sides with different scope. If

aligning y axis with the length of projection and

letting x axis be the intensity along the width of

projection, we can define line spread function

(LSF) A ( x ) , which is one-dimensional and un-

symmetrical in mathematical form, to describe

the intensity distribution of space straight l ine's

projection in image. Fig. 2 shows its front view

and elevation view.

A (x) �9

ol = X

y

�9 X

Fig. 2 Line spread function

3) "Steep edge" curve (SEC)

For any object in a three-dimensional space, its

boundary is always located on some space plane,

and when the object is opaque, each boundary of

the object can be looked as the division line of

two half planes: one is visible and the other is

shadowed. Naturally, the visible half space can

be supposed to be the combination of infinite ad-

jacent bright lines parallel to the boundary and

the projection, called "steep edge" curve because

it is similar to push one edge on emulsion to ex-

pose, of the boundary is the combination of pro-

jection of those parallel lines1,2.,3,4..-in the vis-

ible half space plane. As a result, the intensity

I(xo) of any point located at x0 in SEC is the

summary of intensity of corresponding point in

different LSFs and the final shape of SEC is like

stretched "S" (see Fig. 3).

/ ( , ~ E d g e f I I ((d/////////////'''''''''~" 1 2 3 . ~

I I

lens l(x)

I ,

4~1 3i~1

X O X 0

Fig. 3 Formation of "steep edge" curve

From Fig. 1 to Fig. 3, we can draw a conclu-

sion that, through the optical imaging system, it

WU Jun, et al/Deblurring Texture Extraction from ... 41

is the "intensity accumulation" caused by the

diffraction of sunlight at shooting that change

the theoretic step intensity distribution of line

feature into gradually sloped SEC (Fig. 4) and

thus lead to blurred digital aerial image. This

phenomenon can be explained by considering

people's visual system. Many experiments have

showed that people 's visual system trends to

amplify or minify the intensity difference to de-

tect interested region in image. In addition, the

eye ' s sensitivity to interested regions is related

to the background to a great extent. The greater

the intensity difference between object and its

background is, the higher the differentiation is

in detail. However, opposite to people's visual

system, the forming of SEC is actually a reverse

process, which always minifies the intensity

difference between two sides of the boundary.

And the smoother the shape of SEC is, the

smaller the intensity difference is. Once the in-

tensity difference is beyond eye ' s minimum

adaptability and sensitivity to the intensity, then

we get blurred image.

/ X T X 0 X B

X

Fig. 4 "Steep edge" curve and theoretic step distribution

2 Implementation of deblurring tex-

ture from digital aerial image

Intuitively, the above blurring model shows

that a blurred image can be deblurred by refor-

ming SEC of line feature into its theoretic step

distribution. Although absolutely restoring SEC

into theoretic step distribution is difficult (and

unnecessary), we can adjust the intensity value

of each pixel in SEC to make SEC much steeper

and approximate to the theoretic step distribu-

tion. More specifically, three key problems are

involved in reforming SEC automatically.

(~) Locating step center x0 in SEC. We suppose

the step center x0 in SEC is the point with big-

gest intensity difference along SEC and is identi-

cal with the jumping point in the theoretic step

distribution (Fig. 4). Then, according to the

definition, the step center x0 can be automatical-

ly located by use of highty precise edge detector

such as CANNY, LOG, etc. Once the step cen-

ter x0 is located, can the corresponding SEC be

attained by starting at the step center x0 and

tracing along the maximum gradient direction

(vertical to the line feature).

@ Locating boundary point xr and x , (xr

x , ) in SEC. Theoretically, whole SEC is un-

bounded away from its step center x0 and we de-

fine boundary point xr with minimum intensity

in SEC and xB with maximum intensity in SEC.

In practice, however, the main slope of SEC is

within very narrow neighborhood around its step

center x0 and we can simply consider the SEC a

monotone curve centered at x0 and take the first

intensity minimum/maximum as boundary Point

XT/X~ (see Fig. 4). Obviously, with the above

simplification, the boundary point x r / x , can be

easily obtained by calculating intensity difference

starting from the step center x0 and comparing

its sign. Actually, only the "intercepted" SEC

determined by Xr, x , and :Co is what we empha-

size on in this paper and will have it reformed.

@ Selecting "reforming function" for SEC. Let

f ( x ) be the original SEC and g ( x ) be the re-

formed SEC. For each point x in SEC, the re-

forming transformation can be expressed as

g(x ) = A ( x ) [ f ( x l ) -- f (x0 ) ] + f ( x o ) (1)

where A (x) is called "reforming function" and

related to f ( x ) ; f ( x o ) is called "reforming cen-

ter" and is a constant in its neighborhoodl f ( x l )

is called "reforming boundary", xl =Xr/XB.

To avoid the unnecessary influence of Mach E~] ,

"reforming function" A (x ) for SEC must be

carefully selected to make sure that the reformed

g(x ) is not only steeper than the original f ( x )

but also have the same continuity, smoothness

and monotone consistency. Here, we select the

function y = a t a n ( k x ) , ( x ~ R , k E R or k ~ 0 ) as

the " reforming function". Compressing SEC

along y axis into [-~/2,n/2~ and comparing it

42 Geo-spatial Information Science (Quarterly)

with the selected "reforming function", we can

find that the function y=atan(le:c) has good geo-

metrical condition as we expected and is specially

good in "steepening" (see Fig. 5).

Y

y=a tan(k, x ) ~

. . . . . . . . . . . . . . . ~ - - - - - +~/2

y=a tan(k2x) - -

Steep edge curve g , 9~

. . . . . . . . . . . . . . . . . . . . . . . ~ / 2

Fig. 5 "Transform" function y = a tan(kx)

and "steep edge" curve

Generally, because of the perspective relation,

geometrical correction is necessary for textures

extraction from digital aerial image to overcome

distortion in texture rending [2]. As a result,

many line features which are located in windows

or doors appear to be vertical and/or horizontal

in geometrically corrected textures, and with

this obvious direction information, main line fea-

tures in texture and step center z0 in each SEC

can be easily attained by establishing and analy-

zing the histogram of the edge point in texture,

which is similar in strategy to the well-known

Chinese Characters Recognition Technology [3].

In a word, considering the necessary color space

transformation [4] , our algorithm on texture de-

blurring by reforming SEC can be stepped as fol-

lows (see Fig. 6).

nnnnflnnn

Fig. 6 Procedure for

1) Color space transformation RGB-> HIS

(Fig. 6-@@). Because colored texture is usually

expressed in RGB color space and we only focus

on intensity images here, the color space trans-

formation RGB--~ HIS is necessary to gain

blurred gray-scale texture f ' (x , y) for advanced

operation by only considering I vector in HIS

color space. The corresponding color space

transformation formula is ['~] �9

�9 M i n ( R , G , B ) = R:

S = 1 - - R / I ; H =

�9 M i n ( R , G , B ) = G-

S = l - - G ~ 1 ; H =

�9 M i n ( R , G , B ) = R:

I = ( R + B 4 - G ) / 3 ;

( B - - R ) ~ 3 . ( I - - R ) + I

I = ( R + B + G ) / 3 ;

( B - - G ) ~ 3 . ( I - - G ) + 2

I = ( R 4 - B + G ) / 3 ;

S = 1 - - R / I ; H = ( B - - R ) / 3 . ( I - - R ) §

2) Establishing horizontal/ vertical gray-scale

histogram M for texture f ' (x , y) (Fig. 6-@).

Take the height/width as X axis, and the aver-

age gray-scale of all horizontal/vertical pixels as

Y axis, then the horizontal/ vertical gray-scale

histogram M for texture f ' ( c r , y ) can be estab-

texture debluring operation

(;',

lished. Because main line features in windows or

doors always parallel to texture 's boundary, the

histogram M actually contains corresponding

SEC for all line features and with the accumala-

tion and average of gray-scale, the influence of

random noise will be greatly reduced.

3) Attaining line features by edge detecting and

establishing/analyzing the histogram of edge

point in texture. First, CANNY Edge Detector

is used to extract edge point in single pixel and

generate an edge map for whole texture

(Fig. 6-@). Then, the horizontal/vertical histo-

gram of the edge points in the edge map can be

established by taking the height/width of the

edge map as X axis and the number of all hori-

zontally/vertically edge point as Y axis

(Fig. 6-@@). Last, we filter out each "top of

wave" from the histogram as precise location of

line features.

4 ) Tracing out and reforming SEC (see

Fig. 6 @)from located line features and gray-

WU Jun, et al./Deblurring Texture Extraction from -.. 43

scale histogram M. This operation includes the

following steps.

�9 Locating point xo, Xr and x~ in SEC. Ac-

cording to the definition of the step center in

SEC, Xo is actually the location of line feature

detected by CANNY operator in this paper. Fur-

thermore, after x0 has been located in gray-scale

histogram M, the point X r / X , can be located by

orderly comparing points' (because of very nar-

row slope range in SEC, only continuous four

pixels adjacent to x0 are considered here) inten-

sity value in gray-scale histogram M with the in-

tensity of x0 and finding the point with maxi-

mum/minimum intensity difference. Namely, let

M0 be the intensity value of x0, Ms be the inten-

sity value of x . , Mp be the intensity value of xp

E x 0 , x . ~ , then we have M0% Mp%Ms and

similarly, for x~ ~ ~ x r , x o ~ , M r % M p % M o is

necessary.

�9 Estimating "reforming function" A ( x ) . Ac-

tually, estimation to function A ( x ) : y =

a tan(kx) is the estimation to parameter k ( k ~ R

and k > 0 ) . Then define the parameter L i m = ~ / 2

- - a t an (k �9 X ) ( X = x r , X . ) , and obviously, k

and Lira are similar in determining how quickly

the A ( x ) approaches to -r-~/2 at the point X and

how steep the A ( x ) is at the original point.

Therefore, we can use Lira to calculate k be-

cause of its narrow varying range ( [0 .01 ,0 .05~

is good enough in the experimental result) .

Take f ' ( x o ) as the "reforming center", let M0 =

f ' ( x o ) , L e n r = xo -- x r , L e n s = x ~ - - x o , t = Xo +-

0.5( i f Len j3>Lenr ,4 - ;else . . . . . ) , translate the

SEC to t and s u p p o s e , L e n u > L e n r , A ( x ) ~ O ,

then the parameter k is estimated as follows.

atan(k �9 x , ) = r r / 2 - L i n ~ k =

tan(n/2 -- Lim ) /x ,=>k = - - c tg(Lim ) / x . (2)

By substituting Eq. ( 2 ) i n t o Eq. ( 1 ) a n d uniti-

z ing,A(x) can be re-expressed as:

y = 2 �9 a t a n ( - - c t g ( L i m ) x / x ~ ) / n

(Lira > O,x~ > O,y ~ [0 ,1 ] ) (3)

�9 Intensity adjustment for each point xj ( X r %

xj% xs) in SEC. Actually, Eq. (3) gives differ-

ent proportion with which we scale the SEC and

the final intensity variation for each point in SEC

have to be multiplied by intensity difference be

tween Mr and Ado or M~ and M0, namely,

Eq. (1) can be rewrite as:

M r ~ M , and A(x) ~> 0.

g (x ) = A ( x ) ( M r --Mo) +Ado

MT ~> MB and A ( x ) < 0:

g ( x ) = A(x)(M~ M0)-I-M0

MT % MB and A ( x ) % 0..

g (x ) = A(x) (Mr -- M0 ) -t- M0

MT % MB and A(x) ~> 0.

g ( x ) = A ( x ) ( M ~ - -Mo) +M0

5) Color space transformation HS--~ RGB

(Fig. 6-@). To gain color texture, the color

space transformation HS--*RGB is implemented

to deburred gray-scale texture f ' (x , y) and the

corresponding transformation formula are :~2 �9

�9 O ~ H ~ I : R = I . ( 1 - k 2 . S - - 3 . H . S ) ;

G = I . ( 1 - - S - 1 - 3 . H . S)~ B = I . ( 1 - - S )

�9 l ~ H % 2 ; R = I . ( 1 - - S ) ;

G = I . ( 1 + 5 . S - - 3 . H . S) ;

R = I . ( 1 - - 4 . S § H . S )

�9 2 ~ H % 3 : R = I . ( 1 - - 7 . S + 3 . H . S ) ;

G = I - ( 1 - - S ) ;

B = I . (1 + 8 . S - - 3 - H . S)

3 Experiment and conclusions

In this paper, several textures, which are ex-

tracted from digital aerial image and geometrical-

ly corrected (see Fig. 7 O Q @ ) , are tested with

the proposed strategy and algorithm. For each

step of the algorithm, corresponding temporary

output result is demonstrated in Fig. 6. Fig. 6-@

shows the original blurring texture. The gray-

scale texture f ' ( x , y ) (Fig. 6-@) is the gray-

scale representation for original blurring texture

after applying color space transformation RGB-+

HIS to it and using l vector as the value of pix-

el. Fig. 6-(3) is the gray-scale histogram of tex-

ture f ' ( x , y ) by taking its width as the horizon-

tal axis and its height as the vertical axis. Based

on the gray-scale histogram, SEC for each verti-

cal line feature can be exactly traced out when

the corresponding step center x0 in SEC is loca-

ted. Fig. 6-@ shows the edge map of texture

f ' ( x , y ) after applying the edge detection algo-

rithm CANNY to it. Because of C A N N Y ' s high

44 Geo-spatial Information Science (Quarterly)

precision in location, those edge points in the

edge map are simply thought as step center at0 in

SEC of certain line feature. Fig. 6-@ is the edge

point his togram of the edge map by taking its

width as the horizontal axis and the height as the

vertical axis. With the noise removal operat ion,

the edge point h is togram is obviously improved

(see Fig. 6-@) and the " top of wave" separated

from the improved his togram is thought as ex

pected step center :r0 for each ESC. Once the

step center ac0 is found, we can easily search out

@ Z

SEC from the gray-scale h is togram of texture

f ' ( a c , y ) and thus gain deblurred gray-scale tex-

ture (Fig. 6-@) by SEC steepening operation.

Finally, the deblurred color texture is generated

by applying color space t ransformat ion HIS--*

RGB to deblurred gray-scale texture. The exper-

imental results (Fig. 8) show .that the proposed

method based on reforming the " s t e ep edge"

curve to deblur texture extract ion from digital

aerial image is reasonable and good for automa-

tion.

re,

Fig. 7 Original 3D building without texture debluring

REFERENCES

1 Zhang Y J (1999) Image engineer-image processing

and analyzing. Beijing: Publishing House of Tshing-

hua University. (in Chinese)

2 Wu J (2004) The research on rapidly reconstructing

texture for facades in 3DCM: [-Ph. D dissertationS.

Wuhan: Wuhan University. (in Chinese)

3 Zhang X Z (1992) Technology on chinese characters

recongination. Beijing: Publishing House of Tshing

hua University. (in Chinese)

4 Yi Y H (2000) Research on adjusting hue of digital

color remote imagery: [-master thesis~. Wuhan: Wu-

han Technical University of Surveying and Mapping.

(in Chinese)

5 Gruen A, Wang X (1998) CC-Modeler: a topology

generator for 3-D city models. ISPRS J. Photogr.

Fig. 8 Improved 3D building with texture debluring

and Rein. Sens. , (53)5:286-295

6 Pratt W K (1972) Generalized wiener filtering compu-

tation techniques. IEEE Trans. on Computers, C-21

(7) :636-641

7 Hunt R (1973) The application of constrained least

squares estimation to image restoration by digital com-

puter. IEEE Trans. on Computers, (;-22(9) :805-812

8 Yu H Q (1999) Photography and aerial photogramme-

try. Beijing: Publishing House of Surveying and Map-

ping. (in Chinese)

9 Sun M, Chen J (2000) Reiew of data capture method

in 3D city model. Bulletin o f Surveying and Map-

ping (11) :4-6 (in Chinese)

10 Wan J H (2001) Research on modeling for three-di-

mensional urban geographical information system:

[-Ph. D dissertationS. Wuhan: Wuhan University. (in

Chinese)