Roof Edge Detection using RegularizedCubic B-Spline Fitting

Kung-Hao Liang & Tardi Tjahjadi Yee-Hong YangDepartment of Engineering, Department of Computer Science,University of Warwick, University of Saskatchewan,Coventry CV4 7AL, Saskatoon, Saskatchewan,United Kingdom Canada S7N 5A9Correspondence to T.Tjahjadi, Fax: +44 1203 418922, email: tt@eng.warwick.ac.uk1

SummaryThe extraction of roof edges requires the reconstruction of the original signal from a lattice ofsampled values of grey level (i.e. the sampled image). This is an ill-posed task and therefore,adequate constraints of regularization have to be incorporated. A scheme employing one-dimensional Regularized Cubic B-Spline (RCBS) �tting has been used successfully on the taskof step edge detection, in which the regularized �tting is transformed into a quadratic energyequation to simplify the computation. However, this scheme has three major limitations:it is non-linear; it has a limited accuracy; and it is computationally expensive. This paperpresents a modi�ed scheme which overcomes these limitations. The modi�ed scheme employsthe 1-D RCBS �tting on the horizontal and the vertical orientations of a window of an imageto generate two 1-D signals, which provide su�cient information about the local property ofthe sub-image for roof edge detection. This paper also shows that the regularization factor �should be su�ciently large to make the computation well-conditioned.The modi�ed RCBS �tting is used in a roof edge detector. Two parameters, the regularizationfactor � and a threshold, are to be given a priori. Since � determines the degree of smoothing,a larger value of � should be used when the signal is noisy. Conversely, if the signal is noise-free, a small value of � should be used so as to preserve the signal. The threshold re ects thesize of an edge, i.e. the larger the edges to be detected, the larger the threshold. The False-Correct Ratio is used to quantitatively evaluate the performance of the roof edge detector onsynthetic images. The roof edge map of a real image with step and roof edges shows that theproposed roof edge detector is more sensitive to arbitrary signal than a step edge detector.Abstract-A scheme employing one-dimensional Regularized Cubic B-Spline (RCBS) �tting[18] has been used successfully on the task of step edge detection. The regularized �ttingis transformed into a quadratic energy equation to simplify the computation. This schemehowever has three major limitations: it is non-linear; it has a limited accuracy; and it iscomputationally expensive. This paper presents a modi�ed scheme which overcomes theselimitations. The modi�ed scheme employs the 1-D RCBS �tting on the horizontal and thevertical orientations of a window of an image to generate two 1-D signals, which providesu�cient information about the local property of the sub-image for roof edge detection.Experimental results show that the scheme of roof edge detection is very sensitive to smallsignals.Key words-Edge detection; roof edge; regularization; B-Spline; curve �tting; quadratic en-ergy equation. 2

1 IntroductionA digital image is composed of a two-dimensional (2-D) array of grey levels which representthe sampled light intensity in light intensity images, or the depth value in range images. Inthese images, a roof edge is an important feature, which is required by various image process-ing tasks [1]-[9]. A roof edge is generally de�ned as a discontinuity in the �rst order derivativeof a grey-level pro�le [7, 10]. This de�nition is adopted in this paper. However, in certaininstances a sign change in the �rst order derivatives on the two sides of the discontinuity isalso required [3], i.e. f 0(to+) � f 0(to�) � 0, where the discontinuity occurs at the positionto. Thus, a roof edge is a local maximum or a local minimum of a grey-level pro�le, whichare referred to as a ridge or a valley, respectively, in the literature [3, 5, 9] Fig.1 illustratesan example of a roof edge occurring at the position 0 on the coordinate of the principleorientation, which will be de�ned later in the paper.The importance of roof edge detection has been discussed in the literature. For example, it isargued that human facial expressions in an image are better depicted by valleys of grey levels(i.e. roof edges) than by step edges [8]. The extraction of ridges and valleys from digital terrainmodels plays an important role in lithology, structural geology and geomorphology [9]. Roofedges are also important in the analysis of aero-magnetic images [5] and the segmentation ofrange images [4, 7].(Insert Fig.1.)Ill-posed formation of early vision tasks, including several roof edge detection schemes, is acommonly encountered problem [11]. This is because the tasks of early vision involve thereconstruction of the original signal from a lattice of sampled values of grey level (i.e. thesampled image). A task is ill-posed if it violates the three requirements of well-posedness:(1) a solution exists, (2) the solution is unique, and (3) the solution depends continuouslyon the input data [11]. To transform an ill-posed task into well-posed, the technique ofregularization is generally employed to impose adequate constraints which enable a uniqueand stable solution to be determined [11]-[14].Various schemes for detecting roof edges (or ridges and valleys) have been proposed. Pearsonand Robinson [8] propose a valley detector, which uses a set of criteria to examine the relativegrey levels of a group of neighbouring pixels. These criteria determine the local extremum ofthe second order derivative of the underlying grey-level pro�le, which indicates the occurrenceof a roof edge. Unfortunately, this scheme requires three thresholds to be given a priori andtherefore, a large amount of trial-and-error is required to determine the adequate thresholds.Furthermore, it is an ill-posed task because the resultant edge map varies in a manner whichdoes not depend continuously on the input signal, even though the level of noise is small.The active contour scheme [15] for the extraction of various contours is an example of aregularized method. To extract the contour of valleys, this scheme minimises the functionalE: 3

E = Z (�(s)jvs(s)j2 + �(s)jvss(s)j2)ds+Xs g(v(s))where s denotes the spatial coordinates of the snaxel; �(s) and �(s) are weightings; vs(s)and vss(s) are, respectively, the �rst and the second order derivatives of the contours v(s)of valleys. g(v(s)) denotes the grey level of the image along v(s). The solution v(s) is thendetermined by minimising E.The Haralick schemes of step [16] and roof [3] edge detection employ the technique of least-square optimisation. They use a set of discrete orthogonal polynomials to reconstruct theunderlying grey-level surface of the image. Step edges and roof edges are then determinedwhen the corresponding criteria are applied. Least-square optimisation methods minimise awell-posed quadratic energy function: de�ne the energy e to be k Ac�g k2, where k � k denotesthe 2-norm; g is the input signal; c is the solution of the optimisation; and matrix A mapsthe signal from c space to g space. Hence, the minimum of e occurs when c = (ATA)�1ATg,which indicates that c is continuously dependent on g. This paper will show a regularizationformula can be transformed into a quadratic equation to simplify the computation.To design an optimised 1-D kernel which produces maximum responses at the positions ofstep or roof edges, Canny [2] proposed a method of computational optimisation, where theoptimisation is achieved by the suppression of noise and the localisation of edges. He showedthat the optimised kernel for step and roof edges can be approximated by the �rst and thesecond order derivatives of a Gaussian respectively. The �rst order derivative gives a measureof the slope of a 1-D signal, and has been used to detect step edges. The second orderderivative is used in this paper to indicate the presence of roof edges (see Section 3). Thefact that the optimised 1-D roof edge detector is the second order derivative of the Gaussiansuggests that the Laplacian of Gaussian (LoG) kernel, an augmentation of the 1-D secondorder derivative of the Gaussian to 2-D, is a candidate for roof edge detection on images.Note that the LoG kernel has been used for step edge detection, and can be approximatedby the Di�erence of Gaussian (DoG) kernel when the di�erence between the two standarddeviations of the Gaussians tends to zero [17]. In addition, Torre and Poggio showed thatregularisation can be achieved by convolving the data with a cubic-spline �lter, which has ashape similar to a Gaussian [14].Recently, the technique of Regularized Cubic B-Spline (RCBS) �tting has been proposed forthe tasks of single-scale [18] and multiscale [19] step edge detection. All the tasks formulatedusing the RCBS �tting are regularized, and hence are well-posed. However, the originalRCBS scheme for edge detection has three limitations. First, before the �tting is applied,an equal-weighted averaging of grey levels is applied on the neighbouring pixels along theorientation perpendicular to the �tting (see Fig.2A). This is to suppress noise along both theorientation of the �tting and perpendicular to the �tting. However, the process of the equal-weighted averaging is non-linear in the sense that choosing di�erent number of pixels forthe averaging a�ects the result of the �tting non-linearly. Second, the Prewitt edge detector,an ill-posed operator, is used to determine the edge orientation, along which the �tting is4

applied. Hence, the accuracy of the RCBS scheme is also limited by the Prewitt operator.Third, the required computation of the RCBS scheme is high because the �tting is appliedalong various orientations, along which the pixels required for the �tting do not coincide withthe grid pixels of the image. Interpolation is thus required, which increases the computationtime. These limitations degrade the performance of the RCBS scheme.(Insert Fig.2.)This paper presents a modi�ed RCBS scheme which overcomes the above three limitations.This scheme employs the 1-D RCBS �tting on the horizontal and the vertical orientationsof a window of an image to generate two 1-D signals (see Fig.2B), which provides su�cientinformation about local property of the 2-D sub-image to enable edge detection. A roof edgedetector is then designed based on this modi�ed RCBS scheme.The organisation of this paper is as follows. Section 2 describes the principle of the RCBS�tting, and shows how a regularized formula is transformed into a quadratic energy equation.Section 3 presents the modi�ed RCBS scheme and the roof edge detector. Section 4 discussesthe roles of the various parameters in the roof edge detector, and presents an evaluation ofthe performance of the detector. Finally, Section 5 concludes this paper.2 Regularized Cubic B-Spline FittingThe RCBS �tting, which uses the principle of the regularization theory [11, 13], is used toreconstruct a continuous signal (e.g. the grey-level pro�le) from a set of sampled data (e.g.the sampled image) in a well-posed way. In 1-D, the process of regularization is generallyexpressed as a functional [18, 19]:E =Xj (f(xj)� g(xj))2 + � Z (d2f(x)dx2 )2dx; (1)where g(xj) denotes the grey level of an image at xj, the sampled pixels with a spatialcoordinate x. The parameter � represents the degree of the regularization e�ect. The solutionf (i.e. the �tted curve) is determined by minimising the functional E.The space of the solution for a regularized functional is constrained by suppressing the secondorder derivative of f, which corresponds to the second term on the right hand side of equation( 1). The justi�cation is that the spatial frequency of an image is physically bounded [14], thathigh frequency signals tend to be noise and should be suppressed. The variational approach iscommonly used to solve various functionals [13]. However, techniques such as curve �tting cantransform a functional problem into a problem of solving a quadratic energy equation, andthe extremum of which can be obtained easily by solving a set of �rst order linear equations.Among various curves used for �tting, a spline under a certain premise has a least value ofthe second order derivative according to the Holladay theorem [20]:5

Holladay TheoremLet 4 : a = x1 < x2 < ::::: < xM = b, and a set of real numbers fg(xk)g(k = 1; 2; :::::;M)be given. Then among all the functions f(x) with a continuous second derivative on [a,b],and such that f(xk) = g(xk), the spline function S4(x) with junction points at xk and withS004(a) = S 004(b) = 0 minimise the intergalZ ba (f 00(x))2dx:Here the spline function S4(x) is de�ned to be composed of cubic polynomials in each sub-interval xk�1 � x � xk (k = 2; 3; :::::;M), and satis�es S 04(xk+) = S 04(xk�) as well asS004(xk+) = S 004(xk�), (k : 2; 3; ::::;M � 1).Assuming that the sampling rate for a signal is �xed, then every interval [xk�1; xk] in equation(1) is equidistant. Hence a spline S4(xk) can be approximated by shifting and the superpo-sition of a third-order basis function provided that the derivatives of the basis function arezero at the boundary points. This concept motivates the RCBS �tting [18, 19], which usesthe cubic B-spline (Fig.3) as the basis function: = 8><>: 0 2 � jxj�jxj3=6 + jxj2 � 2jxj+ 4=3 1 < jxj < 2jxj3=2� jxj2 + 2=3 jxj � 1By shifting the basis function over the interval [1,M],ei(x) = (x� i) 0 � i �M + 1 i 2 Z:The solution f in equation ( 1) is then represented as the linear combination of the basisfunctions: f(x) = M+1Xi=0 ciei(x);where ci is the coe�cient of the basis function ei.In this way, the regularized functional in equation (1) becomes a quadratic energy equation:E = MXj=1(M+1Xi=0 ciei(xj)� g(xj))2 + � MXj=1(M+1Xi=0 cid2ei(xj)dx2 )2; (2)The �rst term of equation (2) is the summation of the square of the error between f(xj) andg(xj), while the second term determines the summation of the second order derivatives in xj.Minimising E represents the reduction in the error of the �tting as well as the second orderderivatives in xj, and � is used to adjust the relative importance between the two terms.Given g(xj) and �, ci is then determined. The solution f and its derivatives can then be6

easily obtained by multiplying ci with the corresponding derivatives of ei:f (n)(x) = M+1Xi=0 cie(n)i (x)where f (n)(x) and e(n)i (x) represent the nth order derivatives of f(x) and ei(x), respectively.(Insert Fig.3.)2.1 Quadratic Energy EquationDe�ne C � 26666664 c1:::cM+1 37777775 A � 26666664 e1(xo) : : : : : eM(xo): :: :: :e1(xM+1) : : : : : eM(xM+1) 37777775A2 � 26666664 d2e1(x0)dx2 : : : : : d2eM (x0)dx2: :: :: :d2e1(xM+1)dx2 : : : : : d2eM (xM+1)dx2 37777775 G � 26666664 g(x1):::g(xM) 37777775 ;where A, A2 and G are known, and C is the unknown variable. In this way, equation ( 2)can be represented asE = (ATC �G)T(ATC �G) + �(AT2C)T(AT2C)= (ATC)T(ATC)�GTATC� (ATC)TG+GTG+ �CTA2AT2C= CT(AAT + �A2AT2 )C � 2CTAG +GTG:De�ne P = AAT + �A2AT27

= h A A2 i " I 00 �I # " ATAT2 # :Since the row vectors of [A A2] are independent of each other, the matrix P is positivede�nite as long as � > 0. Therefore,E = CTPC � 2CTAG +GTG;which is a standard quadratic energy equation. Its graph is a paraboloid in the (M+2)dimensional hyperspace, and the minimum of E occurs at C = P�1AG [21].A quadratic energy equation is well-posed. However, it can be ill-conditioned, which meansthat a small perturbation in the input signals results in a large variation of the output. When�! 0, P! AAT, which is a singular matrix because the rank of A is M, and the dimensionof the matrixAAT is M+2 by M+2 such that det(AAT) = 0. The singularity ofAAT causesthe computation of C, which requires the inverse of P, to be ill-conditioned. Since there is noclear boundary between ill-conditions and well-conditions [11], we use MATLAB to simulatethe �tting with a decreasing value of �. The results (Fig.4) show that when � equals to10�7; 10�10; 10�13, satisfactory �ttings are achieved. However, when � is further decreased to10�16; 10�19 as well as 0, the results of the �tting are unsatisfactory, i.e. the results do notrepresent roof edges. Similarly, � ! inf causes the computation to be ill-conditioned as well.(Insert Fig.4.)3 Design of a Roof Edge DetectorA roof edge is de�ned as a discontinuity in the �rst order derivative of a 1-D grey-level pro�lef [10]. Thus, the location of a roof edge corresponds to the extremum in the second orderderivative f 00, and the zero-crossing in the third order derivative f 000. The zero-crossings of f 000can accurately indicate the position of edges, but they are very sensitive to high frequencysignal such as noise. Thus, we introduce a threshold in f 00 to make the criteria more robustagainst noise. As a result the criteria for a roof edge are:1. f 00 � threshold;2. zero-crossing occurs in f 000.Since an image is a 2-D signal, the above 1-D criteria cannot be used directly. There aretwo approaches to solve this problem: (1) augment the 1-D criteria to 2-D; and (2) use a1-D signal in an appropriate orientation to represent the 2-D image. In this paper the latterapproach is used to simplify the computation.8

3.1 Principal Cross SectionThe grey level of a 2-D image describes a 3-D surface. To use a 1-D signal f (a grey-levelpro�le) to represent this 3-D surface as far as roof edge detection is concerned, a cross-sectional plane perpendicular to the 2-D image, onto which f can be projected is needed. Wede�ne the cross-sectional plane which is also perpendicular to the isophote curves (i.e. thecurves which connect pixels of the same grey level) as the principal cross section. Also,the orientation of the principal cross section is de�ned as the principal orientation (seeFig.1).Let S(x; y) be the 3-D grey-level surface, where x and y are spatial coordinates. Let x bethe principal orientation, and y be the orientation of the isophote curve, then @S@y = @2S@y2 = 0.Given an arbitrary direction t inclined at an angle � to the direction of x, thenx = tcos� and y = tsin�:Therefore, dSdt = @S@x � cos� + @S@y � sin� = @S@x � cos�; (3)and d2Sdt2 = @2S@x2 � cos2� + 2� @2S@x@y � sin� � cos� + @2S@y2 � sin2� = @2S@x2 � cos2�: (4)We denote f 0P = @S@x ; f 00P = @2S@x2 ;and f 0� = dSdt ; f 00� = d2Sdt2 :Therefore, according to equations ( 3) and ( 4), the derivatives of f along � at a pixel (x; y)have the following relationships with those along the principal orientation:f 0�(x; y) = f 0P (x; y)� cos�;f 00� (x; y) = f 00P (x; y)� cos2�: (5)These relationships infer that f along the principal orientation (� = 0) has the maximum�rstand second order derivatives. Note that these relationships are obtained under the assumptionthat the isophote curves are parallel to the roof edges. Although this is not always true, theassumption generally approximates the real cases.9

3.2 Horizontal-Vertical DecompositionTo obtain the derivatives along the principal orientation (i.e. f 0P (x; y) and f 00P (x; y)), we de-compose a 2-D image into two 1-D signals which are perpendicular to each other, e.g. thehorizontal and the vertical components of the signal. The derivatives along the principal ori-entation are then determined from the derivatives of these signals. Given a cross section withan angle (� + 90o), equation (5) becomesf 0�+90o(x; y) = f 0P (x; y)� cos(� + 90o) = f 0P (x; y)� sin�;f 00�+90o(x; y) = f 00P (x; y)� cos2(� + 90o) = f 00P (x; y)� sin2�:Therefore,(f 0�(x; y))2 + (f 0�+90o(x; y))2 = (f 0P (x; y))2cos2� + (f 0P (x; y))2sin2� = (f 0P (x; y))2;f 00� (x; y) + f 00�+90o(x; y) = f 00P (x; y)cos2� + f 00P (x; y)sin2� = f 00P (x; y): (6)Equation (6) shows that f 0P (x; y) and f 00P (x; y) can be obtained from the derivatives along anytwo perpendicular orientations. Note the two formulae in equation ( 6) are identical to thede�nition of the gradient and the Laplacian, where f 0P (x; y) corresponds to the gradient, andf 00P (x; y) corresponds to the Laplacian. To simplify the computation, the two perpendicularorientations are chosen to be the horizontal and the vertical orientations so that the greylevels on the sampled image can be directly used for the �tting.3.3 Modi�ed RCBS scheme and roof edge detectionIn the modi�ed RCBS scheme, the RCBS �tting is applied without the non-linear process ofthe equal-weighted averaging, along the horizontal and the vertical orientations to reconstructtwo continuous grey-level pro�les (Fig.2B). The derivatives of the �tted curve f along thesetwo orientations are then obtained as described in Section 2. Since the basis function, thecubic B-spline, is a third order piecewise polynomial, the third order derivative of the �ttedcurve f is not continuous at the grid pixels (see Fig.3).The criteria for detecting a roof edge (Section 3) are then applied along the principal orien-tation. The �rst criterion requires f 00P , determined using equation ( 6), to be greater than athreshold. The second criterion requires a zero-crossing in f 000, i.e. f 000P (to+) � f 000P (to�) < 0.To simplify the application of these criteria, the third order derivatives are examined alongthe horizontal and the vertical orientations. If a sign change occurs along either of the twoorientations, the second criterion is satis�ed. The �rst criterion is then used to detect theroof edge. 10

The magnitude of f 00P is used as an indication of roof edges in the proposed scheme, where f 00Pis determined from two regularised 1-D signals, and equals approximately to the Laplacianof Gaussian 52G � f . The di�erence between f 00P and 52G � f is the shape of the kernel,i.e. a cross kernel (Fig.2B) is used to compute f 00P , while a square kernel is used to compute52G � f .The pseudo codes of the proposed roof edge detector are:BEGINInput (Image(x; y), �, threshold);Assign n = the size (in pixels) of the image;For (x = 1; 2; ::::n)(y = 1; 2; ::::n) doBEGINApply the RCBS �tting along the horizontal orientation;Determine f 00horizontal(x; y), f 000horizontal(x�; y), and f 000horizontal(x+; y);Apply RCBS �tting along the vertical orientation;Determine f 00vertical(x; y), f 000vertical(x; y�), and f 000vertical(x; y+);If (f 000horizontal(x�; y)� f 000horizontal(x+; y) < 0)or (f 000vertical(x; y�)� f 000vertical(x; y+) < 0)thenBEGINf 00P (x; y) = jf 00horizontal(x; y) + f 00vertical(x; y)jIf f 00P (x; y) > thresholdthen Set EdgeMap(x; y)=Edge;else Set EdgeMap(x; y)=Not an Edge;ENDENDOutput (EdgeMap(x; y)); 11

END4 Performance EvaluationIn the proposed RCBS scheme, two parameters, the regularization factor � and the threshold,are to be given a priori. It has been pointed out that �, which re ects the degree of smoothingon the signal, represents the scale of the processing [19]. However, a scale also represents thedegree of \globalness" or \localness" in the processing of an image, i.e. the standard deviationof the Gaussian pre�lter. Research on multiscale image processing aims to provide the linkagebetween global processing method and local processing method. One of the major interestsis to determine the appropriate scale(s) for the di�erent image processing tasks.In the proposed roof edge detector, � determines the degree of smoothing. Therefore a largervalue of � should be used when the signal is noisy. Conversely, if the signal is noise-free, asmall value of � should be used so as to preserve the signal. Several methods which adaptivelyadjust � according to the property of the signal have been proposed [12, 22]. These methodspropose various models which iteratively cause � to converge to an appropriate value. Thesemodels are qualitatively reasonable, e.g. � is a monotonically increasing function of the noiselevel. However, di�erent models result in di�erent values of � for the same signal, whichillustrates quantitatively that these models are only optimum according to their de�nitions.In this paper, a quantitative measurement of performance is used to indicate \how good"the performance of the roof edge detector is. The measurement depends on the de�nitionof \good", and the False-Correct Ratio (FCR) [18] is used for this de�nition. To determineFCR, edge pixels are classi�ed to be either true-positive, true-approximate, true-missing,neutral-extra, or false-positive. The number of falsely-detected pixels are regarded as thesum of true-missing and false-positive pixels, while the number of correctly-detected pixelsare regarded as the sum of the true-positive, true-approximate, and neutral-extra pixels. FCRis the ratio of the number of falsely-detected pixels to the number of correctly-detected pixels.In this sense, the lower the value of FCR, the better is the performance of the edge detector.A series of tests on various synthetic images using various parameters was conducted. Theparameters which generate an edge map with the least value of FCR are considered optimumparameters. The synthetic images contain a roof edge of a \ring" shape so that the perfor-mance of the edge detector on edges of all orientations can be measured. The regularizationfactor � was adjusted according to the noise level of an image, while the threshold usedre ects the size of an edge, i.e. the di�erence of the slopes at the two sides of the roof edge[10]. The larger the edges to be detected, the larger the threshold.In test 1, a synthetic image which contains a roof edge with slope of 20 at the both sides of theedge, i.e. the size of the edge is 40, is used to represent a typical roof edge (Fig.5A). The imageis contaminated with zero-mean Gaussian noise of various standard deviation (NSD), and theFCR's of the edge maps produced by the proposed roof edge detector are determined. In the12

noise-free case, the edge map has a FCR of 0 (i.e. the edge map is perfect in the context ofFCR) when the regularization factor is 0.1 and the threshold is 10. The threshold is then setto 10 for all the noise levels since the regularization factor alone should re ect the noise level.The optimum regularization factors and their corresponding FCR's are then measured undervarious NSD's. The results (Fig.6) show that the optimum regularization factor increases asthe NSD increases. The corresponding optimum edge maps are shown in Fig.5.(Insert Fig.5.)(Insert Fig.6.)In test 2, noise-free images are used and the size of the roof edge is decreased gradually so asto determine the e�ect of the threshold on the performance of the edge detector. The resultis shown in Fig.7. As the size of the roof edge decreases, the optimum thresholds becomesmaller. The perfect edge map (i.e. FCR=0) cannot be produced when the size is reduced to10. This is because as the size is reduced, the edge and non-edge points become more andmore similar, and hence more and more di�cult to distinguish. Fig.8 shows the correspondingoptimum edge maps.(Insert Fig.7.)(Insert Fig.8.)In test 3, a real image of \Trevor" (Fig.9A) is used as a test image, and the correspondingroof edge map generated by the proposed scheme is shown in Fig.9B. Fig.9C shows the stepedge map produced by the Haralick step edge detector [16] for comparison. It shows that thefeatures on the step edge map and the roof edge maps are di�erent. For example, the stepedge map shows the wrinkles on the shirt, while the roof edge map shows the stripes on theshirt. The pattern on the tie appears to be better detected using the proposed detector thanthe Haralick step edge detector. Note that a step edge represents the �rst order extremum ofthe signal, while a roof edge re ects the second order extremum. Hence roof edge responsesare more sensitive than step edge responses to an arbitrary signal. This is why the roof edgemap contains more random dots.(Insert Fig.9.)5 ConclusionThis paper presents a modi�ed Regularized Cubic B-Spline �tting for roof edge detection. Themodi�ed scheme is linear, it does not rely on the Prewitt edge detector used in the originalscheme, and the algorithm is much simpli�ed because an interpolation process, the bottleneckin the original algorithm, is not required. The Horizontal-Vertical decomposition is employedso that a 2-D image can be analysed on a 1-D basis. Although the RCBS �tting is well-13

posed, the regularization factor � used should be su�ciently large to make the computationwell-conditioned.Since the edge detection scheme is applied locally on a sub-image, it can be performed onother sub-images in parallel, and thereby speeding up the processing time.The modi�ed RCBS scheme has been used successfully in roof edge detection. The roof edgemap of a real image with step and roof edges shows that the proposed roof edge detector ismore sensitive to arbitrary signal than a step edge detector.Acknowledgements: The authors would like to thank the NATO Collaborative Research GrantsProgramme (CRG 931451) for providing funds to support the international collaboration, andG.R. Martin for providing the image \Trevor" used for CCITT codec standards. Financialsupport from NSERC of Canada through grant number OGP0000370 and the Linda SumartiniFoundation are gratefully acknowledged.References[1] P.J.Besl and R.C.Jain, \Segmentation through Variable-order Surface Fitting," IEEETrans. Patt. Anal. Machine Intell., vol. PAMI-10, pp. 167-192, 1988.[2] J.Canny, \A Computational Approach to Edge Detection," IEEE Trans. Patt. Anal.Machine Intell. vol. PAMI-8, pp. 679-698, 1986.[3] R.M.Haralick, \Ridges and Valleys on Digital Images," Comput. Vision Graphics ImageProcessing, vol. 22, pp. 28-38, 1983.[4] R.Ho�man and A.K.Jain, \Segmentation and Classi�cation of Range Images," IEEETrans. Patt. Anal. Machine Intell., vol. PAMI-9, pp. 608-620, 1987.[5] J.Hou and R.Bamberger, \Orientation Selective Operators for Ridge, Valley, Edge andLine Detection in Imagery," in Proc IEEE ICASSP, vol. 5, pp. 25-28, 1994.[6] O.Monga, N.Armande and P.Montesinos, \Thin Nets and Crest Lines: Application toSatellite Data and Medical Images," in Proc IEEE ICIP'95, vol. 2 pp. 468-471, 1995.[7] T.Pajdla and V.Halvac, \Surface Discontinuities in Range Images," in Proc IEEE 4thInt. Conf. Comput. Vision, pp. 524-528, 1993.[8] D.Pearson and J.Robinson, \Visual Communication at Very Low Data Rates," in Proc.IEEE, vol. 73, pp. 795-812, 1985.[9] S.Riazano�, B.Cervelle and J.Chorowicz, \Ridge and Valley Line Extraction fromDigitalTerrain Models," Int. J. Remote Sensing , vol. 9, pp. 1175-1183, 1988.[10] D.Lee, \Coping with Discontinuities in Computer Vision: Their Detection, Classi�cation,and Measurement," IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-12, pp. 321-343,1990. 14

[11] M.Bertero, T.Poggio and V.Torre, \Ill-Posed Problems in Early Vision," in Proc. IEEE,vol. 76, pp. 869-889, 1988.[12] M.G.Kang, A.K.Katsaggelos, \General Choice of the Regularization Functional in Reg-ularized Image Restoration," IEEE Trans. Image Processing, vol. 4, pp. 594-602, 1995.[13] D.Terzopoulos, "Regularization of Inverse Visual Problems Involving Discontinuities",IEEE Trans. on Patt. Anal. Machine Intell., vol. PAMI-8, pp. 413-424, 1986.[14] V.Torre and T.Poggio, \On Edge Detection," IEEE Trans. Patt. Anal. Machine Intell.,vol. PAMI-8, pp. 147-163, 1986.[15] M.Kass, A.Witkin and D.Terzopoulos, \Snakes: Active Contour Models," Int. J. Com-put. Vision, vol. 1, pp. 321-331, 1988.[16] R.M.Haralick, \Digital Step Edges from Zero Crossing of Second Directional Deriva-tives," IEEE Trans. Patt. Anal. Machine Intell., vol. 6, No. 1, pp. 58-68, 1984.[17] D.Marr and E.Hildreth, Theory of Edge detection, Proc. Royal Soc. London Ser. B, vol.207, pp. 187-217, 1980.[18] G.Chen and Y.H.Yang, \Edge Detection by Regularized Cubic B-Spline Fitting," IEEETrans. Syst. Man Cybern., vol. 25, no. 4, pp. 636-643, 1995.[19] K.Liang, T.Tjahjadi and Y.H.Yang, \A Regularized Multiscale Edge Detection Schemeusing Cubic B-Spline," in Proc. U.K. Symposium Time-Frequency Time-Scale Methods,IEEE Signal Processing Chapter (UKRI section), pp. 58-65, 1995.[20] J.Ahlberg, E.Nilson and J.Walsh, The Theory of Splines and Their Applications, inMathematics in Science and Engineering, vol. 38, New York: Academic, 1967.[21] G.Strang, Introduction to Applied Mathematics, MA: Wellesley-Cambridge Press, 1986.[22] V.Z.Mesarovic, N.P.Galatsanos and A.K.Katsaggelos, \Regularized Constrained TotalLeast Squares Image Restoration," IEEE Trans. Image Processing, vol. 4, pp. 1096-1108, 1995.About the author{Kung-Hao Liang received the B.Sc. degree in power mechanical en-gineering from National Tsing-Hua University, Hsin-Chu, Taiwan, R.O.C. in 1992. He iscurrently pursuing his Ph.D. degree in engineering at the University of Warwick, U.K.. Hisresearch interests include image processing, image coding, and multivariable control.About the author{Tardi Tjahjadi received the B.Sc. (Hons.) degree in mechanical engi-neering from University College London, U.K., in 1980, the M.Sc. degree in managementsciences (operational management) and the Ph.D. in total technology from the Universityof Manchester Institute of Science and Technology, U.K., in 1981 and 1984, respectively. Hejoined the Department of Engineering at the University of Warwick and the U.K. DaresburySynchrotron Radiation Source Laboratory as a Joint Teaching Fellow in 1984. Since 1986he has been a lecturer in computer systems engineering at the same university. His researchinterests include multiresolution image processing, model-based colour image processing andfuzzy expert systems.About the author{Yee-Hong Yang received the B.Sc. degree (with �rst class honours) inphysics from the University of Hong-Kong, the M.S. degree in physics from Simon Fraser15

University in Canada, the M.S.E.E. and Ph.D. degree in electrical engineering from theUniversity of Pittsburgh.From 1980 to 1983. Dr.Yang was a Research Scientist in the Computer Engineering Center ofMellon Institute, a division of Carnegie-Mellon University, where he was involved in the VeryHigh Speed Integrated Circuit (VHSIC) programme. He joined the Department of Compu-tational Science at the University of Saskatchewan in 1983, and is currently a Professor andthe Director of the Computer Vision Lab. From July 1, 1989 to June 30, 1990, he spent hissabbatical at the McGill Research Center for Intelligent Machines of McGill University inMontreal, Quebec, Canada. His research interests include image processing, computer vision,motion analysis, biomedical image processing, and computer graphics.Dr.Yang is a member of the ACM, the Pattern Recognition Society and the Optical Societyof America. He is also a senior member of the IEEE, and serves on the Editorial Board ofPattern Recognition.

16

LegendsFig.1 An example of a roof edge.Fig.2 (A) The mask of the original RCBS scheme, where the orientation of the �tting isdetermined by the Prewitt operator. (B) The mask of the proposed scheme.Fig.3 From top to bottom: cubic B-spline and its �rst, second and third order derivatives.Fig.4 The �tted curve (solid line) on a roof edge (dashed line) using the RCBS �tting withvarious regularization factors �. From top to bottom: �=10�7,10�10,10�13,10�16,10�19,and0.Fig.5 The optimum edge maps produced when the test image (edge size=40) is contaminatedwith noise of various standard deviations (NSD's). (A) The test image. (B) The idealedge map. (C) NSD=0. (D) NSD=5. (E) NSD=10. (F) NSD=15. (G) NSD=20.Fig.6 Top: The optimumregularization factors obtained when the test image is contaminatedwith noise of various standard deviations (NSD's). Bottom: the corresponding FCR'swhen the optimum regularization factors are used. In this test, the size of the roof edgeand the threshold are 40 and 10 respectively.Fig.7 Top: The optimum thresholds obtained in noise-free images with roof edges of varioussizes. Bottom: The corresponding FCR's. In this test, the regularization factor � is 0.1.Fig.8 The optimum edge maps obtained for noise-free roof edges of various sizes: (A) Theideal edge map; (B) Edge size=10; (C) Edge size=20; (D) Edge size=30; (E) Edgesize=40.Fig.9 (A) The \Trevor" image. (B) The edge map of \Trevor" produced by the proposedroof edge detector (�=0.1; threshold=6). (C) The edge map of \Trevor" produced bythe Haralick step edge detection scheme with the threshold of 4.

17

−6−4

−20

24

6

−1

−0.5

0

0.5

10

50

100

150

200

Coordinate on Principal orientationSpatial coordinate (pixel)

Gre

y−Le

vel

Figure 1: An example of a roof edge.18

(A) (B)Orientation

of the

Equal-Weighted

Averaging

Orientation of the Fitting

Grid Pixel Centre of the MaskGrid Pixel

Centre of the Mask

Horizontal RCBS Fitting

Vertical

RCBS

Fitting

Figure 2: (A) The mask of the original RCBS scheme, where the orientation of the �tting isdetermined by the Prewitt operator. (B) The mask of the proposed scheme.19

−2 −1.5 −1 −0.5 0 0.5 1 1.5 20

0.5

1

Spatial coordinate (pixel)

Mag

nitu

de

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−1

0

1

Spatial coordinate (pixel)

Mag

nitu

de

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−2

0

2

Spatial coordinate (pixel)

Mag

nitu

de

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−5

0

5

Spatial coordinate (pixel)

Mag

nitu

de

Figure 3: From top to bottom: cubic B-spline and its �rst, second and third order derivatives.20

0 1 2 3 4 5 60

50

100

150

Inte

nsity

Spatial coordinate (pixel)

0 1 2 3 4 5 60

50

100

150In

tens

ity

Spatial coordinate (pixel)

0 1 2 3 4 5 60

50

100

150

Inte

nsity

Spatial coordinate (pixel)

0 1 2 3 4 5 60

100

200

Inte

nsity

Spatial coordinate (pixel)

0 1 2 3 4 5 6−200

0

200

400

Inte

nsity

Spatial coordinate (pixel)

0 1 2 3 4 5 6−200

0

200

400

Inte

nsity

Spatial coordinate (pixel)Figure 4: The �tted curve (solid line) on a roof edge (dashed line) using the RCBS �tting withvarious regularization factors �. From top to bottom: �=10�7,10�10,10�13,10�16,10�19,and 0.21

(A) (B) (C) (D)(E) (F) (G)

Figure 5: The optimum edge maps produced when the test image (edge size=40) is contam-inated with noise of various standard deviations (NSD's). (A) The test image. (B) The idealedge map. (C) NSD=0. (D) NSD=5. (E) NSD=10. (F) NSD=15. (G) NSD=20.22

0 2 4 6 8 10 12 14 16 18 200

1

2

3

4

5

Reg

ular

izat

ion

Fac

tor

Noise Standard Deviation

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

2

FC

R

Noise Standard DeviationFigure 6: Top: The optimum regularization factors obtained when the test image is contam-inated with noise of various standard deviations (NSD's). Bottom: the corresponding FCR'swhen the optimum regularization factors are used. In this test, the size of the roof edge andthe threshold are 40 and 10 respectively.23

10 15 20 25 30 35 403

3.5

4

4.5

5

5.5

6

Thr

esho

ld

Size of the Roof Edge

10 15 20 25 30 35 400

0.1

0.2

0.3

0.4

FC

R

Size of the Roof EdgeFigure 7: Top: The optimum thresholds obtained in noise-free images with roof edges ofvarious sizes. Bottom: The corresponding FCR's. In this test, the regularization factor � is0.1.24

(A) (B) (C)(D) (E)

Figure 8: The optimum edge maps obtained for noise-free roof edges of various sizes: (A) Theideal edge map; (B) Edge size=10; (C) Edge size=20; (D) Edge size=30; (E) Edge size=40.25

(A)(B) (C)

Figure 9: (A) The \Trevor" image. (B) The edge map of \Trevor" produced by the proposedroof edge detector (�=0.1; threshold=6). (C) The edge map of \Trevor" produced by theHaralick step edge detection scheme with the threshold of 4.26