In Proceedings of DAFQA lmage Understandinq Workshop 1989, pp. 1076-1088, May 1989.

Computer Analysis of Regular Repetitive Textures Leonard G. C. h e y '

Take0 Kanade School for Computa Science

PiUsburgh PA 15213 CZlllbegkMelloaUniveniry

Abstract

Regular rcpttitive texam arc common in rcal-workl scaux, occurring in both n a n d and man-made environments. Their analysis is important for image segmentatim and fos shape recovery from surf= texture. There are two fundamental problems in analyzing reguiar repetitive texture. F d y , rbc frcsucncy interpretation of any regular t e x m is ambiguous since that arc many altanative imqmatiap tbatccmspmd to the same texture. Secondiy, the v a y definition of regular repetition is c i m h since the elanencard the repetitive hzquency are defined in tamsof each otber. In this papa, we address tbcsc t w o p o b l a n s a n d ~ a n a u s w c r to each. To address the ambiguity of finqpmcy htmpmaa 'on we p~lll to the lattice theory and choose sucassiivc minima as the most fpndamental fnquaq vccuxs of the t~topre. To deal with the- dcfidkm OfregularnpetitiOn, we compare t h e S t C U C t U d ~ - topnmimtntfcaMesinthe~Tbesttbearmcal concepsarehrporatcdintoa working systrm, capable of d y z i a g and scgmauing regular repshive tcxtlacs in real-world images. In contrast with pevioruwark, ora tcchniqueinvolv csentirely localanalysisaadistherebyrobusttotcxture distonion.

Regular repetitive exaxes are CQI~IIY)EI in real-worfd scenes. Tbey occur both as arcSulL of natural pmcesses (e-g. the repetitive mom of reptile skin) and the eff.m of man (eg. man's aukiag of a city scc11c). Understanding these textures is imponam not only as abasis for image SCgmGntation butaiso because regular repetitive textures can provi&valuabIcinfonnationfor~gsrtrfactopieatatioa

A fundamental problem in analyzing regular textures, howeva, is that the &finition of regular repetitive texture is circular. The fnsoencr of the texture is Mined as the spatial dispiacancnt becwccn elantnts of the texture. but the element of the texture is defined as that portion of the image that is ngulariy rcpeatcd This circular dependency is usually handXed by obtaining information about the reptitivc iiesuenCy without considaing the natllre of the texture element or vice vasa. In both applloaches a global adysis of the textme is pufkxmcd, restricting the applicability oftheaIglxithstoundistoro#i samplesdasin@crepetitivetcxture

In comas to these approaches, omwurkanploys apmdyfocalaaaiysis toidentify tbe repetitive structure of the m o s ~ (&minant) f- in nsplarnqeitive ocrmres i n d - d images. In this way, we identify the resnlarnpecitive=hti-m * between textme elemenowithoatirtntifvinp the texMeeianeau themselves.

A second problem in aaaIyzing regular repetitive textmw is that: cvtll whea we know the locations of textw eluxmts, tbue are many alternative pairs of fhqwncy vccmrs hat equally dcscribc the pattcnr of the texture

I

c

elements. This problem has been haadled in the past by choosing either the shortest frequency vectors [8,14,151 or those vectors which provide the simplest grammatical suucture for the texture region [IO]. The former approach has the heuristic advantage that the frequency vectors are independent of the shape of the texture region. In this paper, we develop additional reasons for preferring the shonest frequency vectors (which we call the fundamental frequency vectors based on fesults fiom lattice tbeory. These results paovide additional properties of the fundamental frequency vectols that are UsGful for extracting the freqpeacy of d - w d d kxllms.

2 Existing Frequency-Based Approaches

Both of tbcsc paoblans imply that an analysisncais tocuuidersmail samphofthe image (of the order of a few texmre elements) ata time. Becapse tbe size of the tcxrmcrJanrnh is afnnnim of the fnquency of the repetition, the appropriate image sample sizevaries as a frmction oftk hypmks& ' npecitin frequency. It therefore becomes ~~CCCSS~C~ to p~ct3s a large nambaofsampies of differem sizes i m u d of simply applying a global analysis t~ a Single image. It is nasarpriSing then that these t d m i p c s have yero be applied to reai-wmid images.

3 Existing Element-Based Approaches

from each other p v i d e d that their texture elements m sufficiently diffennt 'Ihis capability is very useful when dealing with real-world images where it is common to encounter more than one regular repetitive texture in the same image.

The element-based approaches do have one major -. the reptcitive firesuency is determined by a global analysis of the tcxtue. Such a global analysis wil l fail if the i k q u a ~ ~ y varies within the texture. Frequency varhthns commonly occur in textures in real-world imagts as a rcsult of ptrspective imaging, and existing

8 techniquesare Unabletoanal~such images.

4 The Dominant Feature Assumption

Ratha than working with the raw image intensities on the one hand or tcxplre dements on the other, we work with fmaves of the image in somearbitrary featrae space. Weconsider the imagc to consist of a set of these features, appropriately located (Conceptually. we convat the h a p into its fcature-space rtpresentation). We define a texture element T as a set of feaopcs (FI, F2,.J',) axh of which has a location x(FJ relative to the origin of T. We assume that a feature dots not span two or more t c x m elancny but allow atCxture elanent to span any number of feaaves

We now defiw a hrnction P, called thepromtrcncc functh, wbae P(Fi) is a scalar value. The function P is amtinuous, in thatlmilnfCgtmtShavesimilar~ofP. Acoavemmt * famforPisafuactionthatmeasures infonnacioa coaoemdthe fcatare. It is highly d l d y thuaq two fcatlnes in Twill have the samc pmminence value P. In taq fa anradomly paatai T, all values o f P wil l be distina with probabii 1. It follows that we can idtnafy any partiEularftanrrc by its pmhmce valw P. In paticular, wc can uniquely ida~tify one of the features - of T, sayFpa3 ltre m o s t p o m b c n t f ~ in T. Condczthefolbwing:

vi PCFr) LPCF,)

We call Fp thedomtranrfeaure of the texture clement T. Notice thatifP isan infamation measure. then F,, is that feaaue in Twhich comains the most information. Forthisrcason, tbe feature with tbe largest d u e of P is selected asthedominantfeaane.

Consider now a regular repetitive tcxtrnc consisting of texture ekmcnfs TI, T2,....Tm that art identical copies of T. If we amiyze this tcxm and extract the feaauw FI1, Fu,,E' we will fmd that thac is a group of features FpI. Fpz,...,Fh that are all equally prominent and are morc prominexu than any orha feaaucs in the texture. The &mbuuu feonvc assumption starea that such a set of dominant features always exists

Dominant Feature Assumption: Every regular repetitive pattern exhibits some feaaae that occurs once in each texture clement and is more prodneat thanany abafeenneoccnrringin tbe same mum clement

Undathedorniaantf~asSMlpaon * * i t i s p o J s i b l e t o ~ * the strrrctm~ of- qetitive tcxplrts from 6 the ft!anlK+space rqxcsaltatim of the texm sincc thae ism dnninnnt fcaam f a eacb tex4 the smlcaae of the

Q m i n a n t f e o m r e s ~ t h e ~ 0 f t h e t e x e b , a s ~ i n f i i g r m c 1 .

5 Fundamental Frequency Vectors

The fimdmentaI ficqmcy vectors are defined= thcshatestpairof fnsllrerrcy vcct~3 in the texture. They have a number of useful propaties that am be derived from lattice theory. In this theory, wtassannc that we arc dealing

I

with an undistorted regular repetitive textme. Undistort#i regular @ve textures are closely related to lattices. We define an lmdistorted regular repetition as follows.

Defmition 1: A regular repetition R is a set of points (x&u+p: i. j integers) in the plane where ~0 is an arbiaary point and u and v are lineariy independent vectors. R is the transhe of a latrice. The vectors u and v ate a basis for R.

We ckfm tbe fmvlamrntnl freqpency vectors a’and v’ofaregular repctitionR as the sbaroesr and second-shortest 0 lineatlyindependcnt f . iqmcy vccfors dR.

Dcfinitb. 2: M a e M y , kt V=(xi -%:xia€ &xi#%) be the 9 c ~ of all possible frequency ve~tQS of R. Define the first fdamaml- 1llect01 u’ to be any one dtbt shortesr vtct~n (me3surtd with Euclidean length) in V. Let U = (id: i intega). Define the s a m d -tal firesucncy vcctoz v’ to be any one d t h e shortesr vcc&xs in V-U. Tbe V~EOPJ a ’ d #are known as suaxssh minima in lattice *. tbyaretbefrmdamentplfreqmcy MctasofR.

TbisdefmiIion of tb fudmWal- vccto~3 has saneambiguity in the choice of u’and v’. lhis ambiguity has thrte @bIc solt~cts. Fdy. thae is the ambiguity between u’ and -u’ and between v’ and -4. Secondly, if la’l=lvl, ambiguity can occur in the labeling of the funrlwnclltal fnquarcy vcctocs. However, both cases are not s a i o u s p r o b l e m s d w i u n o t ~ w ~ .

1.- fmulanrmtai fnqparcy vcctcxs u’and v‘ f a m a basis faR; Le. the set (+,+h’+lv: k, 1 integers) is

2 Tbcredoes notexist foaR apairofbasis vccoonaand b for which IrJdr7aodIbklv’l; is. thac is no basis

3. Thae &a notexist apair of basis VCC~QJ a d b f a R fa which IaMbkbaWfi Le. u’and v’ describe

4. The fundamanal f i l q m c y vectors u’ aad v’ are tbe most p c q d d a r * basis for R; ie. l u ’ d / lu’llvl is

idtntical toR

forR Consistingof Veaonshorotrthan the baga f d a m a m l hqucacy vcum.

the minimnm-- !slmcad Inlit OfR.

minimai among aIl bases of R.

related to the Relative Neighburhood Graph (RNG) of tbe texture elements. The RNG [16] is a bidirectional graph on a set of points P = ( p , , h ....,pn) in the plane. The RNG connects two points pi and pj if and only if there exists no other element pi t€ P such that Ipl-pj<lp,-pj and Ih-p)dp,pJ. The RNG has been proposed as a model of human perceptual grouping of dot pattern (12). We have shown in [s] that the RNG of an undistorted regular repetitive texture cap- exactly the hdamental fresuency nlatiaaships between the texture elements. More specifically, if W is the set Of all possible kqtmlcy vecms of R (induding all forms of ambiguity) then the RNG o f R is wraaly that graph rhatconnectseach pointx leR to a l l points in the comsponding set [x;ewxve w)).

6 Analyzing Real-World Textures

In analyzing real-world tex~llrw. our aim is to discma the f d a r m m i frequency vectors betsvecn the dominant fcaturcs of the texture elements. The applicarh of the thearwcal cmccpts of dominant features and fundamental frequency vccoors is MIL direct, however, because of the variations that can o c ~ m in real-world textures. Firstly, the texture elements of r e a l - d textures often vary from cach othet, giving rise to variarions in the prominence of the dominant feaanes. Secondly, real-dd regular npetitive textures am often distorted as a d t of effects such as surface curva~eandpaspective imaging, causing theresub obtainedfroa3 lattice theory to be violated.

la ordu to deal with tbe Variations ratha than starching far absolutdy rLminant Rathex than computing the Relative Nughbourhood Graph, we dcvtlop asQuctlnaj dcsaiption in which miouspropadtsofthe f o d u n a d frequency vectcm are combined to produce agraph with weighted edges. Successive dinanem steps lead to ambust algorithm that can reliably

real-worldregalartnanes, w w tbsnlative dominance of the features

extract the - frequency desaiptioa

Tbc system that we have implcmenwr ..

illl algaidnn cxl&?bg O f f O r p m a j Q ~ Thc first phase is feature detectioa. It extram blob-like f- hm the i n p u t i m a g e a n d ~ a p o m i # n c e value and image location with each feattm. 'Ibe amnaphaseestablishesbesic sullcad retationshrpr betwcenthefeatuns,basedonthe dominant feature assIIIllptioll and PropatitJ of the fundamental fnsparcy & Tbe suucplral relationships are rcpresarted as weighted links berwtca the f- Thc links and theirssoch@d weights are passed to the third phase which collstrtlcts l d y regular repetitive sutmmes and aoachs evaluaMns to each of them. Multiple candidate repetitive sauuures are evaluated far cach iomgc feamre. 'Ihe ~ I I phase decides up on a locally canSiStcnt repetitive stracture intaprctatiOn based upon the canpetingrepecicive stnmms. A relaxation algorithm is used to obtain c0nsistcncy by wnstraintppgaha

7 Smooth Thresbolding

m u g h w the algolihms described below, we express evahaom * 00 rbnmgs0.0 to 1.0. Thesc evaluations have

applied to a mcasmui value. Instcad of directly thresholding the dam, me emplay "smooth tksholding" functions that convat measured values into cvahations on the mge 0.0 to 1.0. 'Ibesmooth thdwlding functions Tand TL

a similar role to fuzzy reasolling or probability vahm~ Namrl ' ofbm EniDes the fam of thresholding

b

arc based on the tunh function. In the following equations. r is the nominal threshold d u e such that 77% T. a) is 0.5. and CJ is the spread such that T(.r+a,r,a) is 0.75. T is a particular parameterization of the Sigmoid dismbution: T, is a logarithmic version which is mOct appropriate for evaluating ratios.

'

8 Feature Extradon

- 9 Basic Structural Relationships

1. Prejudice urincblc Fattrrres prefer to be Iinkcd to otha fames which are equally or m m prominent than themselvts.

I

dirtctional link between Fi and Fi. The evaluation is suppressive, in tbat large values of S, weaken the link. The constants 8 and 1.5 are the values used in our experiments.

2 Interference Rinciuk The link buwcut two fuulrres is only suong if them is no other equally or more prominent feature within thedominated region d tbe twofeann#.

TheintafcrmapinciplecaptunscancepUdaind~bothtbe Qminant feaopre assumption and the relationship between tbe f imhnad kqumcy vcctm a d theRNG. R a d thattbeRNG linlrs mgethetpairs of points where tbae is no otbapointpoint dasa to both points rmlercnnirlantwL 'Ibis is eqnivalart to linking pats of points when thac is no other point within the "Irme" of the rwo points rmdcr&ckath (the "lune" is the intersection of two circles as shown by the doatdlinein figure 3).

In adapting the RNG ztsult to nal-worjd image data, we must allow fa siightdisaepaacles * in the positions of the dominant feaarres, panimWiy as a FGsult of distorrioa dtbe textare. The 'lune" is much too sensitive to small Variatiwu in tbe pasitions oftbe featllns and tbe "true" timAamenlnl lkquency vcc t~s would ohen be disallowed, so we use an- thatlieswithin tbe "tune". We call thiarcgbu thedomhtcdngbn of the two features. As with the application ofthe &diceprincipiewe do not simply rnakcabinary decision about whethaaparticuiar feature lies inside ocoutsidt the dombcd region ofotha feanrns Ratha, the in tdamcc ofa feaauc incrtases from 0 towanis 0.5 as the feamn appaacbes the cam dthe Itmirraml region. Figum 3 shows some contours of the dominatedrtgion-

k what

. 11 Relaxation

After phase three of the algorithm, we have computed rnany competing regular repetitive smcture hypotheses for each feaarre in the image. If a particuiar feature is a dominant feature of a regular repetitive texture. we expect that neighboring dominant featurcs will also exist aad that thesc neishbaiDg farapnes will have compatible hypotheses about tbe local repuitivc sansmre 'Ibis coasmmc * ~~g~ ' shouldbeconsistentis implcmenrcdinadaxamn - alguithmtbatdeDcnruaes ' atmast- npecitivc s&manrc f a cach texture clement In the e vent that a fcamre is not pmicipm . g in a mreper i t i v t smame, thac is a lack of supprt fkom neighboringfqandtbenlrutatlon - aleorithm- * thatnorepecitivestrPctmcisplesent

l2 Results

Our systan pafornu region segmentation on xcgu&r repetitiw tcxmres p l y as a SidGeffCa of the detailed

analysis. The results ipc srpprisingly good ascan be scen in both figures 7 and 9. In addition, the smctural description produced by our system is very detailed. It may thucfore be useful as a basis for a detailed shape-hm- texture analysis. For example, the oneeyed stereo approach of [13] may be directly applicable m the structural grids extracted by our system.

W Conclusion

Figure 1: Repetitive strucnue of dopninruu features ofa tatwe.

- - . . * . e * * 0- . .....*.e. -.

.......e.. . 4 . 0 0 * e 0 . 0 - . . 0 . . * 0 ' . . . . . * . a m . * * .

0 . *.em.. . . 0

0-m . c- ' .' - - . -