veridicality of three-dimensional shape perception ... zaidi - 200… · veridicality of...

2430 J. Opt. Soc. Am. A/Vol. 18, No. 10 /October 2001 A. Li and Q. Zaidi

Veridicality of three-dimensional shape perceptionpredicted from

amplitude spectra of natural textures

Andrea Li and Qasim Zaidi

SUNY State College of Optometry, 33 West 42nd Street, New York, New York 10036

Received January 12, 2001; accepted March 6, 2001; revised manuscript received March 29, 2001

We show that the amplitude spectrum of a texture pattern, regardless of its phase spectrum, can be used topredict whether the pattern will convey the veridical three-dimensional (3-D) shape of the surface on which itlies. Patterns from the Brodatz collection of natural textures were overlaid on a flat surface that was thencorrugated in depth and projected in perspective. Perceived ordinal shapes, reconstructed from a series oflocal relative depth judgments, showed that only about a third of the patterns conveyed veridical shape. Thephase structure of each pattern was then randomized. Simulated concavities and convexities were presentedfor both the Brodatz and the phase-randomized patterns in a global shape identification task. The concor-dance between the shapes perceived from the Brodatz patterns and their phase-randomized versions was 80–88%, showing that the capacity for a pattern to correctly convey concavities and convexities is independent ofphase information and that the amplitude spectrum contains all the information required to determinewhether a pattern will convey veridical 3-D shape. A measure of the discrete oriented energy centered on theaxis of maximum curvature was successful in identifying textures that convey veridical shape. © 2001 Opti-cal Society of America

OCIS codes: 330.0330, 330.5020, 330.5510, 150.0150, 100.2960.

1. INTRODUCTIONThe pattern of markings on a surface is a powerful visualcue to the three-dimensional (3-D) shape of the surface.1

The two-dimensional (2-D) image in Fig. 1A (left) conveysthe percept of a corrugated 3-D surface solely from thesurface markings, i.e., without stereo, motion, silhouettes,or other cues to 3-D shape. Li and Zaidi2 showed that ob-servers perceived veridical 3-D shape only in the presenceof surface markings oriented along lines of maximum cur-vature. In Fig. 1A these markings are the noisy sparsecontours that appear to run along the curvature of thesurface from left to right.

Before corrugation and projection, the texture patternused in Fig. 1A consisted of eight different complex grat-ings, spaced 22.5 deg in orientation beginning at 0° (hori-zontal), each complex grating being the sum of three dif-ferent spatial frequencies in random phase. The visiblecontours along projected lines of maximum curvature areformed by the horizontal components of the texture. Theright panel of Fig. 1A is the 2-D amplitude spectrum ofthe pattern. (0, 0) in spatial frequency lies at the centerof the panel, and higher energies are indicated by darkerpoints. The vertical axis in frequency space, i.e., thehorizontal axis in the image, is the axis of maximum cur-vature for this 3-D shape.

Figure 1 illustrates the main results of Li and Zaidi.2

Besides patterns in which energy along the axis of maxi-mum curvature is relatively discrete from energy atneighboring orientations (Fig. 1A), the other class of pat-terns that convey veridical shape are those whose ampli-tude spectra contain energy that is oriented predomi-nantly along the critical axis, as in Fig. 1B. The critical

0740-3232/2001/102430-18$15.00 ©

surface markings in this image are visible as shortstreaks lying along projected lines of maximum curva-ture. Synthetic patterns not belonging to the aboveclasses (Fig. 1C–1E) did not convey veridical shape.2

The amplitude spectra of these patterns either do not con-tain any components along the critical axis (Fig. 1C), areoriented along a noncritical axis (Fig. 1D), or are isotropic(Fig. 1E). The left panel of Fig. 1C contains similar tex-ture gradients3–8 and changes in frequency9 as in Fig. 1Abut does not convey veridical shape. Hence texture gra-dients and frequency modulations are neither necessarynor sufficient for conveying veridical 3-D shape.2

Li and Zaidi10 showed mathematically that only energycomponents oriented within a few degrees of the axis ofmaximum curvature will distinguish concavities fromconvexities. This explains why patterns missing thesenecessary components will not convey the veridical shapeof a corrugation. Components at other orientations pro-vide information sufficient only to distinguish extrema ofcurvature from planar portions but not to distinguish be-tween signs of curvatures or signs of slant. This math-ematical result links the perception of shape from com-plex textures to the perception of shape from parallelrulings studied by Stevens11 and may remove the need toincorporate Stevens’s strong constraint that rulings areinterpreted as following lines of maximum curvature onthe surface. Knill12 has recently derived a strong corol-lary to the above result—i.e., the texture flow on a devel-opable surface follows parallel geodesics of the surface.However, since some patterns containing the critical com-ponent also do not convey veridical shape (Figs. 1D and1E), sufficiency conditions may be more complicated.

2001 Optical Society of America

A. Li and Q. Zaidi Vol. 18, No. 10 /October 2001 /J. Opt. Soc. Am. A 2431

The goal of this paper was to identify sufficiency condi-tions by using a set of naturally occurring textures. Wefirst measured how well each of a set of naturally occur-

ring patterns conveyed 3-D shape when overlaid on a flatsurface that was then corrugated and projected in per-spective. Second, we determined whether a pattern’s ca-pacity for conveying veridical shape was contingent uponits amplitude spectrum regardless of its phase structure.This question was tested by generating a set of test pat-terns that had amplitude spectra identical to the originaltextures but randomized phase spectra. Our resultsshow that the amplitude spectrum can be used to predictwhether a pattern will convey veridical shape. Third, wedevised objective measures based on the amplitude spec-tra of patterns to separate patterns that convey veridicalshape from those that do not.

2. EXPERIMENT 1: PERCEIVED ORDINALSHAPE FROM BRODATZ TEXTURESThe purpose of the first experiment was to quantitativelydocument the shapes conveyed by various everyday tex-ture patterns. Local relative depth judgments were usedto reconstruct perceived shape for corrugated surfaces inperspective projection.

A. Texture PatternsTexture patterns from the Brodatz13 collection were usedin this study. They consist of gray-level frontoparallelphotographs of various man-made and natural materials,such as woven textiles and animal skins. The collectionprovides images containing textures at various scales andorientations and has been extensively analyzed.14 Un-like many of the artificial texture patterns used in previ-ous shape-from-texture studies,3–8 most of these naturalpatterns cannot be easily parsed into individual textureelements. Patterns were eliminated if they containedshading caused by the 3-D texture of the material, as thiswould have resulted in projected images with shadingcues that were inconsistent with the corrugated shape ofthe surface. They were also eliminated if the textureconsisted of just a few large objects, as in the case of cer-tain lace patterns that contained large floral patternsonly once or twice across the entire pattern. Of the 56patterns used, 40 were man-made materials (mainly tex-tiles), and the remaining 16 fell into one of three natu-rally occurring categories: animal pelts (8), terrains suchas pebbles or grass (4), and solid surfaces such as woodand stone (4).

Fig. 1. Patterns that convey veridical shape: A, Left: Octotro-pic plaid pattern drawn on a surface corrugated in depth as afunction of horizontal position and projected in perspective withthe center of the image at eye height. The pattern consists ofeight compound gratings, each oriented 22.5 deg from the next.Each compound grating is the sum of three frequencies at ran-dom phases. Right: The amplitude spectrum of the uncorru-gated pattern. (0, 0) in spatial frequency lies at the center of thepanel, and energy amplitude increases with increasing darkness.B, Pattern of white noise filtered with an elliptical filter orientedalong the axis of maximum surface curvature. Patterns that donot convey veridical shape: C, Same complex plaid pattern as inA minus the horizontal compound grating. The pattern containsall the relevant texture gradients and frequency modulationsconsistent with the corrugated surface. D, Pattern of whitenoise filtered with an elliptical filter oriented along the axis ofminimum curvature. E, Pattern of isotropic broadband noise.


Patterns were scanned in at 150 dpi on a Hewlett-Packard 4p Scanner and saved in PCX format. Each pat-tern was 600 3 600 pixels. They were then ported intoMatlab, which was used for all image computations. Theamplitude spectrum of each pattern was computed.These spectra are shown in the leftmost columns of Fig. 2.The corresponding corrugated, projected images of thepatterns are shown in the second column. The patternnumber is shown above the two panels to the right.

B. Corrugation and ProjectionShape-from-texture studies have conventionally used ei-ther flat or singly curved surfaces such as cylinders.3–9

Results from Li and Zaidi2,10 showed the necessity of us-ing stimuli that contain both convex and concave curva-tures. For example, although Sakai and Finkel9 showedthat one-dimensional frequency modulations were effec-tive at conveying the shapes of convex cylindrical sur-faces, Li and Zaidi2 showed that they are not sufficient fordistinguishing convexities from concavities, even underperspective projection.

Each pattern was overlaid on a flat surface that wasthen corrugated sinusoidally in depth (z) as a function ofhorizontal position (x):

z 5 A3D cos~2pf3Dx 1 f ! 1 d, (1)

where A3D and f3D are the amplitude and frequency of thecorrugation, respectively, and f is the phase of the corru-gation at the center of the image. We then computed theperspective projection of each corrugated pattern onto theplane of the CRT display (see Appendix of Ref. 2). Theamplitude of the depth modulation (A3D) was computed tobe 8 cm from peak to trough for a viewing distance d of 44cm. The projected image was 381 pixels square and sub-tended 21 deg visual angle. Each image contained threefull cycles of the corrugation. The patterns in Fig. 2 haveall been corrugated and projected with a central phase ofzero.

C. EquipmentStimuli were presented using a Cambridge Research Sys-tems Visual Stimulus Generator (CRS VSG2/3) on aBARCO 7651 color monitor with a 736 3 550 pixel screenrunning with a refresh rate of 100 frames/s. Through theuse of 12-bit DACs, after gamma correction, the VSG2/3 isable to generate 2861 linear levels for each gun.

D. Psychophysical MethodPerceived ordinal shape was reconstructed for each pat-tern from a series of local relative depth judgments.2

Each projected image contained a central square fixationflanked by two smaller test dots positioned to the left andright of the fixation along a 245° or 145° diagonal. Thetest dots were 0.5° visual angle away from the fixationand abutted it at its diagonal corners. Observers wereasked to fixate the central dot and indicate whether thelocation corresponding to the left or right dot appearedcloser or whether the two locations appeared at equaldepths. To sample both convex and concave portions ofthe surface, the central fixation location was set to corre-

spond to one of 12 different phases along a single cycle ofthe corrugation. The surface was projected in perspec-tive with the center of each image at eye height.

One texture pattern was used for each session. Werandomly interleaved 5 trials for each of the 12 phases fortest dots along the 145° diagonal, and the same for testdots along the 245° diagonal. Images were presentedagainst a black background for 1.5 s followed by a graymask of the same size and average luminance that re-mained on until the observer made a response.

Observers rested their chins in a headrest, and viewingwas monocular in a dark room. Each session lasted ap-proximately 5 min. One of the authors (AL) and a stu-dent (JR) served as observers. JR had participated insimilar experiments on synthetic textures.

E. Ordinal Shape ReconstructionEach relative depth judgment provides the sign of the lo-cal slope of the surface. The perceived ordinal shape ofthe surface was reconstructed by integrating these localsigns cumulatively from left to right in the following way.Initially, the estimate of perceived relative depth at eachtest location was set to zero (i.e., equivalent to the planeof the screen). Distances beyond the plane of the screenwere indicated by positive numbers, distances in front ofthe screen by negative numbers. From the results ofeach trial, the estimated perceived depth of the right testlocation was incremented and decremented according tothe observer’s response: If the right test location ap-peared closer, the depth estimate at that location was dec-remented by 1; if the left location appeared closer (i.e., theright location appeared farther away), the depth estimateat the right test location was incremented by 1. Depthestimates were left unchanged if the two locations werereported at equal depths. Since there were five trials foreach test dot configuration at each location, resultingrelative depth estimates ranged from 25 to 5. The finalarray of relative depth estimates was then added from leftto right, starting at zero. For example, the final array ofdepth estimates of a surface for which all local relativedepth relationships were seen correctly was (5, 5, 5, 0, 25,25, 25, 25, 25, 0, 5, 5) which yielded a reconstruction of(5, 10, 15, 15, 10, 5, 0, 25, 210, 210, 25, 0) (Fig. 3, top).Results across the two diagonal test dot configurationsdid not differ systematically, and so depth estimates wereaveraged across the two.

This reconstruction provided an ordinal estimate of theperceived shape. Hence global percepts of a triangularwaveform and a flattened sinusoid could yield the samereconstruction, as could percepts of different amplitudesof depth modulation. However, the method successfullyreveals perceived convexities and concavities.2

F. Classification of VeridicalityThe reconstructed perceived shape corresponding to eachtexture pattern is shown for each of the two observers inthe rightmost columns of Fig. 2. Each plot shows per-ceived shape for one cycle of corrugation as a function ofthe phase (in radians). Observers’ initials are indicatedat the top of each column. The dashed vertical line in theplots indicates the central phase of the corrugated pat-


Fig

.2.

Am

plit

ude

spec

tra,

corr

uga

ted

patt

ern

s,an

dda

tafo

r56

Bro

datz

patt

ern

s,ar

ran

ged

ingr

oups

offo

ur

pan

els

per

patt

ern

.A

mpl

itu

desp

ectr

aof

un

corr

uga

ted

patt

ern

sar

esh

own

inth

efi

rst

colu

mn

,cor

ruga

ted

proj

ecte

dpa

tter

ns

inth

en

ext

colu

mn

toth

eri

ght,

and

data

for

two

obse

rver

sin

the

nex

ttw

oco

lum

ns

toth

eri

ght

(AL

onth

ele

ftan

dJR

onth

eri

ght)

.T

he

patt

ern

nu

mbe

ris

indi

cate

dab

ove

the

two

data

plot

s.T

he

dash

edve

rtic

alli

ne

inth

epl

ots

indi

cate

sth

ece

ntr

alph

ase

ofth

eco

rru

gate

dpa

tter

ns

show

nto

the

left

.T

he

perc

eive

dsh

ape

iscl

assi

fied

acco

rdin

gto

the

best

-fitt

ing

tem

plat

e(s

eeF

ig.3

),w

hic

hap

pear

sin

the

upp

erri

ght

oru

pper

left

corn

erof

each

data

pan

el.

V,ve

ridi

cal;

HR

,hal

f-re

ctifi

ed;F

R,f

ull

yre

ctifi

ed;

F,fl

at;

O,

oth

er.

Pat

tern

sth

atw

ere

perc

eive

dve

ridi

call

yby

both

obse

rver

sar

epr

esen

ted

firs

t,fo

llow

edby

thos

epe

rcei

ved

veri

dica

lly

byon

lyon

eob

serv

er,

and

fin

ally

byn

eith

erob

serv

er.

(Con

tin

ues

onn

ext

six

page

s.)


Fig

.2.

(con

tin

ued

)


terns shown on the left. A number of different types ofshapes resulted from the relative depth judgments, in-cluding half-rectified, fully rectified, and flat surfaces.There was one notable tendency in the data: In almostall cases, if any portion of the surface was seen incor-rectly, it was the concave portion of the corrugation.

To classify the perceived ordinal shapes, we determinedthe similarity between the reconstruction for each patternand each of the templates in Fig. 3. The veridical tem-plate for the simulated surface is shown in the top panel.Three nonveridical templates, half-rectified, fully recti-fied, and flat (frontoparallel), were also tested against thereconstructions (see Appendix A for details). If the datadid not fit any of the templates, then the shape was clas-sified as ‘‘other nonveridical.’’

A veridical reconstruction results from the percept of asurface whose convexities and concavities are seen cor-rectly with a high degree of certainty; that is, this recon-struction is achieved only if the observer makes correctrelative depth judgments at all locations on every trial.By simple statistical considerations, if the difference inperceived relative depth between two locations in the re-construction is either 4 or 5, we can reject the hypothesisthat the slant between the locations is perceived as flatand that the value results from random response. Per-ceived relative depths of 3 or less between two locations inthe reconstruction can result from a surface that is per-ceived with a high degree of certainty as frontoparallel orfrom an uncertain percept resulting in random relativedepth responses. Regardless of the certainty with whichthe different nonveridical shapes are seen, it is the veridi-cal versus nonveridical classification that is the main fo-cus of this paper.

G. ResultsThe classification of the perceived ordinal shape is indi-cated in the upper right or left corner of each reconstruc-tion panel in Fig. 2 (V, veridical; FR, fully rectified; HR,half-rectified; F, flat; O, other). The patterns are groupedby frequency of veridicality. That is, patterns that wereperceived as veridical by both observers are shown first,followed by those that were perceived as veridical by oneobserver, and finally by neither observer. The resultsshow that not all natural textures convey veridical ordi-nal shape. Observer AL and JR, respectively, perceived37.5% and 45% of the patterns veridically, with 29% of the56 patterns seen as veridical by both.

We informally tested the prediction that the amplitudespectra of the patterns that convey veridical ordinalshape contain dominant or discrete energy along the criti-cal axis, whereas the spectra of patterns that convey non-veridical shapes contain dominant or discrete energyalong noncritical axes, or are isotropic.2 A classificationof the spectra by eye by one of the authors (performedwithout knowledge of which spectrum belonged to whichpattern to minimize bias) yielded 28 patterns that werepredicted to convey veridical shape and 28 patterns pre-dicted to not convey veridical shape. Approximately 80%of the classifications were consistent with the perceivedshapes obtained for each observer. Therefore classifica-tion by eye of the amplitude spectrum might be reason-ably predictive of whether or not a pattern will convey

Fig. 3. Template shapes to which individual data were com-pared. Top, veridical; bottom, nonveridical: half-rectified, fullyrectified, and flat.


veridical ordinal shape. In a later section we examinewhether an objective classification can do better, but be-fore doing so, we test whether the capacity for a pattern toconvey veridical shape is also contingent upon its phasestructure.

3. EXPERIMENT 2: RANDOMIZING PHASESTRUCTUREA. Generation of Phase-Randomized PatternsThe amplitude spectrum of each pattern quantifies thepower at each spatial frequency in the pattern. Thephase spectrum quantifies the phase relations betweenthe various frequencies in the pattern. To test whetherthese phase relations play a role in determining whethera pattern will convey veridical shape, for each of the 56Brodatz patterns we generated a pattern that had thesame amplitude spectrum but whose phase spectrum wasrandomized.

The 2-D Fourier transform of a pattern T can be ex-pressed in terms of its amplitude-spectrum matrix A andphase spectrum matrix P: F(T) 5 AeiP. To generatethe phase-randomized pattern (Tr), we used the ampli-tude spectrum A with a random-phase-spectrum matrixPr to create F(Tr) 5 AeiPr. The new pattern was the in-verse Fourier transform of F(Tr). All images were nor-malized to have the same mean luminance. Each of theBrodatz and the phase-randomized patterns was overlaidon a flat surface, corrugated, and projected in perspectiveas either a central concavity or a central convexity. Thecentral 7° 3 21° vertical strip containing 80% of a fullcycle of the sinusoidal corrugation was used as the stimu-lus. The two left columns of Fig. 4 show some examplesof concavities and convexities of normalized Brodatz pat-terns and the two right columns show their phase-randomized equivalents. Randomizing the phase struc-ture removes many of the sharp contours that are createdby precise phase relations between harmonics.

B. Psychophysical MethodThe number of stimulus patterns was large (56 Brodatzpatterns plus 56 phase-randomized patterns); thereforewe used a global shape identification paradigm in whichobservers were asked to identify the 3-D surface shape ina three-alternative forced-choice task (concave, convex, orother). We have used a similar paradigm previously10

and found it to yield reliable results.The experiment consisted of 7 sessions, each repeated 5

times. Each session contained 320 trials: 32 stimuli (8Brodatz patterns and their 8 phase-randomized analogs,each at two simulated curvatures), presented for 10 trialseach randomly interleaved. Each session lasted approxi-mately 10 min. Each of the 224 stimuli were thus pre-sented 50 times. There was no feedback. Observers’

Fig. 4. Left: examples of Brodatz patterns corrugated and pro-jected in concavity and convexity phase. From top to bottom,Brodatz pattern numbers are D003, D034, D052, D054, andD068. Right: phase-randomized versions of the patterns in theleft column. Patterns have all been normalized to have thesame mean luminance.


heads were fixed, and viewing was monocular in a darkroom.

The same observers from Experiment 1 also partici-pated in this experiment. Observers fixated a centralsquare in a mid-gray screen for 1 min at the beginning ofeach session. The stimuli were presented for 1 s each,followed by the mid-gray screen. A three-button re-sponse box was used to indicate whether the surface ap-peared concave, convex, or other. The third choice of‘‘other’’ was included to minimize random biases towardone or the other curvature on trials where the observerwas unsure of either. Stimuli were presented by usingthe VSG2/3 on a SONY GDM-F500 flat-surface colormonitor with a 800 3 600 pixel screen running with a re-fresh rate of 100 frames/s.

C. ResultsIn Fig. 5, each panel shows the number of correct re-sponses out of 50 for each Brodatz pattern plotted againstthe number of correct responses for the phase-randomized

analog. Each point thus corresponds to one of the 56 tex-ture patterns. The data for simulated concavities areplotted in the left column, and data for simulated convexi-ties are plotted in the right column. The solid horizontaland vertical lines in each panel indicate 35 out of 50 cor-rect responses. In 50 repetitions, the probability that anobserver has made 35 or more correct responses by chanceis less than 0.05; therefore stimuli that were identifiedcorrectly in more than 35 out of 50 trials were consideredveridical identifications. Stimuli that were perceivedveridically for both the Brodatz and the phase-randomized patterns fall in the small square at the upperright of each panel; those that were perceived nonveridi-cally for both patterns fall in the larger square at thelower left of each panel. Most of the data fall in thesetwo squares, suggesting that there is a high degree ofagreement between the veridicality of percepts for theBrodatz patterns and their phase-randomized analogs.In all four panels, the cluster at the upper right corner in-dicates that a number of surfaces were reported as veridi-

Fig. 5. Number of correct responses out of 50 for each pattern plotted for the Brodatz patterns along the abscissa and for the phase-randomized patterns along the ordinate. Data are plotted by simulated curvature in the two columns and by observer in the two rows.Each panel contains 56 points, each corresponding to one of the Brodatz/phase-randomized patterns. The solid vertical and horizontallines indicate the 35/50 correct response boundaries. Patterns for which observers perceived veridical shape in both the Brodatz and thephase-randomized conditions fall in the square at the upper right of each panel; those for which observers perceived nonveridical shapesin both conditions fall in the square at the lower left of each panel.


cal with a high degree of certainty. The clustering at thelower left corner suggests that nonveridical shapes werealso reported with a high degree of certainty. Very few

points fall around 25/50, which would have indicated ran-dom responding. Stimuli that were perceived veridicallyfor only the Brodatz patterns fall in the rectangle at thelower right. Stimuli that were perceived veridically foronly the phase-randomized patterns fall in the upper leftrectangle.

Table 1 shows the numbers of patterns in each subdi-vision of each panel in Fig. 5. The number of patterns forwhich the correct curvature was reported for both theBrodatz and the corresponding phase-randomized patternis tabulated in the first row, the number of patterns forwhich the correct curvature was reported for only the Bro-datz pattern in the second row, for only the phase-randomized pattern in the third row, and for neither pat-tern in the fourth row. The perceived shapes for theBrodatz and the phase-randomized patterns were in closeagreement. For observers JR and AL, 87.5 and 80.4% ofthe simulated concave patterns and 85.7 and 83.9% of thesimulated convex patterns fall in the first and fourthrows. The high concordance between the shapes con-veyed by the Brodatz and the phase-randomized patternsindicates that the amplitude spectrum contains the criti-cal shape-conveying information about a pattern; ran-domization of the phase structure had little or no effect onthe veridicality of the perceived shape.

For observer AL, there were four patterns whoseshapes were perceived veridically for the Brodatz patternand nonveridically for the phase-randomized pattern(D035, D051, D054, and D066). These four patternswere, however, perceived as nonveridical in Experiment1. For observer JR, there were also four patterns per-ceived as veridical for the Brodatz pattern and as nonver-idical for the phase-randomized pattern (D008, D016,

Table 1. Number of Patterns Seen Veridically forSimulated Curvatures

Pattern

Type of Curvature

Concave Convex

AL JR AL JR

Brodatz andphase-randomized

26 17 35 28

Brodatz only 4 4 3 6Phase-randomized

only7 3 6 2

Neither 19 32 12 20Total 56 56 56 56

a 5 tan21S d sin v 2 y cos v sin u

d cos u cos vD ,

s 5f@d

$cos2 u@cos2~v 1 p/2!~d2 1 y2! 1 d2 s

D081, and D082). These patterns were perceived as ver-idical in Experiment 1. It is possible that for these cases,for this observer, phase randomization reduced the per-ceived contrast of critical contours in the image that led tothe nonveridical percepts.

4. ANALYSIS OF THE AMPLITUDESPECTRUMTo develop an objective metric that can be used directly toindicate whether a pattern will convey veridical 3-Dshape, we consider how the amplitude spectra for pat-terns that convey veridical shape differ quantitativelyfrom those of patterns that do not convey veridical shape.

Li and Zaidi10 showed that for a developable surface oflocal slant u at a distance d, a texture component of spa-tial frequency f, oriented at v with respect to the horizon-tal, will have an image orientation a and spatial fre-quency s given by

The slants for the concavity and convexity simulationsin Experiment 2 range from 284 to 184 deg. For theseslants, only components oriented within 3 deg of the axisof maximum curvature were found to provide sufficientdifferences in a to distinguish concavities fromconvexities.10 We therefore derived measures to distin-guish patterns that convey veridical shape based onamount and discreteness within a 6-deg wedge centeredaround the axis of maximum curvature.

The first pair of measures was based on the energy inthe critical orientations. The energy in the amplitudespectrum of each Brodatz pattern within the 6-deg wedgecentered on the axis of maximum curvature (gray wedgein Fig. 6) was measured in two ways. At each spatial fre-quency f within the wedge, we took either the maximumor the mean energy across the 6 deg (ucrit) and weightedthe resulting profile by the human contrast-sensitivityfunction15:

m~ f ! 5 mean~Ef, ucrit! * CSFf , (4)

m~ f ! 5 max~Ef, ucrit! * CSFf . (5)

The filtered energy was then summed over frequencies toyield a single value, in units of canonical contrast thresh-old, for each texture pattern. The two values were com-puted:

Cmean 5 (f

m~ f !, (6)

Cmax 5 (f

m~ f !. (7)

In Fig. 7 the cumulative numbers of veridical (solidcurves) and nonveridical (dashed curves) patterns from

(2)

os~v 1 p/2!cos u#

~v 1 p/2! 2 2yd sin~v 1 p/2!cos~v 1 p/2!sin u#%1/2 . (3)
1 c
in2


Fig. 6. Discreteness index defined as the ratio of the energywithin the critical 6-deg wedge of orientations of the amplitudespectrum (gray) and the surrounding 18-deg wedge (dashedlines).

Fig. 7. Cumulative distributions for Cmean (top row) and Cmax(bottom row) based on the classifications from Experiment 1.Distributions for observer AL are plotted on the left, for JR onthe right. Cmean and Cmax are in units of contrast threshold.For Cmean , the mean energy was computed at each spatial fre-quency within the slice. The resulting profile was filtered by thehuman CSF before summing. For Cmax , the maximum energywas computed at each spatial frequency within the slice, beforefiltering with the human CSF and summing. For patterns thatconveyed veridical shape (solid curves) values were accumulatedfrom left to right. For patterns that did not convey veridicalshape (dashed curves) values were accumulated from right toleft.

Experiment 1 are plotted versus Cmean and Cmax . Thecumulative frequency at each point on the solid curve rep-resents the number of veridically perceived stimuli forwhich Cmean or Cmax was less than or equal to the corre-sponding value along the ordinate. Cmean and Cmax val-ues for patterns that did not convey veridical shape wereaccumulated in decreasing order. The cumulative fre-quency for each point on the dashed curve represents thenumber of nonveridically perceived stimuli for whichCmean or Cmax was greater than or equal to the corre-sponding value along the ordinate. These curves showthe costs and benefits of using any value of Cmean or Cmaxto classify patterns as capable of conveying veridicalshape or not. The abscissa of the solid curve at the cri-terion value indicates the number of patterns that will beincorrectly classified as veridical, and the abscissa valueof the dashed curve indicates the number of patterns thatwill be incorrectly classified as nonveridical. The derivedcriterion value should be set to minimize the total costsassociated with each type of minimization.

In the absence of different cost considerations for mis-classification of veridical or nonveridical patterns, wechose the criterion that minimized the total number ofmisclassified patterns. The minimum number of mis-classified patterns are tabulated by experiment and ob-server in Table 2. Criterion values of Cmean yielded 11and 15 misclassified patterns for observers AL and JR, re-spectively, while criterion values of Cmax yielded slightlyfewer, 10 and 13 patterns respectively.

Similar cumulatives for Experiment 2 are shown inFig. 8. Even though the psychophysical task and shapeclassification methods were different from those used inExperiment 1, and even though the perceived shapes dif-fered for some patterns, the cumulatives for both Cmaxand Cmean for Experiment 2 are similar to those from Ex-periment 1. The criterion values of Cmean yielded 18 and11 misclassified patterns for observers AL and JR, respec-tively, and criterion values of Cmax yielded 18 and 9 pat-terns, respectively.

The difference in shapes conveyed by the isotropic pat-tern in Fig. 1E versus the patterns in Figs. 1A and 1B in-dicates that the discreteness of the energy in the criticalorientation with respect to a local neighborhood of orien-tations may also be a factor. In two additional measures,we used a discreteness index to weight the critical energy.The discreteness index D is defined for each spatial fre-quency f as the ratio of the energy in the critical 6-degwedge to the energy in a surrounding 18-deg wedge (Fig.6). D for a particular spatial frequency f is thus the ratioof summed energies at that spatial frequency for the

Table 2. Minimum Number of MisclassifiedPatterns

CriterionValue

Experiment 1 Experiment 2

AL JR AL JR

Cmean 11 15 18 11Cmax 10 13 18 9Rmean 11 11 16 6Rmax 10 10 14 4


6-deg wedge (ucrit) with respect to the summed energiesfor the surrounding 18-deg wedge (usur):

Df 5

(ucrit

Ef,u

(usur

Ef,u

. (8)

Fig. 8. Cumulative distributions of Cmax and Cmean based on theclassifications from Experiment 2. See Fig. 7 for details.

Fig. 9. Cumulative distributions for Rmax and Rmean for Experi-ment 1. The maximum or the mean energy at each frequencywas weighted by a discreteness index (the ratio of energy in thecritical 6-deg wedge to the energy in the larger 18-deg wedge) be-fore being filtered with the human CSF and summed.

For an isotropic pattern like Fig. 1E, Df will be equal to0.33 at each frequency. For discrete and dominant pat-terns like Figs. 1A and 1B, Df will be greater than 0.33.For discrete or dominant noncritical patterns like Figs.1C and 1D, Df will be less than 0.33. The new measureswere computed as

Rmean 5 (f

Dfm~ f !, (9)

Rmax 5 (f

Df m~ f !. (10)

Figure 9 shows the cumulative curves based on Rmaxand Rmean for Experiment 1. Rmax and Rmean yielded thesame number of misclassifications as Cmax and Cmean forobserver AL (11 patterns for Rmean and 10 patterns forRmax) and fewer respective misclassifications for observerJR (11 patterns for Rmean and 10 patterns for Rmax). ForExperiment 2 (Fig. 10), Rmax and Rmean yielded fewer mis-classifications than Cmax and Cmean for both observers (16patterns for Rmean and 14 patterns for Rmax for observerAL, 6 patterns for Rmean and 4 patterns for Rmax for ob-server JR). Thus Rmax consistently yielded the fewestnumber of misclassifications. If it is necessary to choosepatterns that convey veridical shape, judging from our ob-servers’ percepts, any pattern for which Rmax exceeds 15units will work.

5. DISCUSSIONThe results of this paper show that texture patterns con-vey veridical 3-D shape if the magnitude of energy in anarrow wedge of orientations about the axis of maximumcurvature is large relative to the energy in an encompass-

Fig. 10. Cumulative distributions of Rmax and Rmean based onthe classifications from Experiment 2. See Fig. 9 for details.


ing wider wedge. The span of the critical wedge will de-pend to some extent on the 3-D curvature of the surfaceand the viewing distance.

A. Comparison between the Experimental MethodsExperiments 1 and 2 used local and global shape percep-tion tasks, respectively. Experiment 1 involved the re-construction of ordinal shape based on a series of localslant judgments, whereas Experiment 2 involved the di-rect judgment of concavity versus convexity. Compari-sons of the numbers of patterns perceived veridically andnonveridically showed that a majority of the stimuli wereperceived similarly in the two experiments: 87.5% of thepatterns for AL and 75% for JR. Observer AL perceivedmore patterns as veridical in Experiment 2 and observerJR in Experiment 1. It is possible that some observersare better at extracting local slant while others are betterat extracting global curvature.

B. Materials That Convey Veridical Three-DimensionalShapeTo get an idea of how well different classes of materialsconvey veridical shape, we grouped the patterns as man-made versus natural (animal pelts, terrains, or solid sur-faces). In Table 3 the total number of patterns in eachcategory is listed for each category heading. The num-bers of patterns in each category that conveyed veridicalshape are listed for Experiment 1 and Experiment 2. Be-tween 16 and 27 of the 40 man-made patterns conveyedveridical shape. Many man-made materials such as tex-tiles contain gridlike patterns that can fall along lines ofmaximum and minimum surface curvature. When thepattern is tilted so that the seams follow nonprincipallines of surface curvature, the projected image no longerconveys veridical shape. Only two to four of the eightanimal pelt patterns conveyed veridical shape. The onesthat did (e.g., D003, D022) contained visible contours orflows along projected lines of maximum curvature; thosethat did not (e.g., D036, D092, D093) contained eitherstrong contours along nonprincipal lines of curvature orwere isotropic. The four terrain and four solid-surfacepatterns were either isotropic or dominant along noncriti-cal axes. Only one of these patterns conveyed veridicalshape for only one observer in Experiment 2.

C. Contrast Reversals and Changes in ScaleContrast reversals had little or no effect on perceivedshape. For the five pairs of patterns that were approxi-mate contrast reversals of one another—D101/D102,

Table 3. Numbers of Patterns in Each MaterialCategory Conveying Veridical Shape

MaterialCategory

Experiment 1 Experiment 2

AL JR AL JR

Man-made (40) 16 27 23 19Natural

Animal pelts (8) 2 4 4 7Terrains (4) 0 0 1 0Solid surfaces (4) 0 0 0 0

D103/D104, D109/D110, D111/D112, and D105/D106—theveridicality of the perceived shape agreed within eachpair. This is not surprising since the spectral propertiesof contrast-reversed patterns are identical. Smallchanges in scale in relation to the shape of the surfacealso had little effect. For the three pairs of patterns thatdiffered in magnifications—D016/D017 (43), D032/D033(1.53), D066/D067 (0.333), and D035/D036 (0.53)—achange in scale brought about only minor changes in theperceived shape without changing the veridicality. Inonly one case (D035/D036 for observer JR), the perceptchanged from nonveridical to veridical when the scale ofthe texture was made finer.

D. Shape from Natural TexturesIn our earlier work we used a small number of syntheticpatterns to show that the amplitude spectra of patternsthat conveyed veridical shape were either ‘‘discrete’’ or‘‘dominant’’ along lines of maximum curvature. Thestudy reported in the current paper extends this work to alarge number of naturally occurring textures with com-plex amplitude and phase spectra. The results show thatthe shape information conveyed by a texture pattern isdetermined by its amplitude spectrum. In fact, the suc-cess of the Rmax measure confirms that the information isconveyed by a narrow range of orientations in the pat-terns. Our results suggest that the critical shape infor-mation can be directly processed through independent,band-limited, orientation-selective neurons and suggestthat matched-filter paradigms16 may provide a goodmodel for extracting 3-D shape from 2-D texture cues.

This paper also provides a quantitative method for pre-dicting whether a texture pattern will convey veridical3-D shape when markings corresponding to lines of maxi-mum curvature are visible. These results can be usefulin the 3-D rendering of objects on computer screens. Asan alternative to computing the lines of maximum curva-ture at every point on the surface,17 our results suggestthat a pattern containing a critical amount of discrete en-ergy at a small number of different orientations will effec-tively convey the veridical 3-D shape of most surfaces.

Note added in proof: The results of this paper were ob-tained for upright surfaces. For corrugated surfacespitched toward or away from the observer, Zaidi and Li18

have recently shown that pairs of components orientedsymmetrically away from the axis of maximum curvaturewill produce patterns of orientation modulations in theimage that are critical for distinguishing concavities andconvexities. As a result, for pitched corrugated surfaces,single oriented components, including those parallel tothe axis of maximum curvature, will not be sufficient forveridical shape perception. The results of Zaidi and Li18

showed that the critical pattern of orientation modula-tions that allows distinction of concavities from convexi-ties matches the pattern that arises from a single compo-nent parallel to the axis of maximum curvature forupright corrugations. In alddition, the shape and pitchof the surface can be used to predict which componentpairs will combine to provide the critical orientationmodulations. Thus for pitched surfaces, one can applymethods similar to those reported in this paper to deter-mine whether the amplitude spectrum of a texture pat-


tern contains sufficient discrete energy in the relevantpairs of orientations to convey veridical shape.

APPENDIX ATo determine the template that was most similar to thedata, we first computed the difference between the arrayof local perceived slopes and the array of local slopes cor-responding to each of the five possible templates. Theseerror values were then normalized by the maximum pos-sible error for that particular template. To simplify theexplanation of maximum error, taking the veridical tem-plate as an example, the fixation locations in Fig. 3 werenumbered from 1 through 12 from left to right. For eachof locations 1–3 and 11–12, observers should respond forall five trials at each of these locations that the left testlocation is closer in depth than the right test location.That is, the slope of the surface at each of these locationsshould be 15. Measured perceived slopes at these loca-tions can vary between 25 and 15, which yields a maxi-mum error of 10 at each location. Similarly, the slope forlocations 5–9 should be 25; that is, observers should re-spond for all five trials at each of these locations that theright test location is closer in depth than the left test lo-cation. This again yields a maximum error of 10 at eachof these locations. At locations 4 and 10 the slope shouldbe 0; that is, observers should always see the two flankingtest locations as appearing at the same depth. Since thedifference between veridical and observed responses canvary only between 0 and 15 or 0 and 25, the maximumerror will be only 5 at these two locations. This yields atotal maximum error of 110 for all locations for the ver-idical template. Similarly, the maximum error was 60for the half-rectified and the flat templates and 100 forthe fully rectified template.

The template that yielded the smallest normalized er-ror was the template shape under which we classified thepercept. If the data did not fit any of the templates (nor-malized error .0.25 for all templates), then the shapewas classified as ‘‘other nonveridical.’’

ACKNOWLEDGMENTSThe authors thank Joseph Rappon for his time and dili-gence as an observer and Byung-Geun Khang and FuzzGriffiths for helpful discussions of the manuscript. Partsof this work were presented at the ARVO ‘99 meeting inFt. Lauderdale and at the ECVP ‘99 meeting in Trieste,

Italy. This work was supported by National Eye Insti-tute grants NRSA EY06828 to A. Li and EY07556 andEY13312 to Q. Zaidi.

Address correspondence to Andrea Li, [email protected].

REFERENCES1. J. J. Gibson, ‘‘The perception of visual surfaces,’’ Am. J. Psy-

chol. 63, 367–384 (1950).2. A. Li and Q. Zaidi, ‘‘Perception of three-dimensional shape

from texture is based on patterns of oriented energy,’’ VisionRes. 40, 217–242 (2000).

3. B. G. Cummings, E. B. Johnston, and A. J. Parker, ‘‘Effectsof different texture cues on curved surfaces viewed stereo-scopically,’’ Vision Res. 33, 827–838 (1993).

4. J. E. Cutting and R. T. Millard, ‘‘Three gradients and theperception of flat and curved surfaces,’’ J. Exp. Psychol. 113,196–216 (1984).

5. D. C. Knill, ‘‘Discrimination of planar surface slant fromtexture: human and ideal observers compared,’’ VisionRes. 38, 1683–1711 (1998b).

6. D. C. Knill, ‘‘Surface orientation from texture: ideal ob-servers, generic observers and the information content oftexture cues,’’ Vision Res. 38, 1655–1682 (1998a).

7. D. C. Knill, ‘‘Ideal observer perturbation analysis revealshuman strategies for inferring surface orientation from tex-ture,’’ Vision Res. 38, 2635–2656 (1998c).

8. J. T. Todd and R. A. Akerstrom, ‘‘Perception of three-dimensional form from patterns of optical texture,’’ J. Exp.Psychol. 113, 221–224 (1987).

9. K. Sakai and L. H. Finkel, ‘‘Characterization of the spatial-frequency spectrum in the perception of shape from tex-ture,’’ J. Opt. Soc. Am. A 12, 1208–1224 (1993).

10. A. Li and Q. Zaidi, ‘‘Information limitations in perception ofshape from texture,’’ Vision Res. 41, 1519–1534 (2001).

11. K. A. Stevens, ‘‘The visual interpretation of surface con-tours,’’ Artif. Intel. 17, 47–73 (1981).

12. D. C. Knill, ‘‘Contour into texture: information content ofsurface contours and texture flow,’’ J. Opt. Soc. Am. A 18,12–35 (2001).

13. P. Brodatz, Textures: A Photographic Album for Artistsand Designers (Dover, New York, 1966).

14. A. R. Rao and G. L. Lohse, ‘‘Towards a texture naming sys-tem: identifying relevant dimensions of texture,’’ VisionRes. 36, 1649–1669 (1996).

15. J. G. Robson, ‘‘Spatial and temporal contrast-sensitivityfunctions of the visual system,’’ J. Opt. Soc. Am. 56, 1141–1142 (1966).

16. Q. Zaidi and A. Li, ‘‘Neural model of shape from texture:developable surfaces,’’ Invest. Ophthalmol. Visual Sci. 41,S219 (2000).

17. V. Interrante, ‘‘Illustrating surface shape in volume datavia principal direction-driven 3D line integral convolution,’’presented at the SIGGRAPH conference, Los Angeles, Cali-fornia, August 3–7, 1997.

18. A. Li and Q. Zaidi, ‘‘Limitations on shape information pro-vided by texture cues,’’ Vision Res. (to be published).

veridicality of three-dimensional shape perception ... zaidi - 200… · veridicality of...

Documents