linear and nonlinear transparencies in binocular vision

Linear and nonlinear transparencies inbinocular vision

Keith Langley1, David J. Fleet2 and Paul B. Hibbard1

1Department of Psychology, University College London, Gower Street, LondonWC1E 6BT, UK2Department of Computing and Information Science, Queen's University, Kingston, Canada K7L 3N6

When the product of a vertical square-wave grating (contrast envelope) and a horizontal sinusoidalgrating (carrier) are viewed binocularly with di¡erent disparity cues they can be perceived transparentlyat di¡erent depths.We found, however, that the transparency was asymmetric; it only occurred when theenvelope was perceived to be the overlaying surface.When the same two signals were added, the perceptof transparency was symmetrical; either signal could be seen in front of or behind the other at di¡erentdepths. Di¡erences between these multiplicative and additive signal combinations were examined in twoexperiments. In one, we measured disparity thresholds for transparency as a function of the spatialfrequency of the envelope. In the other, we measured disparity discrimination thresholds. In bothexperiments the thresholds for the multiplicative condition, unlike the additive condition, showed distinctminima at low envelope frequencies. The di¡erent sensitivity curves found for multiplicative and additivesignal combinations suggest that di¡erent processes mediated the disparity signal. The data are consistentwith a two-channel model of binocular matching, with multiple depth cues represented at single retinallocations.

Keywords: envelope tuning; additive transparency; multiplicative transparency; depth asymmetry;binocular rivalry

1. INTRODUCTION

A central issue in stereopsis is the correspondenceproblem: how one determines the retinal locations in theleft and right eyes that are projections of the same three-dimensional points. Conventional (¢rst-order) modelssolve for correspondence by matching linearly ¢lteredversions of the views of the left and right eyes. One mightconsider cross correlation or phase di¡erences betweensmall neighbourhoods of the ¢ltered left and right signalsto establish the binocular matches (e.g. Fleet et al. 1996).It is common for these techniques to use a uniquenessassumption (Marr 1982) to constrain the matchingprocess.With binocular transparency, however, the corre-spondence problem becomes more di¤cult because of theneed to recover several depth planes, and to deal withdi¡erent types of transparency (Weinshall 1991; Kersten1991).

Transparency occurs naturally in two basic forms(Kersten 1991). One is linear, where the retinal intensitiesare composed from the sum of two surface re£ectancepatterns. This may occur, for example, when looking at ariver-bed through a re£ection on the surface. The secondform is nonlinear, occurring when one looks through atranslucent surface. In this case, the retinal intensitiesstem from the product of the material transmittancefunction of the overlaying surface and the re£ectancefrom the underlying surface. How the visual system inter-prets surface depth for both additive and multiplicativetransparency remains an interesting issue because the twotypes of transparency are incompatible.

Kersten (1991) (see also Frisby & Mayhew 1978)reported that some multiplicative transparencies areasymmetrical, where one's percept depends on the leftand right ordering of the binocular images. Kerstenbelieved that binocular asymmetries re£ect competitionbetween contradicting monocular and binocular depthcues. This view originated from two constraints on one'spercept of monocular transparency as proposed byMetelli (1974): (i) no matter how a multiplicative trans-parency is produced, the overlaying transparent surfacemust not change the values of the order of luminancere£ected from the underlying surface; and (ii) whenvalues of lightness are attenuated by a transparentsurface, local di¡erences in lightness seen through thetransparent surface must be less than those seen withoutthe transparent surface.

Beck (1984) reasoned that Metelli's constraints wereincomplete. From the physics of multiplicative transpar-ency, Beck showed that the surface transmittance andre£ectance functions for both the overlaying and theunderlying surfaces must be positive-valued. StudyingMetelli's constraints with overlapping square patches, Becket al. (1984) found that when the luminance values of over-lapping squares satis¢ed Metelli's two constraints, subjectsreported transparency; even when values of the luminanceof one of the squares violated the assumption that its trans-mittance function was positive-valued. However, whenvalues of the luminance violated Metelli's constraints,subjects did not report transparency. To explain theirresults, Beck et al. supposed that the visual system cannotaccess surface re£ectance functions. Rather, it processes

Proc. R. Soc. Lond. B (1998) 265, 1837^1845 1837 & 1998 The Royal SocietyReceived 11 May 1998 Accepted 16 June 1998

perceived lightness, de¢ned as a nonlinear function ofsurface re£ectance.

There are two predictions from Metelli's constraintsthat could in£uence binocular depth perception. One isthat images composed of a product of two positive-valuedsignals may be perceived symmetrically; either signalmay be seen in front of, or behind, the other at di¡erentdepths depending upon the disparity cue present in eachsignal. In a companion paper (Langley et al. 1998), wehave con¢rmed these predictions by using similar stimulito those manipulated here. The other prediction is thatsecond-order binocular signals, like a positive-valuedcontrast envelope and a mean-zero carrier, will beperceived asymmetrically in depth. This is because Metel-li's monocular constraints would be violated when thedisparity cues imply that the carrier is the signal nearestin depth to the binocular observer, thus leading tocon£icting binocular and monocular depth relations.

Metelli's constraints make no predictions concerningadditive (¢rst-order) transparencies. To help explain the¢rst-order case, Weinshall (1991) showed that binocularcross correlation of the left and right images of additivetransparent signals gave two peaks at the disparities ofthe individual signals. One could modify this cross-corre-lation model to account for multiplicative transparencyby introducing an early retinal nonlinearity (e.g. Burton1973) or a later nonlinearity in the cortex (e.g. Langley1997). One nonlinearity that might be considered is loga-rithmic. It is well known that the logarithm of a productof two signals is equal to the sum of the logarithms of theindividual signals. With second-order (non-Fourier) bino-cular signals, the nonlinearity will introduce power at thefrequencies of the second-order envelope so that a single-channel model of transparency, such as the one proposedbyWeinshall (1991), could then detect disparities for both¢rst- and second-order signal combinations. Unfortu-nately, this model may encounter di¤culties whenpresented with ¢rst-order transparencies, because thelogarithm of the sum of two signals is no longer equal to asuperposition of the two signals.Instead of a single-channel model, many researchers

favour a two-channel model. Two-channel models includea separate nonlinear (second-order) channel to detect thesecond-order disparity signal for binocular matching.Models such as this have been proposed to explain theperception of second-order motion stimuli (Chubb &Sperling 1988) and stereopsis (Wilcox & Hess 1996).In analysing properties of second-order stimuli and

models, Fleet & Langley (1994) noted that idealizedsecond-order signals have a simple characterization interms of oriented power concentrations in the Fourierdomain. They showed that orientated power occurs withmultiplicative signal combinations such as those causedby multiplicative transparency and occlusion boundaries.Fleet & Langley further showed that the bandpass-¢ltered image signal may be separated into (¢rst-order)phase and (second-order) amplitude signals by alogarithmic transformation, which is consistent with thetwo-channel hypothesis. This approach is attractivebecause the binocular cross correlation of phase andamplitude may be used to detect transparencies for ¢rst-and second-order signal combinations. Hence, two-channel models may be motivated by a computational

strategy that leads to the recovery of binocular depth forboth ¢rst- and second-order signal combinations for thetwo types of transparency mentioned.In this paper, we examine the one- and two-channel

models, and one's perception of binocular transparencywith additive and multiplicative signal combinations. It isshown that depth perception for contrast envelopes is asym-metrical, consistent with Metelli's constraints. By varyingthe frequency of the contrast envelope, we also show thatminima in transparency thresholds for contrast envelopesoccur for frequencies approximately 2.4 octaves below thecarrier frequency. By comparison, no minima were foundfor ¢rst-order signals which were, broadly speaking,perceived symmetrically. The binocular asymmetry foundfor second-order signals suggests that di¡erent constraintsare used in processing ¢rst- and second-order signal combi-nations, implicating a two-channel model.

2. METHODS

(a) Apparatus and procedureMonocular images of the binocular stimuli were presented on

the left and right sides of a Sony monitor with a refresh rate of76Hz and 256 grey scales. The luminance of the monitor waslinearized by taking luminance measurements with a photo-meter, to which a logarithmic curve was ¢tted. This was thenused to generate a linear look-up table. The residual error fromthe ¢tted curve at any one luminance was no more than 0.2% ofthe ¢tted curve at any one of the sampled points. The meanluminance of the monitor was 37.7 cdmÿ2.

Experiments were done in a darkened room. The only visibleillumination originated from the monitor. A modi¢ed Wheat-stone stereoscope was used to view the binocular images on asingle monitor. The distance from the screen to the stereoscopewas 44 cm. The stereoscope was adjusted so that the stereoimage pairs were correctly aligned using a parallel viewinggeometry. The entire visual extent of each monocular image was7.98. Image pixels were square with a width of 2min. A ¢xationspot was used as a reference point to help keep vergence ¢xed.Subjects were seated with their heads stabilized in a chin rest infront of the stereoscope. They responded by using a computermouse in forced-choice discrimination tasks. Subjects were askedto respond as quickly as they could, but were not otherwiseconstrained by the viewing time.

(b) StimuliBoth contrast-modulated sinusoidal gratings and contrast-

modulated noise signals were used. Some examples are shown in¢gure 1. The raw envelope signal, E(x, y), was an approximationto a vertically orientated, square-wave grating. It was formed bysumming the ¢rst and third harmonics of the square-wave:

E(x,y) � ��sin(!x� �)� (1=3)sin3(!x� �)�, (1)

where ! was the fundamental frequency of the envelope, and �was chosen so that E(x, y) ranged between 1 and 71. The phaseof the envelope, �, was randomized between trials.

Given the envelope E(x, y), o¡set to make it strictly positive,along with carrier signals for the left and right eyes, Cl(x, y) andCr(x, y), the contrast-modulated second-order stimuli were givenby

1838 K. Langley and others Linear and nonlinear transparencies in binocular vision

Proc. R. Soc. Lond. B (1998)

Ml(x,y) � ��1� (a=2)(1� mE(x� d=2, y))Cl(x, y)�Mr(x, y) � ��1� (a=2)(1� mE(xÿ d=2, y))Cr(x,y)�,

(2)

where � is the mean illumination, m is the depth of contrastmodulation, a is the contrast of the carrier, and d is thepositional disparity. In addition, the subscripts l and r areused to denote the left and right eyes. The stimuli werevisible only within a circular window as shown in ¢gure 1.In each condition, the signal C(x, y) was either a horizontal

or vertical sinusoidal grating, or a mean-zero random-dotnoise pattern with zero disparity. The mean value of C(x, y)was always zero. When C(x, y) was a grating, its spatialfrequency was 2.25 or 4.5 cycles per degree (cpd). WhenC(x, y) was a noise pattern, each pixel was randomly assigned avalue of � 1. The depth of modulation m was 0.75, and thecontrast a was 0.98.

Additive combinations of the same signals were also used,and are given by

Linear and nonlinear transparencies in binocular vision K. Langley and others 1839


(a)

(b)

(c)

(d)

Figure 1. Examples of the binocular stimuli used. (a) A multiplicative (square-wave) contrast envelope multiplied by a sinusoidalgrating. (b) The superposition (addition) of a square-wave and sinusoidal grating. (c) A square-wave contrast envelope multipliedby a mean-zero noise pattern. (d) The superposition of a square-wave grating and a noise pattern. For each image triplet, cross-eyed fusion of the left and centre images reveals a vertical structure perceived transparently in front of the plane of ¢xation.However, cross-eyed fusion of the centre and right images may yield either a coherent (a), transparent (b, d) or diplopic (c)square-wave grating. Note that nonlinearities probably caused by the photographic procedure have reduced the saliency of thediplopic envelope shown in (c).

Al(x,y) � ��1� (a=2)Cl(x,y)� (b=2)E(x� d=2, y)�Ar(x,y) � ��1� (a=2)Cr(x, y)� (b=2)E(xÿ d=2, y)�. (3)

For additive signal combination conditions, a/2 and b/2 were¢xed at 0.45. The spatial frequencies of the additive gratingsmatched those used in the multiplicative conditions.

3. EXPERIMENT 1

The ¢rst experiment examined di¡erences between theprocessing of ¢rst- and second-order transparencies, byquantifying the symmetry of the transparent percept. Wemeasured the smallest disparity, d in equations (2) and (3),that was required by subjects to perceive transparency.Disparity was varied between trials using APE, anadaptive probit algorithm (Watt & Andrews 1981). Aftereach session, a psychometric function was ¢t to thesubject's responses byAPE.The disparity at which subjectsreported transparency on 50% of trials was deemed thedisparity threshold for perceived transparency. Eachsession consisted of 64 trials, and was repeated three times.The signal C(x, y) was either a horizontal sinusoidal

grating or a noise pattern. The other signal, E(x, y), wasalways vertical (as in equation (1)). Subjects reportedtransparency when the two signals E(x, y) and C(x, y)

were perceived to be at di¡erent depths. In the multipli-cative condition, when C(x, y) was a horizontal grating,we tested both crossed and uncrossed disparities. When itwas noise, we used only crossed disparities. We did thisbecause, when the disparity of the envelope E(x, y) wasuncrossed, the signal appeared diplopic and could not befused. For the additive signal combinations, crossed anduncrossed conditions were run as separate sessionsbecause there were separate disparity thresholds for each.Finally, note that when C(x, y) was a horizontal grating,

the two monocular images in both additive and multipli-cative conditions were shifted versions of one another.They were, therefore, consistent with a single coherentsurface under binocular viewing. In these cases thepercept was often bistable so that coherent and trans-parent percepts could be reported at di¡erent instances.When perception was bistable, subjects were instructed toreport transparency.

(a) Results and discussionWe summarize the data from the three subjects by

showing how transparent disparity thresholds (TDTs)varied as a function of the spatial frequency of E(x, y).Figure 2a shows the TDTs for each subject in themultiplicative condition when C(x, y) was a horizontal



1

2

4

8

16

PHKL

JB

KLPBHJB

0.20.1 0.4 0.8

Envelope Spatial Frequency (cpd)

0.20.1 0.4 0.8


0.20.1 0.4 0.8


Cro

ssed

Dis

pari

ty (

min

)U

ncro

ssed

Dis

pari

ty (

min

)

(a)

(d) (e)

(b) (c)

4.5 cpd Carrier2.25 cpd Carrier

Noise Carrier

0.20.1 0.4 0.8 1.6

Spatial Frequency (cpd)

16.0

8.0

4.0

2.0

1.0

0.5

4.5 cpd Grating2.25 cpd Grating

Noise Carrier2.25 cpd Grating4.5 cpd Grating

C

ross

ed D

ispa

rity

(m

in)

0.1

16.0

8.0

4.0

2.0

1.0

0.50.2 0.4 0.8 1.6

Spatial Frequency (cpd)

Figure 2. Transparent disparity thresholds are shown as a function of the fundamental frequency of E(x, y). (a) A multiplicativecondition is shown for three subjects. C(x, y) was a 4.5-cpd grating. (b) Mean transparency thresholds for two di¡erent frequenciesof C(x, y) are shown for the multiplicative condition. (c) Individual subject's responses when C(x, y) was noise. (d, e) Thresholdsfor additive combinations of E(x, y) and C(x, y), showing that transparency could be reported for both crossed and uncrosseddisparities. In (a) and (c) the error bars represent standard errors for each subject across di¡erent sessions. In (b), (d) and (e) theerror bars represent the standard error for each subject's thresholds across di¡erent sessions.

sinusoidal grating. The data for each subject show similarbehaviour, with the lowest threshold occurring near0.4 cpd, approximately two octaves lower than the carrierfrequency.

Figure 2b shows the results collapsed across the threesubjects for two di¡erent carrier frequencies, one octaveapart, at 2.25 and 4.5 cpd. The threshold functions inthese two cases appear to be shifted versions of oneanother. In particular, for the lower frequency, theminimum TDT was one octave lower, at about 0.2 cpdinstead of 0.4 cpd This suggests that the envelope TDTsdepended upon the carrier frequency.

For the noise carrier in the multiplicative condition,shown in ¢gure 2c, the TDTs also show a minimum atca. 0.4 cpd, as in ¢gure 2a. However, they do not vary asmarkedly as a function of envelope frequency. This may bedue to the broadband nature of the carrier, in which casethe thresholds measured here could re£ect the combinede¡ects of several spatial frequency tuned channels.

With second-order stimuli, transparency was onlyreported when the depth ordering of the envelope E(x, y)appeared nearer to the observer than the carrier C(x, y);i.e. when the envelope disparities were crossed. Althoughnot reported here, when C(x, y) was a vertical grating, thepercept of transparency (versus coherence) was found todepend upon the relative phase between C(x, y) and E(x, y)for both additive and multiplicative conditions. Similarobservations have been reported in the context of motioncapture (e.g. Gurney & Wright 1996). Capture andbistable percepts are common occurrences with trans-parent stimuli. They probably re£ect competitionbetween di¡erent visual processes (models) when theinput stimuli do not provide a su¤cient number ofconstraints (von Grunau & Dube 1993; Langley 1997).

Finally, ¢gure 2d,e shows the data from the additivecondition collapsed across the three subjects for bothcrossed and uncrossed disparities. The curves show thatfor additive transparency the TDTs decreased as afunction of the spatial frequency of E(x, y). No minimumwas evident over the range of frequencies tested. Figure2d,e also shows a di¡erence between crossed anduncrossed disparity cues when C(x, y) was additive noise.The slope of transparency thresholds as a function offrequency was less steep for the uncrossed than for thecrossed disparities.The transparency asymmetry observed with contrast

envelopes may be explained by Metelli's two constraints.The contrast envelope, 1+mE(x, y), is consistent with atransparent overlaying surface because it is a positive-valued function. The envelope does not change the sign ofthe underlying grating or noise carrier, and the intensitydi¡erences of the carrier at the location of the envelopetroughs are smaller than those at the crests. Allowing thecarrier to be the overlaying surface violates Metelli'ssecond constraint, because a contrast-modulated carrieris not a positive-valued function.These results favour a two-channel model over a

single-channel model. A single-model of transparency,such as that proposed by Weinshall (1991), that incorpo-rates a nonlinearity may detect two disparity signals, butit would not be able to determine the origin of eachdisparity signal (i.e. whether it is ¢rst- or second-order).Hence, this single-channel model would not be consistent

with the asymmetry. A two-channel model could explainthe asymmetry because each disparity signal would havebeen processed by a separate channel.To explain the symmetrical transparency observed in

the additive condition, note that additive transparenciesde¢ne the boundary condition of Metelli's secondinequality constraint (see Beck 1984). This is because theintensity di¡erences taken in a vertical direction for boththe dark and light regions of the vertical square-wavegrating are equal. In addition, the additive transparenciesmay be decomposed into a sum of two positive-valuedintensity functions. Therefore, there are no simple mono-cular cues that constrain either signal to be an overlayingor underlying surface, and so the disparity cues may beused to specify the depth ordering of the binocular signals.Figure 2d,e shows another di¡erence in the additive

noise condition. Although subjects reported transparencyfor both crossed and uncrossed disparities, the slope ofthe curves were less steep in the case of the uncrosseddisparities compared with the crossed. Noise patterns are,however, broadband stimuli. For these binocular signalpairs, there will be interference between the disparitycues present in E(x, y) and C(x, y), especially when theirindividual frequency spectra overlap and their binoculardisparities are di¡erent. The distortion products intro-duced into the image signal by an additive combination ofsignals may be second-order. These distortion productsmay occur as a result of either early visual nonlinearities,or image transformations that follow bandpass ¢lteringand binocular cross correlation. Therefore, the di¡erentslopes found for crossed and uncrossed disparities fromthe additive (broadband) signal combinations furtherimplicate second-order processes in binocular depthasymmetries.

Finally, it is interesting that in the additive conditionwhen C(x, y) was random noise, subjects reportedtransparency when the frequency of E(x, y) was greaterthan 0.8 cpd. In the other conditions, subjects wereunable to report transparency reliably at such higherfrequencies. Wilson et al. (1991) reported that two addedsignals must di¡er in scale by two or more octaves inorder for binocular depth transparency to be perceived.Our data are consistent with these reports but alsosuggest that frequency constraints apply to both ¢rst- andsecond-order signals. The dependence of transparencyupon spatial frequency is likely to re£ect the properties ofthe mechanisms used for binocular matching by thevisual system. This is because a dependency upon spatialfrequency may not be predicted from Metelli's (1974)monocular constraints. A two-channel model of transpar-ency may explain these trends. This model would positthat ¢rst-order transparencies are detected by bandpassprocesses tuned to di¡erent scales and/or orientation,whereas second-order transparent signals are detected bya spatial-frequency-selective second-order channel (e.g.Wilson & Kim 1994).

4. EXPERIMENT 2

Experiment 1 showed that the TDTs for multiplicativetransparency were minimal when the frequency of E(x, y)was relatively low, near 0.4 cpd. For the additivecondition, however, no minimum was found over the



same frequency range. One possibility is that theminimum found for the multiplicative condition isindicative of the peak sensitivity of a second-ordermechanism (see Sutter et al. 1995; Langley et al. 1996).

This possibility was the focus of the second experiment.In this experiment we manipulated the disparity d ofE(x, y)in a disparity discrimination task. However, in this case thesignals Cl(x, y) and Cr(x, y) were allowed to di¡er. Wemeasured the percentage of trials on which subjectsreported that the signal E(x, y) was seen in front of a ¢xa-tion spot. As in the previous experiment, d was variedusing APE. Disparity discrimination thresholds (DDTs)were determined from the slope of the psychometricfunction (¢tted byAPE) at about the point where subjectsreported `in front of ' on 50% of trials. Each sessionconsisted of 64 trials andwas repeated three times.

In multiplicative conditions, Cl(x, y) and Cr(x, y)di¡ered in three ways. In a matched condition, the leftand right gratings were both horizontal with the samespatial frequency. In a Vert^Hor condition, the twosignals di¡ered in orientation (one vertical and onehorizontal), but had the same spatial frequency (cf. Liuet al. 1992). Finally, in an F^3F condition, the twosignals were both horizontal, with frequencies of 1.5 and4.5 cpd. For the additive case, only two conditions werereported, namely the matched and F^3F conditions,because the curve shapes found for the additive condi-tions were similar. The contrast and spatial frequencyfor one of Cl(x, y) or Cr(x, y) was always the same as that

used in experiment 1. Figure 3 shows examples of thestimuli.

(a) Results and discussionResults for this experiment are shown in ¢gure 4.With

multiplicative signal combinations, the DDTs as functionof the frequency of E(x, y) show a minimum near 0.4 cpdfor both F^3F and Vert^Hor conditions. For the matchedcarrier condition, the DDTs £attened out at frequenciesabove 0.4 cpd. In the F^3F condition, subject DS wasunable to discriminate disparities for low or high spatialfrequencies of E(x, y). At these frequencies this subjectreported that he was only able to see the contrastenvelope in front of the ¢xation spot and was, therefore,unable to perform the discrimination task.

When E(x, y) and C(x, y) were added together, ¢gure 4bshows no minimum in DDTs. Rather, the DDTsdecreased as a function of increasing frequency. Thistrend may be explained by a ¢rst-order model ofstereopsis based upon phase di¡erences (Fleet et al. 1996).To transform an interocular phase-di¡erence into a bino-cular disparity, one must divide the phase di¡erence bythe horizontal frequency of the signal. This leads to adisparity error that is inversely proportional to the spatialfrequency of the signal. The predictions from this modelare shown in the left panel of ¢gure 4b. The DDTs shownin ¢gure 4b are broadly similar to those predicted by sucha model (PBH's thresholds were consistently £atter acrossall conditions reported here.)



(a)

(b)

Figure 3. Example of the binocular stimuli used for experiment 2. (a) Multiplicative: crossed-eyed fusion yields a strong perceptof depth when the centre and right images are fused. When the carrier gratings for the envelope are orientated orthogonally, thepercept of depth is much weaker, but disparity discrimination thresholds may still be determined. For these stimuli, the envelopewas not seen in depth behind its carrier grating. (b) Additive: the matching vertical gratings could be seen both in front of andbehind the rivalling gratings. The perceived depth of the rivalling gratings may be seen to lie in the plane of the ¢xation spot (seealso Wurger & Landy 1989).

Schor et al. (1984) have measured DDTs for ¢rst-orderstimuli. They used di¡erence-of-Gaussian (DoG) signalsand found that the DDTs decreased as the peak frequencyof the DoG functions increased (up to 2.5 cpd). Therefore,the decreasing DDTs found over this range of frequenciesare consistent with those expected from a ¢rst-ordermechanism, as explained above. (The £attening of DDTsabove 2.5 cpd may re£ect position-shifted disparity detec-tors (cf. Fleet et al. 1996).) These trends were found in¢gures 2d,e and 4b when E(x, y) was an additive (¢rst-order) signal. The similarity between the disparity thresh-olds for DoG stimuli, transparency thresholds (experiment1) and DDTs (experiment 2) suggest that the binoculardisparities for additive conditions were detected by ¢rst-order processes.When E(x, y) was multiplied (second-order) with C(x, y),

the thresholds for transparency and DDTs showed adi¡erent trend. Figures 2a^ c and 4a show that a minimumoccurred near 0.4 cpd. Such a minimum does not re£ectthe curve shapes that would be expected from a ¢rst-ordermodel of binocular matching.The similarities between thecurve shapes for the envelope signal E(x, y) across the twoexperiments, and the di¡erences between ¢rst- andsecond-order conditions suggest that there was a common,

and probably second-order, process that mediated thedisparity signal.In both experiments 1 and 2, our results show that

transparency thresholds for contrast envelopes ¢rstdecreased, and then increased as a function of thefrequency of E(x, y). This trend was pervasive in our data,and suggests that the processing of disparity fromcontrast envelopes was frequency selective. Consistentwith this, Langley et al. (1996) found that sensitivity forthe detection of envelope spatial orientation was maximalat approximately one-tenth of the carrier spatialfrequency. Sutter et al. (1995), using bandpass-¢lteredrandom-dot noise patterns, reported that sensitivity toenvelope frequencies was maximum when the envelopewas approximately one-eighth to one-sixteenth of thecentre frequency of bandpass noise carriers. In comparingthe results of Sutter et al. and of Langley et al., note thatnarrowband ¢ltering itself introduces contrast variations,which will probably a¡ect the resultant contrast-envelopesensitivity curves. This may explain why the envelopesensitivity curves reported by Langley et al. were moretightly concentrated about their minimum than thosereported by Sutter et al. A similar feature can be seen bycomparing ¢gure 2b with 2c. The curves for the perceived



128.00

32.00

8.00

2.00

0.50

0.13

(a)

(b)

KL DSPBHMatched CarrierVert-HorF-3F

0.1 0.2 0.4 0.8 1.6


Dis

crim

inat

ion

Thr

esho

ld (

min

/min

)

0.1 0.2 0.4 0.8 1.6


0.1 0.2 0.4 0.8 1.6


128.00

32.00

8.00

2.00

0.50

0.130.1

Dis

crim

inat

ion

Thr

esho

ld (

min

/min

)

0.2 0.4 0.8 1.6 0.1 0.2 0.4 0.8 1.6 0.1 0.2 0.4 0.8 1.6

Spatial Frequency (cpd) Spatial Frequency (cpd) Spatial Frequency (cpd)

KL DSPBH

Matched GratingF-3F1/F

Figure 4. (a) DDTs for the multiplicative condition are plotted as a function of envelope spatial frequency on log^log axes. (b)Shows each subject's data for the additive condition. The dashed line labelled 1/F shown in the left panel of (b) is the relationshippredicted if DDTs are proportional to wavelength in a horizontal direction. In these ¢gures, the error bars represent the standarderror of each subject's thresholds taken across di¡erent sessions.

transparency thresholds are more tightly tuned abouttheir minimum for the grating condition than the noisecondition. These data could be explained if the second-order channel pooled the responses from di¡erent spatial-frequency-tuned channels, because random-dot noisepatterns are broadband patterns.

Finally, ¢gure 4a shows that the DDTs for the(second-order) matched condition di¡er from the othermultiplicative conditions. Here the DDTs decreased as afunction of the spatial frequency of E(x, y) and then £at-tened out at frequencies above 0.4 cpd. Morgan & Castet(1997) found that the DDTs for sinusoidal gratings as afunction of orientation, were constant when measured byinterocular phase di¡erences. For the matched condition,this model would posit that DDTs vary as a function ofenvelope spatial frequency with a slope of 1/F, as in theadditive conditions mentioned earlier. This is because anincrease in the envelope spatial frequency will decreasethe orientation of the envelope's sidebands or linearfrequency components. This prediction is inconsistentwith the trend reported for this matched condition. Onecan also note that the £attening of the DDTs occurred ataround 0.4 cpd rather than at 2.5 cpd, as in ¢rst-ordersignals. This di¡erence could re£ect an average takenbetween ¢rst- and second-order processes. For higherspatial frequencies, the increasing DDTs from a second-order process may have been o¡set by decreasing ¢rst-order thresholds (cf. Lin & Wilson 1995).

5. GENERAL DISCUSSION

There is growing evidence for ¢rst- and second-orderprocessing in binocular stereopsis. Hess & Wilcox (1994)showed that envelope disparities can in£uence stereo-acuity, especially when the envelope and the carrier havedi¡erent disparities. They concluded that stereo-acuitydepends on the envelope size when the stimulus band-width is smaller than 0.5 octaves. Sato & Nishida (1993)presented subjects with second-order random-dotstereograms, much like some of the stimuli used in studiesof second-order motion. They found that the upper limitson disparity were lower with the second-order stimulusthan with conventional random-dot stereograms. Liu etal. (1992) using similar stimuli to the ones manipulatedhere, reported that, when presented with binocularGabor stimuli in which the left- and right-eye sinusoidalcarriers were perpendicular to one another, subjectsperceived depth correctly from the envelope while thecarrier components were in binocular rivalry.

Although many of these data support the idea of a two-channel hypothesis, they do not entirely rule out thepossibility that a single-channel model of transparency,which exploits a nonlinearity introduced into the bino-cular pathway, could explain binocular depth perceptionfor second-order signals. A key feature of our results infavour of the two-channel model is the transparentasymmetry reported for second-order signals (see alsoFrisby & Mayhew 1978; Kersten 1991). It would bedi¤cult to explain the depth asymmetry by a single-channel model of transparency because the origin of thetwo disparity signals would be undiscernible. On theother hand, a two-channel model may explain the depthasymmetry because second-order disparity signals would

be represented separately. The frequency tuning curvesreported here for contrast envelopes resemble those foundin spatial vision tasks (Sutter et al. 1995; Langley et al.1996). Again, it would be di¤cult to explain howdi¡erent tuning curves between ¢rst- and second-ordersignals could arise from a single-channel model of trans-parency.

Our data also support the idea that Metelli's (1974)constraints on monocular transparency a¡ect binoculardepth perception (Kersten 1991). However, our dataimplicate one simple strategy by which Metelli'sconstraints may be introduced into visual processes,namely, a second-order channel. This is because a contrastenvelope, as detected by a second-order channel, is apositive-valued signal (Fleet & Langley 1994). The two-channel model posits that ¢rst-order transparencies maybe detected by multiple peaks in the interocular cross-correlation function of a ¢rst-order channel as inWeinshall(1991), whereas second-order signals may be detected byinterocular cross correlations of the second-order channel.Hence, a two-channel model could re£ect a combinedstrategy exploited by the visual system that leads to thedetection of binocular disparities for both ¢rst- andsecond-order signal combinations. This view supports thenotion that the visual system may represent multiple depthcues at common image locations and that the motivationfor a two-channel model stems from the incompatiblenature of additive and multiplicative signal combinations.

Portions of this research were presented at the EuropeanConference on Visual Perception (ECVP) in 1994 and 1995.D.J.F. was funded in part by NSERC Canada, and by an AlfredP. Sloan Research Fellowship.

REFERENCES

Beck, J. 1984 Perception of transparency in man and machine.Hum. Mach.Vision 2, 1^12.

Beck, J. Prazdny, K. & Ivry, R. 1984 The perception oftransparency with achromatic colors. Percept. Psychophys 5,407^422.

Burton, G. J. 1973 Evidence for non-linear response process inthe visual system from measurements on the thresholds ofspatial beat frequencies.Vision Res. 13, 1211^1255.

Chubb, C. & Sperling, G. 1988 Drift-balanced random-stimuli:a general basis for studying non-Fourier motion perception. J.Opt. Soc. Am. 5, 1986^2007.

Fleet, D. J. & Langley, K. 1994 Computational analysis of non-Fourier motion.Vision Res. 34, 3057^3079.

Fleet, D. J., Wagner, H. & Heeger, D. J. 1996 Neural encodingof binocular disparity: energy models, position-shifts andphase-shifts.Vision Res. 36, 1839^1857.

Frisby, J. P. & Mayhew, J. E. W. 1978 The relationship betweenapparent depth and disparity in rivalrous-texture stereo-grams. Perception 7, 661^678.

Gurney, K. N. & Wright, M. J. 1996 A model for the spatialintegration and di¡erentiation of velocity signals. Vision Res.36, 2939^2955.

Hess, R. F. & Wilcox, L. M. 1994 Linear and non-linear¢ltering in stereopsis.Vision Res. 34, 2431^2438.

Kersten, D. 1991 Transparency and cooperative computation ofscene attributes. In Computational models of visual processing (ed. M.Landy & J. A. Movshon), pp. 209^228. London: MIT Press.

Langley, K. 1997 Degenerate models of multiplicative and addi-tive motion transparency. In Proc. British Mach. Vision Conf.Colchester, vol. 2, pp. 440^450.



Langley, K., Fleet, D. J. & Hibbard, P. B. 1996 Linear ¢lteringprecedes nonlinear processing in early vision. Curr. Biol. 6,891^896.

Langley, K. Fleet, D. J. & Hibbard, P. B. 1998 Stereopsis fromcontrast envelopes.Vision Res. (Submitted.)

Liu, L. Schor, C. W. & Ramachandran, V. S. 1992 Positionaldisparity is more e¤cient in encoding depth than phasedisparity. Invest. Ophthalmol.Vis. Sci. 33(Suppl.), 1373.

Lin, L. & Wilson, H. R. 1995 Stereoscopic integration ofFourier and non-Fourier patterns. Invest. Ophthalmol. Vis. Sci.36(Suppl.), 364.

Marr, D. 1982 Vision. NewYork:W. H. Freeman & Co.Metelli, F. 1974 The perception of transparency. Scient. Am. 230,

90^98.Morgan, M. J. & Castet, E. 1997 The aperture problem instereopsis.Vision Res. 37, 2737^2744.

Sato, T. & Nishida, S. 1993 Second-order depth perception withtexture-de¢ned random-dot stereograms. Invest. Ophthalmol.Vis. Sci. 34(Suppl.), 1438.

Schor, C. M., Wood, I. C. & Ogawa, J. 1984 Spatial tuningof static and dynamic local stereopsis. Vision Res. 24,573^578.

Sutter, A., Sperling, G. & Chubb, C. 1995 Measuring thespatial-frequency selectivity of 2nd-order mechanisms. VisionRes. 35, 915^924.

von Grunau, M. & Dube. S. 1993 Ambiguous plaids: switchingbetween coherence and transparency. Spatial Vision 7, 199^211.

Watt, R. & Andrews, D. P. 1981 Adaptive probit estimation ofpsychometric functions. Curr. Psychol. Rev. 1, 205^214.

Weinshall, D. 1991 Seeing g̀host' planes in stereo vision. VisionRes. 31,1731^1749.

Wilcox, L. M. & Hess, R. F. 1996 Is the site of nonlinear¢ltering in stereopsis before or after binocular combination.Vision Res. 36, 391^399.

Wilson, H. R. & Kim, J. 1994 A model for motion coherenceand transparency.Vis. Neurosci. 11, 1205^1220.

Wilson, H. R., Blake, R. & Halpern, L. 1991 Coarse spatialscales constrain the range of binocular fusion on ¢ne scales.Vision Res. 8, 229^236.

Wurger, S. M. & Landy, M. S. 1989 Depth interpolation withsparse disparity cues. Perception 18, 39^54.

As this paper exceeds the maximum length normally permitted,the authors have agreed to contribute to production costs.



linear and nonlinear transparencies in binocular vision

Documents