free-energy and illusions: the cornsweet effectkarl/free-energy and illusions the cornsweet... ·...
TRANSCRIPT
001 058
002 059
003 060
004 061
005 062
006 063
007 064
008 065
009 066
010 067
011 068
012 069
013 070
014 071
015 072
016 073
017 074
018 075
019 076
020 077
021 078
022 079
023 080
024 081
025 082
026 083
027 084
028 085
029 086
030 087
031 088
032 089
033 090
034 091
035 092
036 093
037 094
038 095
039 096
040 097
041 098
042 099
043 100
044 101
045 102
046 103
047 104
048 105
049 106
050 107
051 108
052 109
053 110
054 111
055 112
056 113
057 114
ORIGINAL RESEARCH ARTICLEpublished: xx February 2012
doi: 10.3389/fpsyg.2012.00043
Free-energy and illusions: the Cornsweet effectHarriet Brown* and Karl J. Friston
The Wellcome Trust Centre for Neuroimaging, University College London Queen Square, London, UK
Edited by:
Lars Muckli, University of Glasgow,UK
Reviewed by:
Gerrit W. Maus, University ofCalifornia at Berkeley, USAPeter De Weerd, MaastrichtUniversity, Netherlands
*Correspondence:
Harriet Brown, Wellcome Trust Centrefor Neuroimaging, Institute ofNeurology Queen Square, LondonWC1N 3BG, UK.e-mail: [email protected]
In this paper, we review the nature of illusions using the free-energy formulation of Bayesianperception. We reiterate the notion that illusory percepts are, in fact, Bayes-optimal andrepresent the most likely explanation for ambiguous sensory input. This point is illustratedusing perhaps the simplest of visual illusions; namely, the Cornsweet effect. By usingplausible prior beliefs about the spatial gradients of illuminance and reflectance in visualscenes, we show that the Cornsweet effect emerges as a natural consequence of Bayes-optimal perception. Furthermore, we were able to simulate the appearance of secondaryillusory percepts (Mach bands) as a function of stimulus contrast. The contrast-dependentemergence of the Cornsweet effect and subsequent appearance of Mach bands weresimulated using a simple but plausible generative model. Because our generative modelwas inverted using a neurobiologically plausible scheme, we could use the inversion as asimulation of neuronal processing and implicit inference. Finally, we were able to verify thequalitative and quantitative predictions of this Bayes-optimal simulation psychophysically,using stimuli presented briefly to normal subjects at different contrast levels, in the contextof a fixed alternative forced choice paradigm.
Keywords: free-energy, perception, Bayesian inference, illusions, Cornsweet effect, perceptual priors
INTRODUCTIONIllusions are often regarded as “failures” of perception; however,Bayesian considerations often provide a principled explanation forapparent failures of inference in terms of prior beliefs. This paperis about the nature of illusions and their relationship to Bayes-optimal perception. The main point made by this work is thatillusory percepts are optimal in the sense of explaining sensationsin terms of their most likely cause. In brief, illusions occur when theexperimenter generates stimuli in an implausible or unlikely way.From the subject’s perspective, these stimuli are ambiguous andcould be explained by different underlying causes. This ambiguityis resolved in a Bayesian setting, by choosing the most likely expla-nation, given prior beliefs about the hidden causes of the percept.This key point has been made by many authors (e.g., Purves et al.,1999). Here, we develop it under biologically realistic simulationsof Bayes-optimal perception and try to make some quantitativepredictions about how subjects should make perceptual decisions.We then try to establish the scheme’s validity by showing that thesepredictions are largely verified by experimental data from normalsubjects viewing the same stimuli.
The example we have chosen is the Cornsweet illusion, whichhas a long history, dating back to the days of Helmholtz (Mach,1865; O’Brien, 1959; Craik, 1966; Cornsweet, 1970). This is partic-ularly relevant given our formulation of the Bayesian brain is basedupon the idea that the brain is a Helmholtz or inference machine(von Helmholtz, 1866; Barlow, 1974; Dayan et al., 1995; Fristonet al., 2006). In other words, the brain is trying to infer the hiddencauses and states of the world generating sensory information,using predictions based upon a generative model that includesprior beliefs. We hoped to show that the Cornsweet effect canbe explained in a parsimonious way by some simple prior beliefs
about the way that visual information is generated at differentspatial and temporal scales.
THE CORNSWEET EFFECT AND THE NATURE OF ILLUSIONSFigure 1 provides an illustration of the Cornsweet illusion. Theillusion is the false percept that the peripheral regions of a stimulushave a different brightness, despite the fact they are physically iso-luminant. This illusion is induced by a biphasic luminance “edge”in the centre of the field of view (shown in the right hand col-umn of Figure 1). The four rows of Figure 1 show the Cornsweeteffect increasing in magnitude as we increase the contrast of thestimulus. Interestingly, at high levels of contrast, secondary illu-sions – Mach bands (Mach, 1865; Lotto et al., 1999) – appear atthe para-central points of inflection of the true luminance profile.It is this contrast-dependent emergence of the Cornsweet effectand subsequent Mach bands that we wanted to simulate, underthe assumption that perception is Bayes-optimal.
The Bayesian aspect of perception becomes crucial when weconsider the nature of illusions. Bayesian theories of perceptiondescribe how sensory data (that have a particular likelihood) arecombined with prior beliefs (a prior distribution) to create a per-cept (a posterior distribution). One can regard illusory perceptsas those that are induced by ambiguous stimuli, which can becaused in different ways – in other words, the probability of thedata given different causes or explanations is the same. When facedwith these stimuli, the prior distribution can be used to create aunimodal posterior and an unambiguous percept. If the perceptor inference about the hidden causes of sensory information (theposterior distribution) is different from the true causes used togenerate stimuli, the inference is said to be illusory or false. How-ever, with illusory stimuli the mapping of hidden causes to their
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 1
115 172
116 173
117 174
118 175
119 176
120 177
121 178
122 179
123 180
124 181
125 182
126 183
127 184
128 185
129 186
130 187
131 188
132 189
133 190
134 191
135 192
136 193
137 194
138 195
139 196
140 197
141 198
142 199
143 200
144 201
145 202
146 203
147 204
148 205
149 206
150 207
151 208
152 209
153 210
154 211
155 212
156 213
157 214
158 215
159 216
160 217
161 218
162 219
163 220
164 221
165 222
166 223
167 224
168 225
169 226
170 227
171 228
Brown and Friston Free-energy and illusions: the Cornsweet effect
FIGURE 1 |The Cornsweet illusion and Mach bands. The Cornsweetillusion is the false perception that the peripheral regions of a Cornsweetstimulus have different reflectance values. The magnitude of the effectincreases as the contrast of the stimulus increases. At higher levels ofcontrast, the secondary illusion – Mach bands – appear. The Mach bandsare situated at the point of inflection of the luminance gradient.
sensory consequences is ill-posed (degenerate or many to one),such that a stimulus can have more than one cause. Thus, fromthe point of view of the observer, there can be no “false” infer-ence unless the true causes are known. The perceptual inferencecan be optimal in a Bayesian sense, but is still illusory. However,not all possible causes of sensory input will be equally likely, sothere will be an optimal inference in relation to prior beliefs abouttheir causes. Prior beliefs can be learnt or innate: priors that arelearnt depend upon experience while innate priors can be asso-ciated with architectural features of the visual brain, such as thecomplex arrangement of blobs, interblobs, and stripes in V1, thatmay reflect priors on the statistical structure of visual informationselected by evolutionary pressure.
Prior beliefs are essential when resolving ambiguity or the ill-posed nature of perceptual inverse problems. Put simply, there willalways be an optimal posterior estimate of what caused a sensationthat rests upon prior beliefs. The example in Figure 2 illustratesthis: The central panel shows an ambiguous stimulus (luminanceprofile) that is formally similar to the sort of stimulus that inducesthe Cornsweet illusion. However, this stimulus can be caused in
FIGURE 2 | Contributions to luminance. Luminance values reaching theretina are modeled as a multiplication of illuminance from light sources andreflectance from the surfaces in the environment. The factorization ofluminance is thus an ill-posed problem. One toy example of this degeneracyis shown here; the same stimulus can be produced by (at least) twopossible combinations of illuminance and reflectance. Prior beliefs aboutthe likelihood of these causes can be used to pick the most likely percept.
an infinite number of ways. We have shown two plausible causesby assuming the stimulus is the product of (non-negative) illu-minance and reflectance profiles. The lower two panels show the“true” causes generating stimuli for the Cornsweet illusion. Here,the stimulus has a reflectance profile that reproduces the Corn-sweet stimulus and is illuminated with a uniform illuminant. Analternative explanation for exactly the same stimulus is providedin the upper two panels, in which two isoreflectant surfaces areviewed under a smooth gradient of ambient illumination. In thisexample, we have ensured that both the illuminant and reflectanceare non-negative by applying an exponential transform beforemultiplying them to generate the stimulus.
The key point made by Figure 2 is that there are many pos-sible gradients of illuminance and reflectance that can producethe same pattern of sensory input (luminance). These differ-ent explanations for a particular stimulus can only be distin-guished by priors on the spatial and temporal characteristics ofthe reflectance and illuminance. In this example, the ambiguityabout what caused the stimulus can be resolved if we believe,a priori, that the visual world is composed of isoreflectant sur-faces, as opposed to surfaces that (implausibly) get brighter ordarker nearer their edges or occlusions (as in the lower panels).Under this prior assumption, an observer who infers the pres-ence of spatially extensive isoreflectant surfaces, and explains theedge at the centre with an illuminance gradient, would be infer-ring its most likely cause. The Cornsweet “illusion” is thus only anillusion because the experimenter has chosen an unlikely combi-nation of illuminance and reflectance profiles. In what follows, wewill exploit priors on the spatial composition and generation of
Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 2
229 286
230 287
231 288
232 289
233 290
234 291
235 292
236 293
237 294
238 295
239 296
240 297
241 298
242 299
243 300
244 301
245 302
246 303
247 304
248 305
249 306
250 307
251 308
252 309
253 310
254 311
255 312
256 313
257 314
258 315
259 316
260 317
261 318
262 319
263 320
264 321
265 322
266 323
267 324
268 325
269 326
270 327
271 328
272 329
273 330
274 331
275 332
276 333
277 334
278 335
279 336
280 337
281 338
282 339
283 340
284 341
285 342
Brown and Friston Free-energy and illusions: the Cornsweet effect
visual input to simulate the Cornsweet effect and the emergenceof Mach bands.
The Bayesian approach to visual perception has been exploitedin previous work (Yuille et al., 1991; Knill and Pouget, 2004).In addition, several other visual illusions have been explainedusing Bayesian principles, including motion illusions (Weiss et al.,2002), the sound-induced flash illusion (Shams et al., 2005), andthe Chubb illusion (Lotto and Purves, 2001). Additionally, Purveset al. (1999) demonstrated the Bayesian nature of the Cornsweetillusion: when presented in a context implying an illuminance gra-dient and reflectance step, the Cornsweet illusion is elicited moreeasily.
In terms of the neuronal systems mediating the Cornsweetillusion; some authors have implicated subcortical structures: forexample, Anderson et al. (2009) found that BOLD signal in thelateral geniculate nuclei (LGN) best correlated with perceptionof the Cornsweet illusion, although correlations were also seenin visual cortex. Furthermore, the illusion could be abolished ifthe stimulus was not presented binocularly, suggesting an originbefore V1. Mach bands similarly have been attributed to retinalmechanisms (e.g., Ratliff, 1965); however, Lotto et al. (1999) havesuggested a high-level contextual explanation for their appearance.Irrespective of the cortical or subcortical systems involved, we willassume, in this paper that the same Bayesian principles operateand, crucially, rest on a hierarchical generative model that neces-sarily implicates distributed neuronal processing at the subcorticaland cortical levels.
OVERVIEWThis paper comprises three sections. The first describes a simplegenerative model of visual input that entails prior beliefs abouthow visual stimuli are generated and can be used to infer theircauses. This model is used in the second section to simulatethe perception of the Cornsweet illusion and contrast-dependentemergence of Mach bands. In the third section, we test the pre-dictions of the simulations using a psychophysics study of normalsubjects.
A GENERATIVE MODEL FOR THE CORNSWEET EFFECTOur simulations are based upon the free-energy formulation ofBayes-optimal perception. Put briefly, this is based upon thenotion that self-organizing agents minimize the average surprise(entropy) of sensory inputs through minimizing a free-energybound on surprise. Here, surprise is just the improbability ofsampling some sensory information, in relation to a (generative)model of how those sensations were produced. By adjusting thefree parameters of the model, the sensory information can beexplained or predicted and surprise minimized. Mathematically,surprise is the (negative) log evidence for a model of the worldthat comprises hidden variables (causes and states) that generatesensory information. We have described in many previous publica-tions how this principle leads to active inference and Bayes-optimalperception (Friston et al., 2006; Friston, 2009; Feldman and Fris-ton, 2010). Free-energy is a function of sensory samples and aprobabilistic representation of what caused those samples. Thisrepresentation can be cast in terms of the most likely or expectedstates of the world, under a generative model of how they conspire
to produce sensory inputs. In brief, once we know the agent’s gen-erative model, one can use the free-energy principle to predict itsbehavior and perception. In the present context, our focus willbe on perception and the role of prior beliefs that are an inher-ent part of the generative model. In what follows, we describethe model and then use it to simulate perceptual inference andelectrophysiological responses.
THE GENERATIVE MODELThe generative model we used is straightforward: sensory inputis the product of reflectance and illuminance, where illuminancevaries smoothly over space but can fluctuate with a high frequencyover time. Conversely, the reflectance profile of the visual world iscaused by isoreflectant fields or surfaces that fluctuate smoothlyin time. Crucially, the spatial scales over which these fluctua-tions occur have a scale-free nature, of the sort found in naturalimages (Burton and Moorhead, 1987; Field, 1987; Tolhurst et al.,1992; Ruderman and Bialek, 1994; Ruderman, 1997). To ensurepositivity of the illuminant and reflectance we apply an expo-nential transform to the two factors before multiplying them (asin Figure 2). Equivalently, we can imagine the underlying causes(reflectance and illuminance) as being composed additively in log-space. This model is shown schematically in Figure 3, in termsof hidden causes and states. Mathematically, this model can beexpressed as:
s = g (x , v) = exp (R · x + I · vI ) + ωs
x = f (x , v) = vR − x + ωx(1)
Here, s(t ) are sensory signals generated from hidden statesx(t ) and causes, v(t ) plus some random fluctuations ω(t ). Thedifference between hidden causes and states is that states evolvedynamically, in response to perturbations by hidden causes –these dynamics are described by the equation of motion in thesecond equality above. Hidden causes v(t ) = (vI, vR) have beendivided into those causing changes in luminance and those causingchanges in hidden states that produce changes in reflectance. Thesehidden variables control the amplitude of spatial basis functions(R, I) encoding formal beliefs about the spatial scales of illumi-nance and reflectance. For the illuminant (I) we use a low-orderdiscrete cosine transform, while for the reflectant (R) we use alow-order discrete wavelet transform.
The particular wavelet transform used here is a Haar waveletset that has been thinned by removing high-order wavelets (withhigh spatial frequency) from the periphery of the visual field: Thisrespects, roughly, the increasing size of classical receptive fieldswith retinotopic eccentricity. For simplicity (and ease of report-ing the results), we restrict the simulations to a one-dimensionalvisual field. Because Haar wavelets afford local linear approxima-tions to continuous reflectance profiles, the resulting reflectancehas to be a mixture of isoreflectant surfaces at different spatialscales. To impose the scale-free aspect, we decrease the varianceor, equivalently, increase the precision of the reflectance waveletcoefficients or hidden states in proportion to the order or spatialscale of their wavelet. This is implemented by placing a prior onthe wavelet coefficients with the form p(xk) = N (0, e−3k ), where kis the order of the wavelet. Neuronally, these basis functions could
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 3
343 400
344 401
345 402
346 403
347 404
348 405
349 406
350 407
351 408
352 409
353 410
354 411
355 412
356 413
357 414
358 415
359 416
360 417
361 418
362 419
363 420
364 421
365 422
366 423
367 424
368 425
369 426
370 427
371 428
372 429
373 430
374 431
375 432
376 433
377 434
378 435
379 436
380 437
381 438
382 439
383 440
384 441
385 442
386 443
387 444
388 445
389 446
390 447
391 448
392 449
393 450
394 451
395 452
396 453
397 454
398 455
399 456
Brown and Friston Free-energy and illusions: the Cornsweet effect
FIGURE 3 |The generative model: the generative model employed in thispaper models illuminance as a discrete cosine function and reflectance as aHarr wavelet function with peripheral high frequency wavelets removed. Inaddition, illumination is allowed to change quickly over time, whereasreflectance varies more slowly. This is achieved by making the coefficients on
the reflectance basis functions hidden states, which accumulate hiddencauses to generate changes in reflectance. Inversion of the model providesconditional estimates of the hidden causes and states responsible forsensory input as a function of time. See main text for an explanation of thevariables in this figure.
stand in for a filling-in process such as that described by Gross-berg and Hong (2006). Conversely, the illuminant is modeled asa mixture of smoothly varying cosine functions with a low spatialfrequency. This is easily motivated by the fact that most sourcesof illumination are point sources, which results in smooth illu-minance profiles. These were modeled here with the first threecomponents of a discrete cosine transform (see Figure 4 for agraphical representation of the basis functions and how they areused to generate a stimulus).
By construction, this generative model of visual signals sep-arates the spatial scales or frequencies of the illuminance andreflectance such that all the high frequency components are inthe reflectance profile, while the low frequency components are inilluminance profile. Temporal persistence of reflectance is assuredbecause the reflectance coefficients x(t ) ∈ �16 × 1 are hidden statesthat accumulate hidden causes vR(t ) ∈ �16 × 1. This persistencereflects the prior belief that surfaces move in a continuous fashion.For simplicity, we mapped the hidden causes controlling illumi-nance vI(t ) ∈ �3 × 1 directly to the stimulus (although this is not animportant feature of our model). This can be thought of as accom-modating rapid changes in illuminance of the sort that might beproduced by a flickering candle.
Equation 1 defines our generative model in terms of the jointprobability over sensory information and the hidden variablesproducing fluctuations in reflectance and illuminance. The fluc-tuations in the hidden causes are assumed to be Gaussian with aprecision (inverse variance) of one, while the fluctuations in the
FIGURE 4 |Two-dimension example. The top panels show 2-D examplesof luminance (left) and reflectance (right), created form the generativemodel. The resulting stimulus is the product of the two (bottom panel).
motion of the hidden states are assumed to have a log-precisionof 12. Finally, we assume sensory fluctuations or noise with alog-precision of six. In the next section, we will manipulate the
Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 4
457 514
458 515
459 516
460 517
461 518
462 519
463 520
464 521
465 522
466 523
467 524
468 525
469 526
470 527
471 528
472 529
473 530
474 531
475 532
476 533
477 534
478 535
479 536
480 537
481 538
482 539
483 540
484 541
485 542
486 543
487 544
488 545
489 546
490 547
491 548
492 549
493 550
494 551
495 552
496 553
497 554
498 555
499 556
500 557
501 558
502 559
503 560
504 561
505 562
506 563
507 564
508 565
509 566
510 567
511 568
512 569
513 570
Brown and Friston Free-energy and illusions: the Cornsweet effect
log-precision of the sensory noise as a proxy for changing thecontrast of the stimulus.
Figure 4 shows a snapshot of the sort of visual signals this gen-erative model produces. Here, we have used the outer product ofthe discrete transforms above to generate a two-dimensional stim-ulus. We are not pretending that this is a veridical model of thereal visual world. However, it is sufficient to explain the Cornsweetillusion and related effects by incorporating simple and plausiblepriors on the spatial scales over which illuminance and reflectancechange. In the next section, we use this generative model to sim-ulate perceptual and physiological responses to a stimulus, underthe free-energy principle. This reduces to a Bayesian deconvolu-tion of sensory input that tries to discover the most likely hiddencauses and states generating that input.
PERCEPTION AND PREDICTIVE CODINGThis perceptual deconvolution can be regarded as the inversion ofa generative model that maps from hidden causes (variables in theworld) to sensory consequences. The inverse mapping corresponds
to inferring those variables by mapping back from the sensoryconsequences to the hidden causes and states. This can be imple-mented in a biologically plausible fashion using a generalizedgradient descent on variational free-energy F(s, μ); which is afunction of (generalized) sensory states s(t ) = (s, s ′, s ′′, . . .) andthe expected values μ(t ) = (μx, μv) of hidden variables (see Fris-ton, 2008 for details). In brief, this gradient descent correspondsto a Bayesian filtering, in which expected states of the world arecontinuously optimized using a prediction term and an updateterm:
˙μ = Dμ − ∂
∂μF(s, μ) = Dμ − ∂ εT
∂μξ (2)
Under the simplifying assumption that probabilities are repre-sented as Gaussian densities, this can be regarded as a generalizedform of Kalman filtering, where the second (update or gradient)term can be expressed as a mixture of prediction errors (see theequations in Figure 5). This means that the generalized filtering
FIGURE 5 | Hierarchical message-passing in the brain. The schematicdetails a neuronal architecture that optimizes the conditional expectations ofhidden variables in hierarchical models of sensory input of the sortillustrated in Figure 3. It shows the putative cells of origin of forward drivingconnections that convey prediction error from a lower area to a higher area(red arrows) and non-linear backward connections (black arrows) thatconstruct predictions (Mumford, 1992; Friston, 2008). These predictions tryto explain away (inhibit) prediction error in lower levels. In this scheme, thesources of forward and backward connections are superficial and deeppyramidal cells (triangles) respectively, where state-units are black anderror-units are red. The equations represent a generalized gradient descenton free-energy (see main text) using the generative model of the previousfigure. If we assume that synaptic activity encodes the conditionalexpectation of states, then recognition can be formulated as a gradientdescent on free-energy. Under Gaussian assumptions, these recognitiondynamics can be expressed compactly in terms of precision weightedprediction errors: ξ(i ) :i = s, x, v on the sensory input, motion of hidden
states, and the hidden causes. The ensuing equations suggest two neuronalpopulations that exchange messages; with state-units encoding conditionalpredictions and error-units encoding prediction error. Under hierarchicalmodels, error-units receive messages from the state-units in the same leveland the level above; whereas state-units are driven by error-units in thesame level and the level below. These provide bottom-up messages thatdrive conditional expectations μ(i ) :i = x, v toward better predictions toexplain away prediction error. These top-down predictions correspond tog := g(μ(x), μ(v )) that are specified by the generative model. This schemesuggests the only connections that link levels are forward connectionsconveying prediction error to state-units and reciprocal backwardconnections that mediate predictions. Note that the prediction errors thatare passed forward are weighted by their precisions: Π(i ) :i = s, x, v.Technically, this corresponds to generalized predictive coding because it is afunction of generalized variables, which are denoted by a (∼), such thatevery variable is represented in generalized coordinates of motion: forexample: x = (x , x ′, x ′′, . . .). See Friston (2008) for further details.
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 5
571 628
572 629
573 630
574 631
575 632
576 633
577 634
578 635
579 636
580 637
581 638
582 639
583 640
584 641
585 642
586 643
587 644
588 645
589 646
590 647
591 648
592 649
593 650
594 651
595 652
596 653
597 654
598 655
599 656
600 657
601 658
602 659
603 660
604 661
605 662
606 663
607 664
608 665
609 666
610 667
611 668
612 669
613 670
614 671
615 672
616 673
617 674
618 675
619 676
620 677
621 678
622 679
623 680
624 681
625 682
626 683
627 684
Brown and Friston Free-energy and illusions: the Cornsweet effect
in Eq. 2 to corresponds to a generalized form of predictive coding.Predictive coding has become a popular metaphor for understand-ing perceptual inference in the visual system. For example, Raoand Ballard (1999) used predictive coding to provide a compellingexplanation for extraclassical receptive field effects in striate cortex.
Put simply, in these simulations we assume that neural activ-ity corresponds to the brain’s representation of the most likelyvalues of the hidden causes and states (hidden variables) andthat these are continuously updated to minimize free-energy. Theensuing scheme has been discussed in terms of recurrent message-passing among different cell populations in hierarchical sensorycortex: see Figure 5 and Mumford (1992). This scheme rests uponthe use of bottom-up prediction errors to optimize conditionalestimates of hidden variables. These estimates are then used toproduce top-down predictions that are compared with sensoryinput to form a bottom-up prediction error. In this context, thesum of squared prediction error can be regarded as free-energy.The recursive message-passing used in these schemes tries to min-imize prediction error, such that the predictions approximate thetrue conditional or posterior estimates of the underlying hiddencauses. It is this message-passing that we will stimulate in thenext section and associate with neuronal responses, while usingwhat they represent to predict how real subjects would respondbehaviorally, in terms of their perceptual decisions or inference.
To simulate these responses we simply integrate or solve Eq. 2,using the functions g (x, v) and f(x, v) specified by a generativemodel in Eq. 1. These functions map hidden variables to sen-sory input and encode prior beliefs about the dynamics of hiddenstates. In short, by plugging the equations of our generative modelin Figure 3 into the predictive coding scheme of Figure 5, we cansimulate Bayes-optimal inference about the causes of sensations.Crucially, we can then reconstitute the posterior or conditionalbeliefs about these causes and associate these with percepts. Inparticular, we can take any mixture of the hidden variables andassess the posterior belief about that mixture. We will use thisto quantify the Cornsweet and Mach band percepts, in terms ofreflectance differences among different parts of the visual field.Note that the predictive coding scheme in Figure 5 weights theprediction errors by precision matrices. For example, the preci-sion of sensory signals is Π(s) = I ·exp(γ). These precisions arefunctions of log-precisions γ that encode the expected amplitudeof random fluctuations.
SIMULATED RESPONSESThe simulated responses in Figure 6 were obtained by presentingthe Cornsweet stimulus under uniform illumination. Here, thestimulus was presented transiently by modulating the illuminationwith a Gaussian envelope over time (see image inset). The result-ing predictions are shown in Figure 6A as solid lines, while the reddotted lines correspond to the prediction error. These predictionsare based upon the inferred hidden states and causes shown on theright and lower left respectively. The lines correspond to the poste-rior expectations and the gray regions correspond to 90% Bayesianconfidence intervals. In terms of the underlying causes, the bluecurve in Figure 6B is an estimate of the (log) amplitude of uniformillumination. This should have a roughly quadratic form (giventhe Gaussian envelope), peaking at around bin 30, which indeed
FIGURE 6 | Predictions of the model. (C) Shows the estimates of thehidden states (coefficients of the reflectance basis functions) over time.The hidden state controlling the amplitude of the lowest-frequency basisfunction, which corresponds to the Cornsweet percept, contributessubstantially to the overall perception of the stimulus (green line). Theestimates for the hidden causes are shown in (B). The gray areas are 90%confidence intervals. (A) Shows predictions (solid lines) of sensory inputbased on the estimated hidden causes and states and the resultingprediction error (dotted red lines). The insert on the upper left shows thetime-dependent luminance profile used in this simulation. Please see maintext for further details.
it does. The remaining causes that deviate from zero (Figure 6C)are the perturbations to the hidden states explaining or predictingchanges in reflectance. These drive increases or decreases in theconditional expectations of the hidden states shown on the right.The green line is the coefficient of the second-order basis functionsplitting the visual field into an area of brightness on the left anddarkness on the right. It can be seen that at the point of maximumillumination, there is an extremely high degree of confidence thatthis hidden state is bigger than zero. This is the Cornsweet percept.
The corresponding percepts in sensory space are shown inFigure 7 as a function of peristimulus time. The upper panelsshow the implicit reflectance and illuminance profiles encodedby the conditional expectations of hidden variables respectively.After an exponential transform (and multiplication) these pro-duce the sensory predictions shown on the lower left. By taking aweighted mixture of the perceived reflectance in different regionsof the visual field (shown by the white circles) one can estimate theconditional certainty about both the Cornsweet effect (differencesin perceived reflectance on different sides) and the appearanceof Mach bands (differences in perceived reflectance on the sameside). The weights used to evaluate these mixtures are denoted as
Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 6
685 742
686 743
687 744
688 745
689 746
690 747
691 748
692 749
693 750
694 751
695 752
696 753
697 754
698 755
699 756
700 757
701 758
702 759
703 760
704 761
705 762
706 763
707 764
708 765
709 766
710 767
711 768
712 769
713 770
714 771
715 772
716 773
717 774
718 775
719 776
720 777
721 778
722 779
723 780
724 781
725 782
726 783
727 784
728 785
729 786
730 787
731 788
732 789
733 790
734 791
735 792
736 793
737 794
738 795
739 796
740 797
741 798
Brown and Friston Free-energy and illusions: the Cornsweet effect
FIGURE 7 |The model’s “perceptions.”The upper panels show thepredicted illuminance (left) and reflectance profiles (right), reconstructedfrom the coefficients of the basis functions estimated from the modelinversion shown in the previous figure. An inferred reflectance profiledemonstrating the Cornsweet illusion is apparent, but at this level ofcontrast, Mach bands have not yet appeared. Please see main text furtherdetails.
W corn and W mac for the Cornsweet and Mach band effects respec-tively. The conditional expectation of these mixtures or effectsμmac =W mac·μ(x) and their confidence intervals are shown onthe lower right. At this level of visual contrast or precision (alog-precision of six), the Cornsweet effect is clearly evident witha high degree of certainty, while the confidence interval for theMach band effect always contains zero. In other words, at this con-trast (sensory precision) there is a Cornsweet effect but no Machband effect. In the next section, we repeat the simulation aboveand record the conditional expectations (and confidences) aboutillusory effects at the point of maximum illumination for differentlevels of contrast.
CONTRAST OR PRECISION-DEPENDENT ILLUSORYPERCEPTSUsing the generative model and inversion scheme described above,we repeated the simulations over different levels of sensory pre-cision. This can be regarded as a manipulation of contrast inthe following sense: If we assume that the brain uses divisivenormalization (Weber, 1846; Fechner, 1860; Craik, 1938; Geislerand Albrecht, 1992; Carandini and Heeger, 1994), the key changein sensory information, following an increase in contrast, is anincrease in signal to noise; in other words, its precision increases(see the appendix of Feldman and Friston, 2010 for details). Weuse therefore a manipulation of the log-precision of sensory noiseto emulate changes in visual contrast. It should be noted that wedid not actually add sensory noise to the stimuli. The key quantityhere is the level of precision assumed by the agent which, in these
simulations, we changed explicitly. In more realistic simulations,the log-precision would be itself optimized with respect to free-energy (see Feldman and Friston, 2010 for an example of this inthe modeling of attention).
Figure 8 shows the results of perceptual inference under theBayesian scheme described above. The only thing that we changedwas the log-precision of the sensory input, from minus two (low)through to intermediate levels and ending with a very high log-precision of 16. The two graphs show the conditional expectationand 90% confidence intervals for the Cornsweet effect (upperpanel) and Mach bands (lower panel) respectively, at the pointof maximum illumination. It can be seen in both instances thatunder low levels of contrast (sensory precision) both effects arevery small and inferred with a large degree of uncertainty. How-ever, as contrast increases, conditional uncertainty reduces and,at a critical level, produces a confident inference that the effect isgreater than zero (or some small threshold). Crucially, the pointat which this happens for the Cornsweet effect is at a lower level ofcontrast than for the Mach bands. In other words, the Cornsweetillusion occurs first and then the Mach bands appear as contrastcontinues to rise. The explanation for this is straightforward; theMach band illusion rests upon higher spatial frequencies in thegenerative model, which have a higher prior precision (encodingprior beliefs about the statistical – scale-free – structure of naturalvisual scenes). This means that there needs to be precise sensoryevidence to change them from their prior expectation of zero. Inshort, at high levels of contrast or sensory precision, more andmore fine detail in the posterior percept is recruited to providethe optimum explanation for the stimulus. Interestingly, as thecontrast or sensory precision reaches very high levels, the veridicalreflectance and illuminant profiles are inferred and, quantitatively,both the Cornsweet and mach band effects disappear. The threeimages show exemplar percepts, at low, intermediate and highlevels of contrast respectively. The key difference in the spatialbanding that underlies the Cornsweet and Mach band effects isevident in the difference between the intermediate and high levelsof contrast.
The key prediction of these simulations is that we would expectsubjects to categorize their percepts, following a brief exposureto a Cornsweet stimulus, differently at different levels of contrast.At low levels of contrast, we would expect them to categorize thestimulus as uniformly flat. At intermediate levels of contrast, wewould expect them to categorize the stimulus as a Cornsweet per-cept, with isoluminant and uniform differences in the right andleft hand parts of the visual field; while at higher levels of contrastone would expect the Mach bands to dominate and the stimuliwould be categorized as possessing para-central bands. In princi-ple, at very high levels of contrast, the subject should perceive theveridical stimulus. However, whether this level of contrast can beattained empirically is an open question. In the next section, wetest these hypotheses psychophysically. We conclude this section bylooking not at behavioral responses but at the neuronal responsesimplicit in the simulations.
SIMULATING NEURONAL RESPONSESFigure 9 shows the prediction errors at low, intermediate and highlevels of contrast. These are shown at the sensory level (upper row)
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 7
799 856
800 857
801 858
802 859
803 860
804 861
805 862
806 863
807 864
808 865
809 866
810 867
811 868
812 869
813 870
814 871
815 872
816 873
817 874
818 875
819 876
820 877
821 878
822 879
823 880
824 881
825 882
826 883
827 884
828 885
829 886
830 887
831 888
832 889
833 890
834 891
835 892
836 893
837 894
838 895
839 896
840 897
841 898
842 899
843 900
844 901
845 902
846 903
847 904
848 905
849 906
850 907
851 908
852 909
853 910
854 911
855 912
Brown and Friston Free-energy and illusions: the Cornsweet effect
FIGURE 8 |The effect of contrast. In the results presented here theprecision of the sensory data was used as a proxy for stimulus contrast. Asprecision increases, the strength of both the Cornsweet and Mach effectsincreases until, at very high levels of precision (contrast) the true luminanceprofile is perceived and the illusory percepts fade. Crucially, the Mach bands
appear at higher levels of contrast than the Cornsweet illusion. The inserts inthe lower panels show the inferred reflectance is at different levels of contrast(indicated by the blue dots in the lower graph). The prediction errorsassociated with the processing of stimuli at these three levels are shown inthe next figure. Please see main text for further details.
and at the higher levels of the hidden causes and states (lower row).The key thing to note here is that as contrast increases and thespatial detail of the posterior predictions increases, the sensory
prediction error falls. This is at the expense of inducing predic-tion errors at the higher level, which increase in proportion to theprecision of sensory information. These higher prediction errors
Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 8
913 970
914 971
915 972
916 973
917 974
918 975
919 976
920 977
921 978
922 979
923 980
924 981
925 982
926 983
927 984
928 985
929 986
930 987
931 988
932 989
933 990
934 991
935 992
936 993
937 994
938 995
939 996
940 997
941 998
942 999
943 1000
944 1001
945 1002
946 1003
947 1004
948 1005
949 1006
950 1007
951 1008
952 1009
953 1010
954 1011
955 1012
956 1013
957 1014
958 1015
959 1016
960 1017
961 1018
962 1019
963 1020
964 1021
965 1022
966 1023
967 1024
968 1025
969 1026
Brown and Friston Free-energy and illusions: the Cornsweet effect
FIGURE 9 | Contrast and prediction error. As stimulus contrast increases,prediction error is redistributed from sensory input to hidden variables. Athigh levels of precision, sensory information induces prediction errors athigher levels, which in turn explain away prediction error at the sensorylevel. The higher level prediction errors at high precision reflect increasingconfidence that the reflectance is different from the prior expectation ofzero. Please see main text for further details.
are simply the difference between the posterior and prior expecta-tions and reflect an increasing departure from a prior expectationof zero as contrast (the log-precision of sensory noise) increases.Although these results are interesting in themselves, they can alsobe regarded as a simulation of event related potentials. The reasonthat we can associate prediction error with observed electromag-netic brain responses is that it is usually assumed that predictionerrors are encoded by the activity of superficial pyramidal cells (seeFigure 5). It is these cells that are thought to contribute primarilyto local field potentials and non-invasive EEG signals.
In the high-contrast condition, the prediction errors at thelower level are suppressed by the prediction of the presence ofa Cornsweet stimulus. This sort of phenomenon has been demon-strated using fMRI (Alink et al., 2010; den Ouden et al., 2010);predictable stimuli cause less activation in stimulus-specific areasthan unpredictable stimuli. However, the process simulated hereis likely to produce more complicated neurophysiological corre-lates because of the confounding effect of precision; increasedpredictability (through increasing conditional confidence aboutthe stimulus) will also increase estimates of precision. Since webelieve that the prediction errors reported by superficial pyra-midal cells are precision weighted, decreasing prediction errorin lower sensory areas may be masked by the increasing preci-sion of those errors. We will return to this and related issuesin a subsequent paper looking at the neurophysiological corre-lates of contrast-dependent illusory effects. Here, we focus onpsychophysical correlates:
A PSYCHOPHYSICAL TEST OF THEORETICAL PREDICTIONSIn this section, we report a psychophysics study of normal subjectsexposed to the same stimuli used in the simulations above. Wedepart from the normal procedures for assessing illusions (whichusually involve matching intensity differences) by using a forcedchoice paradigm. This is because we wanted to present stimulibriefly, for several reasons: First, brief presentation avoids the con-founding effects of saccadic eye movements. Second, it allows usto prototype the paradigm for future use in electrophysiological(event related potential) studies, which require transient stimulifor trial averaging. Finally, a forced choice paradigm places con-straints on the subject’s choices that map directly to the modelpredictions. In what follows, we describe the paradigm and inter-pret our results quantitatively, in relation to the predictions of thesimulations above.
SUBJECTS AND EXPERIMENTAL PARADIGMWe studied normal young subjects in accord with guidelines estab-lished by the local ethical committee and after obtaining informedconsent. Eight participants (4 female) completed the Mach bandparadigm; 19 (12 female) completed the Cornsweet paradigm.
EXPERIMENT 1 (CORNSWEET PARADIGM)The Cornsweet illusion was assessed using a two-interval forcedchoice procedure. Subjects were presented with a (set contrast)Cornsweet stimulus and real luminance step for 200 ms (a Gauss-ian temporal envelope was not used), separated by an interval of200 ms. One stimulus appeared to the left of fixation and one tothe right; this was randomized across trials, as was the order ofthe stimuli. Subjects were asked to report the side on which thestimulus with the greatest contrast had appeared (Figure 10).
Six blocks were completed, using Cornsweet stimuli with Webercontrasts of 0.0073–0.734. A Quest procedure (Watson and Pelli,1983) was used to select each step stimulus for comparison. Themean of the psychometric function was taken as the point ofsubjective equality between the Cornsweet stimulus and a realluminance. There were 200 presentations per block.
EXPERIMENT 2 (MACH BAND PARADIGM)The same method could not be used to identify the strength of theMach bands percept, as there is no non-illusory stimulus that canbe used for matching. Consequently, we used a two-alternativeforced choice paradigm: A single Cornsweet stimulus was dis-played for 200 ms to the left or right of fixation and participantswere asked to report if the stimulus contained Mach bands or not.Each participant completed six runs of 200 presentations at 10Weber contrast levels from 0.0204 to 0.2038. The probability ofreporting a Mach band was assessed as the relative frequency ofreporting its presence over trials, within subject (Figure 10).
In both experiments, stimuli were displayed on an LCD mon-itor under ambient room lighting. Subjects were seated 60 cmaway from the monitor, such that stimuli subtended an angle of14.21˚ vertically and 6.10˚ horizontally, at 2.96˚–7.06˚ eccentricity.The luminance ramp of the Cornsweet stimulus profile occupied2.42˚. Only the lowest levels of contrast the monitor was able toproduce were employed; thus, luminance values were linearizedpost hoc.
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 9
1027 1084
1028 1085
1029 1086
1030 1087
1031 1088
1032 1089
1033 1090
1034 1091
1035 1092
1036 1093
1037 1094
1038 1095
1039 1096
1040 1097
1041 1098
1042 1099
1043 1100
1044 1101
1045 1102
1046 1103
1047 1104
1048 1105
1049 1106
1050 1107
1051 1108
1052 1109
1053 1110
1054 1111
1055 1112
1056 1113
1057 1114
1058 1115
1059 1116
1060 1117
1061 1118
1062 1119
1063 1120
1064 1121
1065 1122
1066 1123
1067 1124
1068 1125
1069 1126
1070 1127
1071 1128
1072 1129
1073 1130
1074 1131
1075 1132
1076 1133
1077 1134
1078 1135
1079 1136
1080 1137
1081 1138
1082 1139
1083 1140
Brown and Friston Free-energy and illusions: the Cornsweet effect
RESULTS AND DISCUSSIONThe results of the psychophysics experiments are shown inFigure 11, as a function of empirical (Weber) contrast levels. Theseresults are expressed as the mean over all subjects and associated
FIGURE 10 |Time courses of trials. Experiment 1: before the start of eachtrial, participants fixated a central cross. One Cornsweet and one realluminance step stimulus appeared for 200 ms each, with a 200 ms intervalbetween them. The order of the stimuli, their orientation and the side onwhich each appeared were randomized (although they were constrained toappear on opposite sides within each trial). Participants then had 1750 msto report the side on which the stimulus with the greatest contrast hadappeared (using the arrow keys of the keyboard). Experiment 2: each trialstared with fixation. A Cornsweet stimulus then appeared for 200 ms with arandom orientation to the left or right of fixation. Participants had 1750 msto report, with the “Y” and “N” keys, if they perceived Mach bands.
SE. The reported Cornsweet effect (as indexed by the point of sub-jective equality) peaked, on average over subjects, at a contrast ofabout 0.0025. At higher levels of contrast, as in the simulations,the effect fell quantitatively, plateauing at the highest contrastused in Experiment 1. Conversely, the probability of reporting aMach band increased monotonically as a function of the empiricalcontrast, reaching about 75% at a Weber contrast of about 0.15.
Qualitatively, these empirical results compare well with the the-oretical predictions shown in Figure 9: that is, the subjective orinferred Cornsweet effect emerged before the Mach bands, as con-trast increases. We also see the characteristic “inverted U” depen-dency of the Cornsweet effect on contrast levels. The empiricalprofile is somewhat compromised by the small range of contrastsemployed, however, this range was sufficient to disclose an unam-biguous peak. Clearly, it would be nice to relate these empiricalresults quantitatively to the simulations shown in Figure 9. Thispresents an interesting challenge because the psychophysical dataconsist of reported levels of an effect and the probability of aneffect for the Cornsweet and Mach Band illusions respectively.However, because our simulations provide a conditional or pos-terior probability over the effects reported, we can simulate bothsorts of reports and see how well they explain the psychophysicaldata. This quantitative analysis is now considered in more detail.
A FORMAL BEHAVIORAL ANALYSISThe simulations provide conditional expectations (and precisions)of both the Cornsweet and Mach band effects over a number ofsimulated (Weber) contrast levels, as modeled with the precisionof sensory noise. This means that one can compute a psychome-tric function of contrast c that returns the behavioral predictionsof both illusions respectively; namely, the level of the illusion andthe probability of inferring a Mach band. To predict the reported
FIGURE 11 | experimental results. This figure shows the results of theempirical psychophysical study for the Cornsweet illusion (left panel) andthe Mach band illusion (Right panel). Both report the average effect oversubjects and the SE (bars). The Cornsweet illusion is measured in termsof the subjective contrast level (quantified in terms of subjectiveequivalence using psychometric functions). The Mach band illusion is
quantified in terms of probability that the illusion is reported to bepresent. Both results are shown as functions of empirical (Weber)stimulus contrast levels. We have used the same range for both graphsso that the dependency on contrast levels can be compared. The keything to note here is that the Cornsweet illusion peaks before the Machband illusion (vertical line).
Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 10
1141 1198
1142 1199
1143 1200
1144 1201
1145 1202
1146 1203
1147 1204
1148 1205
1149 1206
1150 1207
1151 1208
1152 1209
1153 1210
1154 1211
1155 1212
1156 1213
1157 1214
1158 1215
1159 1216
1160 1217
1161 1218
1162 1219
1163 1220
1164 1221
1165 1222
1166 1223
1167 1224
1168 1225
1169 1226
1170 1227
1171 1228
1172 1229
1173 1230
1174 1231
1175 1232
1176 1233
1177 1234
1178 1235
1179 1236
1180 1237
1181 1238
1182 1239
1183 1240
1184 1241
1185 1242
1186 1243
1187 1244
1188 1245
1189 1246
1190 1247
1191 1248
1192 1249
1193 1250
1194 1251
1195 1252
1196 1253
1197 1254
Brown and Friston Free-energy and illusions: the Cornsweet effect
level of the Cornsweet illusion we can simply use the conditionalexpectation μcorn(c) scaled by some (unknown) coefficient β1. Topredict the probability of reporting the presence of Mach bands,one can integrate the conditional probability distribution over theMach band effect above some (unknown) threshold β2. However,to do this, we need to know the relationship between the simulatedand empirical contrasts:
As noted above, we used the log-precision of sensory noiseγ to model log-contrast in accord with Weber’s law. This meanswe can assume a linear relationship between the empirical log-contrast and simulated log-precision. This induces two furtherunknown coefficients – the slope and intercept (β3 and β4) thatparameterise the relationship between the simulated and empiri-cal contrasts. Finally, we need to relate the conditional probabilityof a suprathreshold Mach band effect to the probability of report-ing its presence. Here, we assumed a simple, monotonic sigmoidrelationship, under the constraint that when the conditional prob-ability was 50:50, the report probability was also 50:50. The preciseform of this mapping is provided in Figure 11 (left panel) and has a
single (unknown) slope coefficient β5. These relationships providea mapping between the results of the simulations and the observedresponses averaged over subjects (under the assumptions of addi-tive prediction errors). This is known as a response model and isdetailed schematically in Figure 11. The predictions are based onthe simulated responses in Figure 9, assuming a smooth psycho-metric function of contrast that was modeled as a linear mixture ofcosine functions: Xk(γ) = cos(πγk) for k = 1, . . ., 6). The coeffi-cients of this discrete cosine set were estimated with ordinary leastsquares, using the responses of the model (μcorn(γ), μmac(γ)) overdifferent precision levels, at the time of maximum luminance.
Given the form of the relationships between the simulatedand empirical contrasts and between the report probability andconditional probabilities for Mach bands, one can use the psy-chophysical data to estimate the unknown coefficients of theserelationships: βi for = 1,. . ., 5. The results of this computation-ally informed response modeling are shown in the right panel ofFigure 12. The upper panels show the same data as in Figure 10placed over the theoretical psychometric functions based on the
FIGURE 12 | Predicted and experimental results. Theoretical predictionsof the empirical results reported in the previous figure: These predictions arebased upon a response model that maps from the conditional expectationsand precisions in the simulations to the behavioural responses of subjects.This mapping rests on some unknown parameters or coefficients βi:i = 1,. . ., 5 that control the relationship between the simulated γ(c) and empiricalcontrast levels c used for stimuli and the relationship between theprobabilities of reporting a Mach band σ(pmac(c)) and the conditionalprobability that it exceeds some threshold pmac(c) at contrast level c. Theserelationships form the basis of a response model, whose equations areprovided in the left panel (for simplicity, we have omitted the expressions forconditional variance of the Mach band contrast). Given empirical responsesfor the mean Cornsweet effect M (c) and probability of reporting a Mach
band P (c), the coefficients βi can be estimated under the assumption ofadditive prediction errors ε. The predicted responses following thisestimation are shown in the graphs on the right hand side. The upper panelsshow the empirical data superimposed upon conditional predictions fromthe model. The gray lines are the predicted psychometric functions, μcorn(c)and σ(pmac(c)). The red dots correspond to the predictions at levels ofcontrast used in the simulations (as shown in Figure 9), while the black dotscorrespond to the empirical responses: M (c) and P (c). The lower left panelshow the relationship between the empirical and simulated contrast levels(as a semi-log plot of the empirical contrast against the log-precision ofsensory noise). The lower right panel shows the relationship between theprobability of reporting a Mach band is present and the underlyingconditional probability that it is above threshold.
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 11
1255 1312
1256 1313
1257 1314
1258 1315
1259 1316
1260 1317
1261 1318
1262 1319
1263 1320
1264 1321
1265 1322
1266 1323
1267 1324
1268 1325
1269 1326
1270 1327
1271 1328
1272 1329
1273 1330
1274 1331
1275 1332
1276 1333
1277 1334
1278 1335
1279 1336
1280 1337
1281 1338
1282 1339
1283 1340
1284 1341
1285 1342
1286 1343
1287 1344
1288 1345
1289 1346
1290 1347
1291 1348
1292 1349
1293 1350
1294 1351
1295 1352
1296 1353
1297 1354
1298 1355
1299 1356
1300 1357
1301 1358
1302 1359
1303 1360
1304 1361
1305 1362
1306 1363
1307 1364
1308 1365
1309 1366
1310 1367
1311 1368
Brown and Friston Free-energy and illusions: the Cornsweet effect
simulations of the previous section. These predictions are basedon the mapping from simulated to empirical contrast levels (lowerleft) and the relationship between the probability of reporting aMach band and the conditional confidence that it is present (lowerright).
By construction, the relationship between the simulated andempirical contrasts is linear when plotted on a log-log scale. Theslope of this plot suggests that the higher contrasts used in the sim-ulations are, practically, not realizable in an empirical setting. Thisis because as the simulated contrast increases the correspondingempirical contrast increases much more quickly. The implicationof this is that the contrasts employed in the psychophysics studycorrespond to the first few levels of the simulated contrasts. Thismeans that it may be difficult to demonstrate the theoretically pre-dicted attenuation of the Cornsweet illusion at very high levels ofcontrast.
The relationship between the report and conditional probabil-ities suggests that subjects have a tendency to “all or nothing”reporting; in the sense that a conditional confidence that theprobability of reporting a Mach band is slightly greater than theconditional confidence it is above threshold. Conversely, subjectsappear to report the absence of Mach bands with a probabil-ity that is slightly greater than the conditional probability it isbelow threshold. The resulting psychometric predictions (in theupper panels) show a remarkable agreement between the predictedand observed probabilities of reporting a Mach band. The corre-spondence between the predicted and empirical results for theCornsweet illusion are less convincing but show that both asymp-tote to a peak level much more quickly than the probability ofreporting a Mach band.
In summary, this analysis suggests that there is a reasonablequantitative agreement between the theoretical predictions andempirical results. Furthermore, in practical terms, it appears thatthe normal range of Weber contrasts that can be usefully employedcorresponds to a relatively low level of sensory precision in the
simulations. This means that it may be difficult to demonstrate the“inverted U” behavior for the Cornsweet illusion seen in Figure 9.This is because it may be difficult to present stimuli at the ultrahigh levels of contrast required. Note that the empirical proba-bility of reporting a Mach band does not decrease as a functionof contrast level. This is to be anticipated from the theoreticalpredictions: increasing the contrast level increases the conditionalprecision about the inferred level of the Mach band effect, whichmeans that the probability that is above threshold can still increaseeven if the conditional expectation decreases (as in Figure 9).
CONCLUSIONThis paper has reviewed the nature of illusions, in the contextof Bayes-optimal perception. We reiterate the notion that illusorypercepts are optimal in that they may represent the most likelyexplanation for ambiguous sensory input. We have illustrated thisusing the Cornsweet illusion. By using simple and plausible priorexpectations about the spatial deployment of illuminance andreflectance, we have shown show that the Cornsweet effect emergesas a natural consequence of Bayes-optimal perception. Further-more, we were able to simulate a contrast-dependent emergenceof the Cornsweet effect and subsequent appearance of Mach bandsthat was verified psychophysically using a forced choice paradigm.
SOFTWARE NOTEThe simulations and graphics presented in this paper can bereproduced with the DEM toolbox distributed with the academicfreeware SPM from http://www.fil.ion.ucl.ac.uk/spm/. The anno-tated files that implement the Cornsweet illusion simulations andthe more general routines used for model inversion are providedas Matlab code.
ACKNOWLEDGMENTSThis work was funded by the Wellcome Trust. We thank MarciaBennett for helping prepare this manuscript.
REFERENCESAlink, A., Schwiedrzik, C. M., Kohler,
A., Singer, W., and Muckli, L. (2010).Stimulus predictability reducesresponses in primary visual cortex.J. Neurosci. 30, 2960–2966.
Anderson, E. J., Dakin, S. C., andRees, G. (2009). Monocular sig-nals in human lateral genicu-late nucleus reflect the Craik-Cornsweet-O’Brien effect. J. Vis. 9,1–18.
Barlow, H. B. (1974). Inductive infer-ence, coding, perception, and lan-guage. Perception 3, 123–134.
Burton, G. J., and Moorhead, I. R.(1987). Color and spatial structurein natural scenes. Appl. Opt. 26,157–170.
Carandini, M., and Heeger, D. J. (1994).Summation and division by neuronsin primate visual cortex. Science 264,1333–1336.
Cornsweet, T. (1970). Visual Perception.New York: Academic Press.
Craik, K. J. (1938). The effect of adap-tation on differential brightness dis-crimination. J. Physiol. (Lond.) 92,406–421.
Craik, K. J. W. (1966). The Natureof Psychology, ed. S. L. Sherwood.(Cambridge: Cambridge UniversityPress).
Dayan, P., Hinton, G. E., and Neal, R.M. (1995). The Helmholtz machine.Neural Comput. 7, 889–904.
den Ouden, H. E., Daunizeau, J., Roiser,J., Friston, K. J., and Stephan, K.E. (2010). Striatal prediction errormodulates cortical coupling. J. Neu-rosci. 30, 3210–3219.
Fechner, G. T. (1860). “Specielles zurMethode richtiger und falscher Fälle,in Anwendung auf die Gewichtsver-suche,” in Elemente der Psychophysik(Leipzig: Breitkopf and Härtel).
Feldman, H., and Friston, K. J. (2010).Attention, uncertainty, and free-energy. Front. Hum. Neurosci. 4:215.doi:10.3389/fnhum.2010.00215
Field, D. J. (1987). Relations between thestatistics of natural images and theresponse properties of cortical cells.J. Opt. Soc. Am. A 12, 2379–2394.
Friston, K. (2008). Hierarchical mod-els in the brain. PLoS Comput.Biol. 4, e1000211. doi:10.1371/jour-nal.pcbi.1000211
Friston, K. (2009). The free-energyprinciple: a rough guide to the brain?Trends Cogn Sci. (Regul. Ed.) 13,293–301.
Friston, K., Kilner, J., and Harrison, L.(2006). A free energy principle forthe brain. J. Physiol. Paris 100, 70–87.
Geisler, W. S., and Albrecht, D. G.(1992). Cortical neurons: isolationof contrast gain control. Vision Res.32, 1409–1410.
Grossberg, S., and Hong, S. (2006).A neural model of surface per-ception: lightness, anchoring,and filling-in. Spat. Vis. 19,263–321.
Knill, D. C., and Pouget, A. (2004).The Bayesian brain: the role ofuncertainty in neural coding andcomputation. Trends Neurosci. 27,712–719.
Lotto, R. B., and Purves, D. (2001).An empirical explanation of theChubb illusion. J. Cogn. Neurosci. 13,547–555.
Lotto, R. B., Williams, S. M., andPurves, D. (1999). Mach bandsas empirically derived associations.Proc. Natl. Acad. Sci. U.S.A. 96,5245–5250.
Mach, E. (1865). Über die Wirkungder raumlichen Verteilung desLichtreizes auf die Netzhaut.Sitzungsber. Heidelb. Akad.
Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 12
1369 1426
1370 1427
1371 1428
1372 1429
1373 1430
1374 1431
1375 1432
1376 1433
1377 1434
1378 1435
1379 1436
1380 1437
1381 1438
1382 1439
1383 1440
1384 1441
1385 1442
1386 1443
1387 1444
1388 1445
1389 1446
1390 1447
1391 1448
1392 1449
1393 1450
1394 1451
1395 1452
1396 1453
1397 1454
1398 1455
1399 1456
1400 1457
1401 1458
1402 1459
1403 1460
1404 1461
1405 1462
1406 1463
1407 1464
1408 1465
1409 1466
1410 1467
1411 1468
1412 1469
1413 1470
1414 1471
1415 1472
1416 1473
1417 1474
1418 1475
1419 1476
1420 1477
1421 1478
1422 1479
1423 1480
1424 1481
1425 1482
Brown and Friston Free-energy and illusions: the Cornsweet effect
Wiss. Math. Naturwiss. Kl. 52,303–322.
Mumford, D. (1992). On the com-putational architecture of the neo-cortex. II. The role of cortico-cortical loops. Biol. Cybern. 66,241–251.
O’Brien,V. (1959). Contrast by contour-enhancement. Am. J. Psychol. 72,299–300.
Purves, D., Shimpi, A., and Lotto, R. B.(1999). An empirical explanation ofthe Cornsweet effect. J. Neurosci. 19,8542–8551.
Rao, R. P., and Ballard, D. H. (1999).Predictive coding in the visualcortex: a functional interpretationof some extra-classical receptive-field effects. Nat. Neurosci. 2,79–87.
Ratliff, F. (1965). Mach Bands: Quanti-tative Studies on Neural Networks in
the Retina. San Francisco: Holden-Day.
Ruderman, D. L. (1997). Origins of scal-ing in natural images. Vision Res. 37,3385–3398.
Ruderman, D. L., and Bialek, W. (1994).Statistics of natural images: Scalingin the woods. Phys. Rev. Lett. 73,814–817.
Shams, L., Ma, W. J., and Beierholm, U.(2005). Sound-induced flash illusionas an optimal percept. Neuroreport16, 1923–1927.
Tolhurst, D. J., Tadmor, Y., and Chao, T.(1992). Amplitude spectra of naturalimages. Ophthalmic Physiol. Opt. 12,229–232.
von Helmholtz, H. (1866). Treatise onPhysiological Optics, Vol. III, 3rd Edn(trans. by J. P. C. Southall 1925 Opt.Soc. Am. Section 26). New York:Dover.
Watson, A. B., and Pelli, D. G.(1983). QUEST: a Bayesianadaptive psychometric method.Percept. Psychophys. 33,113–120.
Weber, E. (1846). “Der Tastsinn and dasGemeingefuhl,” in Handwörterbuchder Physiologie, ed. R. Wagner(Leipzig: Wilhelm Engelmann).
Weiss, Y., Simoncelli, E. P., and Adel-son, E. H. (2002). Motion illusionsas optimal percepts. Nat. Neurosci.5, 598–604.
Yuille, A., Geiger, D., and Bulthoff,H. (1991). Stereo integration, meanfield theory and psychophysics. Net-work 2, 423–442.
Conflict of Interest Statement: Theauthors declare that the research wasconducted in the absence of any com-mercial or financial relationships that
could be construed as a potential con-flict of interest.
Received: 30 November 2011; paperpending published: 21 December 2011;accepted: 07 February 2012; publishedonline: xx February 2012.Citation: Brown H and Friston KJ (2012)Free-energy and illusions: the Corn-sweet effect. Front. Psychology 3:43. doi:10.3389/fpsyg.2012.00043This article was submitted to Frontiers inPerception Science, a specialty of Frontiersin Psychology.Copyright © 2012 Brown and Friston.This is an open-access article distributedunder the terms of the Creative CommonsAttribution Non Commercial License,which permits non-commercial use, dis-tribution, and reproduction in otherforums, provided the original authors andsource are credited.
www.frontiersin.org February 2012 | Volume 3 | Article 43 | 13