ghost, phantoms and illusions are the neural basis of tonal music dante r. chialvo northwestern...
TRANSCRIPT
![Page 1: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/1.jpg)
Ghost, phantoms and illusions are
the neural basis of tonal music
Dante R. Chialvo
Northwestern University. Chicago, [email protected] ChialvoLab.Northwestern.Edu
![Page 2: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/2.jpg)
Si el ojo puede ser consideradouna cámara fotográfica,
el oido seguro que No es un microfono
Short Summary:
If the eye can be seen as a camera,
the ear surely is NOT a microphone
Resumen corto:
![Page 3: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/3.jpg)
We look at mathematical models to determine how the auditory periphery (the 1st neuron) could solve complex problems usually attributed to central processors.
First, we solve a problem first investigated by Pythagoras, later by Ohm and von Helmholtz: how the brain determines the pitch of a complex sound. We will make three points:
1) The output of even a single noisy neuron driven by multiple frequencies are spikes spaced at the inverse of the “ghost” frequency we call “pitch”.
2) We find an expression to predict the frequency of the “ghost” resonance for some arbitrary inputs.
3) The ghost resonance suggests the presence of phantoms and other sensory illusions which will be discussed ten years from now.
Then, we look at the neural mechanism of consonance, the basis of tonal music, finding that can be mapped almost trivially to the same pitch problem we just solved.
Last, investigating ways to falsify the theory we found new results applying to another classic unsolved problem, the octave enlargement.
![Page 4: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/4.jpg)
The problem: How do we perceive pitch of a complex tone ?
![Page 5: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/5.jpg)
The big big picture
The general tendency in the field is:More complex acoustic attributes => much higher cortical
process.Notice that this chart starts at the brainstem…
![Page 6: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/6.jpg)
Algo asi como que puede hacer una neurona por si sola?
Tres cosas preliminares:
1) Miremos como luce el input2) Que sabemos acerca de como responden las
neuronas- a señales periodicas sin ruido (deterministic)
3) Que sabemos acerca de como responden las neuronas- a señales periodicas ruidosos (stochastic)
Instead we look at generic dynamical properties of neurons
![Page 7: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/7.jpg)
Time 1 2 3 4 5 6 7 8
The “fundamental” is not so fundamental!
x(t)
x(t)
x(t)
x(t)
x(t) = (cos f1 t + cos f2 t +cos f3t+ … cos fnt) /n
fn= q f0
Frequency
q
Spectrum
![Page 8: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/8.jpg)
The “fundamental” is not so fundamental (Regardless of phase or harmonicity)
x(t)
x(t)
x(t)
x(t)
shifted
shifted
Frequency
Time
f
fRandom phase
Random phase
Equal phase
Equal phase
q
Spectrum
1 2 3 4 5 6 7 8
x(t) = (cos f1 t + cos f2 t +cos f3t+ … cos fnt) /n
fn= q f0 + f
![Page 9: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/9.jpg)
N:M1:11:0
2:1
n:m
n+N:m+M
3:23:1
5:2 5:3 4:34:1…
Patterns of spikes are predicted by Farey’s series*
continue
*Chialvo, Nature 330, 1987
M/N 2:1
3:2
5:3
3:1
![Page 10: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/10.jpg)
The devil staircase and the Farey series
![Page 11: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/11.jpg)
1:11:0
2:1
no3:1
no no no4:1
The average pattern of spikes are period adding (N:1) sequences
etc:1
<1:1><2:1>
<3:1>
Inter-spike interval
Log
# S
pik
es
![Page 12: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/12.jpg)
How we perceive pitch of a complex tone ?
Cual es el proceso neuronal mas simple que produce spike trains conteniendo el codigo del pitch de un tono complejo?
Respuesta corta:
The non dynamical threshold-crossing model
If x(t) > U th the system emits a ‘‘spike’’
![Page 13: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/13.jpg)
“Ghost Stochastic Resonance” = noise intensity at the largest proportion of spikes with intervals ~ 1/ f0
The neuron respond to a frequency not present in the input (“ghost”)
f2 = 3*f0
f1 = 2*f0
![Page 14: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/14.jpg)
![Page 15: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/15.jpg)
Ghost resonance for mistuned two-frequencies signals
x(t)= A (cos f1 t + cos f2 t ) +
f1 = qfo + f
f2 = (q+1)fo + f
Neuron response to two-frequencies
tones for increasing f plotted as a
function of f1
with
k=2 k=3 k=4k=5
k=6fp ~ 1/(k+1/2)
![Page 16: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/16.jpg)
The neuron response to mistuned complex tones can be generalized for even or odd number of tones
For inputs composed of N sinusoidal signals of frequencies:
ffNkffkfkf 000 )1(...,)1(,
the resonance occur at frequencies:
Two freq.Two freq. Three freq.Three freq.
![Page 17: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/17.jpg)
Data, Theory and Numerics overimposed (no fitting)
![Page 18: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/18.jpg)
Cariani P.A. and Delgutte B., J. Neurophysiol. 76, 1698-1716, 1996
Center Frequency f2 (Hz)625
100
200
300
150
250Es
t. P
itch
(Hz)
475 775
n=2
n=3
n=4n=5
n=6
How it compares with In Vivo Experiments (Cariani)
Our theory
![Page 19: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/19.jpg)
Short list of recent Ghost Stochastic Resonance work:
• Molecular motors (ratchets) driven by 2F ( Fabio Marchesoni and colleagues).
• Models of climate changes (Holger Braun and colleagues).
• Lasers (Martin Buldu, Jordi Garcia-Ojalvo and colleagues).
• Electric fish sensory system (Longtin and colleagues)
Ghost resonance is a general phenomenon
![Page 20: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/20.jpg)
Missing Fundamental
Ghost resonance in vision*
*K. Fujii et al, Psychological Research (2000) 64:149-154.
![Page 21: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/21.jpg)
Another twist:
As seen, the intervals between spikes is determined by the “shape” in time of the complex tone,
Thus, what will be perceived for amplitude modulated white noise?
Shhhh…Shhhh…Shhhh…Shhhh…Shhhh…
From Ghosts to Phantoms
![Page 22: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/22.jpg)
The input signal frequency spectrum
Input Signal
Notice that the input signal spectrum is flat
noise
1/fo
![Page 23: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/23.jpg)
The input spectrum is flat
The ouput spikes at preferred intervals ~ 1/f0
White noise is perceived not as noise but as a periodic signal with 1/fo frequency
Obviamente si el oido fuese un microfono, debieramospercibir ruido....
![Page 24: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/24.jpg)
White noise is perceived not as noise but as a periodic signal with 1/fo frequency
![Page 25: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/25.jpg)
A short list of what we have learned:
1) The output of even a single noisy neuron driven by multiple frequencies are spikes spaced at the inverse of the “pitch”.
2) A simple expression predicts the pitch for many arbitrary inputs (which perfectly agrees with both psychophysics and neural data).
3) The mechanism (“ghost stochastic resonance”) suggests the presence of phantoms and other sensory illusions.
4) The perception of ghosts and phantoms clearly show that the ear is not a microphone.
![Page 26: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/26.jpg)
Pythagoras discovered that two similar strings under the same tension sounded together sound pleasant if the length of the strings are in the ratio of two small integers …. (1:1, 2:1, 3:2… n:m )
Application
The problem:What is the neural mechanism of consonance
The first law of nature ruled by arithmetic of integers:
Consonance
![Page 27: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/27.jpg)
Inconsistent!
According with this picture pure tones’ consonance is left undefined
A tendency is to think on terms of frequency
More coincident harmonics imply better consonance
1:1 > 2:1 > 3:2 > 4:3 > 45:32
![Page 28: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/28.jpg)
Unison
Octave
Galileo (and independently Mersenne) explained consonance as the regularity of the intervals “such that the eardrum is not kept at perpetual torment”
La solucion: Tonos puros que suenan consonante tienen la misma altura y la neurona descarga a intervals de 1/altura
![Page 29: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/29.jpg)
Interval is more telling than 1/Frequency
Unison
Octave
La solucion: Tonos complejos que suenan consonante tienen la misma altura y la neurona descarga a intervals de 1/altura
![Page 30: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/30.jpg)
Conclusion 1:Two tones (simple or complex) sound consonance when they have the same pitch
Conclusion 2: Altua y consonance son la misma cosa para una neuroan
Conclusion 3:The model objectively judges both pitch and consonance (algo asi como un pichichometro o consonometro)
Conclusion 4: Dynamical properties of the auditory periphery can do highly non-trivial processing including the extraction of the pitch of arbitrarily complex tones and the judgment of consonance.
What else?
![Page 31: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/31.jpg)
The problem: to be consonant higher octaves need a few more %
Psycho-acoustic Data Spike Trains Data
So called “enlargement of the octave”
![Page 32: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/32.jpg)
The solution:
Enlargement of the octave can be explained again on the basis of dynamical properties of neurons, (i.e., incomplete recovery from the previous spike) but I am out of time…
Time to summarize:
![Page 33: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/33.jpg)
BlahBlahlogy
By looking at the dynamics of toy neuron models we work out a long unsolved problem:
The brain (can) determine(s) the pitch of a complex sound by the combination of a linear superposition plus a nonlinear noisy threshold crossing process.
We find the expression predicting the frequency of the “ghost” resonance for some arbitrary inputs.
Phantoms and other illusions can be created in similar ways and surely are hiding new surprises.
Consonance can be judged even by a single neuron in exactly the same way than pitch.
The ear is not a microphone
Papers: www.chialvo.net
![Page 34: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/34.jpg)
Papers: www.chialvo.net
![Page 35: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/35.jpg)
How we perceive pitch of a complex tone ?
Labeling the time of each spike as tspike, average the forcing for t=tspike-t,tspike+t.
What do you see for the case of SR?Answer: The input periodic forcing
What do you see for the case of Ghost SR?Answer: The ghost
What do you see for the case of AM white noise?Answer: Nothing.
![Page 36: Ghost, phantoms and illusions are the neural basis of tonal music Dante R. Chialvo Northwestern University. Chicago, IL. d-chialvo@northwestern.edu ChialvoLab.Northwestern.Edu](https://reader036.vdocuments.mx/reader036/viewer/2022062318/5514d21e550346935c8b4ed6/html5/thumbnails/36.jpg)
Replicated experimentally in semiconductor lasers with optical feedback
UPC (Barcelona): Javier Martin-Buldu , Jordi Garcia Ojalvo, Carme Torrent
and UIB (Mallorca): Claudio Mirasso
Ghost Resonance in a Semiconductor Laser with Optical Feedback