the speech chain (denes pinson, 1993)
DESCRIPTION
What information is embedded within the speech acoustic signal? Phonetic information Affective information Personal information Transmittal information Diagnostic Information Tasko SPPA 6010 Advanced Speech ScienceTRANSCRIPT
Tasko SPPA 6010 Advanced Speech Science
The Speech Chain (Denes & Pinson, 1993)
Tasko SPPA 6010 Advanced Speech Science
What information is embedded within the speech acoustic signal?
Phonetic information Affective information Personal information Transmittal information Diagnostic Information
Tasko SPPA 6010 Advanced Speech Science
Branches of science employed to understand speech communicationPhysics Acoustics Aerodynamics Kinematics Dynamics
Biology Anatomy
Gross anatomy Microscopic anatomy Molecular biology Neuroimaging
Physiology Electrophysiology
Tasko SPPA 6010 Advanced Speech Science
Physical Quantities Basic vs. Derived Scalar vs. Vector Area Volume Displacement Velocity Acceleration Force
Pressure Work Power Intensity Resistance
Ohm’s Law (V=IR)
Tasko SPPA 6010 Advanced Speech Science
Speech anatomy as “tubes” and “valves”
Speech production is achieved through the systematic regulation of air pressures and flows within the lungs and vocal tract.
Tasko SPPA 6010 Advanced Speech Science
Source-Filter Theory of Speech Production
The sounds we hear as speech is the product of a sound source that has undergone filtering by the vocal tract
source and the filter may be considered to be independent of each other
Tasko SPPA 6010 Advanced Speech Science
Source-Filter Theory
Tasko SPPA 6010 Advanced Speech Science
Source-Filter Theory
Tasko SPPA 6010 Advanced Speech Science
Sound: Acoustics review What is sound? Graphic representation of sound Classifying sounds Filters Resonance The decibel
Tasko SPPA 6010 Advanced Speech Science
What is sound? It may be defined as the propagation of a
pressure wave in space and time. propagates through a medium
Tasko SPPA 6010 Advanced Speech Science
What is sound? Mass-spring model
Tasko SPPA 6010 Advanced Speech Science
Wave action of molecular motionTime
1
2
3
4
5
Distance
Tasko SPPA 6010 Advanced Speech Science
Amplitude waveform
Position
Time
Tasko SPPA 6010 Advanced Speech Science
Amplitude waveform
Amplitude
Time
Question: How long will this last?
Tasko SPPA 6010 Advanced Speech Science
Model of air molecule vibrationTime
1
2
3
4
5
Distance
a b c d
Tasko SPPA 6010 Advanced Speech Science
Simple Harmonic Motion: Sine Wave
Features Amplitude Period Frequency
Hz octave
Phase
Pres
sure
Time
Tasko SPPA 6010 Advanced Speech Science
Graphic representation of sound Time domain
Called a waveform Amplitude v. time
Frequency domain Called a spectrum Amplitude spectrum
amplitude vs. frequency Phase spectrum
phase vs. frequency May be measured using a
variety of “window” sizes
Spectrogram frequency v. amplitude v. time
Tasko SPPA 6010 Advanced Speech Science
Same sound, different graphs
Time domain
Frequency domain
From Hillenbrand
Tasko SPPA 6010 Advanced Speech Science
Are all sound waves simply sinusoids?NO! Waves can be summed Simple waves can combine to produce complex waves Fourier: French Mathematician:
Any complex waveform may be formed by summing sinusoids of various frequency, amplitude and phase
Fourier Analysis Provides a unique (only one) solution for a given sound signal Is reflected in the amplitude and phase spectrum of the signal Reveals the building blocks of complex waves, which are sinusoids
Tasko SPPA 6010 Advanced Speech Science
Classification of sounds Number of frequency components
Simple Complex
Relationship of frequency components Periodic Aperiodic
Duration Continuous Transient
Tasko SPPA 6010 Advanced Speech Science
Complex periodic sounds: Graphic appearance
From Hillenbrand
Tasko SPPA 6010 Advanced Speech Science
Complex periodic sounds: Graphic appearance
Tasko SPPA 6010 Advanced Speech Science
Brief Digression
Tasko SPPA 6010 Advanced Speech Science
Amplitude vs. Phase Spectrum
Amplitude spectrum: different
Phase spectrum: same
Tasko SPPA 6010 Advanced Speech Science
Amplitude vs. Phase Spectrum
Amplitude spectrum: same
Phase spectrum: different
Tasko SPPA 6010 Advanced Speech Science
Digression concluded
Tasko SPPA 6010 Advanced Speech Science
Aperiodic sounds: Graphic appearance
From Hillenbrand
Tasko SPPA 6010 Advanced Speech Science
What “class” of sound is speech?
Tasko SPPA 6010 Advanced Speech Science
The “envelope” of a sound wave Amplitude envelope Spectrum envelope
Tasko SPPA 6010 Advanced Speech Science
Amplitude envelope
From Hillenbrand
Tasko SPPA 6010 Advanced Speech Science
Spectrum envelope
From Hillenbrand
Tasko SPPA 6010 Advanced Speech Science
Amplitude Spectrum: Window Size “instantaneous” amplitude spectrum (long term) average amplitude spectrum
Tasko SPPA 6010 Advanced Speech Science
“Instantaneous” Amplitude Spectra
Tasko SPPA 6010 Advanced Speech Science
(Long Term) Average Amplitude Spectrum
Tasko SPPA 6010 Advanced Speech Science
Tasko SPPA 6010 Advanced Speech Science
The Spectrogram
Tasko SPPA 6010 Advanced Speech Science
Rotate90 degrees
F
A F
A
Tasko SPPA 6010 Advanced Speech Science
Rotate it so thatThe amplitude isComing out of thepage
F
AThis is really narrow because it is a slice in time
F
Time
Tasko SPPA 6010 Advanced Speech Science
Dark bands= amplitudePeaks
Time
F
Tasko SPPA 6010 Advanced Speech Science
Two main types of spectrograms Wide-band spectrograms
Akin to spectrum envelopes “lined up” Frequency resolution not so sharp
Narrow-band spectrograms Akin to amplitude spectrums “lined up” Frequency resolution is really sharp
Tasko SPPA 6010 Advanced Speech Science
Highlights harmonic structure
Highlights spectrum envelope
Tasko SPPA 6010 Advanced Speech Science
Filters What is a filter? How are they relevant to speech? Frequency response curve Representing filter operation Types of filters
Tasko SPPA 6010 Advanced Speech Science
Frequency Response Curve (FRC)
Frequencylow high
Gai
n
+
-
Center frequency
lower cutofffrequency
upper cutoff frequency
passband
3 dB
Tasko SPPA 6010 Advanced Speech Science
Operation of a filter on a signal
NOTE: Amplitude spectrum describes a soundFrequency response curve describes a filter
Tasko SPPA 6010 Advanced Speech Science
Source-Filter Theory revisited
Tasko SPPA 6010 Advanced Speech Science
Some frequency selective filtersLow-pass filtersHigh-pass filtersBand-pass filters
Tasko SPPA 6010 Advanced Speech Science
Resonance What is resonance? Free vibration Forced vibration Acoustic resonators Resonance and speech Resonators as frequency selective filters
Tasko SPPA 6010 Advanced Speech Science
Resonance and Speech
Tasko SPPA 6010 Advanced Speech Science
Resonators as frequency selective filters
Tasko SPPA 6010 Advanced Speech Science
Measuring signal amplitude Amplitude vs. loudness Sound intensity vs. sound pressure Decibel scale
Linear vs. logarithmic Absolute vs. relative Reference values Deriving the equations