![Page 1: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/1.jpg)
Prosody: Speech Rhythms and Melodies
6. Generalising Pitch: Stylisation
Dafydd Gibbon
Summer SchoolContemporary Phonology and Phonetics
Tongji University 9-15 July 2016
![Page 2: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/2.jpg)
Revision: What exactly does this graph show?
Endlich gab der Nordwind den Kampf auf.
![Page 3: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/3.jpg)
Phonetic correlates of phonological categories
Endlich gab der Nordwind den Kampf auf.
amplitude; intensity = f(amplitude2)→ factor in stress, focus, contrast, emphasis
![Page 4: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/4.jpg)
Representations of phonetic time functions
voicingpitch track: fundamental frequency, f0
Endlich gab der Nordwind den Kampf auf.
![Page 5: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/5.jpg)
Phonetic correlates of phonological categories
global downtrend model→ declination model
Endlich gab der Nordwind den Kampf auf.
pitch perturbation model → accent/tone and microprosody model
![Page 6: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/6.jpg)
Phonetic correlates of phonological categories
initial & final pitch → boundary tone
![Page 7: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/7.jpg)
Phonetic correlates of phonological categories
pitch deviations → local pitch accents (or tones)
Endlich gab der Nordwind den Kampf auf.
![Page 8: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/8.jpg)
Phonetic correlates of phonological categories
Endlich gab der Nordwind den Kampf auf.
initial & final pitch → boundary tone
pitch deviations → local pitch accents (or tones)
![Page 9: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/9.jpg)
Phonetic correlates of phonological categories
boundary tone
voicingpitch track
f0
Endlich gab der Nordwind den Kampf auf.
amplitude; intensity = f(amplitude2)→ factor in stress, focus, contrast, emphasis
global downtrend model→ declination model
pitch perturbation model → accent/tone and microprosody model
pitch deviations → local pitch accents (or tones)
![Page 10: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/10.jpg)
Phonetic interpretation: parameters and trajectories
● Phonetic domain parameters and trajectories– melody:
● variation of fundamental frequency properties in time
– volume:● variation of intensity properties in time
– duration:● variation of unit duration properties in time
– Note that duration has two temporal dimensions
● Phonological domains– structural and functional units and patterns
![Page 11: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/11.jpg)
Phonetic interpretation: phases
● speaker, production, articulatory phonetics:● articulation rate - effort
● channel, acoustic phonetics:● fundamental frequency - intensity
● hearer, reception, auditory phonetics:● pitch - loudness
SENDER RECEIVERCHANNEL
NOISE
![Page 12: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/12.jpg)
Forms of prosody: phases and subphases
● each of the phases has subphases:
→ brain motor activity → nerves → vocal tract muscles
→ air pressure → ( electronic channel → ) air pressure
→ ear sensors – nerves – brain sensory activity
SENDER:brain motor activity
nervesvocal tract muscles
(articulatory domain)
RECEIVER:brain sensory activity
nervesear sensors
(auditory domain)
CHANNEL:air pressure
electronic subchannel(acoustic domain)
NOISE
![Page 13: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/13.jpg)
Phonetic interpretation: phases
● phonetic domains: information theoretic phases:– sender → channel → receiver
● sender: articulatory domain→ brain
→ nerves
→ articulatory tract muscles & shapes● channel: acoustic domain
→ air pressure
( → electrical voltages )
→air pressure● receiver: auditory domain
→ ear canal, ear drum, ossicles
→ cochlea
→ brain
![Page 14: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/14.jpg)
Phonetic interpretation: methods
observations – measurements – models
![Page 15: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/15.jpg)
Observations and measurements
● analysis methods:– observational methods
● perceptual– quantitative measurements– interpretative judgments
● instrumental– quantitative measurements + interpretative judgments
– experimental methods● production
– quantitative analysis of elicited corpus– quantitative analysis of authentic corpus
● perception– quantitative analysis of same-different judgments, reaction times, ...
![Page 16: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/16.jpg)
Observations and measurements
● analysis methods:– observational
● perceptual– quantitative measurements– interpretative judgments
● instrumental– quantitative measurements + interpretative judgments
– experimental● production
– quantitative analysis of elicited corpus– quantitative analysis of authentic corpus
● perception– quantitative analysis of same-different judgments, reaction times, ...
![Page 17: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/17.jpg)
From phonetic measurements to phonetic models
● First steps: collect data, extract F0
● Induce a prosodic model
● Evaluate prosodic model:– Method 1, machine learning:
● use new data, predict goodness of fit of new data
– Method 2, perception:● re-synthesise prosodic model● test results with perception experiments
– same-different comparison– naturalness judgments– comprehensibility judgments
![Page 18: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/18.jpg)
First steps: from waveform to F0
![Page 19: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/19.jpg)
From waveform to F0
● Time domain methods– frequency = 1 / period
– peak picking: intervals between peaks
– intervals between zero crossing measurement
– autocorrelation
● Frequency domain methods– overtone differences
– spectral comb
– cepstrum
Check Moebius slides
![Page 20: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/20.jpg)
From phonetic measurements to phonetic models
![Page 21: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/21.jpg)
From phonetic measurements to phonetic models
● Phonetic models:– smoothing models:
● median smoothing● global regression● local regression (IPO)
– segment models● voiced signal segments● quadratic interpolation between reference points
– structured models● Fujisaki model● Liberman and Pierrehumbert model● Hirst model
![Page 22: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/22.jpg)
From phonetic measurements to phonetic models
● Phonetic models:– smoothing models:
● median smoothing● global regression● local regression (IPO)
– segment models● voiced signal segments● quadratic interpolation between reference points
– structured models● Fujisaki model● Liberman and Pierrehumbert model● Hirst model
![Page 23: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/23.jpg)
From phonetic measurements to phonetic models
● Phonetic stylisation models:– smoothing models:
● median; global (Huber) and local (IPO) regression
– segment models● voiced segment smoothing● quadratic spline segment interpolation (Hirst)
● F0 stylisation is the simplification of the F0 trajectory to remove
– irrelevant properties
– noise
![Page 24: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/24.jpg)
First steps to stylisation: smoothing filters
![Page 25: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/25.jpg)
Regression smoothing
![Page 26: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/26.jpg)
F0 smoothing: general procedures
1. Identify voiced and unvoiced intervals, extract F0
2. Smoothing and ‘stylisation’ modelling procedures
Local sequencing procedures:● level F0 sequences, e.g. based on median of a sequence: robotic!● median smoothing:
for each F0 ( ti ) measurement :
F0smooth ( ti ) mean <F0 ( ti ), …, F0 ( tn )>
● quadratic spline sequences– (Hirst)
Global reference plus accent/tone excursion procedures● Regression: log, linear, quadratic, … ,
– (Fujisaki, Pierrehumbert & Lieberman, Tilt model)
![Page 27: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/27.jpg)
F0 smoothing: different approaches
● Smoothing by median filter:– the median of sequences of 3 measurements
● Smoothing by linear regression
y = a0 + a1x + ε
● Smoothing by polynomial regression:
y = a0 + a1 · t + a2 · t2 + a3t3 + … + a0tn + ε
● Smoothing by asymptotic descent, effectively log(x):
F0(tt+1) = m · F0(ti) + ε, for m < 0
a + F0(ti+1 ) = a + m · F0(ti) + ε, m < 0 non-zero asymptote
![Page 28: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/28.jpg)
Smoothing: different approaches
● Smoothing by median filter:– the median of sequences of 3 measurements
● Smoothing by linear regression
y = a0 + a1x + ε
● Smoothing by polynomial regression:
y = a0 + a1 · x + a2 · x2 + a3x3 + … + a0xn + ε
● Smoothing by asymptotic descent:
y ∈ <x1,..., x1 > : xi = m · xi-1 + ε
y ∈ <x1,..., x1 > : a + xi = a + m · xi-1 + ε
![Page 29: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/29.jpg)
Regression smoothing examples
1. Identify voiced intervals
2. Extract F0
3. Interpolate silent intervals
Simplified in the following examples:
3rd quartile (75th percentile)
4. Calculate smoothing (declination / accent model)
Linear, quadratic etc. (polynomial) regression over interpolated F0 sequence
5. Calculate residuals (microprosody model):
Subtract regression values from F0 values
![Page 30: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/30.jpg)
F0 smoothing: general procedures
1. Identify voiced and unvoiced intervals, extract F0
2. Smoothing and ‘stylisation’ modelling procedures
Local sequencing procedures:● level F0 sequences, e.g. based on median of a sequence: robotic!● median smoothing:
for each F0 ( ti ) measurement :
F0smooth ( ti ) mean <F0 ( ti ), …, F0 ( tn )>
● quadratic spline sequences– (Hirst)
Global reference plus accent/tone excursion procedures● Regression: log, linear, quadratic, … ,
– (Fujisaki, Pierrehumbert & Lieberman, Tilt model)
![Page 31: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/31.jpg)
F0 smoothing: global procedures
![Page 32: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/32.jpg)
F0 smoothing: procedure
1. Identify voiced and unvoiced intervals, extract F0
2. Smoothing and ‘stylisation’ modelling:– Local smoothing:
● linear, median; quadratic spline (Hirst)
– Global reference plus discrete deviant values for different accents or tones
● Fujisaki model, Liberman & Pierrehumbert’s invariance model, Taylor’s ‘Tilt’ model
– Global smoothing with regression:● log, linear, quadratic, polynomial of degree n
(used for illustration in the following examples)
![Page 33: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/33.jpg)
Smoothing: different approaches, different goals
● Smoothing by polynomial regression (degree n):y = a0 + a1 · x + a2 · x2 + a3x3 + … + a0xn + ε
● Smoothing by linear regression (degree 1)y = a0 + a1x + ε
●
![Page 34: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/34.jpg)
Smoothing: different approaches, different goals
● Smoothing by polynomial regression (degree n):y = a0 + a1 · x + a2 · x2 + a3x3 + … + a0xn + ε
● Smoothing by linear regression (degree 1)y = a0 + a1x + ε
●
![Page 35: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/35.jpg)
Global linear regression contour
Endlich gab der Nordwind den Kampf auf.
smooth(f0) – f0 = accents/tones & microprosody
![Page 36: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/36.jpg)
Global quadratic regression contour
Endlich gab der Nordwind den Kampf auf.
smooth(f0) – f0 = accents/tones & microprosody
![Page 37: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/37.jpg)
Global regression contour, degree 7
Endlich gab der Nordwind den Kampf auf.
smooth(f0) – f0 = accents/tones & microprosody
![Page 38: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/38.jpg)
Global regression contour, degree 11
Endlich gab der Nordwind den Kampf auf.
smooth(f0) – f0 = accents/tones & microprosody
![Page 39: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/39.jpg)
Global regression contour, degree 15
Endlich gab der Nordwind den Kampf auf.
smooth(f0) – f0 = accents/tones & microprosody
![Page 40: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/40.jpg)
Global regression contours, up to degree 20
Endlich gab der Nordwind den Kampf auf.
![Page 41: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/41.jpg)
F0 smoothing: local procedures
![Page 42: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/42.jpg)
Simple median filter (scope: 3), often used
Endlich gab der Nordwind den Kampf auf.
Each F0 value is normalised to the median F0 value of its immediate neighbours
![Page 43: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/43.jpg)
Simple local median levelling filter – robotic!
Endlich gab der Nordwind den Kampf auf.
Each F0 value in a sequence is normalised to the median F0 value for the sequence
![Page 44: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/44.jpg)
Local voicing regression contours, degree 1
Endlich gab der Nordwind den Kampf auf.
![Page 45: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/45.jpg)
Local voicing regression contours, degree 2
Endlich gab der Nordwind den Kampf auf.
![Page 46: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/46.jpg)
Local voicing regression contours, degree 3
Endlich gab der Nordwind den Kampf auf.
![Page 47: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/47.jpg)
Local voicing regression contours (1...5)
Endlich gab der Nordwind den Kampf auf.
![Page 48: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/48.jpg)
Models of f0 patterning: Hirst
● Intsint
● Momel
● ProZed
![Page 49: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/49.jpg)
Hirst: quadratic spline - ‘piecewise quadratic function’
![Page 50: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/50.jpg)
Hirst: quadratic spline - ‘piecewise quadratic function’
Anchor points
quadratic spline transitionbetween anchor points
F0 measurements (blue)
![Page 51: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/51.jpg)
Hirst: micromelody = F0 / quadratic spline function
Macromelody (red), micromelody (blue): micromelody = F0 / spline model
![Page 52: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/52.jpg)
Smoothing by local spline interpolation (Hirst)
Momel:Quadratic splines:● changing an anchor point only affects neighbouring transitions● anchor points correspond to zeros on the first derivative of the spline● the transition between two anchor points:
● symmetrical● maximum slope at the spline "knot" - half way between two anchor points.
Hirst’s f0 formulas:
Cubic spline problem, so not used in Momel:
Changing one anchor point can affect the whole curve.
![Page 53: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/53.jpg)
Many other methods ...
● Straight lines (IPO)
● Baseline + pulse modulation
● Gårding
● Grønnum (Thorsen)
● Asymptotic descent (Liberman & Pierrehumbert)– Tilt
● Spline sequence interpolation (Hirst)
![Page 54: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/54.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
Subtract the reference linefrom the F0 trajectory
Define the asymptotic declination line
Define the relation between focus and non-focus accent
types
Define the relation between first pitch accent and
reference line
Define final lowering
![Page 55: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/55.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
![Page 56: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/56.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
Subtract the reference linefrom the F0 trajectory
Define the asymptotic declination line
Define the relation between focus and non-focus accent
types
Define the relation between first pitch accent and
reference line
Define final lowering
![Page 57: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/57.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
Slope:Xi+1 - r = s x (Xi - r)
r B
A
![Page 58: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/58.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
BA
Slope:Xi+1 - r = s x (Xi - r)
r
![Page 59: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/59.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
Subtract the reference line
Define the asymptotic declination line
Define the relation between focus and non-
focus accent types
Define the relation between first pitch accent
and reference line
Define final lowering
![Page 60: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/60.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
![Page 61: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/61.jpg)
Models of f0 patterning: Liberman & Pierrehumbert
● Zero asymptote:
● X_i+1 = s x X_i
●
● X_i+1 - r = s x (X_i -r)
●
● F0 transform: converts measured F0 values into a new set of values that are assumed to behave in a simpler way - coser to underlying phonetic control parameters for intonation.
●
● Answer-background relation: taken to be constant ratio in transformed F0 values: k
●
● Downstep relation: taken to be constant ratio in transformed F0 values: s
●
● Lowering of F0 targets in utterance-final position; final lowering constant: l, utterance-final is bottom of entire system: b
●
● Transformed value of F0 target P depends on pitch range; reference level for each phrase: r
●
● Transformed value of P is its distance above r
●
● r constrained to remain is above final F0 value: b + d
![Page 62: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/62.jpg)
Evaluation of stylised contours – 2 methods:
Difference between F0 and stylised contour
Difference between contours in perception test
From:Demenko Grażyna, Wagner Agnieszka (2006). The Stylization of Intonation Contours. Proceedings of Speech Prosody 3, May 2-5, 2006, Dresden, Germany.
![Page 63: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/63.jpg)
Evaluation of stylised contours: Demenko & Wagner
D&W 2006 stylisation model (SP3):
IP → IE+
IEi + SLi+1 + IEi+1
IP: Intonation PhraseIE: Intonation EventSL: Straight Line
IE {R, F, C}∈IE parameters:
- slope- Fp (F0 at start of event)- range of F0 change- shape coefficient of curve:
y = yγ for 0<x<1y = 2-(2-x)yγ for 1<x<2
![Page 64: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/64.jpg)
Evaluation of stylised contours: Demenko & Wagner
D&W 2006 stylisation model (SP3):
IP → IE+
IEi + SLi+1 + IEi+1
IP: Intonation PhraseIE: Intonation EventSL: Straight Line
IE {R, F, C}∈IE parameters:
- slope- Fp (F0 at start of event)- range of F0 change- shape coefficient of curve:
y = yγ for 0<x<1y = 2-(2-x)yγ for 1<x<2
straight lines
F, R, C curves
![Page 65: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/65.jpg)
Evaluation of stylised contours: Demenko & Wagner
Evaluation 1: goodness of fit
Compare F0 with stylised functionwith Normalised Mean Square Error:
NMSE(t )=(F0(t i)−Sty (t i))
2
F0(t )⋅Sty (t )
NMSE=mean( )
![Page 66: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/66.jpg)
Evaluation of stylised contours: Demenko & Wagner
NMSE=mean( )Evaluation 2: perception test
1 (identical: F0 & Sty perceived as same)2 (a bit different: small differences in pitch height (<10Hz) perceived between F0 & Sty (e.g. pitch too high at stylized phrase end), from microprosody, errors in F0 extraction or phone or syllable segmentation.3 (very different: F0 & Sty differ significantly – different melody, from unrecognized accents (i.e. syllable accented but not labelled “A”; cf. also #2).Subjects could listen as often as necessary.
Result:
After revision of stylisation criteria, items with score 3 re-tested:
30% still with score 3.
n=400 Test
Score 1: 256
Score 2: 68
Score 3: 76
![Page 67: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/67.jpg)
From phonetic models to phonological models
![Page 68: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/68.jpg)
Phonology: representation systems
● Reminder:– Tonetic
– Conversation analysis
– Levels● 4 levels + junctures: Pike● 2 levels + break indices: ToBI (Pierrehumbert)
– Relations
http://mi.eng.cam.ac.uk/~pat40/examples.html
![Page 69: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/69.jpg)
Phonology: representation systems
● Reminder:– Tonetic
– Conversation analysis
– Levels● 4 levels + junctures: Pike● 2 levels + break indices: ToBI (Pierrehumbert)
– Relations
http://mi.eng.cam.ac.uk/~pat40/examples.html
I will leave the details of relating phonetics and phonology
to your own research!
![Page 70: 6. Generalising Pitch: Stylisation - uni-bielefeld.de...Prosody: Speech Rhythms and Melodies 6. Generalising Pitch: Stylisation Dafydd Gibbon Summer School Contemporary Phonology and](https://reader030.vdocuments.mx/reader030/viewer/2022040923/5e9e139542a44e3da4763b93/html5/thumbnails/70.jpg)
Rank-Interpretation Architecture of Language