bayesian time series
TRANSCRIPT
![Page 1: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/1.jpg)
BAYESIAN TIME SERIES
A (hugely selective) introductory overview
- contacting current research frontiers -
Mike West
Institute of Statistics & Decision Sciences
Duke University
June 5th 2002, Valencia VII - Tenerife
![Page 2: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/2.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Topics
Dynamic linear models (state space models)
• Sequential context, Bayesian framework
• Standard classes of models, model decompositions
Models and methods in physical science applications
• Time series decompositions, latent structure
• Neurophysiology - climatology - speech processing
Multivariate time series:
• Financial applications - Latent structure, volatility models
Simulation-Based Computation
• MCMC - Sequential simulation methodology
![Page 3: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/3.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Standard Dynamic Models
Dynamic Linear Models Linear State Space Models
yt = xt + νt xt = F′
tθt θt = Gtθt−1 + ωt
• signal xt, state vector θt = (θt1, . . . , θtd)′
• regression vector Ft and state matrix Gt
• zero mean measurement errors νt and state innovations ωt
– often zero-mean and normally distributed
![Page 4: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/4.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Examples
“Slowly varying” level observed with noise:
yt = xt + νt xt = xt−1 + ωt
Dynamic linear regression:
yt = xt + νt xt = F′
tθt θt = θt−1 + ωt
Error models for νt, ωt
• normal distributions
• mixtures of normals: outliers and abrupt “structural” changes
![Page 5: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/5.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Simple regression example
yt = xt + νt xt = at + btXt
Ft = (1, Xt)′ and θt = (at, bt)
′ “wanders” through time
X t
6543210
2.2
2
1.8
1.6
1.4
1.2
1
0.8Region 1 Region 2
![Page 6: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/6.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Sales Data Example
4550556065707580
1/75 1/77 1/79 1/81 1/83 1/85
SALES
100110120130140150160170180190
MARKET
![Page 7: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/7.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Sales Data Example
Market
0 25 50 75 100 125 150 175 200
Sales
0
10
20
30
40
50
60
70
80
![Page 8: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/8.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Sales Data Example
0.42
0.43
0.44
0.45
0.46
0.47
QTRYEAR
175
177
179
181
183
185
RATIO
![Page 9: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/9.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Simple Regression Example
Relative to “static” model, dynamic regression delivers:
• improved estimation via adaptation for “local” regression parameters
• and increased (honest) uncertainty about regression parameters
• adaptability to (small) changes → improved point forecasts
• partitions variation: parameter vs observation error
→ increased precision of stated forecasts
i.e., improved prediction: point forecasts AND precision
![Page 10: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/10.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
General Dynamic Linear Model
yt = xt + νt xt = F′
tθt θt = Gtθt−1 + ωt
yt−1 yt yt+1x
x
x
xt−1 xt xt+1x
x
x
−→ θt−1 −→ θt −→ θt+1 −→
• Sequential model definition : Markov evolution structure
• CI structure : θt sufficient for “future” at time t
![Page 11: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/11.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Bayesian Forecasting
Key concepts:
• Bayesian: modelling & learning is probabilistic
• Time-varying parameter models: often non-stationary
• Sequential view, sequential model definitions
– encourages interaction, intervention
Statistical framework:
• Forecasting: “What might happen?” and “What if?”
• Data processing and statistical learning from observations
• Updating of models and probabilistic summaries of belief
• Time series analysis ... Retrospection: “What happened?”
![Page 12: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/12.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Bayesian Machinery
• Inferences based on information Dt = {(y1, . . . , yt), It}
Find and summarise
– p(θt|Dt) and p(θ1, . . . , θt|Dt)
– and update as t → t + 1 → · · ·
• Forecasts:
– p(yt+1, . . . , yt+k|Dt)
– and update as t → t + 1 → · · ·
• Implementation & computations:
– Linear/normal models: neat theory, Kalman filtering
– Extend to infer variance components, non-normal errors ..
∗ need approximations, simulation methods, MCMC
![Page 13: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/13.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Commercial Applications
• Short term forecasting of consumer sales and demand
• Monitoring: stocks and inventories of consumer products
• Many items or sectors: Aggregation and multi-level models
Standard models for commercial applications:
Data = Trend + Seasonal + Regression + Error
↑ ↑ ↑ ↑ ↑
yt = x1t + x2t + x3t + νt
![Page 14: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/14.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Models in Commercial Applications
Component signals xjt follow individual dynamic models
• Trends: e.g., “locally linear” trend
x1t = x1,t−1 + βt + ∂x1t, βt = βt−1 + ∂βt
• Seasonals:
x2t =∑
j
(aj,t cos(2 ∗ πj/p) + bj,t sin(2 ∗ πj/p))
where (aj,t, bj,t) wander through time
Key concept: Model (De)Composition
• Modelling, prior specification, interventions: component-wise
• Posterior inference: detrending, deseasonalisation, etc
![Page 15: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/15.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Special Classes of DLMs
Time Series DLMs – constant F,G
yt = xt + νt xt = F′θt θt = Gθt−1 + ωt
• Includes all “standard” point-forecasting methods
(exponential smoothing, variants, ... )
• Polynomial trend and seasonal components in commercial models
• Includes all practically useful ARIMA models
Multiple representations:
φt = Eθt ↔ G → EGE−1
General class: Time-varying Ft,Gt
Includes non-stationary models, time-varying ARIMA models, etc., ...
![Page 16: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/16.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Simulation-Based Computation
yt = xt + νt xt = F′
tθt θt = Gtθt−1 + ωt
• Normal error models p(νt), p(ωt)
• Fixed time window t = 1, . . . , n
• State vector set: Θ = {θ1, . . . , θn}
• Require: full posterior sample Θ∗ from p(Θ|Dn)
Available via “Forward-filtering: Backward-sampling” algorithm
Carter and Kohn (1994) Biometrika
Fruhwirth-Schnatter (1994) J Time Series Anal
West and Harrison 1997
![Page 17: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/17.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
FFBS Algorithm
Forward-filtering:
• Standard normal/linear analysis: Kalman filter
• delivers normal p(θt|Dt) at each t = 1, . . . , n
Backward-sampling:
• at t = n : sample θ∗n from p(θn|Dn)
• for t = n − 1, n − 2, . . . , 1 : sample θ∗t from normal distribution
p(θt|Dt, θ∗t+1)
Builds up Θ∗ as θ∗n, θ∗n−1, . . .
Exploits Markovian/CI model structure
![Page 18: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/18.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
MCMC in DLMs
DLM parameters: e.g., constant model
yt = xt + νt xt = F′θt θt = Gθt−1 + ωt
Parameters:
• Variances (variance matrices) of νt, ωt
• Elements of F,G
• Indicators in normal mixture models for errors
Add parameters to analysis: MCMC utilising FFBS
![Page 19: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/19.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
MCMC in DLMs
Parameters Φ (may depend on sample size)
Gibbs sampling: p(Θ, Φ|Dn) iteratively resampled via
• Apply FFBS algorithm to draw Θ∗ from p(Θ|Φ∗, Dn)
• Draw new value of Φ∗ from p(Φ|Θ∗, Dn)
• Iterate
“Standard” Gibbs sampling: MCMC
May need “creativity” in sampling Φ∗: Metropolis-Hastings, etc
Often “easy”: as in Autoregressive DLM
![Page 20: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/20.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
MCMC: Autoregressive Model Example
Data: yt = xt + νt
State AR(d) xt =∑d
j=1 φjxt−j + εt
DLM for for xt : xt = (1, 0, . . . , 0)xt, xt = Gxt−1 + ωt
G =
φ1 φ2 φ3 · · · φd−1 φd
1 0 0 · · · 0 0
0 1 0 · · · 0 0...
. . . 0...
0 0 · · · · · · 1 0
, ωt =
εt
0
0...
0
• Parameters: {φ1, . . . , φd; V (νt), V (εt)}
• Conditional posteriors standard: linear regression parameters
![Page 21: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/21.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Models in Scientific Applications
Historical interest in biomedical monitoring (change-points), engineering
applications (control, tracking), environmental monitoring, ...
Goals:
• Exploratory “discovery” of interpretable latent processes
• Nonstationary time series: “hidden” quasi-periodicities
• Changes over time at different time scales
• Time:frequency structure (in time domain)
State-space models:
• Stationary and/or nonstationary, time-varying parameters
• General decomposition theory for state space-space models
• DLM autoregressions and time-varying autoregressions
![Page 22: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/22.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Autoregressive DLM
Latent AR(d) process: xt =∑d
j=1 φjxt−j + εt
Time series: yt = x0t + xt + νt with trend, etc in x0t
DLM for for xt : xt = (1, 0, . . . , 0)xt, xt = Gxt−1 + ωt
G =
φ1 φ2 φ3 · · · φd−1 φd
1 0 0 · · · 0 0
0 1 0 · · · 0 0...
. . . 0...
0 0 · · · · · · 1 0
, ωt =
εt
0
0...
0
– xt latent, unobserved
– G = G(φ) with φ = (φ1, . . . , φd)′ to be estimated
![Page 23: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/23.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Time Series Decomposition
Eigenstructure: G = E−1AE
Reparametrise to “diagonal” model: xt → Ext
Transforms to
xt =dz∑
j=1
zt,j +da∑
j=1
at,j
• z terms: one for each pair of complex conjugate eigenvalues
• a terms: one for each real eigenvalue
• underlying latent processes zt,j and at,j follow “simple” models
– at,j is AR(1) process - short-term correlations
– zt,j is quasi-periodic ARMA(2,1) - noisy sine wave with randomly
time-varying amplitude & phase, fixed frequency
![Page 24: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/24.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Time-varying Autoregression
TV-AR(d) model: xt =∑d
j=1 φt,jxt−j + εt
• AR parameter: φt = (φt,1, . . . , φt,d)′ “wanders” through time:
φt = φt−1 + ∂φt
• stochastic “shocks” ∂φt
• Innovations: εt ∼ N(0, σ2t ) – time-varying variance σ2
t
Flexible representations:
• non-stationary process, time-varying spectral properties
• latent component structure
• other evolutions for φt (e.g., Godsill et al on speech processing)
![Page 25: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/25.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
DLM Form of TV-AR(d):
xt = (1, 0, . . . , 0)′xt
xt = G(φt)xt−1 + (εt, 0, . . . , 0)′
φt = φt−1 + ∂φt
with
G(φt) =
φt,1 φt,2 · · · φt,d−1 φt,d
1 0 · · · 0 0
0 1 · · · 0 0...
. . . 0...
0 0 · · · 1 0
Analysis: Posterior distributions for {φt, σt : ∀t}
Component models: yt = x0t + x1t + νt
→ infer latent TV-AR processes too: {x0t, xt : ∀t}
![Page 26: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/26.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
TVAR Decomposition
xt =dz∑
j=1
zt,j +da∑
j=1
at,j
• z terms: one for each pair of complex conjugate eigenvalues
• a terms: one for each real eigenvalue
• underlying latent processes:
– at,j is TV-AR(1) process - short-term correlations
+ time-varying correlation
– zt,j is TV-ARMA(2,1) - noisy sine wave with
randomly time-varying amplitude & phase
+ time-varying frequency (2π/wavelength)
Number of complex/real eigenvalues can vary over time too!
![Page 27: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/27.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Time:Frequency Analysis
• Fit flexible (high order?) AR or TV-AR models
• Estimate latent components and their frequencies, amplitudes over
time
• Time domain representation of spectral structure
• Often, some zt,j physically meaningful, some (high frequency)
represent noise, model approximation
• at,j – noise, model approximation and and (possibly) low frequency
“trend”
![Page 28: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/28.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Paleoclimatology Example
• deep ocean cores: relative abundance of δ18O
• δ18O ↓ as global temperatures ↑ (smaller ice mass)
• reverse sign: higher recent global temperatures
• “well known” periodicities: earth orbital dynamics → impact
on solar insolation – Milankovitch; Shackleton et al since 1976
eccentricity: 95-120 kyear
obliquity: 40-42 kyear
precession: 19-25 kyear (1 or 2?)
![Page 29: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/29.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Oxygen Isotope Data
• Form of time variation in individual cycles ?
• Timing/nature of onset of “ice-age” cycle ↔ eccentricity component
∼ 1000 kyears ago ?
• Time scale: errors, interpolation, ... measurement, sampling error,
etc
Models: High order TV-AR, p = 20, plus smooth trend (outliers?)
Variance components estimated: changing AR parameters
Decomposition: Posterior mean of xt, φt,j at each t
4 dominant quasi-periodic components: order by estimated amplitude
(innovation variance)
Others: residual structure &/or contaminations
![Page 30: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/30.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
reversed time in k years
oxyg
en le
vel
0 500 1000 1500 2000 2500
3.0
3.5
4.0
4.5
5.0
oxygen isotope series
reversed time in k years
perio
d
0 500 1000 1500 2000 2500
0
50
100
150
trajectories of time-varying periods of components
![Page 31: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/31.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
reversed time in k years
0 500 1000 1500 2000 2500
trend
comp 1
comp 2
comp 3
comp 4
![Page 32: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/32.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Oxygen
• Wavelengths vary only modestly
• Estimated periods/wavelengths consistent with geological
determinations
.... 108–120, peak 110
.... 40.8–41.6, peak 41.5
.... 22.2–23, peak 22.8
• “Switch” due to order of estimated amplitude
– Geological interpretation? Structural climate change ∼ 1.1m yrs?
![Page 33: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/33.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Example: EEG Study
• Clinical uses of electroconvulsive therapy
• Measure seizure treatment outcomes via long, multiple EEG
(electroencephalogram) series – electrical potential fluctuations on
scalp
• Many multiple series: one seizure – 19 channels, 256/sec
Models to:
• Characterise seizure “waveforms” ... time varying amplitudes at
ranges of frequencies (alpha waves, etc)
• Superimposed on “normal” waveform, noise, ...
• Identify/extract latent components: infer seizure effects
• Spatial connectivities: related multiple series
![Page 34: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/34.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Why TVAR Models?
time
0 50 100 150 200
-300
-200
-100
010
020
0
time
0 50 100 150 200
-300
-200
-100
010
020
0
time
0 500 1000 1500 2000
-300
-200
-100
010
020
0
![Page 35: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/35.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
EEG Decomposition Examples
time
0 100 200 300 400 500
data
delta
theta
alpha:theta
alpha
beta
beta
Central section of Ictal19-Cz
![Page 36: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/36.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
time
0 500 1000 1500 2000 2500 3000
beta
alpha
theta2
theta1
delta2
delta1
data
S26.Low Patient/Treatment
![Page 37: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/37.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
EEG Treatment Comparison
Two EEG series on one individual: S26.Low cf S26.Mod
• Repeat seizures with varying ECT treatment
• TVAR(20) with time-varying σ2t
• Evident high frequency structure and “spiky” traces
→ higher order models
• Several frequency bands influenced by seizure – several components
• Identification issues
![Page 38: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/38.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
EEG Comparisons
time
0 1000 2000 3000 4000
0
2
4
6
8
10
theta range
delta range
S26.LowS26.Mod
--
--
--
•
•
•
--
-
-
-
-
•
•
•
--
--
--
•
•
•
--
--
--
•
•
•
--
--
--
•
•
•
--
--
--
•
•
•
-
--
-
-
-
•
•
•
--
-
-
-
-
•
•
•
--
-
-
-
-
•
•
•
--
-
-
-
-
•
•
•
--
-
-
-
-
•
•
•
S26.Low vs. S26.Med: Low frequency trajectories
![Page 39: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/39.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Sequential Simulation Analysis
Time t :
• States Θt = {θt−k, . . . , θt} of interest
• Summarised via set of posterior samples: p(Θt|Dt)
Time t + 1 :
• Observe yt ∼ p(yt|θt)
• Require updated summary, posterior samples: p(Θt+1|Dt)
Issues:
• Expanding/Changing state space and dimension
• Simulation-based summaries: discrete approximations
• Inference on parameters as well as state vectors
• New data may “conflict” with prior/predictions
![Page 40: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/40.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Particle Filtering
Key Goal: sequentially update posteriors
· · · → p(θt|Dt) → p(θt+1|Dt+1) → · · ·
Numerical Approximations (points/weights):
{θ(j)t , ω
(j)t : j = 1, . . . , Nt}
Theoretical update:
p(θt+1|Dt+1) ∝ p(yt+1|θt+1)p(θt+1|Dt)
MC approximation to “prior”:
p(θt+1|Dt) ≈Nt∑
k=1
ω(k)t p(θt+1|θ
(k)t )
• Mixture prior: sample and accept/reject ideas natural
![Page 41: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/41.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Example: Auxilliary Particle Filter
APF state update from t → t + 1:
• for each k,
– “estimates” µ(k)t+1 = E(θt+1|θ
(k)t )
– and weights g(k)t+1 ∝ ω
(k)t p(yt+1|µ
(k)t+1)
• sample (aux) indicators j with probs g(j)t+1
• time t + 1 samples: θ(j)t+1 ∼ p(θt+1|θ
(j)t )
• and weights:
ω(j)t+1 =
p(yt+1|θ(j)t+1)
p(yt+1|µ(j)t+1)
![Page 42: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/42.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Multivariate Models in Finance
(Quintana et al invited talk, Valencia VII)
• Futures markets, exchange rates, portfolio selection
• Multiple time series: time-varying covariance patterns
• Econometric/dynamic regressions/hierarchical models
• Latent factors in hierarchical, dynamic models
• Common time-varying structure in multiple series
• Bayesian multivariate stochastic volatility
yt = (y1t, . . . , ypt)′
e.g., yit is p−vector of returns on investment i
(exchange rate futures, etc.)
![Page 43: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/43.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Dynamic Factor Models
Exchange rate modelling for dynamic asset allocation
• Monthly (daily) currency exchange rates
• Dynamic regression/econometric predictors
• Residual structure and residual stochastic volatility
– Time-varying variances and covariances
– Dynamic factor models
• Dynamic asset allocation & risk management: portfolio studies
• Bayesian analysis: model fitting, sequential analysis, forecasting,
decision analysis
![Page 44: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/44.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Excess Returns on Exchange Rates (monthly)R
etur
ns %
-10
010
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
DEM
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
JPY
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
GBP
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
AUD
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CAD
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CHF
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
FRF
Ret
urns
%-1
00
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
NLG
![Page 45: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/45.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Short Interest Rates (monthly)IS
-0.1
00.
00.
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
DEM
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
JPY
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
GBP
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
AUD
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CAD
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CHF
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
FRF
IS-0
.10
0.0
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
NLG
Short Interest Rate
![Page 46: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/46.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Stochastic Volatility Factor Models
yt = dynamic regressiont + residualt – residualt ∼ N(0,Σt)
residualt = Xtft + et
ft ∼ N(ft|0,Ft)
et ∼ N(et|0,Et)
ft ... k−vector of latent factors et ... q−vector of “idiosyncracies”
Ft = diag(exp(λ1,t), . . . , exp(λk,t)) Et = diag(exp(λk+1,t), . . . , exp(λk+q,t))
Σt = XtFtX′
t + Et
Factor and idiosyncratic latent log volatilities: λt = (λ1,t, . . . , λk+q,t)′
![Page 47: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/47.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Dynamic Factor Model: Factor Loadings Structure
• Constant factor loadings matrix Xt = X
– or “slowly varying” over time –
• Identification constraints on X :
X =
1 0 0 · · · 0
x2,1 1 0 · · · 0...
...... · · · 0
xk,1 xk,2 xk,3 · · · 1
xk+1,1 xk+1,2 xk+1,3 · · · xk+1,k
......
... · · ·...
xq,1 xq,2 xq,3 · · · xq,k
• Series order defines interpretation of factors
![Page 48: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/48.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Stochastic Volatility Models: Factors & Idiosyncracies
Multivariate SV models
• vector AR(1) model for latent log volatilities λt
• volatility persistence: AR(1) coefficients Φ (diagonal)
• marginal volatility levels: time-varying µt
• dependent innovations: shocks to volatilities related across series
global/sector “effects” represented through correlations
λt = µt + Φ(λt−1 − µt−1) + ωt
ωt ∼ N(0,U)
µt ∼ random walk
![Page 49: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/49.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Dynamic Regressions and Shrinkage Models
• several predictors (e.g., interest rates)
• regression coefficients time-varying
• shrinkage models: coefficients related across series
e.g., sensitivity to short-term interest rates “similar” across
currencies
Series j, time t :
βjt = φjtβj,t−1 + (1 − φjt)γt + innovationjt
• global or sector “average” γt
• time-varying degrees of “shrinkage” φjt
• multiple series, several predictors: state-space model formulations
![Page 50: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/50.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Model Fitting & Analysis: MCMC
Inference based on a fixed sample over t = 1, . . . , T
• Bayesian analysis via posterior simulations
• Monte Carlo samples from joint posterior for
– model parameters, dynamic regression parameters, AND
– latent processes: factors & volatilities
{ ft, λt : t = 1, . . . , T }
• examples of highly structured, hierarchical models with many latent
variables and parameters
• custom MCMC built of standard “modules”
• simple MC evaluation of forecast/predictive distributions
![Page 51: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/51.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Model Fitting & Analysis: Sequential Analyses
Sequential particle filtering over t = T + 1, T + 2, , . . .
• “particulate” approximation to posterior distributions
• prior: cloud of particles, associated importance weights
• posterior: sampling/importance resampling to revise posterior
samples
· · · → p(·|Dt−1) → p(·|Dt) → · · ·
• sample “regeneration” by local smoothing of particulate prior
• time-varying latent variables and model parameters
![Page 52: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/52.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Volatilities (SD) of Currency Returns (monthly)st
dev
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
DEM
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
JPY
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
GBP
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
AUD
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CAD
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CHF
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
FRF
stde
v0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
NLG
![Page 53: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/53.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
2 Latent Factors and their Volatilities (SD) (monthly)Fa
ctor
1-0
.10
0.0
0.05
0.10
7602 8102 8602 9102 9602
DEM
Fact
or 1
0.0
0.04
0.08
0.12
7602 8102 8602 9102 9602
Fact
or 2
-0.1
00.
00.
050.
10
7602 8102 8602 9102 9602
JPY
Fact
or 2
0.0
0.04
0.08
0.12
7602 8102 8602 9102 9602
![Page 54: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/54.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
3 Latent Factors and their Volatilities (SD) (monthly)Fa
ctor
1-0
.10
-0.0
50.
00.
050.
10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
DEM
Fact
or 1
0.0
0.02
0.04
0.06
0.08
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
Fact
or 2
-0.1
0-0
.05
0.0
0.05
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
JPY
Fact
or 2
0.0
0.02
0.04
0.06
0.08
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
Fact
or 3
-0.1
0-0
.05
0.0
0.05
0.10
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
GBP
Fact
or 3
0.0
0.02
0.04
0.06
0.08
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
![Page 55: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/55.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Factor Loading Matrix X (monthly)
Factor 1 Factor 2
DEM 1.00 (0.00) 0.00 (0.00)
JPY 0.55 (0.06) 1.00 (0.00)
GBP 0.68 (0.05) 0.48 (0.14)
AUD 0.23 (0.03) 0.35 (0.07)
CAD 0.04 (0.02) 0.03 (0.08)
CHF 1.04 (0.03) 0.23 (0.07)
FRF 0.96 (0.01) -0.01 (0.02)
ESP 0.99 (0.01) -0.01 (0.01)
![Page 56: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/56.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Factor Loading Matrix X (monthly)
Factor 1 Factor 2
DEM 1 ·
FRF 1 ·
ESP 1 ·
CHF 1 0.2
GBP 0.7 0.5
JPY 0.5 1
AUD 0.2 0.3
CAD · ·
![Page 57: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/57.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Idiosyncratic Volatilities (SD) (monthly)Id
iosy
ncra
tic0.
00.
050.
100.
15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
DEM
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
JPY
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
GBP
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
AUD
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CAD
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
CHF
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
FRF
Idio
sync
ratic
0.0
0.05
0.10
0.15
7602 7710 7906 8102 8210 8406 8602 8710 8906 9102 9210 9406 9602 9710
NLG
![Page 58: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/58.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Financial Time Series
• Models/forecasts feed into portfolio decision management
– Quintana et al invited talk, Valencia VII
• Live/Operational assessments of dynamic factor models
• Improved sequential particle filtering: parameters
• Time-varying loadings matrices Xt, parameters
• Uncertainty about number of factors
![Page 59: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/59.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Bayesian Time Series, Currently
Applied aspects:
• Financial modelling and forecasting
• Natural/engineering sciences: signal processing
• Spatial time series: epidemiology, environment, ecology
Models and methods:
• Highly structured multiple time series
• Spatial time series
• Computational methods: Sequential simulation methods
![Page 60: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/60.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Links & Materials
• Books and papers: www.isds.duke.edu/ mw
– Copious references to broad literatures
• 1997 Tutorial – extensive and historical – at website
• Teaching materials, notes, from past time series courses
• Software: www.isds.duke.edu/ mw
– TVAR (matlab, fortran), AR component models, BATS, links
• Encylopedia of Statistical Sciences (1998): Bayesian Forecasting (eds:
S. Kotz, C.B. Read, and D.L. Banks), Wiley.
• 1997 2nd Edition: West & Harrison/Springer book
Bayesian Forecasting & Dynamic Models
• Aguilar, Prado, Huerta & West (1999), Valencia 6 invited paper
![Page 61: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/61.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
Key Recent (since 1995) & Current Authors
– many/most are here! –
Omar Aguilar (Lehman Bros) Chris Carter (Hong Kong)
Sid Chib (Washington Univ/St Louis) Sylvia Fruhwirth-Schnatter (Vienna)
Simon Godsill & Co (Cambridge) Gabriel Huerta (Univ New Mexico)
Genshiro Kitagawa (ISM Tokyo) Robert Kohn (Sydney)
Jane Liu (UBS NY) Hedibert Lopes (Rio)
Viridiana Lourdes (ITAM Mexico) Giovanni Petris (Univ Arkansas)
Michael Pitt (Warwick) Nicolas Polson (Chicago)
Raquel Prado (Santa Cruz) Jose M Quintana (Nikko NY)
Peter Rossi (Chicago) Neil Shephard (Oxford)
![Page 62: Bayesian Time Series](https://reader030.vdocuments.mx/reader030/viewer/2022020101/553253e955034626518b4587/html5/thumbnails/62.jpg)
Mike West - ISDS, Duke University Valencia VII, 2002
... and, of course, ...
Valencia VII Invited Papers
• Quintana, Lourdes, Aguilar & Liu
– Global gambling: multivariate financial time series
• Davy & Godsill
– Signal processing, latent structure, MCMC
... plus a number of contributed talks and posters