multivariate functional data analysis
TRANSCRIPT
Multivariate Functional Data Analysis
ORI ROSEN
Department of Mathematical Sciences, University of Texas at El Paso,
El Paso, Texas, USA
and
WESLEY THOMPSON
Department of Statistics, University of Pittsburgh, Pittsburgh,
Pennsylvania, USA
Outline
ā¢ Motivating example
ā¢ The model
ā¢ Estimation
ā¢ Example
Motivating Example
ā¢ Psychiatric study to compare psychotherapy to pharmacotherapyat the University of Pisa, Italy. Total of 103 acutely depressedsubjects.
ā¢ Response variables: two subscales of the clinician-administered Hamilton Depression rating Scale.
ā¢ HRSD-17 ā measures anhedonia, sleep disturbance,agitation, anxiety, weight loss
ā¢ HRSD-RV ā measures reverse vegetative symptomssuch as weight gain and hyperphagia
ā¢ Covariates: treatment group, lifetime depression spectrum
0 4 8 12 16ā2
ā1
0
1
2
3
4
weeks
stan
dard
ized
scor
e
HRSDā17
0 4 8 12 16ā2
ā1
0
1
2
3
4
5
weeks
stan
dard
ized
scor
e
HRSDāRV
The Model
yi1(tij) = xxxā²ijĀµ1Āµ1Āµ1(tij) + zzzā²ijgi1gi1gi1(tij) + Ī“i1(tij)
yi2(tij) = xxxā²ijĀµ2Āµ2Āµ2(tij) + zzzā²ijgi2gi2gi2(tij) + Ī“i2(tij)... = ...
yip(tij) = xxxā²ijĀµpĀµpĀµp(tij) + zzzā²ijgipgipgip(tij) + Ī“ip(tij)
The Model
yi1(tij) = xxxā²ijĀµ1Āµ1Āµ1(tij) + zzzā²ijgi1gi1gi1(tij) + Ī“i1(tij)
yi2(tij) = xxxā²ijĀµ2Āµ2Āµ2(tij) + zzzā²ijgi2gi2gi2(tij) + Ī“i2(tij)... = ...
yip(tij) = xxxā²ijĀµpĀµpĀµp(tij) + zzzā²ijgipgipgip(tij) + Ī“ip(tij)
ā¢ ĀµĀµĀµk(t) = (Āµk1(t), . . . , Āµkr(t))ā² is an r-vector of fixed functions,k = 1, . . . , p
The Model
yi1(tij) = xxxā²ijĀµ1Āµ1Āµ1(tij) + zzzā²ijgi1gi1gi1(tij) + Ī“i1(tij)
yi2(tij) = xxxā²ijĀµ2Āµ2Āµ2(tij) + zzzā²ijgi2gi2gi2(tij) + Ī“i2(tij)... = ...
yip(tij) = xxxā²ijĀµpĀµpĀµp(tij) + zzzā²ijgipgipgip(tij) + Ī“ip(tij)
ā¢ ĀµĀµĀµk(t) = (Āµk1(t), . . . , Āµkr(t))ā² is an r-vector of fixed functions,k = 1, . . . , p
ā¢ gggik(t) = (gik1(t), . . . , giks(t))ā² is an s-vector of randomfunctions, k = 1, . . . , p
The Model (Cont.)
yyyi(tij) = XijĀµĀµĀµ(tij) + Zijgigigi(tij) + Ī“Ī“Ī“i(tij) ,
ā¢ yyyi(tij) = (yi1(tij), . . . , yip(tij))ā², i = 1, . . . , n, j = 1, . . . ,mi.
The Model (Cont.)
yyyi(tij) = XijĀµĀµĀµ(tij) + Zijgigigi(tij) + Ī“Ī“Ī“i(tij) ,
ā¢ yyyi(tij) = (yi1(tij), . . . , yip(tij))ā², i = 1, . . . , n, j = 1, . . . ,mi.
ā¢ ĀµĀµĀµ(t) = (ĀµĀµĀµā²1(t), . . . , ĀµĀµĀµā²p(t))
ā²
The Model (Cont.)
yyyi(tij) = XijĀµĀµĀµ(tij) + Zijgigigi(tij) + Ī“Ī“Ī“i(tij) ,
ā¢ yyyi(tij) = (yi1(tij), . . . , yip(tij))ā², i = 1, . . . , n, j = 1, . . . ,mi.
ā¢ ĀµĀµĀµ(t) = (ĀµĀµĀµā²1(t), . . . , ĀµĀµĀµā²p(t))
ā²
ā¢ gggi(t) = (gggā²i1(t), . . . , gggā²ip(t))
ā²
The Model (Cont.)
yyyi(tij) = XijĀµĀµĀµ(tij) + Zijgigigi(tij) + Ī“Ī“Ī“i(tij) ,
ā¢ yyyi(tij) = (yi1(tij), . . . , yip(tij))ā², i = 1, . . . , n, j = 1, . . . ,mi.
ā¢ ĀµĀµĀµ(t) = (ĀµĀµĀµā²1(t), . . . , ĀµĀµĀµā²p(t))
ā²
ā¢ gggi(t) = (gggā²i1(t), . . . , gggā²ip(t))
ā²
ā¢ Xij = Ip ā xxxā²ij, Zij = Ip ā zzzā²ij.
The Model (Cont.)
yyyi(tij) = XijĀµĀµĀµ(tij) + Zijgigigi(tij) + Ī“Ī“Ī“i(tij) ,
ā¢ yyyi(tij) = (yi1(tij), . . . , yip(tij))ā², i = 1, . . . , n, j = 1, . . . ,mi.
ā¢ ĀµĀµĀµ(t) = (ĀµĀµĀµā²1(t), . . . , ĀµĀµĀµā²p(t))
ā²
ā¢ gggi(t) = (gggā²i1(t), . . . , gggā²ip(t))
ā²
ā¢ Xij = Ip ā xxxā²ij, Zij = Ip ā zzzā²ij.
ā¢ Ī“Ī“Ī“i(tij) = (Ī“i1(tij), . . . , Ī“ip(tij))ā²
Example
Assume 2 groups of subjects and a bivariate response (p = 2):
xxxij = (xij1, xij2)ā², where xij1 = 1, xij2 =
1 if treatment0 if control
Example
Assume 2 groups of subjects and a bivariate response (p = 2):
xxxij = (xij1, xij2)ā², where xij1 = 1, xij2 =
1 if treatment0 if control
yi1(tij) = (1 xij2)(Āµ11(tij)Āµ12(tij)
)+ gi1(tij) + Ī“i1(tij)
yi2(tij) = (1 xij2)(Āµ21(tij)Āµ22(tij)
)+ gi2(tij) + Ī“i2(tij)
Example
Assume 2 groups of subjects and a bivariate response (p = 2):
xxxij = (xij1, xij2)ā², where xij1 = 1, xij2 =
1 if treatment0 if control
yi1(tij) = (1 xij2)(Āµ11(tij)Āµ12(tij)
)+ gi1(tij) + Ī“i1(tij)
yi2(tij) = (1 xij2)(Āµ21(tij)Āµ22(tij)
)+ gi2(tij) + Ī“i2(tij)
soyi1(tij) = Āµ11(tij) + xij2 Ā· Āµ12(tij) + gi1(tij) + Ī“i1(tij)
yi2(tij) = Āµ21(tij) + xij2 Ā· Āµ22(tij) + gi2(tij) + Ī“i2(tij)
The Error Term
Ī“Ī“Ī“i(t) follows a multivariate Ornstein-Uhlenbeck (O-U) process, i.e.,
dĪ“Ī“Ī“i(t) = āAĪ“Ī“Ī“i(t) +BdWWW i(t) .
ā¢ A and B are p Ć p matrices of full rank common to all i =1, . . . , n,
ā¢ WWW i(t) is the p-dimensional Wiener process.
ā¢ Stationary if the EV of A have positive real parts.
ā¢ The stationary variance-covariance matrix Ī£ is the solution to
AĪ£ + Ī£Aā² = BBā²
The Error Term (Cont.)
In the stationary state, for s < t
Cov(Ī“Ī“Ī“i(t), Ī“Ī“Ī“i(s)) = expāA(tā s)Ī£ ,
so A controls how rapidly the influence of time s dies off by time t > s.
The Transition Density
Let ātij = tij ā ti,jā1 j = 1, . . . ,mi, ti0 = 0.
The transition density of the O-U process is given by
p(Ī“Ī“Ī“i(tij)|Ī“Ī“Ī“i(ti,jā1),ātij) ā |Ī©ātij|ā1/2 exp
ā1
2Ī³Ī³Ī³ā²tijĪ©
ā1ātij
Ī³Ī³Ī³tij
,
where
Ī³Ī³Ī³tij = Ī“Ī“Ī“i(tij)ā exp(āAātij)Ī“Ī“Ī“i(ti,jā1)
Ī©ātij = Ī£ā exp(āAātij)Ī£ exp(āAā²ātij).
Nonparametric Function Estimationā Simple Example
200 observations were generated from
yi = sin(3Ļxi) + Īµi, , 0 ā¤ xi ā¤ 1, Īµi ā¼ N(0, 0.42)
Model:
yi = Ī²0 + Ī²1xi +30āk=1
uk(xi ā Īŗk)+ + Īµi, Īµi ā¼ N(0, Ļ2Īµ )
ā¢ Low rank truncated line basis functions.
ā¢ Fixed effects: Ī²s and us fixed.
ā¢ Mixed effects: Ī²s fixed, u1, . . . , u30iidā¼ N(0, Ļ2
u).
0 0.2 0.4 0.6 0.8 1ā2
ā1
0
1
2
x
yFixed Effects Model
0 0.2 0.4 0.6 0.8 1ā2
ā1
0
1
2
x
y
Mixed Model
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
x0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
x
Nonparametric Function Estimation (Cont.)
We estimate
Āµkl(t) and gikm(t), k = 1, . . . , p, l = 1, . . . , r,m = 1, . . . , s, i = 1, . . . , n
by cubic splines using low-rank radial basis functions (French et al.,2001, Ruppert et al., 2003):
Write Āµkl(tij) = ĻĻĻā²ijĪ²Ī²Ī²kl + ĻĻĻā²ijvvvkl and gikm(tij) = ĻĻĻā²ijwwwikm + ĻĻĻā²ijuuuikm.
Nonparametric Function Estimation (Cont.)
We estimate
Āµkl(t) and gikm(t), k = 1, . . . , p, l = 1, . . . , r,m = 1, . . . , s, i = 1, . . . , n
by cubic splines using low-rank radial basis functions (French et al.,2001, Ruppert et al., 2003):
Write Āµkl(tij) = ĻĻĻā²ijĪ²Ī²Ī²kl + ĻĻĻā²ijvvvkl and gikm(tij) = ĻĻĻā²ijwwwikm + ĻĻĻā²ijuuuikm.
ā¢ Place K knots Īŗ1, . . . , ĪŗK at sample quantiles of tij.
Nonparametric Function Estimation (Cont.)
We estimate
Āµkl(t) and gikm(t), k = 1, . . . , p, l = 1, . . . , r,m = 1, . . . , s, i = 1, . . . , n
by cubic splines using low-rank radial basis functions (French et al.,2001, Ruppert et al., 2003):
Write Āµkl(tij) = ĻĻĻā²ijĪ²Ī²Ī²kl + ĻĻĻā²ijvvvkl and gikm(tij) = ĻĻĻā²ijwwwikm + ĻĻĻā²ijuuuikm.
ā¢ Place K knots Īŗ1, . . . , ĪŗK at sample quantiles of tij.
ā¢ Let ĻĻĻā²ij = (1 tij), Ī¾Ī¾Ī¾ā²ij = (|tij ā Īŗ1|3, . . . , |tij ā ĪŗK|3),
ā¢ Compute ĪK = [|Īŗk ā Īŗkā²|3]1ā¤k,kā²ā¤K and ĻĻĻā²ij = Ī¾Ī¾Ī¾ā²ijĪā1/2K (via
SVD)
Prior Distributions
Priors on the basis function coefficients:
1. Flat priors on Ī²Ī²Ī²kl.
2. vvvklindā¼ N(000, Ļ2
vklIK)
3. wwwikmindā¼ N(000,diag(Ļ2
wkm0, Ļ2wkm1
)).
4. uuuikmindā¼ N(000, Ļ2
ukmIK).
Priors on the variance components:independent IG(ai, bi), e.g., p(Ļ2
vkl) ā (Ļ2
vkl)ā(a1+1) exp(āb1/Ļ2
vkl)
Prior Distributions (Cont.)
Prior on A:
ā¢ Let A = SĪØSā1, where
S =
1 s12 s13 . . . s1ps21 1 s23 . . . s2p... ... ... . . . ...sp1 sp2 sp3 . . . 1
and ĪØ = diag(Ļ1, . . . , Ļp), Ļi > 0, i = 1, . . . , p.
ā¢ sij ā¼ N(0, Ļ2s), logĻi ā¼ N(0, Ļ2
Ļ)
Prior Distributions (Cont.)
Prior on C = BBā²:
ā¢ Let C = LDLā² (modified Cholesky factorization), where
L =
1 0 0 . . . 0l21 1 0 . . . 0l31 l32 1 . . . 0... ... ... . . . ...lp1 lp2 lp3 . . . 1
and D = diag(d1, . . . , dp), di > 0, i = 1, . . . , p
ā¢ lij ā¼ N(0, Ļ2L), log(di) ā¼ N(0, Ļ2
D)
0 4 8 12 16ā2
ā1
0
1
2
3
4
weeks
stan
dard
ized
scor
e
HRSDā17
0 4 8 12 16ā2
ā1
0
1
2
3
4
5
weeks
stan
dard
ized
scor
e
HRSDāRV
Exampleā¢ Psychiatric study to compare psychotherapy to pharmacotherapy
at the University of Pisa, Italy. Total of 103 acutely depressedsubjects.
ā¢ Response variables: 2 subscales of the clinician-administeredHamilton Depression rating Scale (measured at baseline andweekly):
HRSD-17 (0ā31)HRSD-RV (0ā13).
Higher values indicate more depressive symptoms.
ā¢ Covariates:treatment grouplifetime depression spectrum (LDS), assessed at baseline.
Example (Cont.)
yyyi(tij) = XijĀµĀµĀµ(tij) + Zijgigigi(tij) + Ī“Ī“Ī“i(tij) ,where Xij = Ip ā xxxā²ij, Zij = Ip ā zzzā²ij.
In the example yyyi(tij) = (HRSD-17,HRSD-RV)ā²
zzzij = 1xxxij = (xij1, xij2, xij3, xij4)ā²
where xij1 = 1
xij2 =
1 if subject i received psychotherapy0 if subject i received pharmacotherapy,
xij3 = LDS scorexij4 = xij2xij3.
0 4 8 12 16ā2
ā1.5
ā1
ā0.5
0
0.5
1
1.5
2
t (weeks)
Intercept
0 4 8 12 16ā1
ā0.5
0
0.5
1
1.5
t (weeks)
Treatment
0 4 8 12 16ā0.05
0
0.05
0.1
0.15
t (weeks)
LDS Score
0 4 8 12 16ā0.15
ā0.1
ā0.05
0
0.05
t (weeks)
Treatment x LDS
HRSDā17
0 4 8 12 16ā2
ā1.5
ā1
ā0.5
0
0.5
1
1.5
2
t (weeks)
Intercept
0 4 8 12 16ā2
ā1.5
ā1
ā0.5
0
0.5
1
1.5
2
t (weeks)
Treatment Group
0 4 8 12 16ā0.05
0
0.05
0.1
0.15
t (weeks)
LDS Score
0 4 8 12 16ā0.15
ā0.1
ā0.05
0
0.05
0.1
t (weeks)
Treament x LDS
HRSDāRV
Conclusions from the Analysisā¢ Little evidence of a treatment effect
ā¢ Significant effect of LDS score ā higher LDS predicts worseoutcomes
ā¢ LDS effect is more pronounced in the middle period of HRSD-17, and towards the end of the 16 weeks for HRSD-RV.
ā¢ Interaction is significant in the last 8 weeks of HRSD-17, andmarginally significant in the same time period for HRSD-RV.
ā¢ Interaction effect ā LDS is less predictive of poor response inpsychotherapy group than in pharmacotherapy group.
ā¢ Univariate fitting ā interaction is not significant; wider pointwisecredible intervals.