variational inference of the poisson log-normal model some ...€¦ · variational inference of the...
TRANSCRIPT
Variational inference of the Poisson log-normal modelSome applications in ecology
S. Robin
Joint work with J. Chiquet & M. Mariadassou
AIGM, Dec. 2017, Toulouse
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 1 / 34
Multivariate analysis of abundance data
Multivariate analysis of abundance data
Variational inference of PLN
Probabilistic PCA for counts
Network inference
Discussion
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 2 / 34
Multivariate analysis of abundance data Abundance data
Community ecology
Abundance data. Y = [Yij ] : n × p:
Yij = abundance of species j in sample i (old)
= number of reads associated with species j in sample i (new)
Need for multivariate analysis:
I to summarize the information from Y
I to exhibit patterns of diversity
I to understand between-species interactions
More generally, to model dependences between count variables
→ Need for a generic (probabilistic) framework
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 3 / 34
Multivariate analysis of abundance data Abundance data
Community ecology
Abundance data. Y = [Yij ] : n × p:
Yij = abundance of species j in sample i (old)
= number of reads associated with species j in sample i (new)
Need for multivariate analysis:
I to summarize the information from Y
I to exhibit patterns of diversity
I to understand between-species interactions
More generally, to model dependences between count variables
→ Need for a generic (probabilistic) framework
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 3 / 34
Multivariate analysis of abundance data Abundance data
Models for multivariate count data.
Abundance vector: Yi = (Yi1, . . .Yip), Yij = counts ∈ N
No generic model for multivariate counts.
I Data transformation (Yij = log(1 + Yij),√Yij)
→ Pb when many counts are zero.
I Poisson multivariate distributions→ Constraints of the form of the dependency [IYAR16]
I Latent variable models→ Poisson-Gamma (= negative binomial): positive dependency→ Poisson-log normal [AH89]
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 4 / 34
Multivariate analysis of abundance data Abundance data
Models for multivariate count data.
Abundance vector: Yi = (Yi1, . . .Yip), Yij = counts ∈ N
No generic model for multivariate counts.
I Data transformation (Yij = log(1 + Yij),√Yij)
→ Pb when many counts are zero.
I Poisson multivariate distributions→ Constraints of the form of the dependency [IYAR16]
I Latent variable models→ Poisson-Gamma (= negative binomial): positive dependency→ Poisson-log normal [AH89]
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 4 / 34
Multivariate analysis of abundance data Abundance data
Poisson-log normal (PLN) distribution
Latent Gaussian model:
I (Zi )i : iid latent vectors ∼ Np(0,Σ)
I Yi = (Yij)j : counts independent conditional on Zi
Yij |Zij ∼ P(eµj+Zij
)
Properties:
E(Yij) = eµj+σ2j /2 =: λj > 0
V(Yij) = λj + λ2j
(eσ
2j − 1
)(over-dispersion)
Cov(Yij ,Yik) = λjλk (eσjk − 1) (same sign as σjk)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 5 / 34
Multivariate analysis of abundance data Abundance data
Poisson-log normal (PLN) distribution
Latent Gaussian model:
I (Zi )i : iid latent vectors ∼ Np(0,Σ)
I Yi = (Yij)j : counts independent conditional on Zi
Yij |Zij ∼ P(eµj+Zij
)Properties:
E(Yij) = eµj+σ2j /2 =: λj > 0
V(Yij) = λj + λ2j
(eσ
2j − 1
)(over-dispersion)
Cov(Yij ,Yik) = λjλk (eσjk − 1) (same sign as σjk)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 5 / 34
Multivariate analysis of abundance data Abundance data
Poisson-log normal (PLN) distribution
Extensions.
I xi = vector of covariates for observation i ;
I oij = known ’offset’.
Yij | Zij ∼ P(eoij+xᵀi βj+Zij )
Interpretation.
I Dependency structure encoded in the latent space (i.e. in Σ)
I Additional effects are fixed
I Conditional Poisson = noise model
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 6 / 34
Multivariate analysis of abundance data Abundance data
Poisson-log normal (PLN) distribution
Extensions.
I xi = vector of covariates for observation i ;
I oij = known ’offset’.
Yij | Zij ∼ P(eoij+xᵀi βj+Zij )
Interpretation.
I Dependency structure encoded in the latent space (i.e. in Σ)
I Additional effects are fixed
I Conditional Poisson = noise model
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 6 / 34
Variational inference of PLN
Multivariate analysis of abundance data
Variational inference of PLN
Probabilistic PCA for counts
Network inference
Discussion
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 7 / 34
Variational inference of PLN Variational inference
Intractable EM
Aim of the inference:
I estimate θ = (β,Σ)
I predict the Zi ’s
Maximum likelihood. EM requires to evaluate (some moments of)
p(Z | Y ) =∏i
p(Zi | Yi )
but no close form for p(Zi | Yi ).
I [Kar05] resorts to numerical or Monte-Carlo integration.
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 8 / 34
Variational inference of PLN Variational inference
Intractable EM
Aim of the inference:
I estimate θ = (β,Σ)
I predict the Zi ’s
Maximum likelihood. EM requires to evaluate (some moments of)
p(Z | Y ) =∏i
p(Zi | Yi )
but no close form for p(Zi | Yi ).
I [Kar05] resorts to numerical or Monte-Carlo integration.
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 8 / 34
Variational inference of PLN Variational inference
Variational EM
Variational approximation: replace p(Z | Y ) with
p(Z ) =∏i
N (Zi ; mi , Si )
and maximize the lower bound (E = expectation under p)
J(θ, p) = log pθ(Y )− KL[p(Z )||p(Z |Y )]
= E[log pθ(Y ,Z )] +H[p(Z )]
Variational EM.
I VE step: find the optimal p (i.e. mi ’s and diagonal Si ’s)
I M step: update θ.
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 9 / 34
Variational inference of PLN Variational inference
Variational EM
Variational approximation: replace p(Z | Y ) with
p(Z ) =∏i
N (Zi ; mi , Si )
and maximize the lower bound (E = expectation under p)
J(θ, p) = log pθ(Y )− KL[p(Z )||p(Z |Y )]
= E[log pθ(Y ,Z )] +H[p(Z )]
Variational EM.
I VE step: find the optimal p (i.e. mi ’s and diagonal Si ’s)
I M step: update θ.
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 9 / 34
Variational inference of PLN Variational inference
Variational EM
Property: The lower J(θ, p) is bi-concave, i.e.
I wrt p = (M, S) for fixed θ
I wrt θ = (Σ, β) for fixed p (close form for Σ = n−1(MᵀM + S+))
but not jointly concave in general.
Implementation: Gradient ascent for the complete parameter (M, S , θ)
I No formal VEM algorithm.
PLNmodels package:
https://github.com/jchiquet/PLNmodels
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 10 / 34
Variational inference of PLN Variational inference
Variational EM
Property: The lower J(θ, p) is bi-concave, i.e.
I wrt p = (M, S) for fixed θ
I wrt θ = (Σ, β) for fixed p (close form for Σ = n−1(MᵀM + S+))
but not jointly concave in general.
Implementation: Gradient ascent for the complete parameter (M, S , θ)
I No formal VEM algorithm.
PLNmodels package:
https://github.com/jchiquet/PLNmodels
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 10 / 34
Probabilistic PCA for counts
Multivariate analysis of abundance data
Variational inference of PLN
Probabilistic PCA for counts
Network inference
Discussion
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 11 / 34
Probabilistic PCA for counts pPCA
Probabilistic PCA
Dimension reduction. Typical task in multivariate analysis
Model: Probabilistic PCA (pPCA):
(Zi )i iid ∼ Np(0,Σ), rank(Σ) = q p
Yij |Zij ∼ P(eoij+xᵀi βj+Zij )
Recall that: rank(Σ) = q ⇔ ∃B(p × q) : Σ = BBᵀ.
pPCA in the PLN model. Variational inference:
maximize J(θ, p)
→ Still bi-concave in θ = (B, β) and (M, S)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 12 / 34
Probabilistic PCA for counts pPCA
Probabilistic PCA
Dimension reduction. Typical task in multivariate analysis
Model: Probabilistic PCA (pPCA):
(Zi )i iid ∼ Np(0,Σ), rank(Σ) = q p
Yij |Zij ∼ P(eoij+xᵀi βj+Zij )
Recall that: rank(Σ) = q ⇔ ∃B(p × q) : Σ = BBᵀ.
pPCA in the PLN model. Variational inference:
maximize J(θ, p)
→ Still bi-concave in θ = (B, β) and (M, S)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 12 / 34
Probabilistic PCA for counts pPCA
Probabilistic PCA
Dimension reduction. Typical task in multivariate analysis
Model: Probabilistic PCA (pPCA):
(Zi )i iid ∼ Np(0,Σ), rank(Σ) = q p
Yij |Zij ∼ P(eoij+xᵀi βj+Zij )
Recall that: rank(Σ) = q ⇔ ∃B(p × q) : Σ = BBᵀ.
pPCA in the PLN model. Variational inference:
maximize J(θ, p)
→ Still bi-concave in θ = (B, β) and (M, S)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 12 / 34
Probabilistic PCA for counts pPCA
Model selection
Number of components q: needs to be chosen.
Penalized ’likelihood’.
I log pθ(Y ) intractable: replaced with J(θ, p)
I BIC [Sch78] → vBICq = J(θ, p)− pq log(n)/2
I ICL [BCG00] → vICLq = vBICq −H(p)
Chosen rank:
q = arg maxq
vBICq or q = arg maxq
vICLq
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 13 / 34
Probabilistic PCA for counts Illustration
Pathobiome: Oak powdery mildew
Data from [JFS+16].
I n = 116 oak leaves = samples
I p1 = 66 bacterial species (OTU)
I p2 = 48 fungal species (p = 114)
I covariates: tree (resistant, intermediate, susceptible), branch height, distanceto trunk, ...
I offsets: oi1, oi2 = offset for bacteria, fungi
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 14 / 34
Probabilistic PCA for counts Illustration
Pathobiome: PLN model (q = p)
Without covariates: offset only
Regression parameters
−8 −7 −6 −5 −4 −3 −2
01
23
45
Covariance matrix
0 20 40 60 80
020
0040
00
Model parameters
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 15 / 34
Probabilistic PCA for counts Illustration
Pathobiome: PLN model (q = p)
With covariates: offset, tree (suscept., interm, resist.), orientation
Regression parameters
−20 −10 0 10 20
010
2030
Covariance matrix
0 5 10 15 20
010
0020
00
Model parameters
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 15 / 34
Probabilistic PCA for counts Illustration
Pathobiome: PCA rank selection
R2
= 0
.28
R2
= 0
.49
R2
= 0
.6R
2 =
0.6
8R
2 =
0.7
3R
2 =
0.7
8R
2 =
0.8
1R
2 =
0.8
3R
2 =
0.8
5R
2 =
0.8
7R
2 =
0.8
8R
2 =
0.8
9R
2 =
0.9
R2
= 0
.91
R2
= 0
.92
R2
= 0
.92
R2
= 0
.93
R2
= 0
.94
R2
= 0
.94
R2
= 0
.95
R2
= 0
.95
R2
= 0
.95
R2
= 0
.96
R2
= 0
.96
R2
= 0
.96
R2
= 0
.96
R2
= 0
.97
R2
= 0
.97
R2
= 0
.97
R2
= 0
.97
−200000
−150000
−100000
−50000
0 10 20 30
R2
= 0
.33
R2
= 0
.47
R2
= 0
.56
R2
= 0
.64
R2
= 0
.7R
2 =
0.7
3R
2 =
0.7
7R
2 =
0.7
9R
2 =
0.8
2R
2 =
0.8
4R
2 =
0.8
5R
2 =
0.8
6R
2 =
0.8
8R
2 =
0.8
9R
2 =
0.9
R2
= 0
.9R
2 =
0.9
1R
2 =
0.9
2R
2 =
0.9
2R
2 =
0.9
3R
2 =
0.9
3R
2 =
0.9
4R
2 =
0.9
4R
2 =
0.9
5R
2 =
0.9
5R
2 =
0.9
5R
2 =
0.9
5R
2 =
0.9
6R
2 =
0.9
6R
2 =
0.9
6−125000
−100000
−75000
−50000
0 10 20 30
criterion
JBICICL
offset only: q = 24 offset + covariates: q = 21
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 16 / 34
Probabilistic PCA for counts Illustration
Visualization
PCA: Optimal subspaces nested when q increases.
PLN-pPCA: Non-nested subspaces.
→ For a the selected dimension q:
I Compute the estimated latent positions M
I Perform PCA on the M
I Display results in any dimension q ≤ q
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 17 / 34
Probabilistic PCA for counts Illustration
Pathobiome: First 2 PCs
A1.02
A1.03
A1.04A1.05
A1.06A1.07A1.08
A1.09A1.10
A1.11A1.12A1.13
A1.14A1.15
A1.16
A1.17A1.18
A1.19
A1.20
A1.21
A1.22
A1.23A1.24 A1.25A1.26
A1.28
A1.29
A1.30A1.31
A1.32 A1.33A1.34
A1.35
A1.36A1.37
A1.38
A1.39
A1.40
A2.01A2.02
A2.03A2.04
A2.05A2.06A2.07
A2.08
A2.09
A2.10
A2.11
A2.12
A2.13A2.14 A2.15A2.16A2.17
A2.18
A2.19
A2.20
A2.21
A2.22
A2.23
A2.24
A2.25
A2.26
A2.28A2.29
A2.30
A2.31
A2.32
A2.33
A2.34
A2.35A2.36A2.37
A2.38
A2.39A2.40
A3.01
A3.02A3.03
A3.04A3.05
A3.06
A3.08A3.09 A3.10
A3.11A3.12
A3.13
A3.14
A3.15
A3.16A3.17A3.18 A3.19
A3.20
A3.21
A3.22
A3.23
A3.24 A3.25A3.26A3.27
A3.28A3.29A3.30
A3.31A3.32
A3.33
A3.34A3.35
A3.36A3.37
A3.38 A3.39
A3.40
−30
−20
−10
0
10
−25 0 25 50 75axis 1 (48.52%)
axis
2 (
15.6
9%)
A1.02
A1.03
A1.04
A1.05
A1.06
A1.07
A1.08
A1.09A1.10
A1.11
A1.12
A1.13
A1.14
A1.15 A1.16
A1.17
A1.18
A1.19
A1.20
A1.21
A1.22
A1.23
A1.24
A1.25
A1.26A1.28
A1.29
A1.30
A1.31
A1.32
A1.33A1.34
A1.35A1.36
A1.37A1.38 A1.39
A1.40
A2.01 A2.02
A2.03
A2.04
A2.05A2.06
A2.07
A2.08
A2.09A2.10A2.11
A2.12A2.13
A2.14
A2.15
A2.16
A2.17
A2.18
A2.19A2.20
A2.21
A2.22
A2.23A2.24
A2.25
A2.26
A2.28A2.29
A2.30
A2.31
A2.32
A2.33A2.34
A2.35
A2.36
A2.37
A2.38
A2.39
A2.40
A3.01A3.02
A3.03
A3.04
A3.05A3.06
A3.08A3.09
A3.10
A3.11
A3.12
A3.13
A3.14
A3.15
A3.16
A3.17A3.18
A3.19A3.20 A3.21
A3.22A3.23
A3.24
A3.25 A3.26
A3.27
A3.28A3.29
A3.30
A3.31A3.32
A3.33A3.34
A3.35
A3.36
A3.37
A3.38
A3.39
A3.40
−10
0
10
20
−20 0 20axis 1 (31.16%)
axis
2 (
13.4
1%)
treea
a
a
intermediateresistantsusceptible
offset only offset + covariates
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 18 / 34
Probabilistic PCA for counts Illustration
Pathobiome: Precision of Zij
√V(Zij)
1
2
3
45
0 10 100
500
1000
1500
2000
Yij
Due to the link function (log), V(Zij) is higher when Yij is close to 0.
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 19 / 34
Network inference
Multivariate analysis of abundance data
Variational inference of PLN
Probabilistic PCA for counts
Network inference
Discussion
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 20 / 34
Network inference Problem
Problem
Aim: ’infer the ecological network’
Statistical interpretation: infer the graphical model of the Yi = (Yi1, . . .Yip), i.e.the graph G such that
p(Yi ) ∝∏
C∈C(G)
ψC (Y Ci )
where C(G ) = set of cliques of G
Count data: No generic framework (see Intro)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 21 / 34
Network inference Problem
PLN network inference
Cheat: Use the PLN model and infer the graphical model of Z
Graphical model of Z 6= Graphical model of Y
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 22 / 34
Network inference Problem
PLN network inference
Cheat: Use the PLN model and infer the graphical model of Z
Z1
Z2
Z3
Z4 Z5
Y1
Y2
Y3
Y4 Y5
Y1
Y2
Y3
Y4 Y5
Graphical model of Z 6= Graphical model of Y
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 22 / 34
Network inference Problem
PLN network inference
Cheat: Use the PLN model and infer the graphical model of Z
Z1
Z2
Z3
Z4 Z5
Y1
Y2
Y3
Y4 Y5
Y1
Y2
Y3
Y4 Y5
Graphical model of Z 6= Graphical model of Y
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 22 / 34
Network inference Problem
PLN network model
Model:
(Zi )i iid ∼ Np(0,Ω−1), Ω sparse
Yij |Zij ∼ P(eoij+xᵀi βj+Zij )
Interest: Similar to Gaussian graphical model (GGM) inference
Sparsity-inducing regularization: graphical lasso (gLasso, [FHT08])
log pθ(Y )− λ ‖Ω‖1,off
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 23 / 34
Network inference Problem
PLN network model
Model:
(Zi )i iid ∼ Np(0,Ω−1), Ω sparse
Yij |Zij ∼ P(eoij+xᵀi βj+Zij )
Interest: Similar to Gaussian graphical model (GGM) inference
Sparsity-inducing regularization: graphical lasso (gLasso, [FHT08])
log pθ(Y )− λ ‖Ω‖1,off
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 23 / 34
Network inference Problem
PLN network model
Model:
(Zi )i iid ∼ Np(0,Ω−1), Ω sparse
Yij |Zij ∼ P(eoij+xᵀi βj+Zij )
Interest: Similar to Gaussian graphical model (GGM) inference
Sparsity-inducing regularization: graphical lasso (gLasso, [FHT08])
log pθ(Y )− λ ‖Ω‖1,off
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 23 / 34
Network inference Variational inference
Variational inference
Same problem: log pθ(Y ) is intractable
Variational approximation: maximize
J(θ, p)− λ ‖Ω‖1,off = E[log pθ(Y ,Z )] +H[p(Z )]−λ ‖Ω‖1,off
withp(Z ) =
∏N (Zi ; mi , Si )
→ Still bi-concave in θ = (Ω, β) and p = (M, S). Ex:
Ω = arg maxΩ
n
2
(log |Ω| − tr(ΣΩ)
)− λ‖Ω‖1,off : gLasso problem
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 24 / 34
Network inference Variational inference
Variational inference
Same problem: log pθ(Y ) is intractable
Variational approximation: maximize
J(θ, p)− λ ‖Ω‖1,off = E[log pθ(Y ,Z )] +H[p(Z )]−λ ‖Ω‖1,off
withp(Z ) =
∏N (Zi ; mi , Si )
→ Still bi-concave in θ = (Ω, β) and p = (M, S). Ex:
Ω = arg maxΩ
n
2
(log |Ω| − tr(ΣΩ)
)− λ‖Ω‖1,off : gLasso problem
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 24 / 34
Network inference Variational inference
Variational inference
Same problem: log pθ(Y ) is intractable
Variational approximation: maximize
J(θ, p)− λ ‖Ω‖1,off = E[log pθ(Y ,Z )] +H[p(Z )]−λ ‖Ω‖1,off
withp(Z ) =
∏N (Zi ; mi , Si )
→ Still bi-concave in θ = (Ω, β) and p = (M, S). Ex:
Ω = arg maxΩ
n
2
(log |Ω| − tr(ΣΩ)
)− λ‖Ω‖1,off : gLasso problem
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 24 / 34
Network inference Variational inference
Model selection
Network density: controlled by λ
Penalized ’likelihood’.
I vBIC (λ) = J(θ, p)− log n2
(pq + |Support(Ωλ)|
)I EBIC (λ) : Extended BIC [FD10]
Stability selection.
I Get B subsamples
I Get Ωbλ for an intermediate λ and b = 1...B
I Count the selection frequency of each edge
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 25 / 34
Network inference Variational inference
Model selection
Network density: controlled by λ
Penalized ’likelihood’.
I vBIC (λ) = J(θ, p)− log n2
(pq + |Support(Ωλ)|
)I EBIC (λ) : Extended BIC [FD10]
Stability selection.
I Get B subsamples
I Get Ωbλ for an intermediate λ and b = 1...B
I Count the selection frequency of each edge
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 25 / 34
Network inference Variational inference
Model selection
Network density: controlled by λ
Penalized ’likelihood’.
I vBIC (λ) = J(θ, p)− log n2
(pq + |Support(Ωλ)|
)I EBIC (λ) : Extended BIC [FD10]
Stability selection.
I Get B subsamples
I Get Ωbλ for an intermediate λ and b = 1...B
I Count the selection frequency of each edge
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 25 / 34
Network inference Illustration
Oak powdery mildew: PLNmodels package
Syntax:
formula.offset <- Count ∼ 1 + offset(log(Offset))
models.offset <- PLNnetwork(formula.offset)
best.offset <- models.offset$getBestModel("BIC")
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 26 / 34
Network inference Illustration
Oak powdery mildew: no covariates
models.offset$plot()
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
R2
= 0
.99
−42000
−40000
−38000
−36000
1 2 3
penalty
valu
e
criterion
loglik
BIC
EBIC
Model selection criteria
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 27 / 34
Network inference Illustration
Oak powdery mildew: no covariates
best.offset$plot()
Theta
−20 −15 −10 −5
010
2030
40
Sigma
−50 0 50 100 150 200 250
020
0040
0060
00
model parameters
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 28 / 34
Network inference Illustration
Oak powdery mildew: no covariates
best.offset$plot network()
f_1
f_2
f_3
f_4
f_5
f_6
f_7
f_8
f_9
f_10
f_12
f_13
f_15
f_17
f_19
f_20
f_23
f_24
f_25
f_26
f_27
f_28
f_29
f_30
f_32 f_33
f_39
f_40
f_43
f_46
f_57
f_63
f_65
f_68f_79f_317
f_576
f_579f_662
f_672
f_1011
f_1085
f_1090
f_1141
f_1278
f_1567
f_1656
E_alphitoides
b_1045
b_109
b_1093
b_11
b_112
b_1191
b_1200b_123
b_13
b_1431
b_153b_17
b_171
b_18
b_182
b_20
b_21
b_22
b_23
b_235
b_24b_25
b_26
b_27 b_29
b_304b_31
b_329
b_33
b_34
b_35
b_36
b_364
b_37
b_39
b_41
b_42
b_44
b_443
b_444
b_447
b_46
b_47
b_48
b_49
b_51
b_548
b_55
b_56
b_57
b_58
b_59
b_60
b_625
b_63b_662
b_69b_72
b_73b_74
b_76
b_8b_81b_87
b_90
b_98
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 29 / 34
Network inference Illustration
Oak powdery mildew: effect of the covariates
no covariates covariate = tree + orientation
no covar, deg(Ea) = 78
f_1
f_2
f_3
f_4
f_5
f_6f_7
f_8
f_9
f_10f_12
f_13
f_15
f_17
f_19
f_20
f_23
f_24
f_25
f_26
f_27
f_28
f_29
f_30
f_32
f_33
f_39
f_40
f_43
f_46f_57f_63
f_65f_68 f_79
f_317f_576
f_579
f_662
f_672
f_1011f_1085
f_1090
f_1141
f_1278
f_1567
f_1656
E_alphitoides
b_1045
b_109
b_1093
b_11
b_112b_1191
b_1200
b_123
b_13
b_1431
b_153
b_17
b_171
b_18
b_182b_20
b_21
b_22
b_23
b_235
b_24
b_25
b_26
b_27
b_29
b_304 b_31
b_329
b_33b_34
b_35
b_36
b_364
b_37 b_39
b_41
b_42
b_44b_443b_444
b_447
b_46
b_47
b_48
b_49b_51
b_548
b_55
b_56
b_57
b_58
b_59
b_60
b_625b_63
b_662
b_69
b_72
b_73
b_74
b_76
b_8b_81
b_87
b_90
b_98
covar, deg(Ea) = 21
f_1
f_2
f_3
f_4
f_5
f_6f_7
f_8
f_9
f_10f_12
f_13
f_15
f_17
f_19
f_20
f_23
f_24
f_25
f_26
f_27
f_28
f_29
f_30
f_32
f_33
f_39
f_40
f_43
f_46f_57f_63
f_65f_68 f_79
f_317
f_576
f_579
f_662
f_672
f_1011 f_1085
f_1090
f_1141
f_1278
f_1567
f_1656
E_alphitoides
b_1045
b_109
b_1093
b_11
b_112b_1191
b_1200
b_123
b_13
b_1431
b_153
b_17
b_171
b_18
b_182b_20
b_21
b_22
b_23
b_235
b_24
b_25
b_26
b_27
b_29
b_304 b_31
b_329
b_33b_34
b_35
b_36
b_364
b_37 b_39
b_41
b_42
b_44b_443b_444
b_447
b_46
b_47
b_48
b_49b_51
b_548
b_55
b_56
b_57
b_58
b_59
b_60
b_625b_63
b_662
b_69
b_72
b_73
b_74
b_76
b_8
b_81
b_87
b_90
b_98
Ea = Erysiphe alphitoides = pathogene responsible for oak mildew
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 30 / 34
Network inference Illustration
Oak powdery mildew: stability selection
no covariates covariate = tree + orientation
no covar + stabsel, deg(Ea) = 15
f_1
f_2
f_3
f_4
f_5
f_6f_7
f_8
f_9
f_10f_12
f_13
f_15
f_17
f_19
f_20
f_23
f_24
f_25
f_26
f_27
f_28
f_29
f_30
f_32
f_33
f_39
f_40
f_43
f_46f_57f_63
f_65f_68 f_79
f_317f_576
f_579
f_662
f_672
f_1011 f_1085
f_1090
f_1141
f_1278
f_1567 f_1656
E_alphitoides
b_1045
b_109
b_1093
b_11
b_112b_1191
b_1200
b_123
b_13
b_1431
b_153
b_17
b_171
b_18
b_182b_20
b_21
b_22
b_23
b_235
b_24
b_25
b_26
b_27
b_29
b_304 b_31
b_329
b_33b_34
b_35
b_36
b_364
b_37 b_39
b_41
b_42
b_44b_443b_444
b_447
b_46
b_47
b_48
b_49b_51
b_548
b_55
b_56
b_57
b_58
b_59
b_60
b_625b_63
b_662
b_69
b_72
b_73
b_74
b_76
b_8b_81
b_87
b_90
b_98
covar + stabsel, deg(Ea) = 2
f_1
f_2
f_3
f_4
f_5
f_6f_7
f_8
f_9
f_10f_12
f_13
f_15
f_17
f_19
f_20
f_23
f_24
f_25
f_26
f_27
f_28
f_29
f_30
f_32
f_33
f_39
f_40
f_43
f_46f_57f_63
f_65
f_68 f_79
f_317
f_576
f_579
f_662
f_672
f_1011 f_1085
f_1090
f_1141
f_1278
f_1567 f_1656E_alphitoides
b_1045
b_109
b_1093
b_11
b_112b_1191
b_1200
b_123
b_13
b_1431
b_153
b_17
b_171
b_18
b_182b_20
b_21
b_22
b_23
b_235
b_24
b_25
b_26
b_27
b_29
b_304 b_31
b_329
b_33b_34
b_35
b_36
b_364
b_37b_39
b_41
b_42
b_44b_443b_444
b_447
b_46
b_47
b_48
b_49b_51
b_548
b_55
b_56
b_57
b_58
b_59
b_60
b_625b_63
b_662
b_69
b_72
b_73
b_74
b_76
b_8b_81
b_87
b_90
b_98
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 31 / 34
Discussion
Multivariate analysis of abundance data
Variational inference of PLN
Probabilistic PCA for counts
Network inference
Discussion
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 32 / 34
Discussion
Discussion
Summary
I PLN = generic model for multivariate count data analysis
I Allows for covariates
I Flexible modeling of the covariance structure
I Efficient VEM algorithm
I PLNmodels package: https://github.com/jchiquet/PLNmodels
To do list
I Model selection criterion for network inference
I Tree-based network inference (R. Momal’s PhD)
I Other covariance structures (spatial, time series, ...)
I Statistical properties of the variational estimates (for regular PLN)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 33 / 34
Discussion
Discussion
Summary
I PLN = generic model for multivariate count data analysis
I Allows for covariates
I Flexible modeling of the covariance structure
I Efficient VEM algorithm
I PLNmodels package: https://github.com/jchiquet/PLNmodels
To do list
I Model selection criterion for network inference
I Tree-based network inference (R. Momal’s PhD)
I Other covariance structures (spatial, time series, ...)
I Statistical properties of the variational estimates (for regular PLN)
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 33 / 34
Discussion
References
J. Aitchison and C.H Ho. The multivariate Poisson-log normal distribution. Biometrika, 76(4):643–653, 1989.
C. Biernacki, G. Celeux, and G. Govaert. Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans. Pattern Anal.
Machine Intel., 22(7):719–25, 2000.
R. Foygel and M. Drton. Extended Bayesian information criteria for gaussian graphical models. In Advances in neural information processing systems, pages
604–612, 2010.
J. Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432–441, 2008.
D. I. Inouye, E. Yang, G. I. Allen, and P. Ravikumar. A review of multivariate distributions for count data derived from the Poisson distribution. Technical
Report 1609.00066, arXiv, 2016.
B. Jakuschkin, V. Fievet, L. Schwaller, T. Fort, C. Robin, and C. Vacher. Deciphering the pathobiome: Intra-and interkingdom interactions involving the
pathogen Erysiphe alphitoides. Microbial ecology, pages 1–11, 2016.
D. Karlis. EM algorithm for mixed Poisson and other discrete distributions. Astin bulletin, 35(01):3–24, 2005.
G. Schwarz. Estimating the dimension of a model. Ann. Statist., 6:461–4, 1978.
S. Robin (INRA / AgroParisTech) Variational inference of the PLN model AIGM, Toulouse 34 / 34