1

Sparse Inverse Covariance Estimation with Graphical LASSO

J. Friedman, T. Hastie, R. TibshiraniBiostatistics, 2008

Presented by Minhua Chen

2

• Motivation• Mathematical Model• Mathematical Tools• Graphical LASSO• Related papers

Outline

3

Motivation

(M. Choi, V. Chandrasekaran and A.S. Willsky, 2009)

(O. Banerjee, L. Ghaoui,and A. d’Aspremont, 2008)

4

• The optimization problem is concave (M. Yuan and Y. Lin, 2007).

• Various optimization algorithms have been proposed (M. Yuan and Y. Lin, 2007; O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008; N. Meinshausen and P. Buhlmann, 2006).

• The Graphical LASSO algorithm, built on a previous paper (O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008) , is widely used due to its

computational efficiency. • It transforms the above optimization to LASSO regressions.

Mathematical Model

5

• Subgradient (J. Tropp, 2006)Mathematical Tools (1)

Example 1: Example 2:

6

Mathematical Tools (2)• Matrix inversion identity:

• The above equations reveal the relationship between the inverse covariance matrix and the covariance matrix.

7

Graphical LASSO (1)

8

Graphical LASSO (2)

9

Graphical LASSO (3)

10

Graphical LASSO (4)

Ground Truth Inferred

11

Related papers:• N. Stadler and P. Buhlmann, Missing Values: Sparse Inverse

Covariance Estimation and an Extension to Sparse Regression Proposed a MissGLasso algorithm to impute the missing data

and infer the inverse covariance matrix simultaneously.• O. Banerjee, L. El Ghaoui and A. d’Aspremont, Model Selection

Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data

Used a constrained quadratic programming algorithm (COVSEL) to solve the same optimization problem as Graphical LASSO.

• N. Meinshausen and P. Buhlmann, High-Dimensional Graphs and Variable Selection with the Lasso

Proposed a neighborhood selection method to approximate the Gaussian Graph.