tensor’dataanalysis - georgia institute of...
TRANSCRIPT
Some clarificaFons
! Nth order tensor
! DefiniFon ! An element of the tensor product of N vector spaces
! When the choice of basis is implicit we think of a tensor as its representaFon as an N-‐way array
! Difficult to visualize ! We will talk mainly about 3rd order tensors ! Results are extendable to higher orders
! NotaFon ! Not standardized
6
ApplicaFons ! ProbabiliFes
! matrix: joint/cond. probability table of two variables ,
! tensor: joint/cond. probability table of a set of variables
! Text mining: ! matrix: document – term
! tensor: document – term – year – author
! Social networks: ! matrix: find communiFes
! tensor: monitor the change of the community over Fme
! CollaboraFve filtering ! matrix: user – item
! tensor: user – item – Fme
! Signal processing: Example 1 & 3
! Chemometrics: Example 2
! Etc. 7
Outline
! MoFvaFon ! Basic concepts
! Basic tensor decomposiFons
Next lecture:
! Other useful decomposiFons ! Local minima
! Tensors and graphical models
13
Outline
! MoFvaFon ! Basic concepts
! Rank and mulFlinear rank
! Matrix representaFons ! Tensor – matrix mulFplicaFon
! Basic tensor decomposiFons
14
Matrix rank
! # linearly independent rows
! # linearly independent columns ! # rank-‐1 terms
! Singular value decomposiFon (SVD)
15
Useful matrix operaFons
! Kronecker product
! Khatri-‐Rao product ! Column-‐wise Kronecker product
! Let
19
Scalar product, Frobenius norm, contracFon
! Scalar product
! Frobenius norm
! ContracFon
20
4th order tensor
Outline
! MoFvaFon ! Basic concepts
! Basic tensor decomposiFons ! CP / CANDECOMP / PARAFAC
! MulFlinear SVD
21
CP / Canonical decomposiFon / PARAFAC
23
! Polyadic form (Hitchcock, 1927)
! CANDECOMP = Canonical DecomposiFon (Carroll & Chang, 1970) ! PARAFAC = Parallel Factors (Harshman, 1970)
! Vectors are not necessarily orthogonal
CP: uniqueness
24
! Kruskal rank k(A): max k, s.t. any k columns are linearly independent
! Uniqueness
! Up to permutaFon of the terms
! Up to scaling of the factors
! Sufficient condiFon:
! Note: matrix factorizaFons are not unique
€
A = [a1 a2 a R]B = [b1 b2 b R]C = [c1 c2 c R]
ProperFes of tensor rank ! Not bounded by the dimensions of the tensor
! CompuFng R: NP-‐hard problem
! Maximum rank, typical rank
! Best rank approximaFon: ill-‐posed problem
! Rank over ≤ rank over ! Example
25
T. Kolda & B. Bader, Tensor decomposiFons and applicaFons SIAM Review, V. 51, # 3, 2009
CompuFng CP
! Many algorithms
! AlternaFng least squares: ! Repeat unFl convergence:
! OpFmize A
! OpFmize B ! OpFmize C
26
Outline
! MoFvaFon ! Basic concepts
! Basic tensor decomposiFons ! CP / CANDECOMP / PARAFAC
! MulFlinear SVD
29
MLSVD decomposiFon
! 3MFA/Tucker3 = Three-‐mode factor analysis (Tucker, 1966)
! MLSVD = MulFlinear SVD (De Lathauwer, 2000)
! normalized Tucker decomposiFon
! U(n): orthogonal ! All-‐orthogonality:
! Ordering:
30
MLSVD decomposiFon
! Not unique
! ComputaFon
! SVDs of the matrix representaFons A(1), A(2), A(3) U(1),U(2),U(3)
! A, U(1),U(2),U(3) S
31
€
Α = S •1U(1) •2U
(2) •3U(3)
= (S •1 X−1 •2 Y
−1 •3 Z−1) •1U
(1)X •2U(2)Y •3U
(3)Z
€
S = A •1 (U(1))T •2 (U
(2))T •3 (U(3))T
Best rank-‐(R1,R2,R3) approx.: applicaFons
! ApplicaFon areas ! Chemometrics ! Biomedical signal processing
! TelecommunicaFons
35
! Tool for ! Dimensionality reducFon ! Signal subspace esFmaFon
[Example 3]: Parameter esFmaFon
! Tensors can be ill-‐condiFoned in one mode but well-‐condiFoned in other modes. Not possible in matrix case
39
Matlab toolboxes
! Tensor Toolbox ! hbp://csmr.ca.sandia.gov/~tgkolda/TensorToolbox/ ! B. Bader, T. Kolda and others
! N-‐way toolbox ! hbp://www.models.life.ku.dk/nwaytoolbox ! R. Bro and C. Andersson
40