learning potential energy lanscapes with localized graph kernels … · 2016. 11. 18. · ab initio...
TRANSCRIPT
www.enpc.fr
Learning Potential Energy
Lanscapes with Localized Graph
Kernels
IPAM Workshop
Grégoire Ferré – Terry Haut – Kipton Barros
CERMICS - ENPC & Los Alamos National Laboratory
Thursday, November 17th, 2016
Outline
1. Introduction
2. Density kernels and graphs
3. GRAPE
4. Numerical results
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 2 / 20
1. Introduction
Molecular Dynamics
Langevin Dynamics dqt = ptdt ,
dpt = −∇V(qt )dt −γptdt +√
2γβ dWt ,
Schrödinger equation:
Hψ = Eψ
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 4 / 20
Molecular Dynamics
Langevin Dynamics dqt = ptdt ,
dpt = −∇V(qt )dt −γptdt +√
2γβ dWt ,
Schrödinger equation:
Hψ = Eψ
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 4 / 20
Goal of Machine Learning Potentials
Md requires the computation of forces for many configurations
Atomic environment
Objective
Estimate:
• the global energy E(x)
• the forces −∇E(x)
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 5 / 20
Goal of Machine Learning Potentials
Md requires the computation of forces for many configurations
Atomic environment
Objective
Estimate:
• the global energy E(x)
• the forces −∇E(x)
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 5 / 20
Goal of Machine Learning Potentials
Potential at an intermediate level
Ab Initio Calculations
Accurate, long to compute, depend on the number of electrons
Machine Learning Potentials and Forces
Ab Initio accuracy with lower computational cost
Analytical Potentials and Forces
Unable to reproduce all the properties of a complex material
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 6 / 20
Goal of Machine Learning Potentials
Potential at an intermediate level
Ab Initio Calculations
Accurate, long to compute, depend on the number of electrons
Machine Learning Potentials and Forces
Ab Initio accuracy with lower computational cost
Analytical Potentials and Forces
Unable to reproduce all the properties of a complex material
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 6 / 20
Goal of Machine Learning Potentials
Potential at an intermediate level
Ab Initio Calculations
Accurate, long to compute, depend on the number of electrons
Machine Learning Potentials and Forces
Ab Initio accuracy with lower computational cost
Analytical Potentials and Forces
Unable to reproduce all the properties of a complex material
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 6 / 20
Goal of Machine Learning Potentials
Potential at an intermediate level
Ab Initio Calculations
Accurate, long to compute, depend on the number of electrons
Machine Learning Potentials and Forces
Ab Initio accuracy with lower computational cost
Analytical Potentials and Forces
Unable to reproduce all the properties of a complex material
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 6 / 20
Kernel Ridge Regression and localization
Approximated energy E
From a database (xi ,E(xi ))Ni=1 and a kernel K we have
E(x) =N∑i=1
αiK(xi ,x), with α =(K +NλI
)−1EN and Ki ,j = K(xi ,xj ).
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 7 / 20
Kernel Ridge Regression and localization
Approximated energy E
From a database (xi ,E(xi ))Ni=1 and a kernel K we have
E(x) =N∑i=1
αiK(xi ,x), with α =(K +NλI
)−1EN and Ki ,j = K(xi ,xj ).
Localization
Assume that the energy decomposes as E(xi ) =∑ni
j=1 ε(x ji ), then
E(x) =N∑i=1
βi
nx∑j ′=1
ni∑j=1
K(x j′, x ji ), β = (LKLT +NλI)−1EN ,
where the entries of K are the localized correlations K(x ji , xj ′
i ′ ).
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 7 / 20
Difficulties
Physical properties:
• the number of atoms may vary,
• invariance with respect to ordering of atoms,
• rotation and translation invariance,
• stability, differentiability, control of the approximation,
• multiscale features.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 8 / 20
Difficulties
Physical properties:
• the number of atoms may vary,
• invariance with respect to ordering of atoms,
• rotation and translation invariance,
• stability, differentiability, control of the approximation,
• multiscale features.
Extensive literature on this problem:
• symmetry functions (Behler & Parrinello, 2007),
• Smooth Overlap of Atomic Position (Soap, Csányi, Bartók, Kondor,2010),
• Coulomb (Rupp, Tkatchenko, Müller, von Lilienfeld, 2012),
• Internal Vector coordinates (Li, Kermode, De Vita, 2015),
• scattering transform (Hirn, Poilvert, Mallat, 2015),
• Moment Tensor Polynomials (MTP, Shapeev, 2016).
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 8 / 20
Difficulties
Physical properties:
• the number of atoms may vary,
• invariance with respect to ordering of atoms,
• rotation and translation invariance,
• stability, differentiability, control of the approximation,
• multiscale features.
Extensive literature on this problem:
• symmetry functions (Behler & Parrinello, 2007),
• Smooth Overlap of Atomic Position (Soap, Csányi, Bartók, Kondor,2010),
• Coulomb (Rupp, Tkatchenko, Müller, von Lilienfeld, 2012),
• Internal Vector coordinates (Li, Kermode, De Vita, 2015),
• scattering transform (Hirn, Poilvert, Mallat, 2015),
• Moment Tensor Polynomials (MTP, Shapeev, 2016).
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 8 / 20
2. Density kernels and graphs
Functional Representation of Atomic Position
−1 −0.5 0 0.5 1
0
0.25
0.5
0.75
1
x
ϕ(x)
(a)√
2πσe− ‖r‖
2
2σ2
−1 −0.5 0 0.5 1
0
0.5
1
1.5
·10−2
xϕ(x)
(b) e− 1σ2−‖r‖2 1{‖r‖<σ }
−1 −0.5 0 0.5 1
0
0.25
0.5
0.75
1
1.25
x
ϕ(x)
(c) σ−11{‖r‖<σ }(r)
−1 −0.5 0 0.5 1
0
0.2
0.4
0.6
x
ϕ(x)
(d) Hat function
Figure: Functional representation of one atom
Function ϕσ : R3 7→R
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 10 / 20
Functional Representation of Atomic Position
−1 −0.5 0 0.5 1
0
0.25
0.5
0.75
1
x
ϕ(x)
(a)√
2πσe− ‖r‖
2
2σ2
−1 −0.5 0 0.5 1
0
0.5
1
1.5
·10−2
xϕ(x)
(b) e− 1σ2−‖r‖2 1{‖r‖<σ }
−1 −0.5 0 0.5 1
0
0.25
0.5
0.75
1
1.25
x
ϕ(x)
(c) σ−11{‖r‖<σ }(r)
−1 −0.5 0 0.5 1
0
0.2
0.4
0.6
x
ϕ(x)
(d) Hat function
Figure: Functional representation of one atom
Function ϕσ : R3 7→R
«Smoothed» atomic position
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 10 / 20
Functional Representation of Atomic Configuration
Configuration x = (ri ,zi )ni=1 ∈ (R3 ×R+)n .
Associated density : ρ(r) =n∑
i=1
wziϕσ (r − ri ), ϕσ (r) = e− ‖r‖
2
2σ2
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 11 / 20
Functional Representation of Atomic Configuration
Configuration x = (ri ,zi )ni=1 ∈ (R3 ×R+)n .
Associated density : ρ(r) =n∑
i=1
wziϕσ (r − ri ), ϕσ (r) = e− ‖r‖
2
2σ2
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 11 / 20
Density kernel and graphs
Similarity and kernel between two densities ρ, ρ′ :
S(ρ,ρ′) =
∫R
3ρρ′ , K(ρ,ρ′) =
∫SO(3)
S(ρ,Rρ′)pdR , for some p ≥ 2.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 12 / 20
Density kernel and graphs
Similarity and kernel between two densities ρ, ρ′ :
S(ρ,ρ′) =
∫R
3ρρ′ , K(ρ,ρ′) =
∫SO(3)
S(ρ,Rρ′)pdR , for some p ≥ 2.
Actually,
S(ρ,ρ′) =n∑
i=1
n ′∑j=1
wziwzj exp
− (ri − r ′j )2
4σ2
= Tr(AAT ), where
Ai ,j =√wziwzjϕ2σ
(|ri − r ′j |
), is the
adjacency matrix of a bipartite graph.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 12 / 20
Density kernel and graphs
Similarity and kernel between two densities ρ, ρ′ :
S(ρ,ρ′) =
∫R
3ρρ′ , K(ρ,ρ′) =
∫SO(3)
S(ρ,Rρ′)pdR , for some p ≥ 2.
Actually,
S(ρ,ρ′) =n∑
i=1
n ′∑j=1
wziwzj exp
− (ri − r ′j )2
4σ2
= Tr(AAT ), where
Ai ,j =√wziwzjϕ2σ
(|ri − r ′j |
), is the
adjacency matrix of a bipartite graph. Figure: Associated bipartite graph
.Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 12 / 20
3. GRAPE
Graph Representation
Define the adjacency matrix of a configuration as:
Ai ,j =ωziωzjϕσ (|ri − rj |)fcut(|ri |)fcut(|rj |)
Figure: Graph representation of amethane molecule, CH4.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 14 / 20
Graph Representation
Define the adjacency matrix of a configuration as:
Ai ,j =ωziωzjϕσ (|ri − rj |)fcut(|ri |)fcut(|rj |)
Graph kernels since 2002 (Gärtner):
• spectral properties (Kondor et. al,2002),
• shortest paths (Borgwadt et al.,2005),
• graphlets (Shervashidze &Vishwanathan, 2009),
• random walks (Vishwanathan,Schraudolph, Kondor & Borgwardt,2010),
• functional embedding (Shrivastava &Li, 2014).
Figure: Graph representation of amethane molecule, CH4.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 14 / 20
Random walk graph kernels
Adjacency matrices A , A ′ as generators of Markov processes.
Figure: Two systems constituted of 3 and 4particles represented as graphs with weights wikand w′jl . The direct product graph representedbelow has weights wij ,kl = wikw
′jl .
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 15 / 20
Random walk graph kernels
Adjacency matrices A , A ′ as generators of Markov processes.
Figure: Two systems constituted of 3 and 4particles represented as graphs with weights wikand w′jl . The direct product graph representedbelow has weights wij ,kl = wikw
′jl .
Kronecker product:
A ⊗A ′ =
A1,1A ′ . . . A1,nA ′...
...An ,1A ′ . . . An ,nA ′
,Idea:
• starting distributions p , p ′ ,
• stopping distributions q , q ′ ,
• compare the random walksthrough the power iterationsAk , A ′k .
Choose p , p ′ , q , q ′ uniform.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 15 / 20
GRaphe APproximated Energy (GRAPE)
Sum over walks of all lengths with penalization µk :
k(A ,A ′) = (q ⊗q ′)T ∞∑k=0
µk (A ⊗A ′)kp ⊗ p ′ .
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 16 / 20
GRaphe APproximated Energy (GRAPE)
Sum over walks of all lengths with penalization µk :
k(A ,A ′) = (q ⊗q ′)T ∞∑k=0
µk (A ⊗A ′)kp ⊗ p ′ .
For example µk = γk /k !:
k(A ,A ′) = q ⊗q ′eγA⊗A′p ⊗ p ′ .
Renormalization step:
K(A ,A ′) =
k(A ,A ′)√k(A ,A)k(A ′ ,A ′)
ζ ,for some ζ > 0, and derivative:
∂r ′k(A ,A ′) = q ⊗q ′(γA ⊗∂r ′A ′)eγA⊗A′p ⊗ p ′ .
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 16 / 20
4. Numerical results
GRAPE (Graph Approximated Potential)
Procedure for molecules:
• localize molecules into local environments
• define adjacency matrices inspired from SOAP
• run the localized kernel method with the exponential kernel
• adjust the numerical parameters on a validation set
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 18 / 20
Benchmark on molecular Data
Fitting the atomization energy of molecules constituted of C, N, O, S,and H (QM7 database).
-2000
-1500
-1000
-500
-2000 -1500 -1000 -500
GR
AP
E R
egre
ssio
n E
nerg
y [kcal/m
ol]
Reference Energy [kcal/mol]
N 100 300 500
CoulombMAE 25.6 19.8 17.9
RMSE 50.8 33.5 27.1
GRAPEMAE 11.2 10.1 9.6
RMSE 14.9 13.9 13.3
SOAPMAE 15.6 11.3 10.4
RMSE 21.0 15.6 14.5Mean average error and root mean squareerror for Coulomb, GRAPE and SOAP, where
N is the training set size.
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 19 / 20
Conclusion & Tracks
Conlusion
Results:
• density-geometry description
• graph kernel for energy regression
• satisfying results on a standard dataset
Future works• include more physics in the graph design
• uncomplete-overcomplete kernel, multiscale
• forces computation
• transferability capacities
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 20 / 20
Conclusion & Tracks
Conlusion
Results:
• density-geometry description
• graph kernel for energy regression
• satisfying results on a standard dataset
Future works• include more physics in the graph design
• uncomplete-overcomplete kernel, multiscale
• forces computation
• transferability capacities
Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory
Learning Potential Energy Lanscapes with Localized Graph Kernels 20 / 20