2d-pme method and rex-ms method - application of computational chemistry -

National Institute of Advanced Industrial Science and Technology

2D-PME method and REX-MS method

- Application of computational chemistry -

Masaaki KawataMasaaki KawataGrid Technology Research Center, AIST, JAPANGrid Technology Research Center, AIST, JAPAN

[email protected]@aist.go.jp


Outline

IntroductionIntroduction

What is 2D-PME method ?What is 2D-PME method ?

- how fast 2D-PME method is ! -- how fast 2D-PME method is ! -

What is REX-MS method ?What is REX-MS method ?

- how efficient REX-MS method is ! -- how efficient REX-MS method is ! -

Perspective Perspective

- combination of two methods - - combination of two methods -


Introduction

Task parallel calculationTask parallel calculation

ex. Parallel molecular dynamics or ex. Parallel molecular dynamics or Monte Carlo simulations.Monte Carlo simulations.

Data parallel calculationData parallel calculation

ex. optimum-pairing-search of drug ex. optimum-pairing-search of drug compounds (parameter survey)compounds (parameter survey)


Introduction(2)

Task parallel calculationTask parallel calculation

Given a conformation of target compound, Given a conformation of target compound, accurate estimation of physical quantities, accurate estimation of physical quantities, whose accuracy are comparable to whose accuracy are comparable to experiment, requires large amount of experiment, requires large amount of computational resources.computational resources.

⇒⇒ 　　 parallel algorithm of molecular parallel algorithm of molecular simulationsimulation

→ → Parallel calculation of Coulomb Parallel calculation of Coulomb interactioninteraction 　　　　　　　　　　

　　　　　　　　　＝　＝ 2D-PME method2D-PME method


Introduction(3)

Data parallel calculationData parallel calculationStatistical interpretation should be Statistical interpretation should be included in molecular simulation. But included in molecular simulation. But now we can’t do statistical operation now we can’t do statistical operation due to limited computational due to limited computational resources.resources.Statistical operation requires extensive Statistical operation requires extensive survey of parameters space. survey of parameters space.

　　　　⇒⇒　　 New strategy dealing with New strategy dealing with statistical ensemblestatistical ensemble

　　　　　　 → → REX-MS methodREX-MS method


What is 2D-PME method ?

2D-PME method stands for Two-Dimensional Partic2D-PME method stands for Two-Dimensional Particle Mesh Ewald method.le Mesh Ewald method.Fast and accurate method to calculate Fast and accurate method to calculate Coulomb intCoulomb interactioneraction in three-dimensional systems with two-di in three-dimensional systems with two-dimensional periodicity (mensional periodicity (quasi-2D systemsquasi-2D systems).).

xy

z


Why quasi-2D systems ?

Nano applicationNano application⇒⇒surface of nano-structuressurface of nano-structures

Bio applicationBio application ⇒ ⇒ membrane proteinmembrane protein 　　 for drug for drug designdesign

should be treated as Quasi-2D should be treated as Quasi-2D systemsystem


What is quasi-2D system

x y

z

(Left) Quasi two-dimensional simulation box, i.e., three-dimensional box with two-dimensional periodicity in the (x, y) directions and with non-periodicity in the z direction. Original particles are contained in the central box with lengths of the sides, Lx, Ly, and Lz in the x, y, and z directions, respectively. (Right) Self-assembled monlayer membrane system. Images of the simulation box are repeated i

n the (x,y) directions.

H2O

CH3(CH2)nS

Au

H2O


Coulomb interaction

7

6

5

4

3

2

1

0

kcal/mol

500400300200100Angstrom

r

qqClNa

Coulomb interaction has longer tail more thCoulomb interaction has longer tail more than size of simulation boxan size of simulation box→→ 　　 Ewald methodEwald method

More than 90 % of CPU time is consumed fMore than 90 % of CPU time is consumed for calculation of Coulombor calculation of Coulomb 　　 interactionsinteractions →→ 　　 fast Ewald methodfast Ewald method

Acceleration of molecular simulationsAcceleration of molecular simulations　　⇒　　　⇒　 Acceleration of Ewald methodAcceleration of Ewald method


2D Ewald method

yx

N

i

N

j yxji

ji qq

hh hhrr, 1 1

'

2

1

skkr 0k0k

yx

N

i

N

jyxij

yxij

jir qq

hh

hhrhhr, 1 1

' erfc2

1

1

0

1

0

),,(),,(1 x

x

y

y z

G

t

G

t tzyxzyx

yx

k tttQtttQhh

0k

N

ii

kgi

k zBq1

,0k0k

N

ii

s q1

2


2D-PME method

1

0

1

0

),,(),,(1 x

x

y

y z

G

t

G

t tzyxzyx

yx

k tttQtttQhh

0k

Charge qContribution from each grid point: Ｑ

(k1,k2,k3)

N

jjj iqS

1

2exp)( rmm

))(Q(F)()( mmm CS


Results(1)

System 1

System 2

System 3


Results(2)

CPU time on Compaq Alpha Station XP1000 (Alpha21264 667MHz).

CPU timea for a single-step MD calculation for a given accuracy, by using the original method and the 2D-PME method.

System 1 System 2 System 3

Number ofcharges

2928 2955 2817

Lx=Ly (Å) 32.15 22.29 46.35

Lz (Å) 32.15 66.87 15.45

Original Ewaldmethod (s)

1098.29 1455.58 1828.77

2D-PMEmethod(s)

2.38 6.52 2.86

SpeedupSpeedup 461.5461.5 223.2223.2 639.4639.4


Update and redistribute and

Flow chart of parallel MD calculations with the 2D-PME method.

backwardforward

Data transform as illustrated (backward)

jr jr

k0k

),,( zyxM

k0kF

),,(~

),,(~

zyxzyx mmmQmmm

2D FFT and 1D Fourier integral

Calculate and together with van der Waals interactions

Calculate and

Construct from

Calculate bonded interactions

Construct and their derivatives

),,( zyx tttQSet

Calculate

Calculate

Calculate

r rF

k0kF

k0k

jr ju

Data transform as illustrated (forward)

2D FFT and 1D Fourier integral

Sum of all interactions


Results (3)

Speedup factor of the parallel MD calculations for the quasi-2D systems by using the 2D-PME method. The solid line is for calculations with an SP switch (300MB/sec BI-Direction) with user space protocol, and the dashed line is for calculations with internet protocol. The dotted line is the ideal speedup factor, assuming an infinitely fast network connecting the nodes.

60

50

40

30

20

10

Sp

eed

up

fac

tor

605040302010Number of processors


What is REX-MS method?

REX-MS stands for Replica Exchange REX-MS stands for Replica Exchange Molecular Simulation method.Molecular Simulation method.

Computationally efficient sampling Computationally efficient sampling method in the phase space. method in the phase space.

　　　⇒ 　⇒ global optimization problemglobal optimization problem

suitable for grid environmentsuitable for grid environment


In REX-MS method

Replica１

Replica２

ReplicaN

Extended statistical ensemble

1. Run N independent simulation (N replicas) with N different parameters, respectively.

2. Exchange information among replicas during the simulation.

3. Search optimum solution among N simulations.

Sampling by using REX strategy is more efficient than that by the sum of N independent simulations (not REX).


Two type of REX-MS methods

REX-MC method (by t.ikegami & h.takemiya)REX-MC method (by t.ikegami & h.takemiya)Replica exchange Monte Carlo calculationReplica exchange Monte Carlo calculation

REX-MD method (by m.ito)REX-MD method (by m.ito)Replica exchange Molecular Dynamics toolkitReplica exchange Molecular Dynamics toolkit


REX-MC (by t.ikegami)

Potential energy survey of molecules using direct Potential energy survey of molecules using direct method method (Combination of REX-MC with ab-initio MO (Combination of REX-MC with ab-initio MO calculationcalculation ））⇒Superior to random walk survey of complicated

potential surface


Gridifying the program

Bookkeeper

Serversmeta-computing test bed

10 institutes/20 Supercomputers

MonitoringReconfiguration

ab initio calculation

Task allocation

Dynamic scheduling

REXMC client

Reconfiguration request

Two levels of parallelizationTwo levels of parallelizationCoarse grained: parallel monte-carlo samplingFine grained: parallel ab-initio energy calculation

Dynamic task scheduling, machiDynamic task scheduling, machine reconfigurationne reconfiguration

Task scheduling for balancing load on a heterogeneous computing resourcesMachine scheduling for reconfiguring machine sets on the fly


AISTAIST

HPC Challenge in SC2002

10 institutions (3 continentals) / 20 parallel computer (7 types)High Performance Computing Center Stuttgart (HLRS), Sandia National Laboratories (SNL), Pittsburgh Supercomputing Center (PSC), Grid Technology Research Center (AIST), Manchester Computing Centre (MCC), National Center for High Performance Computing (NCHC), Japan Atomic Energy Research Institute (JAERI), Korea Institute of Science and Technology Information (KISTI), European Center of Parallelism in Barcelona (CEPBA/CIRI), Finnish IT center for Science (CSC).

Metacomputing Test-bedMetacomputing Test-bed

Bookkeeper

REXMC ClientFor C20 singlet

REXMC ClientFor C20 triplet


Potential Survey on the Metacomputing 　 testbed

Parallel execution of 1Parallel execution of 1 ６（３６（３２）２） MC samplesMC samples

Running over 150 hours with changing machine configurationsCalculating 145 MC samples/hour using 860 CPU’s (at maximum)

Potential survey for C20 might be completed in a month

(cf. > ~30 years on a single CPU)

Negligible Communication cost< 1% of total time

Dynamic scheduling/configuration Dynamic scheduling/configuration mechanism is useful formechanism is useful for

long time simulationSimulation on the unstable environment


REMD toolkit (by m.ito)

A toolkit was developed to build a replica-exchange A toolkit was developed to build a replica-exchange method program suitable for solving the multiple-method program suitable for solving the multiple-minima problem.minima problem.

It is designed as an object-oriented framework to generate variants of simulation programs by assembling the toolkit components and force field programs.The toolkit components provides the parallelization mechanisms for various computational environments and the sampling methods.An arbitrary force field implementation can be plugged into the toolkit to generate an executable program.

Mindy(NAMD)

MPI Grid Serial

ＭＣＭＤ

Ｎｅｗ　Ｍｏｄｅｌ

REMD toolkit


Why do we need a software framework for REM/REMD simulations?

Estimation of Estimation of thermal distributions for molecular structurethermal distributions for molecular structuress is essential for elucidating biological functionalities, e.g. is essential for elucidating biological functionalities, e.g. a ligand-receptor binding.a ligand-receptor binding.Such estimation, however, is notoriously difficult at low teSuch estimation, however, is notoriously difficult at low temperature because of a multiple-minima problem.mperature because of a multiple-minima problem.

Replica exchange method (REM) and replica-exchange molecular dynamics method (REMD) can overcome the difficulty.

Each component of an REM/REMD algorithm,Each component of an REM/REMD algorithm, a statistical a statistical ensemble, conformational sampling method, and potentiaensemble, conformational sampling method, and potential energy functionl energy function has to be customized to suit a particular has to be customized to suit a particular biomolecular system. biomolecular system. Simulation programs are likely to adjust to various parallelSimulation programs are likely to adjust to various parallelization environments.ization environments.

An object-oriented framework can facilitate to generate a variants of REM/REMD programs suitable for various molecular systems and different computational environments such as PC clusters and Grid.Component-based software development

構造座標

Po

ten

tial

　

En

erg

y


Efficiency of the REMD toolkit

The efficiency of the geThe efficiency of the generated program (toolkit nerated program (toolkit + Mindy(NAMD)) was ex+ Mindy(NAMD)) was examined by estimzating tamined by estimzating the heat capacity.he heat capacity.The error was found to The error was found to decay naturally accordindecay naturally according to the central limit theg to the central limit theorem.orem.


Perspective

Combination of 2D-PME method and REX-MS methCombination of 2D-PME method and REX-MS method leads to fast, accurate and extensive survey of tod leads to fast, accurate and extensive survey of the screening.he screening.⇒⇒higherhigher 　　 throughput in drug design and in matethroughput in drug design and in material design of nano-structure.rial design of nano-structure.Implementation of those methods on grid environmImplementation of those methods on grid environment brings new stage of the design processes. ent brings new stage of the design processes. ⇒⇒ 　　 a promisinga promising 　　 application of grid technology.application of grid technology.


Reference

ForFor 　　 more details about: more details about: 2D-PME2D-PME, ask me or , ask me or [email protected]@aist.go.jpREX-MCREX-MC, ask h.takemiya (here), ask h.takemiya (here) 　　 or mail to or mail to [email protected]@aist.go.jpREMD toolkitREMD toolkit, mail to , mail to [email protected]@aist.go.jp

2d-pme method and rex-ms method - application of computational chemistry -

Documents