References[1] P. Neumann and N. Tchipev, A Coupling Tool for Parallel Molecular Dynamics–Continuum Simulations, Proceedings of the International Symposium on Parallel and Distributed Computing, 2012[2] R. Delgado-Buscalioni, P.V. Coveney. USHER: An algorithm for particle insertion in dense fluids. J. Chem. Phys. 119 (2): 978–987 (2003).

[3] A. Dupuis, E.M. Kotsalis, P. Koumoutsakos. Coupling lattice Boltzmann and molecular dynamics models for dense fluids. Phys. Rev. E 75, 046704 (2007).

1 Department of Computer Science, Technische Universität München, Boltzmannstr. 3, D-85748 Garching, Germany, neumanph@in.tum.de 2 Department of Applied Physics, Technische Universiteit Eindhoven, P.O. Box 513, 5600 MB Eindhoven, The Netherlands, j.d.r.harting@tue.nl

'

&

$

%

Outlook: Nanofilters

• Flow between two reservoirs through amolecular filter• Flow simulation in reservoirs: LB• Simulation of flow through filter: MD• Fully three-dimensional steady-state cou-

pling based on [3]

'

&

$

%

MaMiCo: Macro-Micro-Coupling Tool

ParticleInsertion

MomentumInsertion

MacroscopicCellServiceMacroscopicCellService

MomentumController

KineticEnergyController

MoleculeWrapper

MoleculeIterator

MDSolverInterfaceMacroscopicSolverInterface

MDSimulation ContinuumSimulation

TransferStrategy

implements

utilises

utilises

sendsdata to

receivesdata from/

sendsdata to

implements

utilises utilises

MaMiCo

User/Programmer

uses/ provides newimplementation of

•C++-development for spatiotemporal hybridmolecular–continuum simulations [1]• Support of 2D/ 3D hybrid scenarios•Macroscopic cells within MaMiCo are used to

– map macroscopic quantities between themacroscopic solver and MaMiCo

– map macro-/microscopic quantities betweenthe MD solver and MaMiCo

• Particle insertion/ deletion [2] encapsulatedwithin MaMiCo

Macroscopic Solver

MaMiCo

Molecular Dynamics

Particle Insertion/Deletion

Momentum

MassMomentum

MassMomentum

MassMomentum

'

&

$

%

Macroscopic Cell Exchange•MPI-based exchange of macroscopic cell information between MD and LB• Exchange (mostly) hidden from user• Example: LB on one process, MD on four processes

LB

MD

LB

MD

MacroscopicCellService→ Send to MD

Start: traverse LB cells. Step 1: track cells that should be sentto MD

LB

MD

MacroscopicCellService→ Receive from MD

LB

MD

MacroscopicCellService→ Setup Send-Recv

Step 2: track cells that should be re-ceived from MD

Step 3: setup send-recv topologywithin MacroscopicCellService

'

&

$

%

Parallel Results

MD on rank 0

LB domain andembedded MD

•Channel flow simulation with MD region embedded in the center• LB solver: sequential, 54× 54× 54 cells, BGK•MD solver: parallel, 1 000 000 single-centered LJ molecules• LB→ MD: Velocity relaxation in (MD) boundary cells, retaining mass in outermost cells

using USHER•MD→ LB: Only map velocities and incorporate them into particle distribution functions

Proc. Shaheen HuygensMD-LB MD MD-LB MD

1 1.0 1.0 1.0 1.08 7.2 7.3 7.3 6.9

64 46.0 49.6 45.7 45.3512 244.0 321.0 235.5 249.7

1728 484.4 814.5 456.7 494.7

Strong scaling for channel flow scenario. MD-LB: coupled simulation.MD: reference MD simulation for same MD domain without coupling.

1 10 100 1000 10000

0.7

0.78

0.86

0.94

1.02

Huygens t0Huygens t1Huygens t2Shaheen t0Shaheen t1Shaheen t2

# cores

Effi

cien

cy

Weak scaling for parallel USHER-based particle insertion [1]

'

&

$

%

Overview

Efficient implementations of hybrid molecular-continuum flow solvers are required toallow for fast and massively parallel simulations of large complex systems. Several cou-pling strategies have been proposed over the last years for 2D/ 3D, time-dependent/steady-state or compressible/ incompressible scenarios. Despite their different applica-tion areas, most of these schemes comprise the same or similar building blocks. Still, tothe authors’ knowledge, no common implementation of these building blocks is availableyet. In this contribution, the Macro-Micro-Coupling Tool is presented which is meant tosupport developers in coupling mesh-based methods with molecular dynamics. It iswritten in C++ and supports two- and three-dimensional scenarios. Its design is re-viewed, and aspects for massively parallel coupled scenarios are addressed. Scalingresults are presented for a hybrid simulation which couples a molecular dynamics codeto the Lattice Boltzmann application of the Peano framework.

Philipp Neumann1 Jens Harting2

Massively Parallel Molecular-Continuum Simulationswith the Macro-Micro-Coupling Tool