large scale parallel materials simulations with …...industrial forum for multiscale materials...

Industrial Forum for Multiscale Materials Modeling and Training, 11-13 Sept. 2013, Bologna, Italy

Large Scale Parallel MaterialsSimulations with LAMMPS

Dr. Axel Kohlmeyer

Senior Scientific Computing ExpertThe Abdus Salam International Centre

for Theoretical Physics&

Associate Dean for Scientific ComputingCollege of Science and Technology

Temple University

http://sites.google.com/site/akohlmey/

http://sites.google.com/site/akohlmey/

2Industrial Forum for Multiscale Materials Modeling and Training, 11-13 Sept. 2013, Bologna, Italy

What LAMMPS Is

● Large-scale Atomic/Molecular Massively Parallel Simulator● Classical Molecular Dynamics (MD) framework for a variety of

force fields and methods: biomolecules and polymers (soft matter), materials science (solids), mesoscale to continuum

● Open source, portable and extensible C++● Home page: http://lammps.sandia.gov

http://lammps.sandia.gov/


LAMMPS General Properties

● Serial execution, MPI parallel, OpenMP parallel, CUDA and OpenCL acceleration

● Easy to compile and install, few dependencies● Many pairwise, many-body and bonded models● Analysis and post-processing during MD run,

“rerun” option for additional post-processing● Easy to extend with custom features

(usually sufficient to just add C++ classes)● Active developer and user community

(Most of LAMMPS written by >100 contributors)


Not so Commonly Found Features

● Generalized Aspherical Particles (lines/meshes, “multi-bodies” with internal+external integrator)

● Stochastic Rotation Dynamics (SRD)● Grand Canonical Monte Carlo (GCMC)● Peridynamics mesoscale modeling● Atom-to-Continuum coupling to FE solvers and back● Discrete Element Modeling with CFD coupling

(LIGGGHTS + OpenFOAM = CFDEM)● Electron Force Field (eFF)● Kinetic Monte Carlo Coupling (to SPPARKS)


Popular in Materials Modeling● Integrated scripting language

● Loops, conditions, variables/functions etc. allow to build workflows for complex multi-step operations

● Build complex systems from inside the MD code● Python interface for more complex scripting and

integration into external frameworks

● Hybrid and multi-scale models● Combine multiple force field types via overlays

or mechanical coupling● Coupling to external codes: FE, kMC, QM/MM


Hybrid Models● Solid/liquid interfaces, solid/solid interfaces

=> mechanical coupling, simple model (LJ) for interface● Metal islands on amorphous

(LJ) substrate● Specify 2 (or more) pair

potentials:A-A, B-B, A-B, etc

● Overlay potentials:add explicit h-bondsadd coulomb

● Hybrid in two ways:potentials (pair, bond, etc)atom style (bio, metal, etc)


LAMMPS Development Trends

Coupling to Other Length Scales

Enhanced Sampling Methods

More AccuratePotentials

(QM w/o QM)Coarse Grain& Mesoscale

Optimization,Load Balancing,Parallelization,Accelerators

Classical MDNot Much To Do

GUI Frontends,Visualization,

Interactive MD


Methods to Bridge Time Scales

● Metadynamics, Adaptive Biasing Forces, Steered MD, Targeted MD and more=> available via specialized fixes or commands => available via packages: USER-COLVARS (ported from NAMD) or PLUMED (external)

● Temperature Accelerated MD, Parallel Replica MD, Parallel Tempering Replica Exchange=> available via internal commands=> extensions under development externally (not always or not yet contributed back)


Moore's Law for Potentials


Mesoscale Modeling via Peridynamics

● Particle-based meshless non-local continuum model

● Constituent model implemented as pairwise force

● Particularly suited to model fracture of brittle materials=> mesh-based models always exhibit bias in fracture=> peridynamics leads to consistent results, even when using a coarse representation (few particles)


Stochastic Rotation Dynamics

● Effective model for solvated nanoparticles● No solvent-solvent interactions to compute● Simulation of micron-size partcles for seconds● Study particle diffusivity, viscosity, aggregation


Atom-to-Continuum Package (AtC)

● Estimate continuum fields from atomistic simulation data through consistent coarse graining methods

● Couple atomistic and finite element regions for thermal, mechanical, electrostatic, charge and mass transport simulations

Stress field around crack at finite temperature

Compressive stress field for an atomic simulation of shock loading


AtC Library Components

● Meshes (hexagonal, tetragonal)● Filters (temporal and spatial)● Coarse grain (transfer to mesh):

– mass/charge density, dipole moment– dislocation, velocity, temperature– electric potential, stress and more

● Coupling:– charge diffusion, two-temperature– mechanical (momentum)– thermal (energy) and more

Saltwater-electrode-CNT system: mesh overlaps exactly

with water-CNT atom region


AtC Example: Electrostatics

- Surrogate model for electron density- Electrons segregate to tip- Potential drop across short axis of CNT- Mutual repulsion opens tip- Net charge causes net tip displacement - CNT anchored in a warm substrate


AtC: Electrical Double Layers

z

V

Compact DiffusePolarization across the channelComputing the polarization field


LIGGGHTS: Mesoscale Modeling for Granular Materials

● LIGGGHTS: fork of LAMMPS enhanced for discrete element modeling with CAD meshes

● Coupling to OpenFOAM (=> CFDEM) allows modeling of dry and wet granular materials


CFDEM Modeling Approaches

• Immersed Boundary Method• Fictitious Domain Method

Computational Fluid Dynamics (CFD)

Discrete Element Method (DEM)

Fluid-particle interaction

• unresolved CFD-DEM

• resolved CFD-DEM

+

• CFD-DEM1• coarse grained CFD-DEM 2 • MP-PIC3 1) Goniva, C., Kloss, C., Deen, N.G., Kuipers, J.A.M. and Pirker, S. (2012): “Influence of Rolling Friction Modelling on Single Spout Fluidized Bed

Simulations”, Particuology, DOI 10.1016/j.partic.2012.05.002

2) Radl S., Radeke, Ch., Khinast, J., Sundaresan, S. (2011) : ”Parcel-Based Approach for the Simulation of Gas-Particle Flows” , Proc. CFD 2011 Conference, Trondheim, Norway

3) Andrews, M.J., O’Rourke, P.J. (1996): “The multi-phase particle-n-cell (MP-PIC) method for dense particle flow”, Int. J. Multiphase Flow, 22, 379-402


Transfer Chute Dust EmissionOriginal Geometry

Cross windentrains dustfrom transfer-chute

Dust is sucked into transfer-chute

Optimized Geometry

- Transported particles modeled via LIGGGHTS- Conveyor and chute geometries from CAD meshes- Dust modeled as continuum interacting with wind via OpenFOAM


Modeling of River Bed Erosion

- turbulent lift force is essential; best performance with Zanke (2003) model

Erosion profile

Publications:

Gruber, K., Kloss, C., Goniva, C: NUMERICAL AND EXPERIMENTAL STUDY OF EROSION IN OPEN CHANNEL FLOW; Proc. IHAR 2012

Zanke, U. (2003). On the influence of turbulence on the initiation of sediment motion. International Journal of Sediment Research, pp. 17-31 Chepil, W. (1961). The use of evenly spheres to measure lift and drag on wind-eroded soil grains. Soil Sci. Soc. Am, pp. 343-345


Movies From Internal Visualization

● Sputter example with ambient occlusion

● Indent example using time averaged virial stress for colorization


LAMMPS for Outreach:The Nano Dome

● Single person immersive, stereo-3d, haptic, and interactive simulation/visualization environment

● Combines HPC, visualization, molecular simulation, virtual reality, and STEM outreach


LAMMPS MVPs

● Steve Plimpton (The Man)● Aidan Thompson, Paul Croszier (His Deputies)● Reese Jones, Jeremy Alan Templeton,

Jonathan Zimmerman (Atom-to-Continuum)● Christoph Kloss, Christoph Goniva (CFDEM)● Greg Scantlen (Nanodome)● ... and many more that don't all fit into this slide

or the photo in the background


lammps.sandia.gov


LAMMPS Workshop & Symposium@ ICTP, March 17 – 21st 2014

● First in such activity europe after three successful workshops in Albuquerque(Feb 2010, Aug 2011, Aug 2013)

● Bring together LAMMPS contributors and users● Easier to reach from Europe and ICTP support

for researcher from developing countries● Two Tracks: a) In depth tutorial for beginners,

b) developer training for experienced users● General Symposium on March 21st 2014

large scale parallel materials simulations with …...industrial forum for multiscale materials...

Documents