parallel multiphysics simulations of particles in ... · simulations of particles in electrokinetic...

Computer Science X - System Simulation Group Dominik Bartuschat ([email protected])

Parallel Multiphysics Simulations of Particles in Electrokinetic FlowsD. Bartuschat1, K. Masilamani2, S. Ganguly3,

C. Feichtinger1, D. Ritter1, U. Rüde1

1

October 5, 2011

1 Chair for System Simulation, University Erlangen-Nürnberg, Germany2 German Research School for Sciences, Aachen, Germany3 Research & Development Division, Tata Steel, India

ParNum Conference - Leibnitz, Austria

mailto:[email protected]



Outline

Motivation

Lattice Boltzmann Method

Electro-osmotic Flow

Uncharged Particles in Electro-osmotic Flow

Charged Particles in Non-electrolyte Solutions

Parallel Performance

2




Lab-on-a-chip

Performing laboratory operations on small scale.

Advantages:Highly portable due to downscaling (Point-of-care diagnostics).

Small volumes needed reduce time to synthesize and analyze samples.

Separation, manipulation and analysis of single cells, DNA, proteins, ...

3

Taken from Pont-Tech Corporation,Italy (2004)




Lab-on-a-chip

Performing laboratory operations on small scale.

Mechanism: Electrokinetic motion of micro- and nano-particles inside microchannels under applied electrical field.

Design optimization of lab-on-a-chip systems by means of microfluidic simulations.

4

Taken from Pont-Tech Corporation,Italy (2004)




Electrokinetic Phenomena

Charged surface causes formation of electrical double layer (EDL) of counter-ions.Electric field applied tangentially causes EDL migration by Coulomb force acting on ions.

Fluid viscosity causes movement of surrounding fluid, leading to fluid motion in whole cross-section.

For microfluidic pumping and mixing.

5

Electro-osmosis:

Taken from www.kirbyresearch.com



http://www.kirbyresearch.com

http://www.kirbyresearch.com


Electrokinetic Phenomena

Movement of charged particles relative to a fluid in an applied (uniform) electric field.

Separation of particles, dependent on charge or friction.

Cell sorting by combination of electrophoretic and pressure-driven flow

6

Electrophoresis:

Taken from Kang, Y. and Li, D. „Electrokinetic motion of particles and cells in microchannels“ Microfluidics and

Nanofluidics




Outline

Motivation






7





8

Discrete lattice Boltzmann equation (single relaxation time).

Discretize domain in squares or cubes (cells).

Discrete velocities per cell and associated distribution function.

fi (x + ci∆t, t +∆t)− fi (x , t) = −1

τ(fi − f eqi ).




The streaming step

Stream-Collide

9

The equation is solved in two steps:

fi (x + ci∆t, t +∆t) = f̃i (x , t +∆t)

fi (x + ci∆t, t +∆t)− fi (x , t) = −1

τ(fi − f eqi ).




f̃i (x , t +∆t) = fi (x , t)−1

τ(fi − f eqi )The collision step

Stream-Collide

10

The equation is solved in two steps:

fi (x + ci∆t, t +∆t)− fi (x , t) = −1

τ(fi − f eqi ).




Outline

Motivation






11




Electro-osmotic Flow with LBM

12

* Z. Guo, Zeng, Shi „Discrete lattice effects on the forcing term in the lattice Boltzmann method“

LBE with forcing term *

Forcing term *

Fi =

�1− 1

2τ

�ti

��−→ci −−→u�

c2s+

�−→ci ·−→u�

c4s

−→ci

�·�−→F�

Lattice

considers influence of electrical field (and external pressure): −→F = ρe (x) ·

−→E ext −∇P

Poisson equation for electric potential Φ

−∆Φ(x) =ρe (x)

�r �0 with charge density ρ and permittivities ε




Gouy-Chapman Equation

13

Poisson-Boltzmann equation - steady-state of EDLDescribes electrostatic interactions between molecules in ionic solutions

−∇ · (�∇Φ) =�

i

zi · e · c∞i e−zi eΦkBT

zi: valence of ions, e: elementary charge, ci∞: bulk ionic concentration

Gouy-Chapman equationFor binary symmetric (z1=z2) dilute electrolyte solutions

−∆Φ(x) = −2 z e c∞

�r �0sinh

�z e

kB TΦ (x)

�

Debye-Hückel approximation for |Φ| < 25mV: −∆Φ(x) ≈ −κ2Φ (x)

Electrical double layer thickness

λd =1

κ=

��r �0 kBT

2 z2 e2 c∞




Implementation

waLBerla:widely applicable Lattice Boltzmann framework.

Suited for various flow applications.

Large-scale, MPI based parallelization.

Dynamic application switches for hetero- geneous architectures and optimization.

Solver module (waLBerla):Goal: Provide an efficient, black-box linear systems solver in waLBerla for PDEs on structured grids.

Module for implementation of linear solvers (SOR, Multigrid).

14

EOF Algorithmforeach time step, do

// solve Gouy-Chapman equation (GCE)while residual too high do

set RHS of GCEapply iterative solver to GCE (SOR, MG, ... )communicate potential in ghost layers via MPIcompute and MPI Reduce residual norm

// couple GCE and LBEcalculate external force// solve lattice Boltzmann equation (LBE)begin

stream PDFs (stream step)calculate macroscopic variables considering forcingrelax towards equilibrium PDF (collide step)communicate PDFs in ghost layers via MPI

end




Physical Setup and Validation

15

Φ(z) = 2 ln

�1 + zeζ

kBTexp(− z

λd)

1− zeζkBT

exp(− zλd)

�Analytical solution* for Gouy-Chapman in 1D:

Physical Setup:

Dimensions: W=H=L=0.5µm.

1:1 electrolyte solution

D3Q19 with SRT and external forcing.

Periodic BCs in y-direction, otherwise No-slip/Dirichlet BCs (electric ζ-Potential). *Patankar, Hu „Numerical simulation of electroosmotic flow“

Electric Potential validation:




Physical Setup and Validation

16

Physical Setup: Macroscopic Velocity validation

Analytical solution* for velocity in 2D

*Tian et al. „Lattice Boltzmann simulation of of electroosmotic flows in micro- and nano-channels“

u(z) = − ��0ζEy

µ

�1− e

κz+eκH−κz

1+eκH

�

Parameters:

E = 500V /m

c∞ = 10−4M

� = 6.95 · 10−10C 2/JmT = 273K

ζ = −25mV µ = 10−3Ns/m2

ρ = 103kg/m3

Dx = 1.25 · 10−9τ = 1.7

Re = 4 · 10−6




EOF - Parameter Study

17

Influence of different parameters on velocity:

Electric Field Ionic molar concentration

Increasing c∞ - reduces double layer thickness- but increases charge density

Linear dependence of velocity on electric field




3D Flow Formation

18




Outline

Motivation






19




Uncharged Particles

20

Particles are mapped onto lattice Boltzmann grid.

Each lattice node with cell center inside object is treated as moving boundary.

Hydrodynamic forces of fluid on particle computed by momentum exchange method*.

Fluid-particle interaction by coupling waLBerla to pe:

D.Yu, R. Mei, L.-S. Luo, W.Shyy „Viscous flow computations with the method of lattice Boltzmann equation“




Uncharged Particles

21

Uncharged particles in EOF inside microchannel




Outline

Motivation






22




Charged Particles in Fluid Flow

Fluid-particle interaction like for uncharged particles.

Electrostatic force on particle, resulting from electric potential gradient.

Simulation: Agglomeration of charged particles in water with inflow velocity 250µm/s on charged plane.

23

Potential of plane: -50mVChannel length: 90µmDx=10µm, Dt=4⋅10-5s

Charged Particles Algorithmforeach time step, do

// solve Poisson equation with particle charge density

while residual too high doset RHS of Poisson equation

apply iterative solver to Poisson equation

communicate electric potential via MPI

compute and MPI Reduce residual norm

// couple PE with LBM and potential solver

begincalculate and add hydrodynamic force on particles

calculate and add electrostatic force on particles

move particles depending on forces

end// solve lattice Boltzmann equation

beginstream PDFs (stream step)

calculate macroscopic variables

relax towards equilibrium PDF (collide step)

communicate PDFs via MPI

end




24

Particle charges: 4000⋅e Particle charges: 16000⋅e

Charged Particles in Fluid Flow




Outline

Motivation






25




Results - Parallel Performance

130x130x130 lattice cells per core

Dx=1nm, ζ=-25mV Dt=0.4⋅10-12 s

physical parameters as before

100 timesteps

Executed on RRZE‘s lima cluster: 500 compute nodes with 2 Xeon 5650 "Westmere" chips (12 cores) @2.66 GHz with 24 GB of RAM.

5500 SOR iterations in first timestep (Residual L2 Norm: 10-9)

26

Weak scaling of EOF simulation

SOR performance about 4 times higher - only necessary in first timestep

External force computation about 5% of total runtime

Parameters:

0

500

1000

1500

2000

12 24 48 96 192 384 768

27,9 55,0 107,3219,2

437,1

854,9

1726,3

LBM Weak Scaling

Ideal Scaling Actual performance

MFLUPS

Number of cores




27

Thank you for your attention

Questions (?)



parallel multiphysics simulations of particles in ... · simulations of particles in electrokinetic...

Documents