2008 First journal publication

on HemeLB.

2010 Development of steering and visualization client.

2009 HemeLB run across sites

Using MPI-g.

2011 Improved domain

decomposition with ParMETIS.

2013 Multiscale 3D-1D couplingwith Python Navier Stokes.

2013 Up to 50% faster

Calculations with SSE.

2014 Better load balance using weighted decomposition.

2012 Support for 2nd order

accurate wall conditions.

2014 First comparison tests with

clinical data.

2014 HemeLB-Chaste coupling

to simulate vascularremodelling processes.

2013 More stable model

constructionwith CGAL.

2014Gained support for

implementing immersed boundary conditions.

2012 Performance prediction

Model.

2012 Improved scalability

With coalesced comms design pattern.

2011 Code reengineered forimproved accuracy and

stability.

2014Support for PT-Scotch,

Zoltan and ParMETIS indomain decomposition.

Performance improvements

Accuracy improvements

Scientific advances

New functionalities

2013 Framework for

convenient propertyextraction

Evolution of the HemeLB bloodflow simulation environment

The performance of HemeLB has improved considerably over the past 7 years:

Indeed, for smaller geometries we can now do production simulations (e.g., 5M time steps)in less than an hour. However, simulating a full Circle of Willis model on a relevant time scale (e.g., 25M time steps) will still take about a day.

Circle of Willisrun

(first trial)

HemeLB models sparse vasculature geometries as a lattice of fluid sites. These geometries contain bulk sites, wall sites and in/outlet sites. Both wall and in/outlet sites are generally more expensive to compute than bulk sites, leading to load imbalance among different processes.

To reduce this load imbalance, we assign weights to each lattice site (see right below) before we partition and distribute the domain among

the processes. Using this approach resulted we have managed to reduce the calculation load imbalance by up to 85%.

Highlight: Weighted Decomposition

Recommended readingContributorsMarco Mazzeo, Steven Manos, Gary Doctors – initial developersRupert Nash, Hywel Carver, James Hetherington, Timm Krueger – Main developers during 2010-2013.Miguel Bernabeu, Derek Groen, Sebastian Schmieschek – current HemeLB developers.Dan Holmes – Colloids code 2012.Jiri Jaros, David Abou Chacra – Performance optimizations during 2013.Jens Nielsen – CGAL setup tool optimizations 2013.Gregor Matura, Fang Chen – pre- and post-processing 2014.Aditya Jitta – Comparison against clinical data, 2013.

2013 Comparison of different

Rheology models.

2012 Comparison of different

wall conditions.

Cerebrovascular bloodflow Introducing HemeLB

Derek Groen, Miguel Bernabeu, Rupert Nash, Sebastian Schmieschek and Peter Coveney

2013 Support for

Colloidal particles

2010 Python model construction

tool developed.

● Stroke is the main cause of about 1.1M deaths per year in Europe. ● About 15% of these strokes are caused by bleeding in the brain, e.g. due to the rupture of brain aneurysms. ● These brain aneurysms frequently reside in arteries branching from the Circle of Willis.

● HemeLB is a feature-rich simulation environment for modelling blood flow in sparse geometries.● It relies on the lattice-Boltzmann method and is well suited for execution on large supercomputers.

HemeLB consists of several key components:● The main simulation code, which can be coupled to other codes.● The setup tool, for constructing 3 dimensional geometries from segmented angiography scans.● A Python-based framework for constructing initial conditions and analyzing output data.● An interactive steering and visualization tool.● A Python-based automation environment for deploying and executing the code on remote machines.

Circle of Willis with one diminshed artery. © Nevit Dilmen

Simulating bloodflow

Clinicians can conveniently measure blood pressure and flow velocities on patients at rest with limited resolution.

Simulations allow us to estimate and predict flow properties in other regimes as well. These include:● Flow velocity estimates for patients during exercise and other forms of activity.● Wall stress estimations under all these conditions (e.g., wall shear stress).● Flow properties in specific locations within a geometry, e.g. velocities and stresses in an aneurysm sac.

Both very high and very low wall shear stress have been associated with aneurysm formation and rupture.

Sample visualization of a HemeLB simulation.

1. Performance: JoCS, DOI: 10.1016/j.jocs.2013.03.0022. Weighted Decomposition: EASC 2014. Preprint available.3. Boundary Conditions: Phys. Rev. E 89, 023303 (2014).4. Clinical Validation: work in progress. Slides available.5. Multiscale: Interface Focus, DOI: 10.1098/ rsfs.2012.00876. Retinal blood flow: Interface (submitted) arXiv:1311.1640.

Access the source code at:http://ccs.chem.ucl.ac.uk/hemelb

For requesting preprints, please send an e-mail to Derek Groen (djgroennl@gmail.com).

Highlight: Comparison against clinical data

We have compared flow predictions from HemeLB with clinical measurements. We have done this work in collaboration with Fergus Robertson and Hoskote Chandrashekar from UCL Hospital.

● We obtained rotational angiography images of a middle cerebral artery, as well as velocity measurements (TCD) in 5 planes within this artery.● We imposed one plane as a velocity-based inlet in HemeLB, and ran the code to predict the velocities in the four other planes (we used pressure outlets).● See below for an overview of the geometry, and a comparison between the HemeLB flow predictions and the TCD measurements on the plane furthest away from the inlet.

● We are currently working to repeat this exercise with a second patient, and to improve our comparison techniques.

2012 Tools for automatic

compilation and execution on remote machines.

inlet @63mm.

v plane @49mm.

2014 Prediction of vascular

development in retinas.

2020SCIENCEwww.2020science.net

UKCOMESUK Consortium On Mesoscale

Engineering Sciences

Legend