cfd modelling with openfoam -...

CFD Modelling with OpenFOAM

Hrvoje Jasak

Faculty of Mechanical Engineering and Naval Architecture, Uni Zagreb, Croatia

Wikki Ltd. United Kingdom

Oxford University, 13 October 2017

CFD Modelling with OpenFOAM – p. 1

Outline

Objective

• Present the activities of the CFD Research group in Zagreb and Wikki Ltd.

• Review progress on selected topics related to turbomachinery CFD

Topics

• CFD Group at University of Zagreb: Research Activity

• Harmonic Balance Solver for Turbomachinery flows

• Block matrix and pressure-based coupled implicit solver

• Native Overset Mesh in OpenFOAM

• NUMAP-FOAM Summer School

• Summary


Overview of Research Activity

Research Group Members, CFD Group at University of Zagreb

• CFD Research Group attached to the Chair of Turbomachinery

• 2 professors: Prof. Hrvoje Jasak, Prof. Željko Tukovic

• 1 (+ 1) post doctoral researchers: dr. Vuko Vukcevic

• 6 fully funded PhD students

• Larger group of Master Thesis and Diploma Thesis students integrated within thegroup: results of Master Thesis projects directly used in further research

• Regular external (foreign) visitors working with the group: 3-6 months

Communication and Activity

• Leading developer of OpenFOAM: http://foam-extend.fsb.hr/

• Integrated work effort, modern communication and data integration tools

• Private Media-Wiki: http://spirit.local/mediawiki/

• YouTube Channel: 8th Floor CFD@FSB

• Public web site: http://www.fsb.hr/cfd

• Approximately 25 (significant) publications per year



Open Source Software in Research and Engineering

• Open Source tools are ideal for a research environment: industrial partner gainsaccess not only to physical model equations but also to a working and validatedimplementation by expert users

• Deployment of results of research is faster and more reliable

• Proven track record of model development and delivery

• First-class students with good technical and CFD background (OpenFOAM)

• Strong multi-disciplinary group: fluids, structure multi-phase flow modelling,turbulence, optimisation. Premier source of numerics knowledge in collaborationwith leading world Universities



Areas of Research Activity, CFD Group at University of Zagreb

• CFD simulations in Turbomachinery: basic validation and verification andpractical industrial simulations

◦ Incompressible flow: pumps and turbines, wind energy devices

◦ Compressible flow: compressors and fans with pressure- and density-basedCFD solution algorithms

◦ Harmonic balance modelling in CFD as a general-purpose tool

• Gradient-based and gradient-free optimisation: continuous and discrete adjoint

• CFD in naval hydrodynamics, wave and off-shore structures

• Fundamental research in numerics: discretisation, solution techniques, HPCperformance, inter-equation coupling

• Complex solid mechanics modelling: large deformation, lubricated contact,conjugate heat transfer

• Fluid-solid interaction and “multi-physics” modelling

• Detailed fuel cell modelling using CFD

• Acoustics modelling using linearised Euler equations and coupling with CFD


Free Surface Flow Simulations


Turbomachinery Simulations

Full Support for Turbomachinery CFD

• Complex rotor-stator interfaces, for all physics models: GGI, mixing plane

• Incompressible and compressible, MRF and transient

• Periodic transient mode: harmonic balance solvers


Complex Physics in Complex Geometry

Simulating Complex Physical Processes in Complex Geometry


Harmonic Balance Solver

Expanding the Horizons: Spectral Modelling of Time-Periodic Flows

• Many CFD problems involve temporally periodic flows◦ Flows induced by periodic boundary condition◦ Flows with periodically moving objects

◦ Wave-like phenomena

• To remove irregular start-up unsteadiness, a number of periods is simulated:expensive, complicated – periodic uncertainty issues

Harmonic Balance Method

• Replacing a transient problem with a set of coupled “steady-state” snapshots byvirtue of using periodicity of the time-signal in the solution



Harmonic Balance Method: Work-Flow

• Variables are developed into Fourier series in time with n-harmonics andsubstituted into transport equation

• Transport equation with n sine and n cosine parts + mean part is obtained andwritten as a set of 2n+ 1 equations in frequency domain

• Equations are transformed back to time domain in order to be able to usetime-domain boundary conditions and time-domain non-linear flow solver

Harmonic Balance in FOAM-Extend

• Harmonic balance decomposition does not relate to a special physics model:implement HB as a choice of “temporal discretisation” scheme

• Geometric aspects of harmonic balance can be tackled without change

• Currently, HB is implemented in a segregated mode: low memory, and explicitinter-mode coupling terms. Analysis shows issues for HB without a dominantmean flow component: working towards coupled implicit HB solver

• Author: Gregor Cvijetic, Uni Zagreb



Rapid Simulation of Non-Linear Periodic Flow: Harmonic Balance

• A variable is presented by a Fourier series, using first n harmonics and the mean:replacing a transient simulation with a set of coupled steady-state problems

• Periodicity is independently developed in each computational point

• Non-linear interaction is captured without simplification

• Example: Harmonic Balance scalar equation set

∇•(uQtj )−∇•(γ∇Qtj ) = −2ω

2n+ 1

(

2n∑

i=1

P(i−j)Qti

)

Pi =

n∑

k=1

k sin(kωi∆t) for i = {1,2n}

• A transient equation is replaced by a set of n coupled quasi-steady coupled

equations of the same type

• Physical justification: if a functional form of temporal variation is known, thefunction and its time derivative can be reproduced from a small number of datapoints by fitting a prescribed harmonic function (spectral time accuracy!)



Harmonic Balance for Navier-Stokes Equations

• Harmonic balance momentum equation

∇•(utjutj )−∇•(γ∇utj ) = −2ω

2n+ 1

(

2n∑

i=1

P(i−j)uti

)

• Harmonic continuity equation∇•utj = 0

• Harmonic scalar transport

∇•(utj Qtj )−∇•(γ∇Qtj ) = −2ω

2n+ 1

(

2n∑

i=1

P(i−j)Qti

)

• Physical justification: each tj instance represents a “single-time-step” solution;time derivative terms couple solution fields to each other



Harmonic Balance Solver: ERCOFTAC Centrifugal Pump

• Validation of harmonic balance in turbulent incompressible periodic flow

• HB simulations performed using 1 and 2 harmonics: rotor and stator blade count

• Results compared against full transient simulation: excellent agreement

◦ Integral properties: typical error of 2%

◦ Local solution features: pressure on surface in time

◦ Mode and nature of flow instability

• Results are significantly better than expected!

• Substantial reduction in simulation time:◦ Intel Core i5-3570K, 3.4 GHz computer with 16 GB memory◦ Transient run needs approx. 50 blade passages to become quasi-periodic

Transient HB, 1 h HB, 2 h

Simulation time 5 hrs/rotation 52 mins 78 mins

Iterations 600, dt = 5e-5 s 3000 24001 rotation = 0.03 s




-0.2 -0.15 -0.1 -0.05 0x-Axis

-1200

-1000

-800

-600

-400

-200

0

Pre

ssu

re,

Pa

TransientHB, 1hHB, 2h

-0.15 -0.1 -0.05 0 0.05x-Axis

-1200

-1000

-800

-600

-400

-200

0

Pre

ssure

, P

a


-0.15 -0.1 -0.05 0 0.05x-Axis

-1200

-1000

-800

-600

-400

-200

0

Pre

ssure

, P

a


Transient HB, 1h err, % HB, 2h err, % MRF err, %

Efficiency 89.72 88.80 1.0 89.76 0.0 89.65 0.07t = T

3Head 81.48 81.80 0.4 80.45 1.3 84.12 3.14

Torque 0.0297 0.0302 1.7 0.0294 0.9 0.0308 3.57

Efficiency 89.92 88.78 1.3 89.81 0.1t = 2T

3Head 81.48 81.85 0.4 80.6 1.1

Torque 0.0296 0.0302 2.0 0.0295 0.4

Efficiency 89.83 88.85 1.1 89.71 0.1t = T Head 81.49 81.79 0.4 80.39 1.3

Torque 0.0297 0.0302 1.6 0.0294 1.0


Block Matrix in OpenFOAM

Background

• OpenFOAM uses equation mimicking to perform field algebra and discretisation:perfect for simple PDE-s or segregated solution algorithms

• . . . but sometimes we use equation segregation inappropriately

• There exists a family of problems that cannot be solved efficiently withoutinter-equation coupling; some simulations “that work” can be performed 10-s or100-s of times faster than with equivalent segregated algorithms

Objective

• Implement flexible and efficient block-coupled solution infrastructure

• Re-use all operator-based discretisation schemes, parallelisation and boundarycondition tools already available in OpenFOAM

• Optimise top-level code for efficient execution and ease of assembly

Examples

• Incompressible steady pressure-velocity system (with turbulence)

• Compressible multi-phase free surface simulations: under-water explosions



Block-Coupled Solution Algorithms

• For cases of strong coupling between the components of a vector, the componentscan be solved as a block variable: (ux, uy , uz) will appear as variables in thesame linear system

• In spite of the fact that the system is much larger, the coupling pattern still exists:components of u in cell P may be coupled to other components in the same pointor to vector components in the neighbouring cell

• With this in mind, we can still keep the sparse addressing defined by the mesh: if avariable is a vector, a tensorial diagonal coefficients couples the vectorcomponents in the same cell. A tensorial off-diagonal coefficient couples thecomponents of uP to all components of uN , which covers all possibilities

• For multi-variable block solution like the compressible Navier-Stokes systemabove, the same trick is used: the cell variable consists of (ρ, ρu, ρE) and thecoupling can be coupled by a 5× 5 matrix coefficient

• Important disadvantages of a block coupled system are

◦ Large linear system: several variables are handled together

◦ Different kinds of physics can be present, e.g. the transport-dominatedmomentum equation and elliptic pressure equation. At matrix level, it isimpossible to separate them, which makes the system more difficult to solve



Matrix Connectivity and Mesh Structure

• Irrespective of the level of coupling, the FVM dictates that a cell value will dependonly on values in surrounding cells

PW E

N

S

• We still have freedom to organise the matrix by ordering entries for variouscomponents of the solution variable x

• Global sparseness pattern related to mesh connectivity: easier coefficientassembly



Coupling Coefficient

• Matrix implemented with block coefficients

• Consider general linear dependence between two vectors m and n

m = Ab

• Component-wise coupling describes the case where mx depends only on nx, my

on ny and mz on nz

1. Scalar component-wise coupling

2. Vector component-wise coupling

3. Full (block) coupling

• Explicit methods do not feature here because it is not necessary to express themin terms of matrix coefficients

• For reference, the linear equation for each cell featuring in the matrix reads

APmP +∑

N

ANmN = R


Coupled p-U Solver

Turbulent Steady Incompressible Flows: SIMPLE or Coupled System

• Equation set contains linear p-U and non-linear U-U coupling

∂u

∂t+∇•(uu)−∇• (ν∇u) = −∇p

∇•u = 0

• Traditionally, this equation set is solved using the segregated SIMPLE algorithm

◦ Low memory peak: solution + single scalar matrix in peak storage

◦ p-U coupling is handled explicitly: loss of convergence (under-relaxation)

◦ Number of iterations is substantial; not only due to non-linearity

◦ Convergence dependent on mesh size: SIMPLE slows down on large meshes

• Block-implicit p-U coupled solution

◦ Coupled solution significantly increases matrix size: 4 blocks instead of 1

◦ . . . but the linear p-U coupling is fully implicit!

◦ Iteration sequence only needed to handle the non-linearity in the U-equation

◦ Net result: significant convergence improvement (steady or transient) at acost of increase in memory usage: reasonable performance compromise!


Coupled p-U Solver

SIMPLE-Based Segregated p-U Solver

// Momentum equation assembly and solution

fvVectorMatrix UEqn

(

fvm::div(phi, U)

+ turbulence->divDevReff()

);

UEqn.relax();

solve(UEqn == -fvc::grad(p));

// Pressure equation assembly and solution

U = UEqn().H()/UEqn.A();

phi = fvc::interpolate(U) & mesh.Sf();

fvScalarMatrix pEqn

(

fvm::laplacian(1/UEqn.A(), p) == fvc::div(phi)

);

pEqn.solve();

phi -= pEqn.flux();

p.relax();


Coupled p-U Solver

Block-Coupled u− p System Matrix Structure

u1 u2p1 p2

(

aP (uu) aP (u p)

aP (pu) aP (p p)

) (

aN (uu) aN (u p)

aN (pu) aN (p p)

)

· · ·

.

(

aP (uu) aP (u p)

aP (pu) aP (p p)

)

· · ·

......

. . .

u1

p1

u2

p2...

=

bu 1

bp 1

bu 2

bp 2

...


Coupled p-U Solver

Coupled Implicit p-U Solver: Source Code

fvVectorMatrix UEqn

(

fvm::div(phi, U)

+ turbulence->divDevReff()

);

fvScalarMatrix pEqn

(

- fvm::laplacian(rUAf, p) == -fvc::div(fvc::grad(p))

);

BlockLduSystem<vector, vector> pInU(fvm::grad(p));

BlockLduSystem<vector, scalar> UInp(fvm::UDiv(U));

BlockLduMatrix<vector4> A(mesh);

blockMatrixTools::insertEquation(0, UEqn);

blockMatrixTools::insertEquation(3, pEqn);

blockMatrixTools::insertBlockCoupling(3, 0, UInp, false);

blockMatrixTools::insertBlockCoupling(0, 3, pInU, true);

UpEqn.solve();

UpEqn.retrieveSolution(0, U.internalField());

UpEqn.retrieveSolution(3, p.internalField());


Performance of the Coupled p-U Solver

Performance of the Coupled p-U Solver: Speed and Robustness



Performance of the Coupled p-U Solver: Speed and Robustness



Performance of the Coupled p-U Solver: External Aerodynamics

1e-08

1e-07

1e-06

1e-05

1e-04

1e-03

1e-02

1e-01

1e+00

0 500 1000 1500 2000 2500

Res

idual

Iteration

Ux (simpleFoam)

Uy (simpleFoam)

Uz (simpleFoam)

p (simpleFoam)

BiCGStab Ux

BiCGStab Uy

BiCGStab Uz

BiCGStab p

SAMG Ux

SAMG Uy

SAMG Uz

SAMG p



Performance Improvements and Extensions in the Coupled p-U Solver

• Improvements in performance for the coupled solver: consistency, numerics

• Extension to compressible flows, MRF and porous media (implicit!)

• Major performance jump: block-coupled AMG with additive correction(Hutchinson 1988)

• Block-coupled k − ǫ and k − ω SST turbulence models

◦ Turbulence equations solved in a single block-coupled system

◦ Analysis of source terms to establish favourable cross-equation coupling

◦ Implemented in Diploma Thesis assignment: Robert Keser, Uni Zagreb

• Example: steady (MRF) and transient centrifugal pump



Performance Improvements and Extensions in the Coupled p-U Solver

0 100 200 300 400 500Iterations

0.0001

0.01

1

Init

ial

Res

idu

als

Ux_0 (coupled)

Uy_0 (coupled)

Uz_0 (coupled)

p_0 (coupled)

Ux_0 (segregated)

Uy_0 (segregated)

Uz_0 (segregated)

p_0 (segregated)

Comparison between MRFcoupled and MRFsegregated

initial residuals

0 100 200 300 400 500Iterations

1e-05

0.0001

0.001

0.01

0.1

1

Init

ial

Res

idu

als

k_0 (segregated)

omega_0 (segregated)

k_0 (coupled)

omega_0 (coupled)

Comparison between MRFcoupled and MRFsegregated turbulence

initial residuals

0 100 200 300 400 500Iterations

-100

-80

-60

-40

-20

0

20

40

60

80

100

Hea

d [

m]

Head (segregated)

Head (coupled)

Comparison between MRFcoupled and MRFsegregatedOtaBm1 pump head

0 100 200 300 400 500Iterations

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

1e+05

Po

wer

[W

]

Power (segregated)

Power (coupled)

Comparison between MRFcoupled and MRFsegregatedOtaBm1 pump power

0 100 200 300 400 500Iterations

0

20

40

60

80

100

120

140

Eff

icie

ncy

[%

]

Efficiency (segregated)

Efficiency (coupled)

Comparison between MRFcoupled and MRFsegregatedOtaBm1 pump efficiency


Overset Mesh

Open Source Native Overset Mesh Capability in foam-extend

• We have been involved in several previous Overset projects

• . . . but none reached the maturity or provided flexibility required for ageneral-purpose CFD platform

• Project Objective: Complete rewrite without limitations on release

◦ Native implementation of Overset Mesh technology in FOAM-Extend withoutreference to external libraries

◦ Operator-based support for Finite Volume (FV) discretisation: implicit andexplicit operators (fvc and fvm)

◦ Setup via overset patch type, with minimal impact on existing codecomponents: full re-use of existing physical models

◦ Consistent support for strict mass conservation and second order accuratediscretisation in space and time

• Further requirements

◦ Overset mesh functionality must operate on unstructured polyhedral meshhierarchies

◦ Basic layout for massive parallelisation in domain decomposition mode

◦ Support for dynamic overset mesh simulations


Overset Mesh

Overset Mesh Capability in foam-extend: An Update

• Revisited native implementation of Overset Mesh: public release

• Rewrite of basic Overset functionality to improve performance and parallel scaling

Overview of New Overset Capability

• Explicit and Implicit fringe treatments: chosen per-field, depending on application

• Rewrite of parallelised Overset assembly

• New research item: Implicit conservative Overset

• Additional fringe reduction algorithms

• Rewrite of parallel communications to improve scaling on HPC machines

Publicly available Overset Solver forfoam-extend-4.0


Overset Mesh: Example

Overset Mesh Validation: DTC Propeller Case

• Transient propeller simulation for a model scale propeller for DTC hull (DuisburgTest Case, a container ship)

• Case details◦ Model scale propeller for DTC hull, Diameter, D = 0.15 m

◦ n = 890 RPM (14.8343 RPS)

◦ Advance coefficient (J = v/(n*D) = 0.75)

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65Time; t, s

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Thru

st &

torq

ue

coef

fici

ents

; K

T &

10K

Q

KT, CFD (overset,transient)

KT, EXP

KQ

, CFD (overset, transient)

KQ

, EXP


Overset Mesh: Parallel Scaling

Overset Mesh: Parallel Scaling Test


Overset Mesh: Parallel Scaling

Parallel Efficiency of the Overset Mesh

• Implementation of Overset interpolation performed similar to GGI

◦ Interpolation performed in out-of-core multiplication with parallel comms◦ Parallelised using mapDistribute tool

• Parallel scaling test case

◦ Scaling test performed on 20M cells submarine mesh

◦ Approximately 40K donor/acceptor cells (0.4% of total cell count)

◦ Performed 20 iterations with explicit and implicit Overset fringe

• Parallel speed-up on 64 cores: 41 (implicit) and 46 (explicit)

• Parallel efficiency on 64 cores: 64% (implicit) and 74% (explicit)

1 2 4 8 16 32 64Number of cores, n

1

2

4

8

16

32

64

Par

alle

l sp

eed-u

p, S

Explicit fringe

Implicit fringe

Linear scaling

1 2 4 8 16 32 64Number of cores, n

0

20

40

60

80

100

Par

alle

l ef

fici

ency

, E

, %

Explicit fringe

Implicit fringe


NUMAP-FOAM Summer School

NUMAP-FOAM Summer School 2017

• 12th Edition of NUMAP-FOAM Summer School: 21/Aug-1/Sep/2017https://www.fsb.unizg.hr/numap

The idea of the Summer School is to expand the physical modellingknowledge, numerics and programming skills of attendees usingOpenFOAM in their research through direct supervision and one-to-onework.

This is NOT an introductory OpenFOAM course: significantunderstanding of the project and software is a pre-requisite forapplication.

• The School accepts 10-15 attendees bringing their own projects to the School overa period of 10 working days

• Work is embedded in the research group with 4–6 tutors providing daily one-to-oneattention

• School is open to “young researchers” (typically PhD students) but also toindustrial users, government labs and professors

• Strong follow-up collaboration and extensive publication lists

• Approx 150 attendees to NUMAP-FOAM, from the start in 2008


Summary

Summary

• Ongoing research activity at Uni Zagreb on naval hydrodynamics, basic numericsand turbomachinery CFD

• Actively looking for collaboration partners

Current Work Topics

• Naval hydrodynamics: added resistance in regular and irregular waves, full-scaleship simulation, self-propulsion and manoeuvring, green water and freak waveimpact, modelling of irregular sea states

• Numerics: strongly coupled solution algorithms, Discontinuous Galerkindiscretisation, Overset Mesh and Immersed Boundary

• Turbomachinery: quasi-periodic methods (harmonic balance), LES and instabilitymodelling, implicit pressure- and density-based solvers, turbulence and transition

• Solid mechanics and FSI: coupled non-linear FSI problems

• Optimisation: Gradient-free and adjoint-based methods; uncertainty propagationand robust design


About Me

Hrvoje Jasak

• First degree: mechanical engineering, University of Zagreb, Croatia 1992

• PhD, Imperial College London 1993-1996

• Senior development engineer, CD-adapco (Siemens), 1996-2000

• Technical director, Nabla Ltd. 2000-2004

• Consultant on CFD software, numerics and modelling, ANSYS Fluent 2000-2008

Current Work

• Director, Wikki Ltd: UK-based consultancy company 2004-

• Professor, University of Zagreb, Croatia 2007-

• Mercator Fellow, TU Darmstadt, 2016-

• Various software development and commercial support projects based onOpenFOAM with consultants and large industrial partners

• Coordinating open source OpenFOAM development to allow contributions from thepublic domain developers

• OpenFOAM workshops, lectures and seminars, visiting professorships (TU Delft,Chalmers University and others)


cfd modelling with openfoam -...

Documents