ParCFD’2020

32nd International Conference on Parallel Computational Fluid Dynamics

May-11-13 2020, Nice FRANCE

IN-SITU DATA ANALYSIS AND VISUALIZATION OF DNSCOMBUSTION SIMULATIONS WITH PARAVIEW

CATALYST

Minh Bau Luong∗, Glendon Holst†, Francisco E. Hernandez Perez∗,

Hong G. Im∗

∗ Clean Combustion Research Center (CCRC)4700 King Abdullah University of Science and Technology (KAUST)

Thuwal 23955-6900, Kingdom of Saudi Arabiae-mail: minhbau.luong@kaust.edu.sa

† KAUST Visualization Core Lab (KVL)email: glendon.holst@kaust.edu.sa

Key words: ParaView Catalyst, In-situ visualization, direct numerical simulation (DNS),superknock, deflagration to detonation transition (DDT).

Abstract. ParaView Catalyst is adapted to our DNS solver named KARFS to performin-situ data analysis and visualization for combustion simulations to overcome the IObottleneck–the gap between IO and computing performance of the high-performance het-erogeneous computing architectures. The in-situ integration of modeling, simulation, anal-ysis, and visualization components in KARFS to capture abnormal combustion events,such as super-knock, will be presented.

Towards achieving exascale direct numerical simulation (DNS) of reacting flows, a DNSsolver named KARFS (KAUST Adaptive Reacting Flows Solver) has been developed withstate-of-the-art programming models for the next-generation high-performance heteroge-neous computing architectures of many-core and graphics processing unit (GPU) proces-sors [1]. KARFS is developed in C++ using modern parallel programming patterns, dis-tributed memory parallelism through message passing interface (MPI), and portable on-node parallelism with the Kokkos C++ programming model. KARFS has performance-portable capabilities for multi- and many-core heterogeneous platforms, which is instru-mental to meet emerging demands of exascale computing machines. The employed Kokkosframework allows KARFS to straightforwardly utilize the same source code on CPUs andGPUs while maintaining performance comparable to the code that is optimally tuned toeither processing unit.

The conventional workflow of simulations performs the computations first and analyzesthe output data later. This workflow does not scale on the exascale high-performancecomputing architectures. Exascale computing features a widening gap between computepower and available input/output (I/O) rates that will impose challenges to save data ata high enough temporal frequency for post-processing. For example, a high frequency ofcheckpoint data is required to capture extreme short-time-scale detonation events that

1

M.B. Luong, G. Holst, F.E. Hernandez Perez, and H.G. Im

places significant stress on the I/O storage system and generates a massive amount ofdata. In the exascale era, this I/O constraint will be further exacerbated by the diffi-culty of moving large volumes of data through deep, complex memory hierarchies andacross the machine network to hard disks on a heterogeneous supercomputer As such, anew concurrent workflow design, including in-situ massively parallel visualization coupledwith analytics, is needed. The in-situ integration between modeling, simulation, analy-sis, and visualization components must be performed concurrently rather than workingindependently as they have in the past.

First-principle direct numerical simulation allows unraveling the complex interplay be-tween turbulence and chemical reactions to provide better understanding of the mecha-nism of detonation development encountered in modern combustion devices under extremehigh-load operating conditions, and to develop a reliable predictive model for real-worldindustrial applications. However, large-scale turbulence, combustion and detonation sim-ulations pose a significant challenge in terms of a wide spectrum of length and time scales,which requires extremely fine spatial and temporal resolutions, and highly intermittentlocalized phenomena, resulting in an insensitive checkpoint output data and extensivecomputational resources. Multidimensional simulations of super-knock phenomena incombustion devices face such highly intensive I/O and computational resource require-ment [2, 3].

Downsized and boosted technology is being a major development trend for advancedengines. The engines become more compact with a higher power density per volume, andthus provide higher thermal efficiency and cleaner combustion processes that reduce green-house CO2 gas as well as NOx and soot emissions. However, the elevated pressure andtemperature of the in-cylinder fuel/air mixture under the high-load operating conditionsmay induce undesired pre-ignition, knock, and even super-knock phenomena [4]. Suchabnormal combustion phenomena are also encountered in shock tubes, rapid combustionmachines, and gas turbine engines [4].

Super-knock is featured by excessive pressure oscillations and extremely high-pressurespikes that may lead to mechanical failure [4]. The fundamental understanding of thedeveloping super-knock mechanism and a reliable criterion to predict super-knock areneeded to prevent destructive operation of combustion devices [5, 6, 7]. High fidelitydirect numerical simulations with the capability of fully resolving all temporal and spatialscales and the complex interaction of thermochemistry and turbulence will help addressthe knocking issues [2, 3]. With the aid of ParaView Catalyst, coupled with our DNSsolver, an integrated framework has been set up and deployed, which allows us to capture,visualize and analyze in-situ the highly localized detonation events occurring at sub-nanosecond timescales, as shown representatively in Fig. 1 and Fig. 2.

The homogeneous ignition delay time, τ 0ig, and equilibrium pressure, Pe, of the ethanol/airmixture at the initial conditions, temperature T0 of 1200 K, pressure P0 of 35 atm, andequivalence ratio φ0 of 1.0 are 75 µs, and 99 atm, respectively. Other relevant idealone-dimensional detonation parameters associated with this initial condition are the vonNeumann pressure, PV N of 315 atm, Chapman–Jouguet pressure, PCJ of 185 atm, and

2

M.B. Luong, G. Holst, F.E. Hernandez Perez, and H.G. Im

Figure 1: Instantaneous isocontours of heat release rate (top row), temperature (middlerow), and pressure (bottom row) for a case labeled as G1

5 (the most energetic length scaleof temperature and turbulent field, lT of of 5 mm and le of 1 mm, turbulent velocityfluctuation u′ of 83.3 m/s, and the ratio of the ignition delay time to turbulent timescale τig/τt of 5.0) with increasing time from left to right [3].

T(K) �

1235

1230 24 1225 20

18 1220

1215

16 1210

1205 14

1200 12

10 1195

1190

1185 8

1180 6 1175 4

2 1170

1165

1160 1

Figure 2: The joint probability density function (PDF) of the normalized pressure-specificvolume (in log scale) [2].

3

M.B. Luong, G. Holst, F.E. Hernandez Perez, and H.G. Im

Chapman–Jouguet speed, VCJ of 1836 m/s. Further details will be provided in the pre-sentation.

Acknowledgments

This work was sponsored by King Abdullah University of Science and Technology(KAUST) and used the computational resources of the KAUST Supercomputing Lab(KSL).

REFERENCES

[1] F. E. Hernandez Perez, N. Mukhadiyev, X. Xu, A. Sow, B. J. Lee, R. Sankaran,H. G. Im, Direct numerical simulations of reacting flows with detailed chemistry usingmany-core/GPU acceleration, Computers & Fluids 173 (2018) 73–79 (2018).

[2] M. B. Luong, S. Desai, F. E. Hernandez Perez, R. Sankaran, B. Johansson, H. Im,A statistical analysis of developing knock intensity in a mixture with temperatureinhomogeneities, Submitted to Proc. Combust. Inst. (2019).

[3] M. B. Luong, S. Desai, F. E. Hernandez Perez, R. Sankaran, B. Johansson, H. Im,Effects of turbulence and temperature fluctuations on knock development in anethanol/air mixture, Submitted to Flow Turbul. Combust. (2019).

[4] Z. Wang, H. Liu, R. D. Reitz, Knocking combustion in spark-ignition engines, Prog.Energy Combust. Sci. 61 (2017) 78–112 (2017).

[5] N. Peters, B. Kerschgens, G. Paczko, Super-knock prediction using a refined theoryof turbulence, SAE Int. J. Engines 6 (2013) 953–967 (2013).

[6] K. Grogan, M. Ihme, Identification of governing physical processes of irregular com-bustion through machine learning, Shock Waves (2018) 1–14 (2018). doi:10.1007/

s00193-018-0852-y.

[7] C. A. Z. Towery, A. Y. Poludnenko, P. E. Hamlington, Detonation initiation by com-pressible turbulence thermodynamic fluctuations, Combust. Flame 213 (2020) 172–183(2020).

4