illuminating openmp + mpi performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97...

of 14 /14

Illuminating OpenMP + MPI Performance Beau Paisley, [email protected]

Upload: others

Post on 02-Oct-2020

36 views

Category:

Documents

0 download

Report

Download

Embed Size (px)

TRANSCRIPT

Page 1: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Illuminating OpenMP + MPI Performance

Beau Paisley, [email protected]

Page 2: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

HPC means being able to work productively on remote machines

Page 3: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Calculating pi with 1 thread, 1 process

Page 4: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Profile with 1 process, 1 thread

Page 5: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Using MAP to analyze our 1 process, 1 thread run

Page 6: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Zooming in on the IO portion of our run

Page 7: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Always be thinking about vectorization

Page 8: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

While still connected to the server we can switch to the debugger

Page 9: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Adding some OpenMP pragmas to multithread

Page 10: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Adding MPI support and submitting to and HPC batch system

Page 11: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Analyzing our OpenMP + MPI results

Page 12: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Why all this synchronization?

Page 13: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

The workflow of pi from serial to HPC

caption

Page 14: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

https://www.allinea.com/products/downloads/free-trial

MPI and OpenMP - Archer · • Both OpenMP and MPI are themselves highly portable (but not perfect). • Combined MPI/OpenMP is less so • main issue is thread safety of MPI •

Introduction to MPI, OpenMP, Threads

Hybrid MPI/OpenMP Power-Aware Computing

Hybrid Programming with MPI and OpenMP B. Estradeestrabd/intro-hybrid-mpi-openmp.pdf · not as straightforward as retro-fitting an MPI application with OpenMP because global program

Introduction to MPI and OpenMP - Electrical & Computer ...courses/ee8218/mpi_openmp.pdf · Introduction to MPI and OpenMP myson @ postech.ac.kr ... GCC and MPICH2 for MPI GCC-4.2

Hybrid Programming: MPI+OpenMP - HPC-Forge · 2013-07-22 · MPI+OpenMP hybrid paradigm is the trend for clusters with SMP architecture. Elegant in concept: use OpenMP within the

Hybrid Programming with OpenMP and MPI

OpenMP Basics and MPI/OpenMP Scaling - NERSC

Hybrid OpenMP and MPI Programming and Tuning

チュートリアル: MPI/OpenMP* ハイブリッド・アプリケーション … · 2018-03-23 · チュートリアル: MPI/OpenMP* ハイブリッド・アプリケーションの解析

MPI and OpenMP · •In general, MPI + OpenMP does not improve performance (and may be worse!) in the regime where the MPI application is scaling well. •Benefits come when MPI scalability

Parallel Programming with MPI and OpenMP

Programming with OpenMP and mixed MPI-OpenMP

Parallel Programming in C/C++ - OpenMP versus MPI MPI ...gibson/Teaching/CSC5021/L4-Parallel... · Note: OpenMP and MPI are oftencombined-in a hybridsystem-to provide parallelismatdifferentlevelsof

Splotch: High Performance Visualization using MPI, OpenMP ... · Splotch: High Performance Visualization using MPI, OpenMP and CUDA Klaus Dolag (Munich University Observatory)

MPI/OpenMP Hybrid Parallelism for Multi-core Processors · Historical usage of MPI/OpenMP • MPI-OpenMP hybrid programming is not new • It was used to parallelise some codes on

Programmation hybride MPI-OpenMP - IDRIS · Programmation hybride MPI-OpenMP [email protected] P[email protected] CNRS — IDRIS / LA Version 2.3 —

Performance Characteristics of Hybrid MPI/OpenMP ...faculty.cs.tamu.edu/wuxf/papers/IJNDC2013.pdf · Performance Characteristics of Hybrid MPI/OpenMP Scientific Applications on a

Hybrid MPI-OpenMP versus MPI Implementations: A Case Study · Hybrid MPI-OpenMP versus MPI Implementations: A Ca se Study Osvaldo Mangual +, Marvi Teixeira *, Reynaldo Lopez #, Felix

Hybrid MPI+OpenMP Parallel MD

E5 Pthread OpenMP MPI Molinas Luque

Hybrid MPI+OpenMP Parallel MDcacs.usc.edu/education/cs653/HMD.pdf · Hybrid MPI+OpenMP Parallel MD Aiichiro Nakano Collaboratory for Advanced Computing & Simulations Department of

Insieme-an Optimization System for OpenMP, MPI and ...invasic.informatik.uni-erlangen.de/publications/Moritsch.pdf · Insieme-an Optimization System for OpenMP, MPI and OpenCLPrograms

MPI n OpenMP

Tutorial Exercise NPB-MPI/BT - Prace Training Portal: Events · 2018-07-10 · NPB-BT Intermediate-level tutorial example Available in MPI, OpenMP, hybrid OpenMP/MPI variants also

Introduction to OpenMP. OpenMP Introduction Credits: allans/cs260/lectures/OpenMP.ppt douglas/Classes/cs521-s02/...openmp/MPI-OpenMP.ppt

HPC course on MPI, PETSC, and OpenMP

MPI and OpenMP - archer.ac.uk · Styles of MPI + OpenMP programming • Can identify 4 different styles of MPI + OpenMP programming, depending on when/how OpenMP threads are permitted

Hybrid programming MPI+OpenMP · MPI+OpenMP hybrid paradigm is the trend for clusters with SMP architecture. Elegant in concept: use OpenMP within the node and MPI between nodes,

Performance Analysis of Large-scale OpenMP and Hybrid MPI ... · Performance Analysis of Large-scale OpenMP and Hybrid MPI/OpenMP Applications with VampirNG Holger Brunst1 and Bernd

Performance Analysis of Large-scale OpenMP and Hybrid MPI/OpenMP … · 2005-06-07 · Performance Analysis of Large-scale OpenMP and Hybrid MPI/OpenMP Applications with VampirNG

Combining MPI and OpenMP

Hybrid OpenMP and MPI Programming - SHARCNETjemmyhu/tutorials/MPI+OpenMP... · 2016-07-20 · • From sequential code, parallel with MPI first, then try to add OpenMP. • From MPI

Developing Hybrid OpenMP-MPI Parallelism for Fluidity ... · Developing Hybrid OpenMP-MPI Parallelism for Fluidity - Next Generation Geophysical Fluid Modelling Technology Xiaohu

1 Parallel Programming With OpenMP. 2 Contents Overview of Parallel Programming & OpenMP Difference between OpenMP & MPI OpenMP Programming Model