parallel programming environment

7/30/2019 Parallel Programming Environment

1/34

ComputerComputer and Automation Research Instituteand Automation Research Institute

Hungarian Academy of SciencesHungarian Academy of Sciences

P-GRADEP-GRADE

VisualVisual

ParallelParallel

Programming EnvironmentProgramming Environment

Robert Lovas

Laboratory of Parallel and Distributed SystemsMTA SZTAKI Research Institute

[email protected]


2/34

P-GRADE :P-GRADE :HighlightsHighlights

P-GRADE is a parallel programming environmentwhich supports the whole life-cycle of parallel

program development For non-specialist programmers it provides a

complete solution for efficient and easy parallel

program development Fast reengineering of sequential programs for parallel

computers

Unified graphical support in program design,debugging and performance analysis

Portability on supercomputers and heterogeneous

workstation/PC clusters based on PVM (and MPI)


3/34

Tools ofTools ofP-GRADEP-GRADE

GRAPNEL: Hybrid Parallel Prog. Language

Graphics to express parallelism

C/C++ to describe sequential parts

GRED: Graphical Editor

GRP2C: Pre-compiler to (C/C++)+(PVM/MPI) DIWIDE: Integrated distributed debugger and

animation system

GRM: distributed monitoring system PROVE: Integrated visualisation tool

TLC Engine: Model checker of Temporal Logic

Specifications --> ongoing project with LINZ/GUP


4/34

Parallel Program Design

GRAPNEL GRED

Mapping

User mapping

GRP file

Pre-compilation

GRP2C

C source code, Cross-ref file, Make file

Building executables

C compiler,

linker

GRP-PVM

GRM Library

PVM Library

GRP-MPI

GRM Library

MPI Library

executables

Trace file

Monitoring

GRM

Visualisation

PROVE

Life-cycle ofLife-cycle of

ParallelParallel

ProgramProgram

DevelopmentDevelopment

and itsand its

support insupport inP-GRADEP-GRADE

GRP file

Debugging

DIWIDE

Model Checking

TLC Engine !!


5/34

DesignDesignGoalsGoals ofofGRAPNELGRAPNEL

Graphical interface

to define all parallel activities

Strong support for hierarchical design Visual abstractions to hide the low level details of

message-passing

C/C++ (or Fortran) to describe sequential parts Strong support for parallelizing sequential

applications

Support for programming in large

No steep learning curve

GRAPNEL = (C/C++) + graphics


6/34

GRAPNEL:GRAPNEL:GRaphical Process NEt LanguageGRaphical Process NEt Language

Programming paradigm: message-passing

component processes run in parallel and can

interact only by means of sending and receivingmessages

Communication model:

point-to-point, synchronous/asynchronous

collective (e.g. multicast, scatter, reduce, etc.)

Process model: single processes

process groups

predefined process communication templates


7/34

Three layers of GRAPNELThree layers of GRAPNEL


8/34

GRAPNELGRAPNEL

Hierarchical design levels:

Graphics used at applicationlevel:

Defines interprocesscommunication topology

Port protocols

Graphics hides PVM/MPIfunction calls

Support for SPMD programming

style

Predefined communicationpatterns

Automatic scaling of parallelprograms


9/34

Communication TemplatesCommunication Templates

Pre-defined regularprocess topologies

process farm

pipeline

2D mesh

tree

User defines:

representative

processes actual size

Automatic scaling


10/34

Mesh

MeshT

emplat

e

T

emplat

e


11/34

TreeT

emplate

TreeTemplate


12/34

Process GroupsProcess Groups

Hierarchical design(subgraphabstraction)

Collective communication(group ports)

multicast

scatter

gather

reduce


13/34

GRAPNELGRAPNEL

Hierarchical design levels:

Graphics used at processinternal level

C/C++ used at the text level

Synch/asynch. comm.

Programming in large: Any C/C++ library call can be

included in text blocks

Graphical support for object-based programming


14/34

GRAPNELGRAPNEL

Structuring facilityby macro graphs


15/34

GRED EditorGRED Editor

Supports the creation of allthe elements of GRAPNEL

Drag-and-drop style ofdrawing

Cut/copy/paste/move ongraphical objects

Automatic port positioningwith minimal lengths andcrossing of communication

channels


16/34

GRED EditorGRED Editor

Extremely easy and fastconstruction of process graph

Automatic arrange of theprocess graph

Automatic resizing of processwindows

Cut/copy/paste on graphicalobjects

Macro graph construction atarbitrarily nested level

C/C++ code can be edited byany standard text editor


17/34

GRP2CGRP2CPrePre--compilercompiler

Automatic generation of PVM and MPI callsbased on GRAPNEL graphics

GRP2CC/C++

graphics

GRAPNEL

Automatic code instrumentation for debuggingand performance monitoring

C/C++

PVM/MPI

Generated code


18/34

Debugging byDebugging by

DIWIDEDIWIDE


19/34


20/34

Performance monitoring and analysisPerformance monitoring and analysis

with PROVEwith PROVE

High-SpeedSwitch

Observing?


21/34

Phases of Performance Visualisation

Source Code

Instrumentation

(GRAPNEL/GRED)

Runtime

Monitoring

(GRM)

Visualisation

(PROVE)

Data

Analysis

(PROVE)


22/34

Source Code

Instrumentation

Source Code

Instrumentation

AutomaticAutomatic Monitoring

modes

Monitoring

modesFilteringFiltering Click-back

facility

Click-back

facility

Selectableprogram

units

Selectableprogram

units

IndividualEvents

IndividualEvents

On/offfacilityOn/offfacility

StatisticsStatistics


23/34

Source code click-backSource code click-back

facility and click-forwardfacility and click-forward

facilityfacility


24/34

Behaviour Window ofBehaviour Window ofPROVEPROVE

Scrolling visualisation windows forward and backwards

User controlled focus on processors, processes and messages

Zooming, event filtering facilities


25/34

PROVEPROVEPerformance analyserPerformance analyser

Various views for displayingperformance information

Synchronised multi-window

visualisation


26/34

PROVEPROVESummarySummaryWindowsWindows

Various views for displayingsummary information

Synchronised multi-window

visualisation


27/34

PROVEPROVEStatisticsStatisticsWindowsWindows

Profiling based on counters Analysis of very long running

programs is enabled


28/34

The GRM MonitorThe GRM Monitor

Off-line monitoring (GRADE) stores trace events in a (local or global) storage and

makes it available after execution for post-mortemprocessing.

Semi-on-line monitoring (P-GRADE)

stores trace events in a storage but makes it available for the visualisation tool any time

during execution if the user asks for it interactive usage of PROVE user can remove already inspected part of the trace evaluation of long-running programs macrostep debugging in P-GRADE with execution

visualisation


29/34

Application-level monitor

Tracing + statistics collection

Semi-on-line

P-GRADE

DIWIDE GRED

GRM

(MM) Trace file

LM LM LM

Host 1 Host 2 Host n

socket file operation

Local host

Server host

Remote cluster

PROVE

Prove-rdd

GRM monitorGRM monitor

MM

LM

proc 1 proc 2

Shared-memorybuffer

Host

socket

memory op.

pipe


30/34

PortabilityPortability

Supported Hardware/Software PlatformsSupported Hardware/Software Platforms

Workstation clusters

SGI MIPS

/ IRIX

5.x/6.x (MTA SZTAKI, Univ. of Vienna) SunUltraSPARC / Solaris 2.x (Univ. of Athens)

Intel x86 / Linux (MTA SZTAKI)

Supercomputers

Hitachi SR2201 / HI-UX/MPP(Polish-Japanese School,Warsaw)

Cray T3E/ UNICOS(Jlich, Germany)

Sun Enterprise 10000 / Solaris 8 (Budapest, Hungary)

International installationsInternational installations


31/34

Current UK Austria

Spain

Portugal

Poland

Germany

Slovakia Greece

Japan

Mexico USA


32/34

Further DevelopmentsFurther Developments

Family of parallel programmingenvironments

P-GRADE VisualMP VisualGrid

- checkpointing

- dynamic load

balancing

- fault tolerance

(ongoing project)

- grid resource

management

- grid monitoring

- mobile processes

ConclusionConclusion


33/34

ConclusionConclusion

Current applications in physics andweather forecast

Download version and further

information:

www.lpds.sztaki.hu


34/34

Thank You ...Thank You ...

?

parallel programming environment

Documents