Parallel Programming in Perl Is MPI still an efficient technique for scripted languages?

Zachary A. Levine1,2 1 MOSIS, Information Sciences Institute, Viterbi School of Engineering (VSoE),

University of Southern California (USC), Marina del Rey, USA 2 Department of Physics and Astronomy, College of Letters, Arts, and Sciences, USC,

Los Angeles, USA

Introduction

Methods Our lab (Gundersen/Vernier) runs molecular dynamics simulations of phospholipid bilayers (cell walls) under the influence of external electric fields. These bilayers tend to electro-permeabilize, and the simulations can output human-readable trajectories of these events. Perl is an excellent language with which to analyze these trajectories due to it’s efficient regular expression matching. As a result, meaningful physical phenomena can be extracted from these trajectories quickly and efficiently with perl. Here we incorporate a perl extension module from the Comprehensive Perl Archive Network (CPAN) called Parallel:MPI by Josh Wilmes and Chris Stevens of Rensellaer Polytechnic Institute into existing perl code. We will observe the performance increase in these routines and determine whether a speedup is possible, and if so, how significant.

Sixteen years ago, message passing interface (MPI) was implemented in C, C++, and Fortran as a groundbreaking yet simplistic way of using multi-core architectures for performing concurrent floating point operations. Since then, many languages have tried to incorporate MPI, but some (e.g. Java) have had difficulties doing so due to inefficiencies in transferring and combining large multidimensional arrays. Recently however, scripting languages such as Python and Perl have tried, with varying success, to do the same. In particular, Perl lacks a compilation step, is difficult to crash, contains simple datatypes, and supports all modern language features, however Perl’s simple yet high-level nature can adversely affect it’s performance. Here we test if a scripted language such as Perl can achieve the same performance increases with MPI as C or Fortran.

Conclusions Implementation of MPI in the Perl programming language is possible using external modules which feed MPI commands to a C interpreter.

Although some critical MPI functions lack efficiency in Perl, MPI can be used to dramatically speed up perl scripts which iterative run on a large number of files (e.g. Molecular Dynamics analysis).

Hardware: HP Elite 360Z Workstation 3.2 GHz AMD Phenom-II Hex-Core Processor (3MB L2 + 6 MB L3) Cache 16 GB DDR3 SDRAM

Software: OS: Ubuntu Linux 11.04 MPI: MPICH2 Version 1.9 Perl: Version 5.10.1 CPAN Module: Parallel:MPI

Results (Part I) For an initial proof of concept of MPI in Perl, we look first at the parallel calculation of π = Σ(4/1+xi

2) averaged over three trials.

The Parallel:MPI module in perl slightly decreases the total runtime, but MPI_Bcast and MPI_Reduce functions are not very efficient.

Results (Part II) Perl also allows us to search thousands of files quickly and data mine, so we can also measure the effect of MPI on pre-existing scripts which can, for instance, measure biological properties such as electropore radii as a function of time if a molecular dynamics trajectory is provided. This way we will use MPI as a way of opening and analyzing multiple files at once, and since the results are independent (and do not need to be recombined) we can avoid inefficient perl functions such as the MPI_Reduce (All_Reduce) routines.

When MPI is used around the original perl script and recombination is not required, speedups are linear and nearly ideal. Similarly, this appears to be very efficient in perl.

use Parallel::MPI qw(:all);

MPI_Init()

$numprocs = MPI_Comm_size(MPI_COMM_WORLD); $myid = MPI_Comm_rank(MPI_COMM_WORLD);

for ($a = $myid; $a <= $max; $aa+= $numprocs) {

Run Original Perl Code

}

MPI_Finalize();

MPI Performance