legacy code wrapping yongwei wu tsinghua university

Grid programming with components: an advanced COMPonent platform for an effective invisible grid

© 2006 GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n°034442)

Legacy Code Wrapping

Yongwei WUTsinghua University

outline

Problem statement CharacteristicsObjectiveImplementsFeatures added to the GCMUsageApplicationFuture Work

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 2

Problem Statement

Legacy codes written in Fortran or C/C++ using MPI has to be usable if the objective is to build an effective grid component framework;

In many cases it is not feasible or affordable to rewrite existing code;

Using existing works is most important for IT development;


Objective

Objective:

To develop techniques and methods for wrapping legacy code as GCM components which can run over grid without re-engineering the code, or even access the source files;

Legacy code component will be a module with standardized GCM description, and can be manipulated by tools for composition and deployment;


Characteristics

The general characteristics of legacy code are as follows:

The source code is not available.

The program is poorly documented and the necessary expertise to do any modifications has long left the organization.

The application has to be ported onto the grid within the shortest possible time and smallest effort and cost.

The functionalities are offered to partner organizations but the source is not.


Implements(1/5): Actions

According to our research, grid-enabling legacy code includes the following actions:

For the legacy code, provide some APIs in a standard interface to describe the legacy code attributes, such as the command line format and parameters.

For the related file operations, define some APIs in the interface to transfer the files and set the files’ attributes.

For the resource requirement of the legacy code, include it in the “GCM application description”.

For the running process of the legacy code, define the needed server and client interfaces to manipulate and control the legacy code.


Implements (2/5) : The architecture of the Legacy Code Component

The architecture of the legacy code component


Controller

Content

Java Class：LegacyCodeProxy

Legacy Code

The AttributeController :LegacyComponentAttributes

The server interface for the LegacyComponent:LegacyCodeControllerInterface

File transfer and setting the files attributions

Description of the Legacy code

Implements (3/5) : Description of the Legacy Code

Extending the ADL Not a flexible and extensible solution. First version

Providing some API in a standard Interface Have to create a component with the defined

factory (LegacyComponent class). Set attributes with the AttributeController. Our current solution


Implements (4/5) : Related File Operations

Reuse the file transfer mechanism provided by the ProActive middleware ProActive File Transfer Protocol (PFTP), SSH,

RSH and Nordugrid File Transfer will take place before the

deployment of the component or after the successful execution of the legacy code at the target computing node

Have to set the permissions of the related files


Implements(5/5) : Execution Management

The execution status transition of the legacy code


UNSTARTED

LegacyComponent.newInstance()

RUNNING

KILLED FINISHED

illegalStateException

startLegacyCode()

killLegacyCode() Wait-by-necessity

startLegacyCode()

restartLegacyCode()

killLegacyCode(), restartLegacyCode()

restartLegacyCode()

startLegacyCode() startLegacyCode()

Features added to the GCM(1/5) :API describing the Legacy Code

package org.tsinghua.gcm.legacyComponent.attribute

public interface LegacyComponentAttributes extends AttributeController {

public void setComment (String value);

public void setExecutable (String value);

public void setParameters (String value);

}


Features added to the GCM(2/5) : The Running Process of the Legacy Code

package org.tsinghua.gcm.legacyComponent.legacyCode

public interface LegacyCodeControllerInterface {

public LegacyCodeResult startLegacyCode();

public LegacyCodeResult reStartLegacyCode();

public boolean killLegacyCode();

public String getStatus();

public void setLegacyCodeCommand (String arguments);

}


Features added to the GCM(3/5) : API for Related Files Operation

package org.tsinghua.gcm.legacyComponent.relatedfile

public interface LegacyComponentAttributes extends AttributeController {

//only transfer the file from the local to the remote node where running the legacy code.

public void fileTransfer (File srcFile, File dstFile);

public void fileTransfers (File[] srcFiles, File[] dstFiles);

//set the relatedfile’s permissions, such as the “read”,”write” or “execute”

public void setFilePermission(File file, Permission permission);

public void setFilePermissions(File[] file, Permission[] permission);

public void setFilePermission(String permission);

//delete the no-useful files after running the legacy code successfully

public void setFileDelete(File file, FileDelete delete);

public void setFileDeletes(File[] file, FileDelete[] delete);

public void setFileDeletes(String delete);

}Grid programming with components: an advanced COMPonent platform for an effective invisible grid 13

Features added to the GCM(4/5) : Resource Requirement of the Legacy Code

The definition of a virtual node in a “GCM application descriptor” file with the specification of the following elements: operatingSystem CPUArchitecture CPUSpeed CPUCount memory networkBandwidth diskSpace


Features added to the GCM(5/5) : Wrap the Legacy Code to Component

package org.tsinghua.gcm.legacyComponent

//wrap the legacy code to Component

public Interface LegacyComponent{

public Component newLegacyComponent();

}


Usage

An example implemented By APIAn example implemented By ADL


Usage

BY API


Usage

BY ADL

A common legacy code component ADL


Usage


BY ADLThe base ADL file


Application: Bioinformatics

Belong to Scientific Computing Application This Bioinformatics program can be divided into 3 sub

program{CAP3 ， Tigr (Gene Sequence Assembly Tool) ,Blast (Sequence Alignment Tool)}, and they have natural parallel characteristics.

OutputsInput Files

Cap3

Tigr

Blast

Blast

Application: Bioinformatics 1)Mapping & get Nodes

(2)File transfer

(3)Legacy code ->primitive Component by using API

(4) primitive Component ->Composite Component(CC)

(5)Component deployment

(6) Composite Component1&2 are started parallel

(7)In Node1&3, the component Cap3 in CC 1 and component Tigr in CC2 are working parallel.

(8)When Cap3&Tigr finish their work ,they transfer the output files to node2&4

(9)The two Component Blast on node 2&4 begin to work

(10)Return each result of the CC to the Result Integrate


(9)

(8)

(7)

(9)

(8)

(7)

Composite Component 1

Input Files

Input Files

Node1

Virtual Node


starte(1)

(2)

(3)

Legacy code Cap3

(3) (3) (3)

Legacy code Tigr

Legacy code Blast

Legacy code Blast

(4) (4)

Thred Thred

(2)

(5)(5)

(6) (6)

(10)

Result Integrate


Component Cap3


Component Blast

Component Blast

Component Tigr

Component Cap3


starte

Component Blast

Caps3 output Files

Node2


Input Files

Node3

Component Tigr

Component Blast

Tigr output Files

Node4

(10)

ADLAPI

Deployment XML files

Legacy Code Wrapping: Future work

Future Work Add some APIs ,such as setStopCommandLine(),

setPauseCommandLine() to make good use of the various of commands of legacy code.

Implement the request interface of LegacyComponent so that it can communicate with another component

Execute legacy code asynchronously to increase the efficiency of program


The end

Thanks


legacy code wrapping yongwei wu tsinghua university

Documents

advanced component platform

legacy code attributes

gcm components

source code

deploymentgrid programming

developmentgrid programming

legacy codeextending

gcm application description