problem solving with netsolve

Problem Solving with NetSolve

miller@cs.utk.edumoore@cs.utk.edususan@cs.utk.edu

www.cs.utk.edu/netsolvenetsolve@cs.utk.edu

Michelle Miller,Keith Moore,Susan Blackford,NetSolve group, Innovative Computing Lab,University of Tennessee

NetSolve

• Problem Statement– Software libraries could be easier to install

and use• Locate library• Configure and install library on local machine

– Need access to bigger/different machines

NetSolve

• Solution– Simple, consistent interface to numeric

packages– Ease of use – locate/use already

configured/installed/running solvers through simple procedure call

– Resource sharing (hardware & software) • Greater access to machine resources• New software packages made available simply

NetSolve ArchitectureServer 1blas, petscAgent

Client Proxy

Server 2lapack, mcell

Server 3blas, itpack

Server 4superLU

Server 5aztec, MA28

MatlabClient

NetSolve ArchitectureServer 1blas, petscAgent

Client Proxy

Server 2lapack, mcell

Server 3blas, itpack

Server 4superLU

Server 5aztec, MA28

MatlabClient

netsolve(‘problemX’, A, rhs)

NetSolve ArchitectureServer 1blas, petscAgent

Client Proxy

Server 2lapack, mcell

Server 3blas, itpack

Server 4superLU

Server 5aztec, MA28

MatlabClient

netsolve(‘problemX’, A, rhs)

Servers?

NetSolve ArchitectureServer 1blas, petscAgent

Client Proxy

Server 2lapack, mcell

Server 3blas, itpack

Server 4superLU

Server 5aztec, MA28

MatlabClient

netsolve(‘problemX’, A, rhs)

Servers?

workload

Server1,Server3

NetSolve ArchitectureServer 1blas, petscAgent

Client Proxy

Server 2lapack, mcell

Server 3blas, itpack

Server 4superLU

Server 5aztec, MA28

MatlabClient

problemX, A, rhs

NetSolve ArchitectureServer 1blas, petscAgent

Client Proxy

Server 2lapack, mcell

Server 3blas, itpack

Server 4superLU

Server 5aztec, MA28

MatlabClient

result

Parallelism in NetSolve

• Task Farming– Single request issued that specifies data

partitioning– Data parallel, SPMD support

Task Farming Interface

/*** BEFORE ***/preamble and initializations;

status1 = netslnb(‘iqsort’, size1, array1, sorted1);status2 = netslnb(‘iqsort’, size2, array2, sorted2);

.

.

.status20 = netslnb(‘iqsort’, size20, array20, sorted20);

program continues;

/*** AFTER ***/preamble and initializations;

status_array = netsl_farm(‘iqsort’, “i=0,19”, netsl_int_array(size_array, “$i”),

netsl_ptr_array(input_array, “$i”), netsl_ptr_array(sorted_array, “$i”));

program continues;

Request Sequencing

• Sequence of computations• Data dependency analysis to reduce

extra data transfers in sequence steps• Transmit superset of all input/output

parameters and make persistent near server(s) for duration of sequence execution.

netsl(“command1”, A, B, C);netsl(“command2”, A, C, D);netsl(“command3”, D, E, F);

Client Server

command1(A, B)

result C

Client Server

command2(A, C)

result D

Client Server

command3(D, E)

result F

netsl_begin_sequence( );netsl(“command1”, A, B, C);netsl(“command2”, A, C, D);netsl(“command3”, D, E, F);netsl_end_sequence(C, D);

Client Server

sequence(A, B, E)

Server

Client Serverresult F

input A,intermediate output C

intermediate output D,input E

Data Persistence

NetSolve Applications

• MCell (Salk Institute)– Monte Carlo simulator of cellular

microphysiology – synaptic transmission– Large numbers of same computation with

different parameters (diffusion and chemical reaction calculations)

– Task farming used for parallel runs

NetSolve and Metacomputing Backends

NetSolveClient

NetSolve Servers

NetSolve Agent

NetSolve Services

MDS

GASSGRAM

Client-ProxyInterface

NetSolveProxy

Client-ProxyInterface

GlobusProxy

Client-ProxyInterface

NinfProxy

LegionProxy

NinfServices Legion

Services

Client-Proxy Interface

NetSolve Authenticationwith Kerberos

• Kerberos used to maintain Access Control Lists and manage access to computational resources.

• NetSolve properly handles authorized and non-authorized components together in the same system.

NetSolve Authenticationwith Kerberos

NetSolve client

NetSolve agent

NetSolve servers

KerberosKDC

Typical NetSolve Transaction

Kerberized Interaction

NetSolve Authentication with Kerberos

NetSolve client

NetSolve agent

NetSolve servers

KerberosKDC

Servers registertheir presencewith the agent

and KDC

Client issues problem request;Agent responds with list of servers

Client sends work request to server; server replies requesting

authentication credentials

Client requests ticket from KDC

Client sends ticket and input to server; server authenticates and

returns the solution set

NWS Integration

NetSolveServer CPU

sensor

Host Machine

NetSolveServer CPU

sensor

Host MachineHost Machine

NetSolveAgent

NWSForecaster

NWSMemory

Sensors report to NWS memory.

NWS Integration

NetSolveServer CPU

sensor

Host Machine

NetSolveServer CPU

sensor

Host MachineHost Machine

NetSolveAgent

NWSForecaster

NWSMemory

NWS Integration

NetSolveServer CPU

sensor

Host Machine

NetSolveServer CPU

sensor

Host MachineHost Machine

NetSolveAgent

NWSForecaster

NWSMemory

Agent probes NWS Forecaster

NWS Integration

NetSolveServer CPU

sensor

Host Machine

NetSolveServer CPU

sensor

Host MachineHost Machine

NetSolveAgent

NWSForecaster

NWSMemory

Forecaster probes memory.

NWS Integration

NetSolveServer CPU

sensor

Host Machine

NetSolveServer CPU

sensor

Host MachineHost Machine

NetSolveAgent

NWSForecaster

NWSMemory

Forecaster makes forecast

Agent chooses server

NWS Integration

NetSolveServer CPU

sensor

Host Machine

NetSolveServer CPU

sensor

Host MachineHost Machine

NetSolveAgent

NWSForecaster

NWSMemory

Newly enabled libraries

PETSc

SuperLU

SPOOLES

MA28

Sparse Matrices/Solvers• Support for compressed row/column sparse

matrix storage -- significantly reduces network data transmission.

• Iterative and direct solvers: PETSc, Aztec, SuperLU, Ma28, …

• All available solver packages will be made available from UTK NetSolve servers and others.

Matlab interface

• Calls to PETSc, Aztec– [x, its] = netsolve(‘iterative_solve_parallel’,‘PETSC’,A,b,1.e-6,500);– [x, its] = netsolve(‘iterative_solve_parallel’, ‘AZTEC’,A,b,1.e-6,500);

• Similar for SuperLU, MA28– [x] = netsolve(‘direct_solve_serial’,’SUPERLU’,A,b,0.3,1);– [x] = netsolve(‘direct_solve_serial’, ‘MA28’,A,b,0.3,1);

• Calls to LAPACK, ScaLAPACK– [lu, p, x, info] = netsolve(‘dgesv’,A,b);– [lu,p,x,info] = netsolve(‘pdgesv’,A,b);

‘LinearSolve’ interface

• Uncertain which library to choose?• ‘LinearSolve’ interface chooses the library

and appropriate routine for the user– [x] = netsolve(‘LinearSolve’,A,b);

Heuristics

• Interface analyzes the matrix A– matrix shape (square, rectangular)?

• If rectangular, choose linear least squares solver from LAPACK

– matrix element density• If square, is the matrix sparse or dense?• Manually check the percentage of nonzeros and

transform to sparse matrix format• If dense, is it symmetric?

Heuristics cont’d

• If sparse, and dense band, use LAPACK• If sparse, and if there is a block or diagonal

structure (Aztec), can yield higher performance (level 3 BLAS)

• If sparse, direct or iterative solver?– Size of the matrix (large matrix, fill-in for a

direct method can be larger than iterative)

Heuristics cont’d

– Numerical properties (direct solvers can handle more complicated matrices than iterative methods)

• How to estimate fill-in and gauge numerical properties of A?

• Future Work: ‘Eigensolve” interface

Interfacing to Parallel Libraries

• Improved task scheduling and load balance– NWS memory sensor, latency and bandwidth

sensors– provide matrix distributions for the user– heuristics for the best choice of the # of

processors, amount of matrix per processor, process grid dimension ...

Get NetSolve1.3 Now!

• Release date -- April 2000!• UNIX client/agent/server source code.• UNIX client binaries available.• Win32 dlls for C/Matlab/Mathematica

clients.

www.cs.utk.edu/netsolve