Network Aware Module

Implementation of the paper: “Forecasting Network Performance to Support Dynamic Scheduling Using the Network Weather Service”.

Its main purpose is to provide the network “Weather” to an interested party. It makes predictions based on past performance.

The 3 phases in NAMThis project was broken up into three stages:

1. Sensing the network performance (Netperf) and recording to RDMS (MySQL).

2. Analyzing the data statistically. (Mean, Median and Bivariate Regression)

3. Providing an interface for any software tool to get this information (Implemented via RMI Server, with a sample client included).

Stage 1 of NamA java-based NAM client calls a netperf executable and the output is parsed, and sent to the NAM server.

/home/murphy/nam/netperf/netperf -H 137.207.234.137Waiting to get data from the input streamTCP STREAM TEST to 137.207.234.137

Recv Send SendSocket Socket Message ElapsedSize Size Size Time Throughputbytes bytes bytes secs. 10^6bits/sec

87380 16384 16384 10.01 92.83

Storing the dataThe NAM client sends the NAM server with all the time information, network performance etc.The NAM server stores this information in a MySQL database.About to insert the following SQL command -->INSERT INTO namStat VALUES ( '2002/04/13, 13:59:55' , 'socr.uwindsor.ca' , 87380 , 16384 , 16384 , 10.01 , 93.30 )

NAM FeaturesSince there are many network variables, you have to fill out a data file which NAM reads when it starts.Either a text editor or the included GUI based tool should do the trick.

Multithreaded Design

The periodicity of the client “pings” to the server can be set at each client.The Server runs on at least 4 threads. MySQL writer, server thread, prediction timer, and RMI Server

Each “active client” is stored in a hash table that the prediction timer periodically scans through and removes out of date objects. This was a new concept, and only the next version of java will have this new “conveyor belt” data structure.

Phase Two: Statistical Analysis

I took three statistical measures: Mean, Median and Regression (Least squares method)

Beyond these three methods, I also have a “Floating K value” where K is the statistical sample size between 10-40

Each client contact initiates an intense self analysis routine that checks all three methods, then three values for each 0+1-1

Example outputsocr.uwindsor.ca Median prediction 0 : 93.065 for tru: 93.42 Difference of 0.35499573socr.uwindsor.ca Median prediction 1 : 92.88 for tru: 93.42 Difference of 0.5400009socr.uwindsor.ca Median prediction 2 : 93.16499 for tru: 93.42 Difference of 0.25500488

socr.uwindsor.ca Mean prediction 0 : 93.496 for tru: 93.42 Difference of 0.07600403socr.uwindsor.ca Mean prediction 1 : 93.506676 for tru: 93.42 Difference of 0.08667755socr.uwindsor.ca Mean prediction 2 : 93.54858 for tru: 93.42 Difference of 0.12857819

socr.uwindsor.ca Regressive prediction 0 : 93.424164 for tru: 93.42 Difference of 0.0041656494socr.uwindsor.ca Regressive prediction 1 : 93.44312 for tru: 93.42 Difference of 0.023124695socr.uwindsor.ca Regressive prediction 2 : 93.530754 for tru: 93.42 Difference of 0.11075592

Conclusive Results:Median_low 2 with value 93.16499 for tru: 93.42 0.25500488Mean_low 0 with value 93.496 for tru: 93.42 0.07600403Regress_low 0 with value 93.424164 for tru: 93.42 0.0041656494

Regression more accurate

Stage 1 & 2 review

Stage 1: Collect data, using Netperf as a black-box tool.

Store data in MySQL, which is client server and can be set to run on remote servers

Stage 2: Process the data that is in the MySQL database.

Dynamically slide the K size up or down based on comparative performance

Award the best statistical technique.

Phase 3 Provide info to others…

RMI will allow any JVM to access this data with relative ease.

Software could be upgraded to use CORBA if C programs wanted access

NAM server has to be started with a security parameterjava –Djava.security.policy=policy Server

Sample RMI Client nam/server/HelloClient.java out put (phase 3 final)[root@linuxaka server]# java -Djava.security.policy=policy HelloClient LAMBTON's Previous throuput: 91.64 Present throuput: 92.43 Predicted throuput: 93.275--------------- linuxaka's Previous throuput: 775.25 Present throuput: 521.56 Predicted throuput: 978.73--------------- socr.uwindsor.ca's Previous throuput: 89.54 Present throuput: 90.77 Predicted throuput: 90.76601---------------This worked out very well[root@linuxaka server]#

Suggestions for Upgrade

Tune Netperf to tread more lightly (Phil Smith)Add NAM Client functionality to detect screen-saver mode and CPU usageImplement an optional CORBA moduleSet up a distributed system with and without NAM… Test for performance gains/losses.

Where can I get NAM?

Presently at my SOCR site:

http://socr.uwindsor.ca/~murphy/nam

**** ** ** **** ****

/**/** /** **** /**/** **/**

/**//** /** **//** /**//** ** /**

/** //** /** ** //** /** //*** /**

/** //**/** ********** /** //* /**

/** //**** /**//////** /** / /**

/** //*** /** /** /** /**

// /// // // // //