ugc tutorial octave jchaves 062406 v2

Distribution Statement A: Cleared for Public Release. Distribution is unlimited. 1

Non-traditional High-Level Languages for HPC Applications:

GNU Octave

Juan Carlos Chaves, Ph.D. SIP On-site at ARL MSRC

Ohio Supercomputer Center

Users Group Conference (UGC2006)26 Jun 2006


Purpose of This Tutorial• Provide an introduction to the Octave system

• Teach how to obtain and configure Octave for your own PC

• Teach how to configure, build and install Octave at the HPCMP HPC platforms

• Teach how to exploit Octave HPC technology at the HPCMP Centers

• Be a concise guide to relevant Octave info on the Web


What is Octave?• High-level language, primarily intended for

numerical computations initiated and maintained by John W. Eaton (University of Wisconsin)

• A stated goal of Octave is to be compatible with MATLAB

• Provides a convenient command line interactive interface for solving scientific and technical problems numerically

• Has built-in a large collection of mathematical algorithms and convenience functions


Typical uses of Octave• Prototyping: an interactive Octave session becomes a

system-prototyping environment rivaling systems such as the well-known MATLAB

• Programming language: a powerful but easy to use programming language, mostly compatible with MATLAB, for use in developing sophisticated programs and algorithms

• Batch-oriented HPC applications: Octave and its parallel extensions may also be run under a batch system such as LSF, the mandate queuing system at the DoD HPCMP Centers


Octave Advantages

• Free program available under the GNU General Public License (GNU GPL) and runs on Windows, Linux and Mac OS X including architectures unsupported by MATLAB

• MATLAB clone: syntax is very similar to MATLAB. Careful programming will allow a script to run on both Octave and MATLAB

• Open source community: large and very active support community all over the world

• High productivity system: interactive - code development proceeds incrementally. Excellent development and rapid prototyping environment


Octave Disadvantages• Relatively complex installation procedure (But

fortunately, you’re taking this tutorial!)

• Unlike MATLAB, it does not support object-oriented programming

• Unlike MATLAB supports only basic graphic capabilities, limiting the application of Octave in scientific and engineering graphics, including the creation of GUIs.

• Unlike MATLAB, lacks support from a multi-million dollarcompany such as the MathWorks, Inc.

• Unlike MATLAB, fewer toolboxes are available


Octave – basic features• No declarations needed:octave:1> 14 + 28ans = 42

• Mixed types:octave:2> product = 16*16.34 product = 261.44

• Semicolons suppress output:octave:5> product = 16*16.99;octave:6> productproduct = 271.84


Octave – Matrix notation• Working with matrices:octave:7> a = [ 1, 1, 2; 3, 5, 8; 13, 21, 34 ]a =

1 1 23 5 813 21 34

• Row vectors and the colon operator :octave:8> row_vector = 1:5row_vector =

1 2 3 4 5


Octave – Matrix Subscripting• Working with subscripts:octave:13> b = rand (3, 3);octave:14> b(3,1), b(3,2), b(3,3)ans = 0.27638ans = 0.94927ans = 0.36208

octave:15> b(3,2) = 1;octave:16> bb =0.46079 0.31457 0.536000.55636 0.22555 0.119400.27638 1.00000 0.36208


Octave – Multidimensional Arrays• Example of a 3D random array filled with normal data:octave:20> r = randn(2,3,4)r =ans(:,:,1) =-1.05765 -0.16023 0.722080.48319 0.84360 0.20065

ans(:,:,2) =-0.31952 -0.79753 1.012040.53750 -0.94572 -0.64897

ans(:,:,3) =-1.62063 1.80322 0.592280.42084 1.21434 2.09797

ans(:,:,4) =-1.724684 0.537532 -1.0945330.786163 -0.568425 -0.043409


Octave – Matrix Operations• Example of a Hilbert matrix and its inverse:octave:29> H = hilb(4)H=1.00000 0.50000 0.33333 0.250000.50000 0.33333 0.25000 0.200000.33333 0.25000 0.20000 0.166670.25000 0.20000 0.16667 0.14286

octave:30> H_inv = inv(H)H_inv =16.000 -120.000 240.000 -140.000-120.000 1200.000 -2700.000 1680.000240.000 -2700.000 6480.000 -4200.000-140.000 1680.000 -4200.000 2800.000

octave:31> H*H_invans =1.0000e+00 5.6843e-14 -2.2737e-13 1.1369e-131.6209e-14 1.0000e+00 -1.6032e-13 1.4477e-131.3137e-14 3.4139e-15 1.0000e+00 3.0947e-149.7387e-15 1.9165e-14 -1.1285e-13 1.0000e+00


Octave – Dot Operator• Another example of a Hilbert matrix and its inverse:H =1.00000 0.50000 0.33333 0.250000.50000 0.33333 0.25000 0.200000.33333 0.25000 0.20000 0.166670.25000 0.20000 0.16667 0.14286

octave:33> H_inv = invhilb(4)H_inv =

16 -120 240 -140-120 1200 -2700 1680240 -2700 6480 -4200-140 1680 -4200 2800

octave:34> H.*H_invans =

16 -60 80 -35-60 400 -675 33680 -675 1296 -700

-35 336 -700 400


Octave – Matrix manipulation • Is at the heart of Octave including addition, subtraction, multiplication (matrix

and component-wise), division, transposition, etc.

• Has a number of functions for generating common matrices including:octave:35> eye(3)ans =1 0 00 1 00 0 1

octave:36> ones(3)ans =1 1 11 1 11 1 1

octave:37> zeros(3)ans =0 0 00 0 00 0 0


Octave – Groups of Specialized Functions • Including:- input and output; - plotting; - matrix manipulation; - linear algebra; - non-linear equations; - differential equations; - optimization; - statistics; - financial functions; - sets; - polynomial manipulations; - control theory; - signal processing and image processing;


Octave – Getting help • Getting help from Octave is very easy. For example:

octave:6> help svd

– Will provide extensive help for the Singular Value DecompositionOctave routine, including relevant examples.

• Another useful command is help -i <topic>, which will search the Octave manual index. For example,

octave:7> help -i svd

– Will search for svd in the function index.


Octave – Useful Web Links• 375 page manual available on-line at Octave's home site at:

http://www.octave.org/doc/index.html

• The Octave Wiki: user-contributed advice, examples, tips and tricks at:

http://wiki.octave.org/

• Octave Forge: GNU Octave Repository is a central location for the collaborative development of toolboxes for GNU Octave:

http://octave.sourceforge.net/

http://www.octave.org/doc/index.html

http://wiki.octave.org/

http://octave.sourceforge.net/


Octave – Useful Web Links (2)• The Octave-forge toolboxes:http://octave.sourceforge.net/index/index.html

• The Octave Script Archive at: http://wiki.octave.org/wiki.pl?CategoryScriptArchive

• On-line access to Octave (up to 4 minutes of run time allowed) :

http://www.online-utility.org/math/math_calculator.jsp

http://octave.sourceforge.net/index/index.html

http://wiki.octave.org/wiki.pl?CategoryScriptArchive

http://www.online-utility.org/math/math_calculator.jsp


Getting and Configuring Octave for the PC• 1) The Octave under Cygwin Solution

Download Octave + the Octave-forge extensions (toolboxes) under Cygwin, a Linux-like environment for Windows which consists of two parts:

- A Linux API emulation layer (cygwin1.dll DLL) for Linux API functionality.

- A collection of tools, which provide Linux look and feel.

- Available at:http://www.cygwin.com/

http://www.cygwin.com/


Point and Click PC Octave Installation


Octave on the Windows PC

Linux like environment

Octave-forge toolboxes include


Getting and Configuring Octave for the PC2) The Octave under Quantian Solution

• Quantian is a remastering of Knoppix, a self-configuring and directly bootable CD/DVD that turns any pc or laptop (provided it can boot from cdrom/dvd) into a full-featured Linux workstation.

• Quantian is tailored to numerical and quantitative analysis.

• http://dirk.eddelbuettel.com/quantian.html

• Contains ready to use Octave, with add-on packages octave-forge, octave-sp, octave-epstk, matwrap and Inline-Octave.

http://www.knopper.net/knoppix/index-en.html

http://dirk.eddelbuettel.com/quantian.html


Octave under The Quantian Scientific Computing Environment

Get the latest ISO image and burn it to a blank DVD: http://quantian.fhcrc.org/Quantian_0.7.9.2.iso

http://quantian.fhcrc.org/Quantian_0.7.9.2.iso


Installing Octave at the HPCMP systems• Prerequisites:1) Download Octave (version 2.1.72)ftp://ftp.octave.org/pub/octave/bleeding-edge/octave-

2.1.72.tar.gz

2) Download Octave-Forge extensions that go with Octave 2.1.72

http://sourceforge.net/project/showfiles.php?group_id=2888

3) Download ImageMagickftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.2.6-7.tar.gz

4) Download the jpeg library (version 6b) if not present:http://www.ijg.org/files/jpegsrc.v6b.tar.gz

ftp://ftp.octave.org/pub/octave/bleeding-edge/octave-2.1.72.tar.gz

ftp://ftp.octave.org/pub/octave/bleeding-edge/octave-2.1.72.tar.gz

http://sourceforge.net/project/showfiles.php?group_id=2888

ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.2.6-7.tar.gz

ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.2.6-7.tar.gz

http://www.ijg.org/files/jpegsrc.v6b.tar.gz


Octave – Installation• Step # 1: Octave 2.1.72 install:

> gzip -dc octave-2.1.72.tar.gz | tar -xvf -> cd octave-2.1.72> /configure --prefix=$HOME/octave --enable-dld --enable-

shared

• Note that the “--enable-shared” option must be activatedin the “./configure” script for the Octave-Forge extensions to work.

> make> make install

• Octave building takes a long time. So grab a cup of coffee and relax!


Install the Octave-forge toolboxes


Octave-Forge – Installation • Installing Octave-Forge extensions that go with Octave

2.1.72

> setenv PATH $HOME/octave/bin:$PATH> gzip -dc octave-forge-2006.03.17.tar.gz | tar –xvf –> cd octave-forge-2006.03.17> ./configure --with-path=$HOME/octave-forge --

prefix=$HOME> make -k> make check> make icheck> make install


Octave-Forge – Configuration• Configuring Octave-Forge extensions:

Edit the file:

$HOME/octave/share/octave/2.1.72/m/startup/octaverc

and include the following lines:

LOADPATH = [ getenv("HOME"), "/octave-forge//:", LOADPATH ];EXEC_PATH = [ getenv("HOME"), "/octave-forge/bin:", EXEC_PATH ];


Octave-Forge – Required Libraries• The Octave-Forge function “$HOME/octave-

forge/image/imwrite.m” requires the ImageMagick"convert" utility:

> gzip -dc ImageMagick-6.2.6 | tar –xvf –> cd ImageMagick-6.2.6> ./configure --prefix=$HOME/ImageMagick> make> make install> setenv PATH $HOME/ImageMagick/bin:$PATH

• Note: ImageMagick needs the jpeg library. If this library is not installed on your system then install the jpeg library (version 6b):




Running Octave at the HPCMP systems• 1) Running executable Octave scripts to solve

cheerfully parallel programs

• 2) Use MatlabMPI with Octave. This customization of MatlabMPI is sometimes called OctaveMPI

• Use bcMPI a full MPI interface for Octave to create complex parallel programs (coming soon from OSC!)


Octave - Cheerfully Parallel Programs• Executable Octave script files (like executable shell

scripts)

• Excellent feature of Octave very useful for large cheerfully parallel problems or where a mixture of programs, tools or applications may be needed to solve the problem

• A Octave code my_code.m can be easily converted to an executable Octave program my_executable_script.sh by adding a single line to the beginning of the Octave m-file

#! /usr/bin/octave –qf

• For example at the ARL MSRC JVN Cluster:#! /usr/cta/unsupported/octave/2.1.72/bin/octave/octave -

qf


Octave - Cheerfully Parallel Programs

• To make the script executable and run it at JVN input:

jvn-l1> chmod u+x my_executable_script.shjvn-l1> ./my_executable_script.sh

• Then, it is possible to use LSF queuing system at the HPCMP Centers to schedule large batch jobs.

• The SIP team has advanced tools to facilitate this process: The SIP Productivity Enhancement Tool


OctaveMPI: MatlabMPI under Octave

• OctaveMPI is the name usually given to customization (port?) of MatlabMPI to Octave

• For example, this straightforward tailoring has been done by our collaborators at Indiana University (Professor Arun Chauhan):

www.cs.indiana.edu/~achauhan/publications/ScriptingLangs/IU-TR2006-631.pdf

• We have run OctaveMPI programs at the Seafarer Linux Cluster (SSC – San Diego)

• This tutorial will teach you how to do it!

http://www.cs.indiana.edu/~achauhan/publications/ScriptingLangs/IU-TR2006-631.pdf


OctaveMPI Performance

• Experiments performed under controlled conditions using a 8 node Linux cluster at Indiana University (Intel Xeon 2.5 GHz)

• Interconnect: Gigabit Ethernet

• NFS optimized for MatlabMPI

• Ref: http://www.cs.indiana.edu/~achauhan/publications/ScriptingLangs/abs-iu-tr2006-631.html

Comparing OctaveMPI, MatlabMPI, MPI-TB for MATLAB and C/MPI

http://www.cs.indiana.edu/~achauhan/publications/ScriptingLangs/abs-iu-tr2006-631.html




MatlabMPI Configuration for Octave• Customize MatMPI_Comm_settings.m at /MatlabMPI/src/

% Set location of matlab on unix systems.% Generic location. %matlab_location = ' matlab '; % [OK TO CHANGE.]matlab_location = ' octave '; % [OK TO CHANGE.]

% Hard code location of matlab on remote unix systems.matlab_location = ' /usr/bin/octave';

% Build unix matlab launch command based. [DON'T CHANGE]%machine_db_settings.matlab_command = [matlab_location '-display null -nojvm -nosplash'];

machine_db_settings.matlab_command = [matlab_location ' --no-history --no-line-editing '];


MatlabMPI Configuration for Octave (2)• Fix the Octave special character “\” string problem. For

example at file MPI_Bcast.m:link_script = ['MatMPI\Link_Commands_t' num2str(tag) '.bat'];

Octave line must read:link_script = ['MatMPI\\Link_Commands_t' num2str(tag) '.bat'];

Explanation: in Octave the backslash character ‘\’ starts the escape sequence character:

MATLAB: > backslash = ‘\’Octave: > backslash = ‘\\’

• Disable (comment out) the pc/unix translation of “pwd”implemented at MatMPI_dir_map.m


diff between MatlabMPI and OctaveMPI


MatlabMPI Configuration for Octave (3)• To avoid Octave warnings, for example at MPI_Recv.m,

change the open mode to binary:temp_fid = fopen(lock_file,'r');

For Octave change to:temp_fid = fopen(lock_file,'rb');

• In each directory where you run OctaveMPI code include the dot file “.octaverc” containing the location of OctaveMPI source directory. For example:

addpath('/mnt/sda1/OctaveMPI/src')


OctaveMPI Live Demo

• A live demo of the Octave/OctaveMPIparallel capabilities using the QuantianScientific Computing Environment(live bootable Linux DVD) running a image processing application!


Acknowledgement

“This publication was made possible through support provided by DoD HPCMP PET activities through Mississippi State University under contract No.

GS04T01BFC0060. The opinions expressed herein are those of the author(s) and do not necessarily reflect the

views of the DoD or Mississippi State University.”

ugc tutorial octave jchaves 062406 v2

Documents