christian kocks april 3, 2012 high-performance computing cluster in aachen
Post on 11-Jan-2016
214 Views
Preview:
TRANSCRIPT
Christian KocksApril 3, 2012
High-Performance Computing Cluster in Aachen
Christian KocksApril 3, 2012
Slide 2ww
w.k
omm
unik
atio
nste
chni
k.or
g
Typical Simulation Workflow
Simulator development
Simulation
Evaluation and visualization of results
Christian KocksApril 3, 2012
Slide 3ww
w.k
omm
unik
atio
nste
chni
k.or
g
Workflow for Simulations on HPC Cluster
• Develop simulator on local PC / server• Perform short tests on local PC / server• Transfer simulator to HPC cluster (Subversion!)• Perform short tests on HPC cluster using the Linux login shell• Enqueue simulations on HPC cluster• Wait for notification e-mail from HPC cluster• Transfer results to local PC / server (e.g. with WinSCP using SCP
protocol)• Evaluate and visualize results on local PC
HPC High-Performance ComputingSCP Secure Copy
Christian KocksApril 3, 2012
Slide 4ww
w.k
omm
unik
atio
nste
chni
k.or
g
Local Simulations vs. HPC Cluster Simulations
Local simulations HPC cluster simulation
Execution of simulations can be directly controlled Simulations must be enqueued
Very low setup time necessary to start simulation Some preparation necessary for running simulations
Number of available resources very limited “Unlimited” resources
Number of parallel jobs limited “Unlimited” number of parallel jobs
Arrangements with colleagues necessary No arrangements necessary
number of simulations
tota
l sim
ulat
ion
time
local
HPC cluster
Christian KocksApril 3, 2012
Slide 5ww
w.k
omm
unik
atio
nste
chni
k.or
g
Using Matlab on HPC Cluster
• Connect to Linux login shell using SSH client (e.g. PuTTY):• Server name: cluster-linux.rz.rwth-aachen.de
• Load Matlab modules:module load MISCmodule load matlab
• Start Matlab:matlab -nodisplay -nodesktop -nosplash -nojvm
-logfile job.log -c @lic-serv.uni-due.de
• Alternative: use graphical remote session (with NX client)
Christian KocksApril 3, 2012
Slide 6ww
w.k
omm
unik
atio
nste
chni
k.or
g
Using Subversion on HPC Cluster
• Use Subversion module svntest for first tests• Check out Subversion module (e.g. svntest) from KT server:
svn checkout https://134.91.99.160/svn/svntest/trunk [dst]
• Update local working copy:svn update
• Add file test.m:svn add test.m
• Commit changes to KT server:svn commit
Christian KocksApril 3, 2012
Slide 7ww
w.k
omm
unik
atio
nste
chni
k.or
g
Sample Queue Script Filehttps://versions1.kt.uni-due.de/svn/lib/trunk/simulators/sim_awgn_matlab/sample_simple.sh
#!/usr/bin/env zsh#BSUB -J sim_awgn_matlab # job name#BSUB -o sim_awgn_matlab.%J # job output (use %J for job id)#BSUB -e sim_awgn_matlab.e%J # error output#BSUB -W 0:20 # hard limits in hours:minutes#BSUB -M 512 # memory in MB#BSUB -u my.name@kommunikationstechnik.org # e-mail address for notification#BSUB -N # enable e-mail notification#BSUB -n 2 # request number of compute slots#BSUB -a openmp # use esub for OpenMP/shared memory jobs
### load matlab modulesmodule load MISCmodule load matlab
### change to the work directorycd $HOME/svn/lib/simulators/sim_awgn_matlab
### run matlabmatlab -nodisplay -nodesktop -nosplash -nojvm -logfile job.log
-c @lic-serv.uni-due.de <sim_awgn_matlab_run.m
Christian KocksApril 3, 2012
Slide 8ww
w.k
omm
unik
atio
nste
chni
k.or
g
Sample Queue Script File – Advancedhttps://versions1.kt.uni-due.de/svn/lib/trunk/simulators/sim_awgn_matlab/sample_advanced.sh
#!/usr/bin/env zsh#BSUB -J sim_awgn_matlab # job name#BSUB -o sim_awgn_matlab.%J # job output (use %J for job id)#BSUB -e sim_awgn_matlab.e%J # error output#BSUB -W 0:20 # hard limits in hours:minutes#BSUB -M 512 # memory in MB#BSUB -u my.name@kommunikationstechnik.org # e-mail address for notification#BSUB -N # enable e-mail notification#BSUB -n 2 # request number of compute slots#BSUB -a openmp # use esub for OpenMP/shared memory jobs
### load matlab modulesmodule load MISCmodule load matlab
### change to the work directorycd $HOME/svn/lib/simulators/sim_awgn_matlab
### run matlabmatlab -nodisplay -nodesktop -nosplash -nojvm -logfile job.log -c @lic-serv.uni-due.de <<EOFsim_awgn_matlab('ebn0', 6.78952, 'ModulationOrder', 2, 'Log', 'true', 'NumSymbols', 1000,
‚Filename', 'sample_advanced');EOF
Christian KocksApril 3, 2012
Slide 9ww
w.k
omm
unik
atio
nste
chni
k.or
g
Job Management
• Enqueue a job:bsub <myscript.sh
• Query unfinished jobs:bjobs
• Kill unfinished job:bkill [job ID]
Christian KocksApril 3, 2012
Slide 10ww
w.k
omm
unik
atio
nste
chni
k.or
g
General Hints for using HPC Cluster
• Ask Mrs. Tiedtke from Uni DuE for HPC cluster account• Write the simulator in a way to allow the execution of multiple small
simulations instead of one long simulation• Collect simulation parameters and results in MAT file• Deactivate all graphical outputs• Read “HPC Primer” for further information on using the cluster• Visit http://www.rz.rwth-aachen.de/aw/cms/rz/Themen/~mem/hochleistungsrechnen
Christian KocksApril 3, 2012
Slide 11ww
w.k
omm
unik
atio
nste
chni
k.or
g
High-Performance Computing Cluster in Aachen
Demonstration…
top related