© geodise project, university of southampton, 2003. geodise middleware & optimisation graeme...
Post on 05-Jan-2016
217 Views
Preview:
TRANSCRIPT
© Geodise Project, University of Southampton, 2003.
Geodise Middleware& Optimisation
Graeme Pound, Hakki Eres, Gang Xue
& Matthew Fairman
Summer 2003
© Geodise Project, University of Southampton, 2003.
Computational Toolbox
• Access to Globus compute resources from Matlab
• Proxy certificate management, job submission and file transfer
• Complex grid-enabled workflows may be coordinated from Matlab
• Uses the Java CoG 1.1
© Geodise Project, University of Southampton, 2003.
Computational Toolbox
gd_createproxy.m Creates a Globus proxy certificate for the user's credentials
gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate
gd_certinfo.m Returns information about the user's certificate
gd_proxyinfo.m Returns information about the user's proxy certificate
gd_proxyquery.m Queries whether a valid proxy certificate exists
gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager
gd_jobstatus.m Gets the status of a Globus GRAM job
gd_jobpoll.m Queries the status of a Globus GRAM job until complete
gd_jobkill.m Kills a Globus GRAM specified by job handle
gd_putfile.m Puts a remote file using GridFtp
gd_getfile.m Retrieves a remote file using GridFtp
gd_rmfile.m Deletes a remote file using GridFtp
gd_makedir.m Creates a remote directory using GridFtp
gd_rmdir.m Deletes a remote directory using GridFtp
© Geodise Project, University of Southampton, 2003.
Jython Computational Toolbox
• Python interface to functionality of Matlab Computational Toolbox
• Jython is a 100% Java implementation of the Python scripting language
• Lightweight alternative to the Matlab Computational Toolbox
© Geodise Project, University of Southampton, 2003.
Job Submission Service Client
• Matlab client to job submission web service • User detached from compute endpoint – Condor pool• Machines publish available resources:
– HasProEngineer = TRUE– ProEngineerVersion = "2001400"– ProEngineerPath = "C:\Program Files\proe2001\bin
• DIME file transfer over http• MATLAB Client Functions:
– grid_platform – Describes the platform requirements of the job– grid_submit – Submits the job to the web service, returns a job ID– grid_poll – Polls job ID– grid_status – Queries job status– grid_results – Retrieves the output files
© Geodise Project, University of Southampton, 2003.
OptionsMatlab
• Matlab interface to the Options design exploration system
• State of the art design search and optimisation algorithms
• Objective and constraint functions exposed as Matlab functions
• Grid-enabled job brokers easily incorporated
© Geodise Project, University of Southampton, 2003.
OptionsMatlab Parallel Architecture
• optjob.m allows evaluation of multiple design points
• Concurrent evaluation of optfun.m and optcon.m using Objective Function Service Client
• Single evaluation via optjob.m from the optfun and optcon subroutines
Matlab environment
OptionsMatlab.dll
optjob.m
optjob optfun optcon
optfun.moptcon.m
© Geodise Project, University of Southampton, 2003.
Objective Function Client
• Grid-enabled objective function evaluation• Enables multiple concurrent evaluations• Matlab functions provided:
– gd_objsubmit - Transfers required files, submits job and returns handle
– gd_objvalue - Retrieves function evaluation, prints standard output of failed jobs and cleans up files
– gd_objcleanup - Removes remote files of specified jobs
• Multiple Matlab processes, one Matlab licence • Configuration file defines available objective function
servers
© Geodise Project, University of Southampton, 2003.
Objective Function Client
Matlab
Geodisefile archive
Globus server
gd_archive.m
gd_objsubmit.m
gd_jobpoll.m
gd_objvalue.m
Matlab
objfun.m
1
2
3
4
5
© Geodise Project, University of Southampton, 2003.
A Level 2 Grid Application
• The aim is to find enhanced vibration isolation performance for satellite designs
• Matlab is the scripting environment• Geodise toolkit is utilized• The GA toolbox for Matlab is used for
optimization• UK e-Science Grid resources used
– Cambridge e-Science Centre– CLRC e-Science Centre– London e-Science Centre– Oxford e-Science Centre– Southampton e-Science Centre– Southampton Information Systems Services
© Geodise Project, University of Southampton, 2003.
Parallel Job Submission (11x11)
© Geodise Project, University of Southampton, 2003.
Results (Sample points = 11x11)
© Geodise Project, University of Southampton, 2003.
Conclusions
• Submitting compute jobs is the easy bit.• Complexity of grid computing includes:
– applying for authorisation at a numerous locations– managing the transfer of files– keeping track of heterogeneous compute resources
• Requirements for:– Grid enabled data management– Transparent Grid-enabled tools– Consistent computing environments– Reservation and co-allocation of resources– Grid debugging
top related