© geodise project, university of southampton, 2003. geodise middleware graeme pound, gang xue &...
TRANSCRIPT
![Page 1: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/1.jpg)
Geodise Middleware
Graeme Pound, Gang Xue
& Matthew Fairman
Summer 2003
![Page 2: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/2.jpg)
Computational Toolbox
• Access to Globus compute resources from Matlab
• Proxy certificate management, job submission and file transfer
• Complex grid-enabled workflows may be coordinated from Matlab
• Uses the Java CoG 1.1
![Page 3: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/3.jpg)
Computational Toolbox
gd_createproxy.m Creates a Globus proxy certificate for the user's credentials
gd_destroyproxy.m Destroys the local copy of the user's Globus proxy certificate
gd_certinfo.m Returns information about the user's certificate
gd_proxyinfo.m Returns information about the user's proxy certificate
gd_proxyquery.m Queries whether a valid proxy certificate exists
gd_jobsubmit.m Submits a compute job to a Globus GRAM job manager
gd_jobstatus.m Gets the status of a Globus GRAM job
gd_jobpoll.m Queries the status of a Globus GRAM job until complete
gd_jobkill.m Kills a Globus GRAM specified by job handle
gd_putfile.m Puts a remote file using GridFtp
gd_getfile.m Retrieves a remote file using GridFtp
gd_rmfile.m Deletes a remote file using GridFtp
gd_makedir.m Creates a remote directory using GridFtp
gd_rmdir.m Deletes a remote directory using GridFtp
![Page 4: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/4.jpg)
Jython Computational Toolbox
• Python interface to functionality of Matlab Computational Toolbox
• Jython is a 100% Java implementation of the Python scripting language
• Lightweight alternative to the Matlab Computational Toolbox
![Page 5: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/5.jpg)
Job Submission Service Client
• Matlab client to job submission web service • User detached from compute endpoint – Condor pool• Machines publish available resources:
– HasProEngineer = TRUE– ProEngineerVersion = "2001400"– ProEngineerPath = "C:\Program Files\proe2001\bin
• DIME file transfer over http• MATLAB Client Functions:
– grid_platform – Describes the platform requirements of the job– grid_submit – Submits the job to the web service, returns a job ID– grid_poll – Polls job ID– grid_status – Queries job status– grid_results – Retrieves the output files
![Page 6: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/6.jpg)
Geodise Optimisation
Graeme Pound & Hakki Eres
Summer 2003
![Page 7: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/7.jpg)
OptionsMatlab
• Matlab interface to the Options design exploration system
• State of the art design search and optimisation algorithms
• Objective and constraint functions exposed as Matlab functions
• Grid-enabled job brokers easily incorporated
![Page 8: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/8.jpg)
OptionsMatlab Parallel Architecture
• optjob.m allows evaluation of multiple design points
• Concurrent evaluation of optfun.m and optcon.m using Objective Function Service Client
• Single evaluation via optjob.m from the optfun and optcon subroutines
Matlab environment
OptionsMatlab.dll
optjob.m
optjob optfun optcon
optfun.moptcon.m
![Page 9: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/9.jpg)
Objective Function Client
• Grid-enabled objective function evaluation• Enables multiple concurrent evaluations• Matlab functions provided:
– gd_objsubmit - Transfers required files, submits job and returns handle
– gd_objvalue - Retrieves function evaluation, prints standard output of failed jobs and cleans up files
– gd_objcleanup - Removes remote files of specified jobs
• Multiple Matlab processes, one Matlab licence • Configuration file defines available objective function
servers
![Page 10: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/10.jpg)
Objective Function Client
Matlab
Geodisefile archive
Globus server
gd_archive.m
gd_objsubmit.m
gd_jobpoll.m
gd_objvalue.m
Matlab
objfun.m
1
2
3
4
5
![Page 11: © Geodise Project, University of Southampton, 2003. Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003](https://reader036.vdocuments.mx/reader036/viewer/2022083009/5697c0271a28abf838cd61f1/html5/thumbnails/11.jpg)
Conclusions
• Submitting compute jobs is the easy bit.• Complexity of grid computing includes:
– applying for authorisation at a numerous locations– managing the transfer of files– keeping track of heterogeneous compute resources
• Requirements for:– Grid enabled data management– Transparent Grid-enabled tools– Consistent computing environments– Reservation and co-allocation of resources– Grid debugging