a pparc funded project workflow in astrogrid jeff lusted dept physics and astronomy university of...
TRANSCRIPT
![Page 1: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/1.jpg)
A PPARC funded project
Workflow in Astrogrid
Jeff Lusted
Dept Physics and Astronomy
University of Leicester
![Page 2: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/2.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Part One...
A Quick Tour of Workflow Architecture
![Page 3: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/3.jpg)
15 Dec 2004 Workflow in Astrogrid 1
What is the purpose of Workflow?
To accomplish a complex piece of work:
It takes more than one step to achieve.
The workflow is executed asynchronously.
Each step is executed asynchronously.
![Page 4: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/4.jpg)
15 Dec 2004 Workflow in Astrogrid 1
A Skeleton Workflow<workflow name="ThreeStepWorkflow" >
<sequence>
<Activity name="Step One" >
<tool name="query-6df">
...
</tool>
</Activity>
<Activity name="Step Two" >
<tool name="query-2Mass" >
...
</tool>
</Activity>
<Activity name="Step Three" >
<tool name="federator" >
...
</tool>
</Activity>
</sequence>
</workflow>
![Page 5: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/5.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Expansion of Step One...
<Activity name="Step one" >
<description>...</description>
<tool name="query-6df" interface="adql">
<input>
<parameter name="Format" indirect="false">
<value >VOTABLE</value>
</parameter>
<parameter name="Query" indirect="true">
<value >ivo://...#jl99/query/6dfQuery.xml</value>
</parameter>
</input>
<output>
<parameter name="Result" indirect="true">
<value >ivo://...#jl99/votable/6dfQueryResults.vot</value>
</parameter>
</output>
</tool>
</Activity>
![Page 6: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/6.jpg)
15 Dec 2004 Workflow in Astrogrid 1
The View from 10,000 Metres
A workflows exist at:
Design Time
Execution Time
*** When submitted for execution, a workflow becomes a job. ***
![Page 7: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/7.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Design Time – From 5000 Metres
A Design:
Is held as an XML file in MySpace.
The Design process:
Is managed by the Workflow Editor in the Portal.
Is aided by holding metadata in the Astrogrid Registry.
![Page 8: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/8.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Execution Time – From 5000 Metres
Job Execution System:
Central controlling element.
Interprets the workflow design.
Manages the flow of control between Steps.
Common Execution Architecture:
Manages the local environment for each Step executed.
Manages the flow of data required by each Step.
![Page 9: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/9.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Overview of Workflow Architecture
![Page 10: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/10.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Part Two...
Workflow as a Programming Language
![Page 11: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/11.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Back to Design...
Steps and Tools
Sequences and Flows.
Variables and Scopes.
Loops and Conditional Execution.
Scripting and Script Objects.
![Page 12: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/12.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Sequence – Sequential Execution <sequence> <Activity name="Step One" > <tool name="query-6df"> ... </tool> </Activity> <Activity name="Step Two" > <tool name="query-2Mass" > ... </tool> </Activity> <Activity name="Step Three" > <tool name="federator" > ... </tool> </Activity> </sequence>
![Page 13: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/13.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Flow – Parallel Execution <sequence> <flow> <Activity name="Step One" > <tool name="query-6df"> ... </tool> </Activity> <Activity name="Step Two" > <tool name="query-2Mass" > ... </tool> </Activity> </flow> <Activity name="Step Three" > <tool name="federator" > ... </tool> </Activity> </sequence>
![Page 14: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/14.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Setting Variables
Setting...
Using variables to construct a filename...
<set var="namestub" value="${votable}" /><set var="filecount" value="${1}"/>
<output> <parameter name="result" indirect="true"> <value>${namestub}-${filecount + 1}</value> </parameter></output>
![Page 15: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/15.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Conditional Execution
<set var="x" value="${1}" />...<if test="${x > 0}"> <then> <sequence> ... </sequence> </then> <else> ... </else></if>
![Page 16: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/16.jpg)
15 Dec 2004 Workflow in Astrogrid 1
While Loop
<while test="${results == null || results.size() < 1}">
<step result-var="results"> ... </step>
</while>
![Page 17: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/17.jpg)
15 Dec 2004 Workflow in Astrogrid 1
For Loop
<for var="x" items="${1...10}>
<step result-var="results"> ... </step>
</for>
![Page 18: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/18.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Scripting
Scripting Expressions
Script
<set var="x" value="${1}" /><if test="${x > 0}"> ...</if>
<script> <body> print("Step produced " + ${count} + "votables"); </body></script>
![Page 19: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/19.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Scripting Example plus Script Objects
<step result-var="results"> ...</step><set var="urlList" /><script> <body> if (results.size() != 1) { jes.error("didn't produce expected number of results"); } else { votable = results.get('votable'); parser = new XmlParser(); nodes = parser.parseText(votable); urlList = nodes.depthFirst() .findAll{it.name() == 'STREAM'} .collect{it.value()}.flatten(); print(urlList); } </body></script>
![Page 20: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/20.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Conclusion – Astrogrid Workflow Capabilities
Distributed workflow executing on an asynchronous basis
Highly programmable...
... Workflows from the simple to the complex
Can integrate astronomical tools
![Page 21: A PPARC funded project Workflow in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester](https://reader036.vdocuments.mx/reader036/viewer/2022062409/5697bfa61a28abf838c982a6/html5/thumbnails/21.jpg)
15 Dec 2004 Workflow in Astrogrid 1
Contacts.
Work Group Leader: Jeff Lusted [email protected]
Workflow and JES: Noel Winstanley [email protected]
Common Execution Architecture: Paul Harrison [email protected]
Workflow Editor: Phil Nicolson [email protected]
www.astrogrid.org