![Page 1: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/1.jpg)
1GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
D. Kopanski*, J. Borkowski*, M. Tudruj* x
*Polish-Japanese Institute of Information Technology, 86 Koszykowa Str., 02-008 Warsaw, Poland
xInstitute of Computer Science, Polish Academy of Sciences21 Ordona Str. 01-237 Warsaw, Poland
{janb, damian, tudruj}@pjwstk.edu.pl
GRID applications control based on synchronizers
![Page 2: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/2.jpg)
2GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Introduction
The principles of application control
Implementation issues
Conclusions
Contents
![Page 3: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/3.jpg)
3GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
•Monitoring global states
•Strongly Consistent Global States (SCGS)
•Observed Global States (OGS)
•Activation and cancellation
•P-GRADE system
•PS-GRADE system
Introduction
![Page 4: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/4.jpg)
4GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Arrows represent reliable, asynchronous communication channels
P1
P2 P3 P4
S a
state information
S b
control
Processes can communicate with a number of Synchronizers. Synchronizers learn state information from processes and send back control information.
synchronizers
processes
Monitoring global states
![Page 5: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/5.jpg)
5GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
•There is no global clock, no shared memory
•Synchronizer must be able to order properly incoming events to build Strongly Consistent Global States (SCGS)
•SCGS is a combination of process local states, one state from each process, such that the local states are pairwise concurrent. E.g. <s1,t1> is a SCGS, <s1,t2> is not.
P1
P2
sync
e1 e2
f1 f2
s1
t2t1
m1m2
Monitoring consistent global states
![Page 6: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/6.jpg)
6GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
•Events must have timestamps to be able to order messages correctly. Logical vector clocks or real time intervals based on roughly synchronized local clocks can be used
•If process local clocks are synchronized with a known accuracy, then real time interval timestamps can be used to identify SCGS
process 1
process 2
e1 e2 e3 e4 e5
e1 e3 e2 e4
SCGS duration period e1 event occurrence interval, an event occurred somewhen within this time interval
S1 S2 S3
S1 a SCGS on a linear lattice
Strongly Consistent Global States
![Page 7: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/7.jpg)
7GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Observed states
P1
P2
sync
e1 e2
f1 f2
-f1
e1f1
e1f2
e2f2
•We don’t need a clock synchronization•We don’t need QoS (Quality Of Service) •The reaction is fired immediately
![Page 8: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/8.jpg)
8GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
computations
activated procedure
continued computations
activity
cancellation handling
cancelled fragment
control signal
control signal
another activity
spared time
Computation activation and cancellation caused by predicate evaluation
![Page 9: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/9.jpg)
9GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
A complete graphical programming environment for developing message passing applications designed at Parallel and Distributed Systems Laboratory of the SZTAKI Institute of Hungarian Academy of
Sciences
•Application level specifies processes
and their interconnections
•Process level defines control flow diagram of a process
•Text level is used to enter sequential C code into elements of a flow diagram
loopbegin
receivefrom port 0
send throughport 1
sequential code in C
GRADE system
![Page 10: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/10.jpg)
10GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
standard message passing channels
local state info transfer channels
signal transfer channels
GRADE extension – state information acquisition
![Page 11: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/11.jpg)
11GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Start
Wait for a process/application state information message
Is a new SCGS / OGS detected?
Predicate 1 satisfied?
...
Update state records and make them available for
control predicates
Send sync signals
Yes
Predicate 2 satisfied?
Predicate k satisfied?
Send sync signals
Send sync signals
Yes Yes Yes
No
State messages
No No No
Condition 2 Condition k Condition 1
GRADE extension - synchronizer
![Page 12: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/12.jpg)
12GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
GRADE extension – synchronizerCondition Window
![Page 13: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/13.jpg)
13GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
condition
send signal
reception of state variables
GRADE extension – synchronizercontrol flow window
![Page 14: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/14.jpg)
14GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Start signal-sensitive region "watching-
signal"
End signal-sensitive region "endwatching-
signal"
Resume interrupted
computations
Cancel computations
Send state
End signal- insensitive region
Start signal- insensitive region
GRADE extension – Process - control flow window
![Page 15: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/15.jpg)
15GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
GRADE extension – synchronizer hierarchy
![Page 16: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/16.jpg)
16GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
The principles of application control
Control of GRID application by:•Data control flow (similarly to P-GRADE Workflow implemented by SZTAKI) , based on input and output files for cluster application •GRID Synchronizer :
•Collects information (vector of state) about application state•Detects SCGS or OGS •Computes conditions •Sends signals to the application
![Page 17: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/17.jpg)
17GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
A GRID-level synchronizer inserted into a workflow graph A1
1
A2 A3
1 1
A4
1
4 4 5 5
A5
1 2
Synch1
2 1 2 2
3 3 4 3
5 6
2 3
A6
3 4
1
![Page 18: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/18.jpg)
18GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
A GRID-level synchronizer and an application (example)
GRID Synchronizer
Application A2
Application A3
Application A4
Application A5
![Page 19: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/19.jpg)
19GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Implementation concepts
Hosting environment
Web Service Engine
Grid Service Container
User Defined Service
Base Service
System Level Service
OGSI Spec Implementation Security Infrastructure
GGWS GGMS
GT3 core architecture and PS-GRADE service
•We use the Globus Toolkit v3 (GT3) to implement web service infrastructure •User Defined Service layer will be used for signal and message delivery = Grade-Globus-Web-Service (GGWS) and for the application maintenance service on the GRID= Grade-Globus-Maintenance-Service (GGMS)•Inter-grid communication
with SOAP protocol •We also use :
•GridFTP services for input, output files and program code transfer•GRAM serviced for local job managment
![Page 20: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/20.jpg)
20GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Grade-Globus-Maintenance-Service
GGMS will be the central maintenance service of GRID-Grade application.Main function of GGMS :•Checks the start conditions for applications and if they are true, initiates execution of application•Makes the ”GRID execution map” with information about current running applications and sends it to the GGSW (if GGSW need it)•Makes the global (model) clock available to all GGSW•GGCSW use :
•GridFTP services for input, output files and program code transfer•GRAM services for job managment •SOAP for communication with GGSW
![Page 21: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/21.jpg)
21GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
GGMS – model of execution •Start = execute GGCSW on selected GLOBUS Server•LOAD Data = All executions, input files , execution Map to the selected GLOBUS Server with the use of GridFTP•Starts the Global Synchronizers•Main LOOP - Until all applications complete
•Checks all starting conditions•If any fulfilled then
•Run the ready Grid applications •Updates the GRID execution map•Sends the execution map to the GGSW (if GGSW need it)
•Checks the state of all running application •If any application completes then
•Gets the output files •Updates the GRID execution map•Sends the execution map to the GGSW
•Maintains the time synchronisation of all running GGSW•Stops the Global Synchronizers •Waits for output data request
![Page 22: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/22.jpg)
22GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
GGMS – an application starting pattern •Makes a new instance of GGSW on selected GLOBUS server
•Sends the execution file and input file to the GLOBUS server throught GridFtp service
•Executes program by the use of the relevant GRAM service in interactive mode or in the batch mode
•Creates then communication interface between the GGSW and the started application
![Page 23: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/23.jpg)
23GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Globus Resource Allocation Manager – GRAM service •GRAM simplifies the use of remote systems by providing a single standard interface for requesting and using remote system resources for the execution of jobs. •The most common use of GRAM is remote job submission and control. •Grid Service Factory Pattern
•Create Service •Service instance is created •The request is validated •User’s job request is *ready* to be started
•Start operation •User’s job request is started •Service instance monitors job request •Updates request SDE (Service Data Element)
•Job control •Ensures client received a handle to the job before resources have been consumed
GGMS
GRAM
Local managment Machanism
PS-GRADE Application
GGMS and GRAM
![Page 24: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/24.jpg)
24GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
The clock synchronisation
•If an application uses the Strongly Consistent Global States we have to implement a clock synchronisation mechanism
•We will use the GGMS’s clock as a reference clock
•After clock synchronisation call, all the GGWSs measure the difference between their local clocks and the referenced clock
•When the local synchronizer sends a state message to the global synchronizer, GGWS does the timestamp correction by adding to the timestamp the value of time difference versus the reference clock.
![Page 25: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/25.jpg)
25GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Grade-Globus-Web-Service (GGWS)•Main functions of GGSW
•Implements communication between local synchronizers and the global synchronizer (with SOAP Messaging)
•Corrects timestamps attached to state messages(in the SCGS mode)
•Maintains the GRID CENTER time-synchronisation process
•Monitors running application
![Page 26: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/26.jpg)
26GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
Inter GRID Communication concepts
![Page 27: 1 GRID applications control based on synchronizers, D. Kopanski *, J. Borkowski *, M. Tudruj The Cracow Grid Workshop 2004 D. Kopanski *, J. Borkowski](https://reader035.vdocuments.mx/reader035/viewer/2022062619/5515426d550346c77d8b5ff1/html5/thumbnails/27.jpg)
27GRID applications control based on synchronizers, D. Kopanski*, J. Borkowski*, M. Tudruj
The Cracow Grid Workshop 2004
The paper has presented how the synchronization-based parallel application control can be extended and ported onto the GRID level. With the use of the proposed method we can create an advanced control of many applications running in the GRID environment. Inter–application coordination between programs, which are executed on different GRID sites, is supported.
We have employed the Globus Toolkit as the required middleware implementation platform.
Full implementation of the described extension of the PS-GRADE system will be done in co-operation with P-GRADE authors i.e. SZTAKI Institute of Hungarian Academy of Sciences
Conclusions