Parallel Job Submission In Grid Environment Using

Parallel Particle Swarm Optimization

Dr. G. Sudha SadhasivamAsst. ProfessorDept. of CSE.PSG College Of Technology.

D. Komagal Meenakshi (07MW05)

PSG College Of Technology.

Outline• Scheduling in Grid.• Problem Statement• Need For Job Grouping in Scheduling• Previous Work Done in Job Grouping• Proposed System• Trust Based Filtering of jobs• Particle Swarm Optimization• Parallel PSO• Model for PPSO• Dynamic jobs• Results• Conclusion and Future work• Bibliography

Scheduling in Grid.• Grid computing is a high performance computing

environment to solve large scale computational demands.

• Task scheduling is a fundamental issue in achieving high performance in grid computing systems.

• Reason: Large numbers of tasks are computed on the

geographically distributed resources, a reasonable scheduling algorithm must be adopted order to minimize job completion time with uniform load distribution.

Need

An unorganized deployment of grid applications with a large amount of fine-grain jobs

Leads to

communication overhead dominate the overall processing time

Low computation-communication ratio.

Results

Need For Job Grouping in Scheduling

• Efficient job grouping-based scheduling system is required.

• A Grid Scheduler shouldReduce the total transmission of user jobs to/from

the resources.Reduce the overhead processing time of the jobs

at the resources.

Job Grouping

Dynamically assemble

Transmit

Grid resources

job groups [ coarse grained ]

Jobs of an application [ fine grained ]

Previous Work Done in Job Grouping

• Comparison of Scheduling algorithms with and without job grouping.

• In the context of DAG scheduling, grouping of jobs into clusters to reduce inter-job communication.

• Job Grouping strategy, adaptive to run time environment

• Job Grouping with PSO.

Proposed System• A novel job grouping method using Parallel PSO

• To reduce the communication overhead.

• Enhance the speed of completion of processes.

• Improve resource utilization.

• Improve parallel efficiency.

• Uses PPSO to select the resources to minimize the make span.

• Trust level and dynamism of jobs is considered

• Tool Used - Gridsim-4.2-beta.

The Project aims at …

• Job Grouping based on trust Using PPSO• Parallel Job Submission• Enhancing Computation-communication Ratio• Reducing The Overall Processing Time Of Jobs Using

Parallelization • Improving Resource Utilization In The Grid Environment.• Trust based job filtering• Dynamic job submission

Dynamically assemble Using PPSO

Transmit to

Grid resources

job groups [ coarse grained ]

Filtered Jobs of an application [ fine grain] based on Trust

Grid resources Grid resources

In Parallel

1. Job Grouping

Total number of jobs

Average MI rate of job

MI deviation Percentage

Overhead processing time

Granularity time

Grid Resource

Grid resource 0

Grid resource 1

Grid resource N

Grid Resource File

User Input

GridletsGrid resources’ characteristics

Gridlet MI Resource MIPS Granularity time

Total MIPS

Grid resource 0

Gridlet group 0

Grid resource 1

Gridlet group 1

Grid resource 2

Gridlet group 2

Gridlet groups Resource IDs

…..

Gridlet Scheduler

(1)

(3)

(4)

(5)

(6)

(7)(2)

Trust level

In parallel

Filter jobs based on trust

Job Grouping

2. Trust Based Filtering of jobs• The Grid Information Service GIS gives the information

about all the trust level of the resources .

• The user submits the jobs with different trust values.

• From this, the jobs that have trust values greater than the resource's trust value are filtered out.

• Trust aware resource management and scheduling offer Quality of Service at application layer in grid environment.

3. Particle Swarm Optimization• If large numbers of tasks are computed on the

geographically distributed resources, a reasonable scheduling approach must be adopted in order to get the minimum completion time.

• Task scheduling is a NP-Complete problem

• Heuristic optimization algorithm can be used to solve NP-complete problems.

• Particle Swarm Optimization (PSO) is an evolutionary optimization technique inspired by nature.

• It simulates the process of a swarm of birds preying.

• Its global searching ability can be used for neural network training, control system analysis and design, structural optimization.

• It also has fewer algorithm parameters than genetic algorithm.

• PSO algorithm works well on most global optimal problems.

PSO Concept

• A swarm intelligence based algorithm finds a solution to an optimization problem in a search space.

• Proposed solution exists in the form of a fitness function. • The swarm is typically modeled by particles in

multidimensional space that have a position and a velocity.

• A Particle is a candidate solution in the population and represents a task.

• Particles fly through hyperspace .

• An iterative process to improve candidate solutions is set in motion. The particles iteratively evaluate the fitness of the candidate solutions.

• Particles posses two essential reasoning capabilities– Memory of their own best position and – knowledge of the global best of the swarm.

• As the swarm iterates, the fitness of the global best solution improves.

• All particles being influenced by the global best eventually approach the global best. This phenomenon is called 'convergence'.

PSO Algorithm• Initialize parameters

• Initialize population randomly

• Initialize each particle position vector and velocity vector

Do {• Update each particle’s velocity and position;• Find a permutation according to the updated each particle’s

position;• Evaluate each particle and update the personal best and the global

best;• Apply the local search;• } While (!Stop criterion)

Parallel PSO

Recent advances in computer and network technologies led to parallel optimization algorithms.

Parallel PSO (parallel implementation of stochastic optimization alg)

Parallel PSO design

Intialize

f(x) f(x) f(x)

Check Convergence

Update

# of particles#

of it

erat

ion

s

Model for PPSO

Master

Slave Slave Slave

SEND GLOBAL VALUE RECEIVE INDIVIDUAL VALUE

Gridlet Grouping

Scheduler

Trust based filtered Gridlet list

Resource list

Call PPSO to assign Gridlet To Resources

Create new grouped GridletWith length= Total length

Assign to resources

4. Dynamic jobs

• Dynamic submission of jobs is considered.• User can submit jobs when other jobs are being

processed.• The unused MIPS rating of the resources can be

utilized in a efficient way such that grouping is done by considering the unused MIPS as total MIPS and the jobs are processed.

• Then Parallel Submission of grouped Gridlets to resources is done

Simulation Time for Job Grouping using PSO vs. Parallel PSO

90

100

110

120

130

140

150

20 40 60 80

No of Gridlets

Sim

ula

tio

n T

ime

PPSO

PSO

Total number of processed gridlets for different granularity time and resources

0

20

40

60

80

100

R1 R1-R2 R1-R3 R1-R4 R1-R5

Resources

No

of G

rid

lets

co

mp

lete

d in

gra

n

time

10

20

30

40

50

Load at resources during job grouping with PPSO

0100200300400500600700800900

1000

R1 R2 R3 R4 R5

Resources

loa

d

50 gridlets

60 gridlets

70 gridlets

80 gridlets

90 gridlets

Difference in submission time of gridlets with PSO and PPSO

• Add load balancing feature graph here

Conclusion• The proposed framework using PPSO has less simulation

time compared to job scheduling framework using PSO as the simulation time is reduced.

• Resource selection based on PPSO is used to generate an optimal schedule so as to complete the tasks in a minimum time than PSO as well as utilizing the resources in an efficient way.

• Simulated results demonstrates load balanced resource selection.

• Simulation results demonstrate that PPSO algorithm can get better effect for a large scale optimization problem.

Future Work• Future work would involve developing a more

comprehensive job grouping-based scheduling system that takes into account QoS (Quality of Service) requirements of each user job before performing the grouping method.

• Resource utilization can be done according to the capacity of the resource.

THANK YOU