Introduction to Grid Computing and its components

Grid Computing

• Internet has become an attractive means of sharing information across the globe.

• The idea of grid computing has emerged from the fact that Internet can also be used several other purposes such as sharing the computing power, storage space, Scientific devices and software programs.

• The term ‘Grid’ is chosen, as it is analogous to Electrical Power grid

• A Computational grid is a subclass of general grid, wherein it contains only computing resources.

Issues in Grid Computing

There are three main issues that characterize computational grids:

• Heterogeneity: A grid involves a multiplicity of resources that are heterogeneous in nature and might span numerous administrative domains across wide geographical distances.

• Scalability: A grid might grow from few resources to millions. This raises the problem of potential performance degradation as a Grids size increases. Consequently, applications that require a large number of geographically located resources must be designed to be extremely latency tolerant.

• Dynamicity or Adaptability: In a grid, a resource failure is the rule, not the exception. In fact, with so many resources in a Grid, the probability of some resource failing is naturally high. The resource managers or applications must tailor their behaviour dynamically so as to extract the maximum performance from the available resources and services.

Grid Components

• Portal-User Interface

• Security (Grid Security Infrastructure (GSI)

• Broker (Monitor and Discovery Service (MDS)

• Scheduler

• Data Management (Grid FTP)

• Job and Resource Management (GRAM)

Portal - User interface(The current Globus Toolkit does not provide any user portal)

Grid Components

Portal - User interface(The current Globus Toolkit does not provide any user portal)

A grid portal provides the interface for a user to launch applications that will utilize the resources and services provided by the grid.

user sees the grid as a virtual computing resource just as the consumer of power sees the receptacle as an interface to a virtual generator.

The current Globus Toolkit does not provide any services or tools to generate a portal, but this can be accomplished with tools such as WebSphere.

Grid Components

Security (Grid Security Infrastructure (GSI))

Grid Components

Security (Grid Security Infrastructure (GSI))

A major requirement for grid computing is security. At the base of any gridenvironment, there must be mechanisms to provide security includingauthentication, authorization, data encryption, and so on.

The Grid SecurityInfrastructure (GSI) component of the Globus Toolkit provides robust security mechanisms. The GSI includes an OpenSSL implementation.

It also provides a single sign-on mechanism, so once a user is authenticated, a proxy certificate is created and used when performing actions within the grid.

When designing your grid environment, you may use the GSI sign-in to grant access to the portal or you may have your own security for the portal.

The portal would then be responsible for signing into the grid, either using the user’s credentials, or using a generic set of credentials for all authorized users of the portal.

Grid Components

Broker (Monitoring and Discovery Service (MDS))

Grid Components

Once authenticated, the user will be launching an application. Based on the application, and possibly on other parameters provided by the user, the next step is to identify the available and appropriate resources to utilize within the grid.

This task could be carried out by a broker function. Though there is no broker implementation provided by Globus, there is an LDAP-based information service.This service is called Grid Information Service (GIS), or more commonly the Monitoring and Discovery Service (MDS).

This service provides information about the available resources within the grid and their statuses. A broker service could be developed that utilizes MDS.

Broker (Monitoring and Discovery Service (MDS)).

Grid Components

Scheduler (Globus Toolkit does not provide scheduler)

Grid Components

Once the resources have been identified, the next logical step is to schedule the individual jobs to run on them. If a set of standalone jobs are to be executed with no interdependencies, then a specialized scheduler may not be required.

However, if it is desired to reserve a specific resource or to ensure that different jobs within the application run concurrently (for instance, if they require inter-process communication), then a job scheduler should be used to coordinate the execution of the jobs.

The Globus Toolkit does not include such a scheduler, but there are several schedulers available that have been tested with and can be utilized in a Globus grid environment.

Scheduler (Globus Toolkit does not provide scheduler)

Grid Components

Data management (Grid Access to Secondary Storage GASS))

Grid Components

Data management (Grid Access to Secondary Storage (GASS))

If any data (including application modules) must be moved or made accessible to the nodes where an application’s jobs will execute, then there needs to be asecure and reliable method for moving files and data to various nodes within the grid.

The Globus Toolkit contains a data management component that provide ssuch services. This component, known as Grid Access to Secondary Storage GASS), includes facilities such as GridFTP. GridFTP is built on top of the standard FTP protocol, but adds additional functions and utilizes the GSI for userauthentication and authorization.

Therefore, once a user has an authenticated proxy certificate, she can utilize the GridFTP facility to move files without having to go through a login process to every node involved. This facility provides third party file transfer so that one node can initiate a file transfer between two other nodes.

Grid Components

Job and resource management(Grid Resource Allocation Manager(GRAM)

Grid Components

With all of the other facilities we have just discussed in place, we now get to the core set of services that help perform actual work in a grid environment.

The Grid Resource Allocation Manager (GRAM) provides the services to actually launch a job on a particular resource, check on its status, and retrieve its results when it is complete.

Job and resource management (Grid Resource Allocation Manager (GRAM))

Grid Components

Jobs, via Globus, can be submitted to systems managed by Condor, SGE, PBS and LSF

Grid Scheduling and Resource Management

How Scheduling Works : Grid scheduling involves four main stages:

1. Resource discovery.

2. Resource selection.

3. Schedule generation.

4. Job execution.

1. Resource discovery : The goal of resource discovery is to identify a list of authenticated resources that are available for job submission.

a) The pull model:

• In this modes, a single daemon associated with the scheduler can query Grid resources and collect state information such as CPU loads or the available memory.

• The pull model for gathering resource information incurs relatively small communication overhead, constantly out-of-date, and potentially misleading.

• In centralized scheduling, the resource discovery/query process could be rather intrusive and begin to take significant amounts of time as the environment being monitored gets larger and larger.

b) The push model :

• In this model, each resource in the environment has a daemon for gathering local state information, which will be sent to a centralized scheduler that maintains a database to record each resource’s activity.

• If the updates are frequent, an accurate view of the system state can be maintained over time; obviously, frequent updates to the database are intrusive and consume network bandwidth.

c) The push-pull model :

• The push-pull model lies somewhere between the pull model and the push model.

• Each resource in the environment runs a daemon that collects state information.

2. Resource Selection :

• Once the list of possible target resources is known, the second phase of scheduling process is to select those resources that best suit that constraints and conditions imposed by the user, such as CPU usage, RAM available or disk storage.

• The result of resource selection is to identify a resource list (Rselected) in which all resources can meet the minimum requirements for a submitted job or a job list.

• The relationship between resources available (Ravailable) and resources selected (Rselected) is:

Rselected Ravailable

3. Schedule generation:

• The generation of schedules involves two steps.

1. Resource selection.

2. Job selection.

1. Resource Selection:

• Resource selection process is used to choose resource(s) from the resource list (Rselected) for a given job.

• All resources in the list Rselected could meet the minimum requirements imposed by the job, an algorithm is needed to choose the best resource(s) to executive the job.

• Random selection is a choice, it is not an ideal resource selection policy.

• The resource selection algorithm should take into account the current state of resources and choose the best one based on a quantitative evaluation.

• A resource selection algorithm that only takes CPU and RAM into account could be designed as follows:

WCPU – the weight allocated to CPU speed;CPUload – the current CPU load;CPUspeed – real CPU speed;CPUmin – minimum CPU speed;WRAM – the weight allocated to RAM;RAMusage – the current RAM usage;RAMsize – original RAM size;RAMmin – minimum RAM size.

Example:

The total weighting used in the algorithm is 10, where the CPU weight is 6 and the RAM weight is 4. The minimum CPU speed is 1 GHz and minimum RAM size is 256 MB.

Then, evaluation values for resources can be calculated using the three formulas:

From the results we know Resource3 is the best choice for the submitted job.

2. Job Selection :

• The goal of job selection is to select a job from a job queue for execution. Four strategies that can be used to select a job are given below:

1. First come first serve.

2. Random selection.

3. Priority-based selection.

4. Backfilling selection.

Job execution :

• Once a job and a resource are selected, the next step is to submit the job to the resource for execution.

• Job execution may be as easy as running a single command or as complicated as running a series of scripts that may, or may not, include set up or staging.