distributed systems techs 3. gridsdana.petcu/distrib/tds3.pdf · computing power is produced much...
TRANSCRIPT
![Page 1: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/1.jpg)
Distributed Systems – Techs3. Grids
October 23, 2009
![Page 2: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/2.jpg)
![Page 3: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/3.jpg)
DefinitionA type of distributed system that enables the sharing, selection, &
aggregation of geographically distributed resources:Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;Software – e.g., ASPs renting expensive special purpose applications on demand;Catalogued data and databases – e.g. transparent access to human genome database;Special devices/instruments – e.g., radio telescope –SETI@Home searching for life in galaxy.People/collaborators.
depending on their availability, capability, cost, and user QoSrequirements
for solving large-scale problems/applications. thus enabling the creation of “virtual organization” (VOs)
![Page 4: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/4.jpg)
![Page 5: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/5.jpg)
Grid‘s main idea
To treat CPU cycles and software like commodities.Enable the coordinated use of geographically distributedresources – in the absence of central control and existing trust relationships.Computing power is produced much like utilities such as power and water are produced for consumers.Users will have access to “power” on demand“When the Network is as fast as the computer’s internal links, the machine disintegrates across the Net into a set of special purpose appliances” – Gilder Technology Report June 2000
![Page 6: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/6.jpg)
Computational Grids andElectric Power Grids
Power Grid analogy– Power producers: machines, software, networks, storage systems– Power consumers: user applicationsApplications draw power from the Grid the way appliances draw electricity from the power utility.– Seamless, High-performance, Ubiquitous, DependableWhy the Computational Grid is like the Electric Power Grid– Electric power is ubiquitous– Don’t need to know the source of the power (transformer, generator) or thepower company that serves itWhy the Computational Grid is different from the Electric Power Grid– Wider spectrum of performance– Wider spectrum of services– Access governed by more complicated issues: Security, Performance
![Page 7: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/7.jpg)
![Page 8: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/8.jpg)
![Page 9: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/9.jpg)
![Page 10: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/10.jpg)
![Page 11: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/11.jpg)
![Page 12: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/12.jpg)
![Page 13: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/13.jpg)
![Page 14: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/14.jpg)
P2P, Cluster, Internet computing vs. Grid computing
Peer-to-peer networks (eg Kazaa) fall within the definition of grid computing (the resource shared is the storage capacity of each node)P2P Working Group part of Global Grid ForumA cluster is a resource that can be shared- a grid is a cluster of clustersInternet computing: a VO is assembled for a particular project and disbanded once the project is complete -the shared resource is the Internet connected desktop
![Page 15: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/15.jpg)
Grid evolution: 1st generation
![Page 16: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/16.jpg)
Grid computing: 2nd generation
![Page 17: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/17.jpg)
Grid computing: 3rd generation
![Page 18: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/18.jpg)
OGSA services
Open Grid Service Architecture, defined by GGF groupIn ubiquitous Grid platform, there is common need for some essential set of interfaces, behaviors, resource models, and bindingsdefines the core set of services essential for grid, their functionality and interrelationshipsCore services: service interaction, management, communication, security Non-core: data, program execution, resource management
Grid service is a Web service with extensions, which are:Name (handle GSH, reference GSR)Lifetime management (factories, persistent and transient services)State (Service Data)Notification as well as querying
![Page 19: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/19.jpg)
![Page 20: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/20.jpg)
De facto-middleware: Globus Toolkit
![Page 21: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/21.jpg)
![Page 22: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/22.jpg)
The future?
![Page 23: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/23.jpg)
What do users want ?
Grid ConsumersExecute jobs for solving varying problem size and complexityBenefit by selecting and aggregating resources wiselyTradeoff timeframe and cost
Grid ProvidersContribute (“idle”) resource for executing consumer jobsBenefit by maximizing resource utilizationTradeoff local requirements & market opportunity
![Page 24: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/24.jpg)
Grid ApplicationsDistributed HPC (Supercomputing):
Computational science.High-Capacity/Throughput Computing:
Large scale simulation/chip design & parameter studies.Content Sharing (free or paid)
Sharing digital contents among peers (e.g., Napster)Remote software access/renting services:
Application service provides (ASPs) & Web services.Data-intensive computing:
Drug Design, Particle Physics, Stock Prediction...On-demand, real-time computing:
Medical instrumentation & Mission Critical.Collaborative Computing:
Collaborative design, Data exploration, education.Service Oriented Computing (SOC):
Towards economic-based Utility Computing: New paradigm, new applications, new industries, and new business.
![Page 25: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/25.jpg)
![Page 26: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/26.jpg)
GridCoord Survey
![Page 27: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/27.jpg)
EGEE – largest European Grid & LHC
![Page 28: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/28.jpg)
EGEE usage
![Page 29: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/29.jpg)
RO in EGEE, SEE-Grid & other Grids
Come to: SEE-Grid-SCI training in December 19th, 2008 !!!
![Page 30: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/30.jpg)
Grid Requirements
Identity & authenticationAuthorization & policyResource discoveryResource characterizationResource allocation(Co-)reservation, workflowDistributed algorithmsRemote data accessHigh-speed data transfer
Performance guaranteesMonitoring AdaptationIntrusion detectionResource managementAccounting & paymentFault managementSystem evolutionEtc.
![Page 31: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/31.jpg)
Some Grid Requirements – User Perspective
Single allocation: if any at allSingle sign-on: authentication to any Grid resources authenticates for all othersSingle compute space: one scheduler for all Grid resourcesSingle data space: can address files and data from any Grid resourcesSingle development environment: Grid tools and libraries that work on all grid resources
![Page 32: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/32.jpg)
The Security Problem
Resources being used may be extremely valuable & theproblems being solved extremely sensitiveResources are often located in distinct administrative domains
– Each resource may have own policies & procedures• The set of resources used by a single computation may be large,
dynamic, and/or unpredictable– Not just client/serverIt must be broadly available & applicable– Standard, well-tested, well-understood protocols– Integration with wide variety of tools
![Page 33: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/33.jpg)
The Resource Management Problem
Enabling secure, controlled remote access to computational resources and management of remote computation
– Authentication and authorization– Resource discovery & characterization– Reservation and allocation– Computation monitoring and control
![Page 34: Distributed Systems Techs 3. Gridsdana.petcu/distrib/TDS3.pdf · Computing power is produced much like utilities such as power and water are produced for consumers. Users will have](https://reader035.vdocuments.mx/reader035/viewer/2022062607/605c193b68663a7c561bf3e9/html5/thumbnails/34.jpg)