edas: providing an environment for decentralized adaptive services

Rüdiger Kapitza

AspectIX Research GroupUniversity Erlangen-Nürnberg

EDAS: Providing an Environment for Decentralized Adaptive Services

2

Grid-Computing

Vision

Providing a user supported grid-like infrastructure forlong-term, decentralized and adaptive services

Open-source project management, online communities

EDAS

Peer-to-Peer

Middleware

3

Challenges

Grid inspired organisational architecture• Concept of domain and virtual organisation

Decentralized management tasks via peer-to-peer techniques• Authentication and authorization• Code provisioning• Service placement and resource management

Decentralized adaptive service model • Fault tolerance• Mobility • Scalability

4

Outline

EDAS Architecture Resource Management Decentralized Adaptive Service Agenda Conclusion

5

EDAS - Architecture

Institutions, companies and also single users are willing to support a project by providing resources

How should we use their resources? Where place the service of our project?

6

EDAS - Core Components

Home Environment (HE)- Connects all nodes of an administrative domain

Service Environment1..* *

provides resources to

Decentralized, Adaptive Service

1

1..*

manages the execution of

Home Environment

Node

1

1..*

manages

Service Environment (SE) - Provides a distributed execution environment for services

Decentralized, Adaptive Service (DAS) - Dynamically distributed within the scope of an service environment

7

EDAS - Home Environment

Mediator between peers of an administrative domain and one or more Service Environments (Projects)

Monitors and manages resources of a peer set Controlled by a Domain Manager

• Add and remove peers • Assign and revoke resources to Service Environments

8

EDAS - Service Environment

Collect and combine system information provided by HEs Supports automatic distribution of services

• Based on load, available resources and DAS specific requirements

Controlled by a service manager• Start, stop and configure services• Accept or decline resource offers (manually or via policy rules)

9

EDAS -Decentralized Adaptive Service

Decentralized, Adaptive Services expand or shrink autonomoulsy depending on • Service demand • Fault tolerance reasons etc.

Could have different internal service-structure models and ideally change its service structure over time if necessary

10

Resource Management

Core idea: Controlled resource sharing• Home Environments offer fixed amount of resources to

Service Environments• Service Environment can accept or decline resource offers

Tasks of the Home Environment • Resource monitoring of physical resources: CPU, network,

memory, disk• Local Area Placement

- Manage service placement at domain level

• Keep service environment limits

Task of the Service Environment • Wide Area Placement

- Manage service placement in scope of a project

11

Resource Limits at Node Level

Physical Limit • Provided by the underlying hardware

Node Limit • Bounds the resource usage of a node

Local Limit • Resources of a node assigned to

a service belonging to certain SE NodeLimit

LocalLimit

Node A

Service A (SE 1)

Service B (SE 2)

unassignedPhysical Limit

0%

100%reserved

12

Resource Limits at Domain Level

Global Limit• The overall amount of resources dedicated to

a service environment

Node A Node B Node C Node D

Global Limit

13

Resource Monitoring at Node Level

Prototype based on monitoring and management facility Java Management Extension (SDK 1.5)

Provides beans to monitor CPU and memory usage Implemented own management bean for bandwidth

JMXManager

WebBrowser

SNMPManager

RMI Adaptor SNMP AdaptorHTTP Adaptor

ApplicationsMBeans MBeans

Mbean Server

RemoteManager

ConnectorLevel

AgentLevel

InstrumentationLevel

14

Resource Monitoring at Node Level

15

Service Placement at Domain Level

Problem • Resource demand of services can dynamically change

Restrictions• Collocation and dislocation of services at the same node

Goal• Avoid service migration if possible

Placement algorithm should balance resource usage and keep limits


Service A

16

Keeping Resource Limits

Problem• Resource demand of services can dynamically change but we

have to keep the global limit• More time critical than service placement

Approach• Rededication of limits


17

Keeping Resource Limits

Solution: Diffusive load balancing algorithm• All nodes supporting a service environment at a domain are

connected in small overlapping groups• Frequent exchange and balancing of free resources (free

slack)• If free resources locally reach zero the global limit is reached • Additionally exchange neighbour and resource limit information

to compensate failed nodes

18

Service Placement at Project Level

Problem • Resource demand of services can dynamically change• Resource availability can dynamically change

Restrictions• Collocation and dislocation of services at the same domain

Goal• Avoid service migration if possible• Consider network proximity

19

Related Approaches

Peer-to-Peer query engines• SWORD (Berkley)• XenoServer (Cambridge)

Decentralized approaches based on Peer-to-Peer Networks• Adaptive Service Placement Algorithms for Autonomous

Service Networks (HP)• T. Repantis et. al: Adaptive Resource Management in Peer-to-

Peer Middleware Centralized approaches for closed systems

• HP Internet Utility Farms: Quatermaster• B. Urgaonkar: Application Placement on a Cluster of Servers • CORBA based systems: Realize & MEAD

20

DAS: Decentralized Adaptive Services

Based on the AspectIX Middleware CORBA ORB Offers with Fragmented Objects a distributed object model

• Fragmented Object (FO): Arbitrary set of distributed fragments• Local fragment at every node that accesses FO• FO is encapsulation unit with well-defined interface, but

without any relation to a specific location.

Fragmented Object

Client

Client

Fragment

FragmentImplementation

View

21


Goal• Provide long-term running, user accessed services in

dynamically changing environments

Approach • Active Object Replication• Migration support• Custom client-side interaction via smart proxy approach

22


Support service development • Annotated IDL and a special IDL-Compiler (IDLFlex by Hans

Reiser) to generate custom stubs and skeletons to interact with a group communication framework (Jgroups)

New method modifier• local - stub method local to the client• private - stub method is only object internally accessible • admin - credential guarded method• read - non state mutating method

23


Replicas can be dynamically created and moved inside the scope of the fragmented object

Client-side fragments can be exchanged Example Service: Version Control Service

• Replicated service core holds only current file information and manages concurrent access

• Files might be stored on different nodes of the Service Environment

Node A

GC (Jgroups)

Node A Node A

Node A

Client Client

Replica Replica

CustomFragment

CustomFragment

24

Research Agenda

Status Quo 10/2005• Basic resource accounting infrastructure• Framework for decentralized adaptive service• Code loading framework based on JXTA• Integration of JXTA into AspectIX

Current work 10/2005 - 4/2006• Implementation of EDAS infrastructure • Design and implementation of placement and resource

accounting algorithms Next steps 4/2006-9/2006

• Evaluation of algorithms• Design and implementation of a membership service as a DAS • Implementation of example services: source code repository, ...

25

Conclusion

Grid like infrastructure Resource management support Decentralized adaptive service model Long term perspective provide a project management

infrastructure like SourceForge

Further information:

http://www.aspectix.org

Questions?

26

AspectIX Middleware

View• Stores IOR of the object• Manages active fragment

interfaces• Manages the currently active

fragment implementation Fragment Interface

• Automatically generated by the IDL compiler • Coordination mechanisms for safe exchange of the fragment

implementation Fragment Implementation

• The real implementation of the object's functionality• Service developer may define arbitrary fragment

implementation types

Fragment

FragmentImplementation

View

Fragm

entInterface

Client

27


Example Service: Version Control Service (VCS)• Replicated service core holds only information about where to

find the files and manages concurrent access • Files might be stored on different nodes of the Service

Environment or anywhere else• Ensures small service state and makes service easy

relocatable•

Service EnvironmentClient Node

1: getFile()2: url:getFile()

3: data:getFile()

28

Resource Limits at Node Level

Physical Limit • Provided by the underlying hardware

Node Limit • Bounds the resource usage of a node

Local Limit • Resources of a node assigned to

a service NodeLimit

LocalLimit

Node A

Service A (SE 1)

Service B (SE 2)

unassignedPhysical Limit

0%

100%

€

S(n) = s | s is serivce runnig on node n{ }

∀n ∈ Nodes :NodeLimit(n) ≥ LocalLimit(n,s)s|s∈S(n ){ }

∑

reserved

29

Resource Limits at Domain Level

Global Limit• The overall amount of resources dedicated to

a service environment

€

sse = s | s is service of service envrironment se{ }

∀se∈ SE :GlobaleLimit(se) ≥ LocalLimit(s)s|s∈sse∧s∈S(n ){ }

∑n |n∈Nodes{ }

∑


edas: providing an environment for decentralized adaptive services

Documents