ibm initiatives in autonomic computing and policy

1

© 2002 IBM Corporation© 2003 IBM Corporation

Autonomic Computing

IBM Initiatives in Autonomic Computing and Policy

Alan Ganek

Vice President, Autonomic Computing

© 2003 IBM Corporation 2

Today’s Complex Infrastructure

FirewallServers

RoutersSwitches

UI Data

DNSServers

CachingAppliances

Web Servers

SSLAppliances

ApplicationServers

Security &Directory Servers

File/PrintServers

LAN Servers

Database Servers

Business Data

PCs

UNIX

UNIX

PCs

Mainframe


IT Challenges

Complex, heterogeneous environments

Outages of mission-critical systems cost quite a bit

Poorly documented legacy applications make it painful to diagnose and resolve complex cross-product problems

25-50% of IT resources are spent on problem

determination and resolution

Up to 40% of today’s outages are unscheduled stoppages

The skills needed to do manual cross-product problem determination are scarce and expensive

Missing or Loss of critical data is immeasurable

Outages & unscheduled work leads to saturation on backup systems & power systems


Autonomic Vision

“Intelligent” open systems that: Manage complexity Know themselves Continuously tune themselves Adapt to unpredictable conditions Prevent and recover from failures Provide a safe environment

Focus on business, not infrastructure

Providing customer value Increased return on IT investment Improved resiliency and quality of service Accelerated time to value


Autonomic Computing Attributes

Increased Responsiveness

Adapt to dynamically changing environments

Business Resiliency

Discover, diagnose,and act to prevent

disruptions

OperationalEfficiency

Tune resources and balance workloads to maximize use of IT resources

Secure Information

and Resources

Anticipate, detect, identify, and protect

against attacks

Self-managing systems that deliver:


Levels of Autonomic Maturity

• Manual processes• Reactive and tactically

aligned• Resource intensive

• Automated best practices• Proactive and Predictive

• Automated provisioning

• Autonomic, self managed environment

• Dynamic provisioning

IT Focus Business Focus

1

2

3

4

5

BasicBasic

Managed

Predictive

Adaptive

Autonomic

Element Management

Centralized Management

On Demand Management

Evolutionaryapproach;

Revolutionaryoutcome


Open Standards

The Big Picture of Autonomic Computing Technology

Autonomic core capabilities

Products delivering autonomic features

Business policy

Au

ton

om

ic C

om

pu

tin

g A

rch

itec

ture

Solution Install

Problem Determination Admin Console

Policy

Resource Provisioning

• Define a base reference architecture model which creates a common vernacular for autonomic computing

• Deliver core infrastructure technologies that provide for an open framework for the industry

• Deliver products with built-in autonomic capabilities

• Create and leverage open standards for autonomic computing

Workload Management


Autonomic Computing Architecture Overview

Managed Element

Sensors

Resource Manager

Effectors

Sensors Effectors

Knowledge

Plan

ExecuteMonitor

Analyze

Data Action

Autonomic Manager

Manageability Interface


Autonomic control loops: next step evolution

Autonomic featuresLocal view

Global environment view and knowledge

USER RESPONS

E TIMEAVAIL.

RESOURCE

BUSINESS SLA

POLICY


Policy and Autonomic Managers Policy: a set of considerations

designed to guide decisions on courses of actions (Calo, et al.)

In autonomic (self-managing) systems, policies reflect human judgement.

In an autonomic systems, policies are considerations, stored as knowledge, that guide the actions of autonomic managers

Policies can tell an autonomic manager what to achieve (goals) or how to achieve it (actions)

Sensors Effectors

Knowledge

Plan

ExecuteMonitor

Analyze

Autonomic Manager

PolicyPolicy

Policy

Policies allow administrators to configure autonomic systems.Policies allow administrators to configure autonomic systems.


Policy in Practice: eWLMWorkflow -centric management model

Resource allocation based on goal achievementService class goal exceptions trigger adjustment

algorithmsPredictive algorithms avoid nonproductive

adjustmentsBusiness importance determines problem

prioritizationBusiness importantance determines winners and

losers

Internet

Appliance Servers

Web Application

Servers

Data and Transaction

Servers

Business Partners

Employee Benefits

Employee SchedulingEmployee

Scheduling

Policy Based Business Priority based goals90% less than 2 seconds, critical to the

businessAvg response time 3 seconds, important, but

not criticalDefined by user group and/or business

processDynamic, learned topology controls

No static definitions of servers/application environments


Policy in an IT Infrastructure

Directory Directory and Security and Security

ServicesServicesExistingExisting

ApplicationsApplicationsand Dataand Data

BusinessBusinessDataData

DataDataServerServer

WebWebApplicationApplication

ServerServer

Storage AreaStorage AreaNetworkNetwork

BPs andBPs andExternalExternalServicesServices

WebWebServerServer

DNSDNSServerServer

DataData

Dozens of systems and applications

Hundreds of components

Thousands of tuning

parameters

Access control policies

Access control policies

Quality of

service policies

Quality of

service policies

Business

resiliency & data retentio

n policies

Business

resiliency & data retentio

n policies

Utilization

policies

Utilization

policies

Security policiesSecurity policies


(Some) Key issues in policy for autonomic computing

Business driven: how autonomic manages can manage against high-level business policies.

– Reduce the complexity by managing toward business objectives

Canonical specification: autonomic systems are heterogeneous. – Multiple policy specifications leads to overly complex, fragile systems.

Orchestration: how policies resolve conflicting systems needs.

Transformation, validation, user centered design, and many, many more…


Anatomy of an AC Policy: a canonical policy language Four common concepts identified:

– Scope– Specifications to identify what is or is not subject to the intent.

– Precondition– Specifications to express when a policy is to be applied or is active.

– Business Value– Specifications to express utility functions to make economic trade offs

– Decision – Specifications to describe observable behavior or objective.

Designed by adopting concepts from prevalent policy languages

– eWLM, IETF/DMTF standard, WS-Policy, WSLA


AC Policies and WS-PolicyWS-Policy defines a container for assertions.

WS-Policy Document

<wsp:policy>

</wsp:policy>

AC policies are domain assertions

ScopePreconditionBusiness ValueDecision



AC Policies are assertions contained in WS-Policy documents


Defining and Executing Policies

Policies are used by an autonomic manager to control a managed resource.

Therefore, policies must relate to the resource’s sensors and effectors.

At definition time, policies should be associated with a resource type

At run-time, autonomic managers apply the policies appropriate for the types of resource under management

Observations:Observations:

The next two charts illustrate these points.The next two charts illustrate these points.


A 4-tuple is a type of “K”.

Defining Policies

Managed Resource

EffectorSensor

ManagedResource’sdescriptor

Policy 4-tuple

Scope:

Condition:

Business value:

Decision:

List of SensorsList of Effectors

E

ES

M

A P

Autonomic Manager

K

4-tuple

The managed resource’s descriptor enumerates a resource’s sensors and effectors

The policy is a function of the MR’s externalized sensors and effectors.


How autonomic managers work with policy

Managed Resource

Execute

EffectorSensor

Monitor

Analyze Plan

Autonomic Manager

EffectorSensor

Knowledge

Monitors for events. Deliver to “K””.Monitors for events. Deliver to “K””.

Retrieves state and consults K as needed for policy evaluation.

Retrieves state and consults K as needed for policy evaluation.

Performs the command as indicated by the policy decision

Performs the command as indicated by the policy decision

Evaluates the policy conditions and business value

Evaluates the policy conditions and business value

CBE

“A” gets data from “K”.“A” gets data from “K”.

CBE


WebSphere Application Server v5

an example – policy-driven self-optimizing solution:IBM Server Allocation for WebSphere

IBM Server Allocation for WebSphere

ForecasterAdvice Application

Internet

Load Balancer WebSphere Transactional Grid

Mid priority

Low priority

High priority

DatabaseServer

Account Manager

Stock Trading

ParallelServices

ApplicationProvisioning

•Multiple WebSphere transactional applications

•Multiple Service Level objectives

•Dynamic and automated application provisioning


Mustang: Objective and Approach

Objective– Maintain service level objectives (e.g. response time) in the face of

varying workload

Approach – Configuration management for rapid resource addition/removal

– Online capacity planning to estimate resource needs

– Policy enabled controller


Mustang: Policy enabled self-optimizing controller

ExecuteMonitor

Analyze PlanAutonomic Manager

Knowledge

CommandCommandRetrieve State

Retrieve State

SPEC driver

trade driver

batch driver

WAS 5.1WAS 5.1WAS 5.1


Server pool

DB2 8.1

DB2 8.1


EffectorStart/Stop Server

SensorrespTime, thruput

EffectorSensor

Managed Resource

Condition Evaluator

Data Provider

Optimizer

Server Agent Notification

WSLAPolicy

Policy

Configuration Transition

Engine


Sample policy on min nodes: determined by time-of-day

During daytime (6am -6pm) Trade has priority and is assigned high min #nodes (not shown: SPEC assigned min #nodes = 1)


More sample Mustang time-of-day policies

During nighttime (6pm –6am) SPEC has priority and is assigned high min #nodes (not shown: Trade assigned min #nodes=1)


WAS Server deployment transition: night day

SPEC has 5 WAS nodes

{Trade has 1 WAS nodes

NIGHTTIME

SPEC has 1 WAS nodes

Trade has 5 WAS nodes

DAYTIME

}


Unity system and policy issues

Unity is an autonomic system integration project Focus issues:

– Interactions between autonomic managers– Conflict resolution– Resource allocation– Self-configuration (dynamic discovery and runtime binding)– Self-healing

Policy focus:– Extending policy language and framework to support utility functions– Utility functions enable:

– Principled policy decomposition– Conflict resolution– Continual optimization


An Autonomic System

ResourceArbiter

Registry

PolicyRepository

Application Environment

Application

Manager

Database

Router

Server

ServerDatabase

Router

ServerStorage

Application

Manager

Unity ArchitectureAC system composed of interacting autonomic components

Demand Demand

Sentinel

. . .

Application Environment


Resource Allocation in Unity (utility-based approach)

Demand

ResourceArbiter

Application

Manager

Application

Manager

Demand

PolicyRepository

UI/Editor

. . .Server

ServerServer Server Server

The policy repository holds and distributes policies that specify a business value for the behaviors of each application.

The policy repository holds and distributes policies that specify a business value for the behaviors of each application.

Scenario: an overall solution contains a number of applications, and resources that must be allocated among the applications.

Scenario: an overall solution contains a number of applications, and resources that must be allocated among the applications.

The application managers use these policies to tell the resource arbiter the likely effects of changing allocations.

The application managers use these policies to tell the resource arbiter the likely effects of changing allocations.

The utility functions provide a principled basis for resolving the resource contention, and for converting business-value policies into action.

The utility functions provide a principled basis for resolving the resource contention, and for converting business-value policies into action.

The arbiter allocates the available resources in a way that maximizes the expected business value of the overall solution.

The arbiter allocates the available resources in a way that maximizes the expected business value of the overall solution.


Unified Policy Management

Policy Editing Tools

Managed Element

Decision FederatorAPIs

Decision Federator APIs

Decision Maker APIs

Policy Definition

(Policy Grammar)E

ES

M

A PAutonomic Manager

K

Decision Maker

RMES RM

ES

DecisionFederator

Scope: database

Preconditions: Log file size > 100,000 entries

Business value: high

Decision (Action): incremental backup

Scope: database

Preconditions: Log file size > 100,000 entries

Business value: high

Decision (Action): incremental backup

Managed Element

Standardized approach to enable integration of policy-based solutions Independent of any product infrastructure Rich syntax for the definition of policies Persistence and distribution of policy definitions Allow policy guidance to be stated in a variety of forms (Results, Actions, and Goals)


Research Challenges

Human Interface Authoring and understanding policies Avoiding or ameliorating specification errors

Developing a universal representation and grammar Many different application domains, disciplines Many different flavors of policy Covers service agreements as well ?

Algorithms that operate upon policies (and agreements ?) Automated derivation of actions (e.g. planning, optimization) Automated derivation of lower-level policies from high-level policies (e.g. maximize

profit from this set of service contracts)

Conflict Resolution Both design time and runtime Need to establish protocols, interfaces, algorithms

Policy: Set of guidelines or directives provided to autonomic element to influence its behavior.Policy: Set of guidelines or directives provided to autonomic element to influence its behavior.


The Journey Has Started

Products, technologies and services are available today

IBM is working with academia, business partners and standards organizations to

develop open standards for self-managing systems

Broad IT industry participation is needed – this is an industry-wide initiative

Policy is a critical element and one of the most challenging aspects of AC

Innovation & Collaboration are a must!! Aggressive research is essential!!

Freeing people to focus on their business instead of

their infrastructure


Questions?

Useful info– [email protected]

– URLs– www.ibm.com/autonomic

– www.research.ibm.com/autonomic

– www.alphaworks.ibm.com/autonomic

ibm initiatives in autonomic computing and policy

Documents