- 1 - dpnm lab. autonomic network management may 2008 joon-myung kang distributed processing &...

- 1 - •DPNM Lab.Autonomic Network Management

Autonomic Network ManagementAutonomic Network Management

May 2008

Joon-Myung Kang

Distributed Processing & Network Management Lab.Dept. of Computer Science and Engineering

POSTECH, [email protected]

2008’ Future Internet


Contents1. Introduction

2. Autonomic Computing vs. Autonomic Networking

3. Autonomic Computing Environment

4. Research Challenges

5. Conclusions


The Problem of Complexity


Earlymajority

Earlymajority Convenient

Complexity

CCS

Moving


Network Environment

Core Network

Satellite

BroadcastNetworks (DAB, DVB-T)

The Internet

ISP

CDMA, GSM, GPRS

IP-based micro-mobility

Wireless LANs

Signalling Gateway

WAPAccounting

Context-aware informationCentre

Billing VHE SIP Proxy Server

WiBro, HSDPA

4G

Bluetooth Zigbee


A Traditional OSS/BSS

Autonomic Network Autonomic Network ManagementManagement

Autonomic Network Autonomic Network ManagementManagement


The Problem – Managing Complexity The Complexity of system design and management keeps increasing

– Stovepipe systems: best-of-breed functionality but integration nightmares– Increased technology overwhelms users and administrators

• Different devices have different programming models and interaction models• Different management tasks and integration types require different skill levels

The complexity of business is also increasing– People are demanding a pervasive presence– Many types of business LOSE MONEY if they can’t react fast enough– Varieties of threats, problems, and non-optimized behavior keeps increasing

Behavioral complexity is also increasing– Everything is interconnected, requiring different policies and functions– Too complex to predict, needs too high a skill level, not enough people!


Shortcomings - Infrastructural

Architectural issues– Data redundancy– Synchronization problems– Application authorization issues– Vendor and Application “lock in”

Integration issues– Isolated Data Silos– Administrative nightmare– Integration/customization

nightmare– Transition from legacy systems to

a new OSS


Stovepipes are Everywhere!


Some Effects of Complexity Expensive

– Cost of management by administrators is increasing (CAPEX, OPEX) Fragile

– Complex interdependencies make it hard to diagnose and fix problems– More prone to human error (additional cost)– Upgrades, performance tuning, re-purposing all suffer

Inflexible– Reluctance to change infrastructure once it is working– Does not support agile business (new software, business processes)

Worsening– Technology innovations typically exacerbate the problem, preventing product

innovations from being deployed

Solution: Self-managing systems

CAPEX(Capital Expenditure): expenditures creating future benefits.OPEX (Operational Expenditure): on-going cost for running a product, business, or system


More Effects – Constituency Separation Different constituencies have different terms, grammars, and needs

– Service Level Agreement meaning changes– Business “speak” vs. networking commands– Different representations (e.g., use of UML)

Relating network services and resources to business needs– Not reflected in EMS and NMS design– Lack of policy controlling allocation– Lack of ability to

• Incorporate new knowledge• React in a timely manner to changes


Motivation of Autonomic Management Complexity is growing

– Telecommunications industry has changed dramatically– Explosive growth of the Internet– The proliferation of mobile technologies– Fixed mobile convergence

Autonomic Network Management– Simplify network management process by automating and

distributing the decision making processes involved in optimizing network operation

– Enable expensive human attention to focus more on business logic and less on low level device configuration processes


What is Autonomic Computing? Autonomic

– Pertaining to operating system that responds automatically to problems or system failures.

Autonomic Nervous System– handles many crucial (but what we'd consider mundane) functions without

requiring any conscious though on our part.– When we run, it increases our heart and breathing rates. If we get too hot, it

redirects blood flow to "cool us". If we turn a light on or off, it adjusts our pupils for maximum visual activity.

– This leads us to describe an approach we call "autonomic computing.“ Autonomic computing

– A computing environment with the ability to manage itself and dynamically adapt to change in accordance with business policies and objectives.


Future Vision of Autonomic Computing? Machines will take over all management tasks,

rendering humans superfluous.

Machines will free system administrators to manage system at a higher level.

Wrong

Right


A Misconception About Autonomics Autonomics is not the four (in)famous self-functions (self-configure, -

protect, -heal, and –optimize)– These do not define an autonomic system– These are benefits resulting from an autonomic system

Autonomics is rooted in the following capabilities– Self-knowledge

• We can’t configure what we don’t know!– Ability to understand what is happening to our surroundings

• Learn from and reason about sensed data– Inspiration from biology, sociology, economics, …

• New ways to build and organize management functionality• Notion of maximization of “social welfare” of network service

– Link to business rules• Network services and resources adapt to change


Autonomic Computing Attributes

Increased Responsiveness

Adapt to dynamically changing environments

Business Resiliency

Discover, diagnose,and act to

prevent disruptions

OperationalEfficiency

Tune resources and balance workloads to maximize use of IT resources

Secure Information

and Resources

Anticipate, detect, identify,

and protect against attacks


Self-Configuration Configuration is governed by high-level policies

– BUSINESS objectives that specify WHAT is to be accomplished, but not HOW

New way of component interaction– New component adapts itself to how other components in the environment are

working– Existing components adapt to the presence of the new component– This cannot be done unless the component “knows itself” and its environment

DEN-ng models this interaction using the concepts of capabilities and constraints– DEN-ng: new version of Directory Enabled Networks (DEN)– Common information model to translate business rules into device configuration


Self-Healing Error detection and correction is HARD

– Network management is a good (well, bad) example of how NOT to do this– Predictive failure analysis is still magic. Why? Because there is no self-

knowledge!• Multiple incompatible knowledge sources• Self-healing affects ALL phases of the control loop

Autonomic computing detects errors– Based on self-knowledge (critical role of DEN-ng)– Once an error is known, it can be repaired– or can it? (critical role of DEN-ng)

• The problem is more difficult than this – component and system behavior can change• How do we know if the change is desired?


Self-Optimization Current research efforts are

– Oriented mostly on optimizing system performance– Just the aspect of tuning is complex

• Hardware and software dependencies, backwards compatibility, unclear semantics, etc.

• Tuning one component can adversely affect others

It’s also about– Using the right resources for a given task– Ensuring that tasks with higher business importance get the resources they need– Adapting to recognized environmental and service usage patterns– Learning through action


Self-Protecting Current problems that need to be addressed

– Detecting threats and malicious operations– Prevent the cascading of uncorrected errors– Transitioning from reactive to proactive systems

It can NOT be done without using an information model to represent the behavior– Capabilities and constraints– Patterns and roles

Interaction between hardware and software to protect a system


Any Self-Function Can Change The system must have models pre-defined

– Characteristics and behavior of itself– Needs of the user– Environmental constraints

Self-configuration can change– Based on different user needs– Based on environmental conditions

Decision-making requires policy to govern the system and its interactions– Reasoning enables deduction of cause and effect– Learning functions enable the system to improve


Just Add Water and Stir The integration of self-configuration, -healing, -protection, and –

optimization is critical– If that happens, these separate concepts will merge into more powerful concepts– For example, self-maintenance is the holistic combination of all four of these

principles• Using anti-virus software as an example, the system will proactively and upgrade its

functionality• Adjustment of workload in response to changing conditions (e.g., component failures)

and environment (e.g., new users running new apps)

Self-management is more than the sum of the self-management of its individual components


Autonomic computing architecture concepts

Autonomic computing system– A computing system

that senses its operating environment,

– models its behavior in that environment,

– and takes action to change the environment or its behavior.

- Autonomic Computing reference architecture -


Autonomic computing architecture concepts Managed resource

– An entity that exists in the run-time environment of an I/T system and that can be managed.

Touchpoint– The interface to an instance of a managed resource, such as an

operating system or a server.– A touchpoint implements sensor and effector behavior for the managed

resource.– And it maps to the sensor and effector interfaces to existing interfaces

Touchpoint autonomic managers– An autonomic manager that works with managed resources through their

touchpoints.


Orchestrating autonomic managers– An autonomic manager that works with other autonomic managers to

provide coordination functions. Integrated solutions console

– A technology that provides common, consistent user interface, based on industry standards and component reuse, and can host common system administrative functions.

– The IBM Integrated Solutions Console is a core technology of the IBM Autonomic Computing initiative that uses a portal-based interface.

Autonomic computing architecture concepts


Autonomic manager internal structure

– Knowledge• Standard data shared among the monitor, analyze, plan and execute

functions of an autonomic manager, such as symptoms and policies.

Autonomic computing architecture details


Autonomic computing architecture details Knowledge types

Solution Topology Knowledge

• Captures knowledge about components and constructions and configuration for business system.• For AM to install or configure components, installation and configuration knowledge is captured in a common installable unit.

Policy Knowledge

• A policy is a knowledge that is consulted to determine whether or not changes need to be made in the system.• Autonomic computing system requires a uniform method for defining the policies that govern decision-making for autonomic managers

Problem Determination Knowledge

• It includes monitored data, symptoms and decision trees. • As the system responds to actions, newly learned knowledge can be collected within AM.• Autonomic Computing System requires a uniform method for representing problem.


Autonomic computing architecture details Autonomic manager is a component that implements the control loop.

– Monitor Function• the function that collects, aggregates, filters and reports details (e.g.

metrics, topologies) – Analyze Function

• the function that models complex situations to understand current system state.

– Plan Function• the function that structures the actions needed to achieve goals and

objectives.– Execute Function

• the function that changes the behavior of the managed resource using effectors.


Autonomic computing architecture details Managed Resource

– A controlled system component. ex) a server, a router, a cluster or business application etc.

Manageability Interface– A service of the managed resource that includes the Sensor and the

Effector used by an autonomic manager. – This is for autonomic manager to monitor and control the managed

resource.


Autonomic computing architecture details Sensor

– A set of “get” operations that retrieve information about the current state of a managed resource.

– A set of management events (unsolicited, asynchronous messages or notifications) that can occur.

Effector– A collection of “set” operations that

allow the state of the managed resource to be changed in some important way.

– The operations that managed resource can use to make request.


Autonomic computing architecture details Evolving towards Autonomic Computing Systems

Increasing Autonomic Functionality

Instrument and Monitor

Analysis Closed Loop Closed Loop with Business Priorities

Monitoring is done by sensors in the technology, consistent with open standards

Analysis of monitored activity and associated actions performed IT staff

Actions can be taken autonomically by the technology, based on accumulated knowledge without human interaction

Control is exerted through policy statements

Analysis provided by autonomic functionality

Technology develops a plan recommending course of action

IT staff can choose to implement plans put forth by the technology

IT components able to analyze and execute a plan with minimal human intervention

Humans notified of conditions, plan, executed actions, and results

Manual

Little to no data available

Flow from analyze to execution of plan all manual

Trial-and-error

1 2 3 4 5


Autonomic computing architecture details An evolution, not a revolution


Current Network Management Deficiencies Aggregates of elements may exhibit behavior not predictable from

knowledge of individual behaviors– “The whole may be greater than the sum”

Causal determinacy still limited by simple statistical analysis and rudimentary correlation approaches– Precompiled diagnostic processes required to guide approaches

No ability of the system to “go beyond” precompiled knowledge and procedures– “Human-in-the-loop” is still the order of the day

All current techniques require “human-in-the-loop” back-end analysis– Extensive system, deployment, and technology knowledge– Drives us CAPEX and OPEX


Autonomic Networking Biology, Sociology, and Economics can Inspire Better Networks!

– Technical complexity: human body technology, devices– Business complexity: macro-economics e- and m-Commerce– Behavioral complexity: social interaction service composition– Operational complexity: healing anti-virus, configuration management


Business to System Interactions


Autonomic Control Loop


Autonomic Management System Conceptual Representation


Autonomic Management System The control loop

– Controlled by an autonomic manager that influences the deployment of the policies that effect decision making within the loop

DEN-ng (Directory Enabled Network new generation)– Finite state machines to model behavior and augmenting with ontological models

that embody semantic information that cannot be represented in the UML– A comprehensive information model for telecommunications, capturing

everything from business concepts (products, service level agreements, and customers) to low-level device functionality (packet marking, forwarding, and queuing)

Model-based policy processing component– Incorporate policy conflict analysis algorithms that (1) elaborate newly

defined/modified policies so that conflicts are easier to detect, (2) detect sets of policies that will or potentially could conflict, given certain network context, and (3) resolve conflicts by modifying or removing policies based on separate resolution policies or by referring back to the appropriate policy author for a decision


Autonomic Management System Autonomic Management Architecture


Autonomic Management System FOCALE

– Foundation Observation Comparison Action Learn rEason– Based on the observation that business objectives, user requirements, and

environmental context all change dynamically– Two control loop

• Maintenance control-loop– Used when no anomalies are found

• Adjustment control loop– Used when one or more policy reconfiguration actions must be performed, and/or new

policies must be codified and deployed

– It is unreasonable to assume that a single entity can maintain all the information required to realize the FOCALE control loops for large scale networks containing large numbers of heterogeneous devices

– FOCALE must be a distributed architecture, to the degree that even individual network devices may incorporate autonomic management software, implementing the maintenance and adjustment control loops


FOCALE Autonomic Network Management Architecture


FOCALE Autonomic Network Management Architecture

Main functional components– AM

• Independent of the vendor-specific functionality/data of the underlying managed resource(s), which facilitates easier communication between AMEs for coordination of management decision making

– MBTL• Indepth knowledge of the managed resource(s) to enable it to translate normalized

vendor-specific data gathered from the managed resource(s) into DEN-ng compliant vendor-neutral data to pass to the policy analyer/PDP and vice versa for configuration commands


Basis of MBTL Use of ontologies to identify cognitive equivalence across

heterogeneous data models


Prototype Implementation Implementation

– Single FOCALE AME that targets aspects of traffic conditioning in a simulated IP-based network of an ISP, over which customers are offered a small number of communications services

– Simulated network is very loosely coupled to the AME implementation

– Plan to replaced the simulation with real routers that will be configured by CLI commands generated by the AME and that will provide context information to the AME via SNMP

– OPNET based simulation is configured to emit information relating to network events and read and apply new router configurations generated by the AME


Prototype Implementation


Autonomic Networking Scenario Different networks, technologies, and business rules Conflicting resource and service requirements Different capabilities

Maximize network services according to business rules

across all customers


Autonomic Computing and Autonomic Networking

Autonomic Computing– Coined by IBM as an analogy to the autonomic nervous system– Attempts to manage the operation of individual pieces of IT infrastructure through the

introduction of an autonomic manager that implements an autonomic control loop in which the managed element and the environment in which it operates is monitored

– Autonomic control loop: monitor, analyze, plan, and execute components The vision of autonomic computing

– Self-managing IT infrastructure• Self-configure, self-optimize, self-heal, self-protect (self-* behavior)

Autonomic Networking– Burgeoning research area that seeks to integrate results from disciplines ranging from

telecommunications network management to artificial intelligence and from biology to sociology


Autonomic Computing and Autonomic Networking

Focus of research in autonomic network management– On the development of highly distributed algorithms that seek to optimize one or

more aspects of network operation and/or performance– Investigating the potential use of biologically-inspired algorithms and processes– Although work on the development of decentralized, self-management

algorithms is crucial, the deployment of these algorithms will not be sufficient– Equally important will be the flexible specification and enforcement of the goals

these algorithms collectively seek to achieve Policy-based Network Management

– Appropriate management paradigm to facilitate higher-level, human-specified cognitive decision making

Little work has been done to date on integrating distributed self-management algorithms with policy-based management


Autonomic Computing/Networking People express at a high level what they want the system to achieve

– Level could be business, or IT The system strives to manage its own behavior to optimally satisfy

these multiple criteria, given resource constraints– Resources: Hardware, software, cost– Tradeoffs among multiple criteria must be clear– Self-{configuration, healing, optimization, protection, …} are general classes of

behavioral criteria, but don’t define Autonomic Computing– The AC challenge is to develop the right technologies and architecture

People and self-managing systems will work together iteratively, in partnership with one another– People will do what they’re best at– Systems will gradually assume more management burden

• As they become more competent to do so• As people become more comfortable with this


Autonomic Computing Element (ACE)

An ACE is an abstractionthat enables an autonomic

system to manage the functionality of a new or

legacy managed resource

ACEs manage their ownbehavior in a standard way

Does not have autonomiccapabilities


How to Build an Autonomic Element

AIDE is part of IBM “Build To Manage” initiative

– WSDM (Manageability) support Eclipse tool generates java ‘stubs’

for multiple runtimes– Apache Muse (default)– OSGi, Eclipse 3.1– WebSphere App Server

Currently available at– http://www.alphaworks.ibm.com

In use by several IBM business partners and customers

AC toolkit: Autonomic Integrated Development Environment (AIDE)


IBM’s Autonomic Control Loop ACEs are building blocks that are arranged to provide higher-level system behavior


Autonomic Networking in the ACF


Autonomic Computing Element (ACE)


IBM Touchpoint vs. FOCALE MBTL


Autonomic Computing Scalability


Motorola Labs Autonomic Element


Scalability in an Autonomic Network


Autonomic Computing vs. Autonomic Networking Autonomic COMPUTING

– Assumes homogeneous elements– Control loop is straightforward– System adaptation based on ITIL (or eTOM)– Focused on self-config, -heal, -optimize, -protect– Policies determine what actions to take– Goal is to orchestrate static behavior

Autonomic NETWORKING– Assumes heterogeneous elements– Control loop adapts based on network, task, …– System adaptation based on DEN-ng & ontologies– Focused on self-governance and –knowledge, which drive all other self-*

functions– Policies determine overall behavior of the ACE– Goal is to orchestrate dynamic behavior


Research Challenges Key Challenges

– Providing self-* functionality• Mapping management information and commands in the legacy programming

models to an autonomic programming model• Orchestrate behaviour in a way that legacy devices can understand

– The complexity of applying various techniques may preclude their use in certain platforms

• Model-Driven Engineering• Aspect-Oriented Software Development• Reverse Engineering• Generative Programming

- 65 - •DPNM Lab.Autonomic Network Management65

Challenge: Architecture AE: How to coordinate multiple threads of activity?

– AE’s live in complex environments– Multiple task instances and types

– concurrent, asynchronous– Multiple interacting expert modules

AE: How to detect/resolve conflicts arising from– Internal decisions by independent expert modules– External directives (possibly asynchronous)– Internal policies vs. external directives

System-level: Enable more flexible, service-oriented patterns of interaction

– As opposed to traditional top-down, hierarchical systems management– Multi-agent architecture

– Communication– Representing and reasoning about needs, capabilities, dependencies

Managed Element

ES

Monitor

Analyze

Execute

Plan

Knowledge

Autonomic Manager

An Autonomic Element

ES

Define set of fundamental architectural principles from which self-* emerges


Policy: “Set of guidelines or directives provided to autonomic element to influence its behavior”

Challenge: Policy

Managed ElementES

Monitor

Analyze

Execute

Plan

Knowledge

Autonomic ManagerES

Human interface– Authoring and understanding policies– Avoiding or ameliorating specification errors

Developing a universal representation and grammar– Many different application domains, disciplines– Many different flavors of policy– Covers service agreements too?

Algorithms that operate upon policies (and agreements?)– Automated derivation of actions (e.g. planning, optimization)– Automated derivation of lower-level policies from high-level policies

• E.g. “Maximize profit from this set of service contracts”

Conflict resolution– Both design time and run time– Need to establish protocols, interfaces, algorithms


Three flavors of (policy = “decision-making guide”)

CurrentState

S

PossibleState

s1

PossibleState

s2

PossibleState

s3

a1

a2

a3

Action rule– If (S) then do a2

– Results implicitly in desired state 2

Goal– Achieve a most desired state 2

– Compute a2 most likely to result in 2

– Assumes that most desired state can be determined a priori Utility function

– Achieve state with maximal net value V() – C(aSd)– Benefit and burden of being explicit about value– States have intrinsic value; value of policy is a derived quantity

Element utility functions

System utility functions

Machinecode

Rules

Actions

ElementGoals

Workflows

[More levels of code hierarchy]

Higher-level specifications

GenerativePlanning

Optimization Modeling,

Optimization

Adapters,TranslatorsProgramming

Decision-theoreticPlanning


Challenge: Human-System Interface Develop new languages, metaphors and translation technologies that

enable humans to monitor, visualize, and control AC systems– Specify goals and objectives to AC systems, and visualize their potential effect– Techniques must be

– Sufficiently expressive of preferences regarding cost vs. performance, security, risk and reliability

– Sufficiently structured and/or naturally suited to human psychology and cognition to keep specification errors to an absolute minimum

– Robust to specification errors


Challenge: Learning

Single element level– AE needs to learn a model of itself and environment quickly; environment is noisy, and

dynamic in both state and structure– On-line, so exploration of the space can be costly and/or harmful– May be several hundreds of tunable parameters!

– Maybe only a few dozen are relevant, but which ones?– Some of them can only be changed upon reboot – is it worthwhile?

System level– Multi-agent system: several interacting learners – What are good learning algorithms for cooperative, competitive systems?

– What are conditions for stability?– What is sensitivity to perturbations?

– Opportunities for layered learning

Establish theoretical foundation for understanding and performing learning and optimization in multi-agent systems.


Challenge: Negotiation Develop and analyze

– Methods for expressing or computing preferences

– Negotiation protocols– Negotiation algorithms

Establish theoretical foundation for negotiation– Explore conditions under which to apply

– Bilateral– Multi-lateral (mediated, or not)– Supply-chain

– Study how system behavior depends on mixture of negotiation algorithms in AE population


Challenge: Control and Harness Emergent Behavior

Understand, control, and exploit emergent behavior in autonomic systems– How do self-*, stability, etc. depend on

– Behaviors and goals of the autonomic elements– Pattern and type of interactions among AEs– External influences and demands on system

– Invert relationship to attain desired global behavior– How?– Are there fundamental limits?

Develop theory of interacting feedback loops– Hierarchical– Distributed


Conclusions Autonomic Computing is a grand challenge, requiring advances in several fields of

science and technology– Policy, planning, learning, knowledge representation, multi-agent systems, negotiation,

emergent behavior– Human-system interfaces

Integrating these technologies to support self-management in complex, realistic environments is a research challenge in itself

– What are the best architectures and design patterns? Role of (multi-)agent systems?– Building system prototypes is key to developing and validating AC technology and

architecture Technology to Manage Technology

– Autonomic Computing/Networking does not replace people, it empowers them– Context-awareness is achieved by enabling the system to reconfigure itself, based on

• Changing business goals• Changing user needs• Changing environmental conditions

– Reconfiguration is model-driven, and uses ontologies to reason to build its conclusions– This is slowly becoming a reality, but should not be a panacea


How to realizeHow to realizeAutonomic Network Autonomic Network Management?Management?

Thank you for your Thank you for your attentionattention Joon-Myung Kang

[email protected]


References1. IBM Corporation, “An architectural blueprint for autonomic computing,” White paper, 20032. J.O. Kephart and D.M. Chess, “The vision of autonomic computing,” Computer, vol. 36, no. 1,

Jan. 2003, pp. 41-523. J.O. Kephart., “Research Challenges of Autonomic Computing,” Proceedings of the 27th

International Conference on Software Engineering, 20054. John Strassner, “Autonomic Systems and Networks: Theory and Practice”, NOMS 2008

Tutorial, 20085. R. Sterritt, “A concise introduction to autonomic computing,” Journal of Advanced Engineering

Informatics, Engineering Applications of Artificial Intelligence, Elsevier Publishers, Vol. 19, pp. 181-187, 2005

6. Brendan Jennings, “Towards Autonomic Management of Communications Networks”, IEEE Communications Magazine, Vol. 45, Issue 10, pp. 112-121, 2007

7. http://www.autonomiccomputing.org/8. http://www.research.ibm.com/autonomic/9. http://www.autonomic-communication-forum.org/10. http://dnac.org/autonomic-networking/

- 1 - dpnm lab. autonomic network management may 2008 joon-myung kang distributed processing &...

Documents