autonomic computing - 123seminarsonly.com · dream of pervasive computing … or nightmare! mobile...
TRANSCRIPT
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 1
CSIE50600: Advanced Database Systems
Mobile and Pervasive Computing
Autonomic Computing Shiow-yang Wu
Department of Computer Science and Information Engineering
National Dong Hwa University
(Part of the Slides are taken from Prof. Chung-Ta King of NTHU and Prof. Friedemann Mattern of ETH Zurich)
Mobile and Pervasive Computing Autonomic Computing 1
ReferencesJeffrey O. Kephart and David M. Chess. “The vision of autonomic computing”, IEEE Computer, Jan 2003.IBM Autonomic Computing Manifesto (http://www.ibm.com/research/autonomic)
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 2
Mobile and Pervasive Computing Autonomic Computing 2
Trillions of heterogeneous computing devices connected to the Internet
Dream of Pervasive Computing …
or Nightmare!
Mobile and Pervasive Computing Autonomic Computing 3
Core of the ProblemComplexity in systems themselves and in the operating environment
As systems become more interconnected and diverse, architects are less able to anticipate and design interactions among componentspush to runtime, late bindinge.g., hot-plug, JVM, JIT compilation, service discovery, mobile agents, …
Complexity managementhuman intervention and IT costsEven in uncertain economic times, demand for skilled I/T workers is expected to increase by over 100 percent in the next six years.
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 3
Mobile and Pervasive Computing Autonomic Computing 4
Need Complexity ManagementBut complexity is beyond that human can handleHuman out of the control loop autonomicEven though we are moving along this direction, is there any systematic way of addressing this issue?
Autonomic ComputingComputer systems should ack like our autonomic nervous system that governs our heart rate and body temperature, thus freeing our conscious brain from the burden of dealing with these and many other low-level, yet vital, functions.
Mobile and Pervasive Computing Autonomic Computing 5
Alan G. Ganek, Vice PresidentAutonomic ComputingIBM Software Group
http://www.ibm.com/autonomic/
Autonomic Computing
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 4
Mobile and Pervasive Computing Autonomic Computing 6
Directory Directory and Security and Security
ServicesServicesExistingExisting
ApplicationsApplicationsand Dataand Data
BusinessBusinessDataData
DataDataServerServerWebWeb
ApplicationApplicationServerServer
Storage AreaStorage AreaNetworkNetwork
BPs andBPs andExternalExternalServicesServices
WebWebServerServer
DNSDNSServerServer
DataData
Dozens of systems and applications
Hundreds of components
Thousands of tuning
parameters
Complex Heterogeneous Infrastructures Are a Reality!
Mobile and Pervasive Computing Autonomic Computing 7
Industry TrendsAdministration of systems is increasingly difficult
100s of configuration, tuning parameters for DB2
Heterogeneous systems are increasingly connectedIntegration becoming ever more difficult
Architects can't plan interactions among componentsIncreasingly dynamic; frequently with unanticipated components
More burden must be assumed at run timeBut human administrators can't assume the burden
6:1 cost ratio between storage admin and storage40% outages due to operator error
Need self-managing computing systemsBehavior specified by sys admins via high-level policiesSystem and its components figure out how to carry out policiesself-managing, self-diagnostic, and transparent to the user
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 5
Mobile and Pervasive Computing Autonomic Computing 8
Autonomic Computing Vision“Intelligent” open systems that…
Manage complexity“Know” themselvesContinuously tune themselvesAdapt to unpredictable conditionsPrevent and recover from failuresProvide a safe environment
Self-management:free administrators from details of operationsprovide peak performance 24/7Concentrate on high-level decisions and policies
Flexible, accessible (always on), and transparent
Mobile and Pervasive Computing Autonomic Computing 9
Increase ResponsivenessAdapt to dynamically changing environments
Business ResiliencyDiscover, diagnose, and act to prevent disruptions
Operational EfficiencyTune resources and balance workloads to maximize use of IT resources
Secure Information and ResourcesAnticipate, detect, identify, and protect against attacks
Self-managing Systems That …
Aware/Proactive
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 6
Mobile and Pervasive Computing Autonomic Computing 10
Self-Configuring Example:DB2 Configuration Advisor
H/Wdetection
Expertheuristics
Basicdescription
Configurationmodel
Configurationsettings
Mobile and Pervasive Computing Autonomic Computing 11
Problem Problem DatabaseDatabase
Electronic Response
Dispatch CEService agent
detects a hardware problem
Voice Support
Sends error symptomsto IBM
Sends Symptoms
for Diagnosis
DataDataCatcherCatcher
AnalysisAnalysisofof
Problem Problem RecordRecord
Fully Automatic
Faster problem resolutionHigher availability/resiliencyLower maintenance cost
""IBM's eService Agent allows me to sleep soundlyknowing the system is being monitored 24x7."
Alex Tambellini, 7-Eleven Stores Pty Ltd.
Self-Healing Example: IBM Electronic Service Agent
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 7
Mobile and Pervasive Computing Autonomic Computing 12
InternetInternet
Appliance Appliance ServersServers
Web Web Application Application
ServersServersData and Data and
Transaction Transaction ServersServers
Internet/Internet/ExtranetExtranet
Business Business PartnersPartners
SelfSelf--tuning, endtuning, end--toto--end performance end performance managementmanagement
Dynamic allocation of network resourcesDynamic allocation of network resourcesWorkload balancing & routingWorkload balancing & routingCross platform reportingCross platform reportingPolicyPolicy--based for various classes of users & applicationsbased for various classes of users & applications
Heterogeneous, distributed components working together
Self Optimizing: Enterprise Workload Management
Mobile and Pervasive Computing Autonomic Computing 13
Automate incident responseProtect systems and dataHelp prevent service disruptions
Risk MgrIDS Rules
Event Database
CorrelationEngine
Intrusion Detection System (IDS)
RouterWebServer
Firewall
ApplicationServer Intrusion
Detection
InternetIntranet
RiskManagerSecurity Event
ApplicationServer
"The Tivoli security management software portfolio is helping our clients extend their businesses to the
Internet while providing security and privacy..."Mark Ford, Principal
Deloitte & Touche
Rapid / automated analysisof complex situations
Self-Protecting Example: IBM Tivoli Risk Manager
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 8
Mobile and Pervasive Computing Autonomic Computing 14
Evolving towards Self-management
Mobile and Pervasive Computing Autonomic Computing 15
THE 8 ELEMENTSSystem needs to "know itself“Configure and reconfigure itself under varying (and in the future, even unpredictable) conditionsNever settles for the status quo - it always looks for ways to optimize its workingsMust perform something akin to healing (self-healing)Must be an expert in self-protectionMust know its environment and the context surrounding its activity, and act accordinglyMust function in a heterogeneous world and implement open standardsAnticipate the optimized resources needed while keeping its complexity hidden
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 9
Mobile and Pervasive Computing Autonomic Computing 16
Manual Autonomic
Ben
efits
Skill
sC
hara
cter
istic
s
ManagedLevel 2
PredictiveLevel 3
AdaptiveLevel 4
AutonomicLevel 5
BasicLevel 1
Multiple sources of
system generated data
Requires extensive,
highly skilledIT staff
Basic Requirements
Met
Evolving to Autonomic Computing
Mobile and Pervasive Computing Autonomic Computing 17
Manual Autonomic
Ben
efits
Skill
sC
hara
cter
istic
s
BasicLevel 1
PredictiveLevel 3
AdaptiveLevel 4
AutonomicLevel 5
Multiple sources of
system generated data
Requires extensive,
highly skilledIT staff
Basic Requirements
Met
ManagedLevel 2
Consolidationof data and
actions through
managementtools
IT staffanalyzes andtakes actions
Greater system
awarenessImproved
productivity
Evolving to Autonomic Computing
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 10
Mobile and Pervasive Computing Autonomic Computing 18
Manual Autonomic
Ben
efits
Skill
sC
hara
cter
istic
s
BasicLevel 1
ManagedLevel 2
AdaptiveLevel 4
AutonomicLevel 5
Multiple sources of
system generated data
Requires extensive,
highly skilledIT staff
Basic Requirements
Met
Consolidationof data and
actions through
managementtools
IT staffanalyzes andtakes actions
Greater system
awarenessImproved
productivity
PredictiveLevel 3
Systemmonitors,
correlates and recommends
actions
IT staffapproves and
initiates actions
Reduced dependency on
deep skillsFaster/better
decision making
Evolving to Autonomic Computing
Mobile and Pervasive Computing Autonomic Computing 19
Manual Autonomic
Ben
efits
Skill
sC
hara
cter
istic
s
BasicLevel 1
ManagedLevel 2
PredictiveLevel 3
AutonomicLevel 5
Evolving to Autonomic Computing
Multiple sources of
system generated data
Requires extensive,
highly skilledIT staff
Basic Requirements
Met
Consolidationof data and
actions through
managementtools
IT staffanalyzes andtakes actions
Greater system
awarenessImproved
productivity
Systemmonitors,
correlates and recommends
actions
IT staffapproves and
initiates actions
Reduced dependency on
deep skillsFaster/better
decision making
AdaptiveLevel 4
System monitors,
correlates and takes action
IT staff manages
performance against SLAs
Balanced human/system
interactionIT agility and
resiliency
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 11
Mobile and Pervasive Computing Autonomic Computing 20
Manual Autonomic
Ben
efits
Skill
sC
hara
cter
istic
s
BasicLevel 1
ManagedLevel 2
PredictiveLevel 3
AdaptiveLevel 4
Multiple sources of
system generated data
Requires extensive,
highly skilledIT staff
Basic Requirements
Met
Consolidationof data and
actions through
managementtools
IT staffanalyzes andtakes actions
Greater system
awarenessImproved
productivity
Systemmonitors,
correlates and recommends
actions
IT staffapproves and
initiates actions
Reduced dependency on
deep skillsFaster/better
decision making
System monitors,
correlates and takes action
IT staff manages
performance against SLAs
Balanced human/system
interactionIT agility and
resiliency
AutonomicLevel 5
Integrated components dynamically managed by
business rules/policies
IT staff focuseson enabling
business needs
Business policy drives IT
managementBusiness agility and resiliency
Evolving to Autonomic Computing
Mobile and Pervasive Computing Autonomic Computing 21
IBM’s Architecture ModelIntelligent control loop:
Implementing self-managing attributes involves an intelligent control loop
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 12
Mobile and Pervasive Computing Autonomic Computing 22
Control Loops Delivered in 2 Ways
Combinations of Management Tools
Recourse Provider
Mobile and Pervasive Computing Autonomic Computing 23
3 Layers of Control Loop Management
Composite resources tied to business decision-making
Composite resources decision-making, e.g., cluster servers
Resource elements managing themselves
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 13
Mobile and Pervasive Computing Autonomic Computing 24
Autonomic Element -StructureFundamental atom of the architecture
Managed element(s)Database, storage
Autonomic managerResponsible for:
Providing its serviceManaging ownbehavior inaccordance withpoliciesInteracting with other autonomic elements
An Autonomic Element
Monitor
Analyze
Sensors
Execute
Plan
Effectors
Knowledge
Aut
onom
icM
anag
erM
anag
edE
lem
ent
Sensors Effectors
Mobile and Pervasive Computing Autonomic Computing 25
Alerts, events & problem analysis request interface
SLA/Policy interface, interprets & translates into "control logic"
PlanPolicy Transforms
Plan Generators
Policy InterpreterAnalyze
Execute
Service Dispatcher
Distribution Engine
Scheduler Engine
Workflow Engine
Monitor
Metric Managers
Filters
Simple CorrelatorsKnowledge
Policy
CalendarTopology
Recent Activity Log
Sensors Effectors
Rules Engines
Analysis Engines
Policy Validations
Policy Resolution
Autonomic Manager -Substructure
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 14
Mobile and Pervasive Computing Autonomic Computing 26
Autonomic Elements -InteractionRelationships
Dynamic, ephemeralFormed by agreement
May be negotiated
Full spectrumPeer-to-peerHierarchical
Subject to policies
Mobile and Pervasive Computing Autonomic Computing 27
Engineering ChallengesLife cycle of an autonomic element
Design, test, and verification.Installation and configuration.Monitoring and problem determination.Upgrading.Managing the life cycle.
Relationships among autonomic elementsSpecification.Location.Negotiation.Provision.Operation.Termination.
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 15
Mobile and Pervasive Computing Autonomic Computing 28
Scientific ChallengesBehavioral abstractions and modelsRobustness theoryLearning and optimization theoryNegotiation theoryAutomated statistical modeling
Mobile and Pervasive Computing Autonomic Computing 29
Multiple Contexts for Autonomic Behavior
System Elements
(Intra-elementself-management)
Groups of Elements
(Inter-elementself-
management)
Business Solutions
(Business Policies, Processes, Contracts)
ServerFarm
EnterpriseNetwork
StoragePool
Customer Relationship Management
EnterpriseResourcePlanning
Servers Storage NetworkDevices Middleware
DatabaseApplications
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 16
Mobile and Pervasive Computing Autonomic Computing 30
Mapping to IT Processes
Mobile and Pervasive Computing Autonomic Computing 31
Levels of Maturity
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 17
Mobile and Pervasive Computing Autonomic Computing 32
Enabled capabilities
Core technologies
Administrative Console
Policy Infrastructure
Data Collection (Logging/Tracing)
Infrastructure Provisioning
Install/Dependency Management
Heterogeneous Workload Management
Solution Management
Policy-based Management
End-to-end Problem Determination
Automated Root Cause AnalysisAuto-Update
Identity/Security ManagementAuto-Detection
Dynamic Provisioning
Autonomic Computing Requires Core Technologies
Mobile and Pervasive Computing Autonomic Computing 33
Integrated Solutions Console for Common Sys Administration
Value:One consistent interface across product portfolioCommon runtime infrastructure and development tools basedon industry standards, component reuseProvides a presentation framework for other autonomic core technologies
...n
Customer pain point:Complexity of operations
Standards-based: J2EE, JSR168
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 18
Mobile and Pervasive Computing Autonomic Computing 34
Log and Trace Tool for Problem Determination
Value:Introduces standard interfaces and formats for logging and tracingCentral point of interaction with multiple data sourcesCorrelated views of dataReduced time spent in problem analysis
Analysis Engine
Data Exploiters
Data Producers
ISC
StandardInterface
LoggingAgent
Common situations and data model
BLog
Embedded adapter
....
Data Store
LoggingAgent
Common situations and data model
eServerLog
Embedded adapter
LoggingAgent
Common situations and data model
ALog
Embedded adapter
Collector Collector....
Parser
Parser
Parser
Viewer....
Customer pain point:Difficulty in analyzing problems in multi-component systems
Standards-based:JSR47, Apache
Mobile and Pervasive Computing Autonomic Computing 35
Install/Config Package for Solution Install
Value:One consistent software installation technology across all productsConsistent and up-to-date configuration and dependency data, key to building self-configuring autonomic systemsReduced deployment time with less errorsReduced software maintenance time, improved analysis of failed system components Component-based install for IBM and non-IBM products
Install package developer
Meta-DataNameUUIDVendorVersion
Configuration PropertiesInstall InputRuntime Attributes
DependenciesHW, SW, OS, ConfigurationExtensions
Install ActionsExtensions
Verification ActionsExtensions
Configuration ActionsExtensions
Package Structure
Product Files (binaries, etc.)Product Files (binaries, etc.)
Deployment Descriptor
Deployment Descriptor
Verification Actions
Verification Actions
DependencyCheckers
DependencyCheckers
Custom Extensions
InstallActionsInstallActions
Configuration Actions
Configuration Actions
GUI Interface
GUI Interface
Customer pain point:Difficulty of deployment in complex systems
Standards-based:OGSA, Web Services
Partnering with InstallShield
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 19
Mobile and Pervasive Computing Autonomic Computing 36
Policy Tools for Policy-based Management
Value:Uniform cross-product policy definition and management infrastructure, needed for delivering system-wide self-management capabilitiesSimplifies management of multiple products; reduced TCOEasier to dynamically change configuration in on-demand environment
Customer pain point:Complexity of product and systems management
Standards-based:DMTF, OASIS, OGSA
Adaptation
Definition
ValidationLocal
Repository
Distribution
EnforcementPoint
Push or pull
Push or pull
Activate
Implement
MONI
TOR
Facts
Analysis
Resource
…
…
EnforcementPoint
Resource Resource
Mobile and Pervasive Computing Autonomic Computing 37
Implementing Autonomic Managers
Value:Components to simplify the incorporation of autonomic functions into applications
Building blocks for self-managementMonitoring, analysis, planning and execution components Including autonomic computing technologies, grid tools, and services
PluggableDefines interfaces and provides implementations for each major toolkit component
Customer pain point: How to implement end-to-end autonomic solutions
Standards-based:OGSA, W3C
Mobile and Pervasive Computing Lecture09: Autonomic Computing
Note 20
Mobile and Pervasive Computing Autonomic Computing 38
Summary of Autonomic Computing ArchitectureBased on a distributed, service-oriented architectural approach, e.g., OGSA
Every component provides or consumes servicesPolicy-based management
Autonomic elementsMake every component resilient, robust, self-managingBehavior is specified and driven by policies
Relationships between autonomic elements Based on agreements established and maintained by autonomic elementsGoverned by policiesGive rise to resiliency, robustness, self-management of system
Mobile and Pervasive Computing Autonomic Computing 39
Summary"Civilization advances by extending the number of important operations which we can perform without thinking about them."
- Alfred North Whitehead