Second GSFC/IEEE Workshop on Radical Agent
Concepts
Second GSFC/IEEE Workshop on Radical Agent
Concepts
Genetically Modified Software: Realizing Viable Autonomic Agency
A.G. Laws, A. Taleb-Bendiab & S.J. WadeLiverpool John Moores University, UK
Introduction & Overview Introduction & Overview
• Our search for a unifying conceptual framework and design model for the development of autonomic computing systems
• Management Models for Self-Managing Systems– Managerial Cybernetics and the Viable System Model
• A Cybernetic Reference Model for Autonomic Systems– and some drawbacks
• Some classical cybernetics and their implications• More speculatively
– Some small scale experiments in adaptation with genetic algorithms
– Lessons, limitations and possibilities– Dynamic Model Development for Open Environments
using LCS• Conclusions
The Viable System Model The Viable System Model • Underpinned by “classical” cybernetics
though structured using the human CNS as a model
• Beer’s VSM implements a control & communication structure via hierarchies of (homeostatic) feedback loops
• 6 major systems ensure ‘viability’ of the system
– Implementation S1– Monitoring S2– Audit S3*– Control S3– Intelligence S4– Policy S5
• Offers an extensible, recursive, model-based architecture, devolving autonomy to sub-systems
AutonomicSystems
AnticipatorySelf-awarenessDeliberative
System 5Policy
System 4Intelligence
System 3Control
System 3*Audit
System One (S1)
5
4
3
2
2
2
2
3*
1
Org
an
iza
tion
al E
nvi
ron
me
nt
System 2Coordination
System 2Coordination
System One (S1)
5
3
43*
1
System 2Coordination
System One (S1)
5
3
43*
1
System 2Coordination
Where this begins to get interesting
T h ree *S p o ra d ic A u d it
a lw a y s sta n d s fo r
T w o
T w o
T w oa n ti
o sc illa to ry
T h reein sid e & n o w se lf-o rg a n iza tio n a u to n o m ic reg u la tio n
F o u ro u tsid e & fu tu re se lf-re fe ren ce sim u la tio n p la n n in g
F iv e p o licy
a lg ed o n ic
lo ca l reg u la to ry
lo ca l reg u la to ry
O n e
O n e
5
5
3
4
4
3
2
2
2
2
3 *
3 *
1
1
• The recursive nature of the underlying model suggests that each System One should develop as a Viable System Model in its own right.
• If the recursion is pursued below the level of the human software team then we arrive at the level of the software system.
• This strongly implies that the software system should assume the same overall management structure as the human software process
• Consequently, we appear to have a theoretically supported management architecture both for the human process and for “self-managing autonomic software”
• This also fits closely with IBM’s “dynamic management by business rules/policies” aim for autonomic systems
BasicLevel 1
ManagedLevel 2
PredictiveLevel 3
AdaptiveLevel 4
AutonomicLevel 5
Multiple sources ofsystem generateddata
Requires extensive,highly skilled I T staff
Consolidation of datathrough managementtools
IT staff analyzes andtakes action
System monitors,correlates andrecommends actions
IT staff approvesand initiates actions
System monitors,correlates and takesaction
IT staff managesperformance againstSLAs
Integratedcomponentsdynamically managedby business rules/policiesIT staff focuses onenabling businessneeds
Geater systemsawareness
Improvedproductivity
Reduceddependency on deepskills
Faster, betterdecision-making
IT agility andresiliency withminimal humaninteraction
Business policy drivesIT management
Business agility andresiliency
ManualAutonomic
From IBM Global Services and Autonomic Computing, IBM White Paper, October 2002.
Bratman et al’s IRMA Architecture
• Bratman et al’s Intelligent Resource-Bounded Machine Architecture (IRMA), a classic BDI agent model
• It is noticeable that many of the elements contained in the model correlate directly to elements of the VSM
• So– Planning & Means-Ends
Reasoning S3– Opportunity Analyzer & Filtering S4– Beliefs about the world S5– Desires and Deliberative Process
S5– Intentions - S5 Policy passed to S3
for enactment• It is also apparent that some elements
are missing for Viability
Desires
Wor
ldM
odel Deliberation
Process
Intentions
OpportunityAnalyser
EnvironmentalScan
FilteringProcess
Plans
S
urvi
vin
g P
lans
Intentions
Planning Process
PlanLibrary
Reasoner
Internal Model
Internal Model
Status
ResourceBargaining
Process
Intentions
Overall Plans
Resou rcesA
cco
un
tab
ilit
y
Inten
tions
System Three
System Four
System Five
OperationalStatus
Scheduling
Overall Schedule
Local Plans
Local Schedule
Local Monitoring data
Local Monitoring data
Local Monitoring data
Ove
rall
Sch
edul
e M
onit
orin
g to
nex
t re
curs
ion
Monitoring data
A Viable Intelligent Agent Architecture?
• Using Bratman et al’s IRMA approach and using the VSM as a reference model, we designed a Viable Intelligent Agent Architecture realizable using a multi-agent approach.
• The design specifies, in some detail, the higher, deliberative processes of adaptation.
• The embedding of this architecture in a hierarchy of VSM’s keeps the “Man in the Loop”
• However, the VSM uses human agents to realize its systems. These provide implicit (unmapped) elements of creativity/learning to the organization.
Desires
Wor
ld
Mod
el DeliberationProcess
Intentions
OpportunityAnalyser
EnvironmentalScan
FilteringProcess
Plans
S
urvi
ving
Pla
ns
Intentions
Planning Process
PlanLibrary
Reasoner
Internal Model
InternalModel
Status
ResourceBargaining
Process
Intentions
Overall Plans
Resources
Acc
ount
abili
ty Intentions
System Three
System Four
System Five
Operational
Status
Scheduling
Ov er a ll Sch e du l e
Local Plans
Local Schedule
Local Monitoring data
Local Monitoring data
Local Monitoring data
Ove
rall
Sche
dule
Mon
itori
ng to
nex
t rec
ursi
on
Monitoring data
Ashbean CyberneticsAshbean Cybernetics• The Homeostat - ultra-stable system
capable of returning to stability after it has been disturbed in a way not envisaged by the designer.
• Self-vetoing homeostasis• The Law of Requisite Variety effective control
can only be attained if the repertoire of responses that the controller has at its disposal at least matches the disturbances the situation to be controlled can display.
• More recently, this has been supplemented by the Law of Requisite Knowledge, which states that the system must know i.e. learn, the most appropriate response to use in a given circumstance – i.e. optimized response
• Taken together these two laws underpin the Conant-Ashby theorem, namely “Every good regulator of a system must be (contain) a model of that system”.
K eyV = V a rie ty
E n v iro n m en t
V V
S y stem
M a n a g e m e n tU n it
V
= A tten u a tion
= A m p lifica tion
A (simple) Scenario to address the Law of Requisite Variety
A (simple) Scenario to address the Law of Requisite Variety
• Ashby’s Law states that a system must “derive” a repertoire of actions that can be deployed according to environmental circumstances
• Developed a “toy” system where the environment simply delivers arrays of integers for sorting
• The system is provide with a range of responses from which it tries to select the “optimal” solution using timings/statistical analysis
• Advantages– Fully controllable environment
• Finite (n = 8 = 40320 permutations)• Smooth/turbulent as you like• Measure the degree “sortedness” of
test cases easily
• However, more interesting is how the system acquires new responses to address new environmental circumstances.
SortingSystem
En
viro
nm
ent Quick
Shell
Merge
Bubble
Insert
Hillis’s Sorting NetworksHillis’s Sorting Networks• Hillis had used a “Connection Machine”
coupled with an approach using Holland’s Genetic Algorithms to “evolve” sorting networks for n = 16 arrays
• Although no Connection Machine was available, we decided to try with n = 8 arrays
• Here, for n = 8, a sorting network is represented by 5 pairs of 12 bit chromosomes
• Each 3 bits represents an array index (0 – 7)
• Each 12 bit chromosome holds 4 array indices and hence 2 comparisons
• E.g. Compare Index 1 with Index 2 and swap if necessary
• Chromosomes are paired so that duplicate entries do not appear in the phenotype.
• This gives a minimum of 10 comparisons and a maximum of 20 for the sorting network. Hence more/less “optimal” solutions?
0 0 1 1 0 1 1 1 1 1 1
1
0
3 7
1
5
(1 , 3) (7 , 5) (6 , 2)
0 1 1 1 0 0 101101
0 1 0 1 1 0 101000
1 1 0 1 0 1 101111
0 1 0 1 0 1 001001
0 1 1 1 0 0 101101
0 1 1 1 0 0 101101
0 0 0 1 0 0 101001
0 1 0 1 0 0 101000
0 0 1 1 0 0 101111
0 1 0 1 0 0 100001
1 0 1 1 1 1 1 0 10 0 1Chromosome A
1 0 1 1 1 0 0 1 00 0 1Chromosome B
Breeding GA’sBreeding GA’s• Used a population of 200
networks organized on a two-dimensional array and randomly filled with bits.
• Testing each against our engineered environmental test arrays and measuring reduction in “unsortedness” allowed the identification of “fitter” individuals and hence the new breeding population.
• Select two fit parents, using random crossover, each parent contributes 5 gametes to make a new 10 chromosome member in the new population.
• The new population is then subjected to random mutation (bit-flipping) using the standard probability rate of 0.001
Chromosome A
10 0 1Chromosome B
0 1 1 1 1 1 0 1
0 1 1 1 0 0 1 0
10 0 1
Parent 1
Chromosome A
10 0 1Chromosome B
0 1 1 1 1 1 0 1
0 1 1 1 0 0 1 0
10 0 1
10 0 1 0 1 1 1 0 0 1 0Gamete
Parent 2
10 0 1 0 1 1 1 0 0 1 0Gamete
10 0 1 0 1 1 1 0 0 1 0Chromosome A
10 0 1 0 1 1 1 0 0 1 0Chromosome B
Offspring
Emergence of Full Fitness IndividualsEmergence of Full Fitness Individuals
Full Fitness Individuals in the Population (200)
0
20
40
60
80
100
120
140
160
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97
Generations
No
. of
Ind
ivid
ua
ls
• Somewhat surprisingly, fully-fit individuals appeared in the population very quickly.
• Here, a sorting network capable of correctly sorting the 100 test cases emerged after 11 generations (i.e. in minutes)
• Admittedly, this was using a “gentle” environment
– Inv. 0 “12345678”– Inv. 9 “12378654”
• Interestingly, more than one “type” of fully-fit individuals emerged – efficiency ?
• Unfortunately, less success in highly disordered environments
– Inv. 19 “87621345”– Inv. 28 “87654321” = 97%
• Of course, we’re not suggesting more generalized use, but perhaps “genetic programming?”
Learning Classifier Systems & The Law of Requisite Knowledge
Learning Classifier Systems & The Law of Requisite Knowledge
ClassifierSystem
cond 1 cond 2 message
IF THEN
MESSAGELIST
RULELIST
match
post
Output interfaceeffectors
Input interfacedetectors
Bucket brigade(adjusts rule strengths)
Genetic algorithm(generates new rules)
Payoff
Environment
Inpu
t mes
sage
s
Output m
essages
• So, assuming our system can develop a repertoire of responses tailored to specific environmental circumstances, we still need a means to achieve Requisite Knowledge
• One attractive candidate is Holland’s Learning Classifier Systems approach
• Here the system constructs “creates” candidate production rules in the form: if <condition> then <action> i.e. situation/response
• Rules compete (bid) to participate in a response to an environmental condition. The “solution” provided by the rules is tested in the environment and a “payoff” determined. This is distributed amongst contributing rules and either strengthens or weakens rules depending on success/failure
• A GA is used to generate new rules using provably fit rules.
• What emerges is an adaptive, learning “model” of the environment.
SummarySummary• So, we have looked at the VSM and what it has to contribute
– It clearly defines the types of systems that must exist for viability– Lead to a recursive architecture that appears to be able to incorporate both human systems
and autonomic software systems– Lends itself to the future development of a design pattern/grammar for autonomic system
development • We have noted some of the difficulties encountered by extending the model from
human to artificial systems• Model completeness in open environments• Although we didn’t refer to it today we need to define communications protocols• Etc.
• We briefly examined some of the cybernetic underpinnings for clues to a way forward• Presented some very small scale examples of the use of genetic algorithms to
generate a repertoire of responses and speculated on the use of other perhaps more generalizable genetic approaches e.g. artifical immune systems
• Speculated (even more briefly) on the potential use of Learning Classifier Systems as an approach to address the environmental modelling problems apparent in open systems
Thanks for your attention&
Any Comments or Questions would be welcome?