managing uncertainty in self-adaptive systems with plan reuse …kinneerc.github.io › assets ›...

Managing Uncertainty in Self-Adaptive Systems with Plan Reuseand Stochastic Search

Cody KinneerSchool of Computer ScienceCarnegie Mellon University

Pittsburgh PAckinneercscmuedu

Zack CokerSchool of Computer ScienceCarnegie Mellon University

Pittsburgh PAzfccscmuedu

Jiacheng WangComputer Science Department

Dickinson CollegeCarlisle PA

wangjiadickinsonedu

David GarlanSchool of Computer ScienceCarnegie Mellon University

Pittsburgh PAgarlancscmuedu

Claire Le GouesSchool of Computer ScienceCarnegie Mellon University

Pittsburgh PAclegouescscmuedu

ABSTRACTMany software systems operate in environments where changeand uncertainty are the rule rather than exceptions Techniquesfor self-adaptation allow these systems to automatically respondto environmental changes yet they do not handle changes to theadaptive system itself such as the addition or removal of adaptationtactics Instead changes in a self-adaptive system often require ahuman planner to redo an expensive planning process to allowthe system to continue satisfying its quality requirements underdifferent conditions automated techniques typically must replanfrom scratch We propose to address this problem by reusing priorplanning knowledge to adapt in the face of unexpected situationsWe present a planner based on genetic programming that reusesexisting plans While reuse of material in genetic algorithms hasrecently applied successfully in the area of automated programrepair we find that naiumlvely reusing existing plans for self- plan-ning actually results in a loss of utility Furthermore we propose aseries of techniques to lower the costs of reuse allowing genetictechniques to leverage existing information to improve planningutility when replanning for unexpected changes

CCS CONCEPTSbull Computing methodologies rarr Control methods bull Computersystems organizationrarr Cloud computing Dependable and fault-tolerant systems and networks bull Software and its engineeringrarrSoftware evolution Search-based software engineering

KEYWORDSplan reuse self- systems planning uncertainty genetic program-ming cloud services

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page Copyrights for components of this work owned by others than theauthor(s) must be honored Abstracting with credit is permitted To copy otherwise orrepublish to post on servers or to redistribute to lists requires prior specific permissionandor a fee Request permissions from permissionsacmorgSEAMS rsquo18 May 28ndash29 2018 Gothenburg Swedencopy 2018 Copyright held by the ownerauthor(s) Publication rights licensed to Associa-tion for Computing MachineryACM ISBN 978-1-4503-5715-91805 $1500httpsdoiorg10114531941333194145

ACM Reference FormatCodyKinneer Zack Coker JiachengWang David Garlan andClaire LeGoues2018 Managing Uncertainty in Self-Adaptive Systems with Plan Reuse andStochastic Search In SEAMS rsquo18 SEAMS rsquo18 13th International Symposiumon Software Engineering for Adaptive and Self-Managing Systems May 28ndash292018 Gothenburg Sweden ACM New York NY USA Article 4 11 pageshttpsdoiorg10114531941333194145

1 INTRODUCTIONSelf- systems lower the costs of operating in complex environ-ments of change and uncertainty by autonomously adapting tochange in pursuit of their quality objectives One way these sys-tems self-adjust is by making run-time adjustments according to anadaptation strategy or plan Humans can proactively plan for vari-ous situations by hand at design time [8] This is a form of offlineplanning requiring a painstaking consideration of the full range ofpossible runtime scenarios the system may encounter Automatedtechniques known as online planners seek to reduce planning costsby synthesizing adaptation strategies at run-time [18 32 43 48]

While these systems can quickly respond to the changing condi-tions that they were designed for they often struggle to handle un-foreseen adaptation scenarios [10] Such ldquounknown unknownsrdquo re-alistically include but are not limited to (1) changes in the cost or ef-fects of available adaptation tactics (eg a provider changes the pric-ing schedule for cloud resources) (2) changes in available adapta-tion tactics or options (eg a new type of hardware or server comesto market) or (3) unexpected changes in environmental conditionsor use cases (eg unexpectedly surging popularity of a servicesuch as via the Slashdot effect [42]) Even expensive human gen-erated plans [8] cannot handle this challenge requiring expensivereplanning post design time in the face of unanticipated changes

One potential way to respond to unanticipated adaptation needsis to automatically reuse or adapt prior knowledge to new situ-ations Indeed research in artificial intelligence [1 45] and case-based reasoning [14 28 33] has explored the potential of plan reuseusing knowledge contained in previously-created plans to speed thesynthesis of new plans in response to unanticipated change How-ever the self- context poses unresolved domain-specific challengessince these systems must autonomously respond to uncertaintyfrom a number of sources throughout the adaptation cycle

SEAMS rsquo18 May 28ndash29 2018 Gothenburg Sweden Cody Kinneer Zack Coker Jiacheng Wang David Garlan and Claire Le Goues

We have previously argued [9] that this is a fruitful potentialdomain for the application of stochastic algorithms to self- systemsStochastic search techniques have been shown to be well-suited forsimilar problems [3 6 11 17 38 39 41] However these approachesdo not address the challenge of replanning for new explicitly un-foreseen contexts post-design time which we propose to addressthrough reusing prior plans Intuitively genetic algorithms shouldbe expected to benefit from reused information since they oper-ate by balancing between exploring new solutions and exploitingexisting solutions from generation to generation in effect reusinginformation from previous generations This observation has beensuccessfully applied in other domains such as automated programrepair [16] We investigate the extent to which reusing existingplans in self- planning can result in an improvement in the ful-filling the systemrsquos quality objectives Surprisingly we find thatreusing plans directly is less effective than replanning from scratchWe further propose a series of techniques to make reusing existingplans more efficient ultimately obtaining a planner that can reuseprior plans to improve the systemrsquos quality objectives

We present a self-adaptive systems planner built on genetic pro-gramming that responds to unforeseen adaptation scenarios byreusing and building upon prior knowledge We represent individ-uals as candidate plans evaluating individual fitness by runningthem against a simulated system Our approach explicitly takesinto account the probability that individual tactics may fail andsupports reasoning about tactic latency and planning time

Our planner reuses past information by initializing the popula-tion with individuals based on an existing plan Our main contri-bution is a series of techniques to support adapting to unexpectedchanges at runtime by lowering the costs of plan reuse during evo-lution and an empirical study investigating the utility of plan reusefor several indicative change scenarios Our key contributions arebull An investigation into plan reuse in genetic algorithm plan-

ning finding counter-intuitively that naiumlve reuse can lowerplanning utility

bull A set of techniques for lowering the cost of plan reuse re-sulting in a self- planner that can reuse past information torespond to unforeseen changes more effectively

bull As a sanity check an empirical comparison of our geneticprogramming planner to a PRISM MDP planner [34] thatshows the genetic programming planner can produce near-optimal plans (005 error in the single objective scenarioand 94 in the multi-objective scenario)

bull An investigation into the time quality and population di-versity produced by planning with reuse when adapting tounforeseen scenarios compared to planning from scratchWe find that while the improvement is often slight effectiveplan reuse can result in a fitness improvement

bull Results that show that the objectives emphasized in a multi-objective plannerrsquos starting plan can influence the qualityand character of the plannerrsquos output

Our initial position [9] identified key research questions that weaddress in this work further expanding upon our prior concept inseveral ways a more expressive individual representation inspiredby true planning languages [8] a significantly more efficient fit-ness evaluation strategy a series of techniques for supporting planreuse by reducing the search cost a comparative evaluation to an

Figure 1 MAPE-K Loop for self- systems

exhaustive planner [34] an experimental study investigating thetrade-offs of plan reuse when adapting to unforeseen scenarios andsupport for multi-objective search [47]

The rest of the paper is as follows Section 2 outlines necessarybackground We next describe the running example we use to bothillustrate and evaluate our technique (Section 3) Section 4 detailsour genetic programming self- planning approach Section 5 de-scribes our evaluation Section 6 outlines related work Section 7concludes and offers discussion

2 BACKGROUNDThis section overviews self- planning (Section 21) and geneticprogramming (Section 22) focusing on the background requiredto understand our approach

21 Self- PlannersSelf- systems typically consist of two subsystems a managedsystem and a managing system Many self- systems follow thewell-known five-component MAPE-K architecture [20] shown inFigure 1 We focus on the planning (P) component which pro-duces strategies consisting of tactics ordered to achieve a particulargoal For our purposes tactics are architectural changes the sys-tem can perform to respond to changes eg ldquoturn off a server atlocation Ardquo While multiple planning languages exist for the self-context [26 43] our approach is closest to Stitch [8] An onlineplanner [43] generates a plan at run-time which can adapt quicklyat a potential cost to optimality An offline planner [8] precomputesstrategies to handle common cases and then chooses between themat run-time This allows for a fast correct response to known or pre-dicted situations but cannot handle unanticipated adaptation needsWe compare to a previous hybrid onlineoffline approach [34] thatrelies in part on Markov decision processes (MDPs) formal modelsthat can explicitly capture probabilistic behavior Model checkerssuch as PRISM [26] can use exhaustive search to compute an opti-mal sequence of tactics to maximize one or more system objective(eg profit) for systems formalized as MDPs

Managing Uncertainty in Self- Systems with Plan Reuse and Stochastic Search SEAMS rsquo18 May 28ndash29 2018 Gothenburg Sweden

22 Genetic ProgrammingGenetic programming [24] (GP) is a stochastic technique mod-eled on the principles of biological evolution GP is well-suitedfor problems with poorly understood search landscapes and thosefor which approximate solutions are suitable [37] Note that theseconditions apply to our problem interacting tactics or quality at-tributes render the search landscape complex large search spacesmay preclude the need for (or feasibility of) computing optimalplans and sub-optimal plans are often acceptable in real systemsIndeed genetic algorithms have been successfully applied to self-systems [6 11 38] although using them to explicitly leverage priorknowledge during replanning has not been investigated in self-systems to the best of our knowledge

At a high level a GP evolves a population of candidate programstowards a goal over successive generations A GP represents andmanipulates individual candidate solutions as trees which are mod-ified and recombined using computational analogues of biologicalmutation crossover and selection Mutation randomly modifies oneor more subtrees in an individual supporting search space explo-ration Crossover randomly combines parent individuals to producenew children supporting exploitation of partial solutions A tree-based representation admits the enforcement of a type system overnodes [31] limiting exploration of some types of invalid solutions

A problem-specific objective or fitness function measures howwell a candidate solution satisfies the search objectives Fitness typ-ically informs the probability with which an individual is selectedfrom one generation to the next for continued iteration and caninform the search stopping criterion (if an optimal value or suitablethreshold is known) In contexts with multiple fitness objectives amulti-objective search can produce a set of individuals or Paretofrontier representing the best possible trade-offs between several ob-jectives We use SPEA2 [47] to implement a multi-objective searchwhich selects a fixed quantity of non-dominated individuals tocreate the next generation

3 RUNNING SCENARIOWe first present a system adapted from prior work [34] that weuse as a running example and in evaluation

31 ScenarioFigure 2 shows a cloud-based self-adaptive website with an N-tieredarchitecture User requests to the system are distributed by a loadbalancer to data centers and then to individual servers Serversprocess requests and then return a response to the user Each datacenter has servers of different types with different attributes Ingeneral the more users a server can handle the higher its cost Thewebsite can also serve ads to increase profit slowing response time

32 Quality objectivesThe system goal is to earn profit while maintaining user satisfactionWe consider three interrelated quality objectives (1) System profitas generated by current users minus operating cost (correspondingto the number and cost of the servers) (2) User latency or the meantime users have to wait for a system response (related to the numberand quality of the running servers) and (3) User-perceived qualitythe percentage of users viewing ads These goals are in tension

Figure 2 Cloud web server architecture

For example while the system uses ads for revenue they increaselatency The system can remove ads to improve user experienceand server load while decreasing profit

33 Adaptation tacticsMultiple tactics can adjust the system in pursuit of its quality objec-tives These tactics can turn on and off different types of servers upto a maximum of five per type Each server type has an associatedoperating cost per second and a number of users it can supportper second with or without ads The systemrsquos load balancer dis-tributes requests among data centers according to a traffic valuethere are five traffic levels per data center and traffic is distributedproportionally The system can modify dimmer settings on eachserver type which controls the percentage of users who receiveads (using a brownout mechanism [23] on a per-data center basis)The dimmer level can be changed at 25 increments At run-timeeach of these adaptation tactics may fail Starting and shuttingdown servers fails 10 of the time modifying the dimmer level andincreasing the traffic level fails 5 of the time and decreasing thetraffic level fails 1 of the time

34 Post-design-time adaptationAlthough synthetic this scenario illustrates a number of ways that aself- adaptation problem can change post-design Quality prioritiesmay change eg the system owner might sell it to a charitableorganization that cares more about user satisfaction than profitThe effects of existing tactics may change eg the cost of addinga new server may increase or decrease based on a cloud serviceproviderrsquos fee schedule New tactics may become available vianew data centers server types or even hardware The use caseor environment may also unexpectedly change To the best of ourknowledge existing self- planning technology must always replanfrom scratch in the face of such adaptation changes


⟨plan⟩ = lsquo(rsquo ⟨operator⟩ lsquo)rsquo | lsquo(rsquo ⟨tactic⟩ lsquo)rsquo

⟨operator⟩ = lsquoFrsquo ⟨int⟩ ⟨plan⟩ (For loop)| lsquoTrsquo ⟨plan⟩ ⟨plan⟩ ⟨plan⟩ (Try-catch)| lsquorsquo ⟨plan⟩ ⟨plan⟩ (Sequence)

⟨tactic⟩ = lsquoStartServerrsquo ⟨srv⟩ | lsquoShutdownServerrsquo ⟨srv⟩| lsquoIncreaseTrafficrsquo ⟨srv⟩ | lsquoDecreaseTrafficrsquo ⟨srv⟩| lsquoIncreaseDimmerrsquo ⟨srv⟩ | lsquoDecreaseDimmerrsquo ⟨srv⟩

Figure 3 Grammar for specifying plans Servers (srv) can beof types A B C or D For loops can iterate up to 10 times

4 APPROACHWe present a planner that reuses previously-known information(Section 44) using GP to efficiently produce nearly optimal resultsin a large uncertain search space in response to unforeseen adap-tation scenarios Our approach reuses past knowledge by seedingthe starting population with prior plans These plans satisfied thesystemrsquos objectives in the past but are currently sub-optimal dueto ldquounknown unknownsrdquo unexpected changes to the system or itsenvironment that the past plans did not address After an unex-pected change occurs the system model must be updated to reflectthe new behavior after the unexpected change The mechanism forsynchronizing the system model with the actual world is outsidethe scope of this paper this may be done manually (likely with lesseffort than replanning) or automatically [21 46] Section 44 ex-plains how our approach reuses prior plans while Sections 41ndash43provide the necessary technical details on the GP implementation

A new GP application is defined by how individuals are repre-sented (Section 41) how they are manipulated through mutationand crossover (Section 42) and how the fitness of candidate solu-tions is calculated (Section 43)

41 RepresentationIndividuals in the population are plans organized as trees Figure 3gives a Backus-Naur grammar for our plans Each consists of eitherone of six available tactics (described in Section 3) or one of threeoperators containing subplans The for operator repeats the givensubplan for 2ndash10 iterations the sequence operator consecutivelyperforms 2 subplans The try-catch operator tries the first sub-plan If the last tactic in that subplan fails it executes the secondsubplan otherwise it executes the third subplan The example planat the top of Figure 4 uses a try-catch operator first attempting tostart a new server at data center A If successful it attempts to starta server at data center B if not it retries the StartServer A tactic

This planning language is a simplified variant of other languagessuch as Stitch [8] Unlike Stitch our language does not consider planapplicability (guards that test state to determine when a plan can beused) which we leave to future work Note that any plan express-ible in our language could be expressed with only the try-catchoperator and that our language can represent any PRISMMDP [26]plan as a tree of try-catch operators with depth 2h where h isthe planning horizon

( T (StartServer A) (StartServer A) (StartServer B) )

StartServer A

StartServer A

010

StartServer B

090

9878

010

11373

090

11373

010

15266

090

Figure 4 Top An example plan Bottom This planrsquos systemstate tree

42 Mutation and CrossoverMutation may either replace a randomly selected subtree withanother randomly-generated subtree or copy an individual unmod-ified to the next generation The distribution between these choicesis a tunable parameter Mutation imposes both size and type limita-tions on generated subtrees which can range from a single tacticto a tree of depth ten The crossover operator [44] selects a subtreein each of two parent plans (selected via tournament selection) andswaps them to create two new plans We enforce syntax rules onboth operators (eg requiring swapped or generated nodes to havethe correct number of children of the correct type) However itis still possible for the planner to generate plans which lead thesystem to an invalid state eg a plan that tries to add more serversthan are available is syntactically correct but invalid We do notprevent this behavior instead penalizing such plans

43 FitnessWe evaluate candidate fitness by simulating the plan to measure theexpected quality of the resulting system Because tactics might failwe must combine multiple eventualities Thus conceptually fitnessis computed via a depth-first search of all possible states that asystemmight reach given a plan captured in a system state tree Treenodes represent possible system states connecting edges representtactic application attempts labeled by their probability (the tacticsuccessfailure probability) Every path from the root (the initialsystem state) to a leaf represents a possible plan outcome Overallplan fitness is the weighted average of all possible paths throughthe state tree Path fitness is the quality of the leaf node systemstate measured as one or more of profit latency and user perceivedquality (Section 3) Each final system state contributes to overallplan fitness weighted by the product of its edge probabilities

To illustrate Figure 4 shows a plan and its corresponding statetree Leaf nodes are labeled with their state fitness (profit in thisexample) edges with their probability Left transitions correspondto tactic failure right-transitions tactic success Following the right-hand transitions shows that if all tactics succeed profit will be


15266 with an 81 probability Following the left hand transitionsshows the expected system state if all tactics fail (1 probability)The weighted sum over all paths (overall fitness) is 145114

The simulator takes into account planning time and tactic latencyEach leaf in the state tree represents a timeline of events (parent tac-tics succeeding or failing) This timeline is simulated to obtain thefitness accrued while the plan was executing as well as the fitnessstate of the system after the plan terminates To support reasoningabout the opportunity cost of planning time the fitness functiontakes as input a window size parameter that specifies how long thesystem is expected to continue accruing the fitness resulting fromthe provided plan If the system will remain in a state for a longperiod of time it may be worthwhile to spend more time planningsince the system has more time to realize gains from the planningeffort On the other hand if the system is expected to need to re-plan quickly spending time optimising for the current state may bewasted since this effort will need to be repeated before gains are re-alized Total fitness accrued is equal to slowastp+d+alowast(wminus(t+p)) wheres is the systemrsquos initial fitness p is the planning time d is the fitnessaccrued during plan execution a is the fitness value after the planis executedw is the window size t is the time planrsquos execution time

We investigated several additional heuristic modifications to fit-ness computation to manage invalid actions and plan size First weinvestigate an invalid action penalty per invalid tactic To manageproduced plan size we include a verboseness penalty which penal-izes a plan proportional to its size and a parsimony pressure killratio which assigns a fitness of zero to a random proportion of in-dividuals larger than the average population size In reporting finalplan fitnesses we report actual fitness unmodified by penalties

44 Plan ReuseOur approach reuses past knowledge by seeding the starting popu-lation with prior plans After the system model is updated to reflectthe unexpected change a starting population of adaptation strate-gies is created These strategies are iteratively improved by randomchanges via mutation and crossover with the most effective plansbeing more likely to pass into the next generation resulting inutility increasing over time Seeding previously useful plans intothe population allows for useful pieces of planning knowledge tospread to other plans during crossover

Preliminary results showed that initializing the search by naivelycopying existing plans did not result in efficient planning and inmost cases was inferior compared to replanning from scratch witha randomly generated starting population This is due to the highcost of calculating the fitness values of long starting plans Torealize the benefits of reuse we introduce several strategies forlowering this cost including seeding the initial population with afraction of randomly generated plans in addition to previous plansprematurely terminating the evaluations of long running plans andreducing the size of starting plans by randomly splitting these plansinto smaller plan trimmings

To reduce the number of long starting plans that the plannerneeds to evaluate we initialize a scratch_ratio percent of the startingpopulation with short (a maximum depth of ten) randomly gener-ated plans and only seed the remaining 1minusscratch_ratio individualswith reused plans This reduces the amount of time spent evaluating

the fitness of the starting plan in the new situation while still allow-ing for the reusable parts of the existing plan to bootstrap the search

Since the evaluation time is exponential with respect to the plansize the few longest plans in the population can take significantlylonger to evaluate than the rest of the population To prevent wast-ing search resources on excessively long plans we introduce akill_ratio parameter that terminates the evaluation of overly longplans and assigns them a fitness of zero When kill_ratio percentof individuals have been evaluated evaluation stops and all out-standing plans receive a fitness of zero This approach leveragesthe parallelizability of GP to avoid hard-coding hardware and plan-ning problem dependent maximum evaluation times but requiresplanning on hardware with multiple cores

Lastly to further reduce the cost of reuse rather than completelycopying large starting plans we initialize the search with small planldquotrimmingsrdquo from the initial plan Our planner generates trimmingsby randomly choosing a node in the starting plan using Kozarsquos nodeselector [25] that can serve as the root of a new tree This subtreeis then added to the starting population The process is repeateduntil the desired number of reused individuals is obtained

5 EVALUATIONWe built the genetic programming planner described in Section 4on ECJ1 This section describes our evaluation We investigated thefollowing research questions

(1) As a sanity check how does the GP plannerrsquos efficiency andeffectiveness compare to an exhaustive planner

(2) Can plan reuse improve planning utility in response to un-foreseen adaptation scenarios

(3) Does our plannerrsquos techniques for facilitating reuse improveplanning effectiveness

(4) How does plan reuse impact population diversityIn all experiments we evaluate on various scenarios based on the

system shown in Figure 2 and described in Section 3 The systembegins each scenario with one server of each type a default trafficsetting of 4 and all dimmers set to 0 The experimental server ran64-bit Ubuntu 14045 LTS with a 16 core 230 GHz CPU and 32GB of RAM but was set to limit the planners to 10 GB of RAMThe GP used 8 of the available CPU cores PRISM experiments useversion 431 and the sparse engine unless otherwise stated Weset the planning horizon to 20 for PRISM and the maximum plantree depth to 20 for the GP planner Each experimental result isthe median of 10 trial runs if randomness or timing is involvedWhere statistical tests are used to asses significance we use theWilcoxon rank-sum test a non-parametric test that does not requirethe samples to follow a normal distribution and is appropriate forsmall sample sizes When P lt 005 we reject the null hypothesisthat the samples arise from the same population In the multi-objective context we compute a SPEA2-defined Pareto optimalfront optimizing for two or more of the given fitness objectivesWe set the SPEA2 algorithm elite set to 50 In experiments thatwe compare to PRISM we disable reasoning about tactic latencysince this is not easily achieved in PRISM Where tactic latency

1ECJ is available at httpscsgmuedu~eclabprojectsecj The sourcecode for our planner is available at httpsgithubcomZackCAdaptiveSystemsGeneticProgrammingPlanner


2750

2800

2850

2900

2950

3000

0 50 100 150 200Runtime (seconds)

Pro

fit PlannerPRISMGP

0

1000

2000

3000

4000

0 2500 5000 7500Latency

Pro

fit

Planner

PRISM

GA planner

Figure 5 Top Profit versus planning time for GP parameterconfigurations Many configurations produce similar profitresults to PRISM significantly faster Bottom Pareto frontsfor profit and latency from both planners

is considered we set the window size to be 10000 seconds unlessotherwise specified Where we compare to searches from ldquoscratchrdquowe use ramped half-and-half[24] to initialize the population

51 Comparative Study511 Efficiency As a sanity check to establish that our stochas-

tic planner achieves reasonable results we first tuned and comparedit to an exhaustive planner from previous work [34] an MDP plan-ner written in PRISM2 We configured the planner with the samesettings as in the previous work adding path probability to the sys-tem specification and planning for a particular environment state

As in many optimization techniques a GP typically includesmany tunable parameters that require adjustment We thus per-formed a parameter sweep to heuristically tune the reproductive

2Because the Pandey et al approach [34] was not named and we assess the limitationsof PRISM rather than the hybrid element we refer to this as the PRISM planner forthe remainder of the paper

strategy (which determines how individuals in the next genera-tion are produced a ratio of crossover mutation and reproduc-tioncopying) and number of generations population size and allpenalty thresholds (Section 43) We generated plans for the sys-temrsquos initial configuration (Section 3) and started each search froma minimal plan of four tactics that does not affect fitness

The dark point at the top of Figure 5 shows the optimal systemprofit (fitness) and planning time (200 seconds) of the PRISM plan-ner Each gray point corresponds to a different parameter configu-ration of the GP planner Many parameter configurations allowedthe GP planner to find plans that were within 005 of optimal butin a fraction of the time (under 1 second in some cases) The bestconfiguration that produced plans in 050 seconds resulted in only029 error which demonstrates the that the planner has the po-tential to be used as an online planner that reacts to change in realtime This top configuration used 30 generations each containing1000 individuals the next generation is produced 60 by crossover20 by mutation and 20 reproduction applied 0 parsimony pres-sure and 001 verboseness penalty (ie a small penalty for largeplans) and an invalid action penalty of 0 We use these values insubsequent experiments unless otherwise indicated

512 Search space Next we evaluate and compare the plan-nersrsquo search space limitations We varied search space size by ad-justing the number of available server types (t ) in our scenariowhich caused the model states to grow exponentially followingthe equation (6 servers_per_type lowast 5 possible_dimmer_values lowast5 possible_tra f f ic_values )t

We found that PRISM can plan to maximize profit for 3 servertypes with a maximum plan size of 20 tactics However PRISM runsout of memory and produces no plans when given four server typesto consider even when searching for only a single tactic Using theexplicit engine which requires less memory but more run-timePRISM could produce a plan for four server types for a plan lengthof up to seven By contrast our GP planner succeeded on the fourserver type case increasing profit from 988 in the initial state to2993 Finally we increased the number of data centers from 4 to 16a state space on the order of 1037 and successfully generated a planafter about 9 minutes These tests demonstrate that the GP plannercan handle a very large search space outperforming an exhaustiveplanner and provides evidence that the planner works correctly tobuild confidence in our core experiments investigating plan reuse

513 Multi-objective search The GP planner can create a Paretofrontier of plans to trade-off between multiple quality attributesallowing system maintainers to evaluate the best possible combina-tions PRISM can also generate a Pareto frontier for two objectivesThe bottom of Figure 5 shows the Pareto fronts for the profit andlatency objectives produced by PRISM and the GP For this exper-iment we set the planning horizon for both planners to 10 PRISMfound 30 points along the curve the GP planner produced 89 afterremoving duplicates PRISM took 1177 seconds the GP planner took751 seconds The front produced by the genetic planner roughly ap-proximates the front produced by PRISM with 94 average error

We also generated three-dimensional Pareto fronts for all threequality objectives with the GP planner PRISM cannot producefronts in this case and the graphs are difficult to display but weobserve that the starting plan influenced the shape of the resulting


Table 1 Improvement obtained by reuse enabling tech-niques

Planning Technique Utility P ValueScratch 1000Scratch amp kill_ratio 1044 lt 001Reuse 0962 006Reuse amp kill_ratio 1072 lt 001Reuse amp kill_ratio amp scratch_ratio 1077 063Reuse amp kill_ratio amp scratch_ratio amp trimmer 1112 lt 001

front If we begin with plans previously optimized for profit wefind Pareto fronts with more high-profit individuals Starting froma lower-quality plan or planning from scratch produced a broaderfront of lower latency individuals In effect these starting plans ledthe search to explore more of the trade-offs between latency andquality We explore the trade-offs of plan reuse more directly in thenext set of experiments

52 Reuse-Enabling TechniquesWhile the previous results inspire confidence that the planner canbe competitive with an optimal planner our primary goal is to usethe GP planner to realize increased planning ability in response tounexpected changes through reusing prior plans Since preliminaryresults showed naiumlvely reusing entire plans in the starting popu-lation resulted in poor planning performance recall we introduceseveral techniques for lowering the cost of reuse (Section 44) thekill_ratio scratch_ratio and plan trimmer

To demonstrate the usefulness of these features we performedplanning for the Request Spike + New Data Center scenariowith a planning window of 10000 incrementally enabling the pro-posed reuse enabling techniques to show the improvement obtainedfrom each feature For comparison we also plan from scratch bothwith and without using kill_ratio When used the values used werekill_ratio = 075 and scratch_ratio = 05 These values were chosenbased on a parameter sweep

Table 1 shows the results normalized to the utility of planningfrom scratch without the kill_ratio such that this utility is 1 Usingthe kill_ratio improved utility to 1044 Without any reuse-enablingtechniques reusing plans by initializing the population with mu-tated versions of the starting plan resulted in a fitness of 0962underperforming compared to planning from scratch Enabling thekill_ratio feature improved the utility obtained by reusing plansto a level slightly better than planning from scratch while usingthe kill_ratio Adding the scratch_ratio resulted in a slight improve-ment of 0005 and trimming the reused plans resulted in a furtherimprovement of 0035 The scratch_ratio did not show a statisticallysignificant improvement for this scenario but did for the IncreasedCosts scenario at the 005 level Trimming plans and the kill_ratioboth showed statistically significant improvements

These results demonstrate that while the costs of evaluating thefitness of prior plans makes improving planning fitness throughreuse nontrivial the proposed enhancements to GP planning can re-duce this cost and achieve higher fitness than planning from scratch

Table 2 Percent change reusing plans instead of planningfrom scratch Statistically significant results (P lt 005) areshown in bold font

Scenario 1K 10kIncreased Costs 002 081Network Unreliability 001 010Failing Data Center -002 014Request Spike -014 -001New Data Center -063 028Request Spike + New Data Center -047 154

53 Unforeseen Adaptation ScenariosOur core experiments investigate the GP plannerrsquos ability to ad-dress unforeseen adaptation needs with plan reuse We do thisby constructing adaptation scenarios that cover different types ofadaptation needs based on different sources of uncertainty and as-sessing the plannerrsquos ability to respond when planning with reusecompared to planning from scratch The considered scenarios arebull Increased Costs All server operating costs increase uni-

formly by a factor of 100 a system-wide changebull Failing Data Center The probability of StartServer C

failing increases to 100 a change in the effect of an existingtactic

bull Request Spike + New Data Center The system experi-ences a major spike in traffic and gains access to a newserver location to help address it This location (D) containsservers that are strictly less efficient than those at locationA (ie they have the same operating cost but lower capac-ity) but would be useful if there were more requests thancould be served by location A This corresponds primarily toan environmental change along with the addition of a newtactic

bull Network Unreliability The failure probability for all tac-tics increases to 67 a change in the effect of an existingtactic

For each adaptation scenario we modified the simulator to be-have according to the change relative to the initial scenario (Sec-tion 3) We maximize profit in all experiments box and whiskerplots show the best individual in the population each generationover ten planner executions We show convergence in terms of thequality (profit) of the produced plans over GP iterations a machine-and problem-independent proxy for evaluation time

Table 2 shows the percent change between planning from scratchand plan reuse for each scenario and for two window sizes Posi-tive values indicate the reuse resulted in an improvement negativevalues indicate a decrease in utility compared to planning fromscratch Most values showed a small difference that was not sta-tistically significant For the smaller window size no values werestatistically significant indicating that there is no statistical differ-ence between plan reuse and planning from scratch For the largerwindow size half of the scenarios showed statistically significantimprovements from planning from scratch with the complex Re-quest Spike + New Data Center scenario showing the largestimprovement Since a larger window size means that the system hasmore time to realize the benefits of a higher quality plan this result


Network Unreliability Request Spike amp New Data Center

17500000

20000000

22500000

25000000

27500000

30000000

25e+07

30e+07

35e+07

40e+07

45e+07

50e+07

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19Generation

Pro

fit

StartingPlan

reuse

scratch

Figure 6 Profit versus generation for two scenarios

Failing Data Center Increased Costs Network Unreliability

New Data Center Request Spike Request Spike amp New Data Center

293e+07

294e+07

295e+07

210e+07

215e+07

220e+07

290e+07

291e+07

292e+07

293e+07

294e+07

295e+07

26e+07

27e+07

28e+07

29e+07

45e+07

46e+07

47e+07

48e+07

46e+07

47e+07

48e+07

49e+07

50e+07

0 25 50 75 100 0 20 40 60 0 25 50 75 100

0 50 100 150 200 0 50 100 150 0 50 100 150 200

Cumulative Evaluation Time (seconds)

Pro

fit

StartingPlan

reusescratch

Figure 7 Profit versus cumulative runtime for all six scenarios

is intuitive Additionally since a more complex change scenariois more difficult to plan for it follows that plan reuse results in agreater improvement for these scenarios While in most cases thedifferences are small these results show that our approach usingplan reuse can result in fitness improvements

531 Modified System or Tactics Figure 6 shows the profit overgeneration produced by the GP for two of the considered scenariosThe left of Figure 6 shows results for the Network Unreliability

scenario For much of the first seven generations of planning planreuse outperforms planning from scratch with the two eventuallyconverging to the same fitness at generation eight The IncreasedCosts and Failing Data Center scenarios showed similar results

532 Changing Environment The right of Figure 6 shows resultsfor the Request Spike + New Data Center scenario in whichthe system replans for a large increase in the number of systemrequests handled by previous plans (eg the Slashdot effect [42])

We also provide the system with a new data center D to possiblyuse to address this issue3 This scenario shows the most pronounceddifferences between plan reuse and planning from scratch withplan reuse performing better for all 20 generations The RequestSpike and New Data Center scenarios alone showed a similarpattern but was less pronounced

533 Wall-clock time Because fitness evaluation time variesby plan size the amount of time needed to evaluate the fitness ofeach generation is variable making the number of generations animperfect proxy for run time Thus Figure 7 shows results in termsof wall-clock time for each scenario The Network UnreliabilityIncreased Costs and Failing Data Center scenarios showed sim-ilar behavior with only a very small benefit from reusing plansTheNew Data Center andRequest Spike + New Data Center

3We also evaluated planning in response to the two changes independently Bothscenarios showed similar results to the Request Spike + New Data Centeralthough the benefits of plan reuse are more prominent in the hybrid scenario


scenarios showed greater differences in particular the RequestSpike + New Data Center scenario showed a clear advantage toreusing existing plans

54 DiversityGenetic programming balances search space exploration to avoid lo-cal optima and exploitation of promising partial solutions Solutiondiversity is necessary to support exploration of good partial solu-tions however it typically decreases as the search converges [29]assuming that the population is sufficiently diverse To gain addi-tional insight into plan evolvability given different scenarios wemeasured the syntactic population diversity over a search by com-puting the average pairwise tree edit distance of the individualsusing the APTED algorithm [36]

Figure 8 shows population diversity across the scenarios Diver-sity values from planning from scratch as well as reusing plans bothwith and without trimming are shown Planning from scratch pro-duces a highly diverse starting population that gradually becomesless diverse as it converges

As shown in Figure 8 reusing existing plans without first trim-ming them results in a less diverse population initially Rather thana gradual decrease in diversity as would be expected in some sit-uations (such as generations 2ndash8 for the New Data Center) thediversity actually increases as the population explores new plansbefore continuing to converge on a good solution However whenusing the trimmer the diversity values start high and smoothlydecrease This observation helps to explain why trimming existingplans resulted in a more significant improvement than the scratchratio alone since the presence of smaller plan trimmings facilitatesa smoother exploration and exploitation trade-off

6 RELATEDWORKPlanning The artificial intelligence (AI) community has produceda considerable body of research in planning including but notlimited to probabilistic approaches like MDP [19 30] Some of thiswork demonstrates the benefits of plan reuse such as by reusingparts of existing plans that target particular goals in new situationsthat share those goals [45] concurrently executing and dynamicallyswitching between plans designed to handle contingencies [4]modifying plans produced under assumed optimal conditions tohandle common problems found in simulation [27] or iterativelytransforming simple plans to produce complex plans [1] Case-basedplan adaption [33] explicitly reuses past plans in new contexts inwhich context GAs have been explored directly [14 28] eg byinjecting solutions to previous problems into a GA population tospeed the solution of new problems Although the mechanism issimilar our approach is importantly novel in that it addresses abroader class of uncertainty

Reinforcement learning has been applied to self- systems tolearn at runtime using Q-learning [18 22] Like our approach thistechnique could be used to aide in adapting to unexpected changesand like GP this technique balances exploration of possible al-ternatives and exploiting solutions that achieve the best resultsOur approach is different from reinforcement learning because ourapproach utilizes a model of the system and environment While re-inforcement learning has the advantage of learning online without

needing to synchronize a model with the environment the systemwill likely perform suboptimally while it performs random actionsto learn This might often be undesirable in many production sys-tems especially if such actions could result in irreparable damageto the system or others

A number of works address the problem of updating systemmodels when they become out of sync with reality such as focus-ing on architectural evolution [2] identifying when unexpectedchanges occur for to assist humans in evolving the system [40] orevolving models at runtime [46]

Self- planningMutliple PRISM MDP planners appear in the self- literature [5 32 34] These techniques are typically offline asis manual human planning [8] and produce good plans but haveissues with problem size limitations Much of the other work inthis space focuses on online planning eg reactively regeneratingfailing parts of a plan [43] or using hill-climbing [48] (anotherstochastic technique) to generate plans quickly in exchange for amoderate loss of optimality Our work focuses in particular on planreuse to handle uncertainty and is not reactive

Plato and Hermes [38] use genetic algorithms to reconfigure soft-ware systems (in the domain of remote data mirroring) to respondto unexpected failures or optimize for particular quality objectivesThe search problems (representation operators and fitness func-tion) differ from ours commensurate with the different domainHowever the key distinction is our focus on information reuseto handle uncertainty That is although both Hermes and our ap-proach are initialized with existing adaptation strategies we focusexplicitly on the utility of alternative starting strategies in the faceof unanticipated scenarios We also compare a GP planner to anoptimal planner GAs have also been used to optimize across mul-tiple quality objectives in quality of service composition [6] andto optimize architectures and associated service providers in self-architecting systems [11] This prior work broadly substantiatesthe utility of stochastic algorithms in an self- planning contextbut otherwise focuses on very different domains than we do

In our own previous work [9] we demonstrated the feasibilityof plan reuse using genetic programming in self- contexts Weexpand upon that initial concept in the ways described in Section 1

Search-based software engineering The field of Search-BasedSoftware Engineering (SBSE) uses meta-heuristic and stochasticsearch to solve multiple software engineering problems [15] Therehas been considerable recent success in reusing and improving exist-ing programs [3] similar to the way we reuse and improve existingplans Such methods have been applied to self-adaptive systems andarchitectural design and evolution [7 35] set configuration param-eters for systems with strict quality requirements [13] and improv-ing self-adaptive system test cases [12] However such work doesnot directly tackle the problem of self-adaptive planning or improveon previous plans and the research space of SBSE as applied tosuch systems remains underinvestigated despite its potential [16]

7 CONCLUSIONFuture generation systems will operate in complex environments ofchange and uncertainty Self- systems lower the costs of operatingin such conditions by autonomously adapting to change in pursuitof their quality objectives and planning is an important component




02

04

06

02

04

06

0 10 20 0 10 20 0 10 20Generation

Nor

mal

ized

Div

ersi

ty(a

vera

ge p

airw

ise

tree

edi

t dis

tanc

e)

StartingPlan

mutator

scratch

trimmer

Figure 8 Diversity versus generation for all six scenarios

of these approaches We propose to use stochastic search to dealwith unexpected adaptation strategies specifically by reusing orbuilding upon prior knowledge Our GP planner can handle a verylarge search space (over 1037 possible states) can produce plans towithin 005 of optimal and can effectively plan with respect tomultiple system quality objectives

Our core results demonstrate the feasibility of reusing pastknowledge in unexpected adaptation scenarios and that the na-ture of both the scenario and of that prior knowledge influencesits effectiveness While naiumlvely reusing existing plans can actuallyresult in worse performance than planning from scratch effectivelyutilizing prior plans can reduce the number of generations requiredto reach a good plan for various scenarios but whether this trans-lates into run-time savings depends on both the size of the startingplan and its relationship to the new scenario Our diversity analysiscorroborates these results

There exist several limitations and threats to the validity ofour results First our parameter tuning is heuristic performing acoarse test of one set of parameters before a finer-grained sweepit is possible that the best configuration is located in an area thatseemed less promising initially Additionally while we took care inour implementation of the PRISM MDP planner mistakes in ourimplementation could affect our results We mitigate the risks ofbias in our Java GP planner representation by releasing it publiclyfor review and replication

Additionally our results may not generalize to other systemsor to other unexpected adaptation scenarios We mitigate this risk

by building our evaluation on a cloud system used to assess priorwork [34] and designed to approximate real-world systems suchas an application running on Amazonrsquos Cloud Web Services Weconstructed our evaluation scenarios to sample as much of the spaceof possible unanticipated adaptation needs as possible and leavethe investigation (or even a taxonomization) of additional suchscenarios to future work Other future research directions includeautonomously determining when to replan and investigating howthe system model can be kept up to date with reality

As systems become larger and more complex the difficulty ofplanning for the unexpectedwill only increaseWe show that knowl-edge reuse is a promising tool for addressing unexpected changeswhile being underexplored in the self- context Future work inplan reuse in self- systems has the potential to enable the nextgeneration of autonomous systems to quickly respond to changesunforeseen at design time

ACKNOWLEDGEMENTSThis research supported in part by the National Science Foundation(CCF-1618220) and the National Science Foundation Graduate Re-search Fellowship Program (DGE1745016) Any opinions findingsand conclusions or recommendations expressed in this material arethose of the authors and do not necessarily reflect the views of thesponsoring agencies


REFERENCES[1] Joseacute Luis Ambite and Craig A Knoblock 2001 Planning by Rewriting J Artif

Int Res 15 1 (2001) 207ndash261[2] Jeffrey M Barnes David Garlan and Bradley Schmerl 2014 Evolution Styles

Foundations and Models for Software Architecture Evolution Softw Syst Model13 2 (May 2014) 649ndash678 httpsdoiorg101007s10270-012-0301-9

[3] Earl T Barr Mark Harman Yue Jia Alexandru Marginean and Justyna Petke2015 Automated Software Transplantation In Int Symp on Soft Testing andAnalysis (ISSTA) 257ndash269 httpsdoiorg10114527717832771796

[4] Micheal Beetz and Drew McDermott 1994 Improving Robot Plans During TheirExecution In Int Conf on AI Planning and Scheduling (AIPS) 7ndash12

[5] Javier Caacutemara David Garlan Bradley Schmerl and Ashutosh Pandey 2015Optimal Planning for Architecture-based Self-adaptation via Model Checking ofStochastic Games In Symp on Applied Computing (SAC rsquo15) 428ndash435

[6] Gerardo Canfora Massimiliano Di Penta Raffaele Esposito and Maria LuisaVillani 2005 An Approach for QoS-aware Service Composition Based on GeneticAlgorithms In Conf on Genetic and Evol Computation (GECCO) 1069ndash1075

[7] Betty H C Cheng Andres J Ramirez and Philip K McKinley 2013 HarnessingEvolutionary Computation to Enable Dynamically Adaptive Systems to ManageUncertainty In Workshop on Combining Modelling and Search-Based Soft Eng(CMSBSE)

[8] Shang-Wen Cheng and David Garlan 2012 Stitch A Language for Architecture-based Self-adaptation J Syst Softw 85 12 (2012) 2860ndash2875

[9] Zack Coker David Garlan and Claire Le Goues 2015 SASS Self-adaptationUsing Stochastic Search In Int Symp on Soft Eng for Adaptive and Self-ManagingSyst (SEAMS) 168ndash174

[10] Rogeacuterio et al de Lemos 2013 Software Engineering for Self-Adaptive Systems ASecond Research Roadmap Springer Berlin Heidelberg 1ndash32

[11] John M Ewing and Daniel A Menasceacute 2014 A Meta-controller Method forImproving Run-time Self-architecting in SOA Systems In 5th ACMSPEC IntConf on Performance Eng (ICPE rsquo14) 173ndash184

[12] Erik M Fredericks Byron DeVries and Betty H C Cheng 2014 Towards Run-time Adaptation of Test Cases for Self-adaptive Systems in the Face of UncertaintyIn Int Symp on Soft Eng for Adaptive and Self-Managing Syst (SEAMS) 17ndash26

[13] S Gerasimou G Tamburrelli and R Calinescu 2015 Search-Based Synthesis ofProbabilistic Models for Quality-of-Service Software Engineering In Int Confon Automated Soft Eng (ASErsquo15) 319ndash330

[14] Alicia Grech and Julie Main 2004 Case-Base Injection Schemes to Case AdaptationUsing Genetic Algorithms Berlin Heidelberg 198ndash210

[15] Mark Harman 2007 The Current State and Future of Search Based SoftwareEngineering In Int Conf on Soft Eng 342ndash357 httpsdoiorg101109FOSE200729

[16] Mark Harman Yue Jia William B Langdon Justyna Petke Iman HematiMoghadam Shin Yoo and Fan Wu 2014 Genetic Improvement for AdaptiveSoftware Engineering (Keynote) In Int Symp on Soft Eng for Adaptive andSelf-Managing Syst (SEAMS) 1ndash4

[17] Yanrong Hu and S X Yang 2004 A knowledge based genetic algorithm forpath planning of a mobile robot In Robotics and Automation Vol 5 4350ndash4355httpsdoiorg101109ROBOT20041302402

[18] Pooyan Jamshidi Amir M Sharifloo Claus Pahl Andreas Metzger and GiovaniEstrada 2015 Self-learning cloud controllers Fuzzy q-learning for knowledgeevolution In Cloud and Autonomic Computing (ICCAC) 2015 International Con-ference on IEEE 208ndash211

[19] Leslie Pack Kaelbling Michael L Littman and Anthony R Cassandra 1998Planning and acting in partially observable stochastic domains ARTIFICIALINTELLIGENCE 101 (1998) 99ndash134

[20] JeffreyO Kephart andDavidM Chess 2003 The Vision of Autonomic ComputingComputer 36 1 (2003) 41ndash50

[21] Narges Khakpour Saeed Jalili Carolyn Talcott Marjan Sirjani and Moham-madreza Mousavi 2012 Formal modeling of evolving self-adaptive systemsScience of Computer Programming 78 1 (2012) 3ndash26

[22] Dongsun Kim and Sooyong Park 2009 Reinforcement learning-based dynamicadaptation planning method for architecture-based self-managed software InSoftware Engineering for Adaptive and Self-Managing Systems 2009 SEAMSrsquo09ICSE Workshop on IEEE 76ndash85

[23] Cristian Klein Martina Maggio Karl-Erik AArzeacuten and Francisco Hernaacutendez-Rodriguez 2014 Brownout Building More Robust Cloud Applications In IntConf on Soft Eng (ICSE rsquo14) 700ndash711

[24] John R Koza 1992 Genetic Programming On the Programming of Computers byMeans of Natural Selection MIT Press Cambridge MA USA


[26] M Kwiatkowska G Norman and D Parker 2011 PRISM 40 Verification ofProbabilistic Real-time Systems In Int Conf on Computer Aided Verification (CAVrsquo11) 585ndash591

[27] Neal Lesh Nathaniel Martin and James Allen 1998 Improving Big Plans InConf on Artificial IntelligenceInnovative Applications of Artificial Intelligence

(AAAIIAAI) 860ndash867[28] S J Louis and J McDonnell 2004 Learning with Case-injected Genetic Algo-

rithms Trans Evol Comp 8 4 (2004) 316ndash328 httpsdoiorg101109TEVC2004823466

[29] Sushil J Louis and Gregory J E Rawlins 1992 Syntactic Analysis of Convergencein Genetic Algorithms In Found of Genetic Algorithms 2 Morgan Kaufmann141ndash151

[30] Mausam and Andrey Kolobov 2012 Planning with Markov Decision ProcessesAn AI Perspective Synthesis Lectures on Artificial Intelligence and Machine Learn-ing 6 1 (2012) 1ndash210 httpsdoiorg102200S00426ED1V01Y201206AIM017

[31] David J Montana 1995 Strongly Typed Genetic Programming Evol Comput 32 (1995) 199ndash230

[32] Gabriel A Moreno Javier Caacutemara David Garlan and Bradley Schmerl 2015Proactive Self-adaptation Under Uncertainty A Probabilistic Model CheckingApproach In European Softw Eng Conf and the Symp on the Found of Soft Eng(ESECFSE 15) 1ndash12

[33] H MuAtildeśoz-Avila and M T Cox 2008 Case-Based Plan Adaptation An Analysisand Review IEEE Intelligent Syst 23 4 (2008) 75ndash81 httpsdoiorg101109MIS200859

[34] Ashutosh Pandey Gabriel A Moreno Javier Caacutemara and David Garlan 2016Hybrid Planning for Decision Making in Self-adaptive Systems In Int Conf onSelf-Adaptive and Self-Organizing Syst (SASO rsquo16) 12ndash16

[35] Gustavo G Pascual Moacutenica Pinto and Lidia Fuentes 2013 Run-time Adaptationof Mobile Applications Using Genetic Algorithms In Int Symp on Soft Eng forAdaptive and Self-Managing Syst (SEAMS) 73ndash82

[36] Mateusz Pawlik and Nikolaus Augsten 2015 Efficient Computation of theTree Edit Distance ACM Trans Database Syst 40 1 Article 3 (2015) 40 pageshttpsdoiorg1011452699485

[37] Riccardo Poli William B Langdon Nicholas F McPhee and John R Koza 2008 Afield guide to genetic programming Lulucom

[38] Andres J Ramirez Betty HC Cheng Philip K McKinley and Benjamin E Beck-mann 2010 Automatically Generating Adaptive Logic to Balance Non-functionalTradeoffs During Reconfiguration In Int Conf on Autonomic Computing (ICAC)225ndash234

[39] V Roberge M Tarbouchi and G Labonte 2013 Comparison of Parallel GeneticAlgorithm and Particle Swarm Optimization for Real-Time UAV Path PlanningIEEE Trans on Industrial Informatics 9 1 (2013) 132ndash141 httpsdoiorg101109TII20122198665

[40] Amir Molzam Sharifloo Andreas Metzger Cleacutement Quinton Luciano Baresi andKlaus Pohl 2016 Learning and Evolution in Dynamic Software Product LinesIn Proceedings of the 11th International Symposium on Software Engineering forAdaptive and Self-Managing Systems (SEAMS rsquo16) ACM New York NY USA158ndash164 httpsdoiorg10114528970532897058

[41] E L Da Silva H A Gil and J M Areiza 2000 Transmission network expansionplanning under an improved genetic algorithm IEEE Trans on Power Syst 15 3(2000) 1168ndash1174

[42] Tyron Stading Petros Maniatis and Mary Baker 2002 Peer-to-Peer CachingSchemes to Address Flash Crowds In Int Workshop on Peer-to-Peer Systems (IPTPSrsquo02) 203ndash213

[43] Daniel Sykes William Heaven Jeff Magee and Jeff Kramer 2007 Plan-directedArchitectural Change for Autonomous Systems In Conf on Specification andVerification of Component-based Syst (SAVCBS rsquo07) 15ndash21

[44] Leonardo Trujillo 2011 Genetic programming with one-point crossover andsubtree mutation for effective problem solving and bloat control Soft Computing15 8 (2011) 1551ndash1567

[45] Manuela M Veloso 1994 Flexible Strategy Learning Analogical Replay ofProblem Solving Episodes In Nat Conf on Artificial Intelligence (AAAI rsquo94)595ndash600

[46] Thomas Vogel and Holger Giese 2010 Adaptation and Abstract Runtime ModelsIn Proceedings of the 2010 ICSE Workshop on Software Engineering for Adaptiveand Self-Managing Systems (SEAMS rsquo10) ACM New York NY USA 39ndash48 httpsdoiorg10114518089841808989

[47] Eckart Zitzler Marco Laumanns and Lothar Thiele 2001 SPEA2 Improving theStrength Pareto Evolutionary Algorithm Technical Report

[48] Parisa Zoghi Mark Shtern Marin Litoiu and Hamoun Ghanbari 2016 DesigningAdaptive Applications Deployed on Cloud Environments Trans Auton AdaptSyst 10 4 Article 25 (2016) 26 pages

Abstract

1 Introduction

2 Background

21 Self- Planners

22 Genetic Programming

3 Running Scenario

31 Scenario

32 Quality objectives

33 Adaptation tactics

34 Post-design-time adaptation

4 Approach

41 Representation

42 Mutation and Crossover

43 Fitness

44 Plan Reuse

5 Evaluation

51 Comparative Study

52 Reuse-Enabling Techniques

53 Unforeseen Adaptation Scenarios

54 Diversity

6 Related Work

7 Conclusion

References


We have previously argued [9] that this is a fruitful potentialdomain for the application of stochastic algorithms to self- systemsStochastic search techniques have been shown to be well-suited forsimilar problems [3 6 11 17 38 39 41] However these approachesdo not address the challenge of replanning for new explicitly un-foreseen contexts post-design time which we propose to addressthrough reusing prior plans Intuitively genetic algorithms shouldbe expected to benefit from reused information since they oper-ate by balancing between exploring new solutions and exploitingexisting solutions from generation to generation in effect reusinginformation from previous generations This observation has beensuccessfully applied in other domains such as automated programrepair [16] We investigate the extent to which reusing existingplans in self- planning can result in an improvement in the ful-filling the systemrsquos quality objectives Surprisingly we find thatreusing plans directly is less effective than replanning from scratchWe further propose a series of techniques to make reusing existingplans more efficient ultimately obtaining a planner that can reuseprior plans to improve the systemrsquos quality objectives

We present a self-adaptive systems planner built on genetic pro-gramming that responds to unforeseen adaptation scenarios byreusing and building upon prior knowledge We represent individ-uals as candidate plans evaluating individual fitness by runningthem against a simulated system Our approach explicitly takesinto account the probability that individual tactics may fail andsupports reasoning about tactic latency and planning time

Our planner reuses past information by initializing the popula-tion with individuals based on an existing plan Our main contri-bution is a series of techniques to support adapting to unexpectedchanges at runtime by lowering the costs of plan reuse during evo-lution and an empirical study investigating the utility of plan reusefor several indicative change scenarios Our key contributions arebull An investigation into plan reuse in genetic algorithm plan-

ning finding counter-intuitively that naiumlve reuse can lowerplanning utility

bull A set of techniques for lowering the cost of plan reuse re-sulting in a self- planner that can reuse past information torespond to unforeseen changes more effectively

bull As a sanity check an empirical comparison of our geneticprogramming planner to a PRISM MDP planner [34] thatshows the genetic programming planner can produce near-optimal plans (005 error in the single objective scenarioand 94 in the multi-objective scenario)

bull An investigation into the time quality and population di-versity produced by planning with reuse when adapting tounforeseen scenarios compared to planning from scratchWe find that while the improvement is often slight effectiveplan reuse can result in a fitness improvement

bull Results that show that the objectives emphasized in a multi-objective plannerrsquos starting plan can influence the qualityand character of the plannerrsquos output

Our initial position [9] identified key research questions that weaddress in this work further expanding upon our prior concept inseveral ways a more expressive individual representation inspiredby true planning languages [8] a significantly more efficient fit-ness evaluation strategy a series of techniques for supporting planreuse by reducing the search cost a comparative evaluation to an

Figure 1 MAPE-K Loop for self- systems

exhaustive planner [34] an experimental study investigating thetrade-offs of plan reuse when adapting to unforeseen scenarios andsupport for multi-objective search [47]

The rest of the paper is as follows Section 2 outlines necessarybackground We next describe the running example we use to bothillustrate and evaluate our technique (Section 3) Section 4 detailsour genetic programming self- planning approach Section 5 de-scribes our evaluation Section 6 outlines related work Section 7concludes and offers discussion

2 BACKGROUNDThis section overviews self- planning (Section 21) and geneticprogramming (Section 22) focusing on the background requiredto understand our approach

21 Self- PlannersSelf- systems typically consist of two subsystems a managedsystem and a managing system Many self- systems follow thewell-known five-component MAPE-K architecture [20] shown inFigure 1 We focus on the planning (P) component which pro-duces strategies consisting of tactics ordered to achieve a particulargoal For our purposes tactics are architectural changes the sys-tem can perform to respond to changes eg ldquoturn off a server atlocation Ardquo While multiple planning languages exist for the self-context [26 43] our approach is closest to Stitch [8] An onlineplanner [43] generates a plan at run-time which can adapt quicklyat a potential cost to optimality An offline planner [8] precomputesstrategies to handle common cases and then chooses between themat run-time This allows for a fast correct response to known or pre-dicted situations but cannot handle unanticipated adaptation needsWe compare to a previous hybrid onlineoffline approach [34] thatrelies in part on Markov decision processes (MDPs) formal modelsthat can explicitly capture probabilistic behavior Model checkerssuch as PRISM [26] can use exhaustive search to compute an opti-mal sequence of tactics to maximize one or more system objective(eg profit) for systems formalized as MDPs






















StartServer A

StartServer A

010

StartServer B

090

9878

010

11373

090

11373

010

15266

090























2750

2800

2850

2900

2950

3000


Pro

fit PlannerPRISMGP

0

1000

2000

3000

4000

0 2500 5000 7500Latency

Pro

fit

Planner

PRISM

GA planner
































17500000

20000000

22500000

25000000

27500000

30000000

25e+07

30e+07

35e+07

40e+07

45e+07

50e+07

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19Generation

Pro

fit

StartingPlan

reuse

scratch




293e+07

294e+07

295e+07

210e+07

215e+07

220e+07

290e+07

291e+07

292e+07

293e+07

294e+07

295e+07

26e+07

27e+07

28e+07

29e+07

45e+07

46e+07

47e+07

48e+07

46e+07

47e+07

48e+07

49e+07

50e+07

0 25 50 75 100 0 20 40 60 0 25 50 75 100

0 50 100 150 200 0 50 100 150 0 50 100 150 200


Pro

fit

StartingPlan

reusescratch


























02

04

06

02

04

06

0 10 20 0 10 20 0 10 20Generation

Nor

mal

ized

Div

ersi

ty(a

vera

ge p

airw

ise

tree

edi

t dis

tanc

e)

StartingPlan

mutator

scratch

trimmer




























































Abstract

1 Introduction

2 Background

21 Self- Planners


3 Running Scenario

31 Scenario




4 Approach

41 Representation


43 Fitness

44 Plan Reuse

5 Evaluation




54 Diversity

6 Related Work

7 Conclusion

References






















StartServer A

StartServer A

010

StartServer B

090

9878

010

11373

090

11373

010

15266

090























2750

2800

2850

2900

2950

3000


Pro

fit PlannerPRISMGP

0

1000

2000

3000

4000

0 2500 5000 7500Latency

Pro

fit

Planner

PRISM

GA planner
































17500000

20000000

22500000

25000000

27500000

30000000

25e+07

30e+07

35e+07

40e+07

45e+07

50e+07

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19Generation

Pro

fit

StartingPlan

reuse

scratch




293e+07

294e+07

295e+07

210e+07

215e+07

220e+07

290e+07

291e+07

292e+07

293e+07

294e+07

295e+07

26e+07

27e+07

28e+07

29e+07

45e+07

46e+07

47e+07

48e+07

46e+07

47e+07

48e+07

49e+07

50e+07

0 25 50 75 100 0 20 40 60 0 25 50 75 100

0 50 100 150 200 0 50 100 150 0 50 100 150 200


Pro

fit

StartingPlan

reusescratch


























02

04

06

02

04

06

0 10 20 0 10 20 0 10 20Generation

Nor

mal

ized

Div

ersi

ty(a

vera

ge p

airw

ise

tree

edi

t dis

tanc

e)

StartingPlan

mutator

scratch

trimmer




























































Abstract

1 Introduction

2 Background

21 Self- Planners


3 Running Scenario

31 Scenario




4 Approach

41 Representation


43 Fitness

44 Plan Reuse

5 Evaluation




54 Diversity

6 Related Work

7 Conclusion

References



















2750

2800

2850

2900

2950

3000


Pro

fit PlannerPRISMGP

0

1000

2000

3000

4000

0 2500 5000 7500Latency

Pro

fit

Planner

PRISM

GA planner
































17500000

20000000

22500000

25000000

27500000

30000000

25e+07

30e+07

35e+07

40e+07

45e+07

50e+07

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19Generation

Pro

fit

StartingPlan

reuse

scratch




293e+07

294e+07

295e+07

210e+07

215e+07

220e+07

290e+07

291e+07

292e+07

293e+07

294e+07

295e+07

26e+07

27e+07

28e+07

29e+07

45e+07

46e+07

47e+07

48e+07

46e+07

47e+07

48e+07

49e+07

50e+07

0 25 50 75 100 0 20 40 60 0 25 50 75 100

0 50 100 150 200 0 50 100 150 0 50 100 150 200


Pro

fit

StartingPlan

reusescratch


























02

04

06

02

04

06

0 10 20 0 10 20 0 10 20Generation

Nor

mal

ized

Div

ersi

ty(a

vera

ge p

airw

ise

tree

edi

t dis

tanc

e)

StartingPlan

mutator

scratch

trimmer




























































Abstract

1 Introduction

2 Background

21 Self- Planners


3 Running Scenario

31 Scenario




4 Approach

41 Representation


43 Fitness

44 Plan Reuse

5 Evaluation




54 Diversity

6 Related Work

7 Conclusion

References




















17500000

20000000

22500000

25000000

27500000

30000000

25e+07

30e+07

35e+07

40e+07

45e+07

50e+07

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19Generation

Pro

fit

StartingPlan

reuse

scratch




293e+07

294e+07

295e+07

210e+07

215e+07

220e+07

290e+07

291e+07

292e+07

293e+07

294e+07

295e+07

26e+07

27e+07

28e+07

29e+07

45e+07

46e+07

47e+07

48e+07

46e+07

47e+07

48e+07

49e+07

50e+07

0 25 50 75 100 0 20 40 60 0 25 50 75 100

0 50 100 150 200 0 50 100 150 0 50 100 150 200


Pro

fit

StartingPlan

reusescratch


























02

04

06

02

04

06

0 10 20 0 10 20 0 10 20Generation

Nor

mal

ized

Div

ersi

ty(a

vera

ge p

airw

ise

tree

edi

t dis

tanc

e)

StartingPlan

mutator

scratch

trimmer




























































Abstract

1 Introduction

2 Background

21 Self- Planners


3 Running Scenario

31 Scenario




4 Approach

41 Representation


43 Fitness

44 Plan Reuse

5 Evaluation




54 Diversity

6 Related Work

7 Conclusion

References


















02

04

06

02

04

06

0 10 20 0 10 20 0 10 20Generation

Nor

mal

ized

Div

ersi

ty(a

vera

ge p

airw

ise

tree

edi

t dis

tanc

e)

StartingPlan

mutator

scratch

trimmer




























































Abstract

1 Introduction

2 Background

21 Self- Planners


3 Running Scenario

31 Scenario




4 Approach

41 Representation


43 Fitness

44 Plan Reuse

5 Evaluation




54 Diversity

6 Related Work

7 Conclusion

References




















































Abstract

1 Introduction

2 Background

21 Self- Planners


3 Running Scenario

31 Scenario




4 Approach

41 Representation


43 Fitness

44 Plan Reuse

5 Evaluation




54 Diversity

6 Related Work

7 Conclusion

References

managing uncertainty in self-adaptive systems with plan reuse …kinneerc.github.io › assets ›...

Documents