optimization of agent-based modelsjasss.soc.surrey.ac.uk/17/2/6/6.pdf · 1.9 agent-based models are...
Post on 28-May-2020
4 Views
Preview:
TRANSCRIPT
©CopyrightJASSS
MatthewOremlandandReinhardLaubenbacher(2014)
OptimizationofAgent-BasedModels:ScalingMethodsandHeuristicAlgorithms
JournalofArtificialSocietiesandSocialSimulation 17(2)6<http://jasss.soc.surrey.ac.uk/17/2/6.html>
Received:28-Jun-2013Accepted:08-Dec-2013Published:31-Mar-2014
Abstract
Questionsconcerninghowonecaninfluenceanagent-basedmodelinordertobestachievesomespecificgoalareoptimizationproblems.Inmanymodels,thenumberofpossiblecontrolinputsistoolargetobeenumeratedbycomputers;hencemethodsmustbedevelopedinordertofindsolutionsthatdonotrequireasearchoftheentiresolutionspace.Modelreductiontechniquesareintroducedandastatisticalmeasureformodelsimilarityisproposed.Heuristicmethodscanbeeffectiveinsolvingmulti-objectiveoptimizationproblems.Aframeworkformodelreductionandheuristicoptimizationisappliedtotworepresentativemodels,indicatingitsapplicabilitytoawiderangeofagent-basedmodels.Resultsfromdataanalysis,modelreduction,andalgorithmperformanceareassessed.
Keywords:Agent-BasedModeling,Optimization,StatisticalTest,GeneticAlgorithms,Reduction
Introduction
1.1 Agent-basedmodels(ABMs)areoftencreatedinordertosimulatereal-worldsystems.Inmanycases,ABMsactasinsilicolaboratorieswhereinquestionscanbeposedandinvestigated;suchquestionsoftenarisenaturallyinthecontextofthesysteminquestion.Forexample,anABMofafinancialnetworkmightbeusedtodeterminewhichpoliciesleadtomaximizedprofit,whileanABMmodelingsocialnetworksmightbestudiedtodeterminethemosteffectivemeansoftransmittinginformation.QuestionsconcerninghowonecaninfluenceanABMinordertobestachievesomespecificgoalareoptimizationproblems.Inothercontexts,optimizationmayrefertoparametersormodeldesign.Itisimportanttoreiteratethatthemeaningoftheterminthisstudyisdifferent–itreferstotheoptimalchoiceofasequenceofexternalinputstoamodelinordertoachieveaparticulargoal.ThestochasticityinherentinmanyABMsmeansthatcaremustbetakenwhenattemptingtosolveoptimizationproblems.Underfixedinitialconditions,datafromindividualsimulationreplicationsoftenvary.Thus,carefulanalysisofABMdynamicsisaprerequisiteforthedevelopmentofoptimizationtechniques.Inparticular,statisticalmethodsmustbebroughttobearontheinterpretationofsimulationresults.
1.2 Inthisstudy,statisticalandoptimizationtechniquesarepresentedwhichcanbeapplieddirectlytoABMs:translationofthemodeltoanequation-basedformisnotnecessary.Thereareseveraladvantagestothisapproach–suchtechniquescanbeappliedtovirtuallyanyABM,andthereisnoneedfortransformationofeitherthemodelorthecontrols.Repeatedsimulationisusedtoobtainreliableresults,andcontrolsareapplieddirectlytotheABMs.Whiletheremaybemodelsforwhichthisapproachfails,thesufficientlybroadexamplesprovidegoodevidencethatforlargeclassesofABMs,meaningfulresultscanbeobtainedbydirectanalysisandoptimization.
1.3 Thegoalofthispaperistointroduceandillustrateaframeworkforsolvingoptimizationproblemsusingagent-basedmodels.Ingeneral,thenumberofpossiblesolutionstoanoptimizationproblemisfartoolargeforenumeration.Thus,heuristicmethodsmustbeemployedtoanswersuchquestions.Computationalefficiencyisakeyfactorinthisprocess;assuch,theuseofscaledapproximationscanbeinvaluable.Aslongasascaledmodelfaithfullymaintainsthedynamicsoftheoriginal,itcanbeusedtosolvetheoptimizationproblem,resultinginareductionofruntimeandcomputationalcomplexity.
1.4 Thepaperisorganizedasfollows:standardsfordataanalysisareestablishedandastatisticalmeasureformodelsimilarityisproposed.AheuristictechniqueknownasParetooptimizationisproposedasameansforsolvingoptimizationproblems.TheframeworkispresentedviatheuseoftwomodelsactingasrepresentativesoflargeclassesofABMs,whichoughttoholdinterestforresearchersfromawidevarietyofdisciplines.Briefmodeldescriptionsareoutlinedinthetext,anddetailedmodeldescriptionsfollowingtheOverview,DesignConcepts,andDetails(ODD)protocolforagent-basedmodels(Grimmetal.2006;Grimmetal.2010)areprovidedintheappendices.Thesedescriptionsoughttoprovideenoughdetailthatthemodel(andresults)canbereconstructedandverifiedbyindependentresearch.
Relatedwork
1.5 Optimizationproblemsofthetypepresentedherehavebeenstudiedinmodelsofinfluenzaandepidemics(Kasaieetal.2010;Yangetal.2011),cancertreatment(Lollinietal.1998;Swierniaketal.2009),andthehumanimmunesystem(Bernaschi&Castiglione2001;Rapinetal.2010),tonameafew.Previousstudieshaveinvestigatedtheeffectofvariousmodelfeaturesonoutcomes–forexample,subwaytravelonthespreadofepidemics(Cooleyetal.2011),mobilityandlocationinamolecularmodel(Klannetal.2011),molecularcomponentsinacancermodel(Wangetal.2011),andstrategiesformitigatinginfluenzaoutbreaks(Mao2011)–whilenotquiteposingformaloptimizationproblems.AstudyontheeffectofABMsindeterminingmalariaeliminationstrategies(Ferreretal.2010)suggeststhatresultsfromagent-basedmodelsareinvaluableintheanalysisofinterventions.
1.6 Inotherstudies,ABMshavebeentransformedintosystemsofdifferentialequations(Kimetal.2008a)andpolynomialdynamicalsystems(Hinkelmannetal.2011;Veliz-Cubaetal.2010),amongothers.Theimportanceofspatialheterogeneityhasbeenexaminedinspecific(Haradaetal.1995)andmoregeneral(Happe2005)cases,andpredator-preyABMshavebeenanalyzedusingstatisticalmethods(Wilsonetal.1993;Wilsonetal.1995).
Aframeworkforsolvingoptimizationproblems
1.7 TheframeworkissummarizedinFigure1;subsequentsectionsmotivateandexplaintheprocessindetail.
http://jasss.soc.surrey.ac.uk/17/2/6.html 1 16/10/2015
Figure1.Anoverviewoftheframeworkpresentedinthiswork.Thethreeshadingsrepresentthephasesofanalysis,scaling,andoptimization.
DataReliability
1.8 Akeyfactorinanalysisofagent-basedmodelsisstochasticity.Theapproachsuggestedhereistoexaminehowdataaverageschangeasthenumberofsimulations(runs)increases.Inmanycases,thedatawillsettleinonsomeaveragethatisnotimproveduponbyincreasingthenumberofruns.Determiningasufficientnumberofrunsisthefirststepinobtainingreliableresults.Theemphasisonsolvingoptimizationproblemsnecessitatesthisprocess:whilesomeofthestochasticityinherenttoanindividualrunislostwhenaveragingoverrepeatedruns,itisnecessaryinordertodeterminethegeneralefficacyofonecontrolversusanother.
1.9 Agent-basedmodelsareoftenimplementedonagrid,representingthe'space'ofthemodel(oftentimes,thegridindeedrepresentssomephysicalspace).Treatingtheoriginalsizeandscopeofthemodelastrue,thegoalofscalingistodeterminetheextenttowhichamodelcanbereducedwithoutalteringpertinentdynamics.Themodelsexaminedherecontainphysicalagentstraversingphysicallandscapes.Inthissetting,thestrategyistograduallyscaledownthemodeluntilthedynamicsnolongerfaithfullyrepresenttheoriginalmodel.Whenapplicable,thisstrategyresultsinreducedruntime–inmanycasessubstantiallyso–reducingthecomputationalrequirementsforthesolvingofoptimizationproblemsandallowingaccesstoawiderrangeofanalyticaltools.
1.10 Determiningtowhatdegreeareducedmodelisafaithfulrepresentationoftheoriginalisanimportantquestion.Intermsofoptimization,itisnecessarytodeterminetheextenttowhichmodelscanbereducedforthepurposeofoptimalcontrol.Inordertoaccomplishthis,asampleofthecontrolspaceisimplementedinboththeoriginalmodelandreducedversions.Foreachreducedversion,thecontrolsarerankedaccordingtotheireffectivenessinregardstotheoptimizationorcontrolobjective.Theaimistouseareducedmodelasaproxyfortheoriginal;thus,therankingofthecontrolsonthereducedmodelmustbecomparedtotherankingofthesamecontrolsappliedtotheoriginal.
1.11 WeproposeCohen'sweightedκ(Cohen1968)asameasureofconcordanceofrankingsfordifferentmodelsizes.Letpobsbetheobservedproportionofagreementinthetwolistsandletpexpbetheproportionofagreementexpectedbyrandomchance.Thenκ=(pobs−pexp)/(1−pexp).Henceifthelistsareinperfectagreement,κ=1;ifthelistsarenomoresimilarthanwhatisexpectedpurelybychance,κ=0.Thissimilaritymetricforrankedlistsdeterminespenaltiesbasedonthemagnitudeofdisagreement.Fordetailsofhowtocalculatepobs,pexp,andweightedpenalties,seeCohen(1968).
1.12 Forexamplesoftheuseofthisstatisticasameasureofagreement,seeFleiss(1971)andEugenio(2000).Cohen'sweightedκischosenbecauseofitswidedocumentationandimplementationinavarietyofstudies;assuch,thereisprecedentforthismeasure.Thereisnoobjectivewaytodetermineabenchmarkvalueforκ.Severalstudiesproposeaκvaluegreaterthan0.75asbeingverygood(Altman1991;Fleiss1981),whileothersrecommendavalueof0.8orhigher(Landis&Koch1977;Krippendorff1980).Inthisstudynobenchmarkisset;rather,κvaluesareassessedaposteriori.Formoredetailsonsettingabenchmarkforκ,seeSimandWright(2005),andElEmam(1999).
1.13 ItisofcoursenotguaranteedthatallABMswillbeamenabletothestrategiespresentedhere(fordiscussiononthisissueseeDurrettandLevin(1994)).Infact,modelsmayexistforwhichnoreductionispossible–nevertheless,reductionstrategiesarefrequentlyusefulandinvariablyinformative.Inparticular,theinvestigationofdifferencesinqualitativebehaviorcanbeservedbythese(andother)methodsofmodelreduction.ForexamplesofmodelreductionstrategiesappliedtoABMs,seeZouetal.(2012),Roosetal.(1991),andYesilyurtandPatera(1995).Itisalsoworthnotingthat'modelreduction'isaphrasewhosemeaningmaybediscipline-dependent:theextenttowhichamodelcanbereducedisdependentonwhichmeaningistakenandwhichmodeldetailsonewishestopreserve.
ParetoOptimization
2.1 Onceasuitablereductionhasbeenmade,anoptimizationproblemcanbesolvedusingthereducedmodelasasurrogatefortheoriginal.PerhapsthemostexploredmethodforoptimalcontrolofABMshascomeintheformofheuristicalgorithms.Giventhatenumerationofthesolutionspaceisofteninfeasible,heuristicalgorithmsareusedtoconductaguidedsearchofthesolutionspaceinordertodeterminelocallyoptimalcontrols.
2.2 SeveralheuristicalgorithmshavebeenutilizedinsolvingoptimizationproblemsforABMs.Examplesincludesimulatedannealing(Pennisietal.2008),tabusearch(Wang&Zhang2009),andsqueakywheeloptimization(Lietal.2011).Inthisstudy,attentionisfocusedonacertaintypeofgeneticalgorithm(GA).Thesealgorithms,firstbroughttogeneralattentionin1989(Goldberg1989),aredesignedtomimicevolution:solutionsthataremorefitareusedto'breed'newsolutions.GAshavebeenusedinconjunctionwithABMstofindoptimalvaccinationschedulesforinfluenza(Pateletal.2005),cancer(Lollinietal.2006),andindeterminingoptimalanti-retroviralschedulesforHIVtreatment(Castiglioneetal.2007).Vaccinationscheduleoptimizationresultsobtainedfromsimulatedannealingandgeneticalgorithmshaveevenbeencomparedandcontrasted(Pappalardoetal.2010).AstheprimaryfocusofthispaperistointroduceageneralframeworkforsolvingoptimizationproblemsforABMs,acomparisonofvariousheuristicmethodsisoutsidethescopeofthisstudy.ForamorecomprehensivelookatheuristiccontrolofABMs,seeOremland(2011).
2.3 Thecontrolproblemsdescribedherehavemultipleobjectives–thisnecessitatesassigningweightstoeachobjective.Determinationofweightsinmulti-objectiveoptimizationproblemscanbeproblematicbecauseapriori,theappropriateweightsmaybeunknown–inparticular,theassignmentisatthediscretionoftheinvestigator.Whiletherehavebeenvariousproposalsfortheseassignments(foranexample,seeGennertandYuille(1988)),anymethodwhichdoesnotrequireweightshasparticularappeal.
2.4 Paretooptimizationisjustsuchaheuristicmethod:insteadofafocusingonasinglesolution,thealgorithmreturnsasuiteofsolutions.SolutionsontheParetofrontierrepresentthosethatcannotbeimproveduponintermsofoneobjectivewithoutsomesacrificeinanother.Inthissense,eachsolutionontheParetofrontierisoptimalwithrespecttosomechoiceofweights.Thus,the'managerial'decisionofhowtoassignweightscanbesettledafterthesearchhasconcluded.
2.5 Anextensivelistofreferencesonmulti-objectiveoptimizationtechniquescanbefoundinCoello(2013).ParetooptimizationhasbeenselectedforthisstudyasitisnovelinitsapplicationtoABMs.ThealgorithmadoptedhereisaminorvariantofthatdescribedinHornetal.(1994):itisaheuristicalgorithmthatsearchesthecontrolspaceinanattempttofindtheParetofrontier.PseudocodeforthealgorithmispresentedinAlgorithm1.
http://jasss.soc.surrey.ac.uk/17/2/6.html 2 16/10/2015
Algorithm1.PseudocodefortheParetooptimizationalgorithm.
Software
3.1 Theproposedframeworkrequirestwotypesofsoftware:modeling,andstatisticalanalysis.Agent-basedmodelscanbeimplementedinavarietyofsoftwarepackages.Someofthese,suchasNetLogo(Wilensky2009),Repast(Northetal.2006),andMASON(Lukeetal.2005)havebeendesignedforgeneralagent-basedmodeling.Othersoftwarehasbeendevelopedforagent-basedmodelinginspecificfields–theseincludeC-ImmSim,VaccImm,andSIMMUNEforthehumanimmunesystem(Castiglione1995;Woelkeetal.2011;Meier-Schellersheim&Mack1999),FluTEforinfluenzaepidemiology(Chaoetal.2010),andSnAPforpublichealthstudies(Buckeridgeetal.2011).Whileonecanalwaysimplementone'sowntoolkitforexaminingagent-basedmodels,theuseofestablishedsoftwarecanreduceboththevariabilitybetweenresearchers'implementationsandthelearningcurveforconductingresearchinthisfield.NetLogowaschosenasthemodelingplatforminthisstudy,thoughthereisnoreasonwhythestudycouldnothavebeenundertakenusinganumberofdifferentsoftwarepackages.AstandardNetLogoinstallationcontainsanextensivelibraryofmodelsfromavarietyoffields;themodelsdiscussedhereareadaptationsofpopularmodelsfromthisbuilt-inlibrary.Statisticalanalysiscanbeperformedbyvirtuallyanystatisticalsoftwarepackage;inthisstudy,MicrosoftExcelwasused.DuetothefactthatsimulationdatawasneededinordertoperformParetooptimization,thisprocesswasimplementedinNetLogoaswell.Ingeneral,thetechniquesdescribedherearesufficientlystraightforwardthathighlyspecializedsoftwareisnotneeded,andtheframeworkisnotlimitedtoanyparticularsoftwarechoice.
TwoModels
RabbitsandGrass
4.1 ThefirstmodeltobeexaminedisbasedonasamplemodelfromtheNetLogolibrary(Wilensky2009)involvingrabbitsinafield.Ateachtimestep,eachrabbitmoves,eatsgrass(ifthereisgrassatitslocation),andthenpossiblyreproducesordies,basedonitsenergylevel.Thereareseveralcompellingreasonsfortheuseofthismodelasatestcasefortheproposedframework.Oneisthatthemodelissufficientlysimpletodescribe,soresultscanbeobtained,interpreted,andunderstoodwithminimaloverhead.Amoreimportantreasonisthatthismodelrepresentsthecategoryofgeneralpredator-preysystems(withgrassfunctioningasprey).Suchmodelsarecommonlyusedinecologyandhavebeenwidelystudied.Thus,theframeworkcanbepresentedthroughanexamplethatappealstoabroadcommunityofresearchers.Indeed,thismodelillustratesmanyconceptscommoninABMscontaininginteractingspecies.AdetaileddescriptionofthemodelandalistofparametervaluesareprovidedinAppendixA.
4.2 Controlconsistsofdeciding(eachday)whetherornottoapplypoisontothegrid(i.e.,uniformlytoallgridcells).Specifically,thecontrolobjectiveistodetermineapoisonscheduleuthatminimizesthenumberofrabbitsalivethroughoutthecourseofasimulationwhilealsominimizingthenumberofdaysonwhichpoisonisused.Notethatitisunlikelythatonecontrolschedulewillminimizebothobjectivessimultaneously:forexample,thecontrolwhereinnopoisonisusedcertainlyminimizesthesecondobjective,butnotthefirst.Thus,thisproblemisagoodcandidateforParetooptimization:asuiteofsolutionscanbefound,eachmemberofwhichisoptimaldependingontheweightsassignedtothetwoobjectives.
Scalingresults
4.3 Oneofthecontrolobjectivesconcernstheaveragenumberofrabbitsaliveoverthecourseofasimulation;thus,thisisthepertinentmetricintermsofmodelreduction(giventhattheothercontrolobjective–minimizingdaysonwhichcontrolisused–isentirelypreservedatanymodelsizeandforanynumberofruns).
4.4 AsnotedinDataReliability,thefirstconsiderationwhenattemptingtoscalethemodelisdeterminingthenumberofrunsnecessaryinordertoachievereliableresults.Tothisend,severalcontrolscheduleswereselectedatrandom.Eachwasappliedtotheoriginal50×50model,andresultsweretalliedupto100runs.PopulationdynamicsforthreerandomlyselectedcontrolschedulesarepresentedinFigure2:plotsshowhowtheaveragenumberofrabbitsaliveoverthecourseoftheschedulechangeasthenumberofrunsincreases,witherrorbarsrepresentingonestandarddeviation.
http://jasss.soc.surrey.ac.uk/17/2/6.html 3 16/10/2015
(a)10controldays
(b)30controldays
http://jasss.soc.surrey.ac.uk/17/2/6.html 4 16/10/2015
(c)50controldaysFigure2.Averagepopulationvaluessettleintoconsistentvaluesaround50runs.
4.5 Thesethreeschedulesrepresentthreedistinctregionsofthecontrolspace,inthateachcontainsadifferentnumberofcontroldays.Notethatinallcases,thereislittlechangeinthemeanorthestandarddeviationbeyond50runs–thissuggeststhatthereisnoadvantageinaveragingovermorethan50runs.Itisimportanttonotethatifthecontrolobjectiveswerealtered(forexample,ifgrasslevelswereofinterestratherthanrabbitlevels)thenthisconclusionmaynothold.Inparticular,thenecessarynumberofrunsdependsuponthemodeldynamicsofinterest–inthiscase,theaveragenumberofrabbits.
4.6 Onceabenchmarkforreliableresultshasbeenestablished,variousmodelreductionscanbeinvestigatedwithrespecttocontrol.Intheoriginalmodel,thereare50×50=2500patchesand
150rabbitsinitially.AmodelsizeofMmeansthattheworldwidthandheightarebothM.Hence,whenreducingthemodeltosizeM,theinitialnumberofrabbitsoughttobe150(M2/2500)inordertomaintainthesameproportionofrabbitstomodelsize.Allotherstatevariablevaluesremainthesame.
4.7 Foreachofasetofcontrolsappliedtotheoriginalmodel,averagerabbitnumbersareobtainedviasimulation;thecontrolscanthenberankedbythesenumbers.Thesesamecontrolscanbeappliedtoareducedmodel,resultingina(potentially)differentrankedlist.Theκstatistic(seeDataReliability)measuresthesimilaritybetweenthetworankings,therebyservingasameasureoftheextenttowhichthereducedmodelservesasasubstitutefortheoriginal.Itisimportantthattheserankingsaremaintainedoverawiderangeofcontrolschedules,sincesolvingtheoptimalcontrolproblemwillinvolvethepotentialexaminationoftheentiresolutionspace.
4.8 Generatingasetofcontrolsrandomlyresultsinanormaldistributioncenteredonsolutionswithfiftyzerosandfiftyones(onesindicatingtimestepsonwhichpoisonisused).Toavoidfocusingontoonarrowaportionofthecontrolspace,astratifiedrandomsamplewastaken:24valuesN1,…,N24werechosenasfrequencynumbers,representingthenumberofonesintheschedule.Thesevalueswerechosenatrandomwithinthefollowingscheme:threevalueswerechosenbetween1and10,threebetween10and20,andsoon,withthefinalthreechosenbetween70and80.Fourcontrolscheduleswerethenrandomlygenerated,eachcontainingNtonesand(100−Nt)zeros(distributedrandomlythroughouttheschedule),fort∈{1,…,24}.Thus,foreachtrial,atotalof96scheduleswereevaluated,chosenasrepresentativesofthesolutionspace.Notethatscheduleswithmorethan80non-zeroentrieswerenotconsidered,aspreliminaryinvestigationshowedthatsuchscheduleswerequicklyeliminatedfromanyheuristicoptimalcontrolsearch.
4.9 Onetrialisdefinedasfollows:96controlschedules(chosenaccordingtotheabovedescription)wererunusingtheoriginalM=50model,andthenagainonthemodelateachofthefollowingmodelsizes:50,40,30,20,10,5,and3.NotethattheschedulesareruntwiceontheoriginalM=50model:thisisdoneinordertoestablishhowconsistenttherankingsarewhenevaluatedtwiceonthesame-sizedmodel.Insomesense,thisservesasvalidationofthechoiceof50simulationsasbeingsufficientforreliableresults,andalsoprovidesinsightintotheanalysisofanappropriatebenchmarkforκ,aswillbeseenbelow.
4.10 Evaluationof150schedules(eachaveragedover50simulations)atmodelsizeM=50requiresapproximately3seconds.Table1givesthenumberofsimulationsthatcanberunforthereducedmodelsinapproximately3seconds.Giventhattheprimaryadvantageforscalingmodelsistoreduceruntime,itismoreappropriatetocomparedatabasedonequivalentruntimeratherthanusingafixednumberofsimulationsforeachsize.Thisdataaidsinscalinganalysis:ifonewishestoreducetheruntimeby50%,thenumberofrunsthatcanbeperformediseasilycalculated.
Table1:Numberofsimulationsinequivalentruntime.
Worldsize Simulations Avg.runtime(sec.)50 50 3.0440 75 3.0030 135 3.0520 290 3.0310 1100 3.035 3500 3.013 5700 3.03
4.11 Figure3summarizesκvaluesforvariousworldsizesandruntimes.Eachdatapointrepresentsthemeantakenovertentrials,witherrorbarsrepresentingonestandarddeviation.Abenchmarkvalueofκ=0.8isplottedaswell–itispresentedtoserveasapreliminarygaugeofhowwellthereducedmodelscapturethedynamicsoftheoriginal.Eachlineonthegraphconnectsdatapointsofequivalentruntime.Figure3helpsidentifyunviablereductions:acceptingabenchmarkofκ=0.8,worldsizesbelow20arenotsufficientlyaccuraterepresentativesoftheoriginalmodel(andthesize20modelisonlysufficientat100%runtime).Thedataalsoshowthatifoneinsistsonusingthesize3model,thebenchmarkforκwillhavetobelowered.Itfurthershowsthatifonewishestousethesize3modelandinsistsonaκvaluehigherthan0.8,itwillcertainlyrequireanincreaseintheruntimeofthemodel(andeventhen,maynotbepossible).
4.12 Severalimportantconclusionscanbedrawnfromthisdata:oneisthatifthepriorityisachievingthehighestpossiblevalueforκ,thentheoriginalsize50modelisalwaysthebestchoiceforanyfixedruntime.Thisisperhapsunsurprising,asonecanonlyexpecttolosesomeaccuracyasmodelsizedecreases.
http://jasss.soc.surrey.ac.uk/17/2/6.html 5 16/10/2015
Figure3.Cohen'sweightedkforvariousworldsizesandruntimes.
4.13 Anotherimportantconclusionisthatiftheonlypriorityisdecreasedruntime,itisalwaysbettertousefewerrunsofthesize50modelratherthanmorerunsofasmallermodel.Thisfollowsbecauseeachlinerepresentsafixedruntime,andforanyfixedruntime,thesize50modelresultsinthehighestvalueforκ.Afixedbenchmarkforκfurtherinformsaresearcherwithapriorityofreducedruntime:asthedatashow,ifonewishestokeepκabove0.8,thenitispossibletoreducetheruntimeby90%,butnotfurther(asindicatedbythedataforthesize50modelat0.3seconds).Thus,notonlycanonedeterminewhichworldsizeshouldbeusedinordertoobtainminimumruntime,butalsotheminimumruntimethatcanbeachievedinordertomaintainapre-setbenchmarkforκ.
4.14 Theremaybecaseswhereareducedmodelisofparticularinterest–forexample,Hinkelmannetal.(2011)describesmethodsfortranslatingABMsintopolynomialdynamicalsystems,offeringadvantagessuchassteadystateandbifurcationanalysis.Thenumberofrequiredequationsmaybetoolargeforthesize50model,butnotsoforthesize10model.Asimilarconcernappliestodifferentialequationsapproximations.ExamplesoftheseconsiderationsarediscussedinKimetal.(2008a)andKimetal.(2008b).Hence,dependingonthepriorityofthemodeler,thedatahereshowwhichworldsizesmaybeusedandwhatκvaluescanbeexpectedwhendoingso.Furthermore,notethatforsmallerworldsizestherangeofκvaluesisdecreased.Inparticular,inordertoachieve1%runtimeonthe50×50model,κdecreasesfrom0.90to0.68.However,inordertoachievea1%runtimeonthe3×3model,κdecreasesfrom0.38to0.33.Thus,forsmallermodelsκmaybelessaffectedbyadecreaseinruntime.
4.15 Inadditiontotheaboveconclusions,thedatainformsthestudyofpointsatwhichthedynamicsofthemodelundergoaqualitativechange.Thereisalargerchangeinreducingfromworldsize10to5thanthereisingoingfromsize20to10;thisindicatesthedynamicsaremorerapidlychangingbetweenworldsizes10and5.Inparticular,thedataseemtosuggestthatthepertinentdynamicsarenotdrasticallyalteredbetweentheoriginalsize50modelandthesize20model,butchangeratherquicklyatsmallersizes.Thisisofparticularinterestinlightofthefactthattheoriginalworldsizewaschosenmoreorlessarbitrarily.Ifonebeganwithasize10model,itmaynotbepossibletoreduceittothesameextentthatonecanreduceasize50model.
4.16 AsmentionedinDataReliability,severalstudiessuggestaκvalueof0.8asabenchmarkforsufficientsimilarity.Whilelargelycitedandused,theapplicabilityofthisvalueoughttobeexaminedinlightoftheresultsobtainedbyheuristicalgorithms.Inparticular,foreachmodelsizeanappropriateκvaluecanbedeterminedaposterioribasedonsaidresults.Thegoalofthismodelreductionanalysisisnottoprescribewhichmodelsizeone'should'use;rather,giventhattheprocessdependsonthepriorityofthemodeler,thegoalistopresentκvaluesandruntimesonecanexpectwhenusingaparticularreducedmodel.
ResultsfromParetoOptimization
4.17 AsdiscussedinParetooptimization,thegoalofaParetooptimizationalgorithmistoreturnasuiteofsolutions,eachofwhichisoptimalforaparticularchoiceofobjectiveweights.Forthismodel,theobjectivesaretominimizethenumberofdaysonwhichcontrolisusedandtominimizethenumberofrabbitsaliveduringthecourseofasimulation.Recallthatacontrolisavectoroflength100withentriesin{0,1}.Figure4showsanexampleoftheParetofrontier.Eachdotcorrespondstoonecontrol,plottedaccordingtothevaluesontheaxes.The×'smakeuptheParetofrontierofthisdataset:foreverypointonthisfrontier,oneoftheobjectivescannotbeimproveduponwithoutsomesacrificeintheother.Ontheotherhand,foreachpointnotonthefrontier,thereexistssomepointinthesetthatimprovesuponbothobjectives:inparticular,foreverysquare(i.e.,non-Paretofrontier)datapoint,thereexistsatleastoneotherpointwithfewercontroldaysandaloweraveragenumberofrabbits.ThegoaloftheheuristicParetooptimizationalgorithmistodetermine,asnearaspossible,thetrueParetofrontierofthecontrolspace.Thus,remainingfiguresconsistofParetofrontiersonlyandnottheentiredatasets.InordertoinvestigateavarietyofκvaluesasdeterminedinScalingresults,severalrepresentativemodelsizesandruntimeswerechosen.ForeachrepresentativemodeltheParetooptimizationalgorithmwasrunandaParetofrontierobtained.IfareducedmodelisasuitablesubstitutefortheoriginalthentheParetofrontierforthereducedmodeloughttobethesameastheParetofrontieroftheoriginalmodel.Foreachreducedmodel,thecontrolsmakingupthefrontierareimplementedintheoriginalmodelinordertodetermineiftheyareactuallyParetooptimal(asthereducedmodelresultshassuggested).NotethatParetooptimizationhasbeenperformedontheoriginalmodelaswellinordertoserveasabasisforcomparison.
4.18 Figure5summarizesresultsforrepresentativemodelswithlowerκvalues.Eachshapecorrespondstoonerepresentativemodel,withresultscomingfromtheimplementationofthesecontrolsintheoriginalmodel.Theseresultssuggestthatmodelswithκvaluesbelow0.5arenotverygoodsurrogatesfortheoriginalmodel.Inparticular,therearefewerdatapoints,andtheytendtoclusternearcertainregionsofthefrontier.Inshort,veryfewofthecontrolsdeterminedtobeParetooptimalbytheserepresentativemodelsareinfactParetooptimalintheoriginalmodel.Figure6showssimilarresultsformodelswithhigherκ
Figure4.AnexampleParetofrontierfortheRabbitsandGrassmodel.Frontierpointsaremarkedwithanxandnon-frontierpointswithasquare.
values.Therepresentativemodelwithaκscoreof0.89producesaParetofrontierveryneartothefrontieroftheoriginalmodel,suggestingthataκvalueof0.89issufficientlyhigh–hence,areducedmodelwithaκvalueof0.89canlikelybeusedasasurrogatefortheoriginalmodel.Thedataforthemodelwithaκvalueof0.76isalsoneartotheParetofrontieroftheoriginalmodel,thoughnottothesameextent.Forthemodelwithκ=0.65,therearefewerdatapoints,andtheseareabitfurtherfromthetruefrontier.
http://jasss.soc.surrey.ac.uk/17/2/6.html 6 16/10/2015
Figure5.Paretofrontiersformodelswithlowerkvalues.
4.19 Finally,Figure7suggeststhatthereisamildlylinearrelationshipbetweentheκvalueofareducedmodelandthenumberofpointsontheParetofrontier.ThesamesettingswereusedforeachParetooptimizationalgorithm;yet,ingeneral,thisdatashowsthatmodelswithlowerκvaluestendtoproducesmallerParetofrontiers(evenwithinthereducedmodelitself).Onepossibleexplanationforthisisthatforareducedmodel,thereisanarrowerrangeinthepossibledynamicsofamodel,andthusthetrueParetofrontierforareducedmodelmayindeedbesmaller.Thus,again,κvaluesindicatetheextenttowhich
Figure6.Paretofrontiersformodelswithhigherkvalues.
modeldynamicsarepreserved.Aqualitativeexaminationofthedatapresentedheresuggeststhataκbenchmarkintheregionof0.75–0.80isinfactagoodbenchmarkforthisexample.Onceagain,thefinaldecisionrestswiththeresearcherandisultimatelydeterminedbythelevelofdesiredaccuracy.
http://jasss.soc.surrey.ac.uk/17/2/6.html 7 16/10/2015
>
Figure7.Plotofkvaluevs.sizeoffrontier,withlineofbestfit(Pearson'sr2=0.66).
SugarScapeModel
4.20 ThesecondmodelisamodifiedversionofSugarScape(Epstein1996),inwhichapopulationofagentstraversealandscapeinsearchofsugar.Thismodelwaschosenforseveralreasons:first,itisspatiallyheterogeneous.ThisisacommonfeatureofmanyABMsandthusitisimportanttodemonstratehowtheframeworkpresentedherecanbeappliedtomodelswhereinspaceisanissue.Second,themodelhasbeenexaminedbyresearchersinavarietyoffields–studiesbasedonSugarScapehavefocusedonmigrationandculture(Deanetal.2000),distributionofwealth(Rahmanetal.2007),andtrade(Dascluetal.1998).Assuch,itisofbroadgeneralinterestasatestcase.Thus,futureworkmaybuildontheframeworkpresentedhereasameansofconductingresearchinareasasdiverseassocialscience,biology,andeconomics.
4.21 ThebasisofthemodelusedhereisincludedwiththestandardNetLogodistribution(Wilensky2009).Thelandscapeconsistsoffixedregionscontainingdifferentamountsofsugar;assuch,thismodelcontainsaspatialheterogeneitynotpresentintherabbitsandgrassmodel.TheoriginallandscapeispresentedinFigure8;darkerregionsrepresentareaswithmoresugar.Periodically,antsaretaxedbasedontheirvision,metabolism,andlocation(e.g.,high-visionantsinsugar-richregionsmaybetaxedathigherratesthanlow-visionantsinregionswithlesssugar).Theoptimizationproblemistodeterminethetaxschedulethatmaximizesthetotaltaxincomecollectedwhileminimizingthenumberofdeaths.Fullmodeldetails,includingthosepertainingtotaxation,areprovidedinAppendixB.Notethatcertainparametervaluesarealteredwhenconsideringreducedmodels;parametervaluesinAppendixBrefertothe50×50model.
Figure8.Landscapeofthe50x50SugarScapemodel.Darkerregionscontainmoresugar.Eachregionislabeledwithanumber;Table8providesmaximumsugarlevelsbyregion.
ScalingResults
4.22 Atotalof100controlsweregenerated,consistingofthreedifferentaveragetaxrates.Thenumberofdeathsandtaxincomeforeachwascollectedoveratotalof100runs.RepresentativedataispresentedinFigure9,witherrorbarsrepresentingonestandarddeviation.Asseeninthefigure,thereisverylittlechangeinthemeanandstandarddeviationofthedatabeyond50runs;hence,thereisnobenefittoaveragingovermorethan50simulations.
4.23 GiventheimportanceofthespatiallayoutoftheSugarScapemodel,itisnecessarytopreservethislayoutasnearlyaspossibleinanyreducedversion.Landscapereductionwasdeterminedbythenearest-neighboralgorithm,ameansofre-samplingtheoriginallandscapeinordertodeterminethelayoutofareducedversion.
4.24 Inadditiontoscalingthemap,thenumberofagentswasalsoscaled.Whilelowvisionisdefinedtobe1atanymodelsize,highvisiondependsonthesizeofthegrid:anagentwithvisionvona50×50gridhasvisionvn=v(n/50)onann×ngrid.Forgridsizes10and5,thiswouldresultinhighvisionbeingequivalenttolow;thusinthesetwocaseshighvisionwasdefinedtobe2.Themetabolismofeachagentisnotscaled:ateachmodelsize,itwasrandomlysetbetween1and4(inclusive).
4.25 Torunthesimulation50timesatmodelsizeM=50(meaninga50×50grid)takesapproximately8.5seconds.Table2showsthenumberofsimulationsthatcanberuninequivalentruntimeforreducedmodelsizes.
(a)Avg.taxrateof0.125
http://jasss.soc.surrey.ac.uk/17/2/6.html 8 16/10/2015
(b)Avg.taxrateof0.25
(c)Avg.taxrateof0.375Figure9.Averagevaluesfordeaths(solid)andtaxincome(dashed)againsettleintoconsistentvaluesby50runs.Taxincomevalueshavebeenscaledlinearlytofitonthe
plots.
4.26 Figure10showsκvaluesforvariousmodelreductions.Sincetherearetwovariablesinthiscase(deathsandtaxincome),controlscanberankedaccordingtoeither,resultingintwodifferentκvalues.Notethatwhenrankedaccordingtothenumberofdeaths,κvaluesareextremelylow–infact,closetobeingcompletelyrandom.Whiletherankingsaccordingtotaxincomeresultinhigherκvalues,theystillfallshortoftheproposedminimumbenchmarkof0.8.
Table2:Numberofsimulationsinequivalentruntime.
Worldsize Simulations Avg.runtime(sec.)50 50 8.5140 88 8.4830 173 8.4920 427 8.5210 1375 8.535 3900 8.48
http://jasss.soc.surrey.ac.uk/17/2/6.html 9 16/10/2015
Figure10.Cohen'sweightedkwithcontrolsrankedbydeaths(left)andbytaxincome(right).
4.27 Aninterestingfeatureofthisdataisthatthereappearstobenogreatdifferenceintheκvaluesobtainedfrommodelsrunat2%oftheoriginalruntimeversusthoseobtainedfrom100%runtime.Thismayindicatethatthenumberofrunsusedforreliabledatawasoriginallysettoohigh,oritmayindicatethatκvaluesatorbelow0.45areequallyunreliable.Nevertheless,thereisacleartrendshowingthatasthemodelsizedecreasestheκvaluesdecreaseaswell.Asinthepreviousexample,thismaybeanindicationofqualitativechangesinmodeldynamicsasthemodelsizeisreduced.
ResultsfromParetoOptimization
4.28 Althoughκvaluesappearlowerinthiscase,itisnecessarytoagainexaminetheperformanceofreducedmodelswithrespecttocontrol.Whilethesuggestedbenchmarkof0.75–0.8provedfittingforthepreviousmodel,itmaybethatalowerκbenchmarkisacceptableinthiscase.ResultsarepresentedinFigure11.
http://jasss.soc.surrey.ac.uk/17/2/6.html 10 16/10/2015
Figure11.ParetooptimizationresultsforSugarScape.4.29 Paretofrontiersfromthreemodelsarepresentedhere:thosewithκvaluesof0.43,0.27,and0.15(withrespecttotaxincome).Theshapesofthedatafromthethreemodelsfollowthesame
basicshapeofthetrueParetofrontier(labeled'Master'inthefigure),butthereisadistinctdifferenceinperformance.Inparticular,noneofthecontrolsfoundbyanyofthesemodelsareontheParetofrontieroftheoriginal.Furthermore,theredoesnotappeartobeanysignificantdifferencebetweensolutionsfoundusingthemodelwithκ=0.43andthosefoundusingthemodelwithκ=0.15.Thisindicatesthatnotonlyarenoneofthesemodelsappropriateasreplacementsfortheoriginal,butthattheremayinfactbeaminimumκvalue,belowwhichallmodelsareunsuitable.Inotherwords,thedownwardtrendindicatedinFigure10maybemisleading:whileitseemstosuggestthatasmodelsizedecreases,themodelsbecomelessrepresentativeofthedynamicsoftheoriginal,theresultsinFigure11suggestthatthisisn'tactuallythecase.Onthecontrary,modelswithaκvalueof0.15maybenoworsethanthosewithκvaluesof0.43.Giventhatnomodelsattainedaκvaluehigherthan0.45,itisimpossibletojudgethebenchmarkof0.75asappropriatelyhigh.Ontheotherhand,itispossibletoconcludethatκvaluesatorbelow0.45arecertainlytoolow.
Conclusions
5.1 Thegoalofthispaperistointroduceaframeworkforoptimizationofagent-basedmodels.Oncereliabledataisobtained,reducedmodelscanbecomparedtotheoriginal.SimilaritycanbemeasuredusingCohen'sweightedκ.Paretooptimizationwasimplementedinordertosolvecontrolproblemsinbothcases,allowingforaposteriorianalysisoftheκbenchmark.Resultspresentedhereshowthatinoneexample,theestablishedbenchmarkintherangeof0.75–0.8wasindeedsufficientformodelreduction,whilethesecondexampleshowedthatvaluesbelow0.45weretoolow.Theseresultssuggestκcanbeameaningfulmeasureformodelreduction.
5.2 Themodelspresentedherewereselectedfortheiruniversalityandpopularity–assuch,theyactasstandardmodelstowhichanyattemptatanalysisshouldbeapplied.InprinciplethereisnoreasonwhythemethodologywouldnotapplytoextensionsofthesemodelsortootherABMs.Inthefuturethiscollectionofmodelsshouldbeexpandedtoincludeawidervarietyofmodelsofincreasingcomplexity.Thismethodologyhasbeenappliedheretomodelswhereinspaceandagentlocationarekeyfeatures;itmayrequiresomemodificationinordertogeneralizetonon-spatialmodels.Inaddition,othermethodsforanalysisofagent-basedmodelsincludetransformationtoequationmodels.Suchwork(usingthesamemodelspresentedhere)isunderway.
5.3 AsABMsareusedmoreandmoretoinvestigatereal-worldsystems,optimizationandoptimalcontrolproblemswillnaturallyariseinthecontextofABMs.Heuristicmethodshaveseveraladvantages:theyareeasytoimplementonacomputerandtheycanbeappliedtovirtuallyanyABM.Thisisparticularlyimportantformodelsthataretoocomplexforconversiontoothermathematicalforms,e.g.,incaseswheredifferentialequationsareinsufficient.Theuserhasdirectcontroloverhoweachalgorithmruns,andcanfine-tuneparametersandsettingstobettersuitthemodel.However,therearedrawbackstothesemethods.Forthoseinterestedinthecertaintyoffindinggloballyoptimalsolutions,heuristicmethodsarelacking.Ontheotherhand,onemayobtainsufficientcontrolsusingthesemethods,andthatisastepintherightdirectionforcontrolofABMs–inparticularwhenone'sgoalistoobtaincontrolsthatareeithersufficientorsimplybetterthananypreviouslyknown.
5.4 Itispossiblethatthecomplexityofagent-basedmodelswillmakeformulaictranslationtorigorousmathematicalmodelsintractable–inthatcase,heuristicmethodsprovidetheonlymeansforoptimizationandoptimalcontrolofagent-basedmodels.Coupledwiththemodelreductiontechniquesandanalysisintroducedhere,thistechniqueprovidesvaluablemethodologyforsolvingcontrolproblemswithagent-basedmodels.
Acknowledgements
FundingforthisworkwasprovidedthroughU.S.ArmyResearchOfficeGrantNr.W911NF-09-1-0538,andsomeideasweredevelopedbytheOptimalControlforAgent-BasedModelsWorkingGroupatNIMBioS,UniversityofTennessee.Additionally,theauthorsaregratefultoreviewersfortheircarefulconsiderationofthemanuscript.Manyhelpfulsuggestionswereimplementedduetoreviewerfeedback.
Appendices
A:Overview,Designconcepts,andDetails(ODD)protocolforRabbitsandGrassThemodeluponwhichthisversionisbasedisincludedinthesamplelibraryofNetLogo(Wilensky2009),apopularagent-basedmodelingplatform.Thedescriptionhereiswarrantedasitincludesthemechanicsofanoptimizationproblem,thedetailsofwhicharenotavailableelsewhere.
Purpose
Thepurposeofthismodelistoexaminepopulationdynamicsofasimpleenvironmentalsystem.Inparticular,itisamodelofrabbitseatinggrassinafield.Oneachdayofthesimulation,poisoncanbeplacedonthefieldinordertokilltherabbits.Thisversionofthemodelisanattempttoanswerthefollowingoptimizationquestion:whatisthebestwayofcontrolling(i.e.,minimizing)therabbitpopulationwhilealsominimizingtheamountofpoisonused?
Entities,statevariables,andscales
Thissectioncontainsadescriptionofthegridcells,spatialandtemporalscales,andtherabbits.Italsocontainsadescriptionoftheformatofapoisonschedule,theinvestigationofwhichisthekeyfeatureofthemodel.
Gridcells,spatialscale,andtemporalscale.Theworldisasquaregridofdiscretecells,representingafield.Thegridistoroidal:edgeswraparoundbothinthehorizontalandverticaldirections.Thedistancefromthecenterofacelltoaneighboringhorizontalorverticalcellis1unit(thusthedistancebetweentwodiagonalcellsissqrt(2)).Unitsareabstractspatialmeasurements.Timestepsarealsoabstractdiscreteunits.Asimulationconsistsofafinitenumberoftimesteps.Theonlystatevariableforeachcellindicateswhetherornotthecellcurrentlycontainsgrass.Whengrassiseatenonagridcellthereisacertainprobabilitythatitwillgrowbackateachtimestep.Thisgrowthhappensspontaneously.
Table3:Gridcellstatevariables.
Statevariable Name ValueSidelengthoffield s 50gridcellsTotalgridcells N 2500Presenceofgrass grass? 0=nograss,1=grassGrassgrowthprobability γ/0.02Simulationtime total_sim_time 100timesteps
Rabbits.Eachtimestep,rabbitsmove,eatgrass(ornot),andreproduce(ornot).Reproductionisasexualandbasedonenergylevel,whichisraisedwhenarabbiteats.Rabbitsloseenergybothbymovingandbyspawningnewrabbits.Ifarabbit'senergyleveldropsto0orlowertherabbitdies.
Table4:Rabbitstatevariables.
Statevariable Name ValueMovementcost move_cost 0.5Energyfromfood food_energy 3Birththreshold birth_threshold 8Currentenergylevel energy varies
http://jasss.soc.surrey.ac.uk/17/2/6.html 11 16/10/2015
Poisonschedule.Apoisonscheduleuisavectoroflengthtotal_sim_timewitheachentryeither0or1.Eachentrycorrespondstoonetimestepinthesimulation;0meansthatpoisonisnotusedand1meansthatpoisonisused.Thus,thereareatotalof2^(total_sim_time)possiblepoisonschedules.Thepoisonhasamaximumefficacythatdegradesovertimewithrepeateduse.Ifthepoisonisnotusedtheefficacyincreasesagain,uptothemaximum.
Table5:Poisonscheduledetails.
Statevariable Name ValueMaximumefficacy p_max 0.3Degradationrate p_deg 5Currentefficacy p_eff Variesin(0,p_max]
Processoverviewandscheduling
Inordertominimizeambiguity,detailsofmodelexecutionarepresentedaspseudocode;seeAlgorithm2.
Designconcepts
IntheupdatedODDprotocoldescription(Grimmetal.2010)thereareelevendesignconcepts.Thosethatdonotapplyhavebeenomitted.
Basicprinciples.Inessence,thismodelisapredator-preysystemwhereintherabbitsarepredatorsandthegrassisprey.Introductionofpoisonintothemodel,andhavingthatpoisonmodeledasadirectexternalinfluenceonpopulationlevels,createsanaturalsettingforanoptimizationproblem.Onecanstudytheeffectofvariouspoisonstrategiesonpopulationlevels–intermsofminimizingtherabbitpopulationitcanbethoughtofasaharvestingproblem,butintermsofminimizingpoisonitcanbethoughtofasresourceallocation.
http://jasss.soc.surrey.ac.uk/17/2/6.html 12 16/10/2015
Algorithm2.PseudocodefortheRabbitsandGrassprocessandscheduling.
Emergence.Rabbitpopulationandgrasslevelstendtooscillateasthesimulationprogresses.Thefrequencyandamplitudeoftheseoscillationscanbeaffectedbyparametersettingsandinitialvaluesandhencemaybedescribedasemergentmodeldynamics.
Interaction.Agentinteractionisindirect:sincerabbitmovementisexecutedserially,itispossiblethatotherrabbitsdepleteallofthegrassinaparticularrabbit'spotentialfieldofmovement,therebyreducingoreliminatingthechanceforthatrabbittogainenergy.
Stochasticity.Rabbitmovementistotallyrandominthattheycannotsensewhetherneighboringgridcellscontaingrassornot.Whethergrassgrowsbackonanemptygridcellisalsorandom,andagridcellthathasbeenemptyforseveraltimestepsisnomorelikelytogrowgrassthanacellthathasonlyjustbecomeempty.
Observation.Rabbitpopulationandgrasscountsarerecordedateachtimestep.Thetotalnumberofrabbitsaliveduringthecourseofasimulationservesasameasureoffitnessofthepoisonschedule.
Initialization
Atinitialization,20%ofthegridcellscontaingrass;thesearechosenatrandom.Thereare150rabbitsplacedatrandomlocationsthroughoutthegrid.Eachbeginswitharandomamountofenergybetween0and9inclusive(rabbitswith0energymaysurvivethefirsttimestepbyeatinggrass).Totalsimulationtimeis100timestepsandeachsimulationcontainsapoisonscheduleu,describedinPoisonschedule.
Inputdata
Thereisnoinputdatatothemodel.
Submodels
Themodelcontainsnosubmodels.
Optimization
Sincethemulti-objectiveoptimizationproblemisthekeyfeatureofthemodelaspresentedhere,afewclarifyingdetailsareinorder.Theobjectivesoftheoptimizationproblemaretodetermine,fortheparametervaluesprovided,asetofParetooptimalpoisonschedulesthatminimizethenumberofrabbitswhilealsominimizingtheamountofpoisonused.Thenumberofrabbitsreferstothetotalnumberofrabbitsaliveduringthecourseofasimulation–notjustthosealiveattheendofthefinaltimestep.Sinceapoisonscheduleisabinaryvectoroflengthtotal_sim_time,theamountofpoisonusedisrepresentedbythesumoftheentriesofthatvector.
B:Overview,Designconcepts,andDetailsprotocolforSugarScapewithtaxationPurpose
TheversionofSugarScapepresentedhereisamodifiedversionoftheoriginalSugarScape(Epstein1996),amodelinwhichabstractentitiesroamalandscapemadeofsugar.Theseagentsareperiodicallytaxedfortheirsugarstores–thetaxrateisconstantbutthefrequencydiffersfromregiontoregion.ThepurposeofthisversionofSugarScapeistoinvestigatetheeffectsofvarioustaxationpoliciesontaxincomeandagentpopulation.Inparticular,themodelisusedtoinvestigatethefollowingquestion:whatistheoptimaltaxationpolicyformaximizingcollectedincomewhileminimizingdeaths?
Entities,statevariables,andscales
Ants.Eachanthasafixedvisionandmetabolismlevelforthedurationofthesimulation;theselevelsvaryfromanttoant.Antscanseeinthefourprincipaldirectionsup,down,left,andright,butcannotseeanyothergridcells.Metabolismdetermineshowmuchsugaranantloses('burns')eachtimestep.Movementisgovernedbyvision:anantmovestothenearestgridcellwithinitsvisionwiththemaximumamountofsugar.Onlyoneantmayoccupyagridcellatanygiventime.Antsdieiftheirsugarlevelreacheszero.Thereisnoupperlimittohowmuchsugaranantmayaccumulate.Lowvisionisdefinedas1,2,or3andlowmetabolismisdefinedas1or2.Thus,eachantbelongstooneofthefollowingfourcategories:lowvision/lowmetabolism(LL),lowvision/highmetabolism(LH),highvision/lowmetabolism(HL),andhighvision/highmetabolism(HH).
Table6:Antstatevariables.
Statevariable Name ValueLocation(currentregion) reg {0,1,…,8}Vision vis randomin{1,2,…,5}Metabolism met randomin{1,2,3,4}Sugar sug variesin{1,2,…}
Gridcells.Whenantsconsumethesugarfromagridcell,thesugargrowsbackatafixedrateoversubsequenttimesteps,uptoapre-determinedmaximumbasedonthelayoutofthelandscape.
Table7:Gridcellstatevariables.
Statevariable Name ValueMaximumsugar s_max oneof{0,1,2,3,4}Sugar s_here {0,1,2,3,4}Growbackrate α/1
Spatialandtemporalscales.ThelandscapeforSugarScapeispresentedinFigure8.ThemaximumsugaramountsforeachregionaregiveninTable8.Therearefiveregiontypes:thosewhosemaximumsugaris0,1,2,3,or4.Eachantoccupiesexactlyonegridcell,andthemapistoroidal–edgeswrapinboththehorizontalandverticaldirections.Thelandscapeisa50×50gridofcells.Giventhefairlyabstractnatureofthemodel,timeandspaceareunitless.Asimulationconsistsofafinitenumberoftimesteps.
Table8:Maximumsugarandgridcellcountsforeachregion.
Region 0 1 2 3 4 5 6 7 8Max.sugar 0 0 1 1 2 3 3 4 4
Taxesarecollectedatregularintervals.Thetaxrateforagivenantdependsonitscategoryandcurrentregion(forexample,anagentwithhighvisionandlowmetabolismmaybetaxedat
http://jasss.soc.surrey.ac.uk/17/2/6.html 13 16/10/2015
rate0.75inahigh-sugarregionbutonlyat0.25inalow-sugarregion).Taxamountsarealwaysroundeduptothenearestinteger–thisensuresthatanynon-zerotaxratealwayscollectsatleast1unitofsugar.Taxesarecollectedonceeverysubsequent5timestepsforatotaloftentaxcycles.Thechoicetotaxevery5ticksismotivatedbythedesiretonotletthedynamicsstabilize–withfrequenttaxationthedynamicsaremoreimmediatelyaffectedbyprevioustaxrates.
Table9:Taxationandtemporalvariables.
Variable Name Value UnitsSimulationduration total_sim_time 50 timestepsPermissibletaxrates tax {0,0.25,0.5,0.75} N/ATaxinterval tax_interval 5 timesteps
Foreachofthefourantcategoriestherearefivepossibletaxratesdependingontheircurrentregionandeachoftheseratesmaybedifferentforeachofthetentaxcycles.Thus,ataxscheduleisavectoroflength5·4·10=200witheachentryin{0,0.25,0.5,0.75}–thismeansthereareatotalof4^(200)differenttaxschedules.Theoptimizationproblemistodeterminethetaxschedulesthatmaximizethetotaltaxincomecollectedwhileminimizingthenumberofdeaths.
Processoverviewandscheduling
TheABMprocessispresentedinAlgorithm3aspseudocode.Theantandtaxroutinesareexecutedfullybyoneant,thenfullybyanother–i.e.,serially.Hencestatevariablesareupdatedasynchronously.Timestepsarediscreteunits,asismovement:antsjumpdirectlyfromthecenterofonegridcelltothecenterofanother.
Algorithm3.PseudocodeforSugarScapeprocessandscheduling.
Designconcepts
Basicprinciples.ThisversionofSugarScapebuildsontheoriginalbyincorporatingtaxation.Ingeneral,thebasicquestionunderinvestigationishowspatially-dependentlocalinputsaffectglobaldynamics.Specifically,themodelinvestigateshowlocaltaxratesaffecttaxincomeandregionalpopulationdistribution,aquestionwhichholdsinterestinavarietyofreal-worldsettings.
Emergence.Spatialpopulationdynamicsoughttobeanemergentpropertyofthemodel:forexample,hightaxratesinhigh-sugarregionsmightsubstantiallyalterregionalpopulationdynamics.Theprecisemechanismdrivingsuchchangesisnotbuiltintothemodelinanydirectsense.
Objectives.Theobjectiveofeachantistomovetoacellwithinitsvisionwiththemaximumamountofsugar.Thereisnootherconsideration,andantsdonothaveknowledgeofpastorfuturetaxratesatanylocation.
http://jasss.soc.surrey.ac.uk/17/2/6.html 14 16/10/2015
Sensing.Antsareawareofthesugarlevelandoccupancyofeachgridcellwithintheirvision.Theyarenotawareofanypropertiesofanyotherants,eventhosewithintheirvision.
Interaction.Antsinteractwithoneanotherindirectlyinthesensethatonlyoneantmayoccupyagridcellatanygiventime.Thusiftwoantshavethesamehigh-sugargridcellwithintheirvision,whicheverantisrandomlyselectedtomovefirstwilloccupythatcell.Thismayverywellalterthemovementoftheotherant.Inthisway,serialexecutionandasynchronousupdatearekeyfeaturesofagentinteraction.Ifantorderwasnotrandom(i.e.,thesameantwasallowedtomovefirsteachtimestep),populationdynamicsmightbefundamentallyaltered.
Stochasticity.Antmovementispartiallystochastic:ifanantseesfourunoccupiedgridcellswith2sugarandthreeunoccupiedgridcellswith3sugar,theantwillchoosethenearestcellwith3sugar.This'minimumdistance'policytendstoleadtoantsclusteringonregionalboundaries,astheyhavelimitedincentivetomovetotheinteriorofaregion.ThismovementfeatureisdiscussedinEpsteinandAxtell(1996).
Collectives.Inasense,antsformcollectivesthataffectindividualsinsideandoutsideofthecollective.Thisarisesbecauseonlyoneantmayoccupyagridcellatanytime.Inhigh-sugarregions,antsinthemiddleoftheregiontendtobecometrappedbecauseallavailablespacesareoccupied.Atthesametime,anindividualontheborderofsucharegionisfrequentlyunabletoenterduetothehighpopulationdensitywithintheregion.Thesecollectivesformentirelyasaresultoflocalinteractions.
Observation.Eachsimulationconsistsofafinitenumberoftimesteps.Ateachtimestep,thefollowinginformationiscollected:thetotalamountoftaxcollected,andthenumberofdeathsthatoccur.Foreachsimulation,thetaxpolicyisrecordedaswell.Attheendofeachsimulationthesedataarewrittentoacommaseparatedvalue(.csv)file,auniversalformatforspreadsheetapplications.
Initialization
Themodelisinitializedwith200ants;eachisplacedatarandomunoccupiedlocationonthelandscape.Antsbeginwitharandomamountofsugarbetween5and25(inclusive);thisvalueisdifferentforeachantandchosenfromauniformdistribution.Antsareinitializedwithvisionchosenatrandombetween1and5(inclusive)andmetabolismbetween1and4(inclusive).Visionandmetabolismofagivenantdonotchangeoverthecourseofasimulation.
Inputdata
Thelandscapeisreadinfroma.txtfile;thishelpswithimplementationandmakesiteasiertomakechanges.Ataxpolicycaneitherbechosendirectlyviacodemanipulationorchosenatrandom.Thepolicymustbechosenpriortosimulation.Assuch,thetaxpolicymaybethoughtofasinputtothemodel.
Submodels
TherearenosubmodelsforthisversionofSugarScape.
References
ALTMAN,D.(1991).PracticalStatisticsforMedicalResearch.London:ChapmanandHall.
BERNASCHI,M.&Castiglione,F.(2001).Designandimplementationofanimmunesystemsimulator.Comput.Biol.Med.31,303–331.[doi:10.1016/S0010-4825(01)00011-7]
BUCKERIDGE,D.L.,Jauvin,C.,Okhmatovskaia,A.&Verma,A.D.(2011).SimulationAnalysisPlatform(SnAP):aToolforEvaluationofPublicHealthSurveillanceandDiseaseControlStrategies.AMIAAnnuSympProc2011,161–170.
CASTIGLIONE,F.(1995).C-ImmSimSimulator.InstituteforComputingApplications,NationalResearchCouncil(CNR)ofItaly.<http://www.iac.cnr.it/~filippo/C-ImmSim.html>.
CASTIGLIONE,F.,Pappalardo,F.,Bernaschi,M.&Motta,S.(2007).OptimizationofHAARTwithgeneticalgorithmsandagent-basedmodelsofHIVinfection.Bioinformatics23(24),3350–3355.[doi:10.1093/bioinformatics/btm408]
CHAO,D.L.,Halloran,M.E.,Obenchain,V.J.&Longini,I.M.(2010).FluTE,apubliclyavailablestochasticinfluenzaepidemicsimulationmodel.PLoSComput.Biol.6,e1000656.[doi:10.1371/journal.pcbi.1000656]
COELLO,C.(2013).Listofreferencesonevolutionarymultiobjectiveoptimization.http://www.lania.mx/~ccoello/EMOO/EMOObib.html.Archivedat:<http://www.webcitation.org/6HhFo4K5H>.
COHEN,J.(1968).Weightedkappa:nominalscaleagreementwithprovisionforscaleddisagreementorpartialcredit. PsycholBull70(4),213–220.[doi:10.1037/h0026256]
COOLEY,P.,Brown,S.,Cajka,J.,Chasteen,B.,Ganapathi,L.,Grefenstette,J.,Hollingsworth,C.R.,Lee,B.Y.,Levine,B.,Wheaton,W.D.&Wagener,D.K.(2011).Theroleofsubwaytravelinaninfluenzaepidemic:aNewYorkCitysimulation.JUrbanHealth88,982–995.[doi:10.1007/s11524-011-9603-4]
DASCLU,M.,Franti,E.&Stefan,G.(1998).Modelingproductionwithartificialsocieties:theemergenceofsocialstructure.In:CellularAutomata:ResearchTowardsIndustry(Bandini,S.,Serra,R.&Liverani,F.,eds.).SpringerLondon,pp.218–229.
DEAN,J.S.,Gumerman,G.J.,Epstein,J.M.,Axtell,R.L.,Swedlund,A.C.,Parker,M.T.&McCarroll,S.(2000).UnderstandingAnasaziculturechangethroughagent-basedmodeling.Dynamicsinhumanandprimatesocieties.OxfordUniversityPress,Oxford,179–206.
DURRETT,R.&Levin,S.(1994).Theimportanceofbeingdiscrete(andspatial).TheoreticalPopulationBiology46(3),363–394.[doi:10.1006/tpbi.1994.1032]
ELEMAM,K.(1999).Benchmarkingkappa:interrateragreementinsoftwareprocessassessments.EmpiricalSoftwareEngineering4(2),113–133.[doi:10.1023/A:1009820201126]
EPSTEIN,J.M.&Axtell,R.(1996).Growingartificialsocieties:socialsciencefromthebottomup.Washington,DC,USA:TheBrookingsInstitution.
EUGENIO,B.D.(2000).Ontheusageofkappatoevaluateagreementoncodingtasks.In:InProceedingsoftheSecondInternationalConferenceonLanguageResourcesandEvaluation.
FERRER,J.,Prats,C.,Lopez,D.,Valls,J.&Gargallo,D.(2010).Contributionofindividual-basedmodelsinmalariaeliminationstrategydesign.MalariaJournal9(Suppl2),P9.[doi:10.1186/1475-2875-9-S2-P9]
FLEISS,J.(1981).StatisticalMethodsforRatesandProportions.Hoboken,NJ:JohnWileyAndSons.
FLEISS,J.L.(1971).Measuringnominalscaleagreementamongmanyraters.PsychologicalBulletin76(5),378–382.[doi:10.1037/h0031619]
GENNERT,M.A.&Yuille,A.(1988).Determiningtheoptimalweightsinmultipleobjectivefunctionoptimization.In:ComputerVision,SecondInternationalConferenceon.[doi:10.1109/ccv.1988.589974]
GOLDBERG,D.(1989).Geneticalgorithmsinsearch,optimization,andmachinelearning.Reading,MA:Addison-Wesley.
GRIMM,V.,Berger,U.,Bastiansen,F.,Eliassen,S.,Ginot,V.,Giske,J.,Goss-Custard,J.,Grand,T.,Heinz,S.K.,Huse,G.,Huth,A.,Jepsen,J.U.,Jrgensen,C.,Mooij,W.M.,Mller,B.,Pe'er,G.,Piou,C.,Railsback,S.F.,Robbins,A.M.,Robbins,M.M.,Rossmanith,E.,Rger,N.,Strand,E.,Souissi,S.,Stillman,R.A.,Vab,R.,Visser,U.&DeAngelis,D.L.(2006).Astandardprotocolfordescribingindividual-basedandagent-basedmodels.EcologicalModelling198(1–2),115–126.[doi:10.1016/j.ecolmodel.2006.04.023]
GRIMM,V.,Berger,U.,DeAngelis,D.L.,Polhill,J.G.,Giske,J.&Railsback,S.F.(2010).TheODDprotocol:Areviewandfirstupdate.EcologicalModelling221(23),2760–2768.[doi:10.1016/j.ecolmodel.2010.08.019]
HAPPE,K.(2005).Agent-basedmodellingandsensitivityanalysisbyexperimentaldesignandmetamodelling:Anapplicationtomodellingregionalstructuralchange.2005InternationalCongress,August23-27,2005,Copenhagen,Denmark24464,EuropeanAssociationofAgriculturalEconomists.
HARADA,Y.,Ezoe,H.,Iwasa,Y.,Matsuda,H.&Sato,K.(1995).Populationpersistenceandspatiallylimitedsocialinteraction.TheoreticalPopulationBiology48(1),65–91.[doi:10.1006/tpbi.1995.1022]
HINKELMANN,F.,Murrugarra,D.,Jarrah,A.&Laubenbacher,R.(2011).Amathematicalframeworkforagentbasedmodelsofcomplexbiologicalnetworks.BullMathBiol73(7),1583–1602.[doi:10.1007/s11538-010-9582-8]
http://jasss.soc.surrey.ac.uk/17/2/6.html 15 16/10/2015
HORN,J.,Nafpliotis,N.&Goldberg,D.(1994).Anichedparetogeneticalgorithmformultiobjectiveoptimization.In:EvolutionaryComputation,1994.IEEEWorldCongressonComputationalIntelligence.ProceedingsoftheFirstIEEEConferenceonComputationalIntelligence[doi:10.1109/icec.1994.350037]
KASAIE,P.,Kelton,W.,Vaghefi,A.&Naini,S.(2010).Towardoptimalresource-allocationforcontrolofepidemics:Anagent-based-simulationapproach.In:WinterSimulationConference(WSC),Proceedingsofthe2010.
KIM,P.S.,Lee,P.P.&Levy,D.(2008a).APDEmodelforimatinib-treatedchronicmyelogenousleukemia.Bull.Math.Biol.70,1994–2016.[doi:10.1007/s11538-008-9336-z]
KIM,P.S.,Lee,P.P.&Levy,D.(2008b).Modelingimatinib-treatedchronicmyelogenousleukemia:reducingthecomplexityofagent-basedmodels.Bull.Math.Biol.70,728–744.[doi:10.1007/s11538-007-9276-z]
KLANN,M.T.,Lapin,A.&Reuss,M.(2011).Agent-basedsimulationofreactionsinthecrowdedandstructuredintracellularenvironment:Influenceofmobilityandlocationofthereactants.BMCSystBiol5,71.[doi:10.1186/1752-0509-5-71]
KRIPPENDORFF,K.(1980).ContentAnalysis:anIntroductiontoitsMethodology.BeverlyHills,CA:SagePublications.
LANDIS,J.R.&Koch,G.G.(1977).Themeasurementofobserveragreementforcategoricaldata.Biometrics33(1),159–174.[doi:10.2307/2529310]
LI,J.,Parkes,A.J.&Burke,E.K.(2011).Evolutionarysqueakywheeloptimization:Anewanalysisframework.EvolutionaryComputation.<http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00033>.[doi:10.1162/EVCO_a_00033]
LOLLINI,P.,Castiglione,F.&Motta,S.(1998).Modellingandsimulationofcancerimmunopreventionvaccine.IntJCancer77,937–941.[doi:10.1002/(SICI)1097-0215(19980911)77:6<937::AID-IJC24>3.0.CO;2-X]
LOLLINI,P.L.,Motta,S.&Pappalardo,F.(2006).Discoveryofcancervaccinationprotocolswithageneticalgorithmdrivinganagentbasedsimulator.BMCBioinformatics7,352.[doi:10.1186/1471-2105-7-352]
LUKE,S.,Cioffi-Revilla,C.,Panait,L.,Sullivan,K.&Balan,G.(2005).MASON:Amulti-agentsimulationenvironment.Simulation:TransactionsofthesocietyforModelingandSimulationInternational82(7),517–527.[doi:10.1177/0037549705058073]
MAO,L.(2011).Agent-basedsimulationforweekend-extensionstrategiestomitigateinfluenzaoutbreaks.BMCPublicHealth11,522.[doi:10.1186/1471-2458-11-522]
MEIER-SCHELLERSHEIM,M.&Mack,G.(1999).Simmune,atoolforsimulatingandanalyzingimmunesystembehavior.CoRRcs.MA/9903017.
NORTH,M.J.,Collier,N.T.&Vos,J.R.(2006).Experiencescreatingthreeimplementationsoftherepastagentmodelingtoolkit.ACMTrans.Model.Comput.Simul.16(1),1–25.[doi:10.1145/1122012.1122013]
OREMLAND,M.(2011).OptimizationandOptimalControlofAgent-BasedModels.Master'sthesis,VirginiaPolytechnicInstituteandStateUniversity.
PAPPALARDO,F.,Pennisi,M.,Castiglione,F.&Motta,S.(2010).Vaccineprotocolsoptimization:insilicoexperiences.Biotechnol.Adv.28,82–93.[doi:10.1016/j.biotechadv.2009.10.001]
PATEL,R.,Longini,I.M.,Jr.&Halloran,M.E.(2005).Findingoptimalvaccinationstrategiesforpandemicinfluenzausinggeneticalgorithms.J.Theoret.Biol.234(2),201–212.[doi:10.1016/j.jtbi.2004.11.032]
PENNISI,M.,Catanuto,R.,Pappalardo,F.&Motta,S.(2008).Optimalvaccinationschedulesusingsimulatedannealing.Bioinformatics24,1740–1742.[doi:10.1093/bioinformatics/btn260]
RAHMAN,A.,Setayeshi,S.&Shamsaei,M.(2007).Ananalysistowealthdistributionbasedonsugarscapemodelinanartificialsociety.InternationalJournalofEngineering20(3),211–224.
RAPIN,N.,Lund,O.,Bernaschi,M.&Castiglione,F.(2010).Computationalimmunologymeetsbioinformatics:theuseofpredictiontoolsformolecularbindinginthesimulationoftheimmunesystem.PLoSONE5,e9862.[doi:10.1371/journal.pone.0009862]
ROOS,A.M.D.,Mccauley,E.&Wilson,W.G.(1991).Mobilityversusdensity-limitedpredator–preydynamicsondifferentspatialscales.Proceedings:BiologicalSciences246(1316),pp.117–122.[doi:10.1098/rspb.1991.0132]
SIM,J.&Wright,C.C.(2005).Thekappastatisticinreliabilitystudies:use,interpretation,andsamplesizerequirements.PhysTher85(3),257–268.
SWIERNIAK,A.,Kimmel,M.&Smieja,J.(2009).Mathematicalmodelingasatoolforplanninganticancertherapy.Eur.J.Pharmacol.625(1-3),108–121.[doi:10.1016/j.ejphar.2009.08.041]
VELIZ-CUBA,A.,Jarrah,A.S.&Laubenbacher,R.(2010).Polynomialalgebraofdiscretemodelsinsystemsbiology.Bioinformatics26(13),1637–1643.[doi:10.1093/bioinformatics/btq240]
WANG,T.&Zhang,X.(2009).3Dproteinstructurepredictionwithgenetictabusearchalgorithminoff-latticeABmodel.In:Proceedingsofthe2009SecondInternationalSymposiumonKnowledgeAcquisitionandModeling-Volume01,KAM'09.Washington,DC,USA:IEEEComputerSociety.[doi:10.1109/KAM.2009.2]
WANG,Z.,Bordas,V.&Deisboeck,T.(2011).IdentificationofCriticalMolecularComponentsinaMultiscaleCancerModelBasedontheIntegrationofMonteCarlo,Resampling,andANOVA.FrontiersinPhysiology2(0).[doi:10.3389/fphys.2011.00035]
WILENSKY,U.(2009).Netlogo.CenterforConnectedLearningandComputer-BasedModeling,NorthwesternUniversity,Evanston,IL.http://ccl.northwestern.edu/netlogo/.
WILSON,W.,Deroos,A.&Mccauley,E.(1993).Spatialinstabilitieswithinthediffusivelotka-volterrasystem:Individual-basedsimulationresults.TheoreticalPopulationBiology43(1),91–127.[doi:10.1006/tpbi.1993.1005]
WILSON,W.,McCauley,E.&Roos,A.(1995).Effectofdimensionalityonlotka-volterrapredator-preydynamics:Individualbasedsimulationresults.BulletinofMathematicalBiology57,507–526.[doi:10.1007/BF02460780]
WOELKE,A.L.,vonEichborn,J.,Murgueitio,M.S.,Worth,C.L.,Castiglione,F.&Preissner,R.(2011).Developmentofimmune-specificinteractionpotentialsandtheirapplicationinthemultiagent-systemVaccImm.PLoSONE6,e23257.[doi:10.1371/journal.pone.0023257]
YANG,Y.,Atkinson,P.M.&Ettema,D.(2011).AnalysisofCDCsocialcontrolmeasuresusinganagent-basedsimulationofaninfluenzaepidemicinacity.BMCInfect.Dis.11,199.[doi:10.1186/1471-2334-11-199]
YESILYURT,S.&Patera,A.T.(1995).Surrogatesfornumericalsimulations;optimizationofeddy-promoterheatexchangers.ComputerMethodsinAppliedMechanicsandEngineering121(14),231–257.[doi:10.1016/0045-7825(94)00684-F]
ZOU,Y.,Fonoberov,V.A.,Fonoberova,M.,Mezic,I.&Kevrekidis,I.G.(2012).Modelreductionforagent-basedsocialsimulation:coarse-grainingacivilviolencemodel.PhysRevEStatNonlinSoftMatterPhys85(6Pt2),066106.[doi:10.1103/PhysRevE.85.066106]
http://jasss.soc.surrey.ac.uk/17/2/6.html 16 16/10/2015
top related