Combining several things (new and improved)

Pablo García Sánchez - University of Granada(but now in Vrije Universiteit Amsterdam)

pgarcia@atc.ugr.es

Outline

• What I am investigating in Granada

• EvAg and MultiKulti

• Evolutionary Robotics

• μ+1 online

• μ+1 online Vs. EvAg (Robert-Jan)

• Global Parent Selector or HybridEvag (Berend-Luis)

• μ+EvAg

• EDEA (?)

• Some ideas about MultiKulti + EvAg

My Work

• SOAP

• BPEL

• WSDL

• JBI

• OSGi

• XPATH

• XSLT

• OMG!

• WTF?

My Work

• SOAP

• BPEL

• WSDL

• JBI

• OSGi

• XPATH

• XSLT

• OMG!

• WTF?

But in my spare time...- XSLT Evolution- Service Oriented Evolutionary Algorithms

EvAg

EvAg Pseudo-code

Sactual←Initialize Agent

loop

Sols←LocalSelection(NeighborsEvAg)

Snew←Recombination(Sols,Pc)

Snew’←Mutation(Snew,Pm)

Evaluate(Snew’)

if Snew better than Sactual then

Sactual←Snew’

end if

end loop

Newscast

• Is the population structure used by Juanlu

• It is the cache of Neighbours. Each entry has:

• Reference to the node

• Time-stamp of the entry creation

• Agent identifier

• Specific application data (?)

• Every cycle EvAgi initiates cache exchange

• Selects randomly a neighbour EvAgj of its cache

• Aggregation of caches. JuanLu uses:

• the c freshest items. i and j share the same cache

Newscast pseudocode

• Active mode:

loop

EvAgj←Uniformily random selected node from Cachei

Send Cachei to EvAgj

Receive Cachej from EvAgj

Cachei←Aggregate(Cachei,Cachej)

endloop

• Pasive mode

loop

Wait Cachej from EvAgj

Receive Cachei to EvAgj

Cachei←Aggregate(Cachei,Cachej)

endloop

EvAg (Results)

• Island model: results are sensitive to the number of islands→vulnerability in dynamic environments (like P2P)

• Outperforms tuned GAs

• Less links (no panmictic)

• C newest items of the both caches are kept (TTL). c<<N

MultiKulti

MultiKulti (results)

• MK policies outperform the classic policies (send the best or the random, that never reach the optimum)

• Specially with low number of individuals but larger number of islands

• It is (a bit) better to send the consensus as a representation

• It is better to send the most different of an elite as response

• Problem: difficult to caracterize a population using a string

Evolutionary Robotics

• Taxonomy:

• When? On-Line vs. off-line

• Where? On-board vs. Off-board

• How? Encapsulated vs. distributed (and hybrid!)

μ+1 online

• Local population

• No migration

• Mantains the μ fittest ones

• Sometimes re-evaluates

EvAgRobot (Robert-Jan - EA2011 October)

• Added genotype exchanges using newscast. Difference: last active solutions in caches, not only connections

Comparison μ+1 vs. EvAg

• Sensibility to parameter tuning: μ+1 is reliable. EvAg is reliable when the number of robots increases

• EvAg Panmictic vs EvAg newscast: the newscast performance is worst, but in a panmictic case a large number of peers could be difficult to manage in real-case scenarios

• Doubt: in newscast is it the topology fixed? Only active solutions are exchanged among neighbours or also the connections? (so a peer can exchange its new cache to new neighbours)

HybridEvag (Berend-Luis Approach)

Source code: PopulationStructure.cppPopulationMember pop; //HivemindInstance or NewscastInstance

Controller[] activeSolutions //although localPopulation is a better name

Controller getNextCandidate(){

currentTime++

pop->update(currentTime)

if(reevaluate) return c else...

parentA and B = BinaryTournament(pop)

crossover, mutation...

return new candidate

}

SetFitness(Controller candidate, double fitness){

if (reevaluated) candidate->fitness = (fitness + oldfitness)/2

else

replace the worst of activeSolutions if candidate is better

if (accepted || reevaluated)

pop->setNews(acceptedCandidate, pos, currentTime)

}

HiveMind.cpp

static vector <vector <newsitems>> allItems;

update() // not implemented

setNews(Controller c, pos, currenttime){

stores the c in the static (shared) vector (of accepted solutions)

}

NewscastInstance.cpp

update(int currentTime){ delete older than “currentTime - itemTTL” from myCache neighbour = select random peer of myCache otherCache = neighbour->exchange(myCache, currentTime) //caches CLONING the objects (no shared data)}

setNews(Candidate c, int currentTime){ stores the last accepted candidate (not the best!) in the cache with my timestamp}

BEFORE SEND: 1) Remove older items of my cache 2) Remove duplicates

AFTER RECEIVE: 1) Mantain the cacheSize freshest solution of each node. DOUBT: this timestamp is different!

μ+EvAg

• Combining previous approaches in a parameter-based algorithm with two parameters:

• μ

• rangeRate (parent selection global or local) (Note: no information about newscast, or TTL, is given in the Robert’s thesis)

• Change from one to another easily

• Allows a whole parameter tuning

EDEA (Georgios)

• One solution per robot

• Two kinds of mating for each robot:

• Eagerly: sends genome to the partner without asking

• Selectively: Binary Tournament from other peers and accepted if fitness is quite good enough: if(otherF/myF*Sc <prob())

• In GECCO‘11→panmictic→Random from all robots (experiments with an small set)

Differences

Algorithm Population Migration Ellitism Type

μ+1 Local population None Replace the worst Centralized

EvAg (Juanlu)

1 SolutionSelection from actual

neighborsReplace if better Distributed

EvAg (Robert)

1 Active Solution + Cache

Cache swappingReplace the active solution if better

Distributed??

HybridEvAg Local populationGlobal (panmictic)parent selection

Replace the worst (local survivor selection)

Hybrid

μ+Evag Previous algorithms combined using two parameters: μ (population) and rangeRate (parent selection global or local)

Previous algorithms combined using two parameters: μ (population) and rangeRate (parent selection global or local)

Previous algorithms combined using two parameters: μ (population) and rangeRate (parent selection global or local)

Hybrid

EDEA One solutionSends eagerly, receives selectively (panmictic in

GECCO’11)

New solution can be worst

Distributed

Differences

Few Local genomes Many

Never mU+1

Genome exchange

Always

Few Local genomes Many

Never

Genome exchange

Always EvAg

Few Local genomes Many

Never

Genome exchange

Always Hybrid Evag

Few Local genomes Many

Never

mu+EvAgmu+EvAgmu+EvAgGenome exchange mu+EvAgmu+EvAgmu+EvAg

Always

mu+EvAgmu+EvAgmu+EvAg

Differences (Population)

• μ+1: Local population of μ individuals

• Juanlu EvAg: 1 Best Solution + Selection from the neighbours. Topology: newscast

• Robert EvAg: 1 Active Solution + cache of other (less diverssity)→Finally, behaves like a whole population

• HybridEvag: local population becomes global population (panmictic) for parent selector (more diversity: not only active solutions are used) and local population for replacement

• All algorithms are special cases of μ+EvAg (Robert thesis)→(few or many local genomes AND never or always genome exchange from the global pool)

Differences (Parent Selection)

• μ+1: from the local population

• JuanLu EvAg: selected from current neighbours (open)

• Robert EvAg: binary tournament of active solutions (obtained from the cache)

• Hybrid EvAg Approach: binary tournament of ALL populations (not the active ones)

Differences (Ellitism)

• mu+1: Replace the worst in the local population

• EvAg (Robert and Juanlu): Replace the actual solution if the new is better, otherwise is dropped

• HybridEvag: replaces the worst individual if better

Ideas (I)

• Distributed Islands with MultiKulti but without EvAg (using local populations)

• Which structure is better? (ring, P2P, grid, random...)

• Which migration policy is better? (best, elite...)

• Compare:

• All islands with the tuned parameter settings (homogeneous)

• All island with random parameter settings (heterogeneous)

• What about robot failures?

Ideas (II) Combining EvAg and Multikulti?

• Combine MultiKulti with EvAg (could be difficult)

• EvAg: keeps one individual + exchange of several

• MultiKulti: keeps several individuals + exchange of one

• So, using EvAg...

• Binary Tournaments from the caches but giving priority to the most different instead the most fitted?

• Sending the most different individuals in cache exchange/aggregating ordered by differences, instead timestamp

• Problems:

• Lost of information (is not the EvAg philosophy)

• Caches not too large