Geometric Unification of Evolutionary

Algorithms

Alberto Moraglioamoragn@essex.ac.uk

EvoPhD 2006

By Unification I Mean…

EAs: Algorithmically irrelevant differences: name/authorship/solution interpretation/domain of application

EAs: Algorithmically relevant differences:solution representation/genetic operators

Unification: A formal framework that applies to all representations

Contents

I – Geometric Interpretation of Crossover

II – Unification of Major Representations

III – Crossover Principled Design

IV – Unity of Evolutionary Search

I. Geometric Interpretation of

Crossover

What is crossover?

CrossoverIs there any

commonaspect ?

Is it possible to give arepresentation-

independent definitionof crossover and

mutation?

100000011101000

100111100011100

100110011101000

100001100011100

Geometric Crossover• Representation-independent generalization of

traditional crossover

• Informally: all offspring are between parents

• Search space: all offspring are on shortest paths connecting parents

Geometric Crossover & Distance• Search Space is a Metric Space: d(A,B) =length

of shortest paths between A and B

• Metric space: all offspring C are in the segment between parents

• C in [A,B]d d(A,C)+d(C,B)=d(A,B)

Example1: Traditional Crossover

• Traditional Crossover is Geometric Crossover under Hamming Distance

Parent1: 011|101

Parent2: 010|111

Child: 011|111

HD(P1,C)+HD(C,P2)=HD(P1,P2)

1 + 1 = 2

Example2: Blending Crossover

• Blending Crossover for real vectors is geometric under Euclidean Distance

P1

P2

C

ED(P1,C)+ED(C,P2)=ED(P1,P2)

Geometric definitions with probability distributions

Uniform geometric crossover:

Uniform geometric ε-mutation:

|],[|

]),[(}2,1|Pr{),|(

yx

yxzyPxPzUXyxzfUX

],[}0),|(|{)],(Im[ yxyxzfSzyxUX UX

|),(|

)),((}|Pr{)|(

xB

xBzxPzUMxzfUM

),(}0)|(|{)](Im[ xBxzfSzxUM M

Representation independentand formal definition of

crossover and mutation in the search space seen as a

geometric space

II. Unification of Major Representations &

Operators

Minkowski spaces – real vectors

22

2

B((2, 2); 1)Euclidean space

2

B((2, 2); 1)Manhattan space

Balls

2

2

B((2, 2); 1)Chessboard space

1

2

1

2

1 3

[(1, 1); (3, 2)]1 geodesic

Euclidean space

1 3

[(1, 1); (3, 2)] = [(1, 2); (3, 1)]infinitely many geodesics

Manhattan space

Line segments

1

2

1 3

[(1, 1); (3, 2)]infinitely many geodesics

Chessboard space

Representation: real vectors

Neighbourhoods: continuous (3 types)

Distances: Minkowski distances

Implementation: algebraic manipulation of real vector (equation of line passing through two points)

Pre-existing recombination operators:- both blend crossovers and discrete crossovers fit geometric definition- extended blend crossovers do not fit

Hamming spaces – binary strings

00 01 02

10 11 12

20 21 22

00 01 02

10 11 12

20 21 22

B(00;1)Hamming space H(2,3)

[00;11]=[01;10]2 geodesics

Hamming space H(2,3)

000 001

010 011

100 101

111110

B(000; 1)Hamming space H(3,2)

000 001

010 011

100 101

111

110

[000; 011] = [001; 010]2 geodesics

Hamming space H(3,2)

Representation: binary/multary strings

Neighbourhoods: bit-flip/site substitution

Distances: Hamming distances

Implementation: symbolic manipulation of multary strings (mask-based crossovers)

Pre-existing recombination operators:- all binary crossovers fit the geometric definition

Cayley spaces - permutationsRepresentation: permutations

Neighbourhoods: adj. swap, swap, reversal, insertion

Distances: corresponding distances

Implementation: “minimal permutation sorting by X move” algorithms:- adj. swap = bubble sort- swap = selection sort - insertion = insertion sort - reversal = approximated MPS by reversals (NP-Hard))

Pre-existing recombination operators:various pre-existing crossover operators are sorting algorithm in disguise (because sorting permutations is easier than sorting vectors of other items)

abc

bac acb

bca cab

cba

B(abc; 1)Adjacent swap space

abc

bac acb

bca cab

cba

[abc; bca]1 geodesic

Adjacent swap space

B(abc; 1)Swap space & Reversal

space

abc

bac acb

bca cab

cba

abc

bac acb

bca cab

cba

[abc; bca]3 geodesics

Swap space & Reversal space

B(abc; 1)Insertion space

[abc; bca]1 geodesic

Insertion space

abc

bac acb

bca cab

cba

abc

bac acb

bca cab

cba

Syntactic tree spaces

Representation: syntactic tree (lisp expression)

Neighbourhood: weighted sub-tree neighbourhood

Distance: structural distance

Implementation: - sub-tree swap crossover - common region mask based crossover

Pre-existing recombination operators:- traditional crossover (non-geometric)- homologous crossover - the geometric framework can help to clarify what is the landscape and distance related to homologous crossover and a distance connected with a geometric crossover which traditional crossover is an approximation

+

sin +

x x x

*

* *

y x*

yy

Parent 1 Parent 2

y

+

sin

x

*

*

yy

x

AlignmentCrossover Point

Swap

*

*

yy

+

x x

Offspring 1Offspring 2

Significance of Unification

• Most of the pre-existing crossover operators for major representations fit geometric definition

• Established pre-existing operators have emerged from experimental work done by generations of practitioners over decades

• Geometric crossover compresses in a simple formula an empirical phenomenon

IV. Crossover Principled Design

Crossover Principled Design

• Domain specific solution representation is effective

• Problem: for non-standard representations it is not clear how crossover should look like

• But: given a combinatorial problem you may know already a good neighbourhood structure

• Geometric Interpretation of Crossover Give me your neighbourhood definition and I give you a crossover definition

+ = ?

Crossover Design Example

Non-labelled graph neighbourhood

MOVE: Insert/remove an edge

Fixed number of nodes

0

1

2

1

2

3

+

Offspring

V. Is Biological Recombination

Geometric?

Yes, come to my other presentation at EuroGP!

VI. Unity of Evolutionary Search

Example of evolutionary search

Abstract convex evolutionary search

Main result: an evolutionary algorithm using geometric crossover with any probability distribution, any kind of representation, any problem, any selection and replacement mechanism, does the same search: convex search

Proof based on abstract convexity (axiomatic geodesic convexity) and axiomatization of search process (abstract search process)

…Nearly Over!

Summary

Unification (meaning): formally dealing with all representations at once

Geometric Definition: unif. is possible by defining operators geometrically

Unification: many interesting recombinations are geometric

Crossover design: by specification of geometric definition to a new representation

General theory: using formal definition only, all EAs do the same search: convex

Thanks to the Reviewers

• Franz: thanks for all your suggestions, I’d be glad to talk with you over a coffee…

• Mario?: thanks for the enthusiastic support

• A fan?: thanks for warning me that I may be victim of a geometric hallucination…

Questions?

Geometric Crossover & Path-relinking• Meta-heuristic Path-relinking: searches on path between solutions in the

neighbourhood structure (not necessarily on a shortest path)• Geometric crossover can be understood as a formalized generalization (to

metric spaces) of PR that elicits the dual relationship between distance and solution representation and gives a formal recipe to design new crossover operators

• Formalized: it allows theory• Generalization: metric spaces are more general than graphs• Elicits duality: syntactic recombination is equivalent to neighbourhood

search• Crossover design: tells how to build crossovers rather than how to search the

search space• Formal recipe: it defines exactly what crossover is for any representation