23/4/18 EIE426-AICV 1

Evolutionary Computation (EC)

eie426-ec-200809.ppt

23/4/18 EIE426-AICV 2

Contents Basic Concepts of EC Genetic Algorithms An Example Chromosome Representation Stopping Criteria Initial Population Selection Mechanisms Crossover and Mutation Fitness Functions Another Example Application: Routing Optimization Advantages and Disadvantages of EC

23/4/18 EIE426-AICV 3

Evolution and Search

Evolution - search through the enormous genetic parameter space for the best genetic make-up.

Borrow ideas from nature to help us solve problems that have equally large search spaces or similarly changing environment.

23/4/18 EIE426-AICV 4

Natural Evolution and Evolutionary Computation

Natural Evolution

Individual

Fitness

Environment

Evolutionary Computing

Candidate Solution

Quality

Problem

23/4/18 EIE426-AICV 5

Different ECsSeveral classes of EC algorithms have been developed:

- Genetic algorithms (GA’s): model genetic evolution- Genetic programming: based on GA’s, but individuals are programs (represented as trees)- Evolutionary programming: from the simulation of adaptive behavior in evolution (phenotype evolution)- Evolution strategies: model the strategic parameters that control variation in evolution, i.e., the evolution of evolution- Culture evolution: models the evolution of culture of a population and how the culture influences the evolution of individuals.- Co-evolution: individuals evolve through cooperation, or in competition with one other.

23/4/18 EIE426-AICV 6

Basic Concepts

• Chromosome: individual• Population: many individuals• Gene: each characteristics of chromosome (one parameter)• Allele: the value of a gene• Crossover: generate offspring by combining parts of the

parents.• Mutation: introduce new genetic material into an existing

individual.• Fitness: the survival strength of an individual• Culling (removing) and elitism (copying)

23/4/18 EIE426-AICV 7

Evolutionary Computation

Recombination(Crossover)

MutationPopulation

Offspring

ParentsSelection

ReplacementThe evolutionary cycle

23/4/18 EIE426-AICV 8

Genetic Algorithms

The GA was the first EC paradigm developed and applied (Holland 1975).

The features of the original GA’s:

(1) A bit string representation

(2) Proportional selection

(3) Cross-over as the primary method to produce new individuals.

Several changes have been made:

(1) Different representation schemes

(2) Different selection methods

(3) Different GA operators (cross-over, mutation and elitism)

23/4/18 EIE426-AICV 9

Random Search

The GA is a search procedure. Random search is possibly the simplest search

procedure. Its training time may be very long before an acceptable solution is obtained.

Procedure:

(1) Start from an initial search point or a set of initial points.

(2) Random perturbations to the points

(3) Repeat until an acceptable solution is reached or a maximum number of iterations is exceeded.

23/4/18 EIE426-AICV 10

General Genetic Algorithm(1) Let g = 0.(2) Initialize the initial generation Cg .(3) While no stopping criterion is satisfied

(a) Evaluate the fitness of each individual in Cg .(b) g g+1.(c) Select parents from Cg-1.(d) Recombine selected parents through cross-over to

form offspring Og (with a probability pc).(e) Mutate offspring in Og (with a probability pm).(f) Select the new generation Cg from (the previous

generation Cg-1, e.g., the best individuals are copied) and the offspring Og.

g: generationNote: The things in () might or might not be carried out.

23/4/18 EIE426-AICV 11

An Example

sequence. 0) toting(approxima

number small decrasing a is ,...2,1 and ,...2,1

,...2,1,20

12

0

,...2,1,20

12

:equation above the tosolutions ofnumber infinitivean are There

1010tan

010cos1010sin

:maxima local thefind funtion to thetedifferntiacan We

2,1 0.210sin

:function following theof maximum theFind

0

ii

ii

x

x

ii

x

xx

xxxxf

xxxxf

i

ii

ii

23/4/18 EIE426-AICV 12

.85.385.1an greater thslighytly is

85.1

[-1,2].in maximum theis

19

1919

19

fxf

x

x

23/4/18 EIE426-AICV 13

Use a genetic algorithm to solve the problem:

Coding (chromosome representation of a solution)

Generation of initial population (solutions) Fitness calculation Genetic operation

23/4/18 EIE426-AICV 14

Coding (chromosome representation):Use a binary string to represent x. If the solution is to be precise to 10-6, then the interval (2-(-1)) = 3 should be divided into 3× 106. At least 22 bits should be used because

mapping genotype phonotype

process coding a ,,...,,

3041944210321520972

02021

22621

bbbx

23/4/18 EIE426-AICV 15

Decoding

21111111111111111111111

10000000000000000000000

637197.012

3967 288 20.1

967 288 21111010100011000101110'

1111010100011000101110 e.g.,12

)1(2'0.1

'2,...,,

mapping phonotypegenotype

process decoding a , ,...,,

2

2

22

2

1

22

10

21

0202021

02021

x

x

s

xx

xbbbb

xbbb

i

ii

23/4/18 EIE426-AICV 16

minmaxmin

10

1

02011

maxmin

12

'

'2,,

:belowgiven is conversion then thestring,binary bit -an by

drepresente is ,in valuesthat takesparameter a if general,In

xxx

xx

xbbbb

m

xx

m

m

i

iim

23/4/18 EIE426-AICV 17

Generation of initial population

A set of N 22-bit binary strings can be randomly generated as the initial population.

23/4/18 EIE426-AICV 18

Fitness calculation

Since f(x) > 0 in the interval, we can directly use f(x) as a fitness function:

f(s) = f(x)

e.g.,

s1 = <1000101110110101000111>, f(s1)=2.586345

s2 = <0000001110000000010000>, f(s2)=1.078878

s3 = <1110000000111111000101>, f(s3)= 3.250650

23/4/18 EIE426-AICV 19

Genetic operation(1) Selection: based on the fitness of individualse.g., roulette wheel selection (fitness proportionate selection)

(2) Crossover (with a probability pc)e.g.,

s2 = <00000 | 01110000000010000>, f(s2)=1.078878

s3 = <11100 | 00000111111000101>, f(s3)= 3.250650After the crossover operation:

s’2 = <00000 | 00000111111000101>, f(s’2)=1.940865

s’3 = <11100 | 01110000000010000 >, f(s’3)= 3.459245

23/4/18 EIE426-AICV 20

(3) Mutation (with a probability pm) e.g.,

s3 = <1110000000111111000101>

f(s3)= 3.250650After the mutation operation:

s’3 = <1110100000111111000101 >

f(s’3)= 0.917743or

s3 = <1110000000111111000101>

s”3 = <1110000001111111000101 >

f(s”3)= 3.343555

23/4/18 EIE426-AICV 21

Simulation results:

N = 50, pc = 0.25, pm = 0.01, at 89 generations, the best individual was obtained:

smax = <1101001111110011001111>

xmax = 1.850 549

f(xmax) = 3.850 274

23/4/18 EIE426-AICV 22

The best individual at each iteration (up to 150 generations)

Generation The chromosome of the best individual

x fitness

1 1000111000010110001111 1.831 624 3.534 806

11 0110101011100111001111 1.854 860 3.833 286

17 1110101011111101001111 1.847 536 3.842 004

54 1000110110100011001111 1.848 699 3.847 155

71 0100110110001011001111 1.850 897 3.850 162

89 1101001111110011001111 1.850549 3.850274

150 1101001111110011001111 1.850549 3.850274

23/4/18 EIE426-AICV 23

Summary on Basic Concepts Evolution is an optimization process, where the aim is to

improve the ability of individuals to survive. An evolutionary algorithm (EA) is a stochastic search for

an optimal solution to a given problem. Evolution - search through the enormous genetic

parameter space for the best genetic make-up. Borrow ideas from nature to help us solve problems that

have an equally large search spaces or similarly changing environment.

23/4/18 EIE426-AICV 24

Genotype: describes the genetic composition of an individual

Phenotype: the expressed behavioral traits of an individual in a specific environment.

Selection: use the fitness evaluations to decide which are the best parents to reproduce.

Crossover: generate offspring by combining parts of the parents

Mutation: introduce new genetic material into an existing individual.

Coding: phenotype genotype Decoding: genotype phenotype

23/4/18 EIE426-AICV 25

Simple Genetic Algorithm (SGA)Representation Binary strings

Recombination

(crossover)

N-point (commonly used 1-point and 2-point) or uniform;

pc typically in range (0.6, 0.9)

Mutation Bitwise bit-flipping with fixed probability pm (typically between 1/pop_size and 1/ chromosome_length)

Parent selection Fitness-Proportionate

Survivor selection All children replace parents

Speciality Emphasis on crossover

23/4/18 EIE426-AICV 26

Genotype space = {0,1}I

I-bit binary stringsPhenotype space

Coding (encoding or(representation)

Decoding(inverse representation)

011101001

010001001

10010010

10010001

Chromosome Representation

23/4/18 EIE426-AICV 27

Binary-valued variables: no extra coding required Nominal-valued variables

D-bit with 2D discrete nominal valuese.g., four colors: red (00), blue (01), green (10), yellow (11)

Continuous-valued variables

I1,0:

23/4/18 EIE426-AICV 28

Other Representations Gray coding of integers (still binary chromosomes)

Gray coding is a mapping that means that small

changes in the genotype cause small changes in the

phenotype (unlike binary coding, e.g., 0111(7) and

1000 (8)).

It is generally accepted that it is better to encode

numerical variables directly as

Integers

Floating point variables

23/4/18 EIE426-AICV 29

Stopping Criteria

The maximum number of generation is exceeded

An acceptable best fit individual has evolved The average and/or maximum fitness value do

not change significantly over the past g generations.

23/4/18 EIE426-AICV 30

Initial Population

The standard way of generating the initial population is to choose gene values randomly from the allowed set of values.

The goal of random selection is to ensure that the initial population is a uniform representation of the entire search space.

A large population covers a larger area of the search space, and may require less generations to converge.

In the case of a small population, the EA can be forced to explore a large search space by increasing the rate of mutation.

23/4/18 EIE426-AICV 31

Selection Mechanisms

Selection operators

(1) Random selection

(2) Proportional selection: roulette wheel selection

(3) Tournament selection

(4) Rank-based selection

23/4/18 EIE426-AICV 32

Random Selection

Individuals are selected randomly with no reference to fitness at all. All the individuals, good or bad, have an equal chance of being selected.

23/4/18 EIE426-AICV 33

Proportional Selection: Roulette Wheel Selection

Individual Chromosome Fitness

fi

Selection probability, Pi

Accumulated probability

1 0001100000 8 0.086 957 0.086 957

2 0101111001 5 0.054 348 0.141 304

3 0000000101 2 0.021 739 0.163 043

4 1001110100 10 0.108 696 0.271 739

5 1010101010 7 0.076 087 0.347 826

6 1110010110 12 0.130 435 0.478 261

7 1001011011 5 0.054 348 0.532 609

8 1100000001 19 0.206 522 0.739 130

9 1001110100 10 0.108 696 0.847 826

10 0001010011 14 0.152 174 1.000 000

0,/10

1

ii

iii fffP

23/4/18 EIE426-AICV 34

Roulette Wheel Selection

It can be visualized as the spinning of the wheel and testing which slide ends up at the top. Fitness values are usually normalized to [0,1].

1

2

3

4

5

6

7

8

9

10

23/4/18 EIE426-AICV 35

Assume that the following random number sequence is generated:0.070 221 0.545 9290.784 567 0.446 9300.507 893 0.291 1980.176 340 0.272 9010.371 435 0.854 641

Compared the random number sequence with the accumulated probabilities, we select the individuals: 1, 8, 9, 6, 7, 5, 8, 4, 6, 10. Individuals 2 and 3 were removed and replaced with individuals 8 and 6. The individuals with high fitness tends to survive but those with low fitness may be removed.

23/4/18 EIE426-AICV 36

The pseudocode:

.individual slected theas chrosomeReturn 5.

sum (b)

1 (a)

, sum While4.

1,0~ number, random uniform aObtain 3.

sum 2.

index chromosome thedenotes where,1 1.

n

P

nn

U

P

nn

n

n

23/4/18 EIE426-AICV 37

Tournament Selection

A group of k individuals is randomly selected. The individual with the best fitness is selected from the group.

The advantage: the worse individuals of the population will not be selected and the best individuals will not dominate in the reproduction process.

For crossover, two tournaments are held to select each of two parents. It is possible that (1) A parent can be selected to reproduce more than once; and(2) One individual can combine with itself to reproduce offspring.

23/4/18 EIE426-AICV 38

Rank-Based Selection

The rank ordering of the fitness values is used to determine the probability of selection and not the fitness values itself.

Non-deterministic linear samplingIndividuals are sorted in decreasing fitness value. (1) Let n = random(random(N))

where N is the total number of individuals and random(N) return a number between 1 and N.(2) Return n as the selected individual

23/4/18 EIE426-AICV 39

Elitism

Elitism involves the selection of a set of individuals from the current generation to survive to the next generation.

The number of individuals to survive to the next generation, without being mutated, is referred to as the generation gap.

Generation gap = k

k best individuals or

k individuals selected using any selection operator

23/4/18 EIE426-AICV 40

Crossover

Crossover

23/4/18 EIE426-AICV 41

Uniform Crossover

A mask (vector) of length I (I-bit binary string) is created at random for each pair of individuals selected for reproduction. A bit with value of 1 indicates that the corresponding allele (bit) has to be swapped.

. fromdifferent isIt

.chromosome in theposition each at y probabilitcrossover theis :Note

.r mask vecto Return the 3.

.1 then , If (b)

.1,0~ valuerandom a Calculate (a)

:,...,2,1each For 2.

.,...,2,1 allfor 0 1.

c

x

ix

i

p

p

mp

U

Ii

Iim

m

23/4/18 EIE426-AICV 42

1 0 1 0 0 1 0 0 0 1

Parent1

Parent 2

Mask

Offspring 1

Offspring 2

23/4/18 EIE426-AICV 43

One-point Crossover

.r mask vecto Return the 4.

.1let ,,...,1each For 3.

.,...,2,1 allfor 0 2.

.1,1~ lueinteger va random a Calculate 1.

m

i

i

mIi

Iim

IU

23/4/18 EIE426-AICV 44

0 0 0 0 0 0 1 1 1 1

Parent1

Parent 2

Mask

Offspring 1

Offspring 2

23/4/18 EIE426-AICV 45

Two-point Crossover

.r mask vecto Return the 4.

.1let ,,...,each For 3.

.,...,2,1 allfor 0 2.

. with ,1~, luesinteger va random twoCompute 1.

21

2121

m

i

i

mi

Iim

IU

23/4/18 EIE426-AICV 46

0 0 1 1 1 1 0 0 0 0

Parent1

Parent 2

Mask

Offspring 1

Offspring 2

23/4/18 EIE426-AICV 47

Arithmetic Crossover

Arithmetic crossover can be used in the case of continuous-valued genes.

.1,02,1with

0.1

0.1

by generated are

and offspring, twoof componentsth The

. and parents woConsider t 1.

,22,12,2

,21,11,1

21

21

Urr

rr

rr

i

ininin

ininin

nn

nn

CCO

CCO

OO

CC

23/4/18 EIE426-AICV 48

Mutation Alter each gene independently with a probability

pm (the mutation rate)

Real-valued representations, mutation occurs by adding a random value (usually sampled from a Gaussian distribution ) to allele. The variance is usually a function of the fitness of the individual. Individuals with a good fitness value will be mutated less, while a bad fitness value will lead to large mutations.

2,0 N

23/4/18 EIE426-AICV 49

Mutation (fox)

0 1 0 10 111

0 1 0 00 111

23/4/18 EIE426-AICV 50

The Evolution Mechanism

Increasing diversity by using genetic operators mutation crossover

Decreasing diversity by selection of parents things to kill

23/4/18 EIE426-AICV 51

Fitness FunctionsCommon fitness functions: Use the objective function f(x) directly

(1) For a maximization problem, Fit(f(x)) = f(x)

e.g.,

(2) For a minimization problem, Fit(f(x)) = -f(x)

e.g., solution for x2 + x = 2, to minimize f(x) = x2 + x - 2

xxxf 10sin

:function following theof maximum theFind

23/4/18 EIE426-AICV 52

Clipping

minimum. theof estimationan is

Otherwise,0

,

problemon maximizati aFor (2)

maximum. theof estimationan is

Otherwise,0

,

problemon minimizati aFor (1)

min

minmin

max

maxmax

c

cxfcxfxfFit

c

cxfxfcxfFit

23/4/18 EIE426-AICV 53

Mapping

function. objective theof range value

theof estimation veconservati a is

0 c 0, c1

1

problemon minimizati aFor (2)

0 -c 0, c1

1

problemon maximizati aFor (1)

c

xfxfc

xfFit

xfxfc

xfFit

23/4/18 EIE426-AICV 54

Linear transformation of fitness functions

avgmultavgavg

avgavg

avg

avg

avg

avg

avg

avgavgmult

avg

avgmult

mult

avgmultavgavg

fcfff

ffffff

ff

ff

ff

f

ff

ffcf

ff

fc

c

fcfff

ff

min

minmax'max

''min

min

min

min

max

max

max

'max

'

'

(3) and , (2) ,0 (1)

at verify theasy to isIt

ft2) Fig. (see and

usecan weation, transformafter the negative is fitness no that sure make To

ft1) Fig. (see and 1

2.0]. [1.0,with

(2) and (1) :conditions Two

23/4/18 EIE426-AICV 55

f

f'

fmaxfavgfmin

cmultfavg

f'avg

f'min

Fig. ft1 Fig. ft2

f

f'

fmaxf avgfmi n

cmul tf avg

f 'avg

23/4/18 EIE426-AICV 56

Genetic algorithms: case study To find the maximum of the “peak” function of two

variables x and y:

-3 ≤ x, y ≤ 3

23/4/18 EIE426-AICV 57

Chromosome representation

23/4/18 EIE426-AICV 58

Initial population

23/4/18 EIE426-AICV 59

The first generation

23/4/18 EIE426-AICV 60

Local maximum

23/4/18 EIE426-AICV 61

Global maximum

23/4/18 EIE557-CI&IA 62

• Robotics• Control• Design• Scheduling/Routing/Resource Allocation• Machine Learning• Pattern Recognition• Market forecasting• Data Mining• Game Playing

- Robocode- Backgammon- Chess

Evolutionary Computation: Applications

Robocode

23/4/18 EIE426-AICV 63

An Open Source educational game started by Mathew Nelson (originally provided by IBM). Currently contributions are being made by various people; officially Flemming N. Larsen is working on Robocode to keep it current and fix the bugs. The game is designed to help people learn to program in Java and enjoy the experience.

Genetic Programming (GP) Robocode

23/4/18 EIE557-CI&IA 64

Application: Routing Optimization The problem: Given a

network of M switches, an origin and a destination switch, the objective is to find the best route to connect a call between the origin and destination switches (Sevenster and Engelbrecht 1996). PSTN: Public Switch Telephone Network

23/4/18 EIE557-CI&IA 65

Chromosome representation: - variable length- each gene representing one switch- integer values for switch numbers- the first gene and last gene representing the origin and last switches, respectivelyExamples: (1 3 6 10)

(1 5 2 5 10) = (1 5 2 10)

Duplicate switches are ignored

23/4/18 EIE557-CI&IA 66

Initialization of population:

- randomly generated with the restriction that the first gene represents the origin switch and the last gene the destination switch

23/4/18 EIE557-CI&IA 67

Fitness function: a multi-criteria objective function was applied.

criteria.each of influence thecontrol and , , , costants The

cost route theminimize to:

links theofn utilizatio themaximize to:

congestion route theminimize to:

length route theminimize to:

switches ofnumber total the:

route in the switches ofnumber total the:,

dcba

F

F

F

F

M

rM

rF

dFcFbFaFF

Costj

Utilj

Blockj

Switchj

j

jSwitchj

Costj

Utilj

Blockj

Switchjj

23/4/18 EIE557-CI&IA 68

Selection: any selection operator Crossover: any crossover operator Mutation: replacing selected genes with a uniformly

random selected switch in the range [1, M].

23/4/18 EIE557-CI&IA 69

Real World EC

Tends include: More complex representations and operators Use of problem specific knowledge for seeding

the initial population and creating heuristic operators

Hybridisation with other methods

23/4/18 EIE557-CI&IA 70

Advantages of EC

Handles huge search spaces Balances exploration and exploitation Easy to try - not knowledge intensive Easy to combine with other methods Provides many alternative solutions Can continually evolve solutions to fit with a

continually changing problem

23/4/18 EIE557-CI&IA 71

Disadvantages of EC

No guarantee for optimal solution within finite time

Weak theoretical basis May need extensive parameter tuning Often computationally expensive, i.e., slow