genetic algorithms for optimization
TRANSCRIPT
GENETIC ALGORITHM FOR OPTIMIZATION
FETHİ CANDAN ANIL ERDİNÇ TÜFEKÇİ
İSMAİL HANCI
NUMERICAL METHODS IN OPTIMIZATION
OUTLINE
▸ What are the Genetic Algorithms (GA)
▸ Characteristics of GA
▸ Darwin’s Principle of Natural Selection
▸ Working of GA
▸ Components of GA
▸ Uniqueness of GA
▸ Procedure of GA
▸ Example
WHAT ARE THE GENETIC ALGORITHMS
Genetic Algorithms are search and optimization techniques based on Darwin’s Principle of Natural Selection.
NUMERICAL METHODS IN OPTIMIZATION
HISTORY OF GA
▸ 1950 : Alan Turing proposed a “Learning Machine”
▸ 1957 : Alex Fraser published simulation of artificial selection of organisms
▸ 1960 : Hans-Joachim Bremermann published a series of papers in the 1960s that also adopted a population of solution to optimization problems, undergoing recombination, mutation, and selection. Bremermann's research also included the elements of modern genetic algorithms.
▸ 1975 : J.H.Holland, Adaptive in Natural and Artificial Systems.
NUMERICAL METHODS IN OPTIMIZATION
CHARACTERISTICS OF GA
▸ Stochastic in nature and less likely to get caught in local minima, so mostly used for global optimization problems
▸ Applies to both continuous and discrete optimization problems
▸ Parallel-Search procedure that can be implemented on parallel processing machines for speeding operations
▸ Heuristic method based on ‘Survival of the fittest’
▸ Useful when search space very large or too complex for analytic treatment
NUMERICAL METHODS IN OPTIMIZATION
DARWIN’S PRINCIPLE OF NATURAL SELECTION
The basic principles formulated by Darwin:
1.The strongest survive and tips die (Natural Selection)
2.The new individual is obtained by crossing and there is mutation
NUMERICAL METHODS IN OPTIMIZATION
WORKING GA
▸ GA encodes each point in a parameter space into a binary bit called chromosome
▸ Each point is associated with a fitness function
▸ Gene pool is a population of all such points
▸ In each generation GA constructs a new population using genetic operators
Crossover
Mutation
NUMERICAL METHODS IN OPTIMIZATION
COMPONENTS OF GA
Encoding Schemes
Crossover Operators
Mutation Operators
NUMERICAL METHODS IN OPTIMIZATION
STOCHASTIC OPERATORS
▸ Selection : Replicates the most successful solutions found in a population at a rate proportional to their relative quality.
▸ Recombination : Decomposes two distinct solution and then randomly mixes their parts to form new solutions.
▸ Mutation : Randomly pertubs a candidate solution.
NUMERICAL METHODS IN OPTIMIZATION
UNIQUENESS OF GA
▸ Works with a coding of the parameter set, not the parameter themselves
▸ Search for a population of point and not single point
▸ Use objective function information and not derivatives or other auxiliary knowledge
NUMERICAL METHODS IN OPTIMIZATION
PROCEDURE OF GA
FlowChart
SELECT THE BEST, DISCARD THE RESTSTART
GENERATE INITIAL POPULATION
CALCULATE THE COST FUNCTION
ALGORITHM END?
SELECTION
RECOMBINATION
MUTATION
NEXT GENERATION RESULT
END
N
N-n
n
N-n
N
YES
NUMERICAL METHODS IN OPTIMIZATION
EXAMPLE
• A simple GA application
• Problem : Reach a target value with simple mathematical calculations.
• We have a target value such as 5, 37, 72.8, 231.38, etc.
• We have numbers { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }.
• We have operators { +, -, *, / }
• Totally we have 14 (10 + 4) different elements.
• We call each of one as GENE.
• A combination of GENEs, becomes a CHROMOSOME.
NUMERICAL METHODS IN OPTIMIZATION
▸ We have to encode all of these different genes
▸ As a general attitude, we use binary bits
▸ Binary bits string
▸ The number of different elements that can be encoded:
▸ We have to chose bit number n so as to have enough space for encoding all our different
NUMERICAL METHODS IN OPTIMIZATION
We can encode our elements as such:
NUMERICAL METHODS IN OPTIMIZATION
▸ Problem (Mathematical Calculations) can be formulated differently
▸ Prob1 : use each number ones
▸ Prob2 : use each operator ones
▸ Prob3 : use each number ones, but there is no restriction about operators
▸ Prob4 : use operator ones, but there is no restriction about numbers
▸ There may be Restrictions on number of elements or Gene. It defines the length of chromosomes.
▸ Chromosome Length = Gene Length x Number of Genes
NUMERICAL METHODS IN OPTIMIZATION
TargetGenes
Chromosome
NUMERICAL METHODS IN OPTIMIZATION
‣We need to decide some parameters beforehand.
‣ Later according to performance, we have to change these parameters to improve the performance considering our restrictions such as
‣ Convergence ratio
‣ Computation number
‣ Computation types
‣ Storage, RAM, time, error tolerance (fitness score and target)…
NUMERICAL METHODS IN OPTIMIZATION
GA DESIGN PARAMETERS
‣Number of POPULATION
‣ Length of CHROMOSOME
‣Value and type of CROSSOVER
‣Value of MUTATION RATE
‣Number of GENERATIONS (stop criteria)
‣ Type of SELECTION
NUMERICAL METHODS IN OPTIMIZATION
‣Population (parents, fathers and mothers)
‣Population number can be 500, 20.000, or 1.000.000
‣We create randomly
NUMERICAL METHODS IN OPTIMIZATION
‣We test each population member for a possible solution to the problem.
‣We decode each gene with the order.
‣Calculate the result mathematically ( 8 + 5 – 3 / 2 *4 / 7 = 2.857)
‣We need to define a Fitness Score for each member of population in every generation.
‣ It is normalized to [0 , 1].
NUMERICAL METHODS IN OPTIMIZATION
▸ Fitness Score Calculation or Fitness Function
▸ There can be different calculations
▸ One of them is
|Target Value - Result|1_____________________
NUMERICAL METHODS IN OPTIMIZATION
‣After assigning a Fitness Score to each population, we have to eliminate weak ones as nature does due to Darwin’s Natural Selection Principle
‣We need a Selection Pattern
‣One method is Roulette Wheel
NUMERICAL METHODS IN OPTIMIZATION
‣ We define a crossover pattern.
‣ In nature, we have dominant and recessive genes.
‣ We model with random numbers.
‣ The aim of crossover is to sustain different members, chormosomes or solution variables.
‣ If the same population we cannot find the solution.
‣ We have to mix population by Crossover
NUMERICAL METHODS IN OPTIMIZATION
‣We define a mutatation pattern
‣As it is also rare in nature, we have low mutation ratio
‣ The aim of mutation is sustain different members as well
NUMERICAL METHODS IN OPTIMIZATION
THANKS FOR LISTENING… ANY QUESTIONS ?
NUMERICAL METHODS IN OPTIMIZATION