Genetic AlgorithmsGenetic AlgorithmsLearning Machines for knowledge discovery

Finding Patterns in DataFinding Patterns in Data Data mining is the task of

digging through this data looking for patterns, associations or predictions and which transform that raw material into useful information.

Evolutionary algorithms evolve the patterns which fit the data using Darwinian principles to weed out the patterns which don't work in favor of those that do. Survival of the fittest ensures that over time it is the patterns which best fit the raw data that are delivered as solutions.

Concept HierarchyConcept Hierarchy All Knowledge Computer Science Artificial Intelligence Evolutionary Computation Evolutionary Algorithms

Genetic Algorithms Genetic Programming

Human Knowledge

Computer Science

Graphics DatabasesArtificial Intelligence Networking

Natural Language Evolutionary Computation

Math Logic Language Physics Biology

Evolutionary Algorithms

Genetic Algorithms Genetic Programming

Swarm Intelligence

Expert Systems

TerminologyTerminology Algorithm

A finite set of rules (a procedure) that solves a problem

Evolution A series of changes in a population over time affected

by biological, chemical, environmental, and technical factors

Evolutionary Algorithm An algorithm that uses selection, crossover and

mutation to produce better and better results

Genetic AlgorithmsGenetic Algorithms The Genetic Algorithm is a model of machine learning

Based on the theory of evolution (Darwin) Accomplished by creating a population of individuals represented by “chromosomes” within a computer

Chromosomes can be just character strings that are analogous to the base-4 chromosomes that we see in our own DNA

The individuals in the population then go through a process of evolution (sexual reproduction followed by survival pressure on offspring)

Evolutionary ForcesEvolutionary Forces Selection

A survival process Crossover

A sexual process Mutation

A random process

SelectionSelection

Fitness to perform Mechanisms for Selection

Survival Quantitative function Human intervention

CrossoverCrossover

MutationMutation

BiomorphsBiomorphs

Visualizing and controlling mutations

http://www.phy.syr.edu/courses/mirror/biomorph/

Genetic ProgrammingGenetic Programming

Genetic programming is the application of genetic algorithms to computer programs themselves

Proposed byJohn Koza (Stanford)

Genetic Programming ProcessGenetic Programming Process Start with a collection of functions randomly combine them into programs run the programs and see which gives the best

results keep the best programs (natural selection) mutate some of the others test the new generation repeat this process until a clear best program

emerges

Genetic Programming ExampleGenetic Programming Example

Data (-1,1,3,5,7,9,11,13,15,17) Input function elements

x (can equal any digit 0..9) +,-,*,/

Starting functions (x,x+0,x*2,1+3,4/2)

Function TreeFunction Tree

x *

x3

*

x2 *

x2

+

1 *

x2

-

1

0,1,2,3,4,5,6,7,8,9

0,3,6,8,12,15,18,21,24,27

0,2,4,6,8,10,12,14,16,18

0,3,5,7,9,11,13,15,17,19

-1,1,3,5,7,9,11,13,15,17

Many more functions

Functional ValuesFunctional ValuesTruth x 3*x 2*x 2*x+1 2*x-1

-1 0 0 0 1 -11 1 3 2 3 13 2 6 4 5 35 3 9 6 7 57 4 12 8 9 79 5 15 10 11 9

11 6 18 12 13 1113 7 21 14 15 1315 8 24 16 17 1517 9 27 18 19 17

FitnessFitnessFitness Fitness Fitness Fitness Fitness

1 1 1 2 00 2 1 2 0-1 3 1 2 0-2 4 1 2 0-3 5 1 2 0-4 6 1 2 0-5 7 1 2 0-6 8 1 2 0-7 9 1 2 0-8 10 1 2 0

SUM -35 55 10 20 0

Genetic Algorithms are FlexibleGenetic Algorithms are Flexible

Can solve hard problems quickly and reliably.

Can be easily adapted to data (simulations, models)

Can be extended (scalable) Can be hybridized

GA SoftwareGA Software Evolver (for Excel)

http://www.jurikres.com/catalog/ms_evol.htmPalisade Corp