genetic ai: technical report

8/13/2019 Genetic AI: Technical Report

1/18

G E NE T I C A R T I F I C I A L I NT E L L I G E NC E

TECHNICAL REPORT

Vivek Deshpande GPA1


2/18


3/18

Vivek Deshpande GPA1 2013-2014

Page 3

INTRODUCTION

In this technical report, I will be explaining the basic concepts of genetics and genetic algorithms. I will

be noting down the current implementations of genetic algorithms and the limitations they face. I will be takinga look at some games that implement evolutionary algorithms1and the problems seen in the AI I will then be re-

modeling the algorithm to over-come these limitations.

I will break down my implementation of this model of genetic algorithms and I will be noting down the

problems I faced while developing the prototype and how I overcame them. Finally, I will explain how my

technical research will benefit me and how it can be used to further improve a game.

SUMMARY

My research is on genetic algorithms and specifically, their implementation in artificial intelligence used

in game development. Genetic algorithms are generally used to mimic natural selection and evolution in many

fields so as to acquire useful solutions to search and optimizing problems.

My interest in genetic algorithms is because of the current AI in games today. All of them are built upon

rigid logic with the AI having no adaptive properties. I agree that rigid logic is sometimes useful for games where

a player can only do some actions and not have the freedom to do everything that is possible in real life.

However, I feel that we have come to a point in game development where immersion is seen by manydevelopers as a USP; but instead of realistic immersion

2, we only have games that give secondary immersive

properties like lighting, graphics, SFX etc.

My goal is to be part of a team that introduces actual immersion inside a game. But since the game in

itself will be as realistic as possible, we also need the artificial intelligence to be realistic too; which means that

bots will change their characteristics and are adaptive towards the players actions. The implementation of

genetic algorithms is my first step towards understanding evolutionary algorithms as a whole and their benefits

to a game developer in designing the perfect artificial intelligence.

1Evolutionary algorithm is a parent category which includes all algorithms that are based on natural occurrences like

genetic algorithms, neural networks etc.2Realistic immersion for me is being able to do any action possible by a human being if he were in the same situation.


4/18


Page 4

UNDERSTANDING GENETICS

Understanding the basic concepts of genetics and evolution theory is extremely important to

understand genetic algorithms. I will be explaining them in brief below.

Genetics is a field of study in biology which explains the concepts of genomes and inheritance through

natural selection. It is the process of inheritance of personal traits from one generation to next by

recombination of DNA.

Each cell in a living organism consists of a number of proteins, carbohydrates and a specific type of protein

called chromosomes. A chromosome is an organized structure of DNA and RNA proteins.

According to Wikipedia3, DNA (Deoxyribonucleic acid) is a molecule that encodes

the genetic instructions used in the development and functioning of all known living organisms and many

viruses. DNA is a nucleic acid; alongside proteins and carbohydrates, nucleic acids compose the three

major macromolecules essential for all known forms of life. A gene is a sequence of DNA that is required ofeach cell is a body to perform a certain function. Hence, each gene decides the size, shape and function of each

cell.

So, in laymans terms, every living beings traits on the planet are defined by the combination of these

DNA molecules. The structure of these molecules remains constant; however because of inheritance of

chromosomes from parents to offspring, the behavior of each gene will emulate either of its ancestors traits.

If traits are only being passed down through each generation, there will not be any actual advance in a

species ability to survive. Mutation is the process where genes mutate in a child and because of which it

introduces new characteristics in the pool which can be harmful or beneficial to the communitys survival.

The ultimate test of good genes in a species is the predator. Evolutionary theory states that a predator is

another species that may or may not kill their prey prior to feeding on them, but the act of predation often

results in the death of its prey and the eventual absorption of the prey's tissue through consumption. This

means that for a species to survive, it needs to defeat its predator through some action; it can be elusive,

stealthy or be powerful enough to drive off the predator. Hence, in every generation, a certain set of individuals

are able to not be preyed upon. These individuals carry the necessary characteristics so that its next generations

survival is ensured. The individuals that fall to the predator had genes that were bad for the communitys

survival.

3Literal Definition : http://en.wikipedia.org/wiki/DNA


5/18


Page 5

IMPLEMENTATION OF EV OLUTION THEORY IN COMPUTER SCIENCE

EVOLUTIONARY ALGORITHMS

Evolutionary algorithms are inspired by evolution and uses biological mechanisms like reproduction,

mutation, re-population and selection to find optimal solutions to a problem. Genetic algorithms are a subset of

evolutionary algorithms that follow a generic guideline regarding the structure of the algorithm. Evolutionary

algorithms also contain evolution strategy and neuro-evolution techniques. All of these algorithms are widely

used in the world today with genetic algorithm being most popular simply because of its flexibility in structure.

GENETIC ALGORITHMS

All genetic algorithms are either derived from or explicitly follow the following model.


6/18


Page 6

In the initial setup phase, we begin by identifying a problem. The problem is re-interpreted so that it is

possible to develop a programmable solution for it. This solution is a set of behaviors or traits which in the

right circumstances is able to fix the problem. Generally, each solution has a finite number of traits; however,

some implementations can have an infinite number of traits.

The initial population is randomly generated and then fed into the program. The most common method

of defining a solutions traits is to represent each trait in a bit -wise format. For the purpose of this report, let us

have a look at a simple problem.

Using only the numbers 4, 5,6,7,8 and 9 and operators + and -, find a sequence that will equate to 25.

Now this problem already defines its traits for us. They are 4, 5, 6, 7, 8, 9, + and -, so in total, we have 8

traits.

If we are to represent these traits in a bit-wise format, we can do so using three digit strings.

STRING TRAIT000 Number 4

001 Number 5

010 Number 6

011 Number 7

100 Number 8

101 Number 9

110 Operator +

111 Operator -

A quick solution that we can think of ourselves would be 4+5+9+7. Using the encoding table above, oursolution would be represented by the string 000110001110101110011.

Depending upon the complexity of the problem, we can either randomly generate a population with a

fixed number of strings or have it completely random by generating any number of strings. Note that in this case

it is possible to have nonsensical strings containing too many operators or too many numbers. To avoid this, we

add a filter after the random generation which eliminates such type of solutions.

REPRODUCTION

After we have our initial population, we start the reproduction process. In the general model, we select

any two parent solutions; apply a cross-over technique to generate a child solution. There are multiple cross-

over techniques which are all viable, but the uniform cross-over and zipper cross-over techniques are the most

popular.


7/18


Page 7

The general model of genetic algorithms will not mix bits between separate strings. This way the

developers can ensure that the traits of the parents will be consistent and will appear again in the child solution.

However, because of the flexible nature of genetic algorithms, if required we can have cross-over between two

strings.

MUTATION

A mutation function in a genetic algorithm will change the bits of a child solution (after cross-over). The

rate at which child population mutates can be controlled by the engineer by setting a threshold value which

triggers once there are a certain number of un-successful generations. It can also be designed to introduce

mutation randomly based of a specific percentage. This method is most popular as it we ensure at least a

portion of the child population undergo mutation and hence have some unique traits to them.

From Parent A From Parent B

Before Mutation

After Mutation


8/18


Page 8

FITNESS FUNCTION

Once the entire child population is generated, a fitness function is run on each individual. A fitness

function determines the fitness score of each child solution by running each solution into the problem. It then

checks how much of the problem was the solution able to solve and assigns a score accordingly.

A fitness function in a genetic algorithm removes the unfit individuals from reach the reproduction

stage. This ensures that we obtain the best genes possible going forward. For our example stated earlier since

the problem is simple, our fitness function can also be simple. I believe a fitness score which is equal to the

absolute value of the difference between the target number and the solution number would be ideal. This way a

solution with the fitness score 0 would be our ideal solution.

The fitness function is the most important process in a genetic algorithm. If the fitness function is poorly

designed, the algorithm would be sub-optimal as either every generation would not come close to the perfect

solution or possible solutions would get removed by the fitness function. For complex problems, a fitness

function may be an entire project.

Good Fitness Score Next Gen Parent

Good Fitness Score Next Gen Parent

Bad Fitness Score -Discarded

Bad Fitness Score -Discarded


9/18


Page 9

LIMITATIONS OF CURRENT APPROACH

Even though the genetic algorithm is very flexible, it still has some limitations. These limitations are the

reason that other evolutionary algorithms like neural networks are preferred over genetic algorithms.

VIABILITY OF GENETIC ALGORITHMS

Genetic algorithms are generally computationally expensive. A fitness function for a complex, multi-

dimensional problem takes generally hours of high-end processing time. Sometimes, it is not viable to use

genetic algorithms for such problems as the cost of such a project overbears on the efficient solution that it

provides.

CONVERGENCE TOWARDS A NON-OPTIMUM SOLUTION

Fitness functions are designed according to the problem that we are facing. However, since most

problems are much more complex than our example, they are broken down in sub-problems and GA (genetic

algorithm) is applied to all of them. However, this way we sometimes lose focus on the big-picture and design

fitness functions which will give us an optimum solution to a sub-problem. But once this solution is integrated

into the bigger parent, we realize that it wasnt that optimal at all.

Because of this, fitness functions need to be precise and also need to consider the entire problem.

TIME-CONSUMPTION

As mentioned before, fitness functions on huge problems are computationally expensive. To check the

fitness score for an entire generation, it takes an abnormally large amount of processing time. Also, because of

incorrect logic in the mutation, cross-over or fitness functions, it is entirely possible to scrap days of work and

start from beginning. Sometimes, the fitness function in itself wastes time as developers redo it because of bad

initial design.


10/18


Page 10

IMPLEMENTATION OF GA IN GAME DEVELOPMENT

In the current video game market, there are not a lot of games that actually deploy genetic or

evolutionary algorithms to create a better AI Most work in this field still falls under the score of R&D.Unfortunately, until someone designs the perfect algorithm that can adapt the AI to any circumstances,

genetic algorithms are going to remain obscured.

Here are some of the projects that use genetic algorithms for AI

INVAIDERS

InvAIders was designed by Michael Martin as a genetic AI test. His goal was the same as mine,

which is to design an AI that can adapt endlessly to the players actions. For performing this experiment,

Michael designed a shoot-em-up genre with invaders style gameplay. He released the game on Xbox Liveand it was considered to be a moderate success for a one-man development team.

InvAIders pits the player against generations of enemy AI. When the player beats a generation,

the genetic algorithm ranks the AI and uses them to create a new generation of enemy to fight the player.

Through successive generations, the AI is able to learn every tactic that the player can potentially deploy.

Although, this technique can be considered by some as trial-and-error, it is still effective. The

blog post4describes the entire development process and the pros and cons of the technique too.

InvAIders screenshot

4

http://gamasutra.com/blogs/MichaelMartin/20110830/8325/Using_a_Genetic_Algorithm_to_Create_Adaptive_Enem

y_AI.php


11/18


Page 11

TOWER OF REUS

Tower of Reus was developed by a team at DeVry University in Irving, Texas as a part of their

semester paper. The game is a tower-defense type with genetic algorithm applied to an AI which places

towers along the enemies path. The enemies in each wave are fixed and over a period of successive

generations, the algorithm is able to generate an AI which can withstand the wave until the end.

Tower of Reus was interesting during my research because they implemented the most basic model

of genetic algorithm quite well. As it can be seen in the video, each AI is given a base logic of where to NOT

place towers and then the process is started. The USP of the game is that the player creates maps and then

lets the AI try out and beat the map. If the map is beaten, the player is given the chance to play his own map

and have a better score than the AI. Its an interesting twist of the classic tower defense genre.

Tower Of Reus Screenshot


12/18


13/18


Page 13

REMODELING GENETIC ALGORITHM

For my implementation of genetic algorithms I decided to not use them to completely

model my AI. Instead of having the algorithm let the AI do whatever it wants, I gave every AI a base logic. I usedthe genetic algorithm to moderate the values used in my logic. This technique is called evolutionary

programming6and is one of the four major evolutionary paradigms.

The basic EP method involves 3 steps (Repeat until a threshold for iteration is exceeded

or an adequate solution is obtained):

1. Choose an initial POPULATION of trial solutions at random. The number of solutions in a population ishighly relevant to the speed of optimization, but no definite answers are available as to how many

solutions are appropriate (other than >1) and how many solutions are just wasteful.

2. Each solution is replicated into a new population. Each of these offspring solutions are mutatedaccording to a distribution of MUTATION types, ranging from minor to extreme with a continuum of

mutation types between. The severity of mutation is judged on the basis of the functional change

imposed on the parents.

3. Each offspring solution is assessed by computing its fitness. Typically, a stochastic tournament is held todetermine N solutions to be retained for the population of solutions, although this is occasionally

performed deterministically. There is no requirement that the population size be held constant,

however, or that only a single offspring be generated from each parent.

Generally EP does not have crossover as a genetic operator. However, for the sake of

experiment, I decided to include crossover before mutating a child solution. This way, I could adhere to the basic

model of a genetic algorithm and still perform EP on my project.

6http://en.wikipedia.org/wiki/Evolutionary_programming


14/18


Page 14

MY IMPLEMENTATION

Before starting implementation, I had to think a lot about the genre of the game in which I was going to

implement it. Since turn-based strategy and tower defense games require a complex base which I didnt have at

the beginning. So for the sake of simplicity, I decided to develop upon a 2D platformer base which we had made

for our class assignment.

My 2D Platformer

Initially, in my project the 2D platformer had platforms of varying width placed at a random width

and height from the last platform. The player was a cube that had to traverse this level which didnt have an

end. I changed the varying width to be a constant width and slightly altered the randomness in the separation of

the platforms so that they werent too far away from each other.

One of the major mistakes I made during my implementation was to do too much at once. I addedsmall rectangles that would move in the negative x direction and imitate flying enemies. I also had the platforms

move vertically on their own and added vertical obstacles on each platform which would be placed at any

possible position.

After designing this endless level I started designing my AI. Unfortunately because of the complexity

of the game (where the player had to dodge enemies, jump precisely and negotiate theobstacles), the rigid AI


15/18


Page 15

logic wasnt working like I thought it would. I wasted a lot of time in tweaking the AI logic until I realized my

mistake.

Then I decided to scrap all the extra elements I added in the platformer leaving only platforms which

didnt move on their own. My primary logic for the AI in this level is below:

AI Logic


16/18


Page 16

The values for Walk Acceleration, Jump Acceleration, Maximum X velocity, Maximum Y velocity,

Minimum X distance from next platform, Minimum Y distance from next platform were parameters that the EP

algorithm would solve for me.

After running the algorithm for one whole day, it came upon the perfect solution in its 30th

generation. I tested the solution exhaustively until I concluded that I can continue further and added obstacles

at the center of each platform. I had to change the AI logic so that it would jump over the obstacle.

New AI Logic


17/18


Page 17

The fitness function for this new AI was designed so that if current solution distance is greater than20*(generation count+1) then advance it to the next generation else cull it.

Also because some AI were not even able to get past the first obstacle, I added in a destruction timer

on each so that after 30 seconds the AI would self-destruct (even if it was able to get past the first obstacle) and

the next AI would start the level again. This proved to be good since I was able to calculate how much variation

there was in a parent and a child solution. As the results indicated, there wasnt a noticeable difference.

EXTENDING THE CURRENT PROTOTYPE

From the beginning my goal has been to develop adaptive AI. Using the techniques learnt in this

technical research I feel that designing an adaptive AI would be possible. Currently I was using EP to alter AI logic

values; I believe if it would be possible to encode all AI actions into a bit string, it would also be possible to use

bit-swapping crossover techniques so that in any situation our AI will be able to respond with any action from its

list. Once it performs this action, we can assess it with our fitness function and then allow the AI with good

fitness scores to go ahead.

Although this theory looks promising it can only be tested once I can apply it to a project which needs its

use. But since most games limit the players actions, I feel using this technique would be a waste of time. Instead

if a project which had real immersion and which allowed the player to perform any action that a human being

can perform, an AI with this technique would be the perfect catalyst for a good game.

CONCLUSION

Genetic algorithms are efficient; there is no doubt about that. For relatively simple solutions which

require a lot of grunt work, it is viable to setup a GA system which does the work for you. GA is also the base for

almost all of the new optimized evolutionary algorithms as well. However, for complex problems it is far more

cost-efficient to use new evolutionary algorithms as they are designed to be better as the problem gets biggerand complex too.

In game development, evolutionary algorithms can be widely used. Currently the only projects using

evolutionary algorithms for artificial intelligence are small indie companies or one man teams doing so for

experimental reasons. I believe if a game is designed for an adaptive AI which uses evolutionary techniques it

would revolutionize the game development industry.


18/18


REFERENCES

1. Wikipedia (Genetic algorithms, Evolutionary algorithms, Evolutionary Programming)a. http://en.wikipedia.org/wiki/Genetic_algorithmb. http://en.wikipedia.org/wiki/Evolutionary_algorithmc. http://en.wikipedia.org/wiki/Evolutionary_programming2. Genetic algorithm tutorial http://www.ai-junkie.com/ga/intro/gat1.html

3. What are Evolutionary Algorithms (EAs)?http://www.cs.cmu.edu/Groups/AI/html/faqs/ai/genetic/part2/faq-doc-1.html

4. H. Chase Stevens Blog http://www.chasestevens.com/blog/read_post.php?time=13566639535. Tower of Reus

a. Main Page http://www.cameronferguson.net/projects/games/towers-of-reusb. Gameplay https://www.youtube.com/watch?v=rHhpyisO0vw

6. Using a Genetic Algorithm to Create Adaptive Enemy AI (InvAIders)http://gamasutra.com/blogs/MichaelMartin/20110830/8325/Using_a_Genetic_Algorithm_to_Crea

te_Adaptive_Enemy_AI.php

7. Genetic Algorithms and Evolutionary Computationhttp://www.talkorigins.org/faqs/genalg/genalg.html

genetic ai: technical report

Documents