Yes,you can play Monopoly with a genetic algorithm

Niels Zeilemaker

About me• PhD in Parallel Computing @ TUDelft• Data hacker at GDD• Data scientist and/or data engineer

• Basically, drink coffee a lot while waiting for

Monopoly• Boardgame created in 1935• For 2-6 players• Buy houses/hotels –>• Last man standing wins

• Pronounced in dutch as moonoopooly

Terminology• Players traverse the board by rolling

two dice• When landing on an unowned property,

a player can decide to buy it• If a player owns all properties of a

single color, he can buy houses• Visiting a property owned by another

player you will need to pay rent

Rules• A player passing Go receives $200• All fines are paid into the free parking space• When a player does not buy an unowned property an

auction is held• Rolling doubles three times in a row results in goto jail• If a player lands on Go he receives $400• Rolling doubles will release a player from jail• Paying $50 will release a player from jail

Rules• A player passing Go receives $200• All fines are paid into the free parking space• When a player does not buy an unowned property an

auction is held• Rolling doubles three times in a row results in goto jail• If a player lands on Go he receives $400• Rolling doubles will release a player from jail• Paying $50 will release a player from jail

How much free time do you have?• Interesting game, • still doesn’t explain why you would attempt to “solve it”

• GDD Friday + = “Monopoly Simulations”

• He implemented a small python script which would roll dice + go to jail

See: http://koaning.io/monopoly-simulations.html

Probability distribution

Datascientists writing code• Initial script, 46 lines• “Improved” script, 800 lines

How much free time do you have?• Implemented • Board object• Player object• Deed object• BankruptException ;)

• Unit-tested, coverage ~80%

Player objectclass Player(object): def buy_position(self, deed, amount): # after landing on a deed def bid_position(self, deed, amount, current_bid): # while in an auction def anything_else(self): # at the end of the turn def raise_amount(self, amount): # when we need to pay for something

First players implemented• BuyNothing• BuyAll• BuyFrom• BuyBetween

Player BuyAll BuyFrom '10' BuyBetween '10-20'0

50

100

150

200

250

300

350

400

450

Wins

Which deeds to buy?

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39€ 0.00

€ 100,000.00

€ 200,000.00

€ 300,000.00

€ 400,000.00

€ 500,000.00

€ 600,000.00

Money raised/invested

raised invested

But, which will help you to win?

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 390

50

100

150

200

250

300

350

Caused bankruptcy

Next up, genetic algorithms• Manually selecting which deeds to buy/how many

houses to build isn’t a lot of fun

• Enter, the GAPlayer

Genetic algoritms?• Search heuristic which mimics natural selection

• Convert search space into chromosome• Eg 1,1,0,3,3, etc.

• Evaluate performance of chromosome using fitness function• 1,1,0,3,3,… = 42• 1,2,0,3,3,… = 42

I'm no expert on genetic algoritms, feel free to correct me

Genetic algoritms?• After each round (or population)• Select top X chromosomes• Append some mutations• Generate offspring by combining those

• Stop after X generations

Genetic algoritms?

Algorithm A: 1,0,0,0,0Algorithm B: 0,0,2,2,2

Child C: 0,0,2,0,2

Child D: 1,0,0,2,2

GAPlayer• Chromosome:• 0,6,0,5,0,1,2,0,3,2,0,5,1,5,4,1,1,0,0,1,0,0,0,0,0,1,1,0,1,0,0,0,1,

0,0,1,0,4,0,4

• 0 = don’t buy• 1 = buy• 2..5 = build 1..4 houses• 6 = build hotel

651

232

Fitness function• Implemented two approaches:• Play vs BuyAll• Use TrueSkill

• Not sure if either is correct ;)

Fitness function, vs BuyAll• Play 1000 matches against BuyAll• Score = #times won

• Evaluate 10 generations• Population size = 56• Select 20% best perfoming• Mutate 1% of the solutions

• Single generation takes 1 minute to evaluate

Scores vs Generations

1 2 3 4 5 6 7 8 9 10400

450

500

550

600

650

vs BuyAll

But wait• So, GaPlayer '[0, 6, 0, 5, 0, 1, 2, 0, 3, 2, 0, 5, 1, 5, 4, 1,

1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 4, 0, 4]‘ is a better strategy than buying everything….

• But is it the best monopoly strategy?

BuyAll GaPlayer….

TrueSkill• A ranking system developed for Xbox player matching• In my understanding• Each player has a skill belief, eg a probability distribution• When you win/lose from another we update the probabilities

accordingly

• Let’s hope Vincent is in the room

An example• Before vs After• Poke1 loses from Poke2

Combining GA + Trueskill• Have a global leaderbord• Each new GA chromosome plays against 100 randomly

selected others• Each game now is best of 10

• Leaderbord is sorted by value of curve descending

Presenting

0,0,0,1,0,1,1,0,1,0,0,1,0,0,0,1,1,0,2,1,0,0,0,0,1,1,4,4,0,5,0,1,0,0,0,1,0,2,0,3

You will never beaten in monopoly again, maybe

Runner ups• 0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,1,6,0,6,6,0,0,0,0,0,1,0,1,0,0,0,5,4,0,4,1,0,2,0,3• 0,0,0,1,0,1,0,0,0,0,0,1,0,0,1,1,1,0,2,1,0,0,0,0,0,1,4,4,0,5,0,0,0,0,0,1,0,2,0,3• 0,1,0,1,0,1,0,0,0,0,0,1,0,0,1,1,1,0,0,1,0,0,0,0,0,1,4,4,0,5,0,0,0,0,0,1,0,2,0,3• 0,0,0,1,0,1,0,0,0,0,0,1,1,0,0,1,1,0,1,0,0,1,0,0,0,1,1,1,0,0,0,0,0,0,0,1,0,3,0,4• 0,1,0,2,0,1,1,0,0,1,0,1,1,1,0,1,1,0,2,2,0,0,0,0,0,1,4,3,1,4,0,0,0,0,0,1,0,2,0,3• 0,2,0,1,0,1,1,0,0,1,0,0,1,0,1,1,6,0,6,6,0,0,0,0,0,1,0,1,0,0,0,5,4,0,4,1,0,3,0,3• 0,2,0,2,0,1,1,0,0,1,0,0,1,0,0,1,6,0,5,6,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,3,0,2• 0,0,0,1,0,1,1,0,0,0,0,1,0,0,1,1,1,0,1,1,0,0,0,0,0,1,4,5,0,5,0,0,0,0,0,1,0,2,0,3• 0,2,0,1,0,1,1,0,2,1,0,0,1,0,1,1,6,0,6,6,0,0,0,0,0,1,0,1,0,0,0,5,4,0,4,1,0,4,0,3

So• We get some probable “best” solutions

• But is monopoly rock paper scissors?• We should be able to notice this by it not converging

• And, TrueSkill has some ”magic” numbers• Blame vincent

Future work/known issues• We implemented a single behavior to• Bid in an auction• Buy/Sell houses• When to start buying/selling houses

• We did not implement• Trying to buy streets from other players

• Opensource the stuffs

We’re hiring / Questions / Thank you

Niels Zeilemaker