games computers (cannot) play graham kendall automated scheduling, optimisation and planning...

81
Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Upload: alyson-stanwick

Post on 31-Mar-2015

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Graham KendallAutomated Scheduling, Optimisation and

Planning Research Group (ASAP)

Page 2: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Checkers: Why was it considered “beaten”?

Two approaches to Checkers

Games in ASAP

Poker (if time)

Contents

Page 3: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

1959. Arthur Samuel started to look at Checkers2

The determination of weights through self-play ( adapted, remained fixed)

39 Features

Included look-ahead via mini-max

Computers & Game Playing : A Potted History

2 Samuel A. Some studies in machine learning using the game of checkers. IBM J. Res. Develop. 3 (1959), 210-229

Page 4: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Samuels’s program defeated Robert Nealy, although the victory is surrounded in controversy

Was he state champion?

Did he lose the game or did Samuel win?

Computers & Game Playing : A Potted History

Page 5: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 23 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : Just about to make move 16

Page 6: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 23 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program)

Forced Jump

Page 7: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 23 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program)

Page 8: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 23 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program)

Strong(Try

to keep)Trapped

Only advance

to Square 28

Page 9: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 23 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program)

Page 10: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 23 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program)

Page 11: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

This was a very poor move.

It allowed Samual to retain his “Triangle of Oreo”

AND.. By moving his checker from 19 to 24 it guaranteed Samuel a King

This questioned how strong a player Nealy really was

Computers & Game Playing : A Potted History

Page 12: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 13: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 14: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

16-12 then 5-1, Chinook said would be a draw

Page 15: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 16: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 17: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 18: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 19: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23K

Page 20: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

This checker is lost

Page 21: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

Page 22: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

KThis checker

could run (to 10) but..

K

Page 23: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

K

Page 24: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

K

Page 25: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

Forced Jump

K

Page 26: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

K

Page 27: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

K

Page 28: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Computers & Game Playing : A Potted History

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

White (Nealey)

Red (Samuel’s Program) : After Move 25

23

K

Victory

Page 29: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Two Mistakes by Nealy

Allowing Samuel to get a King

Playing a move that led to defeat when there was a draw available

Computers & Game Playing : A Potted History

Page 30: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

The next year a six match rematch was won by Nealy 5-1.

Three years later (1966) the two world championship challengers (Walter Hellman and Derek Oldbury) played four games each against Samuel’s program. They won every game.

Computers & Game Playing : A Potted History

Page 31: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Checkers

Chinook

Blondie 24 (aka Anaconda)

Computers & Game Playing : A Potted History

Page 32: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Perfect

Each Player has complete knowledge of the game state

Usually only two players, who take alternate turns

Examples include Chess, Checkers, Awari, Connect-Four, Go, Othello

Types of Games

Page 33: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Imperfect

Some of the game state is hidden

Examples include Poker, Cribbage, Bridge

Types of Games

Page 34: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Games with an element of chance

The game moves have some stochastic element

For example, Backgammon

Types of Games

Page 35: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Types of Games

Solved or Cracked

Over Champion

World Champion

Grand-Master

Amateur

Connect-Four Checkers (8x8)

Chess Go (9x9) Go (19x19)

Qubic Othello Backgammon

Nine Men’s Morris

Go_moku

Awari

6 Jaap van den Herik H., Uiterwijk and van Rijswijck J. Games Solved: Now and in the future. Artificial Intelligence 134 (2002) 277-311

Page 36: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Samuel’s work, perhaps, restricted the research into Checkers until 1989 when Jonathan Schaeffer began working on Chinook

He had two aims

To develop the worlds best checkers player

To “solve” the game of checkers

Page 37: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Chinook, at its heart, had an evaluation function

Piece count (+30% for a King)

Runaway checker

“Dog Hole”

The weights were hand-tuned

Page 38: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Opening game database from published work (with corrections they found)

Initially 4000 openings, leading to an eventual 40,000

“Cooks” – innovative lines of play that could surprise an opponent

The aim was to take opponents into unknown territory

Page 39: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Endgame database: Started writing in May 1989

The 8-piece endgame database finished on February 20th 1994

Page 40: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

1 120

2 6,972

3 261,224

4 7,092,774

5 148,688,232

6 2,503,611,964

7 34,779,531,480

8 406,309,208,481

Page 41: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

9 4,048,627,642,976

10 34,778,882,769,216

11 259,669,578,902,016

12 1,695,618,078,654,976

13 9,726,900,031,328,256

14 49,134,911,067,979,776

15 218,511,510,918,189,056

16 852,888,183,557,922,816

Page 42: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

17 2,905,162,728,973,680,640

18 8,568,043,414,939,516,928

19 21,661,954,506,100,113,408

20 46,352,957,062,510,379,008

21 82,459,728,874,435,248,128

22 118,435,747,136,817,856,512

23 129,406,908,049,181,900,800

24 90,072,726,844,888,186,880

TOTAL 500,995,484,682,338,672,639

Page 43: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

With a 4-piece database Chinook won the 1989 Computer Olympiad

In the 1990 US National Checkers Championship Chinook was using a 6-piece database.

It came second, to Marion Tinsley, defeating Don Lafferty on the way who was regarded at the worlds second best player.

Page 44: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Marion Tinsley

Held the world championship from 1951 to 1994

Before playing Chinook, Tinsley only lost 4 competitive games (no matches)

Page 45: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

The winner of the US Championship has the right to play for the world championship. Finishing second (with Tinsley first) entitled Chinook to play for the world championship

The American Checkers Federation (ACF) and the European Draughts Association (ADF) refused to let a machine compete for the title.

Page 46: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

In protest, Tinsley resigned

The ACF and EDF, created a new world championship, “man versus machine” and named Tinsley as the world champion.

At this time Tinsley was rated at 2812, Chinook was rated at 2706

Page 47: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

The match took place 17-29 Aug 1992.

The $300,000 computer used in the tournament ran at about half the speed of a 1GHz PC

The match finished 4-2 in favour of Tinsley (with 34 draws)

Page 48: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

A 32 game rematch was held in 1994

8-piece end game

Processors four times as fast (resulted in a factor of 2 speed up due to more complex evaluation function and the overhead of parallel processing)

Opening book of 40,000 moves

In preparation Chinook no losses in 94 games against Grandmasters

Page 49: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Six games in (1-1, with 4 draws) Tinsley resigned for health reasons. His symptoms were later diagnosed as pancreatic cancer.

Tinsley died on 3rd April 1995 (aged 68). Undoubtedly the best player ever to have lived

Chinook was crowned the man versus machine champion. The first automated game player to have achieved this.

A 20-match with Don Lafferty resulted in a draw (1-1, with 18 draws)

Page 50: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 1: Checkers

Opening Game Database(40,000) moves

End Game Database(8-pieces)

Hand CraftedEvaluationFunction

(/ search)

Won the World (ManVersus Machine)Championship

in 1994…

…defeating the world whohad held the title

for 40 years

Marion Tinsley lost his 5th,6th and 7th games to

Chinook

Schaeffer J. One Jump Ahead:Challenging Human Supremacy

in checkers, Springer, 1997

Page 51: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Project started in the summer of 1998, following a conversation between David Fogel and Kumar Chellapilla

It was greatly influenced by the recent defeat of Kasparov by Deep Blue

Chess was seen as too complex so “draughts” was chosen instead

The aim is to evolve a player – rather than build in knowledge

Page 52: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Reject inputting into a neural network what humans think might be important

Reject inputting any direct knowledge into the program

Reject trying to optimise the weights for an evaluation function

Page 53: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

The Gedanken Experiment

I offer to sit down and play a game with you. We sit across an 8x8 board and I tell you the legal moves

We play five games, only then do I say “You got 7 points.”I don’t tell you if you win or lost

We play another five games and I say “You got 5 points”

You only know “higher is better”

Page 54: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

The Gedanken Experiment

How long would it take you to become an expert at this game?

We cannot conduct this experiment but we can get a computer to do it

Page 55: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Samuel’s Challenge: “Can we design a program that would invent its own features in a game of checkers and learn how to play, even up to the level of an expert?”

Page 56: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Newell’s Challenge: “Could the program learn just by playing games against itself and receiving feedback, not after each game, but only after a series of games, even to the point where the program wouldn’t even know which programs had been won or lost?”

Newell (and Minsky)7 believed that this was not possible, arguing that the way forward was to solve the credit assignment problem.

7 Minsky M. Steps Towards Artificial Intelligence. Proceedings of the IRE, 1961, 8-30

Page 57: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

I1

I32

HL11

HL140

HL21

HL210

O

.

.

....

.

.

.

# weights=1741

Evaluation used for MiniMax

Later changed toan explicit piece

differential

Page 58: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

K

Page 59: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1

7

8

14

25

27

+1

+1

+1

-K

-1

-1

All other neurons have an value of zero

Page 60: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Algorithm

Initialise 30 Networks

Each network played 5 games as red against random opponents

Games were played to completion or until 100 moves had been made (a draw)

+2 for a win, 0 for a draw, -1 for a loss

15 best performing networks were saved for the next generation and copies were mutated

Page 61: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Observations

The points for a win, lose draw were set such that wins were encouraged. No experimentation with different values were tried

Players could play a different number of games. This was, purposefully, not taken into account

Mutation was carried out using an evolutionary strategy

Page 62: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

After 10 Generations

After 10 generations the best neural network was able to beat both its creators and a simple (undergraduate project) program which, by the authors admission was “weak”

Note: 400MHz PC

Page 63: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

ACF Ratings

Grand (Senior) Master 2400+ Class E 1000-1199

Master 2200-2399 Class F 800-999

Expert 2000-2199 Class G 600-799

Class A 1800-1999 Class H 400-599

Class B 1600-1799 Class I 200-399

Class C 1400-1599 Class J <200

Class D 1200-1399

Page 64: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

After 100 Generations

Playing on zone.com

Initial rating = 1600

Beat a player ranked at 1800 but lost against a player in the mid 1900’s

After 10 games their ranking had improved to 1680. After 100 games it had improved to 1750

Typically a 6-ply search but often 8-ply

Page 65: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Observations

The highest rating it achieved was 1825

The evolved King value was 1.4, which agrees with perceived wisdom that a king is worth about 1.5 of a checker

In 100 generations a neural network had been created that was competitive with humans

It surpassed Samuel’s program

The challenge set by Newell had been met

Page 66: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

The Next Development

Alpha-Beta Pruning introduced and evolved over 250 generations

Over a series of games, Obi_WanThe Jedi defeated a player rated at 2134 (48 out of 40,000 registered) and also beat a player rated 2207 (ranked 18)

Final rating was 1902 (taking into account the different orderings of the games)

Page 67: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

The Next Development

Spatial nature of the board was introduced as at the moment it just “saw” the board as a vector of length 32

Page 68: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

36 3x3 Overlapping

squares

Page 69: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

25 4x4 Overlapping

squares

Page 70: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

16 5x5 Overlapping

squares

Page 71: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

9 6x6 Overlapping

squares

Page 72: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

4 7x7 Overlapping

squares

Page 73: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

21 22 24

25 26 2728

29 30 31 32

23

1 8x8 Overlapping

squares

Page 74: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

The Next Development

36+25+16+9+4+1 = 91 inputs

5,046 weights

Page 75: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

I1

I32

O

.

.

.

# weights=5046

36 3x3

25 4x4

16 5x5

9 6x6

4 7x7

1 8x8

HL1

(91 nodes)

HL2

(40 nodes)

HL3

(10 nodes)

Sum of 32 Board Inputs

Page 76: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

2 months and 230 generations later!!

After 100 games the rating was 1929

A 27 point increase over the previous network. Nice but not decisive

Maybe it was due to there being three times more weights but the training period was the same?

Page 77: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

6 months and 840 generations later!!

After 165 games it was rated at 2045.85 (sd 33.94)

Rated in the top 500 at zone.com (of the 120,000 players now registered)

That is better than 99.61% of the players

Page 78: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Playing Chinook8

In a ten match series against Chinnok novice level it had two wins, two losses and 4 draws

8 Fogel D. B. and Chellapilla K. Verifying Anaconda’s expert rating by competing against Chinook: experiments in co-evolving a neural checkers player, Neurocomputing 42 (2002) 69-86

Page 79: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Blondie

The neural checkers player went through a number of names

David0111

Anaconda

Blondie24

Page 80: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

Page 81: Games Computers (cannot) Play Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Games Computers (cannot) Play

Case Study 2: Anaconda

References

Fogel D.B. Blondie24: Playing at the Edge of AI, Morgan Kaufmann, 2002

Fogel D. B. and Chellapilla K. Verifying Anaconda’s expert rating by competing against Chinook: experiments in co-evolving a neural checkers player, Neurocomputing 42 (2002) 69-86

Chellapilla K and Fogel D. B. Evolving neural networks to play checkers without expert knowledge. IEEE Trans. Neural Networks 10(6):1382-1391, 1999

Chellapilla K and Fogel D. B.Evolution, neural networks, games, and intelligence, Proc. IEEE 87(9):1471-1496. 1999

Chellapilla K and Fogel D. B. Evolving an expert checkers playing program without relying on human expertise. IEEE Trans. Evolutionary Computation, 2001

Chellapilla K and Fogel D. B. Anaconda Defeats Hoyle 6-0: A Case Study Competing an Evolved Checkers Program Against Commercially Available Software. Proc. Of CEC 2000:857-863