Marco AdelfioCMSC 828N – Spring 2009

General Game Playing (GGP)

Classic Game Playing AI

Deep Blue

TD-Gammon

Poki

General Game Playing AI

GGP Agent

General Game PlayingGGP Goals:

Create systems to play arbitrary games (given formal game definitions)

Eliminate game-specific strategiesEmphasize generic strategy formulation

Competition created by Stanford Logic GroupHosted during AAAI conference since 2005$10,000 Grand Prize

General Game PlayingQuestions:

What additional challenges arise for GGP agents?

How should a GGP agent evaluate game states?

Can a GGP agent transfer knowledge between games?

General Game PlayingFinitely many players, statesGame play controlled by Game Manager

over networkPlayers act synchronously (noops allowed)Time limits enforcedBasic agent must:

Understand rule specificationRespond to game states with legal actionsRecognize a terminal state and its payoffs

Game Definition LanguageA game definition must logically define:

Set of states in the gameLegal actions for each player from a given

game stateTransition functionInitial stateTerminal states and their payoffs

Game Definition Language - Example(role p1)

(role p2)

(init (cell 1 1 b))

(init (cell 1 2 b))

…

(init (control p1)

…

(<= (legal ?w (mark ?x ?y))

(true (cell ?x ?y b))

(true (control ?w)))

…

(<= (next (cell ?m ?n x))

(does p1 (mark ?m ?n))

(true (cell ?m ?n b)))

…

(<= (row ?m ?x)

(true (cell ?m 1 ?x))

(true (cell ?m 2 ?x))

(true (cell ?m 3 ?x)))

…

(<= (line ?x) (row ?m ?x))

(<= (line ?x) (column ?m ?x))

(<= (line ?x) (diagonal ?x))

…

(<= (goal p1 100)

(line x))

(<= (goal p1 0)

(line o)

…

(<= terminal

(line x))

Game CommunicationGame Manager Message Game Player Response(START MATCH.435 WHITE description 90 30)

READY

(PLAY MATCH.435 (NIL NIL)) (MARK 2 2)(PLAY MATCH.435 ((MARK 2 2) NOOP)))

NOOP

(PLAY MATCH.435 (NOOP (MARK 1 3))

(MARK 1 2)

(PLAY MATCH.435 ((MARK 1 2) NOOP))

NOOP

... ...(STOP MATCH.435 ((MARK 3 3) NOOP)

DONE

General Game PlayingDesign Challenges:

IndeterminacySizeMulti-game CommonalitiesOpponent Recognition

AAAI Competition – Past Winners2005 - ClunePlayer (UCLA)2006 - FluxPlayer (Technical University of

Dresden)2007 - CADIA (Reykjavik University)2008 - CADIA (Reykjavik University)

Agent 1: ClunePlayerApproach: MinimaxProblem: Needs to assign values to

intermediate game states in arbitrary games.Solution:

1. Calculate a vector of generic features at each node

2. Simulate games to determine which features are “stable” and correlated with either payoff or control

3. When running minimax, use a combination of those scores as the evaluation heuristic

Agent 2: CADIA-PlayerApproach: UCT (Variant of Monte Carlo

simulation)Monte Carlo:

Pick random actions for each player to descend the tree

After reaching a terminal state, update expected payoff Q(s,a) for each visited state s and action a

Introduces explore/exploit tradeoff

Agent 2: CADIA-PlayerUCT (Upper Confidence

bound for Trees)Balance exploration

and exploitationGive “bonus” to less

travelled paths

Agent 3: UTexas LARGApproach: Knowledge TransferUses lessons from past games to improve

play in new gamesWar Games!

Determines whether a new game is isomorphic or similar to a previous game. If so, transfer estimated rewards

SummaryGeneral Game Playing introduces a

different set of challenges than designing game-specific AI

Biggest challenge is evaluating states in a novel game

Better understanding of general strategy formation has many applications

ReferencesGGP Website: http://games.stanford.edu/Hilmar Finnson.

CADIA-Player: A General Game Playing Agent. MSc Thesis, School of Computer Science, Reykjavik University. 2007.

Kuhlmann, Gregory and Peter Stone. Graph-Based Domain Mapping for Transfer Learning in General Games. Lecture Notes in Computer Science, Volume 4701/2007.