The Future of Chess-Playing Technologies and the SignificanceKasparov Versus Deep Blue

Dennis DeCosteJet Propulsion Laboratory / California Institute of Technology

4800 Oak Grove Drive; Pasadena, CA 91009http ://www-aig. jpl. nasa. gov/home/decoste/, decoste~aig, j pl.nasa, gov

of

Abstract

In this paper we argue that the recent GarryKasparov vs. Deep Blue matches are significantfor the field of artificial intelligence in severalways, including providing an example of valuablebaseline benchmarl~s for more complex alterna-tives to contrast and justify themselves. We willalso briefly summarize some of the latest develop-ments on computer chess research and highlighthow our own work on a program called Chestertries to build on those developments to providesuch justifications.

Introduction

Since "falling from grace" (DSg0) by the late 1980’s,computer chess has recently received considerable at-tention in the popular media due to the 1996 matchbetween IBM’s Deep Blue chess machine and GarryKasparov, the reigning World Champion. Further-more, researchers of artificial intelligence seem to beincreasingly willing to cite Deep Blue as an exampleof success (e.g. (SBD+96)). The topic of computerchess, and game-playing in general, has also enjoyed aconsiderable resurgence of significant and creative re-search, particularly in the areas of search and machinelearning (as we shall discuss later).

In our view this reemergence of computer chess wassomewhat simply a matter of time -- requiring bothmaturity of technologies and better appreciation byAI researchers of the importance of empirical evalu-ation of comparative performance on problems withwell-defined success criteria.

In this paper, we will argue that the Garry Kasparovversus Deep Blue match ("GK vs DB") has great sig-nificance to AI in several ways. We will directly avoidphilosophical questions such as whether DB is a "think-ing machine" or how we might test such a proposition.However, we will argue that the development of DeepBlue represents a legitimate, and in some ways even arole model, example of the future of AI research. Wewill also summarize some recent developments in rele-vant AI research areas, including our own.

Significance for AI

Appreciating the significance of GK vs DB for AI re-quires assessing the significance of four distinct items:

1. the task of computer chess,

2. the solution instance represented by Deep Blue,

3. the matches between GK vs DB per se,

4. the closed-nature of Deep Blue development.

We explore these issues in turn below.

Significance of Computer Chess

Criticisms of computer chess that lead to its fall fromgrace partly arose from growing pressure to fund andconduct research that directly addressed "real world"problems. Much early work was particularly vunera-ble to such attack, due to the predominance of manualcrafting and tuning of search and evaluation functionsthat seemed required for competitive performance. Re-cent advances in software engineering, search, and ma-chine learning technology promise to make these issuesless germane in the future.

More fundamental has been the failure to addressthe issue of scaling-up. As Schank forcably argued in(Schgl) a key distinction between AI and engineeringis that AI strives foremost for representations and al-gorithms that scale-up gracefully with task complex-ity. Recent success of predominately brute-force ap-proaches such as Deep Blue seriously bring into ques-tion whether chess is complex enough to require moregeneral AI techniques which scale-up. Indeed, many re-searchers on game-playing have shifted to games suchas Go, whose sheer complexity seems certain to protectit from brute-force ’approaches. Others have arguedfor artificial generalizations, such as META-GAME(Pe194), which would better justify the use of learn-ing and sophisticated search.

Some argue that chess is simply not a rich and im-portant enough problem to warrant significant atten-tion. However, the visibility and attention of the GKvs DB match demonstrates that chess performance re-mains a valued and easy-to-grasp metric. Furthermore,recent work on "rational search" (Bau92) highlights

From: AAAI Technical Report WS-97-04. Compilation copyright © 1997, AAAI (www.aaai.org). All rights reserved.

the fact that even though chess is a deterministic game,the importance of reasoning with uncertainty, whichdominates many real-world tasks, arises even in chess.

Significance of Deep Blue

To evaluate the extent to which Deep Blue and othercomputer chess work is "Ar’, it is useful to recallthat AI has several components, most notably: knowl-edge representation schemes, search, and learning. Thebest performing computer chess projects have often re-quired significant manual knowledge encoding/tuningper se, causing some doubt of thei.r being AI.

In contrast, the brute-force approach of Deep Bluedearly focusses on the issue of search over knowledge.However, Deep Blue’s contributions to our understand-ing of this issue are debatable. For example, recentwork has shown that there appear to be significant di-minishing returns in chess around ply 10 (JSB+), muchas that occuring at shallower plys for simpler gamessuch as checkers. Thus, it may be the case that con-tinued work along Deep Blue’s brute force lines will of-fer very little additional insight. Indeed, the designersof Deep Blue have publically stated that for the up-coming May 1997 rematch, the enhancements to DeepBlue have focussed more on improved knowledge thansearch per se. Whether such knowledge engineeringwill be of the unscalable and chess-specific sorts thatmade early chess work easier to dismiss or not remainsto be seen.

Significance of GK vs DB

In our view, the largest significance of Deep Blue is infact in the GK vs DB matches per se. The role andutility of "competitions" in driving and evaluating AIresearch seems to be getting increased acceptance inrecent years, in part due to the easier disemination ofdata sets and domain knowledge via the advent of theWorld Wide Web. For example, the Sante-Fe Instituterecently organized a successful competition to comparetechniques on blind time-series prediction tasks overseveral data sets (WG94).

Of course, the history of computer chess is full ofcompetitions, most notably the ACM tournaments.However, competition among relatively weak computerprograms, as should be expected, tends to overly-reward short-term competitive advantages, such ascooking opening books. Such tricks are of little useagainst a flexible opponent such as Kasparov, particu-larly given that he is well-versed in the weaknesses andtendencies of existing computer chess programs.

It was thus of considerable interest when Kasparovproclaimed in his TIME article after the 1996 matchthat "I could feel -- I could smell -- a new kind ofintelligence across the table" (Kas96). His surprisinginitial loss in Game 1, due to under-appreciating the ef-fectiveness of Deep Blue’s search, and his final crushingvictory due to vastly superior positional play in Game

6 were dramatic contrasts in the current strengths andweaknesses of human and computer approaches.

We believe that similar evaluations will play an in-creasingly common role in future development of AI.First, because it is a natural human curiousity to wantto compare machine and human approaches to achiev-ing intelligent behavior. Second, because such com-parisons can in fact provide useful scientific feedback.Similar feedback during the development of medicalexpert systems, for example, seems to have been a sig-nificant influence in the development of approaches toreasoning under uncertainty, such as Bayesian beliefnetworks, that are now popular and promising in AIresearch.

Also, the importance of visibility can perhaps not beunderestimated. Indeed, much of the public before the1996 GK vs DB match seemed to have believed thatcomputers had already "solved chess", or at least werebetter than all humans. Anyone who wants assurancethat AI is still no match for human intelligence andflexibility need look no further than Game 6.

In short, we view the current significance of DeepBlue to be largely in providing some highly visible datapoints to better understand how far largely brute-forcesearch can go in one historically-significant and com-plex domain. We believe it extremely important thatsuch work continue to parallel that of more fundamen-tM AI research. At the very core of the notion of intel-ligence is the notion of speed of computation. Indeed,typical IQ tests do not allow one to bring the test homeand work it out in leisure over the course of a year. Inthis sense, better case studies into when specific stylesof search and shortcuts are sufficient for a domain athand are critically important.

Closed-Nature of Deep Blue Work

One criticism of Deep Blue has been that the specifictechniques and knowledge used have not been widelyreported, making it difficult to adequately assess whatis really being demonstrated during these matches. Infact, to many, Deep Blue simply seems synonomouswith "the brute-force approach". Much of our knowl-edge comes from early work on the predecessor, DeepThought (HACN90) (ACH90).

Furthermore, the number of published games ofDeep Blue is relatively very small. All such exam-ples of under-reporting seem to be motivated at leastin part by the desire to keep a competitive edge overKasparov. It would be scientifically far more pleas-ing for the Deep Blue team, for example, to have thematch-ready version of Deep Blue generate hundredsor even thousands of games -- perhaps both from self-play and against their chess consultant Grand MasterJoel Benjamin -- for Kasparov to have in preparationbefore rematches. Similarly, Deep Blue could prepare(with suitable machine learning) from the huge his-toric database of Kasparov’s games. Presumably, Kas-parov’s evaluation function and search techniques will

10

not change as rapidly as Deep Blue’s might.Perhaps if Deep Blue wins an entire match against

Kasparov, it might become more acceptable to allowsuch preparations, leading to more significant evalu-ation of in what ways the human mind may still besuperior in the domain of chess. Thus, we argue thatonce (or if) a computer beats the world champion in match, perhaps the real science will begin in earnest,although presumably with significantly less public in-terest and drama.

Summary of Chess-Playing

Technologies

In light of the diminishing returns of brute-force searchin chess noted above, we have considerable doubts onwhether the brute-force style epitomized by Deep Bluewill actually suffice to prevail against Kasparov. In thissection we highlight a variety of other AI techniquesthat offer great promise in the longer run.

Search

Perhaps due to the success of Deep Blue, many seemto believe or to assume that parallel algorithms are theprime future technology for computer chess. However,in fact, within the last decade there have been a widevariety of advances in research on game-tree search.

It is important to keep in mind that, due to the de-sign requirements that the Deep Blue hardware placeson its search algorithms, it is not clear which of thesetechniques could be easily incorporated into the DeepBlue approach.

Some elegant advances, though relatively modestperformance-wise, have arisen from recent work byPlatt et al. Their MTD(f) procedure (pSPdB95)(PSPdB96a) (Pla) essentially reformulates the plex best-first method of SSS* (Sto79) into an elegantcombination of zero-window alpha-beta searches andextensive use of transposition (hash) tables. This workhas led to modest yet surprising speed improvementsfor chess (15%) as well as significant clarity aboutalpha-beta and SSS* variants in general.

Their work on Enhanced Transposition Cutoffs(ETC) (PSPdB96b) takes advantage of the fact game "trees" are actually graphs. For example, theyexplore heuristics to maximize reuse of transpositiontable results, such as checking moves with transposi-tion table entries first under certain conditions, sincethose might lead to immediate cutoffs. They reportthat these heuristics lead to searches with 25% fewernodes.

Less clear at this point, but potentially much morepowerful, are methods for selective tree growth. Re-cent results (BM96) on approximate probabilistic vari-ants of Berliner’s well-known B* algorithm, for exam-ple, appear promising. B* involves search using in-tervals to represent position values, instead of points.Search continues until enough information is gathered

and propagated up the tree such that the low estimateon the value of one move becomes higher than the highestimate of all alternatives.

Since position evaluators generally only provide "re-alistic" point estimates, B* uses null moves (GC90)(and other heuristics) to generate optimistic and pes-simistic values. To avoid the complexity of managingfull probabilistic distributions across these intervals,they explored the use of probability distributions thatdecay linearly from the realistic value to the optimisticvalue. To handle instability of position evaluations,they use alpha-beta to conduct shallow probe searches(e.g. full 3-ply, plus tactical extensions).

Recently "rational search" methods have been pro-posed for selective expansion based on Bayesian meth-ods. This approach has been championed by EricBaum (Bau92) (Bau93) (BS96). These methods tempt to identify at each iteration some small set ofnodes whose expansion would be most informative,based on detailed probabilistic distributions of theevaluations of the current leaf nodes. Though gen-eral and theoretically sound, use of these techniques isincompatible with alpha-beta cutoffs. This approachhas not yet been demonstrated to be competitive withalpha-beta variants in the domain of chess, but cer-tainly some reasonable approximations may prove use-ful in the future.

Korf and Chickering has proposed an alternativeselective search method called "best-first minimaxsearch" (KC94). Essentially, it always expands the leafof the current principal variation m i.e. the node whichdetermines the minmax value of the root. They arguethat it will not generally suffer from excessive explo-ration of a single path because of the well-known os-cillation of values that occurs during minimax search.

Best-first minimax search is relatively simple yetbeats fixed-depth alpha-beta search in the game ofOthello for comparable-sized trees, for medium depthalpha-beta trees. However, deep fixed-depth alpha-beta search beats it for comparable-sized trees. Thissuggests that its best use might be as a relatively shal-low probe search for method such as B*, although Korfand Chickering did not discuss nor explore that option.

Another promising area is that of generalizing thenotion of a transposition table so that each entry con-tains an equivalence class of positions instead of justone. Ginsberg’s work on partition search (Gin96) hasdeveloped this idea into the basis of a world-classbridge program. Although he has not developed repre-sentations for chess, he notes that this might providea more formal approach to the specialized though in-triguing "method of analogies" (AVAD75).

Knowledge

Knowledge-engineering has long played a role in com-petitive computer chess, particularly in the areas ofendgame table precompilation and manually craftedopening books. Considerable effort typically goes into

11

selecting features and weights for the evaluation func-tion. The requirement of such initial overhead to de-velop a competitive chess program has perhaps gonea long way towards discouraging most AI researchersfrom even attempting, and towards computer chess’sfall from grace. With the advent of mature search andmachine learning techniques, it seems likely that thiswill change in the near future -- particularly if theDeep Blue approach falls short.

Learning

Recent advances in machine learning offer perhaps thebest hope for significant near-term advances in com-puter chess. Recent surveys such as (Fur96) and JayScott’s active web site at (Sco97) are good summariesof a variety of approaches that have been proposed todate. Unfortunately, most of these approaches havenot been within the context of tuning a competitive-level chess program, making their immediate impactdifficult to assess.

For example, there has been a fair amount of workon learning chess evaluation functions, with the Neu-roChess work (Thr95) being perhaps one of the mostinteresting.

Fktrthermore, there has been some work in learningsearch control heuristics for game-playing, althoughthe more successful ones seem particularly well-suitedfor games with simplier features, such as Othello(MM94).

It seems fair to say that machine learning to datehas not had nearly the impact on computer chess thatit could. It seems logical to expect that this situationwill change as the impact of diminishing returns inchess search is better understood -- particularly whenresearch chess programs mature to the level that suchwalls are routinely hit.

Summary of Our Chester Program

In this section we briefly summarize our own researchon computer chess. Our search engine uses a variantof B* to conduct high-level selective search, MTD(f)to conduct the relatively shallow low-level probes, andextremely fast move generation and transposition ta-ble routines, so as to reach a competitive level re-quired for meaningful evaluation. We call this programChester (Chess Terminator), which reflects our seriousbut perhaps naive ambitions. The main novelty of ourapproach is our means of attempting to learn goodinterval-valued evaluation functions, suitable for B*,as opposed .to relying on heuristics such as null-movesto generate the bounds.

The emphasis of our research is on using chess asa domain to help test the generality of our machinelearning techniques being developed for real worldtasks such as monitoring the NASA Space Shuttle.(DeC97a) In reality, though, the causality of this lineof research is not straightforward, since our ideas for

learning functions suitable for monitoring tasks werein fact inspired by ideas from B* search.

We believe and hope that such tight synergy betweenchess and "real world" domains might make serious re-search on chess more easily justifiable and even encour-aged. In the spirit of this workshop, we wish to stressthat one of the significances of the GK vs DB matchesis the simple fact that there is renewed interested incomputer chess and thus it could become a useful fo-rum through which to educate the public about whatAI is about as a whole. This gives added urgency to de-velop more learning-intensive alternatives to the DeepBlue approach, so that we are better able compare andcontrast over spectra of approaches, rather than justthe human against machine theme per se.

Detailed discussion of our techniques are beyond thescope of this paper. However, we wish to outline ourbasic ideas here. In our monitoring work, we havedeveloped an approach called Envelope Learning andMonitoring using Error Relaxation (ELMER), whichessentially learns high and low limit functions for eachsensor, based on historic spacecraft data (DEC96).

Our techniques involve regression using asymmetriccost functions (DeC97b), similarly to the 0-1 loss func-tions common in classification tasks. However, the keydifference is that our cost function is parametric andwe search over those parameters and select the settingswhich give the best fit on cross-validation data.

In this way, we are able to learn envelopes whichdo not necessarily suffer from classic problems suchas variance underestimation (BQ97). In fact, our ap-proach is specifically designed to address cases whichare not handled well by the typical approach of learningconfidence intervals by first estimating means and thenestimating input-conditional variances (e.g. (NW95),(Bis95)). Thus, we can offer better assurance that bound estimates are not too tight, as required both forB* position evaluations and for spacecraft sensor limitsfor automated monitoring with few false alarms.

A simple example which illustrates the usefulnessof our approach is the task of learning high andlow bounds on the complexity of example run timesof an algorithm such as quicksort, given as inputonly the size N of each data set that was sorted.Consider that we wish to use linear regression (forspeed and for understandable results) and use the fol-lowing reasonable set of features for all such tasks:IgN, N, NlgN, N2, N3, 2N. With the high and lowbounds of quicksort actually being (O(N2) and O(N lgN) respectively, it turns out to be difficult to capturethese bounds using standard mean plus/minus stan-dard deviation approaches. Essentially, the problemis that there are critical missing inputs (namely, the"sortedness" of each data set to be sorted).

We believe that such an approach is likely to proveuseful for learning high and low bounds for chess posi-tion evaluation functions suitable for B* style selectivesearch. However, we stress that we have not yet had

12

time to try to develop convincing results to supportthis conjecture. Furthermore, we suspect that goodcontext-sensitive evaluators are likely to require goodfeature construction techniques, such as the greedy in-troduction of products that we are currently experi-menting with in our spacecraft domains (SSM92).

ReferencesT.S. Anantharaman, M.S. Campbell, and F. Hsu. Singularextensions: Adding selectivity to brute-force searching.Artificial lntelligence~ 43(1), 1990.G. Adelson-Velskiy, V. Arlazarov, and M. Donskoy. Somemethods of controlling the tree search in chess programs.Artificial Intelligence, 6, 1975.Eric B. Baum. On optimal game tree propagation forimperfect players. In Proceedings of the Eleventh NationalConference on Artificial Intelligence, pages 507-517, 1992.E. B. Banm. How a bayesian approachs a game like chess.In Games: Plannin 9 and Learning, AAAI Fall Sympo-sium, 1993.Christopher M. Bishop. Neural Networks for PatternRecognition. Oxford University Press, 1995.Hans J. Berliner and Chris McConnell. B* prob-ability based search. Artificial Intelligence, 86(1),1996. (See http:// www.cs.cmu.edu/ afs/cs.cmu.edu/user/ccm/www/home.html#B*).Christopher M. Bishop and Cazhaow S. Qazaz. Regressionwith input-dependent noise: A bayesian treatment. Ad-vances in Neural Information Processing Systems 9, 1997.E.B. Bantu and W.D. Smith. A bayesian approach togame playing. (See http:// www.neci.nj.nec.com/ home-pages/eric/ bayesl.ps), 1996.Dennis DeCoste. Learning and monitoring with families ofhigh/low envelope functions: Maximizing predictive pre-cision while minimizing false alarms. Technical Report D-13418, Jet Propulsion Laboratory / California Institute ofTechnology, March 1996.Dennis DeCoste. Automated learning and monitoring oflimit functions. In Proceedings of the Fourth InternationalSymposium on Artificial Intelligence, Robotics, and Au.tomation for Space, Japan, July 1997.Dennis DeCoste. Bounds estimation via regression withasymmetric cost functions. Japan, August 1997. To bepresented at IJCAI-97 poster session.Mikhail V. Donskoy and Jonathan Schaeffer. Perspectiveson falling from grace. In Computers, Chess, and Cogni-tion. Springer-Verlag, 1990.

Johannes Furnkranz. Machine learning in computerchess: The next generation. International ComputerChess Association Journal, 19(3), 1996. (See ftp://ftp.al.univie.ac.at/ papers/oefal-tr-96-11.ps.Z).

G. Goetsch and M. S. Campbell. Experiments with thenull-move heuristic. In Computers, Chess, and Cognition.Springer-Verlag, 1990.

Matthew L. Ginsberg. Partition search. In Proceedingsof the Thirteenth National Conference on Artificial Intel-ligence, pages 228-233, 1996.F. Hsu, T.S. Anantharaman, M.S. Campbell, andA. Nowatzyk. Deep thought. In Computers, Chess, andCognition. Springer-Verlag, 1990.

Andreas Junghanns, Jonathan Schaeffer, Mark Brocking-ton, Yngvi Bjornsson, and Tony Marsland. Diminishingreturns for additional search in chess. In Advances inComputer Chess VIII. (See http://www.cs.ualberta.ca/" jonathan/Papers/ dim.ps.Z).Garry Kasparov. The day that i sensed a new kind ofintelligence. In TIME Magazine. March 25, 1996.Richard E. Korf and David Maxwell Chickering. Best-firstminimax search: Othello results. In Proceedings of theThirteenth National Conference on Artificial Intelligence,pages 1365-1370, 1994.David E. Moriarty and Risto Miikkulalnen. Evolving neu-ral networks to focus minimax search. In Proceedings ofthe Thirteenth National Conference on Artificial Intelli-gence, pages 1371-1377, 1994.David A. Nix and Andreas S. Weigend. Learning localerror bars for nonlinear regression. Advances in NeuralInformation Processing Systems 7, 1995.Barney Pell. A strategic metagame player for generalchesslike games. In Proceedings of the Thirteenth Na-tional Conference on Artificial Intelligence, pages 1378-1385, 1994.Aske Platt. Mtd(f), a new chess algorithm. WWW page:http://theory.lcs.mit.edu/ " plaat/mtdf.html.Aske Plaat, Jonathan Schaeffer, Wire Pijls, and Ariede Bruin. Best-first fixed-depth game-tree search in prac-tice. In Proceedings of the Fourteenth International JointConference on Artificial Intelligence, pages 273-281, 1995.(See http://www.cs.ualberta.ca/ " aske/bfmms.ps).Aske Plaat, Jonathan Schaeffer, Wim Pijls, and Ariede Bruin. Best-first fixed-depth minimax algorithms. Arti-ficial Intelligence, 1996. (See http://www.cs.ualberta.ca/" aske/aij-final.ps.gz).Aske Plaat, Jonathan Schaeffer, Wire Pijls, and Ariede Bruin. Exploiting graph properties of game trees. InProceedings of the Fifthteen National Conference on Ar-tificial Intelligence, pages 234-239, 1996. (See http://www.cs.ualberta.ca/ " aske/ AAAI96-final.ps.gz).Bart Selman, Rodney Brooks, Thomas Dean, EricHorvitz~ Tom Mitchell, and Nils Nilsson. Challenge prob-lems for artificial intelligence. In Proceedings of the Fifth-teen National Conference on Artificial Intelligence, 1996.Roger Schank. Where’s the AI? In AI magazine. Winter1991.Jay Scott. Machine learning in games. WWW page:http:// forum.swarthmore.edu/ " jay/ learn-game/ in-dex.html, 1997.Terence D. Sanger, Richard S. Sutton, and Christopher J.Matheus. Iterative construction of sparse polynomial ap-proximations. Advances in Neural Information ProcessingSystems 4, 1992.

G.C. Stockman. A minimax algorithm better than alpha-beta? Artificial Intelligence, 12(2), 1979.

Sebastian Thrun. Learning to play the game of chess.In G. Tesanro, D. Touretzky, and T. Leen, editors, Ad-vances in Neural Information Processing Systems, vol-ume 7, pages 1069-1076. The MIT Press, 1995.Andreas S. Weigend and Neii A. Gershenfeld, editors.Time Series Prediction: Forcasting the Future and Un-derstanding the Past. Addison-Wesley, 1994.

13