diaconis, persi - the problem of thinking too much

8/2/2019 Diaconis, Persi - The Problem of Thinking Too Much

1/13

26 SPRING 2003

STATED MEET ING REPORT

The Problem of Thinking TooMuch

Persi Diaconis, Mary V. Sunseri Professorof Mathematics andStatistics, Stanford University

Introduction: Barry C. Mazur,Gerhard Gade University Professor,Harvard University

Barry C. Mazur

Persi Diaconis is a pal of mine. Hes also someonewho, by his work and interests, demonstrates theunity of intellectual lifethat you can have the

broadest range and still engage in the deepest proj-ects. Persi is a leading researcher in statistics, prob-ability theory, and Bayesian inference. Hes donewonderful work in pure math as well, most notablyin group representation theory. He has the gift ofbeing able to ask the simplest of questions. Those

are the questions that educate you about a subjectjust because theyre asked. And Persis research isalways illuminated by a story, as he calls itthat is,a thread that ties the pure intellectual question to awider world.

Persis world is indeed wide. It includes discoveringbeautiful connections among group-representationtheory, algebraic geometry, card-shuffling proce-dures, and Monte Carlo algorithms; studying ran-dom-number generators, both theoretical and verypractical; analyzing and interpreting real-world

applications of statistics, as in voting procedures;critiquing misrepresentations of science and mathe-matics, in particular the protocols of experiments

This presentation was given at the 1865th Stated Meeting,held at the House of the Academy on December 11, 2002.


2/13

SPRING 2003 27

regarding extrasensory perception; writing on thegeneral concept of coincidence; and working onhistorical treatises about probability and magic. Asis well known, Persi is also a magician, credited

with, as Martin Gardner once wrote, inventingand performing some of the best magic tricks ever.

As for honors, theres a long list. He was, forexample, one of the earliest recipients of theMacArthur Fellowship. Hes a member of the Na-

tional Academy of Sciences and was president ofthe Institute of Mathematical Statistics. On top ofall this, Persi has an exemplary gift for explainingthings, so I should let him do just that.

Persi Diaconis

Consider the predicament of a centipede whostarts thinking about which leg to move and windsup going nowhere. It is a familiar problem: Anyaction we take has so many unforeseen conse-

quences, how can we possibly choose?

Here is a less grand example: I dont like movingthe knives, forks, and spoons from dishwasher todrawer. There seems no sensible way to proceed. Ifrequently catch myself staring at the configura-

tion, hoping for insight. Should I take the tallestthings first, or just grab a handful and sort them at

Barry C. Mazur (Harvard University)


3/13

28 SPRING 2003

the drawer? Perhaps I should stop thinking and dowhat comes naturally. Before giving in to thinkingtoo little, I recall a friends suggestion: you can

speed things up by sorting the silverware as youput it into the dishwasher. On reflection, though,this might lead to nested spoons not getting clean.And so it goes.

Im not brazen enough to attempt a careful defini-

tion of thinking in the face of a reasonably well-posed problem. I would certainly include mentalcomputation (e.g., running scenarios, doing back-of-the-envelope calculations), gathering informa-tion (e.g., searching memory or the Web, callingfriends), searching for parallels (e.g., recognizing

that the problem seems roughly like another prob-lem one knows how to solve, or thinking of an eas-ier special case), and, finally, trying to maneuverones mind into places where one is in tune withthe problem and can have a leap of insight.

The problem is this: We can spend endless timethinking and wind up doing nothingor, worse,getting involved in the minutiae of a partiallybaked idea and believing that pursuing it is thesame as making progress on the original problem.

The study of what to do given limited resourceshas many tendrils. I will review work in econom-ics, psychology, search theory, computer science,and my own field, mathematical statistics. Thesearent of much help, but at the end I will note a fewrules of thumb that seem useful.

An Example

One of the most satisfying parts of the subjectiveapproach to statistics is Bruno de Finettis solutionof common inferential problems through exchange-

ability. Some of us think de Finetti has solvedHumes Problem: When is it reasonable to thinkthat the future will be like the past? I want to pres-ent the simplest example and show how thinkingtoo much can make a mess of something beautiful.

Consider observing repeated flips of a coin. Theoutcomes will be called heads (H) and tails (T). In


4/13

SPRING 2003 29

a subjective treatment of such problems, oneattempts to quantify prior knowledge into a proba-bility distribution for the outcomes. For example,your best guess that the next three tosses yieldHHT is the number P(HHT). In many situations,the order of the outcomes is judged irrelevant. ThenP(HHT) equals P(HTH) equals P(THH). Suchprobability assignments are called exchangeable.

Bruno de Finetti proved that an exchangeable prob-ability assignment for a long series of outcomes can

be represented as a mixture of coin tossing: For anysequence a, b, . . . , z of potential outcomes,

withA the number of heads and Bthe number oftails among a, b, . . . , z. The right side of this for-mula has been used since Thomas Bayes (1764) andPierre-Simon Laplace (1774) introduced Bayesian

statistics. Modern Bayesians callpA

(1p)B

the like-lihood and m the a priori probability. Subjectivistssuch as de Finetti, Ramsey, and Savage (as well asDiaconis) prefer not to speak about nonobservablethings such as p, the long-term frequency ofheads. They are willing to assign probabilities to

potentially observable things such as one head inthe next ten tosses. As de Finettis Theorem shows,

Speaker Persi Diaconis (Stanford University)


5/13

30 SPRING 2003

in the presence of exchangeability, the two formula-tions are equivalent.

The mathematical development goes further. Afterobserving A heads and B tails, predictions aboutfuture trials have the same type of representation,with the prior m replaced by a posterior distributiongiven by Bayess formula. Laplace and many follow-ers proved that as the number of trials increases, the

posterior distribution becomes tightly focused onthe observed proportion of headsthat is,A/(A+B)ifA heads and Btails are observed. Predictions ofthe future, then, essentially use this frequency; theprior m is washed away. Of course, with a smallnumber of trials, the prior m can matter. If the priorm is tightly focused, the number of trials required towash it away may be very large. The mathematicsmakes perfect sense of this; fifty trials are oftenenough. The whole package gives a natural, elegantaccount of proper inference. I will stick to flipping

coins, but all of this works for any inferential task,from factory inspection of defective parts to evalu-ation of a novel medical procedure.

Enter Physics

Our analysis of coin tossing thus far has made nocontact with the physical act of tossing a coin. Wenow put in a bit of physics and stir; I promise, amess will emerge. When a coin is flipped and leavesthe hand, it has a definite velocity in the upwarddirection and a rate of spin (revolutions per sec-

ond). If we know these parameters, Newtons Lawsallow us to calculate how long the coin will takebefore returning to its starting height and, thus,how many times it will turn over. If the coin iscaught without bouncing, we can predict whetherit will land heads or tails.

A neat analysis by Joe Keller appeared in a 1986issue of American Mathematical Monthly. Thesketch in Figure 1 shows the velocity/spin plane. Aflip of the coin is represented by a dot on the fig-ure, corresponding to the velocity and rate of spin.

For a dot far to the right and close to the axis, thevelocity is high, but spin is low. The coin goes up


6/13

SPRING 2003 31

like a pizza and doesnt turn over at all. All thepoints below the curve correspond to flips in whichthe coin doesnt turn over. The adjacent regioncontains points at which the coin turns over exactlyonce. It is bounded by a similar curve. Beyond this,

the coin turns over exactly twice, and so on.

As the figure shows, moving away from the origin,the curves get closer together. Thus, for vigorousflips, small changes in the initial conditions makefor the difference between heads and tails.

The question arises: When normal people flip realcoins, where are we on this picture? I became fasci-nated by this problem and have carried out a seriesof experiments. It is not hard to determine typicalvelocity. Get a friend with a stopwatch, practice a

bit, and time how long the coin takes in its rise andfall. A typical one-foot toss takes about half a sec-ond (this corresponds to an upward velocity ofabout 512 miles per hour). Determining rate of spinis trickier. I got a tunable strobe, painted the coinblack on one side and white on the other, and

tuned the strobe until the coin froze, showingonly white. All of this took many hours. The coinnever perfectly froze, and there was variation fromflip to flip. In the course of experimenting, I had agood idea. I tied a strand of dental floss about threefeet long to the coin. This was flattened, the coin

flipped, the flip timed, and then we unwrapped thefloss to see how often the coin had turned over. On

Figure 1. Partition of phase space induced by heads and tails


7/13

32 SPRING 2003

the basis of these experiments, we determined thata typical coin turns at a rate of 35 to 40 revolutionsper second (rps). A flip lasts half a second, so a

flipped coin rotates between 17 and 20 times.

There is not very much variability in coin flips,and practiced magicians (including myself) cancontrol them pretty precisely. My colleagues at theHarvard Physics Department built me a perfect

coin flipper that comes up heads every time. Mosthuman flippers do not have this kind of controland are in the range of 512 mph and 35 to 40 rps.Where is this on Figure 1? In the units of Figure 1,the velocity is about 15very close to the zero.However, the spin coordinate is about 40way off

the graph. Thus, the picture says nothing aboutreal flips. However, the math behind the picturedetermines how close the regions are in the appro-priate zone. Using this and the observed spread ofthe measured data allows us to conclude that cointossing is fair to two decimals but not to three.

That is, typical flips show biases such as .495or .503.

Blending Subjective Probability and Physics

Our refined analysis can be blended into the prob-

ability specification. Now, instead of observingheads and tails at each flip, we observe velocity/spinpairs. If these are judged exchangeable, a version ofde Finettis Theorem applies to show that anycoherent probability assignment must be a mixtureof independent and identically distributed assign-ments:

The meaning of these symbols is slightly frighten-ing, even to a mathematical grownup. On the right,F is a probability distribution on the velocity/spinplane. Thus m is a probability on the space of allprobabilities. Here, de Finettis Theorem tells us

that thinking about successive flips is the same asthinking about measures for measures. There is a


8/13

SPRING 2003 33

set of tools for doing this, but at the present stateof development it is a difficult task. It is evendangerous. The space of all probability measures

is infinite-dimensional. Our finite-dimensionalintuitions break down, and hardened profession-als have suggested prior distributions with thefollowing property: as more and more data comein, we become surer and surer of the wronganswer.

This occurs in the age-old problem of estimatingthe size of an object based on a series of repeatedmeasurements. Classically, everyone uses the aver-age. This is based on assuming that the measure-ment errors follow the bell-shaped curve. Owning

up to not knowing the distribution of the errors,some statisticians put a prior distribution on thisunknown distribution. The corresponding poste-rior distribution can become more and moretightly peaked about the wrong answer as moreand more data come in. A survey of these prob-lems and available remedies can be found in myjoint work with David Freedman in theAnnals ofStatistics(1986).

Whats the Point?

This has been a lengthy example aimed at makingthe following point. Starting with the simple prob-lem of predicting binary outcomes and then think-ing about the underlying physics and dynamics, we

Mitchell Rabkin (Beth Israel Deaconess Medical Center), Ruth Rabkin,Leon Eisenberg (Harvard Medical School), and Carola Eisenberg


9/13

34 SPRING 2003

were led from de Finettis original, satisfactorysolution to talking close to nonsense. The analysisled to introspection about opinions on which wehave small hold and to a focus on technical issuesfar from the original problem. I hope the details ofthe example do not obscure what I regard as its

nearly universal quality. In every area of academicand more practical study, we can find simple exam-ples that on introspection grow into unspeakablecreatures. The technical details take over, andpractitioners are fooled into thinking they are doingserious work. Contact with the original problem is

lost.

I am really troubled by the coin-tossing example. Itshouldnt be that thinking carefully about a prob-lem and adding carefully collected outside data,Newtonian mechanics, and some detailed calcula-

tions should make a mess of things.

Thinking About Thinking Too Much

The problem of thinking too much has a promi-nent place in the age-old debate between theory

and practice. Galens second-century attempts tobalance between rationalist and empiricist physi-cians ring true today. In his Three Treatises on theNature of Science(trans. R. Walzer and M. Frode),Galen noted that an opponent of the new theoriesclaimed there was a simple way in which mankind

actually had made enormous progress in medicine.Over the ages men had learned from dire experi-

Robert Alberty and Stephen Crandall (both, MIT) with George

Hatsopoulos (Pharos L.L.C., Waltham, MA)


10/13

SPRING 2003 35

ence, by trial and error, what was conducive andwhat was detrimental to health. Not only did heclaim that one should not abandon this simple

method in favor of fanciful philosophical theories,which do not lead anywhere; he also argued thatgood doctors in practice relied on this experienceanyway, since their theories were too vague and toogeneral to guide their practice. In my own field ofstatistics, the rationalists are called decision theo-

rists and the empiricists are called exploratory dataanalysts. The modern debaters make many of thesame rhetorical moves that Galen chronicled.

Economists use Herbert Simons ideas of satisfic-ing and bounded rationality, along with more

theoretical tools associated with John Harsanyisvalue of information. Psychologists such asDaniel Kahneman and Amos Tversky accept thevalue of the heuristics that we use when we aban-don calculation and go with our gut. They havecreated theories of framing and support that allow

adjustment for the inevitable biases. These give aframework for balancing the decision to keep think-ing versus getting on with deciding.

Computer science explicitly recognizes the limitsof thinking through ideas like complexity theory.

For some tasks, computationally feasible algo-rithms can be proved to do reasonably well. Here isa simple example. Suppose you want to pack twosuitcases with objects of weight a, b, . . . , z. You

Daniel Bell (Harvard University) and Martin Cohn (MIT)


11/13

36 SPRING 2003

want to pack them as close to evenly as you can. Itcan be shown that this is a virtually impossibleproblem. Despite fifty years of effort, we dont

know how to find the best method of packing, savefor trying all of the exponentially many possibili-ties. Any progress would give solution to thousandsof other intractable problems. Most of us concludethat the optimal solution is impossible to find.

Undeterred, my friend Ron Graham proposed thefollowing: sort the objects from heaviest to lightest(this is quick to do). Then fill the two suitcases bybeginning with the heaviest item, and each timeplacing the next thing into the lighter suitcase.Here is an example with five things of weight 3, 3,

2, 2, 2. The algorithm builds up two groups as fol-lows:

This misses the perfect solution, which puts 3, 3 inone pile and 2, 2, 2 in the other. One measure ofthe goodness of a proposed solution is the ratio ofthe size of the larger pile to the size of the larger

pile in the optimal solution. This is 7/6 in theexample. Graham proved that in any problem, nomatter what the size of the numbers, this greedyheuristic always does at worst 7/6 compared to theoptimal. We would be lucky to do as well in morerealistic problems.

An agglomeration of economics, psychology, deci-sion theory, and a bit of complexity theory is thecurrent dominant paradigm. It advises roughlyquantifying our uncertainty, costs, and benefits(utility) and then choosing the course that maxi-

mizes expected utility per unit of time. A livelyaccount can be found in I. J. Goods book GoodThinking (dont miss his essay on How RationalShould a Manager Be?).

To be honest, the academic discussion doesnt shed

much light on the practical problem. Heres anillustration: Some years ago I was trying to decide


12/13

SPRING 2003 37

whether or not to move to Harvard from Stanford.

I had bored my friends silly with endless discus-sion. Finally, one of them said, Youre one of ourleading decision theorists. Maybe you should makea list of the costs and benefits and try to roughlycalculate your expected utility. Without thinking,I blurted out, Come on, Sandy, this is serious.

Some Rules of Thumb

One of the most useful things to come out of mystudy is a collection of the rules of thumb myfriends use in their decision making. For example,

one of my Ph.D. advisers, Fred Mosteller, told me,Other things being equal, finish the job that isnearest done. A famous physicist offered thisadvice: Dont waste time on obscure fine pointsthat rarely occur. Ive been told that AlbertEinstein displayed the following aphorism in his

office: Things that are difficult to do are beingdone from the wrong centers and are not worthdoing. Decision theorist I. J. Good writes, Theolder we become, the more important it is to usewhat we know rather than learn more. Galenoffered this: If a lot of smart people have thoughtabout a problem [e.g., Gods existence, life on otherplanets] and disagree, then it cant be decided.

There are many ways we avoid thinking. Ive oftenbeen offered the algorithm Ask your wife todecide (but never Ask your husband). One of

my most endearing memories of the great psychol-ogist of decision making under uncertainty, Amos

Helen Pounds,William F. Pounds (MIT), and Paul Doty (Harvard University)


13/13

38 SPRING 2003

Tversky, recalls his way of ordering in restaurants:Barbara? What do I want?

Clearly, we have a wealth of experience, gatheredover millennia, coded into our gut responses. Surely,we all hope to call on this. A rule of thumb in thisdirection is Trust your gut reaction when dealingwith natural tasks such as raising children.

Its a fascinating insight into the problem of think-ing too much that these rules of thumb seem moreuseful than the conclusions drawn from more the-oretical attacks.

In retrospect, I think I should have followed myfriends advice and made a list of costs and bene-fitsif only so that I could tap into what I wasreally after, along the lines of the following grookby Piet Hein:

A Psychological Tip

Whenever youre called on to make up your mind,and youre hampered by not having any,

the best way to solve the dilemma, youll find,is simply by spinning a penny.

Nonot so that chance shall decide the affairwhile youre passively standing there moping;

but the moment the penny is up in the air,you suddenly know what youre hoping.

Remarks 2002 by Barry C. Mazur and Persi Diaconis,respectively.

Photos 2002 by Martha Stewart.

A Psychological Tip is from an English-language edi-

tion ofGrooksby Piet Hein, published in Copenhagenby Borgens Forlag (1982, p. 38); Piet Hein.

diaconis, persi - the problem of thinking too much

Documents