Download - Alan Turing's Forgotten Ideas

Alan Turing, at age 35, about the timehe wrote “Intelligent Machinery”

Copyright 1998 Scientific American, Inc.

Alan Turing’s Forgotten Ideas in Computer Science Scientific American April 1999 99

Alan Mathison Turing conceived of the moderncomputer in 1935. Today all digital comput-ers are, in essence, “Turing machines.” The

British mathematician also pioneered the field ofartificial intelligence, or AI, proposing the famousand widely debated Turing test as a way of determin-ing whether a suitably programmed computer canthink. During World War II, Turing was instrumentalin breaking the German Enigma code in part of atop-secret British operation that historians say short-ened the war in Europe by two years. When he diedat the age of 41, Turing was doing the earliest workon what would now be called artificial life, simulat-ing the chemistry of biological growth.

Throughout his remarkable career, Turing had nogreat interest in publicizing his ideas. Consequently,important aspects of his work have been neglected orforgotten over the years. In particular, few people—even those knowledgeable about computer science—are familiar with Turing’s fascinating anticipation ofconnectionism, or neuronlike computing. Also ne-glected are his groundbreaking theoretical conceptsin the exciting area of “hypercomputation.” Accord-ing to some experts, hypercomputers might one daysolve problems heretofore deemed intractable.

The Turing Connection

Digital computers are superb number crunchers.Ask them to predict a rocket’s trajectory or calcu-

late the financial figures for a large multinational cor-poration, and they can churn out the answers in sec-onds. But seemingly simple actions that people routine-ly perform, such as recognizing a face or readinghandwriting, have been devilishy tricky to program.Perhaps the networks of neurons that make up thebrain have a natural facility for such tasks that standardcomputers lack. Scientists have thus been investigatingcomputers modeled more closely on the human brain.

Connectionism is the emerging science of computingwith networks of artificial neurons. Currently research-ers usually simulate the neurons and their interconnec-tions within an ordinary digital computer (just as engi-neers create virtual models of aircraft wings andskyscrapers). A training algorithm that runs on thecomputer adjusts the connections between the neurons,honing the network into a special-purpose machinededicated to some particular function, such as forecast-ing international currency markets.

Modern connectionists look back to Frank Rosen-blatt, who published the first of many papers on thetopic in 1957, as the founder of their approach. Few re-alize that Turing had already investigated connectionistnetworks as early as 1948, in a little-known paper enti-tled “Intelligent Machinery.”

Written while Turing was working for the NationalPhysical Laboratory in London, the manuscript did notmeet with his employer’s approval. Sir Charles Darwin,the rather headmasterly director of the laboratory andgrandson of the great English naturalist, dismissed it asa “schoolboy essay.” In reality, this farsighted paperwas the first manifesto of the field of artificial intelli-

Well known for the machine, test and thesis that bear his name, the British genius also anticipated

neural-network computers and “hypercomputation”

by B. Jack Copeland and Diane Proudfoot

Alan Turing'sFFoorrggootttteenn

IIddeeaassin

Computer Science

W. H

EFFE

R A

ND

SO

NS

LTD

., C

AM

BRI

DG

E, E

NG

LAN

D


gence. In the work—which remained un-published until 1968, 14 years after Tur-ing’s death—the British mathematiciannot only set out the fundamentals of con-nectionism but also brilliantly introducedmany of the concepts that were later tobecome central to AI, in some cases afterreinvention by others.

In the paper, Turing invented a kind ofneural network that he called a “B-type

unorganized machine,” which consists ofartificial neurons and devices that modifythe connections between them. B-typemachines may contain any number ofneurons connected in any pattern but arealways subject to the restriction that eachneuron-to-neuron connection must passthrough a modifier device.

All connection modifiers have twotraining fibers. Applying a pulse to oneof them sets the modifier to “passmode,” in which an input—either 0 or1—passes through unchanged and be-comes the output. A pulse on the otherfiber places the modifier in “interruptmode,” in which the output is always1, no matter what the input is. In thisstate the modifier destroys all informa-tion attempting to pass along the con-nection to which it is attached.

Once set, a modifier will maintain itsfunction (either “pass” or “interrupt”)unless it receives a pulse on the othertraining fiber. The presence of these inge-nious connection modifiers enables thetraining of a B-type unorganized ma-chine by means of what Turing called“appropriate interference, mimickingeducation.” Actually, Turing theorizedthat “the cortex of an infant is an unor-ganized machine, which can be orga-nized by suitable interfering training.”

Each of Turing’s model neurons hastwo input fibers, and the output of aneuron is a simple logical function of itstwo inputs. Every neuron in the net-work executes the same logical opera-tion of “not and” (or NAND): the out-put is 1 if either of the inputs is 0. Ifboth inputs are 1, then the output is 0.

Turing selected NAND because everyother logical (or Boolean) operation can

be accomplished by groups of NANDneurons. Furthermore, he showed thateven the connection modifiers themselvescan be built out of NAND neurons.Thus, Turing specified a network madeup of nothing more than NAND neu-rons and their connecting fibers—aboutthe simplest possible model of the cortex.

In 1958 Rosenblatt defined the theo-retical basis of connectionism in one suc-

cinct statement: “Storedinformation takes theform of new connections,or transmission channelsin the nervous system (orthe creation of conditionswhich are functionallyequivalent to new connec-tions).” Because the de-struction of existing con-nections can be func-

tionally equivalent to the creation of newones, researchers can build a networkfor accomplishing a specific task by tak-ing one with an excess of connectionsand selectively destroying some of them.Both actions—destruction and creation—are employed in the training of Turing’sB-types.

At the outset, B-types contain randominterneural connections whose modifiershave been set by chance to either pass orinterrupt. During training, unwantedconnections are destroyed by switchingtheir attached modifiers to interruptmode. Conversely, changing a modifierfrom interrupt to pass in effect creates aconnection. This selective culling and en-livening of connections hones the initiallyrandom network into one organized fora given job.

Turing wished to investigate otherkinds of unorganized machines, and helonged to simulate a neural network andits training regimen using an ordinarydigital computer. He would, he said, “al-low the whole system to run for an ap-preciable period, and then break in as akind of ‘inspector of schools’ and seewhat progress had been made.” But hisown work on neural networks was car-ried out shortly before the first general-purpose electronic computers becameavailable. (It was not until 1954, the yearof Turing’s death, that Belmont G. Farleyand Wesley A. Clark succeeded at theMassachusetts Institute of Technology inrunning the first computer simulation ofa small neural network.)

Paper and pencil were enough, though,for Turing to show that a sufficientlylarge B-type neural network can beconfigured (via its connection modifiers)

in such a way that it becomes a general-purpose computer. This discovery illumi-nates one of the most fundamental prob-lems concerning human cognition.

From a top-down perspective, cogni-tion includes complex sequential process-es, often involving language or otherforms of symbolic representation, as inmathematical calculation. Yet from abottom-up view, cognition is nothing butthe simple firings of neurons. Cognitivescientists face the problem of how to rec-oncile these very different perspectives.

Turing’s discovery offers a possible so-lution: the cortex, by virtue of being aneural network acting as a general-pur-pose computer, is able to carry out the se-quential, symbol-rich processing dis-cerned in the view from the top. In 1948this hypothesis was well ahead of itstime, and today it remains among thebest guesses concerning one of cognitivescience’s hardest problems.

Computing the Uncomputable

In 1935 Turing thought up the ab-stract device that has since become

known as the “universal Turing ma-chine.” It consists of a limitless memory

100 Scientific American April 1999

Few realize that Turinghad already investigated

connectionist networks as early as 1948.

In a paper that went unpublisheduntil 14 years after his death (top),

Alan Turing described a network ofartificial neurons connected in a ran-dom manner. In this “B-type unorga-nized machine” (bottom left), eachconnection passes through a modifi-er that is set either to allow data topass unchanged (green fiber) or to de-stroy the transmitted information (redfiber). Switching the modifiers fromone mode to the other enables thenetwork to be trained. Note that eachneuron has two inputs (bottom left, in-set) and executes the simple logicaloperation of “not and,” or NAND: ifboth inputs are 1, then the output is0; otherwise the output is 1.

In Turing’s network the neurons in-terconnect freely. In contrast, modernnetworks (bottom center) restrict theflow of information from layer to layerof neurons. Connectionists aim tosimulate the neural networks of thebrain (bottom right).

Turing’s Anticipationof Connectionism

Alan Turing’s Forgotten Ideas in Computer Science


that stores both program and data anda scanner that moves back and forththrough the memory, symbol by sym-bol, reading the information and writ-ing additional symbols. Each of the ma-chine’s basic actions is very simple—such as “identify the symbol on whichthe scanner is positioned,” “write ‘1’”and “move one position to the left.”Complexity is achieved by chaining to-gether large numbers of these basic ac-tions. Despite its simplicity, a universalTuring machine can execute any taskthat can be done by the most powerfulof today’s computers. In fact, all mod-ern digital computers are in essenceuniversal Turing machines [see “TuringMachines,” by John E. Hopcroft; Sci-

entific American, May 1984].Turing’s aim in 1935 was to devise a

machine—one as simple as possible—capable of any calculation that a humanmathematician working in accordancewith some algorithmic method couldperform, given unlimited time, energy,paper and pencils, and perfect concen-tration. Calling a machine “universal”merely signifies that it is capable of allsuch calculations. As Turing himselfwrote, “Electronic computers are in-

tended to carry out any definite rule-of-thumb process which could have beendone by a human operator working in adisciplined but unintelligent manner.”

Such powerful computing devicesnotwithstanding, an intriguing questionarises: Can machines be devised that arecapable of accomplishing even more?The answer is that these “hyperma-chines” can be described on paper, butno one as yet knows whether it will bepossible to build one. The field of hyper-computation is currently attracting agrowing number of scientists. Somespeculate that the human brain itself—the most complex information proces-sor known—is actually a naturally oc-curring example of a hypercomputer.

Before the recent surge of interest inhypercomputation, any information-processing job that was known to betoo difficult for universal Turing ma-chines was written off as “uncom-putable.” In this sense, a hypermachinecomputes the uncomputable.

Examples of such tasks can be foundin even the most straightforward areasof mathematics. For instance, givenarithmetical statements picked at ran-dom, a universal Turing machine may

not always be able to tell which are the-orems (such as “7 + 5 = 12”) and whichare nontheorems (such as “every num-ber is the sum of two even numbers”).Another type of uncomputable problemcomes from geometry. A set of tiles—variously sized squares with differentcolored edges—“tiles the plane” if theEuclidean plane can be covered bycopies of the tiles with no gaps or over-laps and with adjacent edges always thesame color. Logicians William Hanf andDale Myers of the University of Hawaiihave discovered a tile set that tiles theplane only in patterns too complicatedfor a universal Turing machine to calcu-late. In the field of computer science, auniversal Turing machine cannot alwayspredict whether a given program willterminate or continue running forever.This is sometimes expressed by sayingthat no general-purpose programminglanguage (Pascal, BASIC, Prolog, C andso on) can have a foolproof crash de-bugger: a tool that detects all bugs thatcould lead to crashes, including errorsthat result in infinite processing loops.

Turing himself was the first to investi-gate the idea of machines that can per-form mathematical tasks too difficult


TOM

MO

ORE

(illu

stra

tions

); KI

NG

’S C

OLL

EGE

MO

DER

N A

RC

HIV

ES, C

AM

BRI

DG

E U

NIV

ERSI

TY L

IBR

ARY

(top

); PE

TER

ARN

OLD

, IN

C. (

bott

om ri

ght)


for universal Turing machines. In his1938 doctoral thesis at Princeton Uni-versity, he described “a new kind of ma-chine,” the “O-machine.”

An O-machine is the result of aug-menting a universal Turing machinewith a black box, or “oracle,” that is amechanism for carrying out uncom-putable tasks. In other respects, O-ma-chines are similar to ordinary com-puters. A digitally encoded program is

fed in, and the machine produces digitaloutput from the input using a step-by-step procedure of repeated applicationsof the machine’s basic operations, oneof which is to pass data to the oracleand register its response.

Turing gave no indication of how anoracle might work. (Neither did he ex-plain in his earlier research how the ba-sic actions of a universal Turing ma-

chine—for example, “identify the sym-bol in the scanner”—might take place.)But notional mechanisms that fulfill thespecifications of an O-machine’s blackbox are not difficult to imagine [see boxabove]. In principle, even a suitable B-type network can compute the uncom-putable, provided the activity of the neu-rons is desynchronized. (When a centralclock keeps the neurons in step with oneanother, the functioning of the network

can be exactly simulat-ed by a universal Turingmachine.)

In the exotic mathe-matical theory of hyper-computation, tasks suchas that of distinguishingtheorems from nonthe-orems in arithmetic areno longer uncomput-able. Even a debugger

that can tell whether any program writ-ten in C, for example, will enter aninfinite loop is theoretically possible.

If hypercomputers can be built—andthat is a big if—the potential for crack-ing logical and mathematical problemshitherto deemed intractable will beenormous. Indeed, computer sciencemay be approaching one of its most sig-nificant advances since researchers

wired together the first electronic em-bodiment of a universal Turing machinedecades ago. On the other hand, workon hypercomputers may simply fizzleout for want of some way of realizingan oracle.

The search for suitable physical,chemical or biological phenomena isgetting under way. Perhaps the answerwill be complex molecules or otherstructures that link together in patternsas complicated as those discovered byHanf and Myers. Or, as suggested byJon Doyle of M.I.T., there may be natu-rally occurring equilibrating systemswith discrete spectra that can be seen ascarrying out, in principle, an uncom-putable task, producing appropriateoutput (1 or 0, for example) after beingbombarded with input.

Outside the confines of mathematicallogic, Turing’s O-machines have largelybeen forgotten, and instead a myth hastaken hold. According to this apoc-ryphal account, Turing demonstrated inthe mid-1930s that hypermachines areimpossible. He and Alonzo Church, thelogician who was Turing’s doctoral ad-viser at Princeton, are mistakenly credit-ed with having enunciated a principle tothe effect that a universal Turing ma-chine can exactly simulate the behavior

Alan Turing’s Forgotten Ideas in Computer Science102 Scientific American April 1999

Alan Turing proved that his universal machine—and by ex-tension, even today’s most powerful computers—could

never solve certain problems. For instance, a universal Turingmachine cannot always determine whether a given softwareprogram will terminate or continue running forever. In somecases, the best the universal machine can do is execute theprogram and wait—maybe eternally—for it to finish. But in hisdoctoral thesis (below), Turing did imagine that a machineequipped with a special “oracle” could perform this and other“uncomputable” tasks. Here is one example of how, in princi-ple, an oracle might work.

Consider a hypothetical machine for solving the formidable “terminating program” problem (above). A computer pro-gram can be represented as a finite string of 1s and 0s. Thissequence of digits can also be thought of as the binary rep-resentation of an integer, just as 1011011 is the equivalentof 91. The oracle’s job can then be restated as, “Given an in-teger that represents a program (for any computer that canbe simulated by a universal Turing machine), output a ‘1’ ifthe program will terminate or a ‘0’ otherwise.”

The oracle consists of a perfect measuring device and astore, or memory, that contains a precise value—call it τ forTuring—of some physical quantity. (The memory might, forexample, resemble a capacitor storing an exact amount of

Using an Oracle to Computethe Uncomputable

Even among experts, Turing’spioneering theoretical

concept of a hypermachinehas largely been forgotten.

100001. . .001111

BINARY REPRESENTATION OF PROGRAM

COMPUTER PROGRAM

EXCERPT FROM TURING’S THESIS

PRIN

CET

ON

AR

CH

IVES


of any other information-processing ma-chine. This proposition, widely but in-correctly known as the Church-Turingthesis, implies that no machine can carryout an information-processing task thatlies beyond the scope of a universal Tur-ing machine. In truth, Church and Tur-ing claimed only that a universal Turingmachine can match the behavior of anyhuman mathematician working withpaper and pencil in accordance withan algorithmic method—a considerably

weaker claim that certainly does not ruleout the possibility of hypermachines.

Even among those who are pursuingthe goal of building hypercomputers,Turing’s pioneering theoretical contribu-tions have been overlooked. Expertsroutinely talk of carrying out informa-tion processing “beyond the Turing lim-it” and describe themselves as attempt-ing to “break the Turing barrier.” A re-cent review in New Scientist of thisemerging field states that the new ma-

chines “fall outside Turing’s concep-tion” and are “computers of a type nev-er envisioned by Turing,” as if theBritish genius had not conceived of suchdevices more than half a century ago.Sadly, it appears that what has alreadyoccurred with respect to Turing’s ideason connectionism is starting to happenall over again.

The Final Years

In the early 1950s, during the lastyears of his life, Turing pioneered the

field of artificial life. He was trying tosimulate a chemical mechanism bywhich the genes of a fertilized egg cellmay determine the anatomical structureof the resulting animal or plant. He de-scribed this research as “not altogetherunconnected” to his study of neural net-works, because “brain structure has tobe . . . achieved by the genetical embry-ological mechanism, and this theorythat I am now working on may makeclearer what restrictions this really im-plies.” During this period, Turingachieved the distinction of being the firstto engage in the computer-assisted ex-ploration of nonlinear dynamical sys-tems. His theory used nonlinear differ-ential equations to express the chem-istry of growth.

But in the middle of this groundbreak-ing investigation, Turing died fromcyanide poisoning, possibly by his ownhand. On June 8, 1954, shortly beforewhat would have been his 42nd birth-day, he was found dead in his bedroom.He had left a large pile of handwrittennotes and some computer programs.Decades later this fascinating material isstill not fully understood.


S

The Authors

B. JACK COPELAND and DIANE PROUDFOOT are the di-rectors of the Turing Project at the University of Canterbury, NewZealand, which aims to develop and apply Turing’s ideas usingmodern techniques. The authors are professors in the philosophydepartment at Canterbury, and Copeland is visiting professor ofcomputer science at the University of Portsmouth in England.They have written numerous articles on Turing. Copeland’s Tur-ing’s Machines and The Essential Turing are forthcoming fromOxford University Press, and his Artificial Intelligence was pub-lished by Blackwell in 1993. In addition to the logical study of hy-permachines and the simulation of B-type neural networks, theauthors are investigating the computer models of biologicalgrowth that Turing was working on at the time of his death. Theyare organizing a conference in London in May 2000 to celebratethe 50th anniversary of the pilot model of the Automatic Comput-ing Engine, an electronic computer designed primarily by Turing.

Further Reading

X-Machines and the Halting Problem: Building a Super-Turing

Machine. Mike Stannett in Formal Aspects of Computing, Vol. 2,pages 331–341; 1990.

Intelligent Machinery. Alan Turing in Collected Works of A. M.Turing: Mechanical Intelligence. Edited by D. C. Ince. Elsevier SciencePublishers, 1992.

Computation beyond the Turing Limit. Hava T. Siegelmann in Sci-ence, Vol. 268, pages 545–548; April 28, 1995.

On Alan Turing’s Anticipation of Connectionism. B. JackCopeland and Diane Proudfoot in Synthese, Vol. 108, No. 3, pages361–377; March 1996.

Turing’s O-Machines, Searle, Penrose and the Brain. B. JackCopeland in Analysis, Vol. 58, No. 2, pages 128–138; 1998.

The Church-Turing Thesis. B. Jack Copeland in The Stanford Encyclo-pedia of Philosophy. Edited by Edward N. Zalta. Stanford University, ISSN1095-5054. Available at http://plato.stanford.edu on the World Wide Web.

TOM

MO

ORE

8,735,439

DECIMAL EQUIVALENT OF BINARY NUMBER

ORACLE

0

ORACLE’S MEMORY WITH τ = 0.00000001101. . .

PROGRAMWILL NOT

TERMINATE

electricity.) The value of τ is an irrational number; its written representation wouldbe an infinite string of binary digits, such as 0.00000001101. . .

The crucial property of τ is that its individual digits happen to represent accu-rately which programs terminate and which do not. So, for instance, if the integerrepresenting a program were 8,735,439, then the oracle could by measurementobtain the 8,735,439th digit of τ (counting from left to right after the decimalpoint). If that digit were 0, the oracle would conclude that the program will processforever.

Obviously, without τ the oracle would be useless, and finding some physical vari-able in nature that takes this exact value might very well be impossible. So the searchis on for some practicable way of implementing an oracle. If such a means were found,the impact on the field of computer science could be enormous. —B.J.C. and D.P.


Download - Alan Turing's Forgotten Ideas

Top Related