conscientious software

7/28/2019 Conscientious Software

1/18

Permission to make dig ital or hard copies o all or pa rt o this work or per-

sonal or classroom use is granted without ee provided that copies are notmade or distributed or proft or commercial advantage and that copies bear

this notice and the ull citation on the frst page. To copy otherwise, or re-

publish , to post on ser vers or to redist ribute to lists , requi res prio r speci fc

permission and/or a ee.

OOPSLA06October 2226, 2006, Portland, Oregon, USA.

Copyright 2006 ACM 1-59593-348-4/06/0010$5.00.

Abstract

Sotware needs to grow up and become responsible or itseland its own uture by participating in its own installation andcustomization, maintaining its own health, and adapting itselto new circumstances, new users, and new uses. To create suchsotware will require us to change some o our underlying as-sumptions about how we write programs. A promising approach

seems to be to separate sotware that does the work (allopoietic)rom sotware that keeps the system alive (autopoietic).

Categories and Subject Descriptors D.2.5. [Sotware Engineer-ing] Testing and DebuggingError handling and recovery; D.2.11.[Sotware Engineering] Sotware Architectures; D.3.0. [Program-ming Languages] General; H.1.2. [Models and Principles] User/Machine SystemsHuman actors

General Terms Design, Human Factors, Languages, Reliability

Keywords Robustness, sotware, sel-sustaining systems, emer-gence, autopoiesis, stigmergy, continuous (re)design, sel-testing,eedback, sotware complexity, repair

1. Introduction

Sotware systems today are produced according to a manuac-turing model: A nished product is constructed at the actory

and shipped to its nal destination where it is expected to actlike any other machinereliable but oblivious to its surround-ings and its own welare. Responsibility or testing and devisinghow to install the sotware rests with the development team.Once deployed, sotware is on its own, along with the people whomust use it. The result o this way o doing development has beenbrittle, buggy sotware where the only recourse or end-users isto hope the next release will x the problems (and not add toomany new ones).

We believe that sotware needs to grow up and become re-sponsible or itsel and its own uture. Moreover, the people us-ing the sotware every day must be able to shape and customizeit without reliance on the sotwares original developers. In this

Richard P. Gabriel

Sun Microsystems Laboratories

Menlo Park, California, USA

[email protected]

Ron Goldman

Sun Microsystems Laboratories

Menlo Park, California, USA

[email protected]

Conscientious Software

paper we will argue that uture innovations in sotware will needto produce systems that actively monitor their own activity andtheir environment, that continually perorm sel-testing, thatcatch errors and automatically recover rom them, that automati-cally congure themselves during installation, that participatein their own development and customization, and that protectthemselves rom damage when patches and updates are installed.Such sotware systems will be sel-contained, including within

themselves their entire source code, code or testing, and any-thing else needed or their evolution.

Furthermore, we need to make the state o each componentmore visible and make our components interact more richly &more ully. We need to use soter, more dynamic architecturesthat support adding or replacing modules ater deployment (e.g.plug-ins, dynamic loading o classes) and architectures whereobjects can be repaired in situ, methods changed / added, inter-nal state restructured, and object hierarchies rewired). We alsoneed new types o languages to describe the architecture o oursystems. We will not program in these new languagesthey

probably wont have arithmetic operators, data structures, or low-level control statementsbut instead express things like when tocreate and kill components, where components are run, when to

run tests, and which components interact with each other.As our sotware becomes ever more interdependent, with appli-

cations relying on (remote) services developed by other organiza-tions, it also must paradoxically become more independent, ableto maintain its integrity in a changing environment. Interacesbetween components need to become less brittle and more ac-commodatingnot requiring an exact match like a key in a lock,but based more on pattern recognition such as phenotropics [20].We also need to better isolate components rom each otherinbiology this is reerred to as spatial compartmentalization [19],or computing it may translate into components not sharingcommon memory or other resources.

Recent applications are beginning to exhibit these proper-ties and this trend will increase, refecting the need or sotware

systems to change continuously over their lietimes. Many o theideas we present in this paper are based on the changes we seeoccurring in current sotware. We use the term conscientioussotware to describe code that takes responsibility or itsel andits uture. The ollowing is our vision or the uture.

2. SoftwareComplexity

Over the last orty years we have attempted to create ever moreambitious systems, but their complexity oten exceeds our abili-ties as witnessed by how brittle most sotware is and how many


2/18

sotware projects ail. While our current methods have many goodcharacteristics they do not successully address the complexityo modern sotware systems. We need to adopt new principlesand approaches i we are to progress.

Sotware complexity arises in large part rom the inherentcomplexity o the problems we are trying to solve. We lack goodmodels to describe the organization o complex systems. We know

how to decompose large problems into smaller ones, but we arenot so good at describing the interactions among subcomponents.For example, we do not have a good vocabulary to discuss multipleeedback loops and the emergent behavior they can generate.

We also have trouble when dierent aspects o a problem re-quire undamentally dierent decompositions. For example, whendesigning a spacecrat control system there is a natural decom-position into subsystems and devices. However when dealing withthe limited power budget or the attitude control system, thereare a tangle o special-case subsystem-to-subsystem couplingsbehind a aade o modular decomposition [7]consider thatspinning up a disk drive consumes power, generates heat, andacts as a gyroscope. This interaction o concerns goes beyond theidea oseparation o concerns that aspect-oriented programming

(AOP) tries to address.The sheer complexity o the problem and its solution makesit practically impossible to capture such interactions in a staticspecication. The experience at Tandem Computers o tryingto create reliable systems ound that many ailures were due toaults in the specications. Specications are always incompleteand oten just plain wrongmoreover requirements are alwayschanging. Methodologies with an incremental approach ocon-tinuous (re)design are a better match to changing specications.Related to this is the realization that errors are unavoidable, dueto bugs in the implementation and also rom unexpected inputsrom the environment, so we must design our systems to continu-ally run tests and repair errors.

Going rom problem requirements to system design creates

an explosion o derived requirements (the requirements or aparticular design solution); the list o implicit requirements canbe 50 times greater than the original problem-ocused require-ments. [11] Lots o details need to be lled in when translating

what to do (describing the problem to be solved) into how todo (describing how the problem is to be solved). This includestranslating the problem into computer terms, speciy ing requiredcomponents, and even mapping the work to the organization othe people developing the solution. Again we lack good modelsand higher-level abstractions to talk about complex, interactingprocesses.

Another major source o complexity is unplanned interactionsbetween componentscross-talk where bugs or changes in one

piece o code aect quite distant, seemingly unrelated pieces ocode. These implementation combinatorics make it impossibleto thoroughly test sotware: Even 100% test coverage cannot ac-count or code interactions, e.g. an error maniests only when aparticular combination o logic paths is executed. And beyondthe lowest level o unplanned direct interactions there are theunplanned indirect ones. Examples would be eedback loops cre-ated which cause new and complex behavior to emerge. Anotherwould be system-wide interactions which could cause oscilla-tions (again this involves eedback)sometimes via unplannedand unthought-o shared paths like indirectly shared resources.Lastly would be emergent behavior resulting rom new interac-

tion patterns. Hardware is generally more robust because it ismore modular (and less state dependent). Where it relies onhistory / state inormation hardware too is subject to bugs (e.g.the Pentium math bug [28] in which the results o the divisionalgorithm depend on intermediate partial results and the pathtaken through a aulty lookup table).

As i our problems with current systems were not enough, even

bigger systems are on the drawing board. Plans or ultra-large-scale (ULS) systems consisting o billions o lines o code spreadover thousands o widely distributed multi-cpu computers arenow being proposed. These systems must run continuously, evenas individual components / computers come and go. In order tocreate such systems we need new approaches to how we designsotware. [35]

3. Continuous(Re)Design

The practice o continuous (re)design will move rom the actoryto the eld and large lump development will ade away. Develop-ment strategies have changed over time as its become clear thatits not possible to get static requirements right the rst timearound. From heavy design-beore-coding methodologies to agile,people have tried to capture via processes the elusive nature ogetting things right. The trend has been toward entangled inter-actions with end-users and customers in order to nd out whatthe sotware needs to do and how it is most congenially used.However, the watchword is sti ll getting things right.

But because requirements are static doesnt mean that sot-ware viewed over time stands still . Programs evolve and change,but only through a development process back at the actory. Bugreports, comments, suggestions and requirements determined bya marketing group, bright ideas, and new technology combined

with a development group produce the next version o the sot-ware. Static requirements change over time, and in this narrowsense, its possible to view sotware as a living entityadaptable,fexible, malleable, resistant to ailure. But not sel-sustaining.

Design is less a result opre hoc planning than o slowly dawn-ing insightsinsights derived rom seeing how the thing turnsout and is used. Continuous (re)design will move out into theeld because the eld is where the observations are immediateand changes can be tested rapidlyin situ.

Already some programs oer personalization and customiza-tion allowing users to participate in the design o the sotware.Some basic looks, eels, and behaviors can be changed by tailor-ing menus, adding keyboard shortcuts, and dening macros andscripts that add behavior and provide workarounds. Many o theoptions are available through a mechanism called preerences,

which is a set o attributes whose values can be set, such as colors,eld placement, and the like. Preerences are like wal l coveringsand color, window treatments, artwork, and, to a degree, urnish-

ings. (Furnishings are perhaps more like plug-ins.)

4. Soft/DynamicArchitectures

With the advent o soter / more dynamic architectures, changeto programs will be welcome, even encouraged. Today, sotwaredevelopment is a process o building static artiactsstatic mean-ing the program will not change once delivered, remaining infex-ible or subject only to oreseen adaptations. A good metaphor isa highway. Constructing a highway takes a lot o eort includingexpensive and disruptive land purchases. Once the dimensions


3/18

and course o a highway are set, they are stable and static until amajor change is made. Word processors are near the extreme osuch static programs: useul or writing and revising documents

with a wide variety o styles both visual and artistic, but its ownuture on a writers computer is rigidnothing changes.

Not all sotware is totally static, and such programs begin

to point the way to change. Plug-in architectures and web-ser-vice-based applications can be changed once installed by add-ing modules that provide services based on dened protocolsand interaces. Adobe Photoshop is the classic example; andbrowsers, integrated development environments, and operatingsystems are built this way. Here the good metaphor is a house,

which is also (usually) careully planned (by an architect) , butwhich can be and requently is customized by its occupants inboth small and large ways.

Stewart Brand looks at buildings as having six layers: the physi-cal site, the load-bearing structure, the exterior suraces, the gutso services like wiring and plumbing, the interior layout, and thestu in the building like desks and phones. The timescale andeort needed to make a change varies across these layers; site

is eternal, while stu can easily be changed daily. [3] Our com-puter systems also consist o layers: programming languages areat the bottom,1 then hardware, operating system, libraries, andapplications. Applications general ly consist o one or more levelso compiled code (e.g. a ramework and application logic) with aew preerence settings so that the user can modiy the look andeel. On top o that is the layer o the user data, corresponding toBrands stu in the buildingthough in many applications the

desks are bolted to the foor.When thinking o sotware architecture, you probably are

imagining an architecture diagram made o bubbles with linesbetween themperhaps a ramework (a big bubble maybe) withcomponents (the little ones) where the lines mean communica-tion through a small pipe, like an interace. This is what we have

today. But what would it be like to imagine a diagram like this

Brands layers correspond to permanence. For sotware, changing to aprogramming language with semantics not like Fortran, C, or Java would(rationally) require a new hardware platorm. In act, hardware (even in-struction sets) has changed more than the basic semantic structures oour programming languages, and so the languages orm the most basiclayer or us. Sometimes the stack is drawn as an hourglass with hardwareat the bottom, programming languages at the narrow waist, and applica-tions at the top. This emphasizes how applications are decoupled rom thehardware through the use o a standardized programming language.

where two components (red star and blue ellipse) have meldedwith the ramework (orange blob)? [Figure 1]

Some applications, such as spreadsheets, enable their usersto collaborate by adding new macros and sharing templates. Noteveryone can write macros, but one or two tinkerers in a companycan make extensions to the spreadsheet primitives that others

can then easily use in their ormulas. [26] An architecture thatsupports scripting or a macro language adds another layer thatenables users to modiy an applications behavior without needingto send it back to the actory or the proessional programmersto rework. I such an architecture also supports adding plug-inmodules then that opens the door or the do-it-yoursel user toreally customize the application to his or her local needs.

Beyond such sotnesses are sotware architectures that makechanges to themselves over time: dynamic adjustments to bothstate and behavior as the stu (components / architecture) adapts

to the environment in which it nds itsel. We can imagine ob-jects augmenting themselves, changing their algorithms, evolv-ing through competition,2 and rewiring their relationships asthey evolve.

5. In(ter)dependentSoftware

Sotware will be constructed rom components that want to orma community. Systems will be componentseither within thesame address space or notthat interact as needed, even suppos-edly / original ly monolithic systems. This wil l enable us to treatevery system as i it were a distributed system, which might meanthat systems can be more robust to single-point ailures.

Components will be constructed to minimize or ameliorateversion dierences and calling convention mismatches. Today,strange as it seems, components are writtensuch as dynamiclink libraries (DLLs)with infexible interaces so that slightmismatches halt reasonable operation o a system. The result isalmost as i the designers o the components (or more properly,the overall OS-level system architecture) were more interestedin their code and designs punishing simple mistakes rather thantrying to make that code work around those mistakes to getsomething running.

An approach like Laniers phenotropics could help. Phenotropic

computing uses suraces or interaces rather than direct argu-ment-based / protocol-based interaces. Each component has asurace on which is displayed inormation about what it is doing

Digital sotware evolution

soft architectural elementsoft architectural element

Figure 1: Soft Architecture


4/18

and that it would like to communicate to another component(or other components). Two components interacting observeeach others suraces and react to what they see or sample.Approximation replaces perection, and the result can be moreaccommodating but perhaps less optimized behavior. This is incontrast to our current use o rigid, minimalistic APIs where oneagent essentially reaches inside another and commands it to do

some unction (i.e. a remote procedure call). Instead the rstagent would present a request that the second could interpretand deal with as best it can. Other agents might also choose toparticipate by transorming / translating the original request,orwarding the request to an appropriate agent, or working onit themselves. The agent making the request need not even know

which agent(s) will eventual ly handle it .

6. Self-Installation

Sotware in the uture will take an active role in its own instal-lation. At present, many programs are considered properly in-stalled once the correct bits are in the right locations. And eventhis problem is not as well addressed as it could be: There arenumerous interdependencies that go into proper installation. Inrecent yearswith sotware updatesthe installation scriptshave become more sophisticated, but when more permanentlayers (in the Brand sense) are installed, the process can requireextensive human in(ter)vention. In act, the outermost (appli-cationmost) layers are the places where installation has becomeeasier or the installer.

Ignored as part o instal lation are the sometimes dozens o ad-justments people make to the entire set o installed programs tomake the computing / digital nest comortable (again).3 First arethe obvious, such as: Although there are a handul o commandsor moving through text, the gestures a person likes or variouso themthough knowableare never addressed. I its evenpossible to make a program that manipulates text use Emacscommand gestures, why should a person have to gure out how

to program / customize / personalize the sotware? Gesturesare just part o it. The look and eel should be adaptable too. Asstylesheets or application look and eel become more prevalent,these can orm the basis or adapting to a preerred use scheme.

Each program should endeavor to discover such customiza-tions. Further, it should be possible or one person to transer hisor her customizations to someone else who could then apply someor all o them either temporarily or permanently.

In the same vein, other aspects o the preerred use o sotwarecan be deduced and accommodated. For example, when installinga web browser, the browser can notice that one o its potentialusers is also an ardent user o Adobe products and to congurethe browser to use Acrobats PDF browser plug-in instead osome other one. The overall user color scheme can be intelligently

guessed at and compatible colors can be chosen. A program thatmanipulates text can try to determine how spellchecking is donearound here.5 For example, I (one o the authors), like many inmy generation, like to use Emacs or manipulating text. Many

And this is the problem o (re)installing inner, permanent layers.

Emacs is a text editor widely used by programmers. It is highly custom-izable but has a distinct set o deault key bindings.

At one extreme this can be taken as an example o the idea o commonmodules, typically selected by the user or some specifc advantage.

programs manipulate text. I am not particularly interested in thenext persons bright idea o how to do it (unless it is a abulousidea), so why cant I use not only the style I like (Emacs commandsas mentioned beore), but the very same Emacs in every situation,which I may have customized beyond recognit ion?

Another approach, o course, is a systemwide preerence mech-anism where specic preerences would be looked up in a hierar-

chical structure. This is the standardization approach. Althoughthis would work ne were it to happen, the realistic likelihood othat is low. To get the majority o operating system / application

writers to use such a system would require a mandated standard,which is unlikely or user experience. Moreover, or a long timethere will be a signicant population o legacy code that will notuse the standard. And then there are the renegades.

Moreover, there is a real question about whether and whenstandardization is the right solution to a problem. Standardiza-tion requires universal adoption, and, when achieved, tends to

promote infexibility. And once standardization is taken as thekey, it is typically taken too ar, so that we end up, or example,

with a monoculture.Better, we think, is ad hoc and independent fexibility, recog-

nizing that multiple standards / dialects will always exist, eacho which will change over time.

7. ContinualInstallation

In our vision, the installation process never ends: As new sotwareand hardware are installed, a lready installed sotware shouldcontinue the installation process, learning how the interests andhabits o its users are changing over time. What it learnsnot the

personal things like where its users sur and the kinds o booksand videos he or she preerscan be sent back to the sleepy de-velopers back at the plant who can move their agile ngers intogear to keep progress happening. Later or someday, this knowl-edge can be used to sel-adapt the sotware.

Related to installation is the massive upgrade: buying that new

heavy-liting desktop machine or the spry and miniscule portablelaptop / palmtop / ngertop / pintop. When a new computer sys-tem is acquired, all that should need be done is to point the oldsystem at the new and have them sync. Sync in the sense ocompatibly installing everything and readjusting to the new sur-roundings. Most sotware manuacturers treat the upgrade as anopportunity (provided or ree) or the users to clean housetoreinstall everything that was previously installed.6 However, themassive upgrade is not a resh start: Its an increment over theold environment because people expect the continuity o theirlives to dominate the stutter o technological progress. And anew system should consider itsel (anthropomorphically) to beinstalling itsel in an old, established environment. Both the oldand new must adapt.

O course, none o this should be irrevocable. Any customiza-tions and installations should be trivial to back out o or alter. Itshould always be the case that any installation can be completelyand accurately undone.

Apples OS X has a Migration Assistant which perorms the easier halo this when a new version o the operating system is installedit copiesfles rom the old system, ignoring fles supplied by the new version. Itdoesnt do any readjustments.


5/18

8. BeyondInstallation

Someday the manuacturing model may become pass. Con-sider the notion o no installation or certainly no re-installationat all. In such a world, perhaps the only starting point would bea single object which would, like a seed, begin to use resourcesavailable to it to compose the application, i it even makessense to use that word. The seed would assemble components,download clones, etc. to custom build and wire itsel into the

world in which it lives. When an update was desired, the objectsthemselves might morph, according to instructions receivedrom the original authors, while retaining customizations theyhad adaptively designed. In this way, objects change in situ andthe system / application never really restarts nor does it lose thebenet it has had rom living in the world with its users.

9. BuildablePackaging

We imagine that sotware will be more completely packaged. In-stead o just binaries and some support les, each piece o sot-

ware will contain everything it needs to be urther developed bylocal sotware developers.7 Today, i any sort o local development

is possible, its because the source code and required supportingmaterial can be loaded onto a local machine or such a machine isaccessible over the internet. In general, only open source and othersource-available systems present this optionand even when thesource code is available, the right compiler might not be, etc.

A system delivered completely packaged like this would beeasy to modiy or those capable o it; those whose knowledgeand skills are limited will still be able to make stylesheet-basedand other conguration-lebased changes. Some o the expectedcustomizations include the ability to adapt the code more pre-cisely to the local environment, perhaps by the encapsulated de-velopment system being able to sense its environment and shapethe code a little better.

How to implement this is problematic at the moment because

there are many sorts and levels o dependencies sotware can haveon its build and execution environments. For example, how canthe dependence on a particular version o a library or o the oper-ating system or that matter be captured? These sorts o dicul-ties can be handled by packaging up all the dependencies (and

perhaps storing them in a central location or all instances thatshare dependencies), but such an approach doesnt handle theexecutables, like the compilers. The problem is that a compilerneeds to execute on a machine that exists, and it needs to pro-duce code or the target machine. This is probably a case wherestandardization can help: A given platorm is very likely to havea compiler or a popular or mandated standard programminglanguage. Another possibility or executables is a virtual machinethat executes the compiler; porting the virtual machine should

be relatively easy and possibly standardly done.There can be other advantages that derive rom completepackaging. For example, the existence o sl ightly dierent ver-sions o some sotwarewhich could very well end up interacting

with each otherraises such questions as what constitutes theessence o a program and what kinds o variations are permitted

while retaining its identity. With such a sel-enclosed mechanism

Such a packaging has numerous copyright and intellectual property is-sues, similar to those ound in open source sotware.

comes the reality o a population o individuals8individualinstances o the same program, sotware, component, or sys-temand with this it becomes possible to think about advanc-ing the species by selective crossbreeding and other means typi-cally used in husbandry. Whereas today a monoculture is highlyvalued, in the uture not only will it have lesser or no value butit may become obsolete. In such a world, the value o common,

nonprogram-specic standards or data, control, and behaviorexchange will increase.

Another good result is the possibility o el iminating the prob-lem o lost source code, which is a surprisingly pervasive problem

with legacy sotware. The company that created the sotwareyou run goes out o business and no one cares about the sourcecode or thinks its important; an outside group wrote the sot-ware or your company and its backed up on a tape thats thenlost. This would never happen were sotware always packaged

with everything you need. And there would be ewer problemsre-establishing the work environment.

10. SoftwareasanActiveCollaborator/Participant

Sotware should take responsibility or its own uturenot mean-ing sotware broadly construed but meaning each recognizable

program, package, and application. Ater all , sotware can act,it can sense some part o its environment, and it can react tochanges, so why should sotware remain passive once its beenunleashed to the world? The attitude that a programs actionsshould be limited to what was planned at the actory remindsus o what Marvin Minsky wrote in Why Programming Is aGood Medium or Expressing Poorly-Understood and SloppilyFormulated Ideas:

There is a popular, widespread belie that computerscan do only what they are programmed to do. This alsebelie is based on a conusion between orm and content.

A rigid grammar need not make or precision in describ-ing processes. The programmer must be very precise inollowing the computer grammar, but the content hewants to be expressed remains ree. The grammar is

rigid because o the programmer who uses it, not becauseo the computer. The programmer does not even have tobe exact in his own ideashe may have a range o ac-ceptable computer answers in mind and may be contenti the computers answers do not step out o this range.The programmer does not have to fxate the computerwith particular processes. In a range o uncertainty hemay ask the computer to generate new procedures, orhe may recommend rules o selection and give the com-

puter advice about which choices to make. Thus, com-

puters do not have to be prog rammed with extremelyclear and precise ormulations o what is to be executed,or how to do it.

Marvin Minksy [25]

Sotware should be able to examine its environment prior toinstallation, monitor changes to its operating conditions andadapt to them as best it can, observe the state o its own health

This already is true but not widely recognized. Any real running systemruns a variety o release and patch levels.


6/18

and attend to it, pay attention to how its human users use it andbecome easier to use, provide or its own improvement at thehands o local developers (perhaps by noticing and remarkingon places where users had diculties such as requent requestsin one spot to undo an operation), accept extensions and cus-tomizations, and, nally accept and provide or its own deathand replacement.

In short, sotware should act like a liv ing collaborator in itsown uture and not like a manuactured machine waiting or ob-solescence to overtake itthere isnt even any minimally valuablescrap metal or plastics to salvage. Just about everything else webuy as consumers is subject to being repaired or tinkered withgiven enough knowledge and/or braverythis is the legacy o thephysical world: Its just not possible to wall o easily the businessend o most macroscopic physical machines the way its possibleto wall o rom reasonable intrusion the parts o sot ware (andmicroscopic machines like CPUs) that make it readily malleable.That is, the source code or a corporeal machine is not behind arewall on its makers server.

Some languagesLisp [23, 2, 36], Smalltalk [12], Sel [37],and othersprovided some steps in this direction by beingre-

active. Terms used in those times to describe such systems /environments / languages include exploratory programming,rapid application development, and interpreted. In a reactiveenvironment changes are immediatethe response to a changeis instantaneousso not as much bravery is needed to dive inand see what happens. Such systems increase the possibility osalvaging an old bit o sotware. How oten has it been just onething youd like to change about an applications behavior? Ina reactive world you could just make that change and see whathappens.

11. FailureisCommon

We will assume it. This has been substantiated by numerousstudies o sotware ailures and amously long bug lists or most

sotware systems. David Hovemeyer and William Pugh havereported:

we have ound that even well tested code written by ex-perts contains a surprising number o obvious bugs.

David Hovemeyer [14]

To write a program or system assuming that nothing will gowrongeven in the parts being written at that very momentisoolish, and sotware written this way in a ew years wi ll not betolerated and people who design and program that way will bebarred rom practicing the art. Here Joseph Weizenbaums pointo view is illuminating:

The psychological situation the compulsive program-

mer fnd himsel in while [fxing a bug in his code] isstrongly determined by two apparently opposing acts:frst, he knows that he can make the computer do any-thing he wants it to do; and second , the computer con-stantly displays undeniable evidence o his ailures tohim. It reproaches him. There is no escaping this bind.The engineer can resign himsel to the truth that thereare some things he doesnt know. But the programmermoves in a world entirely o his own making. The com-

puter challenges his power, not his knowledge.

Indeed , the compulsive programmers excitement risesto a evered pitch when he is on the trail o a most re-

calcitrant error, when everything ought to work but thecomputer nevertheless reproaches him by misbehavingin a number o mysterious, apparently unrelated ways. Itis then that the system the programmer has created givesevery evidence o having taken on a lie o its own, and

certainly, o having slipped rom his control. This too isthe point where the idea that the computer can be madeto do anything becomes most relevant and most sound-ly based in reality. For, under such circumstances, themisbehaving artiact is, in act, the programmers owncreation. Its very misbehavior can, as we have alreadysaid, be the consequence only o what he has done.

Joseph Weizenbaum [40]

To understand the source o invasive and pervasive ailureconsider that no apple ailed to all because someone orgot totell it to.

12. SeekOutandRepairErrors

We expect sotware will become like living organisms. We aretalking about sel-sustaining and sel-repairing programsrun-ning code that is buggy but coping with those bugs and perhapsrepairing some o them9 or repairing the damage done. Moreover,

we are talking about sotware that continual ly adapts to its en-vironment, both at installation time and aterward.

In most sotware there are data structures which are used bythe processes the sotware embodies. Typically these data struc-tures are by-products o the computation but sometimes they areused to direct parts o it. Such data can be tested or integrity,consistency, and, sometimes, correctnessrepairing data canbe a potent orm o sel-repair.

Sel-sustaining and sel-repairing programs are possible andnecessary. Today, saety-critical systems are deployed or which

stopping and awaiting human intervention ater a ailure hasoccurred is simply out o the question. Even a system or whichshutdown is reasonable needs to shutdown saely. Some research-ers (Rinard: [31], [32]; Patterson & Fox: [5], [6], [27]; Evans: [10])have been exploring ideas or sel-repair, rapid recovery rom er-rors, and ailure tolerance. To many practitioners today, the idea orecovering rom an error goes against the grain: the error shouldnever have happenedits the result o bad design or bad coding.Or bad requirements or specications, or because the sotwareends up in an unexpected environment, perhaps because theenvironment is being updated / upgraded.

The changes we oresee have as much to do with vocabularyand metaphors as with new techniques and technologies. Letslook at one o the rst instances o a program that used sel-re-

pair rather than absolute correctness.

In 1958, John McCarthy was thinking about a symbolic di-erentiation program in a programming language that was laterto become Lisp. He was concerned about the erasure problem:

Sel-repair o code can be approached variously. One promising vein isto think about building sotware through generationrom some modelor specifcation or by some process whose initial conditions can be variedand the code regenerated.


7/18

no-longer-needed list structure needs to be recycled or uture use.In subsequent languages, such problems were handled either bythe structure o the program being restricted to trees (stack al-location o data and its trivial ly automatic deallocation throughstack popping) or by explicit allocation and deallocation (mal-loc/ree). His comment on erasure / explicit deal location:

The recursive defnition o dierentiation made no provi-sion or erasure o abandoned list structure. No solutionwas apparent at the time, but the idea o complicatingthe elegant defnition o dierentiation with expliciterasure was unattractive.

John McCarthy [24]

Its worth a pause to notice the style o research described. Mc-Carthy and his colleagues were trying to design a programminglanguage. Part o their methodology was to write the programthey thought should be able to do the job and not the programthat a compiler or execution system would require to make the

program run well. In act, thebeautyo the program was oremostin their minds, not correctness down to the last detail . Beauty.Remember that word.

Eventually the rst Lisp implementers decided to ignore thebugthe ault o not explicitly erasing abandoned list cells, caus-ing the error o unreachable cells accumulating in memory, lead-ing to a ailure to locate a ree cell when one is expecteduntil theailure occurred and to repair it then. This avoided the problemo entangling a common set o unctionality (keeping availableall the memory that should be) with a pure and clear program(symbolic dierentiation). The ailure the ault eventually causedwas repaired, along with a lot o other similar errors in a processnamed at the time and still calledgarbage collection .

The entanglement o erasure code with the symbolic dieren-tiation code reminds us o the problem aspects address:

We have ound many programming problems or whichneither procedural nor object-oriented programmingtechniques are sufcient to clearly capture some o theimportant design decisions the program must imple-ment. This orces the implementation o those design

decisions to be scattered throughout the code, resultingin tangled code that is excessively difcult to developand maintain.

Gregor Kiczales et al [16]

The combination o inelegance and the oolishness o pursu-ing perection seem to combine:

The Goal of Perfection is Counterproductive: Theaspiration to eliminate as many programming errors as

possible creates a development process that diuses theocus o the project, wastes engineering resources, andproduces brittle sotware that is helpless in the presenceo the inevitable errors or aults. A more productive as-piration is to develop systems that contain errors andsometimes behave imperectly, but remain within theiracceptable operating envelope.

Flawed Software Has Enormous Value: The mostproductive path to better so tware systems will involvethe combination o partially aulty sotware with tech-niques that monitor the execution and, when necessary,

respond to aults or errors by taking action to appropri-ately adjust the state or behavior.

Martin Rinard [31]

The moral is not that Lisp researchers scooped the aspectsand recovery-oriented programming guys by about 0 years butto note that the metaphor o conscientious computing can beeective i thoroughly adopted. The people who wrote the rst

garbage collectors werent bogged down by the belie that a pro-gram is manuactured in one place or use in another, but believedthat a programming language was based at least in part on aruntime environment in which the sotware would continuallyevolvepossibly in response to changing requirements or thechanging nature o interdependent sotware.

In this case, the simplicity and elegance o the program (sym-bolic dierentiation was the example that drove the thinking)

was not to be compromised by the intrusion o an unrelated as-pect o the program (its memory management needs). Insteadthe program was allowed to ail (to run out o storage becausethe programmer orgot to deallocate used memory cells), andby a process o observing its environment (noticing it was out omemory), it was able to avert disaster and initiate a process o

repair to x the mess the programmers aults created. Impor-tantly, the memory management concern has become isolated in aseparate moduleseparate rom the program the programmer isinterested in, and in act, separate rom every other program.10

This is the sort o thought process we believe will becomeprevalent in the uture, but executed with more gusto (andletsace itmore guts, too).

Garbage collection is not the only example o the sort o pro-gramming practices we are talk ing about. We include as exam-ples util ities that repair and deragment disks, that rotate logles and otherwise cleanup rom the normal (and abnormal)activities o a complex system, and that are used as preventa-tives. Virus-protection sotware similarly looks rom the outsideat messages and other downloaded or created les or signs o

errors or pathology.Some o these utilities are triggered by events (like a le being

downloaded) and others are scheduled (like log le rotation). Butin all cases the problem-detection and repair code are indepen-dent o the code that experiences the problem. The code repairedand the repair code are decoupled, which is central in the sameway that by being decoupled rom the design and implementa-tion o the tested code, testing code is eective. Such a decou-

pling, though, is not eective when there are similar or commondesign / implementation points. I both the repair and repairedcode need to (correctly) implement the same dicult-to-imple-

0 A related idea is leasing: A resource is allocated to a particular processor program, but only or a specifed time. I the process or program de-

sires to retain the resource beyond its lease duration, it must renew thelease beore it runs out. I the process or program ailscrashes, goes intoa loopthe lease will not be renewed and the resource will be returnedto the system. This is like the garbage collection scenario except it is notthe ailure o a particular component within a system that is at stakebut the ailure o the entire system. And in the garbage collection case,a program notices that it has allowed its memory to all into disrepair,while in the leasing situation, the overall system notices the ailure onlythrough a orm o insurance policyor one might view it as a orm oemergence: nothing is specifcally looking or the ailure o a component,but the eects o such a thing happening are handled by a mechanismthat assumes ailure.


8/18

ment algorithm, then perhaps the same errors will appear in bothpieces o code, and the repair will not be eective.

Moreover, some o the preventatives, such as anti-virus sot-ware, per orm automatic sel-updates over the internet. This isin concert with our belie that sotware is continually instal lingitsel and actively improving. Such updates are provided by peo-

ple, but they need not be: They might be rom a design arm that

is continually improving sotware through a process o digitalsotware evolution or other automatic mechanisms.

Errors are typically handled by exception handlerscode thatis activated when a ailure occurs. But the ultimate cause o theailurethe aultmight not be apparent. Thereore, errors mustbe sought out, through aggressive means like continual testingand other bad smell detection. And as some researchers are nowdiscovering (and the inventors o garbage collection discovereda long time ago), repairing damage and the results o erroneousexecution can be an eective way to keep a program or systemoperating properly.

12.1 DontSeekOutandRepairErrors

Some errors dont need to be xed, and in act, the very act o

xing them can cause more harm than good. [32] Sometimes itmakes more sense to observe and note problems, perhaps tryingto keep track o coincidental events, les open, etc., with the ideathat perhaps with this inormationi the error turns out to beserious enough to xrepairs can be made.

As noted by Rinard and his colleagues, code can be parti-tioned into orgiving and unorgivingregions. Errors in unor-giving regions lead to atal errors or unacceptable results. Errorsin orgiving regions may result in bad or marginal results, buttypically useul results. The example they give is o sotware thatdecodes and plays mpeg les. The part o the code that locatesimportant metadata in the input streamwhich enables the resto the code to properly display the video streamis unorgiving;an error here causes an innite loop, early termination, or no

video to be displayed. The code that displays video inormationrom the input streamonce that inormation is oundis or-giving because generally errors there cause only degraded video/ audio quality.

Ideally, the system would be able to determine or itsel whicherrors are bad enough to be xed and which not, perhaps by ob-serving the amount o pain the error causes in users o the sot-

ware or in other parts o the system.

13.WriteTestsandContinuallyRunThem

Components o a system will be continually running testsseltests, environment tests, communications tests, etc. This willorm a matrix within which various eedback loops will keep

the overall system running well. At the moment, most programsand sotware operate like bulls in a china shop: They blast aheadassuming everything will go their way. Instead, well-designedsotware in the uture will constantly be adjusting to circum-stances, righting itsel, keeping clear o bad situations. Sucheedback loops will exist within the sotware and also betweenthe sotware and the sotwares environment, correcting smallerrors and making incremental improvements to the organiza-tion and perormance o the system.

Sel-tests and other sorts o tests will help a system discoverproblems early, so that their correction or amelioration is e ec-

tive;11 such tests can be useul to determine whether the environ-ment is changing or degrading. In such a regime, tests shouldlook outward as well as inwardout toward the environment as

well as measuring the eects o the environment on the sotware.When an update is downloadedalong with its new testsalltests will be (eventually) run and the results sent back to whereverthe changes refected in the update were made. This way, the sot-

ware developers will have the benet o the testing environmentso all instances o the code, and there wont be as much need oran extensive and varied testing arm. Almost every instal lation isdierent, because the exact needs o users depends on the com-plete local computing environment: dierent hardware, dier-ent drivers, dierent optional or custom packages and l ibraries,dierent patches and patch levels, and on and on. Perhaps eventhe order o installation makes a dierence, particularly whensotware is customized based on what is already in the system.We dont need to imagine some complicated learning environ-mentthe person or people using a system can customize it, andhow they do so depends on what they know, the experiences theyhave, and the work they do, which are all infuenced by what isalready installed on the computer.

Many sotware developers dont like to write tests, althoughthose in the agile sotware development movement put writ-ing tests at the center o design and implementation. For them,the tests are executable requirements. Their ervor over testingdoesnt go ar enough: Tests should be designed to run all the timeater the sotware is installed. I the tests are also continuallysampling and testing the environment, they can help a programavoid disasters and alert the users o the host computer o prob-lems. Such testing is part o the concept o eedback, a principleon which lie depends.

When testing is part o the culture o programs, and oncesotware is shipped with everything needed to modiy it, the pos-sibility arises o users creating tests that refect their usage pat-terns. Such tests would be useul to the original programmers.12

14. Exercise

Many o the behaviors we expect to see in uture sotware comerom the idea oexercise. When we currently think about ex-ercising sotware, we imagine simply using it or testing itorperhaps its closer to the details o walking the dog. But or people,exercise is a way to strengthen, and when we exercise our mindsour mental aculties can be improved. Many o our expectationsor the uture involve the idea o experience changing sotware.We have known or decades that one o the deep diculties o

Coupling tests with repair means that the eect o a sotware bug can becorrected or mitigated (possibly through rebooting, reinitializing a datastructure, or returning an acceptable value), but the actual bug remainsin the code until fxed by developers.

To the frst order, unit testing does its job: Components are verifed towork according to the designers notion o what the component is intendedto do. Problems occur when the component is used in an unanticipatedway and/or the component becomes a piece o a larger complex system.The system dynamics can cause unexpected behavior resulting rom acombination o una nticipated use o components and interactions be-tween components. State and behavior are stored in unlikely places inthe system (like the communication pathways between the components);our hope is that a new paradigm o testing could fnd ways to exercise /test such system dynamics.


9/18

producing sotware is that discovering requirements is hard andtrying to do so beore some working version o the sotware isavailable is impossible. So why should the discovery o require-ments stop once the sotware is insta lled when the requirementsthat depend on the local context are all around?

This requires writing sotware dierentlyin a way that per-mits changes in situ, something that is dicult but not impos-sible. [12, 23, 2, 36, 37]

15.UseFeedback

Sotware should adapt to its surroundings and prevailing condi-tions. Feedback is a mechanism ound in nature and in numerousmechanical designs to provide stability and adaptation in theace o changing conditions.

In biological systems, eedback plays a decisive role in main-taining cellular unctions. Chemotaxis [Figure 2] is an excellentexample o nested eedback being used by bacteria such asE. coli

to bias their swimming motion toward ood sources and awayrom toxins. The outmost eedback loop involves sensors that con-trol when the bacteria changes its direction. An inner eedbackloop causes the sensors to adapt so they stay sensitive to smalldierences over a very wide range o concentrations. Together thetwo loops enableE. coli to robustly ollow a gradient. [1]

Some algorithms and sotware that control physical devicesuse eedback, but otherwise it is a rare component o sotware.That is, no programming lang uage incorporates eedback as anessential mechanism, but it is, instead, relegated to the program-mer to explicitly construct out o low-level language eatures.13Building eedback requires variables, numbers, comparisons,and loops. Thereore, sotware developersespecially junioronesdont think in terms o eedback.

Even though eedback is essential to living systems.

16. MakeThingsVisible

System components will be able to see out into their environ-ments and into other components. When sotware has just in-stalled itsel, it doesnt have enough experience (or enough time)

Constraint languageswhich are not widely used or application pro-grammingmay be exceptions, though eedback is typically bur ied incontrol algorithms.

to scope out how the system is being used, so it needs to comeup in a deault modewith text editing, spellchecking, etc., ex-actly as some developer imagined them back at the actory. Byobserving other sotware in action or by looking at the recordedhistories o other sotware running on the system, the newlyinstalled sotware can learn about the usage patterns and pre-dilections o its users.

Visibility into the environment is restricted todaynot be-cause o privacy concerns or inadequate design, but becausethe concept is not in play. Encapsulation inadvertently is in e-ect in both directions. It is not desirable to look into a sotwarecomponent by design and it is not possible to look outside o acomponent (rom within it) because its not a concept in contem-porary computing. Communication across a sotware cell barrieris through an interacesometimes through arguments as in

procedural code or through method cal ls (or message-passing)in object-oriented code. But as any sotware developer knows,once inside the conceptual boundary between inside and outside,

the programmer (and one might also say, the code) has a specicvocabulary and set o concepts and ontologies, as does the envi-ronment outside the component. The bridge that is normally /exclusively built between the two is the signature (the name andshape o the communication) and the types o the pieces o inor-mation that go back and orth. This is a very inormation-poormedium. The very narrowness o the communication channel isthe source o many advances in conventional computing. Whencorrectness and optimality is oremost, the narrower the chan-nels o communication the better.

Visibility requires descriptions that are continually updat-ed1or example, descriptions o whats inside a systems sot-ware components, how a running system is currently conguredand what its working on, which users use which sotware and in

what ways, etc. And visibility benets rom abundancea rich

A description is not a representation. A representation can sometimesbe thought o as a model o something else, usually in the real world. Ma-nipulations o the representation typically mean something regardingthat model. A description cannot be manipulated to create a change inthe thing described. A description can be understood, compared to otherdescriptions, and actions can be taken based upon them. A representa-tion is typically efcient in some wayrepresenting just what is neces-sarywhile a description can be extravagant and mention a number oapparently inessential things.

Z

attractants&

repellents

W A

M

P

Y

Y

BP B

R

+

-

tumble

Figure 2: Chemotaxis


10/18

interace where perhaps pattern matching and some degree ounderstanding is possible.

This sort o visibility is partly what Lanier is aiming at withphenotropic computing. Lanier thinks o suraces that can bepattern-recognized or sampled. We are not particular how this isaccomplishedperhaps only (translatable) blackboards can su-ce with simple textual pattern matching. Or even extensions o

something like Common Lisps keyword / optional argument listsand calling conventions. Or even just posting XML documents.

Making continual testing eective is another reason or vis-ibility: Such things as the overall state o the system, what user-visible windows and processes are available, and what the usersphysical environment is like should be visible and, in some cases,alterable. For example, as I (one o the authors) type this sentence,I am also digitizing some old band tapes (the authors were in arock band together). It would nice i the program Im writing this

paper with knew about the recording sotware and that it wasrunning, so that it wouldnt interere with the program captur-ing all the data in the bitstream rom the digitizing hardware,

which can happen because the operating system, OS X , doesntknow, aside rom some heuristics, how I dierentially value the

operation o the two programs, nor, more importantly, how dis-ruptions aect the fow o the document-preparation program(typically, not at all) and my perception o my interaction withit (minimal or not at all). The document preparation programcould easily surmise that the real-timelike recording is likely tobe a lot happier i the program responding to typing were to get alittle less prioritysuch a diminishment would be unnoticeableto me but degradation o the data stream would be noticeableon the recording to everyone who listens to it. And the record-ing sotware could likewise look out or itsel, noticing I have amulti-CPU machine and conversing with the operating systemabout how to handle the situation.

17. ContinualNoticing

Systems will be aware. For a system to maintain itsel, it needsto see itsel, it needs some level o sel-awareness.15 [33, 3] Itmust be possible or a programmer who wishes to write sotwarethat operates on behal o its own well-being to write that code.A current trend is telemetry. A system sends a stream o datato another which is monitoring its health. A model like this is

part o the solution, but only part. Telemetry represents a time-varying look at one or a ew aspects o a system. The visibilityrequired or continual noticing includes the entire status o thesystemhardware and sotwareas well as some inormationabout each component: exceptions thrown, errors trapped, popu-lation o inputs handled (or example).

This principle also enables the use ostigmergyas a sel-or-ganizing mechanism. According to the Wikipedia:

Stigmergy is a method o communication in decen-tralised systems in which the individual parts o the

system communicate with one another by modiyingtheir local environment.

[http://en.wikipedia.org/wiki/Stigmergy]

Stigmergy is a concept proposed by Pierre-Paul Grass inthe 1950s to describe the indirect communication tak ing place

Sel-aware does not require AI or consciousness. A thermostat hasawareness. A cars electronic ignition system is sel-aware.

among individuals in social insect societies. [13] A termite willpick up some mud (or instance) and invest it with her phero-mones. This mudball will tend to attract other termites carry-ing their own mudballs, and the local eect will be to concen-trate the mudballs in a single place, and the global eect w ill beto create towers and arches which are then made into termitepalaces. See also Prigogine [29] and Camazine [] . In comput-

ing an example o this would be a process that writes data intoa JavaSpace that other processes can then read, do some compu-tation based on what theyve read, and then possibly write newdata into the JavaSpace.

An explicit description o the state and history o the sotwaresystem would enable the construction o sel-organizing sotwaresystemsnot necessarily systems that would sel-organize to ac-complish their designed unctions16 but that would sel-organizeto stay alive and healthy.

18. Autopoiesis/Allopoiesis

Programmed systems will become as living. One way that biologi-cal systems use eedback, visibility, and continual noticing is aspart o their sel -generating nature : Many o the eedback loopsin a liv ing system are involved with the regulation o productiono components, proteins, and other biochemical material thatregulates the production o other things including the regulatorsjust mentioned. This is essentially the concept o autopoiesis.Autopoietic systems are

systems that are defned as unities, as networks o pro-ductions o components, that recursively through theirinteractions, generate and realize the network that pro-duces them and constitute, in the space in which theyexist, the boundaries o the network as components thatparticipate in the realization o the network.

Humberto Maturana [21] (see also [22])

(Simply:) An autopoietic system is one that is continually(re)creating itsel. It is an interrelated network o componentsthat build that interrelated network o componentsthat is, itsel.And even though there might be deep connections to an outerenvironment, the boundaries o the system are distinct.

In living systems, living is the ultimate goal. And so the pro-duction o more living stu and its adaptation o that living stu to the surrounding conditions is in many ways more importantthan the nature o the living stu itselas long as it can sel-per-

petuate (more or less). Our programmed systems are not thoughto this way. What is important about a programmed system iswhat it does. In this sense the sotware systems we produce areallopoietic: Allopoiesis is the process whereby a system producessomething other than the system itsel. We are interested in thisother stu and dont care about the mechanism that creates it

beyond the act that it creates it. The health and stability o themechanism is beside the point as long as the desired productionhappens. Allopoiesis is the manuacturing process abstracted.

This explains a lot about our currently airly ragile systemsbut to see it requires examining longevity. Until recently, notmany programs needed to run or extensive periods o timeop-erating systems, certain equipment- and human-healthmonitor-

Most programs have requirements that prevent them rom naturallybeing written as sel-organizing systems, but this shouldnt stop us romtrying to look or ways to write such systems.


11/18

ing sotware, and some embedded systemsand thereore oureducational system has become geared toward teaching how todesign and code programs that are used once or or short periodso time. With the advent o the Web weve seen more systems thatneed to run or long periods. For example, web servers. These op-erate on essentially a regenerative basis: Each server request ishandled as i it were the rst and only request, and the context

or the execution is created aresh. That is, no state is carriedover rom request to request, and its as i the server were beingcreated each time it is needed.17

When a complex, programmed system needs to live or a longtime, living becomes the ultimate goal. Coupling this with theneed to produce the right answers, we ace the ollowing daunt-ing problem: How can we structure a system which needs torecursively generate, realize, and produce itsel as well as cor-rectly produce something other than itsel? That is, how do

we combine the correct and ecient unction o an allopoieticsystem with the urge to live o an autopoietic one that is continu-ally recreating itsel?

I we try to make a system more robust by adding lots o ex-plicit exception handlers and error detection code the program

becomes hard to understand and very dicult to maintain. Itgoes beyond what people can really write.An obvious alternative answerone that was pursued a bit

during the late 1970s and early 1980s, mostly in the articial in-telligence communityis to create autopoietic systems insteado the allopoietic ones that are too ragile and dont know aboutliving, and or that autopoietic system to also produce the al-lopoietic result.

One way o doing this requires the process that produces ulti-mate results (the allopoietic stu) also to be aware o itsel, thestate its in, and how well it is producing both itsel and its results.The simple garbage collection example demonstrates this. Theallocation o resh storage is part o the correct unctioning o theprogram, while the details o the garbage collect ion algorithm

are concerned with properly and reasonably locating unreach-able storage or reclamation. Some garbage collection algorithmsdont locate all such storage, leading to memory leaks18 or use less-than-perectly timely reclamation. That is, some collectors do agood job, but not a perect job. Most importantly, the operationo the code that does symbolic dierentiation (or example) isseparate rom that which does the memory management / era-sure / garbage collection.

Even this simple example demonstrates the undamental prob-lem with this approach: Techniques and languages suitable tocreate an autopoietic systemsuch as the less-than-perect gar-bage collectorsare (probably) not suitable or writing correct,

predictable, and deterministic code; code that correctly doesbanking, or example.

Moreover it is instructive to note that the concept o garbagecollection was resisted by mainstream language designers untilthe mid-1990s on the grounds that it was too slow or that the in-terruptions were too unpleasant or intolerable. Todayalmost50 years ater the rst garbage collector was writtenresearchpapers on how to improve or ameliorate these bad eects o sepa-

To some approximation.

A memory leak occurs when a program allocates a data structure us-ing storage that is never re-used (i.e. deallocated or reclaimed) once thestructure is no longer needed.

rate garbage collection food peer-reviewed venues, signaling thegrowing acceptance o the idea. Nevertheless, sotware develop-ers today will sometimes still preer to write a program that willleak memory until the program halts rom memory exhaustionover using an automatic system.

The lesson rom this is that there will be resistance to thecreation o an autopoietic mechanism i it is slow or, worse, bug-

gy.19

The faw in the earlier research programme was to attempt todene a single programming language or programming system toserve both purposesto operate both at the level o arithmeticand logic, procedure calls and method invocations, and also atthe level o eedback, visibility, noticing, and living. Any program-ming language that can operate at the allopoietic level is neces-sarily ragile, and so injecting those elements in the autopoieticsystem will break it as well.

Moreover, the principles o the two sorts o systems are incon-gruous. For example, think about visibility. There is an elegance, or

perhaps its just a symmetry, that the principle o visibility wouldapply to the autopoietic part o a system while the principle oinormation hidingwould apply to the allopoietic part:

In computer science, the principle o inormation hid-ing is the hiding o design decisions in a computer pro-

gram that are most likely to change, thus protecting

other parts o the program rom change i the designdecision is changed.

[http://en.wikipedia.org/wiki/Inormation_hiding]

The purpose o inormation hiding plays no role in the execu-tion o a programno role whatsoever. Only a role during thedevelopment o the sotware.20 Thereore, inormation hiding ispart o the process o sotware creation, which takes place withinsomething called the programming environment (in the old days)or the integrated development environment (nowadays). Thereis no reason that the sotware cannot be visible (not necessarily

alterable) during executionor even while the sot ware is idle.Heres an example o why.

When a new piece o sotware is installed, it might wonderasmentioned earlierhow spellchecking is done in these parts. One

way would be to inspect programs that are used a lot and havebeen used requently to see whether they provide spellcheckingand whether there is a common module or at least a dictionarythat the user has augmented. Then the new sotware could adopt(/ adapt to) the existing preerred method.

That is, visibility / invisibility should be valued dierentlywhen viewed vis--vis the two dierent aspects o a program: au-topoietic and allopoietic. Invisibility (ignorance o whats reallyinside a module, a unction, an object, etc.) is valuable when a sot-

ware development team is working to produce a system that wil l

then, while executing, go on to produce something elsethereare two allopoietic processes in play: the design / implementa-tion team producing the system, and the system producing itsresults.21 Limited visibilityo the module interacesat system

Even i there is a simplifcation in the expression o the mechanism. Notethat it takes less eort to write code in a garbage collected language, butprogrammers didnt / dont accept it because o perormance worries .

0 There are, however, execution-time repercussions.

But note: During the design process, the insides o a module are visible


12/18

construction time and at execution time is valuable because itreduces the possibility o change to known interaces, and theuse o an interace enables the ability o the potentially changedmodule to supplant the old one. When interaces are visible, it isreadily noticeable when they changecompile-time, link-time,and execution-time errors are timely.

Visibility is valuable to the operation o the autopoietic parto a sotware system, though it may be a hazard to the allopoi-etic creation o the initial autopoietic system. Perhaps it wouldmake sense to separate the allopoietic part o a system rom theautopoietic part; schematically [Figure 3] .

The encompassing autopoietic system needs to observe itseland its environment to know how to change and recreate itsel.It must also have some ability to observe and aect the operationo components o the allopoietic system.

Other principles are similarly in opposition [Table 1].Because o this, the requirements o the two types o systems

dont line up, and so designing an autopoietic language / environ-ment or allopoietic purposes is unlikely to succeed. The gure[Figure 3] contains the answer. The allopoietic part o the sys-tem must remain in an allopoietic language (and runtime when

thats appropriate) while the autopoietic part can be in a separatelanguagedesigned to do dierent things and with radically d i-erent characteristics.

The autopoietic language must (perhaps) have no ingredientsthat can cause errors o a nature as to crash a program writtenin it. A way to think o this is that it must be impossible to writea program with a certain class o bugs in it. Naturally, its notpossible or a person to write a program w ith no bugs, so whatwe mean by this must be elaborated. Some bugs are about theultimate problem, such as getting di erential equations wrong,updating cells in the wrong order, or orgetting parts o a specica-tion, while others seem centered around the business o program-mingper se, such as typos, encepost errors, ailing to initializevariables, etc. These two classes o errors eel very dierent. One

is when we are surprised or discouraged that the program doesntdo what we want and the other is when we are upset the programalls over, perhaps ater getting nowhere. One is a ailure to writethe right program and the other is a ailure to rightly write the

program (we intended) .In a lithe language, it must not be possible to write a program

that alls over. As an example, consider the game o lie [9]. It has

to some people: Design decisions made within a module are visible to thelocal decision- and sotware makers.

3 simple rules or the birth, death and survival o counters on aninnite checkerboard:

Survivals. Every counter with two or three neighboring coun-ters survives or the next generation.

Deaths. Each counter with our or more neighbors dies (is re-moved) rom overpopulation. Every counter with one neigh-bor or none dies rom isolation.

Births. Each empty cell adjacent to exactly three neigh-borsno more, no eweris a birth cell. A counter is

placed on it at the next move.

We have all seen animations o lie congurations. A programin the game o lie is just any initial conguration o counters onan innite 2-d plane. Some congurations dont do anything in-teresting at all: They grow orever or die o or all into a cyclicstate. Others are quite interestingin act, the mathematician

John Conway proved lie is universal: That is , it can simulate aTuring machine. But, consider that no initial conguration canhave a atal bug in it in the sense that the game o lie will halt orexperience some ailure. The conguration might not do whatyou want (due to a ault in placing an initial counter), but anyinitial conguration will run. This is not true in a general-pur-

pose programming lang uage.O course, this is what the type theorists aim at with (allopoi-

etic) type theory, but they will ail in the sense that atal bugswil l always be possible in the languages they work on because,ater all, those languages deal with numbers and arrays andother infexible and insensible data structuresdata structuresthat dont, or example, enorce application and domain-specic

semantics. 22 And i they were to succeed, well, we would simplyuse what they produce.

But we see autopoietic languages being lithe. Lithe:

Readily bent; supple. Marked by e ortless grace.23

This is true o essentially all programming languagesboth staticallyand dynamically typed.

This defnition is adapted rom The American Heritage Dictionaryo the English Language, Fourth Edition.

allopoietic process

allopoietic process

autopoietic system

stuff

Figure 3: Autopoietic Software System

allopoieticsystem

softwaredevelopers

software

developers

Autopoietic Systems Allopoietic Systems

visibility / transparency inormation hiding

dynamic / exible inter aces static / r ig id inter aces

sel-generating /decentralized

command & control /hierarchical

diversity uniormity

reactive manuactured

abundance / redundancy parsimony / e fciency

emergent(no reifcation)

reifcation o properties

loosely coupled interaction tightly coupled interaction

pattern-based signal-based

local rules global reasoning

lithe languagesJava, C++, C#,usual suspects

Table 1: Autopoietic and Allopoietic Principles


13/18

The structure o a system made this way would be o the allo-poietic part o itthe part that does what the designers intended,such as banking or web servingembedded in the autopoieticpart , which would be responsible, in a sense, or keeping theoverall system alive. In one possible metaphor, it would be like

Einsteins body keeping his brain / mind working properly.

We imagine a possible structure or this by the attachment(we avor this way o thinking o it rather than as a traditionalinterace, or reasons described later) o a monitor or autopoieticinteracewe will call it an epimoduleto each or some allo-

poietic components in an allopoietic system. The purpose o theepimodule is to monitor the behavior o the component and toalter it when needed. For example, in a conscientious allopoieticsystem, the component would include all its test code. The epi-module would be able to pause the component, execute its tests,and examine the results. I a new version o a subcomponentbecomes availablesay a new text processing componentthe

epimodule can direct the component to unload / unpatch / ig-nore the old version, install the new one, run tests, and then eitherproceed or undo the whole operation.

One o the more important sets o capabilities o epimoduleswould be to be able to clone and then kill a module. Imagine ascenario in which a component / module has a memory leak orother error whose bad eects accumulate. The autopoietic sys-tem through the intermediation o the epimodule can observeand track this problem and at some point decide to clone thecomponent. I the component has no persistent state, the tracto the old component can be shunted to the new one. I the com-

ponent has persistent state, the old instance o the componentcan be directed to clone itsel, copying over the persistent state.In some casesin act, in most casesthis will have the eecto doing a stop-and-copy garbage collection o the internal stateo the component, thereby cleaning up / leaving behind the ac-cumulated damage to the component.2

Instead o killing the module, it could be instructed to sel-destruct. This mechanism could have similar advantages asapoptosis, or programmed cell death, does in living systems.In apoptosis, the destruction o a cell is triggered by an internalor external signal, and the way the apoptotic process is execut-

Maybe . I the problem is in a persistent part, other techniques willbe needed.

edhow the cell commits suicideacilitates the sae disposalo cell corpses and ragments. Likewise, a system module askedto kil l itsel can cleanly stop (i possible) by explicitly releasingany system resources it has acquired, thereby working towardthe health o the system.

Epimodules, taken in the aggregate, should be able to rewireand hence recreate an entire allopoietic system, and by continu-ally regenerating it, keep it healthy and alive. The requirement,though, is to keep the allopoietic system operating properly: Itis perorming some precisely specied task, and that must con-tinue. Perhaps some degree o tolerance or uneven perormanceexists, but even that might not be permitted. The autopoietic sys-tem is there to make the overall system more robust, less ragile,more accommodating to users, and its own correct unctioningshould not become a actor in the correct unctioning o the al-lopoietic system.

Schematically [Figure ] .A more radical and interesting example suggested by Armando

Fox [6] is or an epimodule to observe a modules mutterings. A

module can be instrumented with a series o mutter statementswhich when captured as telemetry provides the modules innerdialog. When punctuated by input and output boundaries, themutterings can be considered messages, and a Bayesian learn-ing process can be applied to a testing corpus to learn what aregood and bad messages. Then the epimodule can run the Bayes-ian network over the messages o a live module to get a sense o

whether the module is acting normal ly, regardless o whether itsbehavior is correct. These judgments can lead perhaps to a clon-ing and subsequent death o the original component.

Another possible way to envision the allopoietic / autopoieticstructure is to look at a speculative, dierent, more detailed pic-ture [Figure 5]. The blue boxes represent allopoietic codethecode that does the real work and red blobs indicate autopoieticcode. The main big blue box is a component in a larger (allopoietic)

system. The bump and string on top represent its normal inter-ace and connectionsthink o it as the method signature andcall graph. The darker (lower) bluish box inside can be thoughto as the epimodule interace. Its job is to implement health- andrepair-related operations within the component. We see thingslike the ability to run teststhis includes packaging up the teststhemselves and some way o characterizing the resultsto clonethe component, to saely kill the component, or to repair datastructures. There could be others as well, such as a tap into thelocal exception handling and event environments.

autopoietic interface

allopoietic component

Figure 4: Epimodules

Run

tests

Clone self

Repair

structures

Safe kill self

code & data

epimodules

autopoietic components

Figure 5: Autopoiesis / Allopoiesis


14/18

The red shapes outside the box, including the encompassingreddish rectangle with the rounded corners, represent autopoi-etic code. The odd shapes that change shapes and shades romthe outside to the inside o the component (rom oval to squareand rom red to blue or rom a darker grey to a lighter one) rep-resent the epimodules that assist that component by keeping ithealthy and keeping healthy the entire system within which that

component exists.Epimodules can either be developed by the original developers

o the allopoietic code25 or be developed / supplied by the devel-opers or users o the installation site. There might be a standardecology o health maintenance code at the local site that appli-cation systems tie into.

Further, there is no need or epimodules to execute on thesame computer systems as the allopoietic code it oversees. Epi-modules can execute on another computer system or systemsonsite or osite, perhaps where the developers o the allopoieticsystem are located. This wi ll reduce the perormance impact othe architecture on the implementation.

Epimodule execution can also be asynchronous with respectto allopoietic execution. For example, the epimodules (the auto-

poietic system) can include extensive simulations to set param-eters (e.g. recompiling an allopoietic component with dierentcompiler switches to better tune it or actual use), determine thebest conguration, design and implement more appropriate glue,monitoring, or protection code.26

The red, blobby, autopoietic components communicate witheach other through the reddish background substrate and perhaps

by the action o autopoietic entities that operate in the substratebut which are not attached to specic allopoietic components.

We dont know much about what the autopoietic parts shouldbe like. And in particular we have not much o an idea what lithelanguages will be like. There is probably a continuum o likelypossibilit ies. But our intuition says that nudging, tendencies,pressure, and infuence will be more important than exchanging

control and data or traditional control structures and composi-tion techniques.At the conservative end, lithe languages could be declarative,

mostly descriptiveperhaps a set o wiring diagrams or puttingtogether the system rom its components, and specications othe conditions under which to run tests, to clone components,to do internal repair, etc. These descriptions and specications

would be interpreted much as a constraint system is to maintainthe unctioning o the system and its components. As such, thedescriptions might be ineective but never broken.

At the more speculative (and intriguing) end, the lithe lan-guage might be able to express only programs that are dimlyaware o themselves and their surroundings, but aware never-theless. The red, blobby epimodules could be l ike living cells orother living organisms that are attracted to components eithergenerally or specicallythat is, some epimodules might be tai-lored or certain components while others could be general. Theepimodules would attach themselves to components and senseconditions, both inside and outside the component, and exertpressure on the component to run tests, etc. Not interact through

Perhaps as a paid service.

Modulo bandwidth constraints. Note that a multicore CPU could mini-mize even this problem.

a traditional interace, but sense as i sensing the density o chemi-cals and exert pressure as i by a suggestion or a push.

The substrate could be thought o as a medium or smellsthat would infuence the activities o the epimodules. So, per-ormance monitoring o a component could exude a hotspotsmell, which when sensed by other epimodules would cause amigration o a set o components to a stronger cluster or a aster

machine. Perhaps it would be useul or the substrate to have ageometry so that concepts o distance and expanding signalscould be used to model accumulating evidence or disregard ir-relevant, anomalous spikes.

At this end o the spectrum, concepts like population andcrossbreeding could make sense so that the epimodules could

learn or evolve to be more eective. Perhaps epimodules wouldlive rom nourishment provided by a healthy system and its com-

ponents. Perhaps epimodules would eect the initial installationusing surrogates27 to optimize placement and connectivity, thereal installation taking place ater some initial experimenta-tionor perhaps the surrogates would assemble components orgrow them rom ound / sought parts and pieces in a sort o in-place development / growth process. And so on.

The concept o autopoiesis fows, in a way, rom ImmanuelKants analysis, where he compares living organisms to mechani-cal devices. Living organisms:

In such a natural product as this every part is thoughtas owing its presence to the agency o all the remainingparts and also as existing or the sake o the others ando the whole the part must be an organ producing theother parts, each consequently reciprocally producingthe others.

Immanuel Kant [15]

Mechanical devices:

In a watch, one part is the instrument by which the move-ment o the others is aected , but one wheel is not theefcient cause o the production o the other. One part iscertainly present or the sake o another, but it does notowe its presence to the agency o that other. For this rea-son also the producing cause o the watch and its orm isnot contained in the nature o this material. Hence onewheel in the watch does not produce the other and stillless does one watch produce other watches by utilizingor organizing oreign material . Hence it does not o itselreplace parts o which it has been deprived.

Immanuel Kant [15]

Allopoietic systems are like watches: The components are

there or each other, but not by each other. In other words, eachcomponent does not create the others. Order, perhaps, requirescontinual remaking because the nature o the universe is tounmake order. The digital world is thought to be immune romsuch eects, being timeless, but what is sometimes orgotten isthe allibility o the makersthe sotware designers, developers,and programmerswho continually create faws or ail to graspthe ullness o the requirements. And these faws accumulate and

We imagine a sort o digital stem cell.


15/18

operate on the whole and each other rendering errors over time.This is why rebooting is a common resort.

19. MorePrinciples

Although people have talked about dynamic languages, refec-tion, meta-models, and the like, conscientious sotware written

as an embedded allopoietic system within an autopoietic systemhas never really been attempted beore. The trip-up has been thatwhile the initial impulse or a programming model might be to-ward an autopoietic system, the needs o the allopoietic require-ments eventually come to the ore and any attempts at a radicaldesign old back into a traditional mold.

The autopoietic should not harm the allopoietic system. Thismight mean that the epimodules should not be enabled to actu-ally ki ll a module. One approach to this is to create a design lan-guage or autopoietic systems that is not Turing complete, andin act quite dramatically limited in its abilities. The autopoieticsystem might be programmed in terms ofows, where inorma-tion seeps through the system rather than being instantaneouslytransmitted rom point to point. Markets work because currencyfows, because goods fow. The immune system works becauseblood and other fuids fow. eBay works because reputation (re-lating to buyers and sellers) fows.

In many cases, fow is slow / through di usion. A web servercomponent can emit a slowly dispersing overload or hotspot

smellperhaps its input queues are overfowing. I the overloadpersists, the smell around the component wil l grow stronger andas it disperses it transmits its distress to autopoietic componentsthat, when triggered by a suitable threshold, might replicate thestressed component to relieve the overload. On the other hand, ithe overload diminishes, the smell diminishes and doesnt trig-ger the replication.

Thengradients. This is related to locality. A fow can ollow agradient, perhaps rom some region where sel-recognizers / non-selkillers are created to an area where non-sel has appeared.

Another example is in a network: The autopoietic system can sendout two fows that disperse at dierent rates rom a persistentstore. One fow is to enable a copy o the store to create a replicatedsite and the other fow disables copying. A copy is made at a site

with a probability proportional to the ratio o the concentrationo enabler to disabler. In eect, replicants are made not too close,not too ar, and not in a predictable manner. [10]

Think in terms olayers. A system doesnt

conscientious software

Documents