scalingupself-explanatory simulators: polynomial-timecompilation · 2007. 5. 2. · simgen mklto...

Scaling up Self-Explanatory Simulators : Polynomial-time Compilation

Kenneth D. ForbusThe Institute for the Learning Sciences

Northwestern University1890 Maple Avenue, Evanston, IL,

60201, USA

AbstractSelf-explanatory simulators have many potential applica-tions, including supporting engineering activities, intelligenttutoring systems, and computer-based training systems . Tofully realize this potential requires improving the technologyto efficiently generate highly optimized simulators . Thispaper describes an algorithm for compiling self-explanatorysimulators that operates in polynomial time. It is capable ofconstructing self-explanatory simulators with thousands ofparameters, which is an order of magnitude more complexthan any previous technique. The algorithm is fully imple-mented, and we show evidence that suggests its performanceis quadratic in the size of the system being simulated.

Wealso analyze the tradeoffs between compilers and interpret-ers for self-explanatory simulation in terms of application-imposed constraints, and discuss plans for applications .

1. IntroductionSelf-explanatory simulators [1,2,3,4] integrate qualitativeand quantitative knowledge to produce both detailed de-scriptions and causal explanations of a system's behavior .They have many potential applications in intelligent tutoringsystems and learning environments [5, 6] and engineeringtasks [7,8] . One step towards realizing this potential is de-veloping techniques that can operate efficiently on systemsinvolving many hundreds of parameters, so that, for in-stance, complex training simulators can be generated auto-matically. This paper describes a polynomial-time methodfor compiling self-explanatory simulators and shows that itoperates successfully on models larger than most industrialapplications require. This paper considers only initial-valuesimulations of lumped-element (ordinary differential-algebraic) systems .

Section 2 reviews the basics of self-explanatory simula-tors . Section 3 describes our new polynomial-time compila-tion technique. Section 4 outlines the complexity analysisand Section 5 summarizes empirical results, including evi-dence that its performance is quadratic in the size of the sys-tem being simulated. Section 6 identifies tradeoffs in con-structing self-explanatory simulators in light of task re-quirements . Section 7 outlines our plans for future work .

2.

Self-Explanatory simulation : The basicsTraditional numerical simulators generate predictions ofbehavior via numerical computation using quantitative mod-els of physical phenomena. Most simulators are written byhand, although an increasing number are generated by do-main-specific toolkits .[KDFI1 Modeling decisions, such as

To appear in Proceedings ofIJCAI-95, Montreal, Canada, August 1995

Brian FalkenhainerModeling Research Technology AreaXerox Wilson Center MS 128-28E800 Philli s Road, Webster, NY,

4580, USA

what phenomena are important to consider, how does thephenomena work, how can it be modeled quantitatively, andhow to implement the quantitative model for efficient com-puter solution, are mostly made by hand . Domain-specifictoolkits provide reasonable solutions to the last two prob-lems, and with appropriate libraries they can simplify thefirst problem. However, the choices of how to translate thephysical description into the conceptual entities supportedby the toolkit, and which quantitative model to use from thelibrary to model an entity, are still made by hand . More-over, no existing toolkit provides the intuitive explanationsused by scientists and engineers to describe how a systemworks . These intuitive, qualitative descriptions serve sev-eral important purposes in building and working with simu-lations. First, they guide the formulation of quantitativemodels by identifying what aspects of the physical situationare relevant. Second, qualitative descriptions are used tocheck the results of a simulation, to ensure that it "makessense." Thus two advantages of self-explanatory simulationare increased automation and better explanations [1] .

Self-explanatory simulators harness the formalisms ofqualitative physics to automate the process of creating simu-lators . Given an initial physical description, a qualitativeanalysis of the situation reveals what conceptual entities arerelevant to the task and what causal factors affect a parame-ter under different circumstances . This information is thenused, in concert with quantitative information in the domaintheory, to construct numerical simulation programs for thesystem . By incorporating explicit representations of con-ceptual entities (such as physical processes) in the simulator,causal explanations can be given for the simulated behavior .Moreover, the qualitative representations allow automatingsome of the "reality checks" that an expert would apply inevaluating a simulation . For instance, a simulation of a fluidsystem which reported a negative amount of liquid in a con-tainer is not producing realistic results. This ability is calledself-monitoring .

Figure 1 shows a typical architecture for self-explanatorysimulators . The state vectors are arrays of floating point andboolean values describing the state of the system at a par-ticular point in time . The floating point values represent thevalues of continuous parameters, such as pressure and veloc-ity. The boolean values represent the validity of statementsabout the physical system at that time, e.g ., if liquid existsinside a container or if a particular physical process is act-ing. Given a state vector and a time increment, the evolvergenerates a state vector representing the state of the systemafter that time increment.

In many physical systems, the equations governing thetemporal evolution of the system's behavior can themselveschange, as when phase changes occur or flows start andstop . These changes occur at limit points, which mark theboundaries between qualitatively distinct behaviors. Thetransition finder (again see Figure 1) detects the occurrenceof limit points . The controller uses the transition finder andevolver to "roll back" the simulation to include all limitpoints in the simulated behavior. This increases the accu-racy of the simulation because it ensures that the appropriatesets of equations are always used, and increases the accuracyof explanations because it ensures that causally importantevents (e .g ., reaching aphase transition) are included in thesimulated behavior . The controller also is responsible forrecording a concise history describing the system's qualita-tive behavior over time . This concise history is used in con-junction with a structured explanation system [9] to providehypertext explanations of the system's behavior over time,including physical, causal, and mathematical descriptions .The nogood checker generates a warning if qualitative con-straints are violated by a state vector, thus providing self-monitoring .

The first systems to generate self-explanatory simulatorswere compilers, doing all reasoning off-line to produce soft-ware that approached the speed of traditional simulatorswhile providing causal explanations and self-monitoring.These compilation strategies were computationally expen-sive . SIMGEN MK1 [1] used envisioning, an exponentialtechnique, for its qualitative analysis procedure.

SIMGEN MK2 [2] exploited the observation that simula-tion authors never explicitly identify even a single globalqualitative state of the system being simulated, hence quali-tative simulation is unnecessary for simulation construction .Qualitative analysis is still necessary to determine whatphysical phenomena are relevant, for instance, but most ex-ponential reasoning steps could be eliminated . SIMGENMk2 could construct simulators of systems larger than anyenvisioning-based system ever could (i.e., involving up tohundreds of parameters), such as a twenty stage distillationcolumn [10] . This advance in capabilities was not free: Thetradeoff was that some explanatory capabilities (i .e., effi-cient counterfactual reasoning) and self-monitoring capabili-ties (i .e., the guarantee that every numerical simulator statesatisfied a legal qualitative state) were lost in moving fromSIMGEN Mkl to SIMGEN Mk2. As Section 3 explains,even SIMGEN Mk2 was subject to combinatorial explo-sions, because it was based on an ATMS . The techniques inthis paper trade away more self-monitoring to achieve poly-nomial-time performance.

An alternative to compiling self-explanatory simulatorsis to build interpreters that interleave model-building, modeltranslation into executable code, and code execution [3,4] .For example, PIKA [4] uses Mathematica and a causal or-dering algorithm to decompose a set of equations into inde-pendent and dependent parameters and find an order ofcomputation. Every state transition which changes the set ofapplicable model fragments reinvokes this reasoning to pro-duce a new simulator. ([11] uses an incremental constraintsystem to minimize this cost.) One motivation for interpret-ers was the perceived slowness of compiler techniques ; byonly building models for behaviors that are known to be


relevant, presumably the overall time from formulation ofthe problem to the end of simulation would be reduced, eventhough the simulation time itself might be longer due to run-time reasoning. Such systems are not themselves immunefrom combinatorial explosions, and have never been testedon examples as large as compilers (c.f. [10]), but on smallexamples interpreters can exhibit impressive performance.

It should be noted that the claim that PIKA is "5,000times faster than SIMGEN MK2" [4] is problematic, forthree reasons. First, the domain theories used by each sys-tem were completely different. PIKA used just two modelfragments, with built-in quantitative models containing ex-ample-specific numerical constants. This is not realistic . Bycontrast, in keeping with the goal of increased automation,the domain models used in SIMGEN MK2 were similar toothers used in qualitative physics, with quantitative informa-tion added modularly. For instance, parameters such asfluid and thermal conductances and container sizes andshapes are explicit variables in our models that can be set bythe simulation user at run-time . Second, the hardware andsoftware environments used by the two systems were com-pletely different, making the comparison figures meaning-less . Finally, the factor of 5,000 claimed is based on oneexample only ; the other example for which even roughlycomparable data is available shows a difference smaller by afactor of 10 .

3.

SIMGENMK3 : Compiling self-explanatorysimulators in polynomial-time

We have developed a polynomial-time algorithm forcompiling self-explanatory simulators . The improvementsin complexity are purchased at the cost of reduced self-monitoring and less compile-time error checking . However,the ability to quickly generate simulators for systems con-taining thousands of parameters suggests that these costs areworth it. Here we outline the algorithm and explain how itworks. We begin by describing why using an ATMS [16]was the source of exponential behavior in SIMGEN MK2.Next we examine the inference services needed to generatesimulators, and show polynomial-time methods for achiev-ing them . Finally, we outline the algorithm.

3.1 ATMS: The exponential withinATMS' are often used in qualitative reasoning when manyalternative situations are being explored [12] . A qualitativestate can be defined as a set of assumptions and their conse-quences, so that states (and partial states) can be conciselyrepresented by ATMS labels [13] .

The labels in an ATMSdatabase provide a simple and elegant inferential mechanismfor simulation generation . For instance, the code needed togenerate the truth of a proposition could be generated byinterpreting the label as a disjunction of conjunctions, witheach assumption becoming a procedural test. (For instance,the boolean corresponding to a particular liquid flow occur-ing might be set to TRUE if the boolean corresponding tothe fluid path being aligned was TRUE and the numericalvalue for the pressure in the source were greater than thenumerical value for the pressure in the destination.) Labelsfor causal relationships were used to infer what combina-tions of them could occur together, and hence what mathe-matical models were needed . (For instance, there can beseveral distinct models for flow rates, depending on whetheror not one is considering conductances, but no two of thesemodels can ever hold at the same time .) Using labels madecertain optimizations easy, such as proving that two distinctordinal relationships were logically equivalent and thus al-lowing the same test to be used for each, which enhancesreliability . (For example, in a spring-block oscillator therelationship between the length of the spring and its restlength determines the sign of the force.)

Unfortunately, even when we did not perform qualitativesimulation at all, the number of environments generated bythe ATMS grew exponentially with the size of the systemmodeled. Empirically, we found that the source of thisgrowth was the transitivity inference system, whosejob it isto infer new ordinal relationships via transitivity andmark asinconsistent combinations of ordinal relations that violatetransitivity . This makes sense because dependency networksin which assumptions arejustified by other assumptions, andespecially those containing cycles, lead to exponential labelgrowth [14] . The majority of assumptions in a typicalanalysis are ordinal relations, and cyclic dependencies areinherent in transitivity reasoning. Transitivity reasoningcannot be avoided when using an ATMS in simulationgeneration, because without it the labels will includeimpossible combinations . We conclude that the ATMS mustbe abandoned to generate simulators in polynomial time .

3.2 How to get what you really need withoutexponential performance

What is the minimum reasoning needed to generate asimulator for a physical scenario? (1) The model fragmentsof a domain theory must be instantiated to identify therelevant physical and conceptual entities and relationships(e .g., the existence of contained fluids and phase changeprocesses) . (2) The causal and quantitative relationshipsthat follow from them must be determined, to generate theappropriate causal accounts and mathematical models (e .g .,models that allow the level of a liquid to be computed givenits mass, density, and specifications of its container) . (3)The truth values of propositions corresponding to theserelationships holding and/or entities existing (e .g. whetherthe liquid exists at a particular time, and if so, what


ists at a particular time, and if so, what processes are actingon it) must be inferred to control the operation of the quanti-tative models . The performance of any compiler and thequality of the code it produces depends on how these ques-tions are answered . For instance, if the shape of a containercan be fixed at compile-time, then the model for liquid leveldepending on that shape can be "hard-wired," but if theshape is unknown, a run-time conditional must be insertedand code representing alternative models generated.

Three techniques provide the inferential services neededfor polynomial-time simulation generation . (1) Reificationof antecedents : When instantiating model fragments, createexplicit assertions concerning their antecedents, in additionto installing the appropriate dependency network. One ex-ample is((liquid-flow (C-S water liquid f) PI G):ANTECEDENTS( :AND (> (A (amount-of-in water liquid F)) ZERO)

(aligned PI)(> (A (pressure (C-S water liquid f))

(A (Pressure G)))))(The creation of such assertions is automatic, and is

transparent to the domain modeler.) Such reified antece-dents are also generated for causal relations and mathemati-cal models . These antecedent assertions are used by thecompiler in generating truth value tests for propositions . (2)Symbolic evaluation : If the truth value of a proposition isknown at compile-time, it is presumed to hold universally .For instance, if a valve is known to be open at compile-time,the simulator produced should always presume the valve isopen, but otherwise the simulator should include explicittests as to whether or not the valve is open and change itsoperation accordingly. A simple symbolic evaluator pro-vides this information, using the reified antecedents and alogic-based TMS [16] . Given a proposition, it returnsTRUE, FALSE, or MAYBE, according to whether the propo-sition is universally true, universally false, or can vary atrun-time . (3) Deferred error checking: Finding errors atcompile-time can require exponential work . One example isproving that exactly one of the quantitative models for aspecific phenomena must hold at any given simulated time .Such exponential inferences can be avoided by substitutingrun-time conditionals . For instance, if there are N mathe-matical models of aphenomena, insert a conditional test thatruns the appropriate model according to the simulator's cur-rent parameters, and signals an error if no model is appro-priate . This reduces self-monitoring, in that finding suchproblems at compile-time would be useful . However, thissolution is common practice in building standard simulatorswhen its behavior might enter a regime for which the authorlacks a good mathematical model.

3.3 The SIMGENMx3 AlgorithmThe algorithm is outlined in Figure 2. It is very similar toSIMGEN Mx2 (described in [2]), so here we focus on howthe above techniques are used.

Step 1: Creation of the scenario modelAs before, we assume domain theories are written in a com-positional modeling language, using Qualitative Process

theory [15] for their qualitative aspects.

Since the cost ofqualitative reasoning was the dominant cost in SIMGENMxl and Mx2, the tradeoffs in this step are crucial . Thesource of efficiency in interpreters like PIKA [4] and DME[3] is that they appear to do no qualitative reasoning beyondinstantiating model fragments directly corresponding toequations.

Step 1 uses a qualitative reasoner to instantiate modelfragments and draw certain trivial conclusions (i .e ., if A>Bthen ? A=B) . No transitivity reasoning, influence resolutionor limit analysis is attempted. We use TGIZMO, a publiclyavailable QP implementation [16] for our qualitative reasoner.

We modified it in two ways . First, the pattern-directed rules that implement many QP operations weresimplified, stripping out inferences not needed by the com-piler. Second, we implemented antecedent reification bymodifying the modeling language implementation to assertantecedents explicitly in the database as well as producingLTMS clauses.

Step 2: Constructing the simulator's stateHere the results of qualitative analysis are harvested to spec-ify the contents of state vectors, the concise history, and theexplanation system . The numerical parameters are the quan-tities mentioned in the scenario model. Boolean parametersare introduced for relevant propositions : the existence ofindividuals (EXISTs) and quantities (QUANTITY), the status ofprocesses and views (ACTIVE), and any ground propositionsmentioned in their antecedents, recursively (except ordinalrelations, which are computed by comparing numerical val-ues) . Each boolean parameter has an associated antece-dents statement, a necessary and sufficient condition for thetruth of the corresponding proposition .

As noted above, any proposition known to be true in thescenario model is presumed to hold universally over any useof the simulator. The compiler does not generate simulatorparameters for such propositions, although they are stillwoven into the explanation system appropriately . Everyproposition whose truth value is not known in the scenarioyields aboolean parameter whose value must be ascertainedat run-time . This technique allows the compiler to producetighter code by exploiting domain constraints and user hints .The symbolic evaluator decides what propositions are staticand simplifies antecedents containing them .

Step 3: Writing the simulator codeTo achieve flexibility compositional modeling demands de-composing domain knowledge into small fragments . Thiscan lead to long inference chains, which if proceduralizednaively, would result in simulators containing redundantboolean parameters and run-time testing . Step 3 .1 simplifiesinference chains in order to produce better code . We use twosimplification techniques : (1) Symbolic evaluation exploitscompile-time knowledge to simplify expressions and (2) Aminimal set ofboolean parameters is found by dividing theminto equivalence classes, based on their antecedents . That is,if a proposition A depends only on B, and B in turn dependsonly on C, and C either has an empty antecedent or an ante-cedent with more than one ground term, then A, B, and Cwould be in the same equivalence class, and C would be its


1. Create scenario model by instantiating model fragmentsfrom domain theory2. Analyze scenario model to define simulator state vectorand constituents of concise history and structuredexplanation system.

2.1 Extract physical and conceptual entities2.2 Define boolean parameters for relevant

propositions2.3 Define numerical parameters and relevant

comparisons2.4 Extract influences to create causal ordering

3 . Write simulator code3.1 Simplify antecedents of boolean parameters3.2 Compute update orders

3.2 .1 For numerical parameters,use causal ordering

3.2.2 For boolean parameters,use equivalences and dependencies

3.3 Write evolver code3.4 Write transition finder code3.5 Write nogood checker code3.6 Write structured explanation system

Figure 2: TheSIMGEN Mx_3 Algorithm

canonical member . Each equivalence class is represented inthe simulation code by a single boolean parameter, althoughall original propositions are retained in the explanation sys-tem to maintain clarity .

In Step 3 .2, the state space assumption, common in en-gineering and satisfied by QP models [17], guarantees wecan always divide the set of parameters into dependent andindependent parts, with the independent parameters beingthose which are directly influenced (or uninfluenced) andwith the dependent parameters computed from them. Togain a similar guarantee for boolean parameters we stipulatethat the domain theory is condition grounded [18], an as-sumption satisfied by all domain theories we have seen inpractice .

An independent boolean parameter mentions noother boolean parameters in its antecedents . It could be uni-versally true or false, or its value could be determined atrun-time by ordinal relations (e .g ., the existence of a con-tained liquid depending on a non-zero amount of that sub-stance in liquid form in the container) or by the simulationuser's assumption (e.g., the state of a valve) . A booleanparameter that is not independent is dependent.

Thelogicaldependencies between boolean parameters define an order-ing relation that can be used as the order of computation forthem .

The overall structure of the code produced by the com-piler in steps 3.3 through 3.5 is the same as in SIMGENMx2. The only change in step 3.6 is taking into account thesimplifications in the boolean parameters, which is simple sowe ignore it here . In evolvers, the effects of direct influ-ences are calculated first to estimate derivatives, the de-pendent numerical parameters are then updated, followed bythe boolean parameters . The main impact of restricted in-ferencing in writing evolvers arises in selecting quantitativemodels for updating dependent numerical parameters . Forinstance, a domain theory may have two quantitative modelsfor the level of liquid as a function of mass, depending on

whether the container is cylindrical or rectangular. If thecompiler knows the shape of the container (via symbolicevaluation) it can install the appropriate model, otherwise itmust provide both models in the simulator and write a run-time conditional . The compiler must also handle models inwhich the equations governing a quantity vary over time . InSIMGEN MK2 these cases were handled by using influenceresolution to see what combinations of qualitative propor-tionalities could co-occur, constructing appropriate quantita-tive models for each combination or signaling a compile-time error if the domain theory failed to include an appro-priate quantitative model. In SIMGEN MK3 we instead re-trieve all the quantitative models for a parameter not ruledout via symbolic evaluation and write code that selects therelevant model based on evaluating the models' antecedentsin the current state vector . The evolver code includes a testfor none of the known models being relevant, and generatesa run-time error in such cases .

In generating transition finders, restricted inference canlead to moot run-time tests, corresponding to physically im-possible transitions . However, symbolic evaluation catchesmost of them, and this is not a serious drawback becauseinequality tests are very cheap. Generating nogood checkersis simplified : Previous compilers generated code based onATMS nogoods to detect impossible behaviors. Much effortwas wasted filtering the nogoods, since the vast majority ofthem were transitivity violations, which are irrelevant whenordinal relations are computed from known numerical pa-rameters . SIMGEN Mx3 simply uses the symbolic evalua-tion procedure to see what ordinal relations are known to beimpossible and test for those.

4.

Complexity AnalysisA detailed complexity analysis is beyond the scope of

this paper (but see [19]) ; here we settle for proving that thealgorithm is polynomial .Step 1 : The only exponential behavior in previous compil-ers occurred in this step, so its complexity is crucial. Thetime complexity is the sum of the time to instantiate modelfragments and the time to draw conclusions with them . Thecost of instantiation can be decomposed into two factors:The cost of pattern matching, and the size of the descriptionproduced by the operation of the system's rules. The cost ofpattern-matching is polynomial in the number of antecedents[16] . We assume that both the scenario model and the do-main theory are finite, and that the number of new entitiesintroduced by the domain theory for any scenario model is apolynomial function of the size of the scenario model. Do-main theories with exponential, or even unbounded, creativ-ity are possible in theory [18], but never appear in practice .

The number of clauses instantiated about any particularstatement is bounded by apolynomial, since it is a functionof (a) the number of relevant domain theory statements,which is always small and certainly independent of the sizeof the scenario and (b) the number of entities introduced ispolynomial . Since the work of instantiation is the product ofthe number of instantiations and the work to perform each,the instantiation process is polynomial-time. Furthermore,the dependency network so created is polynomial in size, asa function of the size of the domain theory and scenariomodel. This means that the cost of inference remains poly-


Table 1: Compilation times (in seconds) for SIMGEN MK2 vsMK3 on standard test examples (IBM RS/6000 Model 530,128MB RAM, Lucid Common Lisp 4.01)

nomial in these factors, since we use an LTMS, for whichthe cost of inference is worst-case linear in the size of thedependency network [16] . We thus conclude that the timeand space complexity of this step is polynomial.Step 2 : Most of the work in this step consists of fetchinginformation from the TGIZMO database and constructingcorresponding internal compiler datastructures, which isobviously polynomial time . The only other potentiallyexpensive part of this computation is the symbolic evalua-tion procedure. Symbolic evaluation of a ground term isperformed by checking its LTMS label, which is constanttime . Symbolic evaluation of a compound expression is arecursive analysis of the structure of the expression, endingin ground terms. The size of expressions is determined bythe antecedents in the domain theory, and thus for any do-main model a maximum size can be found for such expres-sions independent of the size of the scenario model. Ergosymbolic evaluation is also polynomial in the size of thescenario model.

Step 3: Each of these computations involves simplepolynomial-time operations (see [2]), the most expensivebeing sorting the numerical parameters via the causal order-ing, sorting the boolean parameters via logical dependen-cies, and computing the equivalence classes for booleanparameters . All of these are simple polynomial-time opera-tions, operating over datastructures whose size is polynomialin the initial scenario description, so they are polynomialtime as well .

Exam tle Mk2 Mk3Twocontainers 22.8 6.42Boilinm water 25 .2 5.32S , rin, -/Block 6.4 1.85

3?3 container grid 16286 54

Table 2 : Results ofSIMGEN MK3 on n ? n Manhattan grid(IBM RS/6000 Model 350, 64MB RAM, Lucid Common Lisp4.01)

5.

Empirical ResultsSIMGEN Mx3 is fully implemented, and has been tested

successfully on the suite of examples described in [2] . In allcases it is substantially faster than SIMGEN Mx2, as Table1 shows. The simulators it produces, like those of SIMGENMx2, operate at basically the speed of a traditional numeri-cal simulator, with the only extra run-time overhead beingthe maintenance of aconcise history for explanation genera-tion . Currently the compiler's output is Common Lisp with-out any numerical declarations, and even with this perform-ance handicap, the simulators it produces run quite well oneven small machines (i .e ., Macintosh Powerbooks).

To empirically demonstrate that SIMGEN Mx3's per-formance is polynomial time, we generated a set of test ex-amples similar to those used in [2] . That is, a scenario de-scription of size n consists of an n by n grid of containers,connected in Manhattan fashion by fluid paths. Figure 3illustrates for the three by three case . We generated a se-quence of scenario descriptions, with nranging from 2 to 10 .(The reason we chose 10 as an upper bound is that the simu-lator which results contains just over 2,400 parameters,which is roughly three times the size of the STEAMER en-gine room numerical model [20] .) Extending the domaintheory in [16], contained liquids include mass, volume,level, pressure, internal energy, and temperature as dynami-cal parameters, as well as other static parameters (e .g ., boil-ing temperature, specific heat, density) . Containers can beeither cylindrical or rectangular, with appropriate numericaldimensions in each case . The liquid flow process affectsboth mass and internal energy . We then ran the compiler toproduce simulators for each scenario, to see how its per-formance scaled. The results are show in Table 2. In an n ?n grid scenario, there are nZ containers and 2[n2-n] fluidpaths, so the numbers of parts in these examples ranges from8 to 280. The count for quantities includes both static anddynamic parameters, and the count for booleans includesboth conditions controllable by the user (e .g ., the state ofvalves) and qualitative state parameters, such as whether ornot a particular physical process is occurring. The proposi-tion count is the number of statements in the simulator'sexplanation system.

The theoretical analysis in previous sections suggeststhat the compile time should be polynomial in the number of


parts in the system . A least-squares analysis indicates thatthis is correct: A quadratic model (0.017PZ + 0.399P +4.586, where P is the number of containers and paths) fitsthis data nicely, with ? z = 0.03. Additional evidence forquadratic performance is found in Table 3, which shows thecompiler's performance on examples constructed out of lin-ear chains of containers . A chain of length N has 2N-1 parts,i.e ., N containers and N-1 fluid paths. Figure 4 illustrates . Aleast-squares analysis indicates again that a quadratic model(0.018Pz + 0.554P + 0.228, where P is the number of con-tainers and paths) fits this data well, with ? z = 0.004 .

Figure 4: Alinear chain of containers, three long.

6. Tradeoffs in Self-Explanatory Simulators:Compilers versus InterpretersDifferent applications entail different tradeoffs : Some poten-tial users have powerful workstations and can afford the bestcommercial software (e .g ., many engineering organizations),and some potential users have only hand-me-down com-puters and publicly available software (e .g ., most USschools) . Here we examine tradeoffs in self-explanatorysimulation methods with respect to potential applications .

Broadly speaking, the computations associated with self-explanatory simulations can be divided into three types: (1)model instantiation, in which the first-order domain theory

Table 3: SIMGEN Mk 3 data, linear chain of containers (IBMRS/6000, 64MB RAM, Lucid Common Lisp 4.01)

is applied to the ground scenario description, (2) modeltranslation, in which the equations associated with a stateare identified, analyzed, and converted into an executableform, and (3) model execution, in which numeric integration

GridSize

#quantities

#booleans

#propositions

Compiletime (sec)

2 83 24 456 63 198 63 1131 194 363 120 2108 495 578 195 3387 1056 843 288 4968 2027 958 399 6851 3568 1523 528 9036 5869 1938 675 11523 92710 2403 840 14312 1429

N #quantities

#booleans

#propositions

CompileTime (sec)

2 38 9 194 2.053 58 15 305 3 .434 78 21 416 5 .025 98 27 527 6.666 118 33 638 8 .727 138 39 749 10 .58 158 45 860 12 .59 178 51 971 14 .810 198 57 1082 17 .811 218 63 1193 19 .912 238 69 1304 22 .613 258 75 1415 25 .614 278 81 1526 28 .415 298 87 1637 31 .916 318 93 1748 35 .1

is used to derive behavior descriptions from initial values .The choice of compiler versus interpreter is mainly achoiceof how to apportion these computations, with tradeoffsanalogous to those of programming language interpretersand compilers . Interpreters are more suited for highly inter-active circumstances, where more effort is spent changingmodels than running them. Exploratory and rapid-prototyping environments for scientists and engineersformulating and testing models of new phenomena, andhighly interactive construction kit simulation environmentsfor education may be two such applications . Compilers aremore suitable for circumstances where the additional cost ofcompilation is offset by repeated use of the model, or whenthe environment for model execution cannot support theresources required by the development environment.Compilers seem to have the edge in engineering analysis anddesign, where a small number of models are used manytimes (e.g ., in numerical optimization), and most educationalsoftware and training simulators, where maximumperformance must be squeezed out of available hardware.

The cost of model generation is dominated by the ex-pressiveness of modeling language and the amount of simu-lator optimization performed. In SIMGEN Mk3, the order ofcomputation is specified as an inherent part of the domaintheory due to the causal ordering imposed by QualitativeProcess theory influences . Thus, no algebraic manipulationis required at model generation time. Other systems allow adomain theory to contain equations in an arbitrary form .Thus, the equations must be sorted (using a causal orderingalgorithm [7]) and symbolically reformulated to match thatsort. This technique provides the ease of using arbitrarilyarithmetic expressions, but can lead to expensive processingfor some classes of equations. [KDF2] Furthermore, the timetaken to switch models (0 .1 seconds for a small model on afast workstation) even with PIKA's incremental constraintalgorithm suggests that switching delays for large models(e.g., training simulators) could be unacceptable.

Another way in which the modeling language affects po-tential applications is in the kinds of explanations that can begenerated. Domain theories that explicitly represent con-ceptual entities as well as equations can provide better ex-planations than those which do not. While in afew domains(e.g., electronics) expert causal intuitions are not stronglydirectional, in many domains (e.g ., fluids, mechanics, ther-modynamics, chemistry, etc.) expert causal intuitions arestrongly directed [21], and there is no a priori guarantee thatthe accounts produced by causal ordering will match expertintuitions [22] .

Using equation-based models reduces theoverhead of formalizing expert intuitions, but at the cost ofreduced explanation quality. Using explicit qualitative rep-resentations provides an additional layer of explanations, butat the cost of increased domain theory development time .Interestingly, TGIZMO accounts for less than 15% ofSIMGEN MK3's time, so the penalty for using rich, compo-sitional domain theories appears to be quite small.

7. DiscussionPrevious work on self-explanatory simulation producedsoftware that could compile systems up to a few hundredparameters . This paper describes a new algorithm for com-piling self-explanatory simulators that extends the technol-


ogy to systems involving thousands of parameters . We haveshown, both theoretically and empirically, that self-explanatory simulators can be compiled in polynomial time,as a function of the size of the input description and the do-main theory . This advance was made possible by the obser-vation that minimizing inference could substantially improveperformance [4]. These gains are not without costs:SIMGEN MK3 does less self-monitoring and less compile-time error detection than previous versions, and the simula-tors produced can contain dead code . However, no explana-tory capability is lost, and the ability to scale up to verylarge systems outweighs these drawbacks for most applica-tions . Even our current research implementation ofSIMGEN MK3 can, running on a PowerBook, compile newsimulators for small systems reasonably quickly.

One open question concerns the possibility of recoveringmost, if not all, of the self-monitoring and error checking ofprevious compilers by the judicious use of hints . Many pro-gramming language compilers accept advice in the form ofdeclarations . Qualitative representations can be viewed asdeclarations, providing advice to self-explanatory simulatorsat the level of physics and mathematics rather than code .Perhaps domain-specific and example-specific hints couldreplace the functionality provided by inference in earliercompilers.

We now believe that the remaining hurdles to using self-explanatory simulators in applications are building domaintheories and software engineering. We are working on twoapplications . First, we are building an articulate virtuallaboratory for engineering thermodynamics, containing thekinds of components used in building power plants, refrig-erators, and heat pumps, using a domain theory developed incollaboration with an expert in thermodynamics [9] . Second,we are also developing a tool for building training simula-tors, such as a self-explanatory simulator for a shipboardpropulsion plant, to finally fulfill one of the early goals ofqualitative physics [23] .

8. AcknowledgmentsThis research was supported by grants from NASA LangleyResearch Center and from the Computer Science Division ofthe Office of Naval Research . We thank Franz Amador forsupplying us with a sample PIKA domain theory .

9. References

1 Forbus, K. and Falkenhainer, B . Self-explanatory simula-tions: An integration of qualitative and quantitative knowl-edge, Proceedings ofAAAI-90.

2 Forbus, K. and Falkenhainer, B . Self-Explanatory Simula-tions: Scaling up to large models, Proceedings ofAAAI-92.

3 Iwasaki, Y. & Low, C. Model generation and simulationof device behavior with continuous and discrete changes.Intelligent Systems Engineering, 1(2), 1993 .

4 Amador, F., Finkelstein, A. and Weld, D. Real-time self-explanatory simulation . Proceedings ofAAAI-93.

5 Forbus, K. Towards Tutor Compilers: Self-explanatorysimulations as an enabling technology, Proceedings of theThird International Conference on the Learning Sciences,

August, 1991 .6 Neville, D., Notkin, D ., Salesin, D., Salisbury, M.,Sherman, J., Sun, Y., Weld, D. and Winkenbach, G. Elec-tronic 'How Things Work' Articles : A Preliminary Report .IEEE Transactions on Knowledge and Data Engineering,August 1993 .

7 Gautier, P. and Gruber, T. Generating explanations of de-vice behavior using compositional modeling and causalordering . Proceedings ofAAAI-93.

8 Forbus, K. Self-Explanatory Simulators : Making com-puters partners in the modeling process. In Carrete, N. P.& Singh, M.G . (Eds .), Qualitative Reasoning and Deci-sion Technologies, CIMNE, Barcelona, Spain, 1993 .

9 Forbus, K. and Whalley, P. (1994) Using qualitative phys-ics to build articulate software for thermodynamics educa-tion . Proceedings ofAAAI-94, Seattle.

10 Sgouros, N. Integrating qualitative and numerical modelsin binary distillation column design, Proceedings of the1992 AAAI Fall Symposium on Design of Physical Sys-tems, October, 1992 .

11Amador, F. 1994. Self-Explanatory Simulation for anElectronic Encyclopedia . Ph.D . dissertation, Departmentof Computer Science and Engineering, University ofWashington .

12Forbus, K. The Qualitative Process Engine . In Weld, D.and de Kleer, J. (Eds .) Readings in qualitative reasoningabout the physical systems, Morgan-Kaufmann, 1990, pp220-235 .

13de Meer, J. An assumption-based truth maintenance system .Artificial Intelligence, 28(1986) : 127--162 .

14 DeCoste, D . and Collins, J. CATMS: An ATMS whichavoids label explosions . Proceedings ofAAAI91 .

15 Forbus, K. Qualitative Process theory. Artificial Intelli-gence, 24, 1984

16 Forbus, K. and de Kleer, J. Building Problem Solvers,MIT Press, 1993 .

17 Woods, E. The Hybrid Phenomena theory . In Proceed-ings ofIJCAI-91, Sydney, Australia.

18 Forbus, K. Pushing the edge of the (QP) envelope . InRecent Progress in Qualitative Physics, Faltings, B. andStruss, P. (Eds.), MIT Press, 1992 .

19 Forbus, K. and Falkenhainer, B . Self-explanatory simula-tors . Manuscript in preparation.

20 Roberts, B. and Forbus, K. The STEAMER mathematicalsimulation . BBNTechnical Report No . 4625, 1981 .

21 Forbus, K. and Gentner, D. Causal reasoning aboutquantities . Proceedings of the Eighth annual conference ofthe Cognitive Science Society, Amherst, Mass., August,1986

22 Skorstad, G. Finding stable causal interpretations ofequations . In Faltings, B. and Struss, P. (Eds.), Recent ad-vances in qualitative physics, MIT Press, 1992 .

23 Hollan, J., Hutchins, E., & Weitzman, L. STEAMER: Aninteractive inspectable simulation-based training system .AI Magazine, 5(2), 15-27.


scalingupself-explanatory simulators: polynomial-timecompilation · 2007. 5. 2. · simgen mklto...

Documents