an experimental study of spreadsheet presentation and error

Proceedings of the 29th Annual Hawaii International Conference on SJlstem Sciences - 1996

An Experimental Study of Spreadsheet Presentation and Error Detection

Dennis F. Galletta Kathleen S. Hat-tzel Susan Johnson Jimmie Joseph

University of Pittsburgh Joseph M. Katz Graduate School of Business

Pittsburgh, PA 15260 USA

Sandeep Rustagi

Abstract Spreadsheet Error Research

Several well-founded concerns exist about the integrity and validity of electronic spreadsheets. One hundred thirteen MBA students sought eight errors planted in a single-page spreadsheet to discover if differences in the presentation format would facilitate error-finding performance. Five presentation formats were used. Spreadsheets were presented on the screen, both with and without formulas. Spreadsheets were also presented on paper with a list offormulas attached, or without formulas. A new integrated forinula paper treatment was introduced, with formulas presented in each cell directly under each calculated value. Subjects found, on average, only about 50% of the errors across all presentation

formats. The on-screen treatments were clearly inferior to the paper treatments, whether or not formulas were presented. Practitioners should be aware of the d&Gxlties in jnding even simple errors, especially on-screen, and should develop training programs to facilitate spreadsheet auditors ’ performance.

Many of the widely-reported problems in the management of end-user computing [I, 16, 251 are symptomatic of inadequacies in assigning roles to end-users and developers [ 1.51. Users face a confusing environment [6, 23, 321, employing tools that are often harder to use than programming languages [34].

Perhaps the ease with which users can perform basic functions [22] creates overconfidence in performing advanced functions or performing basic functions on a large or complex spreadsheet. Indeed, Brown & Gould [7] report users’ strong self-confidence in their spreadsheet work.

Confidence aside, laboratory and field studies [7, lo] demonstrate that somewhere between one-third and one-half of all spreadsheets contain errors. It is relatively easy to find reports of serious errors in spreadsheets sampled from actual practice [9, 111. Error severity can be quite high; errors in actual spreadsheet usage are known to have caused losses of hundreds of thousands [21] to millions [ 11, 131 of dollars.

Introduction

Electronic spreadsheet software continues to be widely used by knowledge workers. Spreadsheet software is a central component in the offerings of many vendors, and competition is intense. However, the recent concern over CPU errors in floating point calculations, even though reportedly rare, has raised everyone’s awareness about the potential impact of errors on decision making. Errors can originate from the most unexpected sources, and manifest themselves in the most unexpected ways. Because we have no reliable method of preventing or predicting these errors, we will need to focus on error detection for the foreseeable future.

Researchers (e.g., [3, 281) and practitioners (e.g., [2, 31, 331) have devoted considerable effort to describing how errors are made while spreadsheets are being built, estimating their effects, and prescribing techniques or software to prevent them.

While the most desirable outcome would be to prevent the errors, there is presently no assurance that it will be possible to eradicate them [19]. Although spreadsheet auditing programs and features are available, the wide variation in the typical functions modeled by users reduces their effectiveness. Training users to be more careful can be explored by firms, but the wide variation in spreadsheet usage, and the propensity of firms to cut back on training expenditures in difficult economic times, tarnish this approach.

Error detection can be affected by several factors. This study examines the effects of varying the method of spreadsheet presentation on users’ error detection performance.

Although spreadsheet audit software continues to improve, automatic detection of all possible errors is not on the horizon. We should therefore devote significant attention to detection of spreadsheet errors. Presently, there is a paucity of research in the empirical literature in this area.

1060-3425/96 $5.00 0 1996 IEEE 336

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 © 1996 IEEE

Proceedings of the 29th Annuul Hawaii International Conference on System Sciences - 1996

Error detection appears to be extremely difficult. In one recent study [14], it was found that even experienced CPAs had difficulty finding accounting errors in simple spreadsheets presented simultaneously on a computer screen and paper. Although the participants considered the errors obvious when they were specifically pointed out, subjects could only find 50% of the errors. This rate appears consistent with other spreadsheet studies [29, JO] and with professional programmers’ 33% to 46% error-finding rates in the programming literature [4, 261. Panko [30] indicates that errors caused by subjects themselves might be more difficult to fmd than the errors of others; only 16% found their own errors.

We then turn to the question of how to improve subjects’ performance; if as many as half of all spreadsheets contain serious errors, and only half are detected, such an inquiry appears urgent. It is important to identify factors that might affect error-finding performance.

To that end, a conceptual model appears in Figure 1. According to the model, individual factors, presentation factors, error factors, and external factors might affect error-finding performance. Each set will be discussed in turn.

Two individual factors examined in a previous study [ 141 include the spreadsheet auditors’ experience (or expertise) in the domain of the problem (for example, cost accounting in a pricing model or competitive intelligence in a :market position analysis), and in the spreadsheet device (software) itself. A host of other individual factors might be important, such as skill or motivation to perform the task.

Another set of factors, that might improve error-finding performance is the presentation of the information. These factors are investigated in this study. There are no firm guidelines on how spreadsheet auditors should examine the spreadsheets. To the best of the authors’ knowledge, researchers have not yet posed some important, yet basic questions: (1) Should spreadsheet auditors print spreadsheets or view them on the screen? (2) Is it helpful to have access to formulas as they are printed or displayed by typical packages? (3) If formulas are printed, are they more helpful if they are placed contiguous to the results in each cell?

There is some guidance on the first question in the literature on reading from paper versus CRT screens. A series of studies attempted to discover why reading from paper was generally about 20% to 30% faster and more effective than reading from a typical CRT screen [ 12, 17, 18, 24, 271. The long-established differential between the presentation formats was explained only by the quality of the text, and disappeared when a special anti-aliasing display was used. The Gould et al. studies employed several experiments that examined a host of other variables, including brightness, contrast, visual angle, distance, polarity, and paper/screen angle, and concluded that only the quality was important: Only if the text appeared continuous, and did not contain jagged curves typical of a matrix of dots, did reading speed and comprehension improve.

Two other sets of factors will be examined in future studies in our program of research. Error factors might include the prominence of the error; the complexity of the spreadsheet or formula containing the error; and the quantity

Individual Factors - domain

experience - device

experience - skill - motivation

Factors (with some examples) \

-- Presentation Factors * - screen vs

paper - location of

formulas

Error Factors

- prominence - complexity - quantity

* This study

\ Figure 1. Conceptual Model

- type

External Factors

- time pressure - desired

accuracy - supervision - uncertainty

I Error-Finding Performance

337


Proceedings of the 29th Annual Hawaii International Conference on System Sciences - 1996

of errors. Errors might also be classified into types, some of which might be easier to detect than others. Our previous study [14] differentiated domain (accounting) and device (spreadsheet) errors, but only as they matched the experience of the subjects. Other types that might be included in future studies are typographical errors (data entry or formula keystroke slips) or labeling errors (items in incorrect positions based on the indicated structure and/or meaning).

External factors might include time pressure, desired accuracy, the extent of supervision, and overall uncertainty in the spreadsheet. Uncertainty can diminish the extent to which items in a model are correct or incorrect, and thus items that are indeed incorrect might be masked by ambiguity in the bulk of the spreadsheet.

print a list of formulas that can be appended to the spreadsheet for later reference. Using either technique, formulas are hidden from the user, requiring several keystrokes and/or references to multiple pages to find them. Because each keystroke and turn of the page will take time [8], we can predict greater efficiency if formulas are integrated into the presentation of the model. Therefore, we create an experimental “integrated’ paper design (see Appendix A) that presents formulas directly in the appropriate cell below the result of computation, and expect this new integrated paper condition to facilitate error-finding performance.

As stated above, we focus this study on a single set of variables, the presentation factors. Future studies will investigate combinations of factors to determine any interactions that might exist.

H3a: More errors will be found when spreadsheet formulas are integrated in the spreadsheet than when they are not.

H3b: Errors will be found more quickly when spreadsheet formulas are integrated in the spreadsheet than when they are not.

Hypotheses Method

We predict that reviewing a spreadsheet presented on paper will result in a larger number of errors found, in less time than reviewing a spreadsheet presented on a screen. Support for this proposition comes from the body of screen versus paper studies [ 12, 17, 18,24,27].

A laboratory experiment was conducted to identify error finding performance variations resulting from different Lotus l-2-3’ spreadsheet formats and modes of presentation. Individual performance was measured in each of five conditions (see Table 1).

Hla: More errors will be found when reviewing a spreadsheet presented on paper than on a screen.

Hlb: Errors will be found more quickly when a spreadsheet is presented on paper than on a screen.

Spreadsheet formulas are useful aids in finding errors; they present an “audit trail” to the user. Reviewing a formula might even make the error obvious without tabulating long lists of results. Therefore, we propose that presenting formulas to users will aid in their performance in error-finding.

H2a: More errors will be found when spreadsheet formulas are available than when they are not.

H2b: Errors will be found more quickly when spreadsheet formulas are available than when they are not.

The following treatments were created: Treatment 1 (PN) included a spreadsheet printed on paper (see Appendix A) with no formulas available to the subjects. Treatment 2 (PI) presented a new formula-integrated spreadsheet created by the researchers (Appendix B), who inserted the formulas in small print directly under the resultant value in each cell. This format allowed the spreadsheet user to examine both the value and the formula by which it was derived in close proximity. Treatment 3 (PS) included a listing of formulas printed automatically by Lotus l-2-3 (Appendix C). Treatments 4 and 5 (SN and SF, respectively) included screen representations of the basic spreadsheet (similar to Appendix B), only the researchers removed the formulas in SN and retained them in SF.

Subjects

Finally, it is expected that formulas that are contiguous to the results will be more useful to subjects than formulas that are hidden. Usually, formulas are hidden on screen spreadsheets, requiring the user to position the cursor in a particular cell to see its formula. On paper, the problem is more difficult; some spreadsheet packages allow the user to ~___-

The subjects were 11.5 volunteers obtained from nearly 250 candidates enrolled in five introductory graduate MIS MBA classes at a large university in the northeastern U.S. After 2 subjects who performed the task were disqualified for failing to follow instructions, 113 subjects remained. A cash prize of $5 was awarded to the top one-third of participants in each condition. For hypothesis testing, performance was

Lotus I-2-3 is a registered trademark of Lotus Development

338



Table 1 - The 5 Treatments ( 1 Abbreviation (Treatment (Description -1

Paper copy of spreadsheet; separate list of formulas attached (standard Lotus l-2-3 treatment) 1 spreadsheet New treatment with formulas integrated into the printed

3 I I pn Paper; no formulas Paper report of spreadsheet results only (normal approach to distributing spreadsheet results) I On-screen spreadsheet but with formulas removed; only cell results

assessed using both the number of errors correctly identified and also the amount of t ime required to complete the: task. A third, exploratory measure assessed the number of incorrectly-identified errors. That is, we also noted how many false positives there were (correct formulas identified by subjects as errors).

Materials

The task was to find 8 errors planted in a simple, one-page spreadsheet that would allow incoming students to do cash budgeting for three terms (see Appendix A). This task was chosen because it was expected to be familiar to the student participants. The errors ranged from values improperly entered from the assumption sheet to errors in entering formulas. As in a previous study [ 141, the impact of each error was tightly constrained to enable scoring of the results. Materials were pilot-tested and modifications to the planted errors, instructions, consent form, and exit questionnaire were made.

Answer sheets contained 1.5 spaces where the location of the error, the time the error was found, and a brief deiscription of the error could be recorded. To make sure subjects had no idea of the actual number of errors planted on the spreadsheet (8), they were told that a “great deal of extra space” (to fill an entire page) was provided.

An overhead projection panel used for all subjects displayed the elapsed time (to the second). All students had the option to use a calculator provided by the researchers. The subjects in the computer conditions used 486, 33 MHz networked AT&T PCs running Lotus I-2-3 version 5 for Windows’, and 14-inch color super-VGA monitors.

2 Windows is a registered trade mark of Microsoft, Inc

Experimental Procedures

Just before class was dismissed 30 minutes early by the cooperating professor, the first author entered the classroom, described in general terms the purpose of the study, and asked for volunteers to serve as experimental subjects. The cooperating professor provided a great deal of encouragement, but about 20% of the students immediately left the room. Those who stayed behind were asked to read portions of the instructions. The experimenter paused from time to time to review those instructions before going on. Subjects were reminded that performance scores would be determined by assessing both the number of errors correctly identified and the speed at which the errors were found. Finally, the subjects were randomly assigned to one of the 5 conditions and the computer-assigned subjects were asked to follow the experimenter to a computer lab. In the first of the five processions, several students (another 20%) left the line that headed to the lab, and fled downstairs to a nearby lounge. In the remaining four sessions, we supplied a second usher near the end of the line to stand innocently in front of the staircase and make it slightly less convenient to escape the experiment.

In all conditions, a researcher instructed the participants to begin the task and then immediately started the LCD-projected timer. Subjects not only recorded the error location and current time displayed on the screen, but also a very brief description of the error so that those scoring the answer sheets could confirm that an error was actually identified. Subjects were told to make the error descriptions as brief as possible to save time.

Subjects were also instructed to answer an exit questionnaire when they felt they had found all the errors they could. The exit questionnaire first asked subjects to record the final t ime for completing the task. Then the

339



Table 2 - Error-Finding Performance: Cell Means for Each Treatment Treatment Number of Subjects Mean (std dev) number of Mean (std dev) Time Taken

Errors Found out of 8 in minutes

PF (Paper + formulas attached) 23 4.2 (1.6) 15.0 (5.4)

PI (Paper + formulas integrated) 22 4.4 (1.8) 15.4 (3.4)

PN (Paper; no formulas) 23 4.6 (1.6) 15.6 (3.8)

SN (Screen; no formulas) 22 3.2 (1.3) 12.2 (5.3)

SF (Screen; formulas intact) 23 3.9 (2.0) 13.8 (6.0)

questionnaire was used to collect demographic (described below) and other data.

Coding of outcomes

Two judges scored the results; both assigned initial scores for each subject without knowledge of the subjects’ experimental condition or the other judge’s score. A third judge compared the findings of the two original coders. Of the 904 judgments (8 for each of 113 subjects), the raters only disagreed on 6 occasions. The third judge analyzed the 6 discrepancies, and served as a “tie breaker,” determining the proper coding of each item. The agreement between the two judges was over 99%, and Cohen’s Kappa, a measure of interrater reliability, was also over .99 [5].

Results

A preliminary ANOVA was used to determine if there were any differences in demographic variables between the groups, which would indicate a bias in random assignment of subjects. Experience variables included in the examination included experience entering data, using a computer, modeling data relationships, auditing for error-fmding, and training users. Other variables included gender, age, foreign status, and English as a primary language. ANOVA revealed no significant differences in any of the variables among the five groups.

Table 2 summarizes the number of errors found and time taken (up to the last error found) for all five conditions.

To examine the first four hypotheses, a two-way ANOVA was run considering screen vs paper as one factor and formulas vs. no formulas as another factor. Hypotheses la and lb predict that performance will be better for paper subjects (treatments 1 through 3) than for screen subjects (treatments 4 and 5). The ANOVA reveals that only the screen effect is significant, both for number of errors found (p=.OO5) and for the time taken (p=.O44; effect is in the wrong direction). Therefore, there is support for Hypothesis 1 a. The descriptive statistics are shown in Table 3.

Table 3. Cell means (standard deviations), Screen Versus Paper

Hypotheses 2a and 2b predict that performance will be better for subjects who were able to refer to formulas (treatments 1, 2, and 5) than subjects who could not (treatments 3 and 4). The ANOVA failed to reveal any differences in performance based on the presence or absence of formulas, and therefore fails to support Hypotheses 2a and 2b. Table 4 contains the descriptive statistics for formula and non-formula treatments.

Table 4. Cell means (standard deviations), Formulas versus No Formulas

For Hypotheses 3a and 3b, our integrated formula treatment was examined by comparing performance of subjects in treatment 2 to that of subjects in all other treatments (see Table 5). Again, no differences were found, and Hypotheses 3a and 3b are not supported.

Table 5. Cell means (standard deviations), Integrated Formulas versus the Other Treatments

340



Finally, two exploratory analyses were conducted. First, the number of false positives in each cell were computed. Second, an error rate was computed by dividing tlhe time taken by the number of errors found, to derive the number of minutes taken per real error detected. The latter analysis would give us an idea if there was a joint effect of t ime and accuracy (suggesting a trade-off effect). All of the preceding ANOVAs were conducted for each of the two new valriables, and differences were nonsignificant (at the .05 level) for screen versus paper, formula versus no formula. and for _ . integrated formulas versus all other treatments. Table shows the means for each cell.

Table 6. Cell Means (std. dev.) for Exploratory Variables

6

PF (paper formulas)

PI (paper integrated)

PN (paper no formulas)

False Positives

2.0 (3.3)

1.2 (1.5)

1.7 (1.6)

SN (screen no formulas) 2.8 (2.6)

SF (screen formulas) 2.0 (2.8)

One note is in order: the number of false positives in the PI treatment approaches significance when compared against all other treatments. This might suggest that there was not enough statistical power to detect what might be a moderate or weak effect.

Discussion and Conclusions

Our study shows that users who attempt to find errors in spreadsheets are not aided by formulas, but are aided by paper copies of a spreadsheet. Interestingly, only about half of all of the errors are found, indicating a great deal (of room (and perhaps great need) for improvement.

Some speculation might be valuable in interpreting these results. First, the paper versus screen effects seen in previous reading research seem to apply well to spreadsheet error-finding. The lack of findings for providing formulas to subjects might indicate that spreadsheet auditors do not rely on them, or perhaps it takes as long and works as effectively to inspect formulas as it takes to check the math in each computed cell using a calculator or mental calculations.

Although we hoped to contribute a new format for printing spreadsheets for most effective error-checking, the integrated treatment did not seem to facilitate spreadsheet debugging in a material way. We would hesitate to discard our new treatment with haste, however, because the lower number of false positives approached significance cp=.O88). More extensive studies using more subjects, different

spreadsheets, and/or different types of errors are needed to draw well-founded conclusions.

Our spreadsheet was very small by typical standards, and perhaps major facilitative effects of formula contiguity will not materialize in such an artificially easy situation. However, because subjects performed very near the 50% level in their effectiveness, increasing the difficulty level substantially will probably cluster subjects at a very low level of performance, and differences among cells will be artificially attenuated. Tests comparing the means in each cell will therefore be less likely to achieve significance.

Future research should focus on other parts of our conceptual model so that we can build more understanding of the error-finding process. Larger sample sizes will enable researchers to consider as many independent variables as possible at one time. If researchers can build understanding of how errors are found, perhaps some of the recent well-publicized disasters can be averted, and thus diminish further the difficulties of end-user computing.

Note

Please contact the first author at the above address (or [email protected]) for a copy of the experimental materials.

References

1. Alavi, M., Nelson, R. R., and Weiss, I. R. (1987-88). “Strategies for End-User Computing: An Integrative Perspective,” Journal of Management Information Systems, 4(3), 28-49.

2. Anderson, K. and Bernard, A. (1988). “Spreading Mistakes,” Journal of Accounting and EDP, 42-45.

3. Ballou, D.P., Pazer, H.L., Belardo, S., and Klein, B. (1987) “Implications of Data Quality for Spreadsheet Analysis,” Data Base, V. 18, N. 3, Spring, pp. 13-19.

4. Basili, V.R. and Selby, R.W., Jr. (1986). “Four Applications of a Software Data Collection and Analysis Methodology,” in J. K. Skwirzynski (ed.), Sofrware System Design Methods, Berlin: Springer-Verlag, 3-33.

5. Bishop, Y.M.M., Fienberg, S.E., and Holland, P.W., (1975). Discrete Multivariate Analysis: Theory and Practice, MIT Press, Cambridge, Mass.

6. Brancheau, J. C., Vogel, D. R., and Wetherbe, J. C. (1985). “An Investigation of the Information Center from the User’s Perspective,” Data Base, 17(l), Fall 1985,4-18.

7. Brown, P. S., and Gould, J. D. (1987). “An Experimental Study of People Creating Spreadsheets,” ACM Transactions on Office Information Systems, 5(3), 258-272.

341



8. Card, S, Moran, T. & Newell, A. (1983). The Psychology of Human-Computer Interaction, Hillsdale, NJ: Laurence Erlbaum.

9. Cragg, P. B. & King, M. (1993). “Spreadsheet modeling abuse: An opportunity for OR?” Journal of the Operational Research Society, 44(8), pp. 743-752.

10. Creeth, R. (1985). “Microcomputers Spreadsheets: Their Use and Abuse,” Journal ofAccountancy, June 1985, 92.

11. Davies, N. and lkin, C. (1987). “Auditing Spreadsheets,” Australian Accountant, December 1987, 54-56.

12. Dillon, A. (1992). “Reading from paper versus screens: A critical review of the empirical literature.” Ergonomics, 3.5(10), pp. 1297-1326.

13. Ditlea, S. (1987). “Spreadsheets Can be Hazardous to your Health,” Personal Computing, January, 1987, 60-69.

14. Galletta, D.F., Abraham, D., El Louadi, M., Lekse, W., Pollalis, Y., and Sampler, J. (1993). “An Empirical Study of Spreadsheet Error-Finding Performance.” Accounting, Management, and Information Technology, V. 3, N. 2, 1993, pp. 79-95.

15. Galletta, D.F. and Heckman, R. (1990) “A Role Theory Perspective on End-User Development,” information Systems Research, Vl, N2, June, 1990, pp. 168-187.

16. Galletta, D.F. and Hufnagel, E. (1992). “A Model of End-User Computing Policy: Determinants of Organizational Compliance,” Information and Management, 22 1 - 18.

17. Gould, J.D., Alfaro, L., Barnes, V., Finn, R., Grischkowsky, N., and Minuto, A. (1987a). “Reading is Slower from CRT Displays than from Paper: Attempts to Isolate a Single-Variable Explanation,” Human Factors, 29(3), 269-299.

18. Gould, J.D., Alfaro, L., Finn, R., Haupt, B., and Minuto, A. (1987b). “Reading from CRT Displays Can Be as Fast as Reading from Paper,” Human Factors, 29(5), 497-5 17.

19. Hayen, R. L. & Peters, R. M. (1989). “How to ensure spreadsheet integrity.” Management Accounting, 70( lo), pp. 30-33.

20. Hendry, D.G. & Green, T.R.G. (1994). “Creating, Comprehending, and Explaining Spreadsheets: A Cognitive Interpretation of What Discretionary Users Think of the Spreadsheet Model,” International Journal of Human-Computer Studies, 40(6), pp. 1033- 1065.

2 1. Kee, R. C. and Mason, J. O., Jr. (1988). “Preventing Errors in Spreadsheets,” Internal Auditor, February, 1988, pp. 42-47.

22. King, M., Lee, R. A., Piper, J. A. & Whittaker, J. (1991). Information Technology and the Management Accountant. CIMA London.

23. Lederer, A. L. and Sethi, V. (1986). “Planning for End-User Computing: Top-Down or Bottom-Up?’ Information Management Review, Fall 1986.

24. McGoldrick, J. A., Martin, J., Bergering, A. J. & Symons, S. (1992). “Locating discrete information in text: Effects of computer presentation and menu formatting.” Journal of Reading Behavior, 24(l), pp. l-20.

25. Munro, M. C., Huff, S. L., & Moore, G. (1987-88). “Expansion and Control of End-User Computing,” Journal of Management Information Systems, q3), S-27.

26. Myers, G.J. (1978). “A Controlled Experiment in Program Testing and Code Walkthroughs/Inspections,” Communications of the ACM, 2/(9), pp. 760-768.

27. Oliver, R. (1994). “Proof-reading on paper and screens: The influence of practice and experience on performance.” Journal of Computer-Based Instruction, 20(4), pp, 118-124.

28. Olson, J. R. & Nilsen, E. (1987-88). “Analysis of the Cognition Involved in Spreadsheet Software Interaction,” Human-Computer Interaction, 3, 309-349.

29. Panko, R.R. (1995). Hitting the Wall: Development and Debugging Errors in a “Simple” Spreadsheet Problem, Working Paper, Department of Decision Sciences, College of Business Administration, University of Hawaii, 2404 Maile Way, Honolulu, HI 96822

30. Panko, R.R. (1996). “Spreadsheets on Trial: A Survey of Research on Spreadsheet Risks,” Proceedings of the 29th Hawaii International Conference on Systems Sciences, forthcoming.

3 1. Pearson, R. (1988). “Lies, Damned Lies, and Spreadsheets,” Byte, December 1988, pp. 299-304.

32. Rockart, J. F. and Flannery, L. S. (1983). “The Management of End-user Computing.” Communications of the ACM, 26, 776-784.

33. Stang, D. (1987). “Spreadsheet Disasters 1 have Known,” Information Center, 3(11), November 1987, pp. 26-30.

34. Xenakis, J. W. (1987). “l-2-3: A Dangerous Language,” Computerworld, March 9, 1987.

342



Appendix A: The Spreadsheet used in the Task

343



Appendix B: New Formula-Integrated Printed Spreadsheet (reduced in size by about 20%)

15 Insurance

16 Living Costs (other)

17 Food

18 Entertainment

19 Transportation

20 Clothing

A

$53 4 $212 $212 $212 $636 15 +C15’B15 +D15 +D15 @?SUM(D15..Fl5)

16

$330 4 $1,320 $1,346 $1,373 $4,039 17 11’30 +C17*B17 @ R O U N D @ R O U N D @[email protected])

(+D17*(I+SDSI),O) (+E17*(I+SD$l),O)

$150 4 $600 $612 $624 $1,836 18 +C18*B18 @ROUND @ROUND @[email protected])

(+DlS*(I+SDSI),O) (+ElS*(I+SDSl),O)

$40 4 $160 $161 $164 $485 19 +Cl9’B19 @ROUND mom @[email protected])

(+D19+(I+SDSI),O) (+El9*(1+SDSI),O)

$21 4 $80 $82 $84 $250 20 +c20*20 @ROUND @ R O U N D @SUM(CZO..FZO)

(+DZO*(l+SDSI),O) (+EZO*(I+SDSl),O)

l3 C D E F G

344



Appendix C: Spreadsheet Formulas

A:Al: {Page l/l) (G) CWZII 'Cash Budget A:Cl: (Page) CG) CW71 'Assume: A:Dl: (Page) (PO) 0.02 A:El: (Page) CG) t inflation/semester A:D2: "Fall A:EZ: (G) "Spring A:FZ: CC) "Sumner A:G2: (G) liOverall A:A3: (Page) (G) CW211 'Cash, beginning A:D3: (CO) 1000 A:E3: (CO) +D8 A:F3: (CO) +E8 A:A4: <Page) CC) [WZll 'Outflows - School A:D4: (CO) @SUM(DlO..DlZ) A:E4: (CD) @SUM(ElO..ElZ) A:F4: (CO) @SlJM(FlO..FlE)

Cost!5

A:G4: (CO) @SUMiD4..F4) A:A5: <Page) (G) CW211 I Living costs A:D5: (CO) @SUM(D14..D20) A:E5: (CO) @SUMCE14..E20) A:F5: (CO) @SUM(FlZ..F19) A:G5: (CO) @SUMCOS..FS) A:A6: {Page) CG) fW211 'Inflows - Loans A:D6: (CO) 3000 A:E6: (CO) 3000 A:F6: (CO) 3000 A:G6: (CO) @SUMCD6..F6) A:A7: CPase) CG) LW211 ' Support from home A:D7: (CD, @ROUND(lOOO+D4+D5-D3-D6,'-3) A:E7: (CO) @ROUND(1000+E4+E5-E3-E6,-3) A:F7: (CO) @ROUND(lOOO+F4+F5-F3-F6]-3) A:G7: (CO) @SUM(D7..F7) A:AB: {Page) (G) CW211 'Cash, end A:D8: (CO) +D3-D4-D5+Db+D7 A:E8: (CO) +E3-E4-E5+Eb+E7 A:F8: (CO) +F3-F4-E5+Fb+F7 A:GB: (CO) @SUMCDB..FB) A:D9: (CO) \= A:EP: (CO) \= A:F9: (CO) \= A:G9: (CO) \= A:AlO: {Page) (G) CW211 'School (contractual): Tuition A:D10: (CO) 4115 A:ElO: (CO) 4115 A:FlO: (CO) 4115 A:GlO: (CO) @SUM(DlO..FlO) A:All: <Page) CC) CW2l l ' Fees A:Dll: (CO) 53 A:Ell: (CO) 53 A:Fll: (CO) 53 A:Gll: (CO) @SUM(Dll..Fl l)

A:A12: (Page) (G) [WZll 'School (other): books/supplies A:D12: (CO) 300 A:E12: (CO) @ROUND(*D12*(l+SDSl),O) A:F12: (CO) @ROUND(+E12*Cl+SDSl),O) A:G12: (CO) @SUMCD12..F12) A:A13: <Page) (G) CW2l l 'Living (contractual) A:B13: (CO) [USI 'MONTHLY'

iCOj [W71 'MONTHS' A:C13: A:A14: A:B14: A:C14: A:D14: A:E14: A:F14: A:G14: A:A15: A:B15: A:ClS: A:Dl5: A:E15: A:F15: A:G15: A:A16: A:A17: A:B17: A:C17: A:D17: A:E17: A:F17: A:G17: A:A18: A:618: A:C18: A:D18: A:E18: A:F18: A:G18: A:AlP: A:BlP: A:C19: A:DlP: A:E19: A:F19: A:G19: A:A20: A:B20: A:C20: A:D20: A:E20: A:F20:

{Page) (G) [W2ll I Housing (CO) CW81 450 (FO) CW71 4 (CO) +C14*B14 (CO) +D14 (CO) +D14 (CO) @SUM(D14..F14) (Page) (CO) CW211 ' Insurance (CO) tW81 53 (FO) tW71 4 (CO) +C15*B15 (CO) +D15 (CO) +D15 (CO) @SUM(D15..FlS) {Page) (G) IW211 'Living Costs ( (Page) CC) IW211 I Food (CO) IW81 11*3D (FO) CW71 4 (CO) +C17*B17 (CO) @ROUND(+D17*(l+tDSl),O) (CO) @RWND(+E17*(1+SDtl),O)

other)

(CO) @SUM(D17..F17) . (Page) (G) CW2l l ' Entertainment (CO) CWBl 150 (FO) CW71 4 (CO) +C18+618 (CO) @ROUNDC+D18*(1+SDtl),O) (CO) @ROUND(+E18*(1+SDS1),0) (CO) @SUM(D18..F18) CPage) (G) tW211 I Transportation (CO) cwa1 40 (FO) CW71 4 (CO) +C19*B19 (CO) @RWND(+D19+(l+SDS1).0) (CO) @ROUND(+E19*(1+$D$l);Oj (CO) @SUMCDl9..FlP) (Page) CG) tW211 I Clothing (CO) tW81 21 (FO) CW71 4 +c20*20 (CO) @ROUND(+D20*(1+bDSl),O) (CO) @RDUNDC+E2O*(l+SDSl),O)

A:G20: (CO) @SUM(C2O..F20)

345


an experimental study of spreadsheet presentation and error

Documents