Acta Psychologica 58 (1985) 1599172

North-Holland

159

SYSTEMS ANALYSIS AND DYNAMIC DECISION MAKING *

Andrew J. MACKINNON and Alexander J. WEARING

Unir~~rr~.r of Melbourne, Austruliu

Accepted April 1984

Prevtous research in dynamic decision making has failed to produce a unified approach to the area. This has resulted in a collection of experiments whose results are difficult to generalise from the

specific task systems used. It is argued that an approach focusing on the structure of the task system and employing well-known methods of systems analysis can overcome this difficulty. In

addition. this technique can order research and assist in the psychological interpretation of system

parameters.

The use of systems analysis m this role is demonstrated in an experiment using a first order

feedback system. SubJects were required to stabilise the system at a level other than its equilibrium.

The results show that subjects quickly adapt to different values of parameters in the system

boundary. However performance was consistently better in more inertial systems. This suggests

that subjects rely on the task system to integrate over- and underadjustments of their decision

inputs.

Dynamic decision making tasks may be characterised as those in which the decision maker is required to make a sequence of decisions in a tusk system, in which each decision affects it so that the following decision is made in a different state of the system.

Omitting multi-stage gambling tasks (e.g. Rapoport and Jones 1970) Dynamic Decision Making (DDM) tasks have fallen into two cate- gories. The first category consists of tasks that are essentially simula- tions of specific real-life situations (see, for example Darner 1980; Mackinnon and Wearing 1980; Weisbrod et al. 1974). These studies

* An earlier version of this paper was presented at the SPUDM-9 conference in Groningen.

Holland. in September, 1983. The authors are grateful to Berndt Brehmer. Katrin Borcherding, Philippa Pattison, Charles Vlek and Willem Wagenaar for their suggestions, as well as three

anonymous referees whose detailed comments were particularly helpful.

Mailing address: A.J. Wearing, University of Melbourne, Dept. of Psychology, Parkville, Vict.

3052, Australia.

OOOJ-6918/85/$3.30 (3 1985, Elsevier Science Publishers B.V. (North-Holland)

provide information about human performance in the task and give rise to hypotheses about strategies employed, but they suffer from a number of limitations:

(1) It is often difficult to determine the demands which the task places on the decision maker (DM). Is a particular task inherently “dif- ficult” or “easy”? What components of the task are responsible for these effects?

(2) Optimal levels of performance are rarely known. It may be difficult to determine how well (in any absolute sense) subjects are perform- ing.

(3) It may be difficult to generalise from one experiment or compare it with others. This is because the characteristics of the system and the population of systems from which it was drawn may be ill defined.

(4) The extent to which DMs apply or misapply knowledge they have of situations they perceive to be similar to the task is impossible to determine.

In contrast to simulation studies the second category includes the work of Rapoport (1966a, b, 1967, 1975) and others (Kleiter 1970; Ebert 1972) have carried out research using tasks amenable to optimisation by dynamic programming techniques (see Bellman and Dreyfus 1962). Their experiments have focused on the sub-optimality of DMs’ deci- sions. Although optimal decisions are known for these tasks, they suffer from similar problems to simulation oriented studies. The prescription of the optimal solution does not describe the task and so it does provide information about the demands the task places on the DM. The problem of generalisability remains.

Thus whilst DDM would appear to be a relevant area of inquiry in a world in which many significant decisions are sequential and depen- dent, research in the area has been confined to a small number of specific tasks, each of unknown generalisability and exploring a limited sector of the universe of task systems.

Instead, a preferable approach is one which: (I) Enables and encourages a systematic exploration of the universe of

dynamic task systems. (2) Permits a precise and formal description of the degree of difficulty

or complexity of the task.

(3) (4)

A.J. Mackrnnon. A.J. Weuring ,’ Dmm~ic deusion nurkrng 161

Allows tasks to be compared. Is capable of providing “base-line” information of human ability in these tasks.

An approach that focuses on the structural characteristics of the task system can meet these criteria. The mathematical concepts required for such a structural analysis are not novel, the application of systems analysis in engineering being commonplace. The usefulness of systems analysis to the study of DDM will be illustrated by its application to the demonstrably simplest DDM task.

A large number of systems may be represented as combinations of linear operators (Cortes et al. 1974). An example is shown in fig. 1. The output ( Y) of such a system can be expressed as a function of the input (I/) thus:

y= ND) u. P(D)

R(D) is the boundary polynomial of the system and P(D) is its character- istic polynomial. Both are polynomials in the differential operator D (D” = d”,‘dt”).

U System Input (decision) Q Boundary

Xl System Variable R Feedback Constant

Y System Output K Equilibrium Constant

1 Identity Operator

D-’ Integration Operator (inverse of differentiation)

Fig. 1. First-order system represented using linear operators

These polynomials may assist in the classification of DDM tasks. The boundary polynomial includes those elements that are involved only in the “feed-through” (left-to-right in the diagram) of quantities. This may be thought of as the transformation undergone by decisions before they reach the system proper. For example if the boundary polynomial was the differential operator D, the system would respond to the rate of change of input rather than its actual value. The response of humans to many forms of stimulation are an example of this type of system boundary: Constant levels of stimulation are not processed but the system responds to changes in level.

The boundary function of the system in fig. 1 is simply a constant of proportionality, Q. The DM’s input may be amplified or attenuated before it reaches the system proper. An example of this type of boundary function might be found in modelling an aid program. A certain proportion of funds applied to the program will be lost in administrative and other costs before being applied to the program proper. Thus Q would represent the proportion of income remaining for use.

The characteristic polynomial includes those parts of the system involved in its feedback loops. The behaviour of the system over time may be decomposed into two components: The autotzotmus re.~pon.w to the structure of the system and its initial state, independent of subse- quent input and the forced response resulting from the DM’s inputs after they are “filtered” by the system boundary.

The autonomous response is found by setting the filtered input to zero:

0 = YP(D). (2)

In the example:

0= Y(D-R). (3)

The restriction of the system operators to constants of proportionality, the differential operator and its inverse (integration), ensures that the solution of eq. (2) will always be that of a linear homogeneous differen- tial equation with constant coefficients (Pontryagin 1962).

Assuming P(D) has wt distinct roots the solution of the autonomous

A./. Mackrnnon, A.J. Weorrng / Dwmrc dec~von makrng 163

response.),,(t) will be of the form:

.vh( t) = c,e-‘I’ + c2epr2’ + . . . +c,,eC’““, (4)

where u, is thejth root of P(D). The roots may be real or imaginary. Imaginary roots will occur as complex conjugates transformable to the form C,eU’sin( ht + C,). The values of the constant, c,, are determined by initial conditions.

As the autonomous response is independent of the DM’s input it would only be of interest in situations where the DM’s response to a specific preexisting situation is studied. However for linear systems of the type described above the forced response may be derived from the autonomous response. The forced response _Q( t) may be expressed by the convolution integral:

yJt)=~b(t-+I(T) dr. (5)

W(t) is the weighting function. It may be derived from the autonomous response by appropriate choice of initial conditions to correspond to a unit impulse.

The weighting function reveals a latent characteristic of task systems essential to the understanding of decision making within them. It shows the relative impact of previous decisions on the current state of the system; how the system is affected by past “errors”, and the duration of their effective impact.

For values of R greater than zero the system in fig. 1 has an exponentially decreasing weighting function. Thus previous inputs are “forgotten” over time. This phenomena is seen when a foreign sub- stance is eliminated from the bloodstream or when sales of a particular product are boosted as the result of an advertising campaign. In both of these examples a “shock” to the system initially destabilises it. How- ever, over time, the shock is forgotten and the system returns to its normal state. In most situations systems are subject to continuing rather than isolated inputs. Thus it is difficult to determine what effect each previous input has had in producing the state of the system at any point in time. This information is however provided by the weighting function.

Because the weighting functions of linear systems assume a limited number of forms this approach enables task systems to be efficiently classified according to the nature of this “memory”. Specifically, the weighting functions of all linear systems are sums of exponents or sinusoidally modulated exponents.

At a very basic level, systems whose weighting functions decrease with time can be distinguished from those whose weighting functions increase over time. The former class of systems eventually “forget” past decisions while. in the latter, previous decisions (including non-optimal inputs) are amplified over time. Similarly, some systems will have weighting functions that are monotonic whilst others will be non-mono- tonic.

At a finer level, the effect of parametrically varying the weighting function of a particular type of task system can be systematically investigated. In addition, these variations can be given psychological interpretations. For example, the inertial characteristics of decision environments have been perceived as critical: The well-known “in- crementalist” heuristic (Braybrooke and Lindblom 1963) presupposes a system possessing considerable inertia. The effect of system inertia may be systematically assessed by parametrically varying the weighting function.

This analysis can be applied retrospectively to DDM tasks previously employed. Briefly, the analysis of tasks used by Rapoport and others reveals that they have all been characterised by simple, exponentially increasing weighting functions. In contrast, many real world task sys- tems may be characterised by negative feedback loops or combinations of positive and negative feedback (Forrester 1968). This implies a paucity of knowledge about DDM ability in the wider range of possible task systems. It also underlines the necessity for an approach that identifies the response of the task to the DM rather than one that simply prescribes the optimal input.

While systems analysis describes the task, the relevance of this to subjects’ performance may be questioned. Recent advances in control theory shed light on the relation between the structure of the task system and the DM’s internal representation of it. To perform optim- ally in a DDM task it has been shown that simplest representation possible is isomorphic with the task system: This “Internal Model Principle” (Francis and Wonham 1976) has been succinctly summarised by Conant and Ashby (1970) as “Every good regulator of a system

A.J. Mockinnon, A.J. Wearing / DIYUVHK decrsrotz muhing 165

must be a model of that system”. The importance of feedback between the task system and the DM has been suggested by Hogarth (1981). However, the Internal Model Principle shows that feedback per se is insufficient for good performance: The DM must incorporate a model of the task in the feedback loop.

Therefore the weighting function not only describes the task system but defines the information that the DM must apprehend in order to understand and thus control the task system.

An experiment

An advantage of the proposed approach is that it suggests an ordering of the complexity of task systems. The simplest dynamic is a first order feedback system (as shown in fig. 1). It has only one differential operator. This system was used as the task system in a DDM experiment by giving Ss a goal value at which to maintain the system variable. The boundary function is:

R(D) = Q.

and the weighting function is:

The overall behaviour of the system is thus:

Y(t)=K+(Y(0)-K).xe-R’+Q O’iY(T)xe~R(“‘dr. / (6)

where the last term is the forced response of the system. For positive values of R this system will asymptote towards the equilibrium value, K. Whilst this is a very simple, system it represents a fundamental characteristic of many systems that humans attempt to manipulate; biological, chemical and economic. Many more complex systems may be thought of as elaborations of the feedforward and feedback paths within this basic structure. Thus this class of system is not only a mathematically convenient starting place but also focuses an elemental aspect of dynamic decision making.

In keeping with the aim of collecting baseline data about dynamic decision making ability a competing random input was not used. Most other research has involved stochastic tasks, making it difficult to separate the effect of the task from the random variation. Further a stationary goal was chosen. This is in contrast to previous experiments that have specified a nonstationary goal for Ss in addition to using a dynamic task system (see e.g. Kleiter 1970; Rapoport 1966a).

Thus there were four parameters and the goal value that could be varied experimen- tally. To keep the experiment to a manageable size only Q and R were systematically

varied. The equilibrium parameter (K) was set to 200.0 and the initial value ( Y(0)) to 100.0. The goal value was 50.0.

Parameters Q and R were varied in the following manner: Q took the values 0.25 and 0.75. While both these values attenuate the input they were chosen to represent reasonable variation without requiring Ss in different conditions to use inputs of differing orders of magnitude.

R also took two values: One corresponded to the weighting of input decreasing by 50% in two trials. The other corresponded to the same decrease within nine trials. These values were chosen relatively arbitrarily with a view to contrasting a quick decrease in decision importance to a long one. Whilst Ss in the short condition might be expected to remember relevant previous inputs (the immediately preceding one or two) they could not be expected to do so in the long condition. Both values were chosen to be greater than zero. The task system therefore forgot DMs’ past inputs over time. This contrasts to previous research that has employed “explosive” tasks.

Two competing hypotheses can be formed concerning this “ memory” parameter: (1) As the ‘short memory’ system forgets inputs (and thus the decision makers previous errors) more quickly than the long memory condition, better performance should be exhibited in controlling it. On the other hand, whilst (2) the long memory system retains errors. it may act to smooth positive and negative deviations. Thus the long memory system may be easier to control.

Competing hypotheses can also be educed concerning the effect of different values of Q. The larger the value of this parameter the greater the effect of any input on the system. Brehmer’s (1974) work on learning to make inferences in bivariate relationships suggests that the more visible a relationship is. the more accurate predictions become. Higher values of Q would make the effect of input more apparent and thus lead to better performance. However an additional complication, not present in Brehmer’s experiment, occurs here. Because the S’s estimate has an effect on the behaviour of the system. higher values of Q will make the system more sensitive to input. This may reduce performance. The relative importance of visibility versus sensitivity can only be determined experimentally.

Method

Ss were undergraduate students at the University of Melbourne. They were re- cruited by advertisement and paid for their services. In total 38 Ss were tested, however six failed to meet a performance criterion detailed below. Thus statistical results are reported on 32 Ss.

Design The two experimental factors - the value of the boundary and the feedback parame-

ter - were crossed to produce four systems. Eight Ss were randomly assigned to interact with each system.

Each system was realised as a computer model based on the equations above. Ss sat at a visual display unit and made numerical inputs to the system via the keyboard.

A.J. Muckrnnon. A.J. Weuring / Dynamic decisron muking 167

There was no restriction on the range of values Ss could use as inputs. After an input was made an algorithm equivalent to that in fig. 1 calculated the new state of the system and displayed this to the S. In addition the previous system state was also displayed as well as the input that resulted in the change displayed. Ss were not allowed to record previous inputs or system values.

There were 75 trials. In this context a “trial” refers to running the computer model for one unit of simulated time (i.e. the model was not run in real time).

Ss were not told about the characteristics of the system: They were introduced to the concept of dynamic systems using a number of economic. biological and social examples. A variety of examples were chosen so as not to suggest any one pattern of behaviour in the task system. The task was presented as a “black box” whose behaviour Ss were to attempt to understand and control. The output of the system was referred to as variable “X”. Their assigned task was to get the output of the system to attain a value of 50.0 as quickly as possible and to maintain it there.

Responses were not speeded and Ss were encouraged to think about the behaviour of the system before making an input. Ss were made aware that the computer model “stopped” and waited for their input. Thus the situation could not deteriorate whilst they thought about their decision.

Results

A wide range in performance was observed: A number of Ss’ met the criterion within the first half of the experiment while a similar number of Ss’ performance declined during the experiment. The correspondingly large within cells variance meant that subsequent statistical tests had very low power. To counteract this it was decided to exclude from these analyses Ss who showed no evidence of learning or improvement. Thus Ss who maintained an average deviation of greater than 50 units from the goal in the last quarter of the experiment were replaced. Six Ss were replaced. There was no tendency for Ss in any one experimental condition to fail this criterion.

An ANOVA was performed on the RMS [l] deviations from the goal. A third design factor of “Stage” was created by dividing the trials into four blocks. At the same time S’s performance was compared to a “minimal intelligence” (MI) strategy as a more stringent test of differences between the task systems. This strategy involved using the difference between the current value of the system variable and the goal as the input. It was chosen as a reasonable strategy that could be adopted without using any informa- tion unavailable to Ss.

The analysis showed that there was an overall improvement during the experiment (F(3,84) = 12.24, p < 0.01). More importantly there was a significant interaction be- tween the feedback parameter and “Stage” (F(3,84) = 4.80, p < 0.01). Fig. 2 shows that the systems with longer memory were better controlled, particularly during the first quarter of the experiment. Except initially, Ss performed significantly better than the MI strategy.

[II RMS Deviation from the Goal = d( system state - 5O)‘/N. where N is the number of trials.

The boundary parameter (Q) was not significant as a main effect or in interaction with other factors. The interaction between Q and “Stage” is shown in fig. 3. While non-significant (F(3.84) = 2.24. p = 0.09). it suggests that while Ss initially better controlled the systems with a higher boundary parameter (favouring a visibility effect), they soon adapted to this parameter.

A second analysis attempted to examine the dynamics of the S/system interaction. Standard time series analysis (Box and Jenkins 1970) was not found to be helpful in this regard: Variance nonstationarity, possible changes in parameters and difficulty in positive model identification were particular problems. In addition parametric tech- niques were unable to incorporate the relation between the system variable and the goal as a factor.

A discrete approach was adopted: Each input may be classified according to its effect on the system variable (Y ). An input may maintain Y at the goal. cause it to converge towards the goal, diverse away from it or cause Y to cross the goal. The latter three transitions may occur on either side of the goal.

Stepwise log-linear analysis (Bishop et al. 1975) was initially used to model the relationship between contingent transitions and the experimental factors. (In order to maintain cell frequencies the “Stage” factor was reduced to two levels.) Models involving all factors were complex and difficult to interpret. Therefore models were produced separately for each system.

The transitions reveal the basic asymmetry of Ss’ control of the system. While 36% of transitions were ‘Converge Down’ only 6 “7 were ‘Converge Up’. Twenty-two percent

100

80

Short Memory

Long Memory

”

; ;

I

Stage 3 I

Fig. 2. R by stage interaction.

A.J. Muc~kwm. A.J. Wemng ,’ Dpwnic decision muking 169

of transitions were ‘Diverge Up’ compared to only 3% ‘Diverge Down’. This indicates that Ss were continually influenced by the tendency of the system to attain an equilibrium (200) higher than their set goal (50). Approximately the same proportion of over- and undershoots occurred (13%). This suggests that these transitions were important in guiding the system around the goal. However, only slightly more of these transitions occurred in the second half of the experiment than did in the first.

Because of the asymmetry about the goal. transitions above and below it were modelled separately. The log-linear models showed that the same type of process could explain first-order transitions in all systems, namely a stationary first-order process. This model implies that the nature of the control exerted by Ss at any point is dependent only on the previous control situation created by the S. This result is consistent with Rapoport’s (1966a) determination of a two-stage ahead planning horizon in his task. This experiment suggests that the “span of control” is independent of the feedback parameter although the success of the planning may differ between systems.

The pattern of transitions was constant over trials. However, the frequency of each type of zero-order transition did change from the first to the second half of the trials.

The performance of those Ss excluded from the analyses requires explanation. Although no formal protocols were taken there appear to be three reasons for their poor performance. Some of these Ss chose large inputs early in the experiment. Seeing the large change in the state of the system their input produced, they confined themselves to smaller inputs. Thus the system remained a large distance from the goal throughout the experiment. A second group of Ss could be characterised as exhibiting improvement during many of the trials followed by a sharp decline in performance.

100

80

60

40

20

Cl = .25

01 1 2 3 4

Stage

Fig. 3. Q by stage interaction.

This appeared to be due to changing strategies (unsuccessfully) in order to accelerate progress towards the goal.

The failure of a third group of Ss can be traced to their inability to take into account the fact that their behaviour (input) affected that of the system. An example of this was an S who changed from a series of positive to a series of negative inputs. Observing the swing in the behaviour of the system she concluded that the .~~.rtrr~ oscillated and proceeded to input a roughly sinusoidal series in order to track and equilibrate it. The resultant oscillation of the system was perceived as evidence for her original hypothesis!

Discussion

The simple experiment described here was intended to illustrate the use of adapting a systems theoretical approach to dynamic decision mak- ing, and the results demonstrate the information to be gained by manipulating system parameters. In particular, it has been shown that performance is differentially affected by system parameters in a way that is psychologically interpretable: Subjects could quickly adapt their inputs to different boundary parameters, but the effect of the inertial characteristics of the system were relatively permanent. Whilst it has previously been speculated (Hogarth 1981) that feedback is important in sequential tasks, this experiment clarifies the role of the task system in this process. Subjects rely on the inertia of the system to damp their under- and overadjustments.

Both the task system and the subjects’ task were deliberately chosen to be simple. Whilst this renders the results interpretable the question must arise as to the generalisability of these results to practical prob- lems. Firstly it should be noted that the task (which might be sum- marised as destabilising a homeostatic system) has counterparts in medicine, industry and economics. Whilst some of these systems have more complex structures the approach demonstrated here enables the source of any difficulty subjects have in controlling the task system to be identified. This experiment has shown that the basic homeostatic mechanism is not likely to cause subjects difficulty in dynamic decision making tasks. Failure in more complex task can thus be attributed to the complexiiving elements of these tasks.

As the task was presented as an abstract system the possibility of context specific effects must be considered. Firstly it should be recog- nised that this mode of presentation represents the hardest test of

subjects ability. It shows that they are capable of progressing from a state of no real knowledge of the system to one that enables them to control its behaviour solely on the basis of the system’s reaction to their input. In many situations, instruction, familiarity and analogy may make dynamic tasks easier.

Simon and Hayes (1976) have found context effects in multistage problems. Therefore it is possible that differences in behaviour may be exhibited between task systems having the same structure but represent- ing different real-word situations. This does not reduce the necessity for analysis of the task. Before an effect can be deemed contextual it must be demonstrated that the tasks are isomorphic. Research into context- ual effects has used easily identifiable task isomorphs; identification of systems that respond in identical ways to subjects’ inputs is not as easy (see Cortes et al. 1974).

The purpose of presenting this experiment was to illustrate the utility of a systems analysis approach to DDM. At the level of a simple task system, it has been possible to give a psychological interpretation to the system parameters and to relate them to human performance. Im- portantly, there is no barrier to applying this approach to more com- plex situations. A systems analytical approach identifies the demands the task makes upon the decision maker, provides a basis for the comparison of tasks and encourages an ordered exploration of the range of possible tasks.

References

Bellman, R. and SE. Dreyfus, 1962. Applied dynamic programming. Princeton, NJ: Princeton

University Press. Bishop, Y.M.M.. SE. Feinberg and P.W. Holland, 1975. Discrete multivariate analysis: theory and

practice. Cambridge, MA: The MIT Press.

Box, G.E.P. and G.M. Jenkins, 1970. Time series analysis forecasting and control. San Francisco,

CA: Holden-Day.

Braybrooke. D. and C.E. Lindblom, 1963. A strategy of decision: policy evaluation and social

process, New York: Free Press.

Brehmer, B.. 1974. Single-cue probability learning as a function of the sign and magnitude of the

correlation between cue and criterion. Organisational Behaviour and Human Performance 9,

377-395. Conant, R.C. and W.R. Ashby. 1970. Every good regulator of a system must be a model of that

system. International Journal of System Science, 89-97.

Cortes. F.. A. Przeworski and J. Sprague. 1974. Systems analysis for social scientists. New York:

Wiley.

Dbwx. D.. 1980. On the difficulties people have in dealing with complexity. Simulatic

Games II, 87-106.

Ebert. R.J.. 1972. Human control of a two-variable decision system. Organisational Behavio

Human Performance 7. 237-264.

Forrester. J.W.. 1968. Principles of systems. Cambridge. MA: Wrlgth-Allen.

Francis, B.A. and W.M. Wonham. 1976. The internal model principle of control theory.

matica 12. 457-465.

Hogarth, R.M.. 1981. Beyond discrete biases: functional and dysfunction aspects of Judge1

heuristics. Psychological Bulletin 90. 197-217.

Klciter. G.D.. 1970. Trend-control in a dynamic decision making task. Acta Phychologi

387-397.

Mackinnon, ,4.J. and A.J. Wearing, 1980. Complexity and decision making. Behavioural S

25. 285-296.

Pontryagin, P.. 1962. Ordinary differential equatmns. Reading. MA: Addison Wrslcy.

Rapoport. A.A.. 1966a. A study of human control in a stochastic multistage decision

Behavioural Science 11, 1X-32.

Rapoport. A.A.. 1966h. A study of multistage decision-making task wth unknown dul

Human Factors 8, 54-61.

Rapoport. A.A., 1975. ‘Research paradigms for studying dynamic decision behavlour’. 1

Wendt and C.A.J. Vlek (eds.). Utility. probability and human decision making. Dord

Reidel. pp. 349-369.

Rapoport. A.A. and L.V. Jones, 1970. Gambling behaviour in multiple-choice multistage t

games. Journal of Mathematical Psychology 7, I-17.

Simon. H.A. and J.R. Hayes, 1976. The understanding process: problem isomorphs. Cq

Psychology 8. 165-190.

Weisbrod, R.L.. K.B. Daws. A. Freedy and G. Weltman, 1974. Adaptive computer aid

dynamic decision processes. JSAS: Catalog of selected documents in psychology. Vol. S.