i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

journa l homepage: www. int l .e lsev ierhea l th .com/ journa ls / i jmi

Clinical information displays to improve ICU outcomes

Judith A. Effkena,∗, Robert G. Loebb, Youngmi Kanga, Zu-Chun Lina

a College of Nursing, The University of Arizona, USAb College of Medicine, Department of Anesthesiology, The University of Arizona, PO Box 210203, Tucson, AZ 85721-0203, USA

a r t i c l e i n f o

Article history:

Received 20 December 2007

Received in revised form

26 March 2008

Accepted 12 May 2008

Keywords:

Clinical information display

Human computer interaction

Ecological display design

a b s t r a c t

Purpose: In a previous study, we compared a prototype ecological display (ED) that repre-

sented physiological data in a structured pictorial format with two bar graph displays [J.A.

Effken, Improving clinical decision making through ecological interfaces, Ecol. Psych. 18

(2006) 283–318]. In ED and the first bar graph display, data were grouped hierarchically based

on a cognitive work analysis (CWA); in the second bar graph display they were grouped as

usually collected. Treatment efficiency (i.e., percentage of time seven variables in the CWA

model were in target range) improved similarly with the two displays incorporating the CWA

order for intensive care unit (ICU) residents, but not for novice ICU nurses. Hypothesized

reasons for this result included: insufficient practice with novel displays; use of identical

histories across displays; insufficient clinical knowledge; and the variables used in the effi-

ciency analysis, which included only one of ED’s four integrated design elements. In the

current study we tested these hypotheses.

Methods: We asked ICU nurses assigned to three knowledge groups based on intensive care

and hemodynamic monitoring pretests to identify and treat oxygenation problems pre-

sented via ED and the first bar graph display (BGD) in an experimental laboratory simulation.

We measured the impact of display, clinical scenario, data level, knowledge, presentation

order, and practice extent on event recognition, treatment efficiency, cognitive workload,

and user satisfaction.

Results: The two displays produced little difference in recognition speed or overall cognitive

workload, but user satisfaction was greater with ED. When 12 variables were included in the

analysis, treatment efficiency improved with ED; when only 7 were measured, BGD prevailed.

The results suggest benefits for the kind of synthesis provided in ED, but also a potential

limitation. If too many different pictorial formats are used in a display, detecting critical

events may be more difficult.

1

Uo$tp

icians must integrate and synthesize to evaluate a patient’s

1d

. Introduction

p to 98,000 hospitalized Americans die annually becausef medical errors. The cost to our healthcare system is over

38 billion [2]. Despite increased use of technology in hospi-als, the patient safety problem persists. In a study of 1000atients in two intensive care units (ICUs) and a surgical

∗ Corresponding author. Tel.: +1 520 626 6307; fax: +1 520 626 7891.E-mail address: jeffken@nursing.arizona.edu (J.A. Effken).

386-5056/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights resoi:10.1016/j.ijmedinf.2008.05.004

© 2008 Elsevier Ireland Ltd. All rights reserved.

unit, both clearly high technology environments, 46% werereported to have had some adverse effect [3]. Contributing tothe safety problem is the huge number of data elements clin-

status. Not only are many data elements needed, but also clini-cians must obtain those data elements from many sources andmany varieties of proprietary clinical and computer systems

erved.

i c a l

766 i n t e r n a t i o n a l j o u r n a l o f m e d

[4,5]. Importantly, physicians and nurses rarely cover only onepatient at a time—and are subject to frequent interruptions [6].Under these conditions, clinicians cannot possibly synthesizeso much data without some kind of information support [7].

Although current health information technology can col-lect and present data from multiple sources on one or severalscreens, to date the technology produces little to no synthesisof the data elements [5,8]. A more systematic, theoreticallybased approach to the design of clinical information sys-tems is needed. The ideal information system should presentsynthesized data in such a way that it enhances clinicians’detection of significant changes and their overall understand-ing of the patients’ conditions while decreasing their cognitiveworkloads [9]. Our research is aimed at facilitating clinicians’decision making through ecologically designed graphical dis-plays that integrate and represent data in structures that helpclinicians visualize a patient’s physiological status. In thispaper, we report the results of a study comparing ICU nurses’performance with a prototype ecological display (ED) and a bargraph display (BGD).

2. Background

Presenting data graphically amplifies cognition by capitalizingon humans’ acute perceptual capabilities [10], so researchersincreasingly are exploring new ways to present visual informa-tion in clinical displays [11–16]. One promising technique is toincorporate graphical objects that represent the relationshipamong several variables as an emergent feature (e.g., [17,18]).

Cole and Stewart demonstrated the efficacy of a “metaphorgraphic” that showed the relationship between respiratoryrate and tidal volume in a single object [13]. In another study,histogram and polygon displays were shown to be superior toa numeric display when compared in a laboratory simulationenvironment [19]. The graphic displays decreased anesthe-sia residents’ response latency and increased their accuracyin detecting changes in physiologic variables. However, therewere no differences among the display formats for non-medical volunteers. Horn et al. tested a single circular objectwith 12 sectors, one representing each variable [20]. Althoughthe nurse subjects in that study reported that deviations fromnormal were easy to recognize, they reported that it was dif-ficult to gain an overall impression of the patient from theabstract display.

Ecological interface design (EID) integrates ecological psy-chology [21,22] with cognitive engineering to design interfacesfor complex socio-technical systems [23]. Four principles ofecological psychology are key [24]: (1) person and environmentare mutual and reciprocal; (2) perception is fundamental tocognition; (3) environmental constraints relevant to the usershould be identified early in the design process; and (4) dis-plays, controls, and evaluations should be ecologically valid[25,26]. EID typically begins with a cognitive work analysis(CWA) that defines the constraints of the work domain (i.e.,what is to be acted upon), and the cognitive tasks that are

performed (what is done) [27]. Vicente first described EID [26]and then applied the technique to the problem of power plantcontrol. By mapping identified work domain constraints ontothe geometry of an interface as perceptually salient objects,

i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

Vicente was able to improve users’ fault detection, diagnosis,and control in a complex thermohydraulic microworld [27,28].

In healthcare, EID has been used to design a neonatalICU display that presented physiological data as graphicalobjects [29]. In a laboratory study, physicians from a neonatalICU more accurately diagnosed clinical incidents when usingthe experimental display than when using a conventionalalphanumeric clinical display. Although the EID interfaceresulted in better overall performance, the greatest gain wasseen in less experienced physicians.

Using similar techniques, Blike developed an anesthesiadisplay that mapped physiologic variables onto display objectsthat had meaningful shapes [30,31]. Anesthesiologists’ prob-lem recognition and diagnostic accuracy for five shock and fivenon-shock states were better with the prototype than witha traditional alphanumeric display. Specific design features(i.e., a pointer and reference scale) improved recognition, butnot accuracy. Providing emergent features (e.g., changing anobject’s shape to depict a dilated blood vessel) improved eti-ology judgments. These studies, as well as others, show theimportance of accurate semantic mapping of the clinical sit-uation onto display elements [32–34].

We used EID techniques to design an oxygen managementdisplay [1,35]. We conducted a CWA that identified functionaldesign constraints in five areas (work domain, tasks, strate-gies, social-organizational and user skills) and resulted in amodel that described the critical physiology (e.g., mean arte-rial pressure), functions (e.g., cardiac output), balances (e.g.,oxygen demand and oxygen delivery), and purposes (cellu-lar oxygenation) of the body’s oxygenation system [1,35,36].We used the CWA model to create a prototype ED that pre-sented clinical data structured at four levels: purpose, balance,processes, and physiology (Fig. 1). Non-medical readers arereferred to the Glossary for definitions of the physiologicalvariables.

In a previous study, we compared ED with two bar graphdisplays containing the same clinical variables in differentarrangements [1]. In the first bar graph display (Fig. 2) and inED, variables were ordered in accordance with the CWA model;in the second bar graph display, variables were arranged inthe order that they are usually collected. Resident physiciansshowed higher treatment efficiency on a simulated clinicaltask (measured as percentage of the trial that seven variableswere kept within target range) when using either of the dis-plays arranged in accordance with the CWA model, but typeof display had no effect on novice ICU nurses’ performance.These results raised a number of questions. Did we not prop-erly analyze treatment efficiency by including only 7 of the 12displayed variables? Did the nurses, who had just completedtheir ICU internship, lack sufficient task-relevant knowledgeto use the displays arranged in accordance with the CWAmodel effectively? Did participants simply need more practicewith the novel displays? Did our use of the same clinical his-tories across displays induce participants to repeat the sametherapies for each? Or perhaps the results were due to somecombination of these factors. To test these hypotheses, in this

study we compared just the two displays in which variableswere arranged hierarchically based on the CWA model, butwhose visual presentation format differed. We added an eval-uation of cognitive workload to further clarify the results.

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777 767

Fig. 1 – Screen shot of ED with the treatment panel early in Scenario 1 when some treatments have been started. EDpresents data in the order suggested by the results of a cognitive work analysis (goal = cellular oxygenation—although thisis currently only a placeholder since no data were available to measure it; balance = DO2, CaO2, VO2, and ER;processes = SaO2, Hgb, and CO; and physiology = CVP, PAWP, MAP, SVR, SV, and HR), as well as clinically importantrelationships among data elements. For example, cardiac output is shown as the product of heart rate and stroke volume.Pressures such as CVP, PAWP, and MAP are shown as they relate to the heart; the central object shows the ratio of CaO2 toCvO2 (venous oxygen content). All objects are color coded to indicate oxygen content (dark red being the most oxygenated;blue being the least). The sizes and shapes of objects change in accordance with the simulated patient’s variables. Thevalues shown here are for a normal patient. Participants use the mouse to click on the treatment buttons at the right to startor stop each therapy. The small circle just to the left of the button turns green when drugs are being given and the text atthe left changes from OFF to ON. The amount of fluid and blood given is calculated and shown as number of ccs given, as isthe patient’s urine output. Only one FiO2 (fraction of inspired oxygen) percentage (30%) is available. More than one therapyc ncesw

risbaiith

HB

HBuwa

H

H

an be given simultaneously. (For interpretation of the refereeb version of the article.)

In this study, we compared ICU nurses’ critical eventecognition, treatment efficiency, cognitive workload and sat-sfaction using ED (Fig. 1) or the first BGD from the previoustudy (Fig. 2) in a simulated oxygenation management task. Inoth displays, the physiologic variables shown were ordered inccordance with the CWA model. Thus, the displays differedn graphic presentation format, but not in data content norn how the data were arrayed. Nurses were used to evaluatehe effect of task-relevant knowledge. We tested the followingypotheses:

1. Critical event recognition will be faster with ED than withGD.

2. Treatment will be more efficient with ED than withGD, as measured by the percentage of appropriate drugssed; but the benefits of ED will be more apparenthen additional integrated variables are included in the

nalysis.

3. Cognitive workload will be less with ED than with BGD.

4. User satisfaction will be greater with ED than with BGD.

to color in this figure legend, the reader is referred to the

H5. Nurses with more task-relevant ICU knowledge will per-form better in the task, and differences in performance withdisplays will be greater with more knowledgeable nurses.

H6. More practice time will improve performance with EDdifferentially because of its novelty.

H7. Display differences will be more apparent with different,but equivalent, patient histories.

3. Methods

3.1. Sample

After approval was received from The University of ArizonaInstitutional Review Board, all adult ICU nurses at a univer-sity medical center were invited to participate in the study.Volunteers received $25/h for participating in the study.

3.2. Design

In the mixed experimental design, display (ED vs. BGD), sce-nario (4 clinical events), and data level (balance, processes,

768 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

Fig. 2 – In this display (BGD), the bar graphs are ordered in the same way as the variables in ED (Fig. 1), and the same colorreatmd to

coding and drug controls are used. The same scenario and treferences to color in this figure legend, the reader is referre

and physiology) were controlled within subjects; and task-relevant knowledge (high, medium, low), display presentationorder (ED first vs. BGD first), and practice extent (short vs.long) were controlled between subjects. Knowledge and prac-tice extent were added for this study (H5 and H6); the otherfour variables replicated the previous study. Participants’ priorcomputer usage and attitudes toward computers were enteredas covariates in the analyses to better control for individualdifferences. We anticipated that the complex design wouldallow us to not only test multiple hypotheses, but also to eval-uate potential interactions. Each participant viewed the samefour clinical scenarios (Appendix A) with ED and BGD, but forhalf the trials, the patient history for a scenario differed witheach display. We examined the effect of these variables onparticipants’ critical event recognition, treatment efficiency,cognitive workload, and user satisfaction. Display presenta-tion order was counterbalanced; scenario presentation orderwas randomized. To allow us to evaluate H7, half of the par-ticipants repeated the training session (practice) with eachdisplay.

3.3. Procedures

Participants were tested individually in a quiet room. Each ses-sion consisted of: (1) written pretests, (2) training on the firstdisplay and experimental task, (3) performance of the experi-mental task on each scenario using the first display (includingcompleting the NASA TLX after each scenario), (4) completion

of the “Your Opinion Please” satisfaction questionnaire, (5) abreak of at least 20-min, and (6) repetition of steps 2–4 usingthe second display. Steps 2–3 were presented on a Dell Lat-itude C840 laptop computer with a 21 in. color monitor. The

ents are shown as in Fig. 1. (For interpretation of thethe web version of the article.)

computer recorded, as a 2-s interval time series, the valuesof all physiological variables and the status of each treatment(i.e., “on” or “off”).

At the start of each session, we collected basic demographicdata including age, gender and prior clinical experience. Par-ticipants then completed three written pretests:

• An adapted version of the Use of Technological InterventionsInstrument developed at The University of Arizona (UA) wasused to measure students’ attitudes toward computers andprevious computer experience. The instrument has fourscales: Technology Used on the Job, Technology Used Outsidethe Job, Computer Skill, and Attitudes toward Using Computers.UA experts had established content validity for the instru-ment, but neither construct validity nor reliability had beenreported.

• An abridged version of Ritchey and Toth’s Basic Knowl-edge Assessment Tool Version 5 (BKAT-5) [37] was used tomeasure participants’ basic critical care knowledge. Apanel of experts had established content validity for theoriginal instrument; reliability (internal consistency viaCronbach’s coefficient alpha) for this version was reportedas 0.82 (http://nursing.cua.edu/research/toth-bkat5s.cfm).The abridged BKAT-5 included 12 questions judged byclinicians on our research team to be relevant for the exper-imental task.

• An abridged version of Iberti’s Pulmonary Artery Catheter: Useand Interpretation of Data [38] test (PCATH) was used to mea-

sure participant’s knowledge of hemodynamic monitoring.The 12 questions were selected by clinicians on our researchteam to be relevant to the experimental task. A Delphi pro-cess had established content validity for the original test.

a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777 769

•

•

•

•

dcclswmSleebfiwtmfi[puwo

3

Wstttseaapaiamvo

i

Fig. 3 – Mean scores on abridged BKAT-5 and PCATH testsby years of experience. Dotted lines indicate the divisionsbetween high, medium and low knowledge groups. Groupswere defined by adding and subtracting 0.5 S.D. to themean score for the two tests. This resulted in 12 (2 male; 10female) in the Low Group, 9 females in the Medium Group,and 11 (3 male, 8 female) in the High Knowledge Group.The correlation between test scores and years of experience

i n t e r n a t i o n a l j o u r n a l o f m e d i c

Reliability (Kuder–Richardson 20 estimate) was reported as0.71.Participants received identical training with each displayprior to the experimental trials with that display. After view-ing video clips introducing each display, participants wereasked to:Identify changes in a single variable when the experimenterinitiated the changes.Use the available treatments to increase or decrease six vari-ables to a specified target.Identify three common hemodynamic/oxygenation prob-lems.Use the available treatments to correct simple hemody-namic/oxygenation problems.

In the experimental task, participants viewed, with eachisplay, 4 clinical scenarios in which 12 physiologic variableshanged dynamically and controlled 8 treatments, whichould be administered concurrently, to correct identified prob-ems. Treatments could be turned “on” or “off,” but wereet to “off” at the start of each trial. Physiologic variablesere updated every 2 s, based upon a running cardiopul-onary model that responded to treatments (Human Patient

imulator version C, METI, Sarasota, FL). Although all prob-ems focused on oxygenation management and mimickedxisting patient data, the underlying physiology was differ-nt in each scenario (Appendix A). Each trial lasted 240 s,ut participants could end a trial whenever they were satis-ed with their patient’s status. After each trial, participantsere asked to describe the clinical problem and complete

he NASA-TLX (Task Load Index), which has six subscales:ental demands, physical demands, temporal demands, per-

ormance, effort, and frustration. Validity of the NASA-TLXs well established, and stability has been reported as 0.8339]. After completing four trials (scenarios) with each dis-lay, participants rated their satisfaction with the displaysing the Your Opinion Please questionnaire. The experimentas videotaped, and participants talked aloud through-ut.

.4. Data analysis

e measured: (1) critical event recognition (elapsed time [in] from the onset of the scenario to the initiation of the firstherapy), (2) treatment efficiency (percentage of the 240 s trialhat variables were kept within the target range), (3) cogni-ive workload, and (4) user satisfaction. Although the displayshow optimal target ranges for each variable, we found thatxperts resisted this degree of optimization as unrealisticnd possibly undesirable. Therefore, for treatment efficiencynalyses, target ranges were instead based on the meanerformance, in a previous study, of three board-certifiednesthesiologists and three nurse clinical specialists certifiedn intensive care nursing and calculated, for each variable,s experts’ mean score ±1S.D. [1]. Trials that participants ter-inated early were normalized to 240 s by repeating the last

alues of the physiologic variables and states of the treatmentptions (on or off).

The Your Opinion Please instrument uses a 26 mm hor-zontal scale anchored by two poles (e.g., wonderful and

was extremely low (r = 0.05).

terrible) to measure user satisfaction and was scored asdistance from the negative pole, expressed as a percent-age. Negatively phrased items in the Attitudes toward UsingComputers and Your Opinion Please instruments were reversecoded.

Descriptive statistics were used to summarize the results.Three separate mixed analyses of covariance (ANCOVAs)were conducted with display, scenario, data level controlledwithin subjects; knowledge, presentation order, and practiceextent controlled between subjects; and prior computer usageat work and home, computer skills, and attitudes towardcomputers entered as covariates to assess: (1) critical eventrecognition (H1), (2) treatment efficiency (H2), and (3) per-ceived cognitive workload (H3) with each display. Analysesof variance (ANOVAs) were conducted to assess differencesin pretests and user satisfaction among groups. Paired t-testswere used to compare users’ satisfaction with the two displays(H4) and to evaluate the impact of same vs. different patienthistories (H7). For analyses, a one-sided p < 0.05 was consid-ered statistically significant and observed power acceptable at>0.80.

4. Results

4.1. Sample characteristics

Thirty-two ICU nurses (5 male and 27 female) were assignedto one of three knowledge groups (High [HK]; Medium [MK],and Low [LK]) based on combined BKAT-5 and PCATH testscores (Fig. 3). BKAT-5 scores ranged from 58 to 92%; PCATHscores ranged from 17 to 83%. Demographics are shown in

Table 1. The entire experimental session took 2–3.5 h. Becausetwo nurses did not view all scenarios, computer-recorded datawere analyzed for 30 nurses only.

770 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

Table 1 – Demographic characteristics of sample byknowledge group (N = 32)

Age (years) Knowledge group

Low Medium High

N % N % N %

21–30 3 25 3 33.2 3 27.331–40 6 50 2 22.2 5 45.441–50 0 0 2 22.2 3 27.351–60 3 25 2 22.2 0 0Total 12 100 9 100 11 100

GenderMale 2 16.7 0 0 3 27.3Female 10 83.3 9 100 8 72.7

AN

CO

VA

resu

lts

byd

isp

lay,

scen

ario

,dat

ale

vel,

kn

owle

dge

leve

l,p

rese

nta

tion

ord

eran

dp

ract

ice

exte

nt

(N=

30)

Dis

pla

ySc

enar

ioD

ata

Leve

lK

now

led

geG

rou

pPr

esen

tati

onO

rder

(Fir

stD

isp

lay)

Prac

tice

Exte

nt

EDB

GD

12

34

Bal

Proc

Phys

Low

Med

Hig

hED

BG

DSh

ort

Lon

g

itia

tetr

eatm

ent

(s)

36.2

40.1

36.7

42.9

43.6

29.3

––

–37

.933

.343

.236

.939

.437

.039

.2(2

.4)

(2.9

)(3

.5)

(3.4

)(2

.0)

(1.8

)–

––

(3.8

)(4

.9)

(3.9

)(3

.2)

(3.7

)(4

.0)

(3.8

)

enta

geof

tim

ein

targ

etra

nge

(7va

riab

les)

29.1

*29

.8*

40.0

35.1

13.1

29.4

12.5

35.5

40.2

29.7

30.3

28.3

30.3

28.6

30.4

28.5

(0.8

)(1

.0)

(1.6

)(1

.6)

(0.5

)(1

.6)

(1.2

)(1

.1)

(0.8

)(1

.1)

(1.6

)(1

.2)

(1.1

)(1

.1)

(1.5

)(1

.3)

enta

geof

tim

ein

targ

etra

nge

(12

vari

able

s)35

.9*

34.8

*43

.833

.925

.937

.922

.4*

37.1

*46

.6*

34.4

36.2

35.5

37.0

33.7

37.1

33.7

(0.9

)(0

.9)

(2.2

)(0

.9)

(0.7

)(1

.3)

(0.9

)(1

.4)

(0.9

)(1

.2)

(1.7

)(1

.3)

(1.2

)(1

.2)

(1.6

)(1

.3)

itiv

ew

orkl

oad

57.0

57.4

56.5

61.4

52.6

58.2

––

–59

.056

.855

.857

.157

.258

.655

.7(3

.5)

(3.5

)(3

.6)

(3.9

)(4

.1)

(3.8

)–

––

(5.4

)(7

.1)

(5.6

)(4

.7)

(5.3

)(5

.8)

(5.5

)

toin

itia

tetr

eatm

ent

and

tota

lcog

nit

ive

wor

kloa

dw

ere

not

mea

sure

dat

the

dat

ale

vel.

S.E.

:sta

nd

ard

erro

r;B

al:b

alan

ce;P

roc:

pro

cess

es;P

hys

:ph

ysio

logy

.

Total 12 100 9 100 11 100

Notes: N: number, %: percentage.

4.2. Instrument psychometrics for this sample

Cronbach’s alpha for the entire Use of Technological Inter-ventions survey was 0.90 and for individual scales 0.57(Technology Used on the Job), 0.78 (Technology Used Out-side the Job), 0.90 (Computer Skill), and 0.92 (Attitudestoward Using Computers). The abridged BKAT-5 and PCATHinstruments demonstrated lower internal consistency, withCronbach’s Alphas of 0.40 and 0.38, respectively. Cron-bach’s Alpha of the Your Opinion Please instrument was0.92.

4.3. Testing the hypotheses

4.3.1. Clinical event recognitionWe had hypothesized that clinical event recognition wouldbe faster with ED than with BGD (H1). Clinical event recog-nition (i.e., time to initiate treatment) was nearly 4 s fasterwith ED (Table 2). A 2 (Display) × 4 (Scenario) × 3 (DataLevel) × 3 (Knowledge) × 2 (Presentation Order) × 2 (PracticeExtent) mixed ANCOVA with time to initiate treatment as thedependent variable and the previously described covariatesrevealed no main effects; but there was a Display × Order inter-action (Fig. 4, top), as well as 3-way interactions of display,scenario and presentation order and of scenario, presentationorder and practice extent. Recognition was quicker with thesecond display used. In the BGD First condition, most improve-ment in subsequent ED trials occurred in Scenarios 2 (adultrespiratory distress syndrome, or ARDS) and 3 (Sepsis) – 17 and28 s, respectively – and was facilitated, at least in Scenario 2,by shorter practice. When ED was used first, most improve-ment (18 s) with subsequent use of BGD occurred in Scenario3.

4.3.2. Treatment efficiencyWe expected that treatment would be more efficient with EDthan with BGD, but the benefits of ED will be more appar-

ent when additional integrated variables are included in theanalysis (H2). We conducted a similar ANCOVA on PercentTime (%Time) in Target Range, using the same variables asin our previous study: balance (delivered oxygen); processes

Tabl

e2

–

Tim

eto

inM

ean

(S.E

.)

Mea

np

erc

Mea

n(S

.E.)

Mea

np

erc

Mea

n(S

.E.)

Tota

lcog

nM

ean

(S.E

.)

Not

e:T

ime

∗p

<0.

01;

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777 771

Fig. 4 – Top panel: Mean percentage of time to initiate treatment by display and target order, showing a statisticallysignificant interaction, F1,14 = 18.024, p = 0.001. Lower left panel: Mean percentage of time in target by display andpresentation order when 7 variables were measured, showing a statistically significant interaction, F1,14 = 9.260, p = 0.009.Lower right panel: Mean percentage of time in target by display and presentation order when 12 variables were measured,s p = 0b

(paoEcplaspDPEpMBapt

Tr(s

howing a statistically significant interaction, F1,14 = 11.687,enefit of ED.

arterial oxygen saturation, cardiac output, and hemoglobin);hysiology (heart rate, mean arterial pressure, and pulmonaryrtery wedge pressure). The ANCOVA revealed a main effectf Display, F1,14 = 9.602, p = 0.008,1 with BGD 0.1% higher thanD (Table 2). The high statistical significance is because theovariates reduced the error variance and provided sufficientower (observed power = 0.82) to detect differences. Physio-

ogical variables were controlled best, followed by processesnd balance variables. Treatment efficiency improved with theecond display (a learning effect); but there was also a Dis-lay by Presentation Order interaction (Fig. 4, left lower panel).isplay entered into 3-way interactions with Knowledge andresentation Order and with Presentation Order and Practicextent. LK scores improved only when ED was the second dis-lay used and improved more in the short practice condition;K scores improved with the second display (either ED or

GD); HK scores were unaffected by display. We also observed6-way interaction of display, scenario, data level, knowledge,resentation order, and practice extent, but will not attempto explain it here.

A similar ANCOVA was conducted for Percent Time withinarget Range, but with five additional variables: balance (arte-

ial oxygen content and extraction ratio) and physiologycentral venous pressure, systemic vascular resistance, andtroke volume). Only variables that could not be controlled

1 F1, 14 = F test statistic with 14 degrees of freedom.

.004. In all three cases, there seems to be a differential

by the available treatments were excluded. The ANCOVArevealed main effects of Display, F1,14 = 9.005, p = 0.001; andData Level, F2,13 = 5.691, p = 0.008. The order of displays wasreversed with ED now 1.1% higher than BGD (Table 2). Phys-iological variables were controlled best; balance variableswere controlled least well. Covariates reduced the error vari-ance, accounting for the statistical significance and adequateobserved power (0.80 and 0.82, respectively). Treatment effi-ciency increased 3% with the second display used, a learningeffect. Display entered into a 2-way interaction with Pre-sentation Order (Fig. 4, right lower panel), and into 3-wayinteractions with Knowledge and Presentation Order, withPresentation Order and Practice Extent, and with Data Leveland Practice Extent, and into a 5-way interaction with Sce-nario, Knowledge, Presentation Order, and Practice Extent. Asin the previous analysis, the HK group did not improve withthe second display used. Treatment efficiency using ED wasbetter with longer practice, but only if it was the first displayused.

4.3.3. Cognitive workloadWe anticipated that cognitive workload would be less with EDthan with BGD (H3). Results for Total Workload are presentedin Table 2. Separate mixed ANCOVAs on Total Workload and

each individual subscale revealed no main effects, with oneexception: for Perceived Performance, the ANCOVA revealed amain effect of computer skill and a significant interaction ofDisplay and Knowledge.

772 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

Table 3 – Mean (M) and standard error (S.E.) for Items in the “Your Opinion Please” survey by display and knowledgelevel (N = 32)

Questions EDa BGDa

Lowb Mediumb Highb Meanb Lowb Mediumb Highb Meanb

M (S.E.) M (S.E.) M (S.E.) M (S.E.) M (S.E.) M (S.E.) M (S.E.) M (S.E.)

1. Wonderful vs. Terrible 69 (5.8) 62 (5.5) 75 (4.7) 69 (3.2) 63 (6.0) 49 (5.9) 57 (4.9) 57 (3.3)2. Satisfying vs. Frustrating 68 (6.6) 61 (3.7) 66 (4.9) 65 (3.1) 56 (7.1) 49 (7.0) 55 (6.2) 54 (3.8)3. Stimulating vs. Dull 78 (5.8) 73 (4.9) 77 (6.0) 76 (3.2) 72 (7.0) 56 (5.1) 68 (6.6) 66 (3.8)4. Easy vs. Difficult 59 (6.9) 51 (5.3) 47 (8.2) 52 (4.1) 56 (7.1) 42 (7.0) 43 (5.9) 48 (4.0)5. Flexible vs. Rigid 65 (7.1) 67 (5.2) 58 (7.5) 63 (3.9) 55 (9.0) 40 (3.7) 60 (6.0) 52 (4.2)6. Beneficial vs. Misleading 78 (5.2) 72 (3.3) 78 (4.2) 76 (2.6) 70 (7.5) 59 (5.4) 66 (7.6) 65 (4.1)7. Availability of clinical information 80 (3.9) 75 (3.5) 69 (5.4) 75 (2.6) 69 (8.1) 56 (7.1) 63 (6.1) 63 (4.1)8. Ease of use 70 (5.1) 70 (5.8) 70 (5.8) 70 (3.1) 70 (8.5) 48 (7.8) 57 (6.9) 59 (4.6)9. Presentation of clinical information 76 (4.7) 74 (2.0) 76 (3.8) 76 (2.2) 68 (8.1) 65 (5.0) 56 (5.6) 63 (3.8)10. Images and characters 66 (6.7) 61 (8.6) 68 (7.4) 65 (4.2) 64 (9.8) 48 (7.9) 54 (8.4) 56 (5.1)11. Use of color 64 (7.1) 71 (2.9) 70 (7.5) 68 (3.8) 64 (8.7) 53 (8.7) 67 (6.3) 62 (4.6)12. Use of metaphors 31 (10.1) 34 (9.5) 43 (11.5) 36 (6.0) 37 (11.3) 29 (8.4) 35 (9.1) 34 (5.6)13. Overall sense of patients 79 (3.5) 64 (8.4) 71 (7.8) 72 (3.8) 69 (7.8) 62 (6.8) 70 (5.6) 67(3.9)

Overall mean 68 (4.0) 64 (2.7) 67 (4.2) 66 (2.1) 62 (5.4) 50 (3.4) 58 (3.8) 57 (2.7)

However, when 12 variables were included, of which 3 wereintegrated objects in ED, efficiency was better with ED, provid-

a Display.b Knowledge level.

4.3.4. User satisfactionWe expected that user satisfaction would be greater with EDthan with BGD (H4). Participants rated ED (M = 67%) higherthan BGD (M = 57%) (t31 = 3.088, p = 0.002). For ED, three itemsscored higher than 75%: stimulating vs. dull, beneficial vs.misleading, and presentation of clinical information (Table 3).Only “use of metaphors” was rated below the scale’s midpoint.For BGD, the item scoring highest was overall sense of patient(67.3%); and two items were rated below the scale midpoint:easy vs. difficult, and use of metaphors.

4.3.5. Task-relevant knowledgeWe hypothesized that nurses with more task-relevant ICUknowledge would perform better in the task, and differencesin performance with displays will be greater with more knowl-edgeable nurses (H5). Knowledge was never a main effect, butdid enter into several interactions, which provides some sup-port for H5. Contradicting H5, there seemed to be more impactof ED on the LK group; the HK group tended to improve withBGD—or show little difference.

4.3.6. Practice timeWe expected that more practice time would improve per-formance with ED differentially because of its novelty (H6).Generally, longer practice sessions tended to degrade perfor-mance, probably due to fatigue induced by the 3-h experiment.However, in one analysis, performance with ED (when usedfirst) was enhanced with longer practice, partially supportingthe hypothesis. NASA TLX results indicated that, with longerpractice, mental demands and effort decreased, but frustra-tion increased, perhaps clarifying the mixed results.

4.3.7. HistoriesWe hypothesized that display differences would be moreapparent with different, but equivalent, patient histories (H7).A mixed 2 (Display) × 2 (History) ANCOVA for each of the four

scenarios with treatment efficiency as the dependent variablerevealed a significant effect of Display, but only in Scenario2 (M = 38% and 49% for BGD and ED, respectively).2 Whenthe patient histories viewed for a scenario were different foreach display, overall treatment efficiency improved slightly(although not statistically significant) and only for Scenario 2.In that scenario, performance with the novel history improvednearly three times as much with ED. The reason for this is asyet unclear.

5. Discussion

We begin with a discussion of the results of our hypothesistesting followed by more general observations.

Critical event recognition will be faster with ED than withBGD (H1). Critical event recognition did not differ significantlyby display (H1), even though recognition was 4 s faster withBGD. Perhaps the use of only one type of display element inBGD made detecting emergent patterns of changes as propor-tional bar heights easier [40]. Critical event recognition wasfaster with the second display used, a transfer (learning) effect;but there also was a differential effect of presentation orderthat suggested a benefit of using ED first.

Treatment will be more efficient with ED than with BGD,but the benefits of ED will be more apparent when additionalintegrated variables are included in the analysis (H2). Whenthe efficiency measure (%Time in Target Range) includedonly seven variables, of which only one was presented as anintegrated object in ED, performance was better with BGD.

ing support for the second part of H2.

2 M: Mean.

a l i n

taaswtfdmTpwtrdltsp

usplsOusom

(n

(a

fwnusshAn

fwtdpaviiHp6

i n t e r n a t i o n a l j o u r n a l o f m e d i c

Although physiology variables were always controlled best,he majority of improvement with ED occurred in balance vari-bles. Two of the three integrated objects (CaO2 and ER) occurt this level and were not included in the seven-variable analy-is. This result is similar to other studies of EID displays [27], inhich the main improvement with EID displays was attributed

o higher level functional information. The high level of per-ormance seen with physiology variables in this study may beue to the fact that drugs more frequently act at this level,aking it crucial to controlling the patient’s disease process.

his observation is consistent with a previous study, in whicherformance improved drastically when patient informationas presented at the same level as drug controls [41]. In

hat study, a many-to-many, non-linear control problem waseduced to three one-to-one linear problems. While clearlyesirable, this kind of simplification is not yet feasible in prob-

ems with many variables to be controlled. Still, coupled withhe lower variability in ED performance, the results provideome support for using this kind of integrated graphical dis-lay.

Treatment efficiency improved with the second displaysed. The Display by Order interactions observed in thistudy suggest differential transfer (learning) from one dis-lay to the other. Something more (or different) apparently is

earned when ED is used. This could be related to ED’s novelty,ince participants reported it as more stimulating in the Yourpinion Please survey. Alternatively, perhaps ED led to betternderstanding of the relationships among data elements, asuggested by participants’ 76% rating of ED for presentationf clinical information vs. 63% for BGD on the same instru-ent.Cognitive workload will be less with ED than with BGD

H3). Contradicting H3, cognitive workload did not differ sig-ificantly across displays.

User satisfaction will be greater with ED than with BGDH4). Users apparently preferred ED because of its interestingppearance and the way data were shown.

Nurses with more task-relevant ICU knowledge will per-orm better in the task, and differences in performanceith displays will be greater with more knowledgeableurses (H5). Finding no main effect of knowledge wasnexpected. Although nurses’ low PCATH scores were con-istent with Iberti’s findings [42], the wide range of BKAT 5cores was surprising. In addition, participants who scoredigh on one test did not always score high on the other.

better measure of task relevant knowledge may beeeded.

More practice time will improve performance with ED dif-erentially because of its novelty (H6). Treatment efficiencyith ED, when it was the first display used, was better in

he longer practice condition; but otherwise longer practiceegraded performance, probably because simply repeating theractice session increased the experiment length significantlynd led to fatigue and/or boredom. ED’s more interestingisual presentation presumably buffered these effects earlyn the experiment. There is conflicting evidence on whether

ntegrated displays such as ED take longer to learn to use [43].ad we been able to space practice sessions over a longereriod of time, the outcomes might have been different. In a-month longitudinal study, researchers found that EID stim-

f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777 773

ulated functional (or goal-oriented) knowledge in users onlyif they actively reflected on display feedback [44]. Althoughour participants seemed very motivated, it is unclear to whatextent they reflected on the relationships shown in the dis-plays.

Display differences will be more apparent with different,but equivalent, patient histories (H7). Some improvementwas noted in treatment efficiency, but only for a single sce-nario.

These results suggest that not finding a treatment effi-ciency benefit for ED over BGD in the previous study may bedue to the set of variables we included. When more synthe-sized variables were included, the benefits of ED became moreapparent. However, the higher order interactions suggest thatonly the combination of the hypothesized factors can providea full explanation for the observed results.

Although never a main effect, scenario entered into 3-way interactions for clinical event recognition and treatmentefficiency; but its effects were largely restricted to Sce-narios 2 and 3. Although further research will be neededto understand exactly why this result occurred, differ-ential benefits of clinical displays across scenarios maybe due to the relative specificity of the semantic map-ping of data relationships to the clinical scenario (e.g.,[33,34,41]).

The current study had several limitations. Extending theabbreviated trials by assuming the last state of the systemcontinued was problematic because participants who stoppedtrials early seemed to assume that their patients would con-tinue to improve. However, projecting actual trajectories foreach variable was not feasible because trajectories changeuniquely and are highly sensitive to changes in other variablesand to initial conditions.

We intentionally compared the two displays that differedonly in design elements because we had previously foundbenefits for the hierarchical ordering used in each [1], aswell as for the benefits of structured pictorial displays overalphanumeric displays [45]. To provide a more ecologicallyvalid data analysis, we used a mixed design with both withinand between subjects factors. The repeated measures plus thecovariates reduced participant variance sufficiently to allow usto detect main effects of displays and levels, as well as higherorder interactions. However, repeated measures can producelearning effects that may mask real differences. Higher orderinteractions are difficult to interpret, and we did not attemptto do so here. Testing each hypothesis individually in multipleexperiments would have provided more easily interpretableresults, but at the expense of detecting complex interactionsreflecting reality. We did not account statistically for multi-ple tests of hypotheses. Because of the complex design andbecause participants were selected from only one medicalcenter, it will be useful to replicate this study in other set-tings.

Studies of ecologically designed displays have demon-strated better user performance for error detection andprocess control in a variety of settings, but such displays

require sensors that can provide accurate data [27]. This canbe a serious problem for healthcare applications of EID [27,46].Necessary sensors are sometimes not available—or not avail-able continuously.

i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

Summary points

What was already known

• Ecological interface design has been shown to be use-ful to clinicians in several settings.

• When clinical data were arranged in a display in theorder that experts use them, as opposed to the waynurses typically collect the data, diagnostic and treat-ment performance of resident physicians improvedbut not nurses; thus, the way data are arrayed matters.

What this study added to our knowledge

• Tested several hypotheses about why intensive carenurses’ performance did not improve with the ecolog-ically designed display format.

• Directly compared two displays that arrayed the datain the same way (as experts use them)

• Results suggested the need to consider a varietyof human-technology factors and how they interactwhen evaluating displays, e.g., display format, priorcomputer usage, task relevant knowledge, clinical sce-nario, and practice.

774 i n t e r n a t i o n a l j o u r n a l o f m e d

EID has been most successful when the functional relation-ships among parts of the system to be controlled are clearlyunderstood. Equations that describe these relationships aremore likely to be known for a physical system than for aphysiological or social system and often fail to take individ-ual patient variation into account [46]. With few exceptions,most physiology studies are too fine-grained to be usefulfor whole-body modeling. Clearly, further basic research isneeded to uncover the relationships among physiologicalvariables to allow for more coherent presentation in clinicaldisplays.

EID may be more beneficial for complex problems [27]. Theoxygenation problem in this study seems simple but, in fact,is highly complex. If one ignores the 2 variables that did notchange, the patient space defined by the problem is com-prised of 12 dimensions (the 12 variables to be monitored);and the treatment space (therapy options) is comprised of 8dimensions. In the future, it might be useful to study experts’strategies for reducing problem dimensionality. An ideal clin-ical display (i.e., one that reduces both cognitive workloadand errors) should accomplish similar dimensionality reduc-tion.

Although ED was designed for only one domain, it couldeasily be extended to encompass any physiological vari-ables for which there are known relationships. Thus thedisplays could be used in various ICU, Emergency, and Traumaunits. However, such displays require constant vigilance.In the clinical setting, constant observation is not feasible;therefore the displays will need to be supplemented and/orrevised to trend critical information and include alarms. Inaddition, several researchers have proposed better spatialand temporal integration across data levels [47,48], as wellas additional presentation modes (e.g., auditory or tactile)[49,50].

6. Conclusion

We compared the impact of two clinical displays on ICUnurses’ event detection, treatment efficiency, cognitive work-load and satisfaction in a simulated oxygenation managementtask. ED, which was designed using EID techniques, presentedphysiological data in a hierarchically structured pictorial for-mat. BGD presented the same data arrayed in the same order,but shown as discrete bar graphs. ED produced higher usersatisfaction and better treatment efficiency when 12 vari-ables (3 of which were synthesized in ED as objects) wereincluded in the analysis. When the analysis was done usingonly seven variables (only one of which was synthesized inED), treatment efficiency was higher with BGD. This observa-tion provides at least a partial explanation for the perplexingresults of the previous study and suggests the benefit of syn-thesizing data as objects in a clinical display. Although EIDrules for mapping data onto appropriate display elements arenow emerging [51], our results suggest that it may be advis-able to limit the number of different types of objects per

display.

We observed no differential effect of display on total cogni-tive workload. Both event recognition and treatment efficiencyimproved with the second display used. However, using ED

first seemed to improve later performance with BGD, sug-gesting that something significant is learned from using thatdisplay, perhaps because its novelty or design invites closerscrutiny, or because some features, such as extraction ratio,are made more salient and therefore that information is betterattended to in BGD.

We hope that our results will encourage further researchin this area. Displays that synthesize clinical data effectivelycould provide another leverage point for reducing clinicalerrors in technology-intensive, high-stress clinical environ-ments.

Acknowledgements

This research was supported by a grant from the LawrenceEmmons Foundation at The University of Aizona and by theDepartment of Army and the National Medical TechnologyTestbed (NMTB) Cooperative Agreement No. DAMD17-97-2-7016. The content of the information does not necessarilyreflect the position or the policy of the government orthe National Medical Technology Testbed, and no officialendorsement should be inferred. Portions of this manuscripthave been presented as poster or podium presentations atthe 2007 and 2008 Western Institute of Nursing ResearchConference, the 2007 American Medical Informatics Associ-ation (AMIA) Spring Symposium, and the 14th InternationalConference on Perception and Action (ICPA 2007). We

acknowledge the many helpful comments of anonymousreviewers.

A

S

1

2

3

4

r

i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777 775

ppendix A. Scenarios with original and alternate patient histories

cenario Original history Alternate history

65-year-old man who 2 days ago had a smallbowel resection for Stage II adenocarcinomaof the colon. He has increasingly complainedof shortness of breath throughout the past12 h. Arterial blood gases revealed a metabolicacidosis with hypoxemia. He is intubated andin the intensive care unit (ICU).

70-year-old woman who a colon resection.During the night she became increasingly shortof breath to the point where she needed to beintubated. She was transferred to ICU and is nowon a ventilator.

25-year-old male who has been hospitalizedfor one week following a motor vehiclecollision (MVC). His injuries included multiplerib fractures, pulmonary contusions, pelvicfracture, right tibia–fibula fracture, andruptured bladder. He has developed adultrespiratory distress syndrome (ARDS) and ison continuous renal replacement therapy(CRRT) for non-oliguric renal failure. He issedated, intubated, and ventilated in the ICU.

28-year-old female who sustained multipleinjuries when she was struck by a car whileriding her bicycle. Injuries included multiplefractured ribs, fractured clavicle, pelvic fractureand ruptured bladder. Three days after theinjury, she developed ARDS and subsequently,kidney failure. She was transferred to ICU, whereshe has been in the ICU for 3 days, where she ison a ventilator.

82-year-old man admitted to the hospitalfrom a nursing home with a diagnosis ofurosepsis. Urine cultures revealed E. Coli. Hewas started on Bactrim, but continued to befebrile despite antibiotic therapy. After72 hours, he was transferred to the ICUbecause of declining mental status. He wasintubated and placed on mechanicalventilation. On morning rounds, you notethat he is still febrile.

85-year-old woman admitted to the hospital 4days ago with pneumonia, multiple decubiti anda UTI. She has been on antibiotics sinceadmission but remains febrile. Yesterday, thenurses noted that she was becomingincreasingly lethargic and confused so she wastransferred to the ICU where she was intubatedand placed on a ventilator.

72-year-old male admitted to the coronarycare unit (CCU) with atrial fibrillation andcongestive heart failure (CHF). He is breathingspontaneously, with oxygen by nasal prong.

68-year-old female who was admitted to CCUfollowing an acute exacerbation of mitralregurgitation. She is receiving oxygen by nasalprongs.

e f e r e n c e s

[1] J.A. Effken, Improving clinical decision making throughecological interfaces, Ecol. Psych. 18 (2006) 283–318.

[2] L.T. Kohn, J.M. Corrigan, M.S. Donaldson, To Err is Human:Building a Safer Health System, National Academy Press,Washington, DC, 2000.

[3] L.B. Andrews, C. Stocking, T. Krizek, L. Gottlieb, T. Vargish, etal., An alternative strategy for studying adverse events inmedical care, Lancet 349 (1997) 309–313.

[4] Y. Donchin, D. Gopher, M. Olin, Y. Badihi, M. Biesky, C.L.Sprung, et al., A look into the nature and causes of humanerrors in the intensive care unit, Crit. Care Med. 23 (1995)294–300.

[6] S.D. Surgenor, G.T. Blike, H.L. Corwin, Teamwork andcollaboration in critical care: lessons from the cockpit, Crit.Care Med. 31 (2003) 992–993.

[7] D.M. Eddy, Practice policies: Where do they come from?JAMA 263 (1990) 1265–1269.

[8] S.B. Wachter, K. Johnson, R. Albert, N. Syroid, F. Drews, D.Westenskow, The evaluation of a pulmonary display todetect adverse respiratory events using high resolutionhuman simulator, J. Am. Med. Inform. Assoc. 13 (2006)635–642.

[9] A.P. Bartolone, L.J. Glaab, M.F. Hughes, R.V. Parrish, Initialdevelopment of a metric to describe the level of safetyassociated with piloting an aircraft with Synthetic VisionSystems (SVS) displays, in: Proceedings, SPIE, 2005. Availableat http://techreports.larc.nasa.gov/ltrs/PDF/2005/mtg/NASA-2005-spiedss-apb.pdf.

[5] P.R. Norris, B.M. Dawant, A. Geissbuhler, Web-Based dataintegration and annotation in the intensive care unit, in:Proceedings AMIA Annual Fall Symposium, AmericanMedical Informatics Association, Bethesda, MD, 1997, pp.794–798.

[10] S.K. Card, J.D. Mackinlay, B. Schneiderman, Readings inInformation Visualization: Using Vision to Think, MorganKaufmann Publishers, Inc., San Francisco, CA, 1999.

[11] C.A. Green, R.H. Logie, K.J. Gilhooly, D.G. Ross, A. Ronald,Aberdeen polygons: computer displays of physiological

i c a l

776 i n t e r n a t i o n a l j o u r n a l o f m e d

profiles for intensive care, Ergonomics 39 (1996)412–428.

[12] M. Weinger, K. Gurushanthaiah, Graphical displays mayallow more rapid and accurate detection of acutephysiological changes, Anesthesiology 81 (1994)A579.

[13] W.G. Cole, J.G. Stewart, Metaphor graphics to supportintegrated decision making with respiratory data, Int. J. Clin.Monit. Comput. 10 (1993) 91–100.

[14] F. Hesselvik, C. Carlsson, B. Brodin, L. Jorfeldt, B. Schildt, Ahemodynamic profile diagram and its application in septicpatients, Acta Anaesth. Scand. 29 (1985) 706–711.

[15] R.J. Marshall, Scaled rectangle diagrams can be used tovisualized clinical and epidemiological data, J. Clin.Epidemiol. 58 (2005) 974–981.

[16] S.S. Hussaini, Simple bar graph displays for control variablesand alarm status, in: Electo/94 International, ConferenceProceedings, Combined Volumes, Cariger, Inc., Londonderry,NH, 1994, pp. 651–656.

[17] D.D. Woods, J.F. O’Brien, L.F. Hanes, Human factorschallenges in process control: the case of nuclear powerplants, in: G. Salvendy (Ed.), Handbook of Human Factors,John Wiley & Sons, New York, 1987, pp. 1724–1770.

[18] C.D. Wickens, Engineering Psychology and HumanPerformance, Charles E. Merrill, Columbus, OH, 1984.

[19] K.I. Gurushanthaiah, M.B. Weinger, C.E. Englund, Visualdisplay format affects the ability of anesthesiologists todetect acute physiologic changes: a laboratory studyemploying a clinical display simulator, Anesthesiology 83(1995) 1184–1193.

[20] W. Horn, C. Popow, L. Unterasinger, Support for fastcomprehension of ICU data: visualization using metaphorgraphics, Methods Inf. Med. 40 (2001) 421–424.

[21] J.J. Gibson, The Senses Considered as Perceptual Systems,Houghton-Mifflin, Boston, 1966.

[22] J.J. Gibson, The Ecological Approach to Visual Perception,Lawrence Erlbaum Associates, Inc., Hillsdale, NJ, 1986 (Firstpublished, 1979).

[23] J. Rasmussen, Information Processing and Human–MachineInteraction: An Approach to Cognitive Engineering,North-Holland, New York, NY, 1986.

[24] J. Flach, P. Hancock, J. Caird, K.J. Vicente, Global Perspectiveson the Ecology of Human–Machine Systems, LawrenceErlbaum Associates, Hillsdale, NJ, 1995.

[25] K.J. Vicente, J. Rasmussen, Ecological interface design:theoretical foundations, IEEE Trans. Syst. Man Cybernet. 22(1992) 589–605.

[26] K.J. Vicente, J. Rasmussen, The ecology of human-machinesystems II: Mediating “direct perception” in complex workdomains, Ecol. Psych. 2 (1990) 207–249.

[27] K.J. Vicente, Ecological interface design: progress andchallenges, Hum. Factors 44 (2002) 62–78.

[28] K.J. Vicente, Cognitive Work Analysis: Toward Safe,Productive, and Healthy Computer-based Work, LawrenceErlbaum Associates, Mahwah, NJ, 1999.

[29] T.D. Sharp, A.J. Helmicki, The application of the ecologicalinterface design approach to neonatal intensive caremedicine, in: Proceedings of the Human Factors andErgonomics Society 42nd Annual Meeting, Human Factorsand Ergonomics Society, Santa Monica, CA, 1998, pp.350–354.

[30] G. Blike, The boundary information in a gray-scale objectdisplay and a color-enhanced variant improve problem

recognition compared to an alphanumeric display,Anesthesiology 87 (1997) 458.

[31] G.T. Blike, S.D. Surgenor, K. Whallen, J. Jensen, Specificelements of a new hemodynamics display improves the

i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777

performance of anesthesiologists, J. Clin. Monit. Comput. 16(2000) 485–491.

[32] P. Michels, D. Gravenstein, D.R. Westenskow, An integratedgraphic data display improves detection and identificationof critical events during anesthesia, J. Clin. Monit. Comput.13 (1997) 249–259.

[33] Y. Zhang, F.A. Drews, D.R. Westenskow, S. Foresti, J. Agutter,J.C. Bermudez, et al., Effects of integrated graphical displayson situation awareness in anaesthesiology, Cogn. Technol.Work 4 (2002) 82–90.

[34] J. Agutter, F. Drews, N. Syroid, D. Westneskow, R. Albert, D.Strayer, et al., Evaluation of graphic cardiovascular display ina high-fidelity simulator, Anesth. Analg. 97 (2003) 1403–1413.

[35] J. Effken, R. Loeb, K. Johnson, S. Johnson, V. Reyna, Usingcognitive work analysis to design clinical displays, in: V.L.Patel, R. Rogers, R. Haux (Eds.), Proceedings of MedInfo-2001,London, England, September 2–5, IOS Press, London, 2001,pp. 27–31.

[36] J.A. Effken, Different lenses, improved outcomes: a newapproach to the analysis and design of healthcareinformation systems, Int. J. Med. Inform. 65 (2002) 59–74.

[37] J.C. Toth, Basic knowledge assessment tool for critical carenursing, version four (BKAT-4): validity, reliability, andreplication, Crit. Care Nurs. 14 (3) (1994) 111–117.

[38] T.J. Iberti, E.P. Fischer, A.B. Leibowitz, E.A. Panacek, J.H.Silverstein, T.E. Albertson, A multicenter study ofphysicians’ knowledge of the pulmonary artery catheter,JAMA 264 (1990) 2928–2932.

[39] S.G. Hart, L.E. Staveland, Development of NASA-TLX (TaskLoad Index): results of empirical and theoretical research, in:P.A. Hancock, N. Meshkati (Eds.), Human Mental Workload,North Holland Press, Amsterdam, 1988, pp. 239–250.

[40] P.M. Sanderson, J.M. Flach, M.A. Buttigieg, E.J. Casey, Objectdisplays do not always support better integrated taskperformance, Hum. Factors 55 (1989) 459–472.

[41] J.A. Effken, N.-G. Kim, R.E. Shaw, Making the constraintsvisible: testing the ecological approach to interface design,Ergonomics 40 (1997) 1–27.

[42] T.J. Iberti, E.K. Daily, A. Leibowitz, C.B. Schecter, E.P. Fischer,J.H. Silverstein, Assessment of critical care nurses’knowledge of the pulmonary artery catheter. ThePulmonary Artery Catheter Study Group, Crit. Care Med. 22(1994) 1674–1678.

[43] P.M. Sanderson, M.O. Watson, W.J. Russell, Advanced patientmonitoring displays: tools for continuous informing, Med.Intell. 101 (2005) 161–168.

[44] K. Christoffersen, C.N. Hunter, K.J. Vicente, A longitudinalstudy of the effects of ecological interface design on deepknowledge, Int. J. Hum-Comput. Stud. 48 (2005) 729–762.

[45] J. Effken, R. Loeb, K. Johnson, S. Johnson, V. Reyna,Preliminary evaluation of an oxygenation managementdisplay, in: N. Callaos, L. Hernandez-Encinas, F. Yetim (Eds.),Proceedings of the 6th World Multiconference on Systemics,Cybernetics, and Informatics July 14–18, Orlando (vol. 1:Information Systems Development I), International Instituteof Informatics and Systemics, Orlando, 2002, pp. 279–284.

[46] J.R. Hajdukiewicz, K.J. Vicente, D.J. Doyle, P. Milgram, C.M.Burns, Modeling a medical environment: an ontology forintegrated medical informatics design, Int. J. Med. Inform. 62(2001) 79–99.

[47] C.M. Burns, Navigation strategies with ecological displays,Int. J. Hum-Comput. Stud. 52 (2000) 111–129.

[48] D.-H. Ham, W.C. Yoon, Design of information content andlayout for process control based on goal-means domain

analysis, Cogn. Technol. Work 3 (4) (2001) 205–223.

[49] R.G. Loeb, W.T. Fitch, A laboratory evaluation of an auditorydisplay designed to enhance intraoperative monitoring,Anesth. Analg. 94 (2002) 362–368.

a l i n

G

CCC

C

CD

E

offered by peripheral circulation primarily due to smallartery tone)

i n t e r n a t i o n a l j o u r n a l o f m e d i c

[50] P. Sanderson, The multimodal world of medical monitoringdisplays, Appl. Ergonom. 37 (2006) 501–512.

[51] J.R. Hadjukiewicz, C.M. Burns, Ecological Interface Design,CRC Press, Boca Raton, FL, 2004.

lossary

aO2: arterial oxygen content (SaO2 × Hgb)I: Cardiac Index (CO × body surface area)O: cardiac output (the amount of blood pumped by the heart

per minute (heart rate × stroke volume)vO2: venous oxygen content (oxygen content of venous

blood)VP: central venous pressure

O2: oxygen delivery (amount of oxygen actually delivered;

depends on hemoglobin for transport, cardiac output andavailable oxygen)

R: extraction ratio (fraction of oxygen removed from theblood)

f o r m a t i c s 7 7 ( 2 0 0 8 ) 765–777 777

Hgb: hemoglobinHR: heart rate (number of heart beats per minute)MAP: mean arterial pressure (average pressure in an artery

over complete cycle of one heartbeat)PAWP: pulmonary artery wedge pressure (intravascular pres-

sure measured by a catheter wedged into the distalpulmonary artery; indirect measure of mean left atrialpressure)

SaO2: arterial oxygen saturation (percentage of availablehemoglobin saturated with oxygen)

SV: stroke volume (volume of blood pumped by the left ventri-cle heart with one contraction)

SvO2: mixed venous saturation (SvO2 × Hgb)SVR: systemic vascular resistance (resistance to blood flow

VO2: oxygen consumption (oxygen actually utilized by thebody; can be increased in a variety of conditions such asfever, infection, anesthesia, etc.)