Intensive and Critical Care Nursing (2004) 20, 62—68

ORIGINAL ARTICLE

Moving on from ‘patient dependency’ and ‘nursingworkload’ to managing risk in critical care

Carol Ball a,b,*, Guy Walkerc, Paul Harperc, Deborah Sandersa,Maura McElligottb

a Royal Free Hampstead NHS Trust, Pond St, London NW3 2QG, UKb City University, London, UKc School of Mathematics, University of Southampton, Southampton, UK

Accepted 3 February 2004

Summary The contribution nurses make to the management of critically ill patients isusually appraised through the use of concepts such as ‘patient dependency’ or ‘nurs-ing workload’. These concepts fail to address the knowledge, skills and experienceof nurses and consequently fail to acknowledge the risk presented by critically ill pa-tients. This paper describes the development of a tool that attempts to measure riskand the process of risk management undertaken by nurses who coordinate the shiftsand lead the nursing team. The results of this pilot study indicated that the tool wasvalid, but reliability has not yet been demonstrated. Thus the tool requires furtherrefinement and testing. We chose to publish at this time because we feel the paperoffers a new way of examining the contribution of nurses working in critical care.© 2004 Elsevier Ltd. All rights reserved.

Introduction

A number of terms or constructs are currently inuse, for example ‘workload’ and ‘dependency’that are associated with the activity of nurses incritical care. In addition a number of instrumentsalso exist which purport to measure the above con-structs. These instruments have sometimes beenused to appraise the work of nurses specifically(Miranda et al., 1996, 1997; Hurst, 2003) and themultidisciplinary team in general (ICNARC, 1999).However, in many cases the terminology used canbe ill defined and the specific instruments utilisedlack validity because they frequently fail to:

*Corresponding author. Tel.: +44-20-7472-6137;fax: +44-20-7472-2469.

E-mail address: carol.ball@rfh.nthames.nhs.uk (C. Ball).

• appraise indirect nursing activity (e.g. supervi-sion of juniors);

• incorporate the expertise and knowledge ofnurses (i.e. assume a nurse is a nurse is a nurse);

• assess the degree of risk particular patients pose(e.g. the complexity of intervention, the vulner-ability of the patient in terms of deterioration).

Simply measuring ‘workload’ or ‘dependency’,however ill defined, does not address the elementof risk associated with the care and managementof the critically ill by nurses (Adomat and Hicks,2003; Garfield et al., 2000). In critical care thenurse is a constant presence, whereas all othermembers of the multidisciplinary team are presentonly on an intermittent basis. It is this that makesthe profession’s contribution unique in this era ofmulti-tasking (Ball and McElligott, 2003). A largenumber of research studies have begun to identify

0964-3397/$ — see front matter © 2004 Elsevier Ltd. All rights reserved.doi:10.1016/j.iccn.2004.02.001

Managing risk in critical care 63

the ‘risk’ patients face in the absence of skilled,experienced and knowledgeable nurses in criticalcare specifically (Amaravadi et al., 2000; Binnekadeet al., 2003; Dang et al., 2002; Dimick et al.,2001; Fridkin et al., 1996; Garfield et al., 2000;Provonost et al., 2001; Robert et al., 2000; Whitmanet al., 2002a, 2002b), in hospitals generally (Aikenet al., 2002) and within the multidisciplinary team(Tarnow-Mordi et al., 2000).The aim of the current study was therefore to

commence the development of an instrument thatassociates the concept of ‘risk’ with the provisionof skilled, experienced and knowledgeable nurses.This was undertaken in conjunction with the Facultyof Mathematical Studies, University of Southamp-ton, who helped with the development of the in-strument and were responsible for data analysis.

Aims of the study

1. To define risk associated with the critically illpatient in terms of a conceptual framework.

2. To develop a preliminary instrument to appraiserisk and associate this with the level of nurserequired to reduce risk.

3. To correlate this with the level of nurse avail-able.

4. Undertake preliminary validation of the instru-ment in one critical care unit.

Methods

The first stage of the study was to clarify a new con-struct to differentiate our work from ‘dependency’and ‘workload’. This was termed: Managing Risk inthe Care of the Critically Ill Patient.

Working definitions

A working party was formed to define the constructand develop the research instrument. This com-prised the principal investigator, research assistant(M.Sc. student from Southampton University, thesenior nurse and nurses in charge of the intensivecare unit on a daily basis). Risk was deemed to bethe possibility of misfortune or loss being incurred.Key factors in the construct were seen to be ‘actual’and ‘potential’ risk. That is to say there could be areal hazard such as the inappropriate managementof a labile blood pressure or a potential problem,such as the risk of aspiration pneumonia as a resultof naso-gastric feeding. In developing an instrumentto measure risk it would be important to incorpo-rate both ‘real’ and ‘potential’ problems.

Managing was associated with being in chargeof the intensive care unit. The key people on theunit who ‘managed’ the critical care units were theNurses in Charge (NIC). This group was thereforechosen to determine the risk posed by critically illpatients. The NIC managed risk through the alloca-tion of nursing staff on the unit to particular pa-tients. In maintaining a ‘safe’ environment for thepatient the knowledge and experience of the nursewas taken into account and allocation made ac-cordingly. That is not to say that nurses were notgiven a choice of patient to care for, but that NICswere aware of the individual abilities of nurses andmanaged this proactively. For example, if a juniornurse requested a complex patient, then provisionwas made to reduce risk by allocating a more seniornurse to care for the patient in the adjacent bedarea or the junior nurse’s mentor was allocated towork with the same patient. The study thereforecomprised two essential elements. The first was thedevelopment of an instrument that represented pa-tient risk, the second determined the levels of nurseworking on the intensive care unit. Essentially, thepatient was thought to be at greater risk if the ap-propriate level of nurse was not, or could not, beallocated to them.

Instrument development

The NICs were interviewed individually to ascer-tain what factors they took in to account whenthey considered allocation for a shift. They wereasked only one broad question, ‘‘What factors doyou take into account when you are considering theskill mix available for a particular shift?’’ A numberof diverse elements were suggested. A previousstudy carried out by two of the authors derived anumber of categories associated with the contri-bution nurses made to the care and managementof critically ill patients. It was not our intention touse these categories in this study but the same is-sues were raised by the NICs repeatedly and thesethen became the categories for the prototype in-strument. Namely, these were:

• Patient centred• Proactive• Vigilance• Emotional support (Ball and McElligott, 2003)

Examples of the items developed can be seen inTable 1. The patient-centred category representeditems associated with the patient’s physical sta-tus. Proactive represented the forward thinkingrequired to move the patient towards recoveryand was associated with ‘real’ problems. Vigilance

64 C. Ball et al.

Table 1 Examples of the items derived under each category.

Patient centred1. Need for increased infection control measures (e.g. multi-drug resistant tuberculosis (TB),

immunocompromised)2. Requiring motivation and encouragement (e.g. weakness as a result of critical care neuropathy)3. Hygiene needs impact on clinical status (e.g. profuse diarrhoea could impact on haemodynamic state due

to need for turning, fluid loss or both)

Proactive1. Titration of multiple vaso-active drugs2. Manipulation of preload, afterload and contractility using the oesophageal Doppler or pulmonary artery

catheter3. Actual haemodynamic instability–—from dysrythmias, severe blood loss requiring multiple colloid

replacement (whole blood, platelets and fresh frozen plasma), septic shock, electrolyte imbalance4. Manipulation of ventilatory parameters for weaning or deteriorating respiratory status5. Need for active rehabilitation following prolonged period in critical care and potential for critical care

neuropathy

Vigilance1. Airway: insecure or grade 4 intubation. Potential for intubation/re-intubation2. Active interventions being taken to avoid intubation3. Potential haemodynamic instability–—from dysrythmias, severe blood loss requiring multiple colloid

replacement (whole blood, platelets and fresh frozen plasma), septic shock, electrolyte imbalance4. Intricate fluid balance manipulation, e.g. multiple drainage, continuous veno-venous haemofiltration

(CVVH), multiple blood products

Emotional support1. Patient–—high requirement for psychological support-anxiety, substance withdrawal, sensory

deprivation/overload2. Relatives–—high requirement for psychological support-anxiety, need for in depth information3. Relatives potentially violent4. Brain stem death or permanent vegetative state tests to be assessed or confirmed

represented issues that might arise but might notand were associated with ‘potential’ risk. Emo-tional support represented the psychological carerequired both of patients, relatives and peers. Indeveloping the instrument two default items weregenerated when the NICs would automatically allo-cate a ‘competent’ level of nurse. These were:

• Patient in side room with multiple organ failure;• Complex transfer for, e.g. computerised tomog-raphy.

The total number of items generated was 37. Thesecond stage of the study was to categorise nursesaccording to their knowledge and experience andexamples of this are represented in Table 2. Nursesworking on the unit were then grouped by two se-nior nurses on the intensive care unit. Each seniornurse undertook this independently. Consensus wasthen achieved through discussion. NICs were not in-formed of the level of nurse allocated so that theycould participate in the validation of the instrumentwithout bias. The final version of the instrument al-lowed the association of a nurse with a particular

patient. This was achieved by deciding which itemsimmediately required a competent nurse and theweighting of the instrument to enable this level ofnurse to be identified.If none of the items generated were scored,

then a novice level nurse was deemed appropriate.The weightings for each category are portrayed inTable 3. Proficiency was not seen by the workingparty to be a particular ability for an individual skill(e.g. the management of continuous veno-venoushaemofiltration) but an ability to manage multi-ple patient problems (e.g. sepsis combined withmultiple organ failure together with relatives whorequired detailed and explicit information and ajunior colleague requiring educational support atthe bedside). Essentially the greater the score(Tables 1 and 3), the higher the level of nurserequired (Table 2).

Ethical issues

The research instrument was completed after theNICs allocation was complete. It was not used topredict allocation. Therefore, because the instru-

Managing risk in critical care 65

Table 2 Level of nurse.

Novice Successful completion of supervised practice period <6 months experience in critical careRequires a lot of indirect supervision in caring for critically ill patientsUnfamiliar with usual ICU technology and equipment

Advanced beginner Completed D grade development programme, or completed transition (Universityprovided), or equivalent from abroadRequires some supervision in the management of critically ill patientsFamiliar with routine equipment and technology in ICU (but not CVVH)

Competent >18 months experience and appropriate intensive care modules (University provided) andfamiliar with routine equipment and technology in ICU (including CVVH)Awareness of Protocols and Policies of the Units/Standards of PracticeDiscusses individual management plan with medical staffAble to carry out management plan independently (usual type of unit patient)

Proficient >2 years experience and and appropriate intensive care modules (see competent)Able to carry out management plan independently (usual type of unit patient and thoseless common to the unit)Able to co-ordinate the care of several critically ill patients (team leading) with noviceand advanced beginnersAble to communicate effectively with members of staffHelps develop–—Protocols and Policies of the Units/Standards of PracticeEvidence of knowledge updateEnsures compliance with unit policies and procedures

ment did not influence patient management ethicalapproval was not required.

Data collection

Data collection was carried out on an intensivecare unit with 20 beds over a 1-month period in Au-gust 2003. The research instrument was completedby the NIC for every patient during each shift

Table 3 Weighting of instrument.

Category Score Level of nurseallocated byinstrument

Default items(2 items)

Competent

Patient centred(4 items)

1—2 Advanced beginner

3—4 CompetentProactive (14 items) 1—2 Advanced beginner

3—7 Competent8—14 Proficient

Vigilance (11 items) 1—6 Competent7—11 Proficient

Emotional support(6 items)

1—2 Advanced beginner

3—4 Competent5—8 Proficient

(07:45—20:00 and 19:45—08:00 h, respectively). Atotal of 878 instruments were completed. The dataset was compiled and analysed using Excel. Analy-sis of the data was limited to descriptive statisticsas the study represented the beginning of the vali-dation process.

Results

Validity

We did not have a defined output (e.g. an out-come, such as life or death). In this case the bestmethod of validation was to use experts in the re-lated field (Ebener, 1985). We counted the numberof occasions where the nurse’s opinion on levelof nurse required matched the level assigned bythe research instrument. To achieve this we askedtwo questions, ‘‘What level of nurse has beenassigned by the research instrument?’’ and ‘‘Inyour opinion what level of nurse should have beenassigned?’’ The level of agreement was 81.2%. Theresearch instrument over-scored (i.e. allocated ahigher level nurse than the opinion of the NIC)on 13.6% of occasions and underscored on 5.1%of occasions. The major difference for under andover-scoring was accounted for by a lack of agree-ment between competent and advanced beginnerlevels.

66 C. Ball et al.

Reliability

To test reliability three nurses assessed 29 patientson three separate occasions. The best scenariowould have been for the three nurses to undertakethis on all three occasions but unfortunately thiswas not possible. Therefore, the same nurses wereinvolved on two occasions and on one occasion twofurther nurses were included. This meant that onlyone nurse was consistent throughout and five nursesappraised reliability altogether. The scores for thenurse who was consistent throughout the appraisalof reliability were used as the ‘gold standard’ andother results from the four remaining were com-pared with these. The gold standard score waschosen as the average score of each patient sincenone of the NICs could be said to be definitivelycorrect. It was assumed that the (score given byone NIC minus the gold standard score) values wereindependent and identically distributed normalrandom variables. A null hypothesis that the meanof the normal distribution was equal to 0 was made.Thus, a Student’s t test was performed to see ifthe null hypothesis should be rejected. The sameprocedure was then repeated on the other fourNICs.Only 51.7% (n = 29) scores derived from the

research instrument agreed. If opinions were com-pared only 44% agreement was obtained. One nursecontinuously scored above and one consistentlyscored below the ‘gold standard’ score. Furtheranalysis demonstrated that even where the scores,and therefore the nurse assigned by the instrumentagreed, the same items were not always chosento derive the level of nurse required. On four oc-casions the nurses involved in testing reliabilityscored levels as diverse as novice, advanced be-ginner and competent for the same patients. Thecategory that caused the greatest divergence whenscoring the research instrument was ‘Vigilance’.In particular item 26–—GCS less than 8 or potentialto drop from 15 caused the greatest diversity ofscores. The most consistent was item 29–—intricatefluid balance manipulation (e.g. CVVH, multipledrainage, multiple blood products required).

Discussion

The instrument is not reliable in its current format.It was assumed that because, during interviews,the same issues were raised by NICs in their man-agement of risk that there was a shared under-standing of what the individual items meant. Thiswas not the case. Further work needs to be under-taken specifically in two different areas. Firstly,

there is a need to define the items present onthe research instrument so that a shared under-standing is attained. Secondly, in the ‘Vigilance’category the word ‘potential’ should be replacedwith the word ‘probability’. The reason for this isthat where over-scoring occurred it was thought,by the nurses participating in the study, that evenpatients ready for discharge had the potential todeteriorate because they had been critically ill. Ifthe word ‘probability’ replaced ‘potential’ nursewould then be required to consider the likelihoodof such an event taking place.Some items also seemed to be more impor-

tant than others, for example, item 29–—intricatefluid balance manipulation (e.g. CVVH, multipledrainage, multiple blood products required). It isalso possible that in the further refinement of theinstrument we need to consider weighting the itemsto reflect this. Possibly this might be achieved bysquaring the numbers derived for each category asis the case with the Injury Severity Scoring System(ISS) (Baker et al., 1974).The participants in the study identified further

problems related to scoring the instrument dur-ing presentation of the findings. Many of the NICsthought that their knowledge of the patient woulddiffer depending on how many shifts they hadworked. The implication of this was that if consec-utive shifts had been worked the NIC would knowthe patient better and allocation of a certain levelof nurse might differ from that made by a NIC whohad been on ‘days off’ and relied on the ‘han-dover report’ to make a judgement. Therefore,the nurses involved in determining the reliabilityof the instrument would have been influenced byhow well they knew the patient.In the future development of the instrument we

hope to have three nurses assessing reliability whoall have equal knowledge of the patients’ severityof illness. To ensure this we aim to invite NICs fromoutside the Trust to make their decision regardingpatient risk based on knowledge of the geographyof the unit but who rely on ‘the handover report’to gauge the risk the patient presents and conse-quently the level of nurse required.If the research instrument had been reliable it

would be able to predict the skill level requiredon a particular ICU. This study demonstrated thatthe following composition of nursing staff wasrequired–—novice 4%, advanced beginner 34%, com-petent 59% and proficient 3%. The lack of reliabilityprecludes any firm conclusion to be drawn fromthis but it does demonstrate the potential of theinstrument to estimate the composition of nursingstaff on the ICU in order to manage risk rather thanonly account for workload or patient dependency.

Managing risk in critical care 67

The validity of the instrument appears to be goodat 81.2%, particularly when the large numbers ofitems within the instrument are considered. How-ever, it could be that the nurse scoring the instru-ment had already decided that a certain level ofnurse was required and then completed the instru-ment in accordance with this belief.

Conclusion

Currently, our instrument, though apparently valid,lacks reliability. However, we feel it is importantto publish our work as it offers a new way of think-ing about nurse staffing in intensive care. That is,to change from the perennial discussion around‘dependency’ and ‘workload’ that both fail to ad-dress the knowledge and experience nurses requireto care for the critically ill, to one that evaluatesthe impact of nursing on patient care in reducingrisk.The ultimate aim, or outcome measure, of this

would be to use the instrument to appraise thequality of care given. Measurable outcomes couldbe length of stay, nosocomial infection rates,critical incidents and even possibly standardisedmortality ratios. For example, if the instrumentdemonstrated that the majority of nurses requiredby the instrument should be competent, by ap-praising the level of risk patients presented, butthe ICU lacked this level and instead had a predom-inance of novices this might explain why criticalincidents and nosocomial infection rates were highand/or length of stay prolonged. The increased in-cidence of such problems has also been the subjectof concern in the Netherlands, where a lack of in-tensive care nurses has led to increased risk in anerror prone environment (Binnekade et al., 2003).It would not be possible to suddenly employ com-petent nurses but it would provide an argument forsupporting novice nurses possibly through the em-ployment of clinical practice educators in order toimprove the competence of novices. It might alsoprovide an opportunity to reconsider why nursesare promoted.Currently, in the United Kingdom, promotion is

given for increased managerial responsibility in ourhigher grade nurses. However, there might also bean argument for nurses being promoted in recogni-tion of their enhanced clinical skills as part of theirjob evaluation (DoH, 2003). These nurses wouldthen be recognised for their ability to supportjunior nurses. This innovation has already beendescribed as the introduction of ACCESS nurses inAustralian ICUs, where the acronym stands for theprovision of Assistance, Co-ordination, Contingency

(for a late admission or staff sickness mid-shift),Education, Supervision and Support (AustralianCollege of Critical Care Nurses, 2002). The guide-lines from the Royal College of Nursing CriticalCare Forum (2003) also emphasise the importanceof the knowledge and experience nurses have whenstaffing is considered (RCN, 2003).Definitive proof that nurses and nursing make a

difference in the care of the critically ill remainsillusive. There is therefore a need to continue workon the refinement of the current ‘Managing Risk’instrument that we hope to undertake in the nearfuture.

References

Adomat R, Hicks C. Measuring nursing workload in intensivecare: an observational study using closed circuit video cam-eras. J Adv Nurs 2003;42(4):402—12.

Aiken L, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospitalstaffing and patient mortality, nurse burnout and job dis-satisfaction. JAMA 2002;288(16):1987—93.

Amaravadi, Dimick JB, Provonost PJ, Lipsett PA. ICUnurse-to-patient ratio is associated with complications andresource use after esophagectomy. Intensive Care Med2000;26:1857—62.

Australian College of Critical Care Nurses. Position statementon intensive care staffing. Aust Crit Care 2002;15: 6—7.

Baker SP, O’Neill B, Hadon Jr W, Long WB. The injuryseverity score: a method for describing patients with mul-tiple injuries and evaluating emergency care. J Trauma1974;14:187—96.

Ball C, McElligott M. Realising the potential of critical carenurses. Intensive Crit Care Nurs 2003;19:226—38.

Binnekade JM, Vroom MB, Kesecioglu J. The risk of nursing in anerror prone environment. In: Vincent J-L, editor. Yearbookof intensive care and emergency medicine. 2003. p. 963—7.

Dang D, Johantgen ME, Provonost P, Jenckes MW, Bass EB. Post-operative complications: does intensive care unit staff nurs-ing make a difference? Heart Lung 2002;31(3):219—28.

Dimick JB, Swoboda SM, Provonost P, Lipsett PA. Effect ofnurse to patient ratio in the ITU on pulmonary complica-tions and resource use after hepatectomy. Am J Crit Care2001;10(6):376—82.

Department of Health. Job evaluation handbook. March 2003.Ebener MK. Reliability and validity basics for evaluating classi-

fication systems. Nurs Econ 1985;3:324—7.Fridkin SK, Pear SM, Williamson TH. The role of understaffing in

central venous catheter-associated bloodstream infection.Infect Control Hosp Epidemiol 1996;17:150—8.

Garfield M, Jeffrey R, Ridley S. An assessment of the staffinglevel required for a high-dependency unit. Anaesthesia2000;55:137—43.

Hurst K. Selecting and applying methods for estimating the sizeand mix of nursing teams. Nuffield Inst Health 2003.

Intensive Care National Audit and Research Centre. The IC-NARC system of patient related activities (SOPRA). London:ICNARC; 1999.

Miranda DR, de Rijk A, Schaufeli W. Simplified therapeutic in-tervention scoring system: the TISS-28 items. Results froma multicentre study. Critical Care Med 1996;24:64—73.

68 C. Ball et al.

Miranda DR, Moreno R, Iapichino G. Nine equivalents ofnursing manpower use scores (NEMS). Intensive Care Med1997;3:760—5.

Provonost P, Dang D, Dorman T, Lipsett L, Garrett E, JenckesM, et al. Intensive care unit nurse staffing and the riskof complications after abdominal aortic surgery. Am CollPhysicians 2001;4(5):199—206.

Royal College of Nursing. Guidelines for nurse staffing in criticalcare. RCN 2003.

Robert J, Fridkin SK, Blumberg HM. The influence of the compo-sition of the nursing staff on primary bloodstream infection

rates in a surgical intensive care unit. Infect Control HospEpidemiol 2000;21:12—7.

Tarnow-Mordi WO, Hau C, Warden A, Shearer AJ. Hospital mor-tality in relation to staff workload: a 4 year study in anadult intensive care unit. Lancet 2000;356:185—9.

Whitman GR, Yookyung K, Davidson LJ, Wolf GA, Wang S-L.The impact of staffing on patient outcomes across specialityunits. J Nurs Admin 2002a;32(12):633—9.

Whitman GR, Yookyung K, Davidson LJ, Wolf GA, Wang S-L.Measuring nurse-sensitive patient outcomes across specialityunits. Outcomes Manage 2002b;6(4):152—8.