Site visits: Validation of vendor marketing claims Author: Vicky Bradley, RN, MS, Lexington, Kentucky

T he sof tware m a r k e t is ve ry compet i t ive . Vendors a re u n d e r t r e m e n d o u s p r e s s u r e to m e e t our

n e e d s . A t t h e s a m e t ime , our r e q u i r e m e n t s for an e lec t ron ic record in t he e m e r g e n c y s e t t i n g are very complex . P rov id ing s o m e a s p e c t s of t he s y s t e m w e would like to have is not poss ib l e at th is s t a g e of de - ve lopment ; t he pe r fec t s y s t e m tha t would m e e t all of

our n e e d s is no t c u r r e n t l y ava i l ab le , nor will i t b e avai lab le in t he future. Our v i s ion for an idea l s y s t e m is a l w a y s a h e a d of w h a t is real or poss ib le . Conse - quently, s i te v is i t s a re an e s sen t i a l c o m p o n e n t in the s y s t e m se lec t ion process , b e c a u s e t h e y s h o w us w h a t is real. S e e i n g a s y s t e m in ac t ion he lps one va l ida te v e n d o r c l a ims a n d o b t a i n v a l u a b l e i m p l e m e n t a t i o n r e c o m m e n d a t i o n s from peers . P l a n n i n g in a d v a n c e he lps m a x i m i z e the benef i t of a s i te visit . Some ven- dors will p a y for s i te visi ts , and o thers will not; e i ther way, c o n d u c t i n g a s i te v is i t is a s ign i f i can t cos t w h e n

the sa la r ies of the p a r t i c i p a n t s a re cons idered .

Site selection To se lec t a s i te for a visit , ob ta in a l ist of s i t es from vendo r s w h o have success fu l ly a n d fully i m p l e m e n t e d the l a t e s t vers ion of the i r sof tware. Ask the vendor s to s u p p l y s o m e b a s i c d e m o g r a p h i c s a b o u t e a c h s i te . Ideally, t he s i te wou ld b e s imilar to your own environ- ment . Table 1 l ists poss ib l e cr i ter ia to b e m a t c h e d . In- t e rv i ew pe r sons a t t he v e n d o r - r e c o m m e n d e d s i t es b y t e l ephone to ob ta in informat ion tha t will he lp de te r - m i n e w h i c h s i te or s i t e s you would like to visit. ~

Preparation for site visit O n c e a s i t e h a s b e e n s e l e c t e d , f inal ize your ob jec - t ives. W h a t major t h ings do you w a n t to s ee or dis- cuss w i th t he use r s? Table 2 p rov ides an example of a tour checkl is t . Send your ob jec t ives to t he vendor and a sk t h a t t h e y b e f o r w a r d e d to t h e s i te . Deve lop a

Vicky Bradley is Director of Nursing Informatics, University of Ken- tucky Hospital, Lexington. J Emerg Nuts 1998;24:341~3. Copyright © 1998 by the Emergency Nurses Association. 0099-1767/98 $5.00 + 0 18/9/90131

s c h e d u l e wi th the vendor and pe r sonne l a t the si te, Persons a t the s i te you are v i s i t ing m a y c o n d u c t s i te v is i t s regular ly and have a se t schedule . Rev iew the s c h e d u l e in a d v a n c e . If you h a v e s p e c i a l r e q u e s t s ,

s u b m i t t h e m a h e a d of t ime. Table 3 p rov ides a sam- ple tour schedule . Frequent ly , s i tes p r e pa re a p a c k e t of in format ion t ha t d e s c r i b e s the i r facility, miss ion , and goals and p rov ides a synops i s of the i r e xpe r i ence

wi th t he sys tem. Ask for th is p a c k e t in a d v a n c e so it can be u s e d to or ient s i te v is i t t e a m m e m b e r s abou t w h a t to e x p e c t . E n c o u r a g e your s i t e v i s i t t e a m to l imit ques t ions in the c lass room s e t t i n g to ensu re tha t t hey have p len ty of t ime to v i ew the s y s t e m in act ion.

The major goal of any s i te vis i t is to s ee the sys- t e m in ac t ion and to talk wi th e n d users . Mos t s i te v is i t s t ake p lace du r ing only par t of one day, and thus t ime is prec ious . Site v is i t s f requent ly b e g i n wi th in- t roduc t ions and a p r e s e n t a t i o n by s o m e o n e at the s i te a b o u t the i r e x p e r i e n c e w i th t he sy s t e m, w h a t t h e y w o u l d do differently, a n d the i r fu ture p lans . Try to k e e p this par t of t he vis i t to an hour or less to ensu re tha t you have p len ty of t ime to v i ew the sys tem. Ask the vendor to b e a b s e n t du r ing the tour so you can hear u n e d i t e d c o m m e n t s from users . After the tour,

August 1998 341

JOURNAL OF EMERGENCY NURSING/Bradley

Site se lect ion criteria

Patient volume Institution type; eg, rural, urban, suburban, university,

or community hospital, multiple facilities (two emer- gency depa r tmen t s or combination of emergency depar tment , urgent care centers, and pediatric after-hours clinic)

Patient population; eg, adult, pediatric, ches t pain cen- ter, t rauma center

Interfaces wi th the same sys tems you have; eg, admis- sion, discharge, t ransfer (ADT) system, laboratory, radiology, pharmacy, drug dispensing, pat ient ac- counting, acute care, ambulatory

Technical features: radio frequency, mobile laptep com- puters, handheld devices, voice dictation

Sample tour schedule

1-Hour introduction and overview: Hospital 's informa- tion sys tems strategic plan, why sys tem was select- ed, functions currently in place, future plans*

2-Hour tour 1-Hour lunch wi th counterparts 1-Hour practice in tes t sys tem 1-Hour topics: What they would do differently, imple-

mentat ion strategies, project t imehne, quest ion and answer period

*Ask ff one RN can shadow his/her counterpart for 4 hours after the initial overview.

Tour checkl ist

Physical env ironment Types of hardware used Location of hardware Ratio of hardware to staff or rooms Location of large display monitors Location of printers Observe staff using devices for data entry Have the staff had any problems related to ergonomics? Staff comments Pros/Cons

Chart View copies of completed charts Can you print any forms, eg, t ransfer record? View copies of pat ient education materials View copies of pr inted logs What were s tandard reports? What reports have you custom designed? Is the report writer user friendly?

Staff quest ions Is the sys tem user friendly? How long did it take you to get used to the system? What functions do you use? What function do you use the most? What functions do you not use?

What would you change? How does it help you provide care? What do you do w h e n it goes down? How does the sys tem help protect pat ient confidentiality? How do they handle charting during t rauma resuscita-

tion or cardiac resuscitation? Did you change how you practice?

Technical Hardware issues How are devices configured? Does staff have access to other systems? Radio frequency issues, eg, problems wi th dead areas,

batteries? Bar coding issues How long are data s tored on the system? How do staff retrieve stored data; have they ever had to

retrieve data? System response time issues? Downtime issues? What tools do you use to monitor performance? What information is provided via interfaces? What were the s tandard interfaces? Any custom interfaces? Interface issues?

a s k if e a c h v i s i t i n g p a r t i c i p a n t c a n s p e n d t i m e w i t h

h i s or h e r c o u n t e r p a r t a t t h e s i te ; for e x a m p l e , a n RN

w o u l d s h a d o w a n RN, a p h y s i c i a n w o u l d s h a d o w a n

e m e r g e n c y p h y s i c i a n , a n i n f o r m a t i o n s y s t e m s r e p r e -

s e n t a t i v e w o u l d i n t e r v i e w a n i n f o r m a t i o n s y s t e m s

r e p r e s e n t a t i v e , a m a n a g e r w o u l d m e e t w i t h a m a n a g -

er, a n d a n e d u c a t o r w o u l d m e e t w i t h a n e d u c a t o r .

T h e n b r i n g t h e g r o u p b a c k t o g e t h e r in t h e s i t e ' s t r a in -

i n g room. A s k s i t e p e r s o n n e l for a c c e s s to t h e i r t e s t

e n v i r o n m e n t so y o u c a n h a v e h a n d s - o n t i m e m a n e u -

v e r i n g in t h e i r s y s t e m . B r i n g a f e w of your t e s t s c e -

n a r i o s a n d t ry to e x a m i n e r e q u i r e d func t iona l i ty . You

c a n eas i ly s p e n d a n h o u r in t h e t e s t s y s t e m . Be s u r e

to t e s t a n y a r e a s a b o u t w h i c h y o u still h a v e q u e s t i o n s

or c o n c e r n s r e g a r d i n g s y s t e m func t iona l i ty . A s k a n y

u n a n s w e r e d q u e s t i o n s . S o m e s i t e s m a y b e wi l l ing to

s h a r e job d e s c r i p t i o n s , p r i n t e d c h a r t c o p i e s , fo rms , re-

po r t s , po l i c i e s , a n d p r o c e d u r e s . B e c a u s e m a n y sof t -

w a r e p r o d u c t s a l low t h e s i t e to c u s t o m i z e s c r e e n s , b e

ca re fu l t ha t , in your eva lua t i on , y o u a re j u d g i n g t h e

s o f t w a r e i t se l f a n d n o t h o w t h e h o s p i t a l h a s i m p l e -

m e n t e d t h e s o f t w a r e . T h e s a m e s o f t w a r e c a n b e s u c -

342 Volume 24, Number 4

Bradley/JOU][hNAL OF EMERGENCY NURSING

cessfully or unsuccessfully installed based on hospital resources and implementation strategies. Question- ing the vendor and the site personnel can help clarify software issues versus implementation issues.

If you visit multiple sites at one facility--for ex- ample, two different emergency depar tments - -con- sider splitting the visit between 2 days. Take an early flight and begin the first session and tour in the after- noon at first site. On day two, visit the second site in the morning and return home in the afternoon. Stay- ing overnight allows the site visit team to compare notes and determine questions or observations they wish to include during day two of the visit, and it keeps them from becoming overwhelmed with too much information. Comments and observations should be recorded after each site visit; waiting until after both site visits can lead to confusion and forget- ting where one saw what.

Ask permission to take pictures of physical de- vices or physical layout. These pictures can be used to facilitate physical redesign and to share with the steering committee and other users.

Site visit participants Who should go on the site visit? The team should con- sist of the project leader, representatives of nursing and physician management, system users (front-line staff), and an information sys tems representative. Each member will bring a slightly different perspec- tive and focus and may be assigned a specific area of responsibility. For example, the frontqine emergency nurse could be charged with investigating how well staff nurses accept and use the new system, the im- pact it has had on their practice, and what they like and dislike about the system. The physicians might be responsible for talking with colleagues and other physician consultants to assess their views of the sys- tem. The project coordinator may pay more attention to physical layout, project timeline, and cost-benefit analysis. The information sys tems representat ive

would likely focus on hardware, system reliability, and interfaces. The ED manager may focus on forms, poli- cies, reports, and patient and staff satisfaction. The educator would probably focus on orientation, imple- mentation, and how staff adapted to the change.

Evaluation of the site visit When evaluating the site visit, ask each participant to share his or her assessment and evaluate the sys- tem's ability to meet your project objectives. Rank the system according to the selection criteria. You can use the same evaluations that you used for the sys- tem demonstrations. 2 Compare what you saw with what the vendor said the system would do in the re- quest for proposal (RFP), the functionality displayed at system demonstrations, and the responses from other references. If any discrepancies in system functional- ity still exist, clarify them with the vendor. Be sure to save a list of any promises made by the vendor so they can be included in the contract negotiations. If you like what you saw on the site visit and believe the system meets your project objectives, you are ready for contract negotiations, the last component of sys- tem selection.

It is a continual wonder that each emergency de- partment believes they have purchased and imple- mented the best system. Each emergency depart- ment may have looked at the exact same systems offered by the same vendors, and each emergency de- partment came to a different conclusion. Much more is involved in successful implementation than just the software and hardware. The site visit process can be used to gain valuable information about implementa- tion, resources provided by the hospital, and the change process, and it can be used as a process to validate that what you see is what you get.

References

1. Bradley V. Reference checks: a tool for system selection. J Emerg Nurs 1998;24:175-7. 2. Bradley V. Scripted demonstration: a screening tool for software system selection. J Emerg Nuts 1998;24:263-6.

Contributions to this column may be sent to 40I 7 Palomar Blvd,

Lexington~ KY 40513; phone (606) 257-2350 (E-mail address: bradley@pop.uky.edu) or .....

i414 Ashland Ave, St Paul, MN 55104,. (612) 347-8796 (E-mail address: marsha.zimmerman @co.hennepin. ran. us).

August 1998 343