CSI Cyberspace: A Multiple Case Study Investigation of the

Untimely Demise of Seven Virtual Reference Services

Marie L. Radford, Ph.D., Rutgers Universitymradford@scils.rutgers.edu

M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign

katkern@uiuc.edu New Reference Research: 11th Annual Reference Research Forum, ALA, Chicago June 25, 2005

 

Problem Statement

Number of chat reference services growing 3000+ chat & IM (Coffman & Arret, 2004)

Alive & Clicking! Many services are successful and viewed as

integral Clicked the Bucket!

However, a number of chat services have died untimely deaths.

Why?

Research Questions What were the deciding factors for ending these

services? Who was involved in making the decision to end the

services? How could these failures/false starts have been

prevented? What has been the impact of discontinuing these

services? What are the similarities and differences among the

cases? What lessons can be learned to ensure the success

of existing services or newly developing services?

Modus Operandi - Method

Multiple Case Study Method Selection of Cases & Informants

7 Cases Data Collection

Email Surveys Telephone Interviews Examination of Documents

Method Continued

Analysis of Results Case Descriptions of Each Case Comparison of Cross – Case Findings

Similarities Differences

Iterative Process Examination of Documents

Results Despite our original & advertised title

we had 7 cases not 6. 5 Academic 1 Public 1 Consortium

One surprise, all 7 agreed to be interviewed & share their experiences.

Another surprise – Most (5-7) had no grant funding!

Starting & Operating the Service – Common Themes

No additional staff hired. Broad target audience – same as for in-

person service. Distance education students a focus. Limited hours of operation.

Average 22.4 hours per week User response positive.

Postmortem - 5 Major Reasons for Discontinuing Services

Funding – Budget CutsLow UseStaffingTechnical Problems Institutional Culture

Reasons for Discontinuing Service

Low

Use

Funding Staffing Tech.

Prob.

Inst.

Cult

Primary 3 3 1 1 0

Note: Totals to 8 since Case B listed 2 primary factors (Technical Problems and Low Use).

Reasons for Discontinuing Service

Low

Use

Funding Staffing Tech.

Prob.

Inst.

Cult

Primary 3 3 1 1 0

Secondary 2 0 2 0 1

Total 5 3 3 1 1

CSI Cyberspace Mystery

Why did these services have such low volume?

Exhibit A - VolumeCase Monthly Qs % of total Qs

A 35 <1%

B 35 1.5%

C 30 <1%

D 36 3% (less if all branches included)

E 69 <1%

F 120-160 Unknown

G 7 2%

Exhibit B – Marketing of Service Case Amount Comments

A Moderate Web, Displays, Flyers, Lib. Instr, Newspaper

B Extensive Web, Branding, Articles, Lib. Instruction

C Moderate Target Pop., Flyers, Bookmarks, Library Instruction, Plan to Increase

D Extensive Web, Newspaper, Ads, Signs, Posters, Table Tents

E None Limited hours precluded marketing push.

F Little Pilot training & exploration period. G Extensive Email Announcement, Meetings, Giveaways,

E-Newspaper, Library Instruction

Exhibit C – Marketing of Service –Low Vol.Case Amount Comments

A Moderate Web, Displays, Flyers, LUI, Newspaper

B Extensive Web, Branding, Articles, LUI

C Moderate Target Pop., Flyers, Bookmarks, LIU, Plan to Increase

D Extensive Web, Newspaper, Ads, Signs, Posters, Table Tents

E None Limited hours precluded marketing push.

F Little Pilot training & exploration period. G Extensive Email Announcement, Meetings, Giveaways,

E-Newspaper, LIU

Exhibit D – Hours of ServiceCase Hours of Service

A M-F 3-8pm 20 hrs. per week (changed hours)

B M-Th 1-7pm, F 1-5pm 28 hrs. per week

C M-Th 5-8pm 12 hrs. per week

D M-F 1-5pm 20 hrs. per week

E Su-Th 7-10pm 18 hrs. per week

F M-Th 2-8pm 24 hrs. per week

G M-F 10am-5pm 35 hrs. per week

Exhibit E – Length of ServiceGiving VR a Fair Trial

Case Trial/Pilot Length of service

A 1/2002 – 4/2004 26 months

B 1/01 – 8/02 9/2001 – 12/2002 7 months pilot

15 months

C 5/2002 – 4/2004 21 months

D 10/2001 – 4/2002 6 months

E 1/02 - 5/02 1/2003 – 5/2004 5 months pilot

14 months

F 9/2004 – 3/2003 6 months

G 7/2002 – 2/2004 20 months

Giving VR a Fair Trial …

Shortest services ran 6 months (2 services) Longest run was 26 months Average of 17 months

Exhibit F – Service Evaluation

User

Eval?

Type of Eval?

Users

Satisfd?

Comments

A Y Survey & Trans.

Yes Mixed librarian perceptions

B Y Survey & Assmt.

Yes Low (19%) return rate.

C N ? Planned 2nd yr. survey

D N ?

E Y Survey Yes 68% “very satisfied”

40% return rate

F Y Survey Yes Staff satisfied too

G Y Survey Yes Users highly satisfied, low return rate.

Impact of Closing Services

Common Trends Demise had limited impact. Users did not notice. Email reference (phone service) more

promotion, more use. Unique Responses

Staff relieved, less pressure.

Lessons Learned

If doing a multiple case study, don’t put a number in your title for conference papers.

Consortium necessities: Full-time coordinator Variety of libraries

Some reference questions are better answered by email.

More Lessons Learned…

Critical factors: Staff training & enough volume for staff to

stay primed, comfortable staffing patterns User needs assessment

What hours for service? What software?

User evaluation Marketing Design prominent & easily identifiable link

The Mystery Remains UnsolvedOngoing Investigation… Season 2? Future Research Directions Low volume, what are critical factors? Have identified 3 additional cases.

Perhaps more will be identified today.

More rigorous examination of documents. Research on successful services – what factors

ensured their success? User expectations of service – what hours, days?

More on users. More research on non-users

Future Directions for Services

Investigate free & simpler software IM or refTracker

Consortia Share Cost Share Staff

Continue to scan landscape

There’s Hope…Future Directions for CasesCase Reinstate? Comments

A Maybe In consortium, if users want it

B Maybe Simple Chat, IM, or RefTracker, Summer ’05

C Yes If budget & staffing restored

D Maybe Possible if consortium

E Maybe Possible if consortium (doubtful)

F Yes Reorganization in process, hope to reopen in Jan. ’06

G Maybe Something free like IM

Conclusion

PPT slides will be posted at:

RUSA Reference & Statistics Committee Sitehttp://www.ala.org/RUSA

Thanks & Questions???

Marie L. Radford, Ph.D., Rutgers Universitymradford@scils.rutgers.eduM. Kathleen Kern, MLS, University of Illinois at Urbana-Champaignkatkern@uiuc.edu