lynn berard / bella gerlich the read scale g. lynn berard bella karr gerlich

Lynn Berard / Bella Gerlich

The READ Scale

G. Lynn Berard

Bella Karr Gerlich


The READ Scale Reference Effort Assessment Data

A six point (1 - 6) sliding scale that asks librarians to assign a number based on effort / knowledge / skill / teachable moment instead of a hash mark after a reference transaction. The READ (Reference Effort Assessment Data) Scale Creative Commons License This work is licensed under a Creative Commons Attribution- Noncommercial- No Derivative Works 3.0 Unported License. Bella Karr Gerlich


Objective

Test the validity of the READ Scale as an additional tool for gathering reference statistics to record and recognize the effort / knowledge / skills / value-added service required during a reference transaction.

READ Scale Study


READ Scale Study

Recruit 9 - 15 academic libraries Public and private Geographically diverse Various enrollment size


14 Institutions, 12 States


The InstitutionsEnrollment under 5,000

Eastern Virginia Medical SchoolEdward E. Brickell Medical Sciences Library, Norfolk, VA

Lawrence UniversitySeeley G. Mudd Library, Appleton, WI

Lewis & Clark CollegeAubrey R. Watzek Library, Portland, OR

Clarke CollegeClarke College Library, Dubuque, IA

Our Lady of the Lake University San Antonio (OLLUSA)Sueltenfuss Library, San Antonio, TX


The InstitutionsEnrollment over 5,000

Georgia College & State UniversityGCSU Library & Instructional Technology Center (2 Service Points) Reference & Special Collections, Milledgeville, GA

Carnegie Mellon UniversityHunt Library, Arts,& Spec Coll, Science Library, Mellon Inst, Music Listening, Software Engineering Institute (6 Service Points) Pittsburgh, PA

Robert Morris UniversityPatrick Henry Center, Pittsburgh Center (2 Service Points)Moon Township, PA

Washburn UniversityMabee Library, Topeka, KS


The InstitutionsEnrollment over 15,000

West Virginia University (3 Libraries)Health Sciences Library, Downtown Campus Library,Evansdale Library, Morgantown, WV

University of California, San DiegoScience & Engineering Library, La Jolla, CA

New York UniversityBusiness & Documents Center – Bobst Library, New York, NY

Georgia Institute of TechnologyGeorgia Tech Library, Atlanta, GA

University of NebraskaLove Library (Chat Service only), Lincoln, NE


Participant Data

14 Institutions 24 Service Points 179 Participants Full / PT / Faculty / Staff Varying experience levels


Study Components

IRB / Consent Forms Timeline (3 week and/or semester long) Pre-test / local calibration Blog Online Survey


Study Components - IRB

IRB (Institutional Review Board) approval Done at GCSU and Carnegie Mellon Done at Institutions if required Consent forms delivered electronically,

signed & returned with data at end of 3 week study period


Study Components - Timeline

3 Week and / or Full Semester Feb. 2 - Feb. 24, 2007 - 14 Institutions (all) Full Semester - 7 Institutions

February dates selected to give institutions time to test as well as minimize chances for spring break, holidays etc.


Study Components - Pre-test, Local Calibration

Sample Questions created to choose from Encouraged to include questions typical to

their home institution (ie collection specific) On-Site Coordinator distributed locally and

calibrated, creating an ‘example key’ for participants

Asked to record time for each during test phase


Study Components - Pre-test, Local Calibration

Pre-test allowed for study-wide calibration - test questions, responses, time and READ Scale category assignments were the same across institutions for the most part

Institutions used their own recording sheets


Study Components - Blog

Blog set up Only one question received during study Online survey responses suggest that the

READ Scale was easy to apply, could explain why blog was not utilized


Study Components - Online Survey

Online Survey sent at the conclusion of the three week study period to all participants

Response rate was high - 102 responses out of 179, or 56%


Study Components - Online Survey - Results

Question 1: Please rank your degree of difficulty using the READ Scale.

Responses

not difficult somewhat difficult

moderately difficult

difficult very difficult

Skipped Question

Number responded

52 (51.0%) 38 (37.3%) 10(9.8%)

2 (2.0%)

0 0 102



Question 2: Was the READ Scale easy to apply?

Responses

very easy to apply

easy to apply

moderately easy

somewhat easy

not easy Skipped Question

Number responded

16 (15.7%)

39 (38.2%)

38 (37.3%)

8 (7.8%)

1(1.00%)

0 102


Study Components - Online Survey - Results Question 3: Please rank the level of perceived "added

value" the READ Scale placed on statistics gathering for reference transactions.

Responses

extreme value added

high value added

moderate value added

minimal value added

no value added

Skipped Question

Number responded

7(6.9%)

46 (45.5%)

35 (34.7%)

9 (8.9%)

4 (4.0%)

1 (.99%)

101


Study Components - Online Survey - Results Question 4: Did you have difficulty in deciding between

ratings? If so check all that apply.

Scale

1-2 2-3 3-4 4-5 5-6 no difficulty

Resp

Total

Skip

Resp

12 (7.6%)

32 (20.4%)

46 (29.3%)

31 (19.7%)

15 (9.6%)

21 (13.4%)

157 99 3


Study Components - Online Survey - Results Question 5: How did you feel about evaluating your own

efforts?

Responses

Extremely Comfortable

Very Mod. Min. NotSkipped Question

Number responded

12 (11.9%)

50 (49.5%)

35 (34.7%)

4 (4.0%)

0 (0%)

1 (.99%) 101



Question 6: Would you recommend the READ Scale to another reference librarian?

Response Response Total

Yes 68 (67.3%)

No 12 (11.9%)

Yes, but with some modifications

21 (20.8%)

Skipped Question 1

Number responded 101



Question 7: Would you like to see this scale adopted for use in your library?


Yes 50 (50.5%)

No 18 (18.2%)

Yes, but with some modifications

31 (31.3%)

Skipped Question 3 (2.97%)

Number responded 99


Study Components - Online Survey - Results - Likes

Question 8 asked participants what they liked about the Scale.

The likes listed by the participants where coded into the six most common reoccurrences:

Effort / Value; (17)Approach to Evaluation; (13)Types / Levels; (9)Time; (5)Staffing Levels; (6)Reporting to Administration (5)


Likes

“It gave me a quick visible check of my recent efforts. This made my desk work more rewarding, since I sometimes feel like I do so many 1s and 2s- but I could see that I was actually doing a higher level of reference than I realized. It added value to the statistics - literally.”


Study Components - Online Survey - Results - Dislikes

Question 9: Participants were also asked to list their dislikes. These were coded into the six most common reoccurrences:

Difficult to Apply / Subjectivity; (19)Types / Levels; (16)Approach to Evaluating; (9)Knowledge of Staff; (6)Effort / Value; (4)


Dislikes

“At times it was difficult to rate effort.”



Question 10: Do you have any modifications you would like to suggest to improve the READ Scale?


No 67 (72.8%)

Yes (please describe) 25 (27.2%)

Total Respondents 92 (90.1%)



Study Components - Online Survey - Results - Modifications

Modifications were put into the following categories:

Delivery Method/READ Scale Appearance; (9)

Time Element; (5)

Skill Level Element; (4)

Clarity of Categories; (4)

Discussion Component; (2)

Comments / Observations. (2)


Modifications

“Perhaps, clarify who is answering the questions” (skill level).

“Collect length of time as another way to gauge level of difficulty”



Question 11: Were there any changes in your personal approach to reference service while you were using the READ Scale?

Responses Response Total

No 88 (89.8%)

Yes (please describe) 10 (10.2%)

Total Respondents 98 (96.0%)



Personal approaches

“More likely to think about the level of service being provided.”

“I gave more conscience thought to the processes or steps involved in order to rate each interaction.”


Survey Results - Overall

No difficulty using the Scale Easy to apply Ranked perception of added value to reference statistics

as ‘high’ Staff comfortable with rating their own efforts 68% would recommend as is, 20% with modifications 50% would adopt as is, additional 31% with

modifications Low percentage of changes in approach (10%)

reinforces ease of use / local adaptability


READ Scale - What works

Local approach to using the READ Scale Pre-testing / common questions Easy to use Adds value to data gathering Adds value to work / satisfaction Records previously unrecorded effort /

knowledge / skills (service point and off-desk)


READ Scale - preliminary data

Comparisons per service point (READ Scale) Comparisons off-desk (READ Scale) Approach type, service points Approach type, off-desk


Three Week Data, Service PointsService Point Comparisions, READ Scale Category Percentages

0% 20% 40% 60% 80% 100%

Service Point 1

Service Point 2

Service Point 3

Service Point 4

Service Point 5

Service Point 6

Service Point 7

Service Point 8

Service Point 9

Service Point 10

Service Point 11

Service Point 12

Service Point 13

Service Point 14

Service Point 15

Service Point 16

Service Point 17

Service Point 18

Service Point 19

Service Point 20

Service Point 21

Service Point 22

Service Point 23

Service Point 24

1

2

3

4

5

6


Three Week Data, Off-Desk

Off-desk READ Study Category Comparisons

0% 20% 40% 60% 80% 100%

1 Off-Desk

2 Off Desk

3 Off-Desk

4 Off-Desk

5 Off-Desk

6 Off-Desk

7 Off-Desk

8 Off-Desk

9 Off-Desk

10 Off-Desk

11 Off-Desk

12 Off-Desk

13 Off-Desk

14 Off-Desk

15 Off-Desk

16 Off-Desk

17 Off-Desk

1

2

3

4

5

6


Full Semester Participants Data, Service Points

Semester Category Percentages READ Scale, Service Points

0% 20% 40% 60% 80% 100%

Semester 1

Semester 2

Semester 3

Semester 4

Semester 5

Semester 6

Semester 7

Semester 8

Semester 9

Semester 10

Semester 11

Semester 12

Semester 13

Semester 14

Service Points

1

2

3

4

5

6


Full Semester Participants Data, Off-Desk

Off-Desk Semester Percentages

0% 20% 40% 60% 80% 100%

Semester Off-Desk1

Semester Off-Desk2

Semester Off-Desk3

Semester Off-Desk4

Semester Off-Desk5

Semester Off-Desk6

Semester Off-Desk7

Semester Off-Desk8

123456


Approach Types, Service Points, All Transactions

In Person81%

Phone 9%

Email8%

Chat2%


Approach Types, Off-Desk, All Transactions,

In Person46%

Phone18%

Email36%


READ ScalePractical Applications

Training / Continuing Education Renewed Personal & Professional Interest Outreach Reporting / Statistics



Training / Continuing Education

“I felt it was very useful because it challenged me to come up higher in those areas where I need improvement in certain concentrations like ____ which is not my specialty. I need to learn so much more.”



Training / Continuing Education

New staff. Develop a training regimen with outcomes, with a similar series of questions to be given at a later date to insure that the staff is developing the necessary skills / knowledge.

Continuous learning. Writing down any questions that elicit an assignment of a category of 4 or higher at the service point, then sharing with colleagues, there by sharing strategies and learning from others.



Renewed Personal & Professional Interest

“Using the READ Scale added to my sense of accomplishment!”



Renewed Personal & Professional Interest Self Assessment / Reference as Activity Reference

staff can rate their effort / knowledge / skills as appropriate. Gives recognition to primary function and can be compared to other libraries / librarians using the Scale. Acknowledges the two activities most important to reference staff in terms of job satisfaction (Gerlich): helping users and detective work.



Outreach

“It gives ME a tangible scale on which to rate my efforts, ultimately spurring me to strive for better service.”



Outreach Recording Liaison Activity Off-desk statistics are often

not recorded, or if they are, given the same hash mark as a directional question. Using the READ Scale in these cases would show case the subject specialization knowledge / needs of the campus. In cases where off-desk statistics are low or READ Scale assignments are in the low end range, outreach activity could be re-examined, surveys taken, etc in that particular area and services redesigned as needed.



Research / Statistics

“An assessment tool that does a better job of reflecting how reference librarians spend their time. It gives more value than tick marks on a page. It's a tool we can use with administrators to show what we really do.”



Research / Statistics Staffing Strategies. Who staffs desk, when. Develop Narrative Statistics. Records hidden

work. Time. Estimate or actual real time statistics for

effort working with patrons. Comparisons with like institutions who use the

Scale.


For further exploration Discussion must be part of implementation /

continued use - integrate with Scale Categories to include more clear examples that

emphasize effort / teachable moment Consider alternate format easier to read (bulleted or

rubric form) Time element more prominent and / or recorded for

sample periods More clear distinction between categories or consider

fewer categories Consider ways to identify levels of skill between staff

at service points


Next Steps

Consider modifications based on feedback Publish articles Host @ Creative Commons Invite other Libraries to use Continue data gathering Create communication tool for users /

continued development of Scale


Questions?

Thank you!

If you are interested in trying the READ Scale, please contact us at:

Lynn Berard, [email protected]

Bella Gerlich, [email protected]