Why Bother? The Road To A Quality Information Literacy

Assessment Study

Lyn LaVigne, M.A., M.L.I.S.Malka Schyndel, M.S.L.S.Darlene Ann Parrish, Ph.D.

Florida Atlantic University Libraries 2010

Special thanks to Tony Scott Parrish, M.S. in M.I.S.,

Systems Administrator at FAU, for his technical assistance.

Introduction Reference/instruction librarians from Florida Atlantic University (FAU) wanted to find out if their library sessions have a lasting impact on student research. For the first time, an effort was made to develop a model that could be used as a method of continuous assessment and improvement.

In partial preparation for a Southern Association of Colleges and Schools (SACS) re-accreditation visit, a committee of reference/instruction librarians developed a 15-item multiple choice questionnaire to measure what specific library research skills first semester freshman students enrolled in a Strategies for Learning Success (SLS 1503) class have prior to library instruction, and if there is any improvement after library instruction.

Introduction (con.) A pretest-post-test study design was used, and the pretest was administered prior to library instruction during a 50-minute on site scheduled library instructional session. The post-test, which was identical to the pretest, was administered at the end of the semester in the students’ regularly scheduled class location. Pretests and post-tests were administered to more than 300 students in a total of 22 SLS classes.

Why is library assessment necessary?

• Accreditation purposes.• Fits into other parts of organization:

strategic plan, mission/vision, annual reports.

• Result of financial constraints and increased accountability.

ASSESSMENT PROCESS FLOW CHART

Start with a strategic plan containing goals and objectives.

Develop self-studies to measure goals and objectives.

Analyze results, implement changes, and revise strategic plan.

Problem Areas in Developing Information Literacy Studies

• Identification of what needs to be assessed (e.g. learning outcomes).

• Determination of population and sample size.

• Time-consuming nature of information literacy studies.

Problems in Developing Information Literacy Studies

• Selection, creation and implementation of appropriate tools/methodologies.–Which assessment models are most

appropriate?–Are there models that already exist?–Do you need to create an in-house

one?• Interpretation and application of results.

“Best Practices”• The information literacy assessment study

should be linked to the library’s strategic plan with goals and objectives.

• It is important to create a time line for any information literacy study.

• The purpose of the study should be clearly defined and agreed upon by all investigators at the beginning.

“Best Practices”• The principal investigator needs to keep

the study on schedule.• If a research methodology is created for

a study, it will increase study time, but can be used for continuous assessment and improvement.

• Coordinators of library assessment need to have some formal training in applied or basic research.

Phases of the SLS Information Literacy Study

Study Design Phase• December 2007 – Committee of reference/

instruction librarians began to meet.• Spring 2008 – Committee began to create

survey instrument for SLS study.• September 2008 – Received Institutional

Review Board (IRB) approval for SLS study.

• Fall 2008 - Pretested survey instrument using students working for the Library.

Data Gathering Phase

• August-October 2009 – Administered pretests to 21 SLS classes in Library, before library instruction.

• November-December 2009 – Administered post-tests to 22 SLS classes during their regular class time.

• March 2010 – Completed the scoring of individual student pretests and post-tests.

• March 2010 – Completed the data input in an Excel spreadsheet.

Test Scoring and Data Input Phase

Data Analysis Phase• March 2010 – Compared the mean for

correct responses for pretests with post-tests to see if there was an improvement.

• April 2010 – Will create a report with recommendations for improvement.

Results• Mean score was 9.46 for pretests and

10.10 for post-tests out of 15 possible correct responses, which is about a 6% improvement.

• Maximum number of correct responses for pretests and post-tests was 14, so there was no change.

• Minimum number of correct responses for pretests was 2 and increased to 3 for post-tests.

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q150

50

100

150

200

250

300

350

400

SURVEY QUESTION TOTALS FOR PRE-TEST AND POST-TESTNu

mbe

r of

cor

rect

ans

wer

s

What have we learned?

• Individual course instructors could have positively impacted survey results:– Some instructors provided library

instruction prior to the library session.– Some instructors required knowledge of

library skills in order to complete assignments.

What have we learned?• Results showed that the SLS students

who were surveyed did better on the pretests than expected.

• Results of study may indicate that “one time” library sessions, if not tied to an assignment, are less effective.

What have we learned?• Results lead to the assumption that

instructors/professors whose students have a “need to know” in order to complete assignments benefit more from library instruction.

• Students are more motivated to learn library skills when a library instructional session is tied to a meaningful assignment.

Where do we go from here?• This study will be used as a baseline for

future assessment studies in information literacy.

• Results of this study support the Library’s current direction of embedding librarians in course instruction.

• Assist course instructors/professors in their understanding of the importance of the relationship between information literacy and its impact on student learning.

Where do we go from here?• Compare results of the SLS study with

a current ENC 1102 (Freshmen English) study that is using the same design and survey instrument.

• The comparison of the SLS and ENC studies may help to determine if project based library instruction is measurably more effective.

Where do we go from here?• Recommendations will be made to

continue supporting the SLS classes with library instruction.

• The emphasis will be on embedding librarians in course instruction, which would include the planning of course assignments.

For More Information