evaluating and assessing library instruction examples & best practices
Post on 17-Dec-2015
222 Views
Preview:
TRANSCRIPT
Evaluating and Assessing Library
Instruction
Examples & Best Practices
Presenters
Martin Garnar – Regis University Rhonda Gonzales & Eleni Adrian –
CSU-Pueblo Lorrie Evans – Auraria Library Patrick McCarthy - CSU
Evaluating and Assessing Library Instruction
Regis University
Martin Garnar
ACRL Guidelines for Evaluation and Assessment
program evaluation plan criteria for evaluation learning outcomes and assessment methods coordination of assessment with teaching
faculty data gathering and analysis
Evaluating and Assessing Library Instruction
CSU-Pueblo
Eleni Adrian & Rhonda Gonzales
CSU-Pueblo Library Instruction Evaluation
Overview
Librarians at CSU-Pueblo administered a brief evaluation form after each instruction session.
Collected these evaluations from 1999-2004. Entered results into an Access Database. Data Clean-up Statistical Analysis Conclusions
CSU-Pueblo Library Instruction Evaluation
Evaluation Form
Background questions
Satisfaction questions
Ranking question
CSU-Pueblo Library Instruction Evaluation
Evaluation Forms
Collected Fall 1999 through Summer 2004.
181 classes
2441 evaluation forms
CSU-Pueblo Library Instruction Evaluation
Access Database
Entered several semesters’ of data
Revealed errors in our database design
New database structure
New input forms
CSU-Pueblo Library Instruction Evaluation
Access Database
CSU-Pueblo Library Instruction Evaluation
Access Database
CSU-Pueblo Library Instruction Evaluation
Data Fixes: New Database Design
Relationship between the classes and the evaluations tables.
No answers for background questions.
CSU-Pueblo Library Instruction Evaluation
Data Fixes: Form Changes
Two questions added, two questions removed.Re-entered all old evaluation form data to
match the final version.Double-checked that the form changes were
accurately re-entered.
CSU-Pueblo Library Instruction Evaluation
Data Fixes: Data Integrity
Duplicate Class records
Duplicate and blank Evaluations records
Typos in Status (year in school)
General data entry errors
CSU-Pueblo Library Instruction Evaluation
Results – Background Questions
20.1%sophomore
16.0%senior
20.6%junior 3.0%
graduate
39.5%freshman
0.7%faculty
Category
seniorsophomore
facultyfreshmangraduatejunior
Status of RespondentsYear in School
CSU-Pueblo Library Instruction Evaluation
Attended Before
Count
YesNoNA
1600
1400
1200
1000
800
600
400
200
0
973
1436
32
Attended Session Before
Results – Background Questions
Attended Previous Session
CSU-Pueblo Library Instruction Evaluation
Had a Specific Assignment
Count
YesNoNA
2000
1500
1000
500
0
1800
579
611
Student Had a Specific Assignment
Results – Background Questions
Specific Assignment
CSU-Pueblo Library Instruction Evaluation
Professor Present
Count
YesNoNA
2000
1500
1000
500
0
1948
349
144
Professor Participated in Session
Results – Background Questions
Professor Participated
CSU-Pueblo Library Instruction Evaluation
A
Count
654321*
1200
1000
800
600
400
200
0
1194
952
243
1714147
This presentation helped me learn how to start my research.
1=strongly disagree 6=strongly agree
Results – Satisfaction Questions
Start Research
CSU-Pueblo Library Instruction Evaluation
B
Count
654321*
1400
1200
1000
800
600
400
200
0
1374
832
193
26763
The presentation was clear.
1=strongly disagree 6=strongly agree
Results – Satisfaction Questions
Clear Presentation
CSU-Pueblo Library Instruction Evaluation
C
Count
654*312
1200
1000
800
600
400
200
0
1173
836
282
64501917
The hands-on experience will help me to complete my assignment.
1=strongly disagree 6=strongly agree
Results – Satisfaction Questions
Hands-on Experience
CSU-Pueblo Library Instruction Evaluation
D
Count
65*413
1800
1600
1400
1200
1000
800
600
400
200
0
1680
361338
5174
The librarian treated each student with respect.
1=strongly disagree 6=strongly agree
Results – Satisfaction Questions
Respect
CSU-Pueblo Library Instruction Evaluation
15.1%5
15.8%4
8.9%3
24.1%2
36.1%1
Category
5
1234
The most important thing that I learned was:
1=how to locate a journal article or book2=how to access the library's databases3=how to use Interlibrary Loan to get my information4=how to decide which databases I should use5=how to search the Internet more efficiently
Most Important Thing Learned
Results – Ranking Question
CSU-Pueblo Library Instruction Evaluation
Comparisons
Significance of Having an Assignment on Students' Feeling that Presentation Helped Them Start Their Research
0.00%
50.00%
100.00%
No Assignment Had Assignment All Respondents
No Assignment 0.52% 1.22% 1.39% 11.98% 43.75% 41.15%
Had Assignment 0.61% 0.39% 0.50% 9.41% 37.79% 51.31%
All Respondents 0.59% 0.59% 0.72% 10.03% 39.23% 48.84%
1 2 3 4 5 6
CSU-Pueblo Library Instruction Evaluation
Significance of Having an Assignment on Students' Feeling that Hand's-On Practice Was Helpful
0.00%
20.00%
40.00%
60.00%
No Assignment Had Assignment All Respondents
No Assignment 1.42% 1.06% 2.48% 15.93% 35.22% 43.89%
Had Assignment 0.57% 0.63% 2.00% 10.61% 34.91% 51.28%
All Respondents 0.78% 0.73% 2.11% 11.91% 34.99% 49.48%
Strongly Moderately Slightly Slightly Agree Moderately Strongly Agree
Comparisons
CSU-Pueblo Library Instruction Evaluation
Conclusions
Distribution of responses skewed towards high end.
Student evaluations of faculty teaching across campus are not normally skewed towards the positive to this extent.
These results show that a large majority of students felt that our instruction sessions were helpful!
CSU-Pueblo Library Instruction Evaluation
Conclusions - Meeting Our Goals?
Since most of our sessions are introductory in nature, one of our primary goals is to show students how to start their research using our library.
88.17% of students that took the survey indicated that they moderately or strongly agreed that the sessions helped them learn how to start their research.
36.12% of students who took the survey responded that learning how to locate a journal article or book was the most important thing they learned.
CSU-Pueblo Library Instruction Evaluation
Conclusions - Factors Affecting Results What factors may have affected student responses?
Chi-square tests indicate that the following factors proved to be statistically significant:
Having a specific assignment in conjunction with this presentation was a significant factor on the number of students that strongly agreed with A & C
Attending a prior session was a slightly significant factor in determining how many students strongly disagreed with D
Survey Design: Even number of response choices forced students to choose positive or
negative Unclear wording – “My Professor participated in this session.” Lack of N/A forced students to choose a response even if it did not apply
Survey Administration Administered at the end of each session, when students were in a hurry to
leave. Tendency of librarians may be to skip the survey if the session hasn’t gone
well.
CSU-Pueblo Library Instruction Evaluation
Indications for Best Practices
Role of Evaluation:combine learning assessment with program evaluation
Survey Design:concise, clearly worded, avoid mild statements, allow neutral and N/A responses, define hypotheses prior to creating the form
Evaluating and Assessing Library Instruction
Auraria Library
Lorrie Evans
Evaluation of instruction at Auraria Library
Survey student learning in single session library classes (random selection).
Survey students in full semester credit bearing classes (assignments, final project)
Survey the faculty toward the end of the semester – after papers and assignment are turned in.
Do the students understand the most important concepts?
No other chance for a follow up or review.
When students leave the room we will not see them again
Challenges in the single session library class
Look in the mirror – are we teaching what we think we are teaching.
Collaborate with campus faculty on a pro-active level.
Make library instruction a more integrated part of the academic culture.
Accreditation – assessment mandates driven externally.
Why?
The tool – short online survey
In class – three to five questions mapped to learning goals.
Customized for each class
Broadcast results, review answers & main concepts.
Students share concerns.
•Students see their answer along with classmate responses. •Librarian can respond to student input.•Review main points.
Sample results
English 1020 Student comments view
Psychology freshman seminar data
Graduate Political Science - single session post test.
Graduate English Literature - semester course
Jump past results
Comments
return
Freshman Seminar
sample items
Psych
return
Graduate English Literature survey:sample items
return
post test
Political Science
return
Right tool for the job question
Faculty survey -- end of the semester
Program effectiveness as reflected in quality of student projects and assignments.
Satisfaction with content, scheduling and teaching.
Faculty Evaluation
survey results
Zoomeranglink
Faculty evaluations
sample items
How can we improve?
Feedback loop
In class survey – instant feedback for both students and librarian. Librarian can go over confusing concepts.
Results discussed at departmental meetings – used as source of information to make adjustments in teaching (more depth with less quantity).
Results made available to instructors and students.
End of the year report submitted to the campus Outcomes Assessment Advisory Committee. The committee provides a review of the report, recommends improvements.
Evaluating and Assessing Library Instruction
CSU – Fort Collins
Patrick McCarthy
Evaluating Learning Outcomes at the Reference Desk - CSU
Why? Determine if learning takes place as a result of
interactions at the reference desk Identify those skills most suitable to reference
desk instruction Establish reasonable reference desk learning
outcome benchmarks Recommend behaviors and methods that lead to
student learning
Categories of Reference Assessment
Accuracy response rate User satisfaction Course/instructor opinion survey Learning outcomes
Methodology?
Quantitative approach Powerful, not practical
Qualitative approach Powerful, versatile
Procedures
1. Patron completes preliminary survey at completion of reference interaction
2. Researcher analyzes preliminary surveys and selects relevant patrons for follow-up assessment
3. Selected patrons are interviewed on specifics of reference transaction
4. Interview recordings are transcribed and coded for assessment
Analysis
Transfer assessment Accuracy assessment Patron must provide evidence Authentic assessment
Group Discussion – Guiding Questions
1. How have you determined the learning needs of your community?
2. What's your best method for evaluating library instruction?
3. What's your best method for assessing progress on learning outcomes?
4. Do you have examples of successful collaboration with classroom faculty for evaluation and/or assessment?
5. How have you used data from your instruction program evaluations and/or assessments?
Assessment of Instruction – Examples & Best PracticesPresenter Contact Information
Martin Garnar, Reference LibrarianDayton Memorial Library, Regis University3333 Regis Blvd., Mail Stop D-20Denver, CO 80221-1099303.964.5459303.964.5497 (fax)
mgarnar@regis.edu
Lorrie Evans,Library Instruction ServicesAuraria Library1100 Lawrence StreetDenver, CO 80204303.556.6776
Lorrie.Evans@cudenver.edu
Patrick McCarthyCoordinator of InstructionMorgan LibraryColorado State Univ.501 University AvenueFort Collins, CO 80523720.491.1881
pmccarthy@manta.colostate.edu
Rhonda Gonzales, Interim DeanEleni Adrian, Instruction CoordinatorUniversity LibraryColorado State University – Pueblo2200 Bonforte Blvd.Pueblo, CO 81001719.549.2361rhonda.gonzales@colostate-pueblo.edueleni.adrian@colostate-pueblo.edu
top related