measuring student information literacy learning outcomes

GABRIELA SONNTAGCALIFORNIA STATE UNIVERSITY

SAN MARCOS

LAC 2008

Measuring student information literacy learning outcomes

“the unanimous conclusion from the testing done and from personal observation is that most students are seriously lacking in knowledge and ability to use books and libraries effectively.”

Felix Snider (1974)

Kellogg library at the California State University in San Marcos

CSUSM facts:Undergraduates 8,577Graduates 582Full-time Faculty 187Bachelor’s Degrees 27Master’s Degrees 10

Top degrees:business , biology, communication, psychology, kinesiology, human development, pre-nursing, sociology.

CSU system: 23 campuses, 450,000+ students , 47,000 faculty and staff.

Three assessments

Overall Results

GEL Pre/Post Scores

GEO Pre/Post Scores

Three assessments

“…discussing assessment methods collaboratively is a very productive exercise in planning a systematic, comprehensive information literacy program. This assessment program …..should make explicit to the institution’s constituencies how information literacy contributes to producing educated students and citizens.”

(Information Literacy Competency Standards, p. 6)

General Education

Cycle 1: Written Communication and Information Literacy

Rounds: each semester select an Area to focus on: several classes in each Area, several sections of each class: Round 1: (GE Writing) and (Critical Thinking) Round 2: (Quant Reasoning) and (US History) Round 3: (Social sciences)

The Assessments

Cycle 1 Rounds:

#Faculty participants

#Different courses represented*

#Pieces of student writing assessed

Round 1 20 4 737

Round 2 9 7 442

Round 3 19 8 617

Total 48

19*multiple sections of the courses were assessed

1796

Information Literacy

Finding appropriate sources: Students can locate appropriate references for their papers and assignments.

Using sources: Students interpret and use the information found in their paper and assignments.

Results

Student Learning Outcome

Meeting minimum

Rated superior

Thesis 86.9% 20.0%

Organization 86.0% 21.1%

Mechanics 86.2% 20.9%

Finding sources 86.5% 25.5%

Using sources 82.9% 20.1%

Comparison possible?

iSkills Using Information Data

Three assessments

Annual Assessments and Program Reviews

Required self-study includes assessment of Programmatic Student Learning Outcomes.

Includes annual assessment findings – benchmarking, changes, evidence of impact.

Plans for next cycle of assessment studies.

IL Assessment

Programmatic Student Learning Outcomes are Standards for Information Literacy (ACRL).

Annual assessments are embedded.

Measured student knowledge of characteristics of scholarly information sources (Evaluate).

Results

#Different courses represented*

# Student participants

Fall 7 471

Spring 17 776

Total 24 1247

*multiple sections of the courses were assessed

Scores by year

Deeper study

Degree #core courses

#cores with IL

#courses with IL

Biology 7 3 9

Comm. 6 4 6

History 1 1 8

H Develop. 5 4 0

Kinesiology 16 1 0

Lit/Writing 8 2 9

Pol. Science 6 4 9

Psychology 6 1 10

Sociology 6 3 10

Mean Scores by Course

Mean Scores by Major

Class score cross tabulation

Failing Adequate Excellent Total

FreshmenCountPercent w/n class

31750.50%

19831.50%

31118.00%

628100%

SophomoreCountPercent w/n class

5959.60%

2323.20%

1717.20%

99100%

JuniorCountPercent w/n class

13368.20%

3920.00%

2311.80%

195100%

SeniorCountPercent w/n class

11866.70%

2614.70%

3318.60%

177100%

TotalCountPercent w/n class

62757.10%

28626.00%

18616.90%

1099100%

Comparison Possible: Evaluate?

iSkills scores

GE Assessment

Annual Assessment

Pre-test score =59%

Post-test score = 64%

n/a Freshmen =50%

Economics

Learning Outcomes measured: Formulate meaningful economic questions (4.57) Retrieve information (2.43) Apply relevant concepts (4.14) Effectively communicate (3.86)

Conclusion: “more direct incorporation of library resources”

History

Learning Outcome measured: “incorporate new digital and multimedia formats into the practice and presentation of history” specifically “questions about what issues are raised in using the Internet for research…”

Conclusion: “we should address this issue in more detail than we do… .discussing not just how one can judge reliability of a source but also how to sift through even the most reliable sources [for evidence]….”

Political Science

Learning Outcomes measured: “Demonstrate working knowledge of research methods”

Pre-test score of 9.8 -weakness in formulating research questions, sampling, drawing conclusions.

Post-test score of 17.7 -general improvement but especially in formulating research questions, hypotheses, and identifying appropriate research methodology.

Psychology

Learning outcomes measured: Information Literacy

Faculty survey on student IL abilities : 57% reported students find it difficult to

locate sources. 45.3% students need help with evaluating

sources 26.4% students lack synthesis skills (using

information)

General Education Assessment

Results: Superior Rating

Student Learning Outcome

Psychology

All GE

Thesis 31.5% 20.0%

Organization 35.6% 21.1%

Mechanics 32.4% 20.9%

Finding sources 39.9% 25.5%

Using sources 30.5% 20.1%

Sociology

Learning Outcome measured: Write a literature review and research report.

Assessed research papers in capstone seminar. Interviewed faculty teaching course.

Locate: 81.5%Understand: 39.5%Summarize/Synthesize (USING): 36.8%Mechanics: 36.8%

Conclusions

Need for multiple measures.

Need for collaboration with academic departments.

Strive for improvement.

Future Goals

Year 2 measure Define.Engage the university community in dialog on teaching, learning, and information literacy.

Look to departmental assessments and program reviews for inclusion of IL.