Information Literacy Learning Outcomes and Student Success
Post on 02-Sep-2016
yStudent Successby Sue SamsonAvailable online 30 March 2010
Information literacy learning outcomes ofrandomly selected first-year and capstone
students were analyzed using an assessmentinstrument based on the ACRL competencystandards. Statistically significant differencesbetween student populations in the selectiveand relative use of information inform thelibrary instruction program and apply to
research and teaching libraries.
the official ratification of information literacy competencies by the
ages 202210Mansfield Library, The University of Montana,Missoula, MT, USA
202 The Journal of Academic Librarianship, Volume 36, Number 3, pSue Samson is Professor and Head, Information and Research Services,
ACRL. Interest in learning outcomes has further expanded as regionalacademic accreditation standards have begun to focus on learningoutcomes. By 2004, the Middle State Association for regional academicaccreditation refers explicitly to the ACRL standards of informationliteracy, and incorporates them into their own expectations for learningoutcomes, confirming the trend toward amore explicit endorsement ofinformation literacy in general, and the ACRL standards in particular.4
Information literacy-related behaviors have also been included on anexperimental basis in the 2006 National Survey of Student Engagement(NSSE), providing interesting results of correlations between informa-tion literacy scales and other NSSE measures.5
In addition to the NSSE experiment, other national and regionalassessment initiatives that survey large groups of students acrossInformation Literac Learning Outcomes and
INTRODUCTIONThe critical need for students to be knowledgeable about finding,retrieving, analyzing, and effectively using information has become acampus-wide issue fostered by librarians and underscored by regionalaccreditation standards. Accreditation standards not only defineinformation literacy but also include learning outcomes, emphasizethe importance of collaboration between librarians and faculty, andare based on the competency standards of the Association of Collegeand Research Libraries (ACRL).1 As information literacy has become animportant aspect of higher education, the need exists for authenticassessment models to identify learning outcomes.
The iterative process of information literacy requires evaluation ofinformation, critical thinking, revision, and integration and is not welladdressed by standardized testing.2 Among alternative assessmentmethodologies is the process of evaluating student work directly. Thismethodology focuses on student-constructed use of informationwithin the context of their curriculum and has the potential toexamine changes in learning outcomes across disciplines and through-out the college experience.
The objectives of this studywere to: 1) quantify learning outcomesbased on the five ACRL standards using student portfolios; 2) comparethe selective use of information resources between first-year andcapstone students; 3) compare the relative use of information re-sources between first-year and capstone students; and 4) identifysignificant patterns of learning outcomes to inform the library'sinformation literacy curriculum.
LITERATURE REVIEWThe term information literacy became part of the American LibraryAssociation (ALA) lexicon in 1989 as part of the Final Report of theALA's Presidential Committee on Information Literacy. Learning out-comes in relation to information literacy appear in the published libraryliterature as early as 19983 and more regularly shortly after 2000 with
academic institutions provide indicators of student learning outcomes.The Education Testing Service (ETS) reports that many collegestudents lack the information and communication technology (ICT)
test the efficacy of a curriculum designed to foster information literacyskills of graduate students in a chemistry bibliography course.20 Anassessment tool was given to students at the beginning and at the endliteracy skills necessary to navigate, evaluate and use informationbased on the administration of their iSkills assessment, an Internet-based assessment that gauges cognitive skills within a digitalenvironment.6 Another assessment instrument developed for largescale assessment and based on the ACRL Information LiteracyCompetency Standards for Higher Education is Project SAILS. Whilethe ETS ICT Literacy Assessment was based on the Standards, ProjectSAILS has mapped its skill sets to outcomes and objectives of theStandards.7 Though promising, analyses of SAILS have the limitation ofproviding only cohort-level information and the application of resultsare not yet appearing in the literature.8
Kuh and Gonyea9 report on the College Student ExperiencesQuestionnaire that provides data from 300,000 students from 1984 to2002. The results of this exploratory study indicate that libraryexperiences of undergraduates positively relate to select education-ally purposeful activities and that the library appears to be a positivelearning environment for all students, especially members ofhistorically underrepresented groups. An example of a system-wideassessment instrument is the South Dakota Information LiteracyExam, a home-grown, dual-measure instrument, developed for asystem of small and medium-sized universities and required to becompleted by all students to document and assess their informationliteracy skills.10 As they have validated and vetted the instrument, theauthors have also developed a generic version available to othersimilar systems. An example of an institution-based assessment is theInformation Seeking Skills Test, a computer-administered, compe-tence test, that measures the ability to find and evaluate informationand is required of all freshmen at James Madison University.11
Student learning outcomes are described by Gratch-Lindauer12 asone of three arenas of information literacy assessment. The other twoarenas are the learning environment and information literacyprogram components. Although these arenas are complementaryand not isolated, this review of learning outcomes literature identifiedmultiple methodologies for making a connection between studentacademic success and level of information literacy knowledge.Tancheva et al.13 emphasize the importance of using a combinationof attitudinal, outcomes-based, and gap-measure assessments toaddress the shortcomings of any one of these. Other authors report alack of success in their methodology. One study gauged theeffectiveness of their information literacy program by comparingaverage course grades in course sections that received informationliteracy instruction to grades in sections that did not and found noclear benefit to the instruction.14 McMillen and Dietering15 found nosingle assessment tool or method proved adequate to effectivelymeasure student learning happening both inside and outside of theirlibrary.
Working from the broad perspective of assessing informationliteracy in general education, Mackey and Jacobson16 found course-specific strategies enhance institutional assessment efforts by pro-viding a range of instruments to measure information literacy withinunique educational contexts. The use of electronic portfolios alongwith rubrics enabled librarians at Washington State UniversityVancouver to evaluate their information literacy program based onACRL best practices guidelines, authentic assessment techniques, andthe tenets of phenomenography, amethodology that is very adaptableto the assessment of information literacy.17
A recent example of pre- and post-testing with 60-min sessionsestablishes this methodology as helpful in identifying areas in whichinstruction could be improved and where positive results were beingachieved.18 Rockman19 reports on the critical issue of integratinginformation literacy into the learning outcomes of academic dis-ciplines. An example is documented in a 3-year study formulated toof the semester, and results indicated marked improvements in theaverage student scores from all 3 years.
Relevant to the focus of this research project, Knight21 reports in astudy using rubrics to assess information literacy that students'academic work is a useful gauge of their achievement of informationliteracy-based learning outcomes and that a rubric is a valuableassessment tool that provides a reliable and objective method foranalysis and comparison. Three additional studies document thevalidity of this assessment methodology. Findings of an assessment byportfolio validated this methodology in a study of an informationliteracy module.22 Scharf et al.23 further support the validity of using arubric and portfolio for program analysis in their careful and rigorousstudy of the effectiveness of information literacy instruction forundergraduates at a technological university. Finally, citation analysisof final year project reports provides further documentation of thevalue of this methodology in identifying information literacy learningoutcomes.24
INFORMATION LITERACY CURRICULUMWith an undergraduate student population of 11,799, a graduatestudent population of 2059, and 581 full-time faculty, UM is a mid-size coeducational, doctoral university. The information literacycurriculum at the Maureen and Mike Mansfield Library is based on arubric that identifies learning initiatives and outcomes for first-yearthrough graduate level coursework. The strategic integration ofinformation literacy into the curriculum begins with first-yearinitiatives that serve as the basis for information literacy instructionin the disciplines at the junior and senior levels. During fiscal year2009, 420 curriculum integrated classes were taught to 9,513students; and, in addition, 3 credit classes were offered.
First-year curriculum is based on a model of teaching the teachers,and integration decisions have been made on the basis of severalfactors: courses that are a part of the standard university curriculum;courses with a research component, usually smaller enrollmentclasses; and courses with a large enrollment through participationin the Freshman Interest Group program, which offers the opportu-nity to provide cross-disciplinary information literacy instruction.Specific standards and teaching strategies have been identified fortargeted courses to establish quality learning opportunities for first-year students. At every opportunity, librarians seek to serve asresearch consultants and pedagogical guides and to facilitate thesuccessful delivery of information literacy content by teaching facultyin the disciplines. Targeted first-year courses include: EnglishComposition, Introduction to College Writing, Introduction to PublicSpeaking, Critical Writing, Freshman Interest Groups, FreshmanSeminar, and Honors College Seminar.
Based on the delivery of lower-division information literacyinstruction, liaison librarians work collaboratively with faculty in alldepartments, schools, and colleges to tailor advanced informationliteracy instruction to upper-division students in their major studies.Liaison librarians target research andwriting courses in all majors andfacilitate the successful delivery of information literacy contentthrough collaboration with faculty that includes: integration ofinformation literacy standards into the curriculum and learningoutcomes of individual academic units; provision of consultativeservices to teaching faculty to develop curriculum-integrated libraryresearch assignments; promotion of instruction in the use of libraryresources to students and faculty, integrating the tiered LibraryInformation Literacy Curriculum; creation of web-based subjectresources for faculty, students, and staff; and provision of regular,advertised office hours, scheduled reference assistance, and smallgroup instruction sessions as part of the Learning Commons.
May 2010 203
METHODOLOGYThe potential survey population consisted of students 18 years old
student population provides another viewpoint of the data (Table 2).Small p-values provide convincing evidence that a significantdifference exists between the student populations in the total numberof citations and total number of primary sources. Nearing significanceis the total use of books and images. First-years students usedsignificantly fewer total citations and primary sources than didcapstone students, while capstones students used books and imagesmore frequently than first-year students. Use of total number ofunique formats, films, interviews, newspapers, peer-reviewed articles,reference tools and web sources were not significantly different.
Standard Two: The Information Literate StudentAccesses Needed Information Effectively and Efficiently
Access to information was quantified by the following questions(Table 3):
Is there evidence that a database has been used?
Is there evidence that Google Scholar has been used?
Is there evidence that interlibrary loan may have been used?
Is there evidence that the student has collected his/her ownresearch for the project:
Describe the nature and extent of that research.
Table 1[ACRL Standard One: Know] Comparison of theand older enrolled in English Composition during fall semester 2007and in capstone courses across the curriculum during spring semester2008. Four randomly selected sections of English Composition andfour randomly selected capstone classes were selected for analysis. Allsections of English Composition were selected from the ENEX 101class number sequence. Capstone classes were identified from theuniversity catalog and were required to use the term capstone as partof either the class title or class description. The randomly selectedcapstone courses included Capstone in Rural and EnvironmentalChange (SOC 460), Senior Thesis (ECON 489), Legal Research andWriting (LEG 287T), and Photojournalism Senior Seminar (JOUR 481).
Within each class section, four studentswere randomly selected fora representative sample of 16 students in the first-year compositionclasses and 16 students in the capstone classes to serve as the studygroup. Student identification numbers were captured to establishquantitative points of comparison between information literacy andstudent demographics and to facilitate a longitudinal assessment upongraduation of current first-year students during phase 2 of the study.The Institutional Review Board reviewed and approved the project,and all participants provided informed, signed consent forms.
For each student [surveyed], class portfolios and/orfinal research projects were collected [and] classgrades were recorded to measure student success
within the curriculum.
For each student, class portfolios and/or final research projectswere collected for analysis using an assessment instrument. Based onInformation Literary Competency Standards for Higher Education,25 theinstrument assesses the performance indicators of each of fivestandards with quantifiable measures. In addition, class grades wererecorded tomeasure student successwithin the curriculumof the c...