The Journal of Academic Librarianship 39 (2013) 414–422

Contents lists available at ScienceDirect

The Journal of Academic Librarianship

Uncovering Impact: The Influence of Archives on Student Learning

Morgan Daniels a,⁎, Elizabeth Yakel b

a School of Information, University of Michigan, 3348B North Quad, 105S. State St., Ann Arbor, MI 48109-1285, USAb School of Information, University of Michigan, 4419 North Quad, 105S. State St., Ann Arbor, MI 48109-1285, USA

⁎ Corresponding author.E-mail addresses: mgdaniel@umich.edu (M. Daniels

0099-1333/$ – see front matter © 2013 Elsevier Inc. Allhttp://dx.doi.org/10.1016/j.acalib.2013.03.017

a b s t r a c t

a r t i c l e i n f o

Article history:Received 14 February 2013Accepted 26 March 2013Available online 13 June 2013

Keywords:ArchivesArchival instructionAssessmentStudentsImpact

University archivists and special collections librarians offer orientations and assistance to students who usetheir resources for course projects, but assessment of the impact of these interactions on students has beenlargely overlooked. We address this gap by presenting the results of a survey of 452 students at two univer-sities who had a formal orientation to the archives as part of their coursework. We assess the impact of theirexposure to archives using several measures, including: confidence in finding and conducting research withprimary source materials; perception of archival research as relevant to the students' goals; student evalua-tion of their experience in the archives; and willingness to return to use the archives in the future. The resultsof this study provide university archivists and special collections librarianswith anunderstanding of the differentlearning impacts that are possible through exposure to the archives andwill be useful in identifying learning out-comes for undergraduate and graduate students using archival and manuscript materials in the future.

© 2013 Elsevier Inc. All rights reserved.

University archivists have long seen the value of incorporating archi-val materials in education (Osborne, 1986). Currently, however, twotrends are converging which highlight the potential for primary sourcesin the classroom. First, the Boyer Commission Report on ReinventingUndergraduate Education (1998), sponsored by the Carnegie Foundationfor the Advancement of Teaching, advocated research experiencesthroughout the college curriculum. The Boyer Commission noted thatworkingwith primarymaterialswas anaturalway for humanitiesmajorsto participate in learning as inquiry as educators seek to teach studentshow to think historically and critically about sources. Rather than simplyasking students to remember and understand historical events, instruc-tors increasingly want them to understand how histories are createdand how records are used as evidence to make arguments. In the fifteenyears since the Boyer Report, the vision its authors presentedhas not beenrealized although there has been substantial activity to integrate archivesand special collections into undergraduate education. The second trend ismore recent and focuses on the value proposition of libraries, which hasnow been extended to special collections and archives. Long a placewhere resources were expended with little analysis of return, academiclibrary administrators and special collections librarians are now examin-ing ways to demonstrate value to the larger university beyond unique-ness and prestige. Lisa Carter (2012) has noted this transition in approach

ARL's struggle to move from celebrating its member libraries' trea-sures to proving their relevance and value is indicative of the chal-lenge that institutions of many sizes and types that hold specialcollections and archives are presently facing. Documented evidence

), yakel@umich.edu (E. Yakel).

rights reserved.

of impact has become particularly important as higher educationmodels transition to global, participatory, online environments andthe continuing economic crisis disrupts traditional bases of financialsupport for academic libraries (p. 89–90).

In this quest, university archivists and special collections librarianshave begun to focus on impact on student learning as a key part of thevalue proposition for special collections.

Archivists have responded by supporting instruction through a vari-ety of activities: from giving presentations to individual class sessions tofacilitating term-long research involving primary sources (Cotton &Sharron, 2011). Yet, while we have increasingly diverse examples ofthe use of archival records in instruction, we have seen only minimalevaluation. As primary sources enter the classroom, we must ask whatimpact interaction with these materials has on students. This articleaddresses this question of impact, presenting the results of a survey com-pleted by 452 students at two universities in the fall semester of 2007.These students participated in archival orientations, which were re-quired as part of their university course work, and in many cases wenton to conduct their own research using primary sources that semester.

This article presents an analysis of this survey data set, in an attemptto answer the research question:what impact does the archives have onstudents? We operationalize impact by addressing the following ques-tions, which each deal with different aspects of student experiences inthe archives through the concepts of value, confidence, and willingnessto return.

• What kinds of experiences in the archives do students value?• Does use of archives make a difference in students' confidence asresearchers?

415M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

• What factors are associated with students' willingness to return tothe archives?

• What factors figure into students' perceptions that archival researchis valuable to their goals?

By taking these questions into consideration, we explore the factorsthat contribute to meaningful impact in students' experiences in thearchives.

LITERATURE REVIEW

In this section, we explore the impact of students' exposure toarchives and special collections from two perspectives: the educa-tional perspective, which has studied the thought processes studentsand experts use to evaluate primary sources and examined learningoutcomes, and the archival studies perspective, from which practi-tioners have grappled with their role vis-à-vis student educationand have been joined by researchers in offering several useful frame-works for understanding the learning that takes place in archives.

LEARNING IMPACT AND USE

In 1987, Bruce Dearstyne called for archivists “to find better meansof measuring and interpreting the significance of research use” (p. 80).Twenty-five years later, archivists are still searching for reliablemetrics.Recently there has been considerable effort to measure the impact ofuse. In this section, we review progress in this area.

Sarah Horton and Jacqueline Spence (2006) state that the impactof archives can best be viewed as a continuum, “which extendsfrom the record creator, via the archivist, to the primary user of the ser-vice, the secondary user of research products, and the ‘non-userʼ at theother extreme” (p. 4). At each point in the process, value is added throughthe description of materials, their use to create new research, and the useof that research by society. This model explains how impact can filterfrom the individual, who will often use archives on his or her own, to agroup and community, through the products of an individual's research.

Dorothy A. Williams, Caroline Wavell, Graeme Baxter, AlanMacLennan, and Debbie Jobson (2005) categorize the impact of mu-seums, libraries, and archives into three groups: social, learning, andeconomic. They explain that,

social impact encompasses inclusion or overcoming exclusionof individuals or groups in terms of poverty, education, race ordisability and may also include issues of health, community safety,employment and education. Learning impact is interpreted in a broadsense to encompass an individual, organisation or community'sformal, informal and lifelong progression towards, and change in,knowledge and skills through a variety of real and virtual chan-nels. Learning can be surface or deep, immediate or long term,the acquisition of skills or an interaction with established knowl-edge. Economic impact encompasses economic implications forindividuals, small or large groups or for communities in terms ofurban or rural regeneration, renewal or sustainability (p. 535).

For students using college and university archives, learning impactis likely to be the most relevant measure, although social and economicimpact cannot be entirely discounted as students complete their educa-tion andmove into other spheres. Furthermore, a student's experiencesin the archivesmay very well have broader social and economic impact,depending on the context of their research.

While often taking the form of a user survey, impact has beenmeasured using a number of different methodologies. The LearningImpact Research Project in the United Kingdom identified five genericlearning outcomes in museums, libraries, and archives: “increase inknowledge and understanding; increase in skills; change in attitudesor values; enjoyment, inspiration, creativity; action, behaviour, pro-gression” (Hooper-Greenhill, 2004, p. 163). These outcomes are

based on the understanding that “‘learning’ is not usually used torefer to knowledge or scholarship; ‘learning’ is used to refer to learn-ing processes” (Hooper-Greenhill, 2004, p. 156). This view of learningimpact corresponds with those developed in the educational psychol-ogy and archives literature dealing with the role of primary sources indeveloping historical and critical thinking skills.

EDUCATIONAL PSYCHOLOGY RESEARCH AND LEARNING IMPACT

Within educational psychology research, studies of high schooland college students' experiences with primary sources indicate thekinds of thinking required to effectively use these sources and arguethat exposure to primary sources may increase these thinking skills.Sam Wineburg (1991) examined the cognitive processes used byeight historians and eight high school students when comparing his-torical documents to paintings depicting the Battle of Lexingtonduring the American Revolutionary War. Wineburg identified threeheuristics the historians used to evaluate primary source documentsduring this task: corroboration, or comparing the content of multipledocuments; sourcing, examining the source of a document beforeexploring its content; and contextualization, situating a documentwithin the time and place of its creation. The students based theirdecisions on less nuanced, binary assessments of the bias of each doc-ument and found textbook accounts to be the most trustworthy doc-uments they were given. Wineburg (2001) speculates about thesedifferences between students and historians using the concept of an epis-temology of text: an individual's understandings about historical inquiry.He writes, “Before students can see subtexts, they must first believe theyexist. In the absence of such beliefs, students simply overlooked or did notknow how to seek out features designed to shape their perceptions ormake them view events in a particular way” (Wineburg, 2001, p. 76).For Wineburg, this kind of critical analysis requires an epistemologicalshift in students' understanding of documents.

In a similar study, Jean-François Rouet, Anne M. Britt, Robert A.Mason, and Charles A. Perfetti (1996) examined the ability of under-graduate students to reason about historical events using primarysources. Rouet et al. concluded that the use of primary sources in col-lege curricula can help students understand how evidence is used tocreate historical accounts and may lead students to engage in moreadvanced reasoning with documents in general. Robert Bain's(2005) research has resulted in a carefully designed group-learningmodule which models historians' thought processes of sourcing, con-textualizing, and corroborating information to lead the students in col-lective interpretation of primary sources. Each of these learning sciencesresearchers has focused learning outcomes away from content, per se,and concentrated on critical thinking and assessment of documents; skillsthat can be transferred to other areas of life. Also striking about thisresearch is the absence of archivists. The one educational researcher,Sharon Cook (1997), who did include archivists as part of a collaborativeteam of archivists and educators emphasized the assistance archivists canoffer but also limited their role as teachers themselves:

…the archivist can perform a major service to teachers andstudents by locating, collating, and publicizing relevant sourceswhile not being called upon to attach pedagogical strategies tothem. Providing these strategies is the role for senior teachers orother educational professionals, ideally working (as the exampleconcluding this article demonstrates) in partnership with archi-vists and archival institutions (p. 108).

ARCHIVISTS' ENGAGEMENTWITH PRIMARY SOURCES IN THE CLASSROOM:MEASURING STUDENT LEARNING OUTCOMES

Hugh Taylor's 1971 address at the Society of American Archivistsannual meeting was one of the earliest calls by an archivist to open

416 M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

up the archives to students. In his address, Taylor provided a narrativeof recent changes in history education and illustrated successfulprojects to engage students with primary source materials. He em-phasized those programs that allowed students to develop theirown interpretations of documents (whether originals or facsimiles),rather than simply reading reproduced excerpts along with ahistorian's interpretation.

Historians and teachers have always used records as a means to anend. Perhaps we, rather than they, are the ones who can introducethe student to a genuine experience by simply placing an unsortedgroup of papers or series with unspecified contents in front of himand saying, in effect, ‘enter into a dialogue with these records, thistiny fragment of thousands of tons that have been written, and askyour own questions and draw your own conclusions; expose yourpersonality to them and see what happens; there is no right orwrong answer.’ I believe the result would be a genuine historicalexperience because it is quite unstructured and far removed fromtextbooks and source books which have their place in anothercontext.

(Taylor, 1972, p. 329)

Taylor called for archivists to engage with teachers and students todiscover how students can best be “let loose among the archives,”providing them with experiential learning and opportunities for per-sonal discovery (p. 330).

More and more archivists have heeded Taylor's call and over theyears have moved more aggressively into the classroom as both pur-veyors of documents and more recently as educators themselves. KenOsborne (1986) offered suggestions to archivists to connect withclassroom teaching, including teacher education, exhibits, directedstudent research in the archives, and archival materials specially pre-pared for students. Marcus Robyns (2001) argued that archivists canhave a broad impact on students' education.

Certainly the time has come for proactive archivists involved ineducational outreach to move beyond showing students how tofind and access information in archives and toward greater in-struction in critical interpretation and analysis of that information.

(Robyns, 2001, p. 365)

Robyns' approach built off the growing body of research in psy-chology and education, such as that previously discussed. In contrast,a majority of the writing on the integration of primary source mate-rials into the classroom written from an archival perspective, whilecognizant of trends in educational psychology and learning styles,has focused on logistical and pedagogical strategies (e.g., Bahde,2011; Carini, 2009; Wosh, Bunde, Murphy, & Blacker, 2007).

When assessment does occur, course evaluation is the mostpopular form of assessment. Peter J. Wosh, Janet Bunde, KarenMurphy, and Chelsea Blacker (2007) and Anne Bahde (2011) eachused course evaluations to assess archival involvement. In their bookon using primary sources in the classroom, Justine Cotton and DavidSharron (2011) brief discussion of instructional assessment focuses oncourse evaluation rather than student learning assessment. BarbaraRockenbach (2011) noted the importance of learning assessment anddiscussed plans for implementing this in her archival education offeringsin the future. In the recent edited volume on the use of special collectionsand archives in undergraduate education, Past or Portal, a number of theauthors mention course evaluation but only four really engage in a dis-cussion of student learning outcome assessment in any detail (e.g.,Grob, 2012; Bean & Anderson, 2012; Sutton, 2012, and Andrews,Cassman, & Johnson Melvin, 2012). Deborah C. Andrews et al. (2012)make this distinction nicely when they cite their learning goal as onethat is an “engaging approach to student learning that emphasizes theprocess of workingwith special collections rather than simply extractinginformational content from them” (p. 290).

Taken together, these studies demonstrate the types of thinkingand learning that can take place in the archives, indicating somepotential areas for impact measurement. Actual studies of impactmeasurement are scarce and varied. Wendy M. Duff and Cherry(2008) used a pre-test/post-test study design to examine the impact ofarchival orientation on undergraduate students utilizing their confidencein their ability to find and use archival sources as the major indicator ofimpact. Students' mean confidence levels rose from 4.1 to 6.0 on a10-point scale. Duff and Cherry's research design provides an excellentproxy for impact, since it isolates changes in student responses beforeand after orientation sessions. While the survey reported in this articleused a post-test only design, it has the advantages of a much larger re-sponse rate and a sample drawn from two universities.

Magia G. Krause (2010) conducted a field experiment using a ru-bric in a pre–post test design to assess student learning outcomesafter using the archives. Using two groups of students, one who re-ceived archival instruction and one who did not, she found that stu-dents who participated in the archival instruction had greaterimprovement on a document analysis exercise, showed more knowl-edge about the characteristics of primary sources and demonstratedmore facility in analyzing them. Krause's rubric used 4 types of archi-val literacy skills: 1) observation, 2) interpretation/historical context,3) evaluation/critical thinking, and 4) research skills. It moves us clos-er to identifying the set of leaning outcomes on which archival in-struction can have an impact.

The literature reviewed here has suggested several avenues formeasuring the impact of the archives on student users. The educa-tional psychology literature emphasizes the critical thinking skillsthat can be gained through primary source research, while the archi-val studies research, particularly Duff and Cherry's recent work,suggests using confidence as a proxy for learning. Krause helps us tothink about learning outcomes and skills that can transfer to otheractivities outside the archives. This literature provides the frameworkfor our approach to impact. To understand how exposure to thearchives has impacted students' valuation of their experiences, wemeasured students' perceptions of value of the archives to their edu-cational goals and which aspects of their experiences were valuableto them. To understand the learning impact of archives, we explorestudents' assessment of their confidence in using archives. Finally,to understand the impact of their experiences on their perceptionsof the archives, we will examine the students' willingness to returnin the future.

METHODS

This research utilized a survey methodology developed as part ofthe Archival Metrics Project (http://www.archivalmetrics.org/).Questionnaires were distributed at two universities to students inclasses that had used the archives as a part of course work duringthe fall semester of 2007. The three-page survey instrument consistsof fifteen questions divided into two sections (see Appendix Afor the questionnaire). The first section asks for feedback about theorientation session while the second section asks for informationabout students' use of the archives since orientation, including theresources they used, what they feel they learned from the experience,and their confidence in conducting archival research. The finaltwo questions ask for demographic information about the student'sacademic level and area of study. The instrument had been testedfive times at two university archives prior to this implementation,which led to a number of iterative improvements.

In the process of planning with course instructors for class orien-tations to the archives during the semester, the archivists at bothuniversities alerted instructors to the upcoming survey, helping toensure participation. Toward the end of the semester, the archivistsreminded them about the survey just before paper copies of the ques-tionnaire were distributed to course instructors in their campus

417M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

mailboxes. Instructors distributed them to their students in class, dur-ing one of the last sessions of the semester, and asked students to fillthem out and return them during that class session. Instructors sentthe completed surveys back to the archivists, who forwarded themto the research team for data entry. Responses to the surveys wereanalyzed by the authors using the data analysis software SPSS.

FINDINGS

We identified several potential approaches to the measurementof impact. These are students' valuation of their experiences in thearchives, overall confidence in their primary source research abilities,willingness to return, and perceiving the project as valuable to largereducational goals. In this section, we will first provide a general over-view of the respondents and then examine the survey data in terms ofthese measures of impact.

CHARACTERISTICS OF RESPONDENTS

While comparable numbers of survey respondents came fromstudents engaged in each university archives, the course configura-tions were quite different in terms of topical areas and class sizes. InTable 1, we provide an overview. History departments were widelyrepresented in the classes that used the archives, but students werealso enrolled in sociology, musicology, and African-American studiesclasses. Class sizes ranged from 5 to 176 students, with large classesutilizing both archives. Response rates ranged from 54.8% to 100.0%for an overall response rate of 85.8%.

The respondents were primarily undergraduates (214 or 99% atArchives A and 220, 96% at Archives B). The greatest difference inrespondents was between freshmen at Archives A and seniors atArchives B. Because the large classes at each university were madeup primarily of freshman in one case and seniors in the other, theyskew the number of students in those groups that took the survey.For Archives A, 57.1% (n = 124) of the respondents were freshmen;at Archives B, 57.0% (n = 131) were seniors.

We asked students to identify their field of study as a free textresponse and then coded them. The most common categories atArchives A were health and physical sciences (n = 62), education(n = 23), and history (n = 23). Sociology (n = 113), history(n = 35), and art and design (n = 20) were the most frequentlycited majors at Archives B. The large number of sociology studentsat Archives B was consistent with the large senior-level sociology

Table 1Courses and questionnaire response rates

Class Home institution Class topic

Archives A1 University A State history2 University A Regional women's histo3 Local community college State history4 Local community college American history to 186Sub-total Archives A

Archives B1 University B Race in America2 University B History of education3a University B Musicology4a University B Musicology5 University B History: World War II6 University B Sociology7 University B African-American studieSub-total Archives BTotal

a Classes 3 and 4 at Archives B were taught by the same instructor.

class using the archives (n = 120). Some of the less frequently sup-plied responses included engineering, agriculture and mathematics.This wide range of fields of study gave a good picture of the types ofacademic interests held by these students.

IMPACT OF THE ARCHIVES

As the Learning Impact Research Project stated, learning impactmay include such things as knowledge and understanding, skills, atti-tudes or values, enjoyment, and behavior (Hooper-Greenhill, 2004, p.163). We used four dimensions to operationalize impact: students'value received from their experiences in the archives, confidence,willingness to return, and perceiving the archives as valuable to largereducational goals. Accordingly, we first review the aspects of the stu-dents' experiences in the archives that they found valuable. Next, weexamine the correlation between confidence in performing archivalresearch and use of the archives after orientation, to learn aboutthe impact of archives in increasing students' perceptions of theirresearch abilities. As Duff and Cherry (2008) found, confidence is auseful measure of student learning to perform archival research.Students' willingness to return to perform research in the archivesis then explored, to learn which factors in their experiences influencethis willingness (or lack thereof). Finally, we discuss the extent towhich students identified the use of archives as valuable to theirgoals as a measure of the impact of archives, returning again to thedefinition of impact offered by Eilean Hooper-Greenhill.

WHAT KINDS OF EXPERIENCES IN THE ARCHIVES DO STUDENTS VALUE?

Students' responses to survey question 8, “What is the most im-portant thing you have learned this semester to help you conductarchival research?” indicated the types of learning they valued mostfrom their experiences in the archives. These responses connectedtwo measures of impact: the facets of the experience that studentsfound important and the learning that took place over the semester.

We organized the responses (n = 276) into two major groups:general awareness and specific skills, then divided these into morespecific categories as seen in Table 2. Students most frequently(n = 78) provided general comments, such as “Where to find andhow to use primary sources” and “The uses of primary sources.”They also frequently made comments concerning general awarenessof the archives (n = 47), “That the [Archives] exists” and “Where togo!” Responses in both of these categories reflected a very basic

Distributed surveys Completed surveys Response rate

176 172 97.7%ry 14 14 100.0%

21 21 100.0%5 21 15 71.4%

232 222

21 18 85.7%5 5 100.0%

11 10 90.9%31 17 54.8%70 46 65.7%

143 120 83.9%s 14 14 100.0%

295 230527 452 85.8%

Table 2What is the most important thing you have learned this semester to help you conductarchival research?

Type of response Number of responses

General awareness How to use the archives generally 78Awareness of the archives 47Available collections 30

Specific skills How to search the archives 33Asking for help 16Study skills/time management 12Research skills 11Preparation 6Procedural 5How to interpret resources 3

Other comments Sardonic comments 4Non-archives related web resources 4N/A or nothing 27Total 276

Table 3Use of the archives after the orientation.

Orientation only Orientation plus later archives use Total

Archives A 89 131 22040.5% 59.5% 100.0%

Archives B 181 49 23078.7% 21.3% 100.0%

Total 270 180 45060.0% 40.0% 100.0%

418 M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

understanding of the archives: where it is and what can be donethere. The responses made clear that simply learning that the ar-chives existed and were available for their use was a revelation formany students. Comments about the types of collections available(n = 30) also provided some indication that students valued thearchives as a resource.

A number of students mentioned that they gained specific skillsrelevant to working in the archives, as well as more general skillsthat could be applied to their other work. Students often mentionedhow to search the archives (n = 33), “Where to find the informationyou need and how to acquire it” and “How to go about finding yoursources and knowing how many boxes are out there to help you getwhat you need for research.” Additionally, some students found theassistance of archivists and staff to be truly helpful (n = 16) as theyidentified asking for help as a skill in such comments as “Use thelibrarians as a resource.” Still other students learned to improvetheir study, preparation, and time management skills over the courseof the semester (n = 12). Comments included “Be specific in mygoals,” “Go to the [Archives] in advance because it may not be there(it may be in storage),” and “To allow substantial time to go throughnumerous boxes + look at everything very carefully.” These com-ments indicated that using the archives had an impact on students'learning how to conduct research, from both a nuts-and-bolts per-spective, such as searching an archives' holdings, as well as in amore broad-based sense, like time management and preparation fora visit to the archives. This learning is an important aspect of theimpact of archives on students.

Table 4Confidence level and extended exposure to archives.

Based on your experience this semester,how confident are you in your ability todo archival research? n = 445

Type of use of archives Mean confidence N Std. DeviationOrientation alone 3.30 266 1.01Orientation plus later archives use 3.87 179 0.80Total 3.53 445 0.97

DOES USE OF ARCHIVESMAKE A DIFFERENCE IN STUDENTS' CONFIDENCEAS RESEARCHERS?

Student use of the archives after the archival orientation variedbetween classes. Some courses required archival research whileothers did not, and in cases where research was required, some stu-dents used resources other than those at the archives where theirorientation had taken place. Table 3 shows that 60% (n = 270) didnot use collections after the orientation session.

It is instructive, therefore, to compare the confidence level ofstudents who used archives after orientation with those who didnot. Question nine in the survey asked “based on your experiencethis semester, how confident are you in your ability to do archivalresearch?” Students were asked to rate their confidence on a scaleof 1 to 5 where 1 was “not confident at all,” 3 was “neutral,” and 5was “extremely confident.” As Table 4 shows, students who usedthe archives after orientation had a mean confidence of 3.87, morethan a half point higher than students who did not use the archives

after orientation. We conducted an independent-samples t-testto compare confidence in students who used the archives after orien-tation and those who did not. We found a significant difference in thescores for those who used (M = 3.87, SD = 0.80) and did not use thearchives after orientation (M = 3.30, SD = 1.01); t (443) = 6.296,p b 0.001. These results suggest that confidence in using archiveswas correlated with use of the archives over the course of the term.

In Duff & Cherry's, 2008 survey report of students in the YaleUniversity archives, confidence in using the archives rose from 4.1 to6.0 on a ten-point scale over the course of the term, where 1 was notat all confident and 10 was extremely confident. While the presentsurvey did not incorporate a pre-test design, as did Duff and Cherry's,our responses from students who used the archives after orientationare comparable, since all of the students in the Yale study wererequired to use primary sources in their coursework. Our mean scoreof 3.87 on a five-point scale is commensurate with the mean confi-dence score of 6.0 on a ten-point scale in Duff and Cherry's results.

WHAT FACTORS TEND TO BE ASSOCIATED WITH STUDENTS'WILLINGNESS TO RETURN TO THE ARCHIVES?

Students who were willing to return to the archives to conductresearch felt, for the most part, that they had been welcomed asstudents in the archives. A chi-square test of independence wasperformed to examine the relationship between willingness to returnand students' perception that their presence was welcomed. Therelationship between these variables was significant, X2 (1, N =433) = 116.10, p b .01. Students who felt their presence as a studentwas welcome in the archives were more willing to return to use thearchives again.

We then compared willingness to return to the archives withuse of the archives before and after the orientation (in Table 5).Two striking details emerge. First, all 36 of the students who hadused archives prior to the orientation said they would be willing toreturn to use them again. This suggests that repeated use leads togreater appreciation of archives. It may also, however, indicate thatstudents who had previous positive experiences in the archiveswere more likely to select a class that required archival research.Second, looking at students' use of the archives after the orientation,we see that almost two thirds (64.7%) of those who were unwillingto return had not used the archives since orientation. While many of

Table 5Willingness to return to the archives, cross tabulated with use of the archives before and after orientation.

Before the orientation, had you ever used thearchives or manuscripts at the [archives]?n = 441

Since the orientation, have you used thearchives or manuscripts at the [archives]?n = 447

No Yes Total No Yes Total

Would you come back to the[archives] if you had anotherproject that would benefitfrom the use ofits collections?

No 17 0 17 11 6 17100% 0% 100% 64.7% 35.3% 100%

Yes 388 36 424 256 174 43091.5% 8.5% 100% 59.5% 40.5% 100%

Total 405 36 441 267 180 44791.8% 8.2% 100% 59.7% 40.3% 100%

419M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

the students who did not use the archives after orientation would stillbe willing to return at a later date, this unwilling group may simplyhave never come to see the value of archives, or they may have hada negative experience during the orientation which colored theiropinion of the institution.

Table 7Willingness to return to the archives cross tabulated with value of archives for goals

WHAT FACTORS TEND TO BE ASSOCIATED WITH STUDENTS' PERCEPTIONTHAT ARCHIVAL RESEARCH IS VALUABLE TO THEIR GOALS?

Question 12 of the survey specifically asked “Is archival researchvaluable for your goals?” Of the 442 students answering this question,294 (66.5%) said yes while 148 (33.5%) indicated no. On their ownthese numbers indicate a large impact of archives on many of thesestudents, but an understanding of the factors associated with theseresponses is needed to give depth to the numbers and make theseresponses meaningful.

In Table 6, student responses to the question of whether archivalresearch is valuable to their goals are cross-tabulated with theirresponses to the question of whether they have developed skillsby doing archival research that help in other areas of their work orstudies. Few students overall said that they had developed new skillsthat were relevant to their other work; however, 91.2% of those whodid develop new skills also said that archival research is valuable fortheir goals. The relationship between these variables was significant,X2 (1, N = 426) = 37.39, p b .001.

Students who said that archival research was valuable to theirgoals also felt that they, as students, were welcome in the archives.Of the 291 students who said that archival research was valuable totheir goals, only 7 (2.4%) did not feel welcome. The relationshipbetween these variables was significant, X2 (1, N = 437) = 17.75,p b .001, suggesting that a perception of archival research as valuableand a feeling of welcome in the archives went hand in hand for mostof these students.

Students who said that archival research was valuable to theirgoals also reported a higher mean confidence level than otherstudents: 3.74 as opposed to 3.10 on a five-point scale. There was a

Table 6Development of skills cross tabulated with value of archives for goals.

Is archival researchvaluable for your goals?n = 426

No Yes Total

Have you developed any skills by doingresearch in archives that help you inother areas of your work or studies?

No 135 189 32441.7% 58.3% 100.0%

Yes 9 93 1028.8% 91.2% 100.0%

Total 144 282 42633.8% 66.2% 100.0%

significant difference in the confidence levels for those who value(M = 3.74, SD = 0.87) and do not value archives for their goals(M = 3.10, SD = 1.02); t (436) = 6.87, p b 0.001. These results sug-gest that confidence in using archives is correlated with perceptionsthat archival research is valuable for student goals.

Students responding to question 12, (“Is archival research valu-able for your goals”) also indicated the alignment of perceived valueof archives with willingness to return. As Table 7 shows, of the 294students who said that archival research was valuable for theirgoals, all but two said they would return to the archives if theyhad another project that would benefit from the use of its collections.The relationship between these variables was significant, X2 (1, N =441) = 21.48, p b .001, using Yates' correction for observed fre-quencies below five. Those who felt they had developed skills usingthe archives that helped them in other areas of their work or studieswere also more likely to return to the archives; however, the relation-ship between these variables was not statistically significant, X2

(1, N = 433) = 2.24, p = .14, using Yates' correction.These results suggest that perceptions of the value and usefulness

of archives are important indicators of willingness to return to the ar-chives. Those who believe that archives are valuable resources thatcan impact their own work are more likely to use them in the future.Because all of the students to complete this survey had an orientationsession at archives, there was an opportunity for archivists to explainand demonstrate the value of archives. These findings suggestthe utility of focusing on a broad definition of value of archives forstudents in the orientation session. The more that students perceivedthe value of archives, the more open they were to using archivesin the future and the more potential impact archives had on thestudents.

and development of skills.

Is archival researchvaluable for yourgoals? n = 441

Have you developedany skills by doingresearch in archivesthat help you inother areas of yourwork or studies?n = 433

No Yes Total No Yes Total

Would you come back to the[archives name] if you hadanother project that wouldbenefit from the use ofits collections?

No 15 2 17 16 1 1788.2% 11.8% 100% 94.1% 5.9% 100%

Yes 132 292 424 313 103 41631.1% 68.9% 100% 75.2% 24.8% 100%

Total 147 294 441 329 104 43333.3% 66.7% 100% 76.0% 24.0% 100%

420 M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

DISCUSSION

We answer the question of whether use of the archives had an im-pact on students in the affirmative, “yes.” Many students associatedfeelings of confidence about their research abilities with the archives,appreciated the archives as a resource, and thought that archival re-search was valuable to their goals. These associations concur withmeasures used by previous researchers of the impact of history edu-cation, archives, and other cultural heritage institutions. We positedfour dimensions of impact: students' valuation of their experiencesin the archives, overall confidence in their primary source researchabilities, willingness to return, and perceiving the project as valuableto larger educational goals. Each of these yielded different informa-tion about impact, both in terms of learning impact and other typesof impact.

In terms of value, we found that while learning how to usearchives was the most valued attribute, some students also valuedother more general and transferrable skills, such as study skills,time management, and skills related to the research project and prep-aration for it. While none of the courses specifically had these latterskills in mind as learning outcomes, a minority of students did pointto them as important take-aways. In looking for more transferrablelearning outcomes, university archivists may want to think abouthow to emphasize these dimensions of archival research.

Students who returned to the archives to conduct their ownresearch, rather than merely attending an orientation session, tendedto have greater confidence in their research abilities. This indicatesthat assignments provide a reinforcing mechanism for the learningprocess. However, these students were also more willing to returnto the archives at a future date, suggesting that students who hadgained more in-depth experience using archives were also morelikely to appreciate the archives as a useful resource. Students' assess-ment of the value of the archival research experience to future educa-tional goals also appears to be linked to confidence: students viewingarchival research as valuable to their goals reported a higher meanconfidence level than those who did not. This has pedagogical impli-cations for designing student learning activities and how the archivalresearch experience is presented, particularly to those in differentdisciplines.

Students who felt that their presence was welcome in the archivesalso appeared to have been impacted more positively by their experi-ences in the archives than other students. Archivists who create awelcoming atmosphere may also be creating an archival halo effectthat colors student perceptions of other aspects of their archivalexperience. While the power to shape student perceptions of the ar-chives as friendly may not be entirely under the control of archivists,staff training on the sensitivities of undergraduate students, in partic-ular, when encountering the new, unfamiliar environment of the ar-chives seems to have far reaching effects on their overall attitudestoward the archival experience.

An auxiliary goal of this research was the testing and refinementof the survey instrument itself, which the Archival Metrics projectteam designed as a mechanism for college and university archivistsand special collections librarians to use to assess their own impacton their student users. A number of Toolkits, designed to help theseorganizations conduct user-based evaluation, are available on theproject website http://archivalmetrics.org. The Toolkits consist ofimplementation instructions, data analysis instructions, pre-formattedspreadsheets and SPSS templates for results, and a sample report,alongwith the questionnaire. In addition to the Student Researcher sur-vey discussed here, other Toolkits assess user interactions generallywith the archives, online finding aids, archival websites, teachingsupport, and the economic impact of archives. We encourage archiviststo explore these toolkits, which are freely accessiblewith registration tothewebsite, to seewhether they can be useful in assessing the impact oftheir own organizations.

In implementing the student survey in particular, we encounteredseveral issues of importance to potential users. First and foremost, welearned that it is important to begin the evaluation discussion early inconsultation with faculty members who will be bringing their stu-dents to the archives. Our methodology includes asking instructorsto distribute the survey (which takes about ten minutes to complete)in class at the end of the term, collect the completed questionnaires,and return them to the archivist via campus mail. Faculty buy-in istherefore very important in assuring that the survey will be complet-ed and available for analysis. The survey itself should be part of archi-vists' discussions with faculty as they plan student interactions withthe archives for the academic term. In addition, we suggest remindingcourse instructors about the survey a few weeks before the end ofthe term and distributing copies to them at that time, to give themflexibility in when they administer it to students. We also recom-mend that archivists collect basic information from the instructorabout the course, including class number and title, number ofstudents enrolled, and a copy of assignments involving work in thearchives. In addition, we suggest keeping a brief summary of theservices provided to the class by the archives. This information willhelp during analysis in terms of factors specific to the course andmake results comparable across classes and over time. We hopearchivists and special collections librarians will find the Toolkits use-ful in performing assessment of their own services to students andother users.

CONCLUSION

In this article, we extend the discussion of archives impact on stu-dent outcomes and suggest several measures such as confidence inusing archives, willingness to return, perceived value of archivalresearch to student educational goals, and the perceptions of skilldevelopment which transfer to other aspects of student work. Wealso explore several factors that influence or moderate these mea-sures: use of the archives beyond orientation, affective feelings ofwelcome by the archives staff, and believing that the use of archivesmay be useful to students' future goals. By identifying and measuringthe factors that help archives to contribute to meaningful changes inthe lives of students and other users, archivists can understand theways that they make a difference for individuals and communities.This understanding can then be used to evaluate programs and activ-ities and to increase the positive impact of archives, an ongoing pro-cess that employs lessons learned from users to drive continualimprovement.

The recently issued Association of College & Research Libraries,“Characteristics of Programs of Information Literacy that IllustrateBest Practices: A Guideline” (2012) cites best practice as programsthat assess both the program (course) and student outcomes. Thusfar archives have primarily focused primarily on course evaluationwith some attention to student content retention. While our researchis far from definitive, our goal has been to further explore what wemean by student impact and to test different measures of impact.Tying the archival mission to student learning is essential in manycolleges and universities but the best means of accomplishing thisgoal has yet to be identified. We hope that future research willamend our measures and introduce more robust impact assessmentinto the discussion of student learning and archives.

ACKNOWLEDGMENTS

This research was funded by a grant from the Andrew W. MellonFoundation.

The authors give their thanks to Paul Conway, who provided com-ments on an earlier draft of this paper, and to the archivists, profes-sors, and students who participated in this research.

Please elaborate on your degree of confidence:

421M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

APPENDIX A. ARCHIVAL METRICS SURVEY QUESTIONNAIRE

Section 1: Orientation

1. How satisfied were you with the orientation you attended at the [Archives] this term?Please circle the appropriate number:

2. What did you find most useful about the orientation? Please choose one of the following:

Learning about the archives’ holdingsSeeing and/or handling the documentsLearning about the archives’ policiesInstruction on how to use access tools (i.e. catalogs, finding aids, online databases)Other ______________________________________________________________

3. What was missing in the orientation that would have been helpful to you?

4. How would you describe the orientation to a fellow student? Please choose one of the following:

Essential knowledge for conducting researchGenerally good knowledge to haveUseful knowledge only if you need primary sourcesInteresting, but not relevantNeither interesting nor useful

Completelydissatisfied Neutral Completely

satisfied

1 2 3 4 5

5. Before the orientation, had you ever used the archives or manuscripts at the [Archives]?

YES NO

Section 2: Use of Archives

6. Since the orientation, have you used the archives or manuscripts at the [Archives]?

YES NO

IF YES, go to question 7, if NO, go to question 8

7. Please rate the extent to which you have used the following sources from the [Archives] for coursework or other research this semester. Please circle the appropriate number:

8. What is the most important thing you have learned this semester to help you conductarchival research?

9. Based on your experience this semester, how confident are you in your ability to do archival research? Please circle the appropriate number.

Type of Source Extent UsedNot At All A Great Deal

Organizational Records 1 2 3 4 5Photographs 1 2 3 4 5Personal papers and correspondence 1 2 3 4 5

Maps 1 2 3 4 5Sound recordings 1 2 3 4 5Film or video 1 2 3 4 5Published books or pamphlets 1 2 3 4 5Newspapers or periodicals 1 2 3 4 5Architectural plans 1 2 3 4 5Government documents 1 2 3 4 5Journal articles 1 2 3 4 5Information published on the web 1 2 3 4 5

Not at allconfident Neutral Extremely

confident

1 2 3 4 5

[Name of College or University][Name of Archives]

Student Researcher Survey v.1.0

We would like to ask you some questions regarding your experience using the materials at the [Archives] the [Name of term]. Your response is voluntary and you may choose to skip any question. YES NO

YES NO

YES (If yes, please describe) NO

11. Did you feel that your presence, as a student, was welcomed?

12. Is archival research valuable for your goals?

13. Have you developed any skills by doing research in archives that help you in other areas of your work or studies?

14. Please note your field of study: ___________________________

15. Please note your year of study:

FreshmanSophomoreJuniorSeniorMaster’sPh.D.

Thank you for completing the questionnaire. We welcome any additional comments you may have, including suggestions for the orientation session.

YES NO

10. Would you come back to the [Archives] if you had another project that would benefit from the use of its collections?

REFERENCES

Andrews, D. C., Cassman, V., & Johnson Melvin, L. R. (2012). Learning as doing: Under-grads using special collections for conservation and material culture studies. InEleanor Mitchell, Peggy Seiden, & Suzy Taraba (Eds.), Past or portal? Enhancingundergraduate learning through special collections and archives (pp. 290–297).Chicago, IL: Association of College & Research Libraries.

Association of College & Research Libraries (2012). Characteristics of programs of infor-mation literacy that illustrate best practices: A guideline. ACRL, January 2012. ( http://www.ala.org/acrl/standards/characteristics (accessed November 5, 2012))

Bahde, A. (2011). Taking the show on the road: Special collections instruction in the campusclassroom. RBM: A Journal of Rare Books, Manuscripts, and Cultural Heritage, 12, 75–88.

Bain, Robert B. (2005). ‘They thought the world was flat?’: Applying the principles ofhow people learn in teaching high school history. In How Students (Ed.), Learn:History, mathematics, and science in the classroom, edited by John Bransford andSuzanne Donovan (pp. 179–214). Washington: The National Academies Press.

Bean, R., & Anderson, L. M. (2012). Teaching research and learning skills with primarysources: Three modules. In Eleanor Mitchell, Peggy Seiden, & Suzy Taraba (Eds.),Past or portal? Enhancing undergraduate learning through special collections andarchives (pp. 156–162). Chicago, IL: Association of College & Research Libraries.

Boyer Commission on Educating Undergraduates in the Research University (1998).Reinventing undergraduate education: A blueprint for America's research universities.Stony Brook, The State University of New York (Accessed November 5, 2012.http://www.niu.edu/engagedlearning/research/pdfs/Boyer_Report.pdf)

Carini, P. (2009). Archivists as educators: Integrating primary sources into the curricu-lum. Journal of Archival Organization, 7(1–2), 41–50.

Carter, L. R. (2012). Articulating value: Building a cultureof assessment in special collections.RBM: A Journal of Rare Books, Manuscripts, and Cultural Heritage, 13, 89–99 (Fall 2012).

Cook, S. A. (1997). Connecting archives and the classroom. Archivaria, 44, 102–117.Cotton, J., & Sharron, D. (2011). Engaging students with archival and digital resources.

Oxford, UK: Chandos.Dearstyne, B. W. (1987). What is the “use” of archives? A challenge for the profession.

The American Archivist, 50(1).Duff, W., & Cherry, J. (2008). Archival orientation for undergraduate students: An

exploratory study of impact. The American Archivist, 71(2), 499–530.Grob, J. (2012). More than gold leaf: Teaching undergraduates in capstone courses

about the scholarly use of medieval manuscripts. In Eleanor Mitchell, PeggySeiden, & Suzy Taraba (Eds.), Past or portal? Enhancing undergraduate learningthrough special collections and archives (pp. 145–150). Chicago, IL: Association ofCollege & Research Libraries.

Hooper-Greenhill, E. (2004). Measuring learning outcomes in museums, archives andlibraries: The Learning Impact Research Project (LIRP). International Journal ofHeritage Studies, 10(2), 151–174.

422 M. Daniels, E. Yakel / The Journal of Academic Librarianship 39 (2013) 414–422

Horton, S., & Spence, J. (2006). Scoping the economic and social impact of archives, reportfor MLA Yorkshire, Department of Information Studies. Aberystwyth: University ofWales.

Krause, M. G. (Fall/Winter). Undergraduates in the archives: Using an assessmentrubric to measure learning. American Archivist, 73, 507–534.

Osborne, K. (1986). Archives in the classroom. Archivaria, 23, 16–40.Robyns, M. C. (2001). The archivist as educator: Integrating critical thinking skills into

historical research methods instruction. The American Archivist, 64(2), 363–384.Rockenbach, B. (2011). Archives, undergraduates, and inquiry-based learning: Case

studies from Yale University Library. The American Archivist, 7(4), 297–311.Rouet, J., Britt, M. A., Mason, R. A., & Perfetti, C. A. (1996). Using multiple sources of

evidence to reason about history. Journal of Educational Psychology, 88, 478–493.Sutton, S. C. (2012). The special collections laboratory: Integrating archival research

into undergraduate courses in psychology and music. In Eleanor Mitchell, PeggySeiden, & Suzy Taraba (Eds.), Past or portal? Enhancing undergraduate learning

through special collections and archives (pp. 175–178). Chicago, IL: Association ofCollege & Research Libraries.

Taylor, H. A. (1972). Clio in the raw: Archival materials and the teaching of history. TheAmerican Archivist, 35(3/4), 317–330.

Williams, D. A., Wavell, C., Baxter, G., MacLennan, A., & Jobson, D. (2005). Implementingimpact evaluation in professional practice: A study of support needs within themuseum, archive, and library sector. International Journal of Information Management,25, 533–548.

Wineburg, S. (1991). Historical problem solving: A study of the cognitive processesused in the evaluation of documentary pictorial evidence. Journal of EducationalPsychology, 83, 73–87.

Wineburg, S. (2001). Historical thinking and other unnatural acts: Charting the future ofteaching the past. Philadelphia: Temple University Press.

Wosh, P. J., Bunde, J., Murphy, K., & Blacker, C. (2007). University archives and educa-tional partnerships: Three perspectives. Archival Issues, 31(1), 83–103.