ONE OR MANY?ASSESSING DIFFERENT DELIVERY TIMING FOR INFORMATION RESOURCES RELEVANT TO ASSIGNMENTS DURING THE SEMESTER.A WORK IN PROGRESS

June 11, 2012

Amy S. Van Epps and Megan R. Sapp NelsonEngineering Librarians, Purdue University

INTRODUCTION

OPPORTUNITY

3

Information Literacy opportunity with different sections of the same first year course

• Fundamentals of Speech Communication • Associated with engineering learning communities

Different delivery models • a single, 50 minute lecture early in the semester• four, approximately 12 minute lectures to be offered just before

each assignment was given

Material presented was coordinated • LibGuide to minimize variation

RESEARCH QUESTION

4

Is there a noticeable difference in the quality of citations in student assignments when “just-in-time” instruction is used as opposed to a “one-shot” session?

Hypothesis Sections which received the just-in-time

instruction would have better citations, both in quantity and quality than the section which received the one-shot session.

METHODS AND SAMPLE

METHODS

6

Study GroupAll students in 3 sections of the course

DataOutlines with references required for each

assignmentData received by librarians after anonymized

Reference AnalysisDone using a framework to identify resource type and quality of intended audience and purpose

SAMPLE SELECTION

7

SamplingRandom sample of outlines for coding

Analysis Set5 bibliographies for each assignment (1-3) per team3 bibliographies for final assignment per team

36 bibliographies233 references

124 references from just-in-time set; 109 from one-shot

Based on the work of Wertz et al (2011) Resource type Quality

BookPeriodicalWebFacts & FiguresUnknown

Inter-rater reliability, consensus estimate • 84.1 percent agreement

CODING FRAMEWORK

8

Scholarly Popular

Informative

Biased

Medium

Medium

High

Low

RESULTS ANDDISCUSSION

RESULTS

10

Average number of references per outline

All assignments required a minimum of 3 references.

  Just-in-time (n=124) One-Shot (n=109) Total (n=233)

Individual Assignments (#1-3)* 4.93 3.86 4.4

Group Assignment (#4) 16.67 17.00 16.8

All assignments 6.88 6.05 6.5

* Statistically significant difference (p<.05)

QUALITY OF RESOURCES USED

11

High* Medium* Low* Unable to classify0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

mini-lectures one-shot

* Statistically significant difference (p<.05)

Quality Embeddedn=124

One-Shotn=109

Significance

High* 43.6% 22.9% (Z= 3.31, p< .001)Medium* 51.6% 65.2% (Z= -2.06, p< .05)Low* 2.4% 9.2% (Z= 2.24, p< .05)Unknown 2.4% 2.7% (Z= -0.16, p= .873)

High, medium, and low quality sources all show statistically significant differences between sections.

COMPLETENESS OF CITATIONS

12

Difference for IncompleteMay reflect differences between raters more than differences in student abilities.

APA format was not taught by the librarians

complete reference = presence of all elements

Complete Improper Incomplete* Unknown0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

mini-lectures one-shot

* Statistically significant difference (p<.05)

Completeness Embed. One-Shot SignificanceComplete 55.7% 60.6% (Z= -0.756, p= .453)Improper 29.8% 32.1% (Z= -0.37, p= .712)Incomplete* 12.1% 4.6% (Z= 2.03, p< .05)Unknown 2.4% 2.7% (Z= -0.16, p= .873)

TYPE OF RESOURCE

13

Significantly different distribution of resources between sections.

One-shot students use many more web resources. Just-in-time students split between journals and web resources.

Monograph Periodical* Web* Interpersonal Unknown Facts & Figures0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

mini-lectures one-shot

* Statistically significant difference (p<.05)

Type Embed. One-Shot SignificanceMonograph 6.5% 4.6% (Z= 0.617, p= .535)Periodical* 48.4% 9.2% (Z= 6.52, p< .001)Web* 41.9% 83.5% (Z= -6.50, p< .001)Interpersonal 0% 0% Unknown 2.4% 2.7% (Z= -0.16, p= .873)Fact & Fig. 0.8% 0%

DISCUSSION

14

An apparent benefit can be see with just-in-time teaching• Use of more high quality resources• Use of more periodicals • Frequently visits = opportunity for follow-up

Overall use of popular and informative resources • Medium quality; 59.9% not surprising to the authors• Speech communication is well suited to popular resources• Less requirement of research or scholarly publications than

design assignments or research papers • 93.4% of the resources used were classified as informative

CONCLUSION

CONCLUSION

16

• A correlation exists embedded just-in-time teaching approach and the quality of reference sources used can be seen.

• Still unknown…if the library instruction model is responsible for the difference

QUESTIONS?