USING STUDENT ACCOUNTS OF THE RESEARCH PROCESS TO IMPROVE AN

INFORMATION LITERACY MODULE

Journals for Learning, Journals for Teaching

Claire McGuinness

Presentation Outline

Context: Authentic Assessment for Information Literacy

Instruction Reflection in the Learning Process Journal-keeping as a reflective learning tool Reflective practice in teaching “Assessment-Evaluation-Revision” Cycle

Information Literacy in UCD: The Case Study Module overview Analysis of student research experience reported in 2007 Using students’ research experience to adapt module

Framework

Authentic Assessment

Traditional methods of assessing IL outcomes (i.e. quantitative, standardised tests, product appraisal, evaluation surveys) – only assess certain level of learning or student satisfaction

Authentic assessment: “an evaluation process that involves multiple forms of performance measurement reflecting the students’ learning, achievement, motivation and attitudes on instructionally relevant activities” (Callison, in Sharma, 2007, p.127)

Reflective Learning

Metacognition - “awareness of [one’s] own cognitive machinery and how the machinery works” (Meichenbaum, in Woolfolk et al, 2008, p.319)

Declarative knowledge – knowing what to do (skills, strategies) Procedural knowledge – knowing how to use the strategies Conditional knowledge - knowing when and why to use strategies

“Learning is enhanced as learners become aware of their own learning strategies and begin to monitor the use of these strategies” (Grassian & Kaplowitz, 2001, p.48)

Three skills which enable individuals to regulate thinking and learning – Planning, Monitoring and Evaluating (Woolfolk et al, p.320)

Learning & Reflective Journals/Portfolios

“Journals, logs, diaries, portfolios are containers for writing that isrecorded over a period of time”

“A learning journal is: diverse; not necessarily written - but most of the time assume written

format; generally reflective and accumulated over a period of time with

the intention to learn, i.e. not purely descriptive; flexible (it can be structured or unstructured) a useful back-up to learning; something that accentuates favourable conditions for learning -

e.g. space, time, reflection. •applicable to all disciplines (not just literary)”

(Jennifer Moon for UCD Centre for Teaching & Learning)

EG’s of Journals/Portfolios for IL instruction

Diller & Phelps (2008): Use of e-portfolio to assess IL in General Education Programme. Students included two pieces of evidence for each of 6 learning goals, e.g. Critical Thinking, IL and Communication, Self in Society

Sharma (2007): Use of a web portfolio to document the process of researching a 20-page paper. Included topic statement, concept maps, research q’s, research log, annotated bibliography & reflection

Scharf et al (2007): Use of existing writing portfolios to assess IL skills at end of 4-year programme. Rubric developed using five traits linked to ACRL standards to measure IL performance in student papers selected from portfolios

Sonley et al (2006): Use of research information literacy portfolio in “Negotiated Learning” module (adult learners), which included a bibliography, and “evidence to demonstrate the process which had been undertaken to lead to this bibliography” (p.48), e.g. Search strategies, evaluation of sources, etc.

Reflective Practice

“If we are going to address the issues of librarians’ roles within educational endeavours systematically, we, as a discipline, need to foster reflective, critical habits of mind regarding pedagogical praxis within ourselves, our libraries and our campuses” (Jacobs, 2008, p.256)

Critical reflection is A process more than an

outcome Recognising and using

trigger events Letting go of assumptions

without feeling guilty Tipping over the ‘rocks’ of

tradition or habit Confronting illusions Deciding on potential

changes in practice (action research)

(Burge, in Thomas et al, 2004, p.13)

Assessment-Evaluation-Revision

“Assessment should be student centred and proactive. The results should be used to implement positive changes in the teaching of information literacy” (Avery, 2003, p.2)

“Authentic assessment is, among other things, iterative. Librarians can evaluate student learning, implement changes, and continue this cycle so that improvement via assessment becomes an inherent part of the instructional process.” (Sonntag & Meulemans, 2003, p.8)

Case Study -Information Literacy Module

Modular structure established in UCD in 2005-6 (“Horizons”)

Overhaul of course offerings – new First Year modules developed by School of ILS

Concerns about quality of First Year student essays – plagiarism and copying a particular issue

Decision to establish new full (12-week) information literacy module, aimed at improving students’ skills in researching and writing academic essays

“Introduction to Information Literacy” (re-named in 2006-7: “Information Literacy: Information Skills for Effective Academic Writing”)

Collaboration with UCD Library staff, who deliver most of the “information searching” sessions

Module Objectives

“On completion of this module students should be able to: Identify and clearly articulate information needs in the context of

specific information tasks. Identify the range of potential information sources, and evaluate

and choose the appropriate source(s) for specific information tasks.

Access and retrieve needed information efficiently from a variety of information media, through the formulation and execution of effective search strategies.

Critically evaluate information and its sources in terms of bias and relevance to specific information tasks.

Synthesise the gathered information to successfully solve an information problem and create a final product.

Recognise and reflect on own information-seeking strategies and behaviour. “

Information Literacy Framework

Stripling & Pitts’ Model of the Research Process (1988)

Step 1: Choose a broad topic Step 2: Get an overview of the topic Step 3: Narrow the topic Step 4: Develop a thesis or statement of purpose Step 5: Formulate questions to guide research Step 6: Plan for research and production Step 7: Find / Analyze / Evaluate sources Step 8: Evaluate evidence / Take notes / Compile bibliography Step 9: Establish conclusions / Organize info. into an outline Step 10: Create and present final product  

Assessment Protocol

Year 1: One-page essay proposal

(10%) – discontinued after year 1

1500-word essay (50%) Reflective Research Journal

(30%) Tutorial attendance (10%)

Year 2: 1500-word essay (50%) Reflective Research Journal

(40%) Tutorial attendance (10%)

From Year 3: 1500-word essay (50%) Reflective Research Journal

(35%) Tutorial attendance (15%)

IL pre-test (MCQ) used in Years 1

& 2, but discontinued as onlyoffered limited insight intostudents’ skills deficits, and

werelabour-intensive to grade

Research Journal

In Year 1: 10 weeks (from Week 3 to Week 12)

Weekly entries comprised:

Goals – planned research activities for week ahead Activities – activities actually accomplished during week Readings – items read or referred to by student during week Problems – honest account of any difficulties encountered Reflection – space to reflect on the experience of research and

of writing the journal Appendices – diagrams, exercises, concept maps, search

printouts, abstracts, pictures, etc... Not compulsory, but encouraged

Research Journal

Reflection in final week:

What was most challenging about the assignment? What was easiest? What have I learned through this assignment? What am I proud of with this assignment? What would I do differently next time I complete a

similar task?What did I enjoy most about writing this assignment?

Analysis of Journals 2005-6

Problems experienced

Narrowing essay topics Orientation in the library Missing/unavailable items in Lib Information overload Fear of overlooking information Identifying relevant information “False hits” when searching Problems with ICT’s Time management Getting side-tracked Problems with organising info Difficulty with essay structure

Over-reliance on print or electronic resources, especially the Internet

Problems with specialist terminology

Not having much to write about some weeks

Problems finding statistics Understanding referencing Technical problems De-motivation Difficulties with search strategies Lack of confidence Fear of accidental plagiarism

Analysis of Journals 2005-6

Students enjoyed/wereproud of:

Opportunity to explore novel topics to build knowledge base in new areas

Freedom to choose own topic Improvement in search skills Appreciation of using

multiple information formats in research

Appendices allowed for creative expression

Drawing evidence together to create arguments

Sense of achievement and personal fulfilment – persistence through difficulties, hard work, effort

Success in completing unfamiliar type of assignment

Motivation to be organised Developing awareness of

multi-disciplinarity and cognate subject areas

Evolution of a Module

Changes made since Year 1:

Length of journal (10 weeks to 8 weeks to 6 weeks) More lab sessions & new lecture content Summative to formative assessment (Year 2 only) Essay topics – broad and unfocused, to narrower ,

more clearly defined questions Increased “portfolio” component – compulsory

worksheets based on lecture and tutorial content More emphasis on “activity” rather than pure

reflection

Summative to Formative

Formative assessment: “The purposes of formative assessment are to guide the teacher in planning and to help pupils identify areas that need work. In other words, formative assessment helps form teaching.” (Woolfolk et al, 2008, p.660)

In Year 2, journal was submitted in two parts (4 weeks each)

Aim was to provide students with feedback before second half of journal submitted

In the event, turnover was not rapid enough to help students – large numbers meant that second half of journal was mostly written before first half returned with comments and grades.

Old Essay Titles Examples

Old Essay Format

“Censorship and the Media”

“Health Promotion and Smoking”

“Health Information and the Internet”

“Terrorism and News Reporting”

“Political Corruption and Investigative Journalism”

“Diet and Popular Media”

Extremely broad topics Student expected to narrow

topic through exploratory research

Students work together on topics in tutorial groups

Student decides on “type” of essay, i.e. Argumentative, persuasive, etc

Unique “thesis statement” developed for each essay

Students’ Research Experience

Students stuck here until close to end Kuhlthau’s Information Search Process

Student Quotes

In Week 6 (2006): “Again, I had great trouble trying to narrow down the topic, wish I could just write about a few different things.”

In week 4 (2006): “Getting worried now as I am still struggling to narrow down my topic. I feel as if I never will be able to narrow it down and would much prefer a different research topic!”

What would you do differently next time? (2006): “I would have tried to narrow my topic in to a specific essay title sooner. I wasted a lot of time researching vast amounts of broad information that ended up being useless to my essay title.”

What was most challenging? (2006): “As probably everybody else will state as well, I found it hardest to actually narrow the research topic down to one specific topic and question relating to it, which was caused by the sheer amount of information that is to be attained.”

Student Quotes

In week 8 (2007): “Last journal [entry] and things have still not clearly formed in my head of what my essay title will be.”

In week 8 (2007): “I am not completely happy with the topic and approach I chose, but at the moment it is too late to alter anything significantly. I am hoping that my good intentions an idea will come through in my writing and the paper will fit the initial, broad topic of Media and Censorship.”

In week 7 (2007): “I do not have a structure for my essay, or [am not] even certain what it will be”

In week 6 (2007): “I feel I made little progress this week; this is alarming as I am over half-way through the journal. This step backwards is quite disheartening, and I’m starting to question the aspect I decided to zoom in on.”

  In week 8 (2007): “Although this is the last week, I am having a change of mind in

the direction of my essay [...]. This is a major change, so I will have to change the majority of my information.”

 

Coursework under the microscope...

Persisted with broad essay topics for 3 years – belief that students should develop skills to explore and narrow topics through background reading and research, and lectures/tutorials were geared towards that

However, evidence of low confidence and frustration until late in process, in some cases never fully resolved. While frustration is normal during process, it should eventually give way to confidence and relief.

Too advanced for First Year students? More appropriate for 3rd or 4th years, or novice postgraduate researchers?

Transition from secondary to tertiary education needs to be carefully managed, to ensure student retention and motivation.

Would a more clearly defined approach to topic selection strike the correct balance between the structured approach of secondary education, and the more flexible, independent approach at university?

What kinds of essays are assigned to undergraduates in other modules?

New Essay Titles (Year 4) Examples

Reflect type of assignments students have to do in other modules

Clearly defined “Essay questions” but topics still quite broad

Students more easily able to articulate information needs?

Students can hopefully move beyond “frustration” stage more quickly

Students can spend more time searching for & evaluating info

“Discuss possible solutions to the problem of racism towards immigrants in Ireland”

“It is widely agreed that poverty is one of the major causes of anti-social behaviour. Do you agree?” 

“Reality television competitors are the modern-day gladiators” Discuss

New Essay Format

From journal to portfolio

Previously, journal was primarily reflectivePossible to complete it all in one go at end (some anecdotal

evidence, and date on printouts in appendices!)“Hands-on” activities (e.g. worksheets) not compulsoryReluctant acceptance (to an extent!) of the “strategic student”

– unless submission is weekly, not possible to monitor entriesRevisited course objectives – importance of developing skillsPortfolio format – students have to complete various exercises,

etc, so work gets done regardless of when they do itWhich is more important – students complete regular weekly

entries, or learn through activities?

Portfolios

“A portfolio is a purposeful collection of student work that exhibits the student’s efforts, progress or achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for judging merit and evidence of the student’s self-reflection” (Paulson et al, in Sonley et al, 2007, p.47)

Convey the “thinking and activity involved in the creation of major projects, and may contain records of original brainstorming sessions, early and current drafts […] and student self assessments, instructor evaluation, and student thoughts on further projects” (Thomas, 1999, p.140).

Journal Format Change

New Template (Year 4)

What stage of the process are you at this week?What research and/or writing tasks have you attempted this

week? Which information resources have you searched and/or read

this week? (List at least 3) Give your reasons for either choosing or rejecting resources

you have looked at this week. How do you feel your research and/or writing is going this

week?Compulsory appendices – students must include all

exercises handed out in class, plus self-selected work

Student Quotes (2008)

Main challenge is now focused on the information search process:

“The most challenging aspect of the assignment was the gathering of relevant sources […] Citing and quoting references was one of the more difficult aspects for me. I found the need to quote quite tedious, however I do understand the importance of it”

  “I found the research of my topic quite difficult as it was hard to find books of

good use [sic] and on the same topic.”  “It was difficult to keep looking at different sources that I could write about

in my journal each week”  “The most difficult part of this assignment was finding enough sources to fill

my research journal, it was very time-consuming to read three sources a week while juggling my other subject assignments”

 

Future Plans?

More reflection required....

Change to e-portfolio format?

Give students option of presenting journal in Blog format?

No longer specify the number of readings to be done each week?

Introduce complete flexibility – students not required to post weekly entries?

Thank you!

Contact details

Claire McGuinnessSchool of Information & Library Studies,

UCDPh. 7167054 (Thurs & Fri)Email: claire.mcguinness@ucd.ie

References

Avery, E.F. Assessing Student Learning Outcomes for Information Literacy Instruction in Academic Institutions. Chicago: Association of College & Research Libraries, 2003

Diller, K.R., & Phelps, S.F. (2008). Learning outcomes, portfolios and rubrics, Oh my! Authentic assessment of an information literacy programme. Portal: Libraries and the Academy, 8(1), pp. 75-89

Grassian, E.S., & Kaplowitz, J.R. Information literacy instruction: Theory and Practice. New York: Neal-Schuman, 2001

Jacobs, H.L.M. (2008). Information literacy and reflective pedagogical praxis. Journal of Academic Librarianship, 34(3), pp.256-262

McGuinness, C. & Brien, M. (2007). Using reflective journals to assess the research process. Reference Services Review, 35(1), pp.21-40.

References

Scharf, D., Elliot, N., Huey, H.A., Briller, V. & Joshi, K. (2007). Direct assessment of information literacy using writing portfolios. Journal of Academic Librarianship, 33(4), pp.462-478.

Sharma, S. (2007). From chaos to clarity: Using the research portfolio to teach and assess information literacy skills. Journal of Academic Librarianship, 33(4), pp.127-135.

Sonley, V., Turner, D., Myer, S., & Cotton, Y. (2007). Information literacy assessment by portfolio: A case study. Reference Services Review, 35 (1), pp.41-70

Sonntag, G., & Meulemans, Y. Planning for assessment. In Avery, E.F. (ed.) Assessing Student Learning Outcomes for Information Literacy Instruction in Academic Institutions. Chicago, Association of College & Research Libraries, 2003, pp.6-21

References

Thomas, D.B., Tammany, R., Daier, R., Owen, E., & Mercado, H. Reflective Teaching: A Bridge to Learning. (Library Orientation Series). Ann Arbor, MI: Pierian Press, 2004

Thomas, N.P. Information Literacy and Information Skills Instruction: Applying Research to Practice in the School Library Media Center. Englewood, CO: Libraries Unlimited, 1999

Woolfolk. A., Hughes, M. & Walkup, V. Psychology in Education. Essex: Pearson Education Ltd, 2008