critical thinking and information literacy: assessing student performance work in progress critical...

1

Work in Progress

Critical Thinking and Information Literacy: Assessing Student

PerformanceThe InfoSkills Research Team

Dr. Senay Purzer, Mr. Michael Fosmire, Ms. Amy Van Epps, and Ms. Ruth Wertz

Project Collaborators

Ms. Megan Sapp Nelson, Dr. Brian Dillman, and Mr. Austin Saragih

Presented at the 120th ASEE Annual Conference & Expo – Atlanta, GAJune 24th, 2013

InfoSkills, 2013

2

Overview

Project Background – Information Literacy and Critical Thinking

Research Methods – RQs and Research Design

Project Results – Correlation of CELT and CAT

Project Conclusions – Implications & Further Study

InfoSkills, 2013

3

A set of skills that enables the ability to recognize the need for information, and the ability to search for, access, evaluate, and

use information to fulfill a specific purpose.1

InfoSkills, 2013

DefinitionsProject Background

Information Literacy

The cognitive process of “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or

generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.”2

Critical Thinking

1. American Library Association. (2000). Information literacy competency standards for higher education. Retrieved from http://www.ala.org/acrl/sites/ala.org.acrl/files/content/standards/standards.pdf.

2. The Foundation for Critical Thinking. (2013). Defining critical thinking. Retrieved from http://www.criticalthinking.org/pages/defining-critical-thinking/410.3. Albitz, R. S. (2007). The what and who of information literacy and critical thinking. portal: Libraries and the Academy, 7(1), 97-109.

recognize need

accesssearch for/gather

evaluateuse/apply

synthesize

analyzeconceptualize

SET OF SKILLS3

COGNITIVE PROCESSES3

4

Impetus for our research:• Important 21st century skills• ABET criterion 3i (lifelong learning)• Engineering decision-making

Primary focus of our research:• How do we assess information literacy and critical

thinking?

Impetus for Our Research

InfoSkills, 2013

Project Background

5

InfoSEAD framework for item generation1

• Seeking – Gathering and access• Evaluation – Assess quality, credibility, etc.• Application – Using information to satisfy a need• Documentation – Citation and in-text referencing

Two Scenarios 18 total items• 16 multiple choice; 2 select all that apply

About the CELT(v2.1)

InfoSkills, 2013

Project Background

1. Wertz, R. E. H., Purzer, S., Fosmire, M. J., & Cardella, M. E. (in press). Assessing information literacy skills demonstrated in an engineering design task. Journal of Engineering Education.

6

Research question:• How well does the CELT (v2.1) measure critical

thinking?

Research method:• Pearson’s correlational analysis between CELT

instrument and the Critical Thinking Assessment Test (CAT) developed by Tennessee Technology University.1

InfoSkills, 2013

Correlational StudyResearch Methods

1. Center for Assessment and Improvement of Learning at Tennessee Technological University. (2010). CAT technical information, from http://www.tntech.edu/files/cat/reports/CAT_Technical_Information_V7.pdf

7

N = 44 first-year engineering students CELT reliability measures• KR-20 = 0.671

• N = 188 first-year students (Fall 2012)• Based on 16 MC questions

CAT reliability measures • 15 constructed response items; scoring reliability 0.822

• Cronbach’s alpha 0.702

InfoSkills, 2013

Population and SettingResearch Methods

1. Wertz, R. E. H., Saragih, A., Fosmire, M. J., & Purzer, S. (2013). An Evaluation of the Critical Engineering Literacy Test (CELT) Instrument through Item Analysis and Comparison to the Critical Assessment Test (CAT). Paper presented at the 2013 Illinois/Indiana - ASEE Section Conference, Angola, IN.

2. Center for Assessment and Improvement of Learning at Tennessee Technological University. (2010). CAT technical information, from http://www.tntech.edu/files/cat/reports/CAT_Technical_Information_V7.pdf

8

Overall Correlational Analysis:• CELT total score was positively related the CAT

total score (r = 0.47, p <0.01)

Itemized Correlational Analysis:• Items 5, 11, 16, 17 had positive associations with

the CAT total score

InfoSkills, 2013

CELT ResultsProject Results

9InfoSkills, 2013

CELT ResultsProject Results

Which of the following comparisons of omega3/omega6 ratios is most relevant in determining whether GE fish is equivalent to

its non-GE counterpart?

Item 11(r = 0.32, p <0.05)

What would help the review panel validate the data presented?Item 16

(r = 0.37, p <0.05)

Where would you likely find authoritative information on a typical omega-3 levels of salmon?

Item 17(r = 0.35, p <0.05)

[Evaluate] which of the citations is incorrect or incomplete?Item 5

(r = 0.51, p <0.01)

10

14 of 18 CELT items positively correlated to some portion of CAT

Correlation between the CELT and CAT total scores is moderately strong (r = 0.47, p <0.01)

CELT items 5, 11, 16, and 17 provide insight into a possible subset of skills where information literacy skills and critical thinking overlap.

InfoSkills, 2013

General ConclusionsProject Conclusions

11

What are the common elements in CELT items 5, 11, 16, and 17 that made them correlate to CAT?• Expand sample• Item-to-item analysis• Verbal protocol

InfoSkills, 2013

Future StudyProject Conclusions

12

Dr. Senay Purzer – senay@purdue.edu

Mr. Michael Fosmire – fosmire@purdue.edu

Ms. Amy Van Epps – vanepa@purdue.edu

Ms. Ruth Wertz – rwertz@purdue.eduInfoSkills, 2013

THANK YOU!

Contact Information