critical thinking and information literacy: assessing student performance work in progress critical...
TRANSCRIPT
1
Work in Progress
Critical Thinking and Information Literacy: Assessing Student
PerformanceThe InfoSkills Research Team
Dr. Senay Purzer, Mr. Michael Fosmire, Ms. Amy Van Epps, and Ms. Ruth Wertz
Project Collaborators
Ms. Megan Sapp Nelson, Dr. Brian Dillman, and Mr. Austin Saragih
Presented at the 120th ASEE Annual Conference & Expo – Atlanta, GAJune 24th, 2013
InfoSkills, 2013
2
Overview
Project Background – Information Literacy and Critical Thinking
Research Methods – RQs and Research Design
Project Results – Correlation of CELT and CAT
Project Conclusions – Implications & Further Study
InfoSkills, 2013
3
A set of skills that enables the ability to recognize the need for information, and the ability to search for, access, evaluate, and
use information to fulfill a specific purpose.1
InfoSkills, 2013
DefinitionsProject Background
Information Literacy
The cognitive process of “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or
generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.”2
Critical Thinking
1. American Library Association. (2000). Information literacy competency standards for higher education. Retrieved from http://www.ala.org/acrl/sites/ala.org.acrl/files/content/standards/standards.pdf.
2. The Foundation for Critical Thinking. (2013). Defining critical thinking. Retrieved from http://www.criticalthinking.org/pages/defining-critical-thinking/410.3. Albitz, R. S. (2007). The what and who of information literacy and critical thinking. portal: Libraries and the Academy, 7(1), 97-109.
recognize need
accesssearch for/gather
evaluateuse/apply
synthesize
analyzeconceptualize
SET OF SKILLS3
COGNITIVE PROCESSES3
4
Impetus for our research:• Important 21st century skills• ABET criterion 3i (lifelong learning)• Engineering decision-making
Primary focus of our research:• How do we assess information literacy and critical
thinking?
Impetus for Our Research
InfoSkills, 2013
Project Background
5
InfoSEAD framework for item generation1
• Seeking – Gathering and access• Evaluation – Assess quality, credibility, etc.• Application – Using information to satisfy a need• Documentation – Citation and in-text referencing
Two Scenarios 18 total items• 16 multiple choice; 2 select all that apply
About the CELT(v2.1)
InfoSkills, 2013
Project Background
1. Wertz, R. E. H., Purzer, S., Fosmire, M. J., & Cardella, M. E. (in press). Assessing information literacy skills demonstrated in an engineering design task. Journal of Engineering Education.
6
Research question:• How well does the CELT (v2.1) measure critical
thinking?
Research method:• Pearson’s correlational analysis between CELT
instrument and the Critical Thinking Assessment Test (CAT) developed by Tennessee Technology University.1
InfoSkills, 2013
Correlational StudyResearch Methods
1. Center for Assessment and Improvement of Learning at Tennessee Technological University. (2010). CAT technical information, from http://www.tntech.edu/files/cat/reports/CAT_Technical_Information_V7.pdf
7
N = 44 first-year engineering students CELT reliability measures• KR-20 = 0.671
• N = 188 first-year students (Fall 2012)• Based on 16 MC questions
CAT reliability measures • 15 constructed response items; scoring reliability 0.822
• Cronbach’s alpha 0.702
InfoSkills, 2013
Population and SettingResearch Methods
1. Wertz, R. E. H., Saragih, A., Fosmire, M. J., & Purzer, S. (2013). An Evaluation of the Critical Engineering Literacy Test (CELT) Instrument through Item Analysis and Comparison to the Critical Assessment Test (CAT). Paper presented at the 2013 Illinois/Indiana - ASEE Section Conference, Angola, IN.
2. Center for Assessment and Improvement of Learning at Tennessee Technological University. (2010). CAT technical information, from http://www.tntech.edu/files/cat/reports/CAT_Technical_Information_V7.pdf
8
Overall Correlational Analysis:• CELT total score was positively related the CAT
total score (r = 0.47, p <0.01)
Itemized Correlational Analysis:• Items 5, 11, 16, 17 had positive associations with
the CAT total score
InfoSkills, 2013
CELT ResultsProject Results
9InfoSkills, 2013
CELT ResultsProject Results
Which of the following comparisons of omega3/omega6 ratios is most relevant in determining whether GE fish is equivalent to
its non-GE counterpart?
Item 11(r = 0.32, p <0.05)
What would help the review panel validate the data presented?Item 16
(r = 0.37, p <0.05)
Where would you likely find authoritative information on a typical omega-3 levels of salmon?
Item 17(r = 0.35, p <0.05)
[Evaluate] which of the citations is incorrect or incomplete?Item 5
(r = 0.51, p <0.01)
10
14 of 18 CELT items positively correlated to some portion of CAT
Correlation between the CELT and CAT total scores is moderately strong (r = 0.47, p <0.01)
CELT items 5, 11, 16, and 17 provide insight into a possible subset of skills where information literacy skills and critical thinking overlap.
InfoSkills, 2013
General ConclusionsProject Conclusions
11
What are the common elements in CELT items 5, 11, 16, and 17 that made them correlate to CAT?• Expand sample• Item-to-item analysis• Verbal protocol
InfoSkills, 2013
Future StudyProject Conclusions
12
Dr. Senay Purzer – [email protected]
Mr. Michael Fosmire – [email protected]
Ms. Amy Van Epps – [email protected]
Ms. Ruth Wertz – [email protected], 2013
THANK YOU!
Contact Information