Assessing (and Improving) Critical Thinking and Quantitative Reasoning in MA 103

Download Assessing (and Improving) Critical Thinking and Quantitative Reasoning in  MA 103

Post on 14-Jan-2016

21 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

Assessing (and Improving) Critical Thinking and Quantitative Reasoning in MA 103. Gerald Kruse and David Drews Juniata College Huntingdon, PA kruse@juniata.edu drews@juniata.edu. MA 103, Quantitative Methods, aka QM - PowerPoint PPT Presentation

TRANSCRIPT

<ul><li><p>Gerald Kruse and David DrewsJuniata CollegeHuntingdon, PA</p><p>kruse@juniata.edudrews@juniata.edu</p></li><li><p>MA 103, Quantitative Methods, aka QMMathematics 103 prepares students to be quantitatively literate citizens in today's world. By learning to think critically about quantitative issues, students will be able to make responsible decisions in their daily lives. as well as to present quantitative output and verbal arguments. </p></li><li><p>MA 103, Quantitative Methods, aka QMMathematics 103 prepares students to be quantitatively literate citizens in today's world. By learning to think critically about quantitative issues, students will be able to make responsible decisions in their daily lives. as well as to present quantitative output and verbal arguments. Three Projects during the semester - began using CLA performance tasks Spring 2009 - authentic and open-ended </p></li><li><p>MA 103, Quantitative Methods, aka QMMathematics 103 prepares students to be quantitatively literate citizens in today's world. By learning to think critically about quantitative issues, students will be able to make responsible decisions in their daily lives. as well as to present quantitative output and verbal arguments. Three Projects during the semester - began using CLA performance tasks Spring 2009 - authentic and open-ended Pre and Post Assessment (Skills and Attitudes) - 55 min exam given on the first and last class of semester - Fall 2009 transition from math skills to CLA performance task </p></li><li><p>The Collegiate Learning Assessment (CLA) provides one possible example of an assessment that fits a situated notion of QR. Richard Shavelson, Reflections on Quantitative Reasoning, an Assessment Perspective, In B.L. Madison &amp; L.A. Steen (Eds.), Calculation vs. context: Quantitative literacy and its implications for teacher education. MAA. </p></li><li><p>The Collegiate Learning Assessment (CLA) provides one possible example of an assessment that fits a situated notion of QR. Richard Shavelson, Reflections on Quantitative Reasoning, an Assessment Perspective, In B.L. Madison &amp; L.A. Steen (Eds.), Calculation vs. context: Quantitative literacy and its implications for teacher education. MAA. </p><p>Being quantitatively literate is being able to think and reason quantitatively when the situation so demands </p></li><li><p>Evaluating Evidence</p><p>Higher-Order Skills Assessed</p></li><li><p>Evaluating Evidence</p><p>Analysis / Synthesis / Conclusion</p><p>Higher-Order Skills Assessed</p></li><li><p>Evaluating Evidence</p><p>Analysis / Synthesis / Conclusion</p><p>Presenting / creating evidenceHigher-Order Skills Assessed</p></li><li><p>Evaluating Evidence</p><p>Analysis / Synthesis / Conclusion</p><p>Presenting / creating evidence Acknowledging alternatives to THEIR conclusionHigher-Order Skills Assessed</p></li><li><p>Evaluating Evidence</p><p>Analysis / Synthesis / Conclusion</p><p>Presenting / creating evidence Acknowledging alternatives to THEIR conclusion CompletenessHigher-Order Skills Assessed</p></li><li><p>Experimental Design</p></li><li><p>Explicit scoring guidelines (based on rubric) established.Fall 2009, Version 1.o</p></li><li><p>Explicit scoring guidelines (based on rubric) established.</p><p>Scoring guidelines gave good reliability.Fall 2009, Version 1.o</p></li><li><p>Explicit scoring guidelines (based on rubric) established.</p><p>Scoring guidelines gave good reliability.</p><p>Encouraging, but not statistically significant, results indicated that students in the section with performance task based projects showed more improvement in critical thinking skills.Fall 2009, Version 1.o</p></li><li><p>Explicit scoring guidelines (based on rubric) established.</p><p>Scoring guidelines gave good reliability.</p><p>Encouraging, but not statistically significant, results indicated that students in the section with performance task based projects showed more improvement in critical thinking skills.</p><p>Use results to prepare for next round of assessment in Spring 2011.Fall 2009, Version 1.o</p></li><li><p>The content of the course Fall 2009 vs. Spring 2011 remained the same, as did 95% of the classroom experience, but the course was reframed with a focus on quantitative reasoning (q. r.) and critical thinking (c. t.)Modifications for V2.0How was the student experience different in Spring 2011 vs. Fall 2009?</p></li><li><p>The content of the course Fall 2009 vs. Spring 2011 remained the same, as did 95% of the classroom experience, but the course was reframed with a focus on quantitative reasoning (q. r.) and critical thinking (c. t.) - syllabus - assignments - opportunities during lecture - salt and pepperModifications for V2.0How was the student experience different in Spring 2011 vs. Fall 2009?</p></li><li><p>The content of the course Fall 2009 vs. Spring 2011 remained the same, as did 95% of the classroom experience, but the course was reframed with a focus on quantitative reasoning (q. r.) and critical thinking (c. t.) - syllabus - assignments - opportunities during lecture - salt and pepper The pre/post assessment was modified: - better linkage with specific learning outcomes - more open-ended scenario - names - one promptModifications for V2.0How was the student experience different in Spring 2011 vs. Fall 2009?</p></li><li><p>Present the idea of a rubric: - objective assessment, trust - familiarize with elements - create one for chips </p><p>Modifications for V2.0 continuedHow was the student experience different in Spring 2011 vs. Fall 2009?</p></li><li><p>Scoring Guidelines</p></li><li><p>Present the idea of a rubric: - objective assessment, trust - familiarize with elements - create one for chips Improved feedback for projects completed during the semester: - students used guideline to score their work - compared this to my scoring - general trends discussed with entire class - scheduled time to meet for specific feedback Each of the three projects emphasized different quantitative content (what we were doing at the time) as well as different categories in the rubric </p><p>Modifications for V2.0 continuedHow was the student experience different in Spring 2011 vs. Fall 2009?</p></li><li><p>Results for V2.0</p><p>Total ScorePrePost% change% of possible improvementmy section9.4613.2540.0632.84other section8.29.2412.688.13</p><p>Evaluating EvidencePrePost% change% of possible improvementmy section1.682.548.8118.98other section0.961.2429.175.56</p></li><li><p>Results for V2.0</p><p>Analysis, Synthesis, ConclusionPrePost% change% of possible improvementmy section2.86439.8636.31other section2.63.2826.1520.00</p><p>Presenting, Creating EvidencePrePost% change% of possible improvementmy section0.381163.1623.66other section0.160.6275.0015.49</p></li><li><p>Results for V2.0</p><p>Acknowledging Alternatives to Their ConclusionPrePost% change% of possible improvementmy section0.710.9635.2110.92other section0.80.44-45.00-16.36</p><p>CompletenessPrePost% change% of possible improve-mentmy section1.892.2820.6335.14other section1.841.840.000.00</p></li><li><p>Rubric ReliabilityDimension% spot on% +/- 1Evaluation56.740.096.7Anal/Synth/Concl58.634.593.1Create 73.326.7100.0Alternatives 51.737.998.7Completeness 62.134.596.6Total Score39.235.775.0</p></li><li><p>Rubric ReliabilityCorrelation of Total ScoresPearson correlation of D and J = 0.927 P-Value = 0.000</p></li><li><p>Critical thinking involves evaluating/making good argumentsMaking good arguments involves. . .</p><p>Clearly stating a conclusionEvaluating and selecting evidenceCreating links between evidence and conclusion</p><p>We consider quantitative reasoning as critical thinking involving numbers/data</p></li><li><p>MA 103, Quantitative Methodsat Juniata CollegeJuniata has a Quantitative Skills Requirement, Q = QM + QS MA 103, Quantitative Methods, is offered for students who dont fulfill the Q in their POE Three Projects during the semester - began using CLA performance tasks Spring 2009 - authentic and open-ended</p></li><li><p>Making good arguments involves. . .</p><p>Clearly stating a conclusionEvaluating and selecting evidenceCreating links between evidence and conclusion</p><p>We can then consider quantitative reasoning as critical thinking involving numbers/data</p><p>Critical thinking involves evaluating/making good arguments</p></li><li><p>Identifying relevant evidence, evaluating evidence credibility, reliability, relevanceNot Attempted (0)Emerging (1,2) Mentions one or two documents, with: - No or wrong evaluation on both (1) - cursory-to-OK eval on document C, flawed on other (2)Developing (3,4) Mentions two documents (one must be C), with: - cursory-to-OK evaluation on both (3) - good evaluation on both (4)Mastering (5,6) Evaluation of C is good, and evaluates two other doc with: - acceptable evaluations (5) - good evaluations (6) Evaluating Evidence Category on Rubric</p></li><li><p>Identifying relevant evidence, evaluating evidence credibility, reliability, relevanceNot Attempted (0)Emerging (1,2) Incorrectly implies , or states directly, agrees that banning aspartame would improve the health of the states citizens with: - no evidence (1) - evidence (2)Developing (3,4) Implies, or directly disagrees, with Sauer, noting inconsistency of claim with data in doc C but reason is: - inaccurate/unclear or incomplete(3) - good (4)Mastering (5,6) - Says C doesnt support claim and is clear about reason and uses F reasonably well (5) - Satisfactorily uses conditional probability when discussing relationship between headaches and aspartame usage (6) Analysis/Synthesis/Conclusion Category on Rubric</p></li><li><p>Sen. Nathan Dulce is running for re-election vs. Pat SauerProposed bill to ban aspartame, an artificial sweetener, from being added to any soft drink or food product, Dulce opposes, Sauer approves.Pat Sauer made two arguments during a recent TV interview:(1) Strong correlation between the number of people who consume aspartame and headaches, so,banning aspartame would improve the health of the states citizens. (2)Aspartame should be banned and replaced with sucralose. Pat Sauer supported this argument by referring to a news release.</p><p>Performance Task Scenario for Pre and Post-Assessment</p></li><li><p>Re-designed MA 103 during 2007 sabbatical, making more use of active-learning and one day each for pre and post assessment.</p><p>Attended CLA in the Classroom Performance Task Academy in December 2008, and immediately began working to incorporate performance tasks in MA 103. Presented my first task to Juniatas SoTL group (bi-weekly Brown Bag lunches) and director suggested I expand, and use performance tasks as the pre/post assessment for students qr skills.</p><p>Note the similarities*Re-designed MA 103 during 2007 sabbatical, making more use of active-learning and one day each for pre and post assessment.</p><p>Attended CLA in the Classroom Performance Task Academy in December 2008, and immediately began working to incorporate performance tasks in MA 103. Presented my first task to Juniatas SoTL group (bi-weekly Brown Bag lunches) and director suggested I expand, and use performance tasks as the pre/post assessment for students qr skills.</p><p>Note the similarities*Re-designed MA 103 during 2007 sabbatical, making more use of active-learning and one day each for pre and post assessment.</p><p>Attended CLA in the Classroom Performance Task Academy in December 2008, and immediately began working to incorporate performance tasks in MA 103. Presented my first task to Juniatas SoTL group (bi-weekly Brown Bag lunches) and director suggested I expand, and use performance tasks as the pre/post assessment for students qr skills.</p><p>Note the similarities*Operationalize our definition of quantitative reasoning linkage to ciritical thinking*Operationalize our definition of quantitative reasoning linkage to ciritical thinking*Inspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classInspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classHigher-order skills (c.t., problem-solving, communicating) more general than c.t., but will use c.t. and q.r. here please forgive</p><p>*Inspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classInspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classHigher-order skills (c.t., problem-solving, communicating) more general than c.t., but will use c.t. and q.r. here please forgive</p><p>*Inspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classInspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classHigher-order skills (c.t., problem-solving, communicating) more general than c.t., but will use c.t. and q.r. here please forgive</p><p>*Inspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classInspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classHigher-order skills (c.t., problem-solving, communicating) more general than c.t., but will use c.t. and q.r. here please forgive</p><p>*Inspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classInspired by CLA, and functionally, our defn. of critical thinking quantitative reasoning = critical thinking in an MA classHigher-order skills (c.t., problem-solving, communicating) more general than c.t., but will use c.t. and q.r. here please forgive</p><p>*Based on Solomons Four-Group Design, able to do it thanks to my helpful and flexible colleagues in the MA dept.*********Rubric (general categories) shared initially. Scoring guideline, very specific, returned with response.**We operationalized quantitative reasoning as critical thinking w/numbers</p><p>Have to be careful, Critical Thinking might be used as Higher-Order skills actually a subset. Analytic reasoning and problem solving included.*Note previous slide with categories of rubric (which we used as functional defn of critical thinking) heres one category, for one question. We had comparable breakout for each category in each question.***</p></li></ul>

Recommended

View more >