Post on 23-Jan-2016




0 download

Embed Size (px)


ASSESSING STUDENTS STATISTICAL REASONING. Robert delMas (Univ. of Minnesota, USA) Ann Ooms (Kingston College, UK) Joan Garfield (Univ. of Minnesota, USA) Beth Chance (Cal Poly State Univ., USA). Statistical Literacy, Reasoning and Thinking. - PowerPoint PPT Presentation


<ul><li><p>ASSESSING STUDENTS STATISTICAL REASONINGRobert delMas (Univ. of Minnesota, USA)Ann Ooms (Kingston College, UK) Joan Garfield (Univ. of Minnesota, USA)Beth Chance (Cal Poly State Univ., USA)</p></li><li><p></p></li><li><p>Statistical Literacy, Reasoning and ThinkingStatistical literacyinvolves understanding and using the basic language and tools of statistics: knowing what statistical terms mean, understanding the use of statistical symbols, and recognizing and being able to interpret representations of data. Statistical reasoningis the way people reason with statistical ideas and make sense of statistical information. Reasoning means understanding and being able to explain statistical processes, and being able to fully interpret statistical results. Statistical thinkinginvolves an understanding of why and how statistical investigations are conducted. This includes recognizing and understanding:The entire investigative process.How models are used to simulate random phenomena.How data are produced to estimate probabilities.How, when, and why existing inferential tools can be used.Utilizing the context of a problem to plan and evaluate investigations.</p></li><li><p>ARTIST Resources Page - Alternative Assessments - Projects</p></li><li><p>Alternative Assessments - References on Student Projects</p></li><li><p>Alternative Assessments - Examples of Student Projects</p></li><li><p>ARTIST Resource Page - References on Assessment</p></li><li><p>References on Assessment in Statistics Education</p></li><li><p>ARTIST Topic Tests</p><p>There are 11 scales, each consisting of 7-15 multiple-choice items, that can be administered online. Reports of student performance can be downloaded. Our goal was to develop high quality, valid and reliable scales that can be used for a variety of purposes (e.g., research, evaluation, review, or self-assessment).</p><p>TOPICSData Collection (data types, types of study, study design)Data Representation (choose appropriate graphs, interpret graphs)Measures of Center (estimate, when to use, interpret, properties)Measures of Spread (estimate, when to use, interpret, properties)Normal Distribution (characteristics, empirical rule, areas under the curve)Probability (interpret, independence, relative frequency, simulation)Bivariate Quantitative Data (scatterplots, correlation, descriptive and inferential methods, outliers, diagnostics, influential observations)Bivariate Categorical Data (two-way tables and chi-square test, association)Sampling Distributions (types of samples, sample variability, sampling distributions, Central Limit Theorem)Confidence Intervals (interpret, confidence level, standard error, margin of error)Tests of Significance (hypothesis statements, p-values, Type I and II error, statistical and practical significance)</p></li><li><p>ARTIST Test Report</p></li><li><p>Comprehensive Assessment of Outcomes in Statistics (CAOS)</p><p>Forty item test that can be administered as an online test to evaluate the attainment of desired student outcomes. CAOS items are designed to represent the big ideas and the types of, literacy and reasoning skills deemed important for all students across first courses in statistics. Unifying focus is on reasoning about variability: in univariate and bivariate distributions, in comparing groups, in samples, and when making estimates and inferences.Not intended to be used exclusively as a final exam or as the sole assessment to assign student grades. CAOS can provide very informative feedback to instructors about what students have learned and not learned in an introductory statistics course (e.g., administered as pretest and posttest). An analysis of responses from a large national sample of students was presented at AERA 2006 (Click the link for ARTIST Publications, Presentations, and Workshops to download a copy of the paper).</p></li><li><p>Accessing the ARTIST Item Database</p></li><li><p>Assessment Builder: Conducting a Search for Items</p></li><li><p>Assessment Builder: Search Results</p></li><li><p>Assessment Builder: Downloading a Test</p></li><li><p>Evaluation of ARTIST</p><p>Evaluations consisted ofSurvey of 98 instructors who used ARTIST resources89 instructors who had not used ARTIST resources Observations of 5 instructors as they used the ARTIST websiteInterviews with 7 non-users and 7 frequent users</p><p>ResultsContent is of high quality and navigation was user-friendlyNon-users were not aware of ARTIST or of its purposeARTIST assessment items were judged to be of the same or higher quality than items from other sourcesFrequent users of ARTIST resources indicated that they are starting to rethink instruction in order to have a greater impact on students statistical reasoning and thinking.</p></li><li><p>Future Plans</p><p>Create large data sets, along with variable descriptions, that can be downloaded for use in open-ended assessmentsProvide an online version of the Student Attitudes Towards Statistics (SATS) survey (Shau, 1994) that assesses six components of attitudes toward statistics (affect, cognitive competence, value, difficulty, interest, and effort). Develop a Statistical Thinking Assessment (STA) to assess students ability to select appropriate procedures, check conditions and assumptions, and draw correct conclusions.Develop a Statistics Teaching Inventory (STI) to gather information on teachers own beliefs about teaching, their instructional practices, and the constraints in which they teach (school and student variables). Collaborate with CAUSE (Consortium for the Advancement of Undergraduate Statistics Education) to use ARTIST assessments and resources in multi-institution statistics education research projects.</p></li><li><p>ARTIST Website contact the ARTIST team with any comments and suggestions you have regarding this presentation, or any questions you have about the materials at the ARTIST website.Thank you for your attention.</p><p>The ARTIST website can be accessed from the URL at the top of the page. This URL will be displayed again at the end of this talk. Read the GOAL of the ARTIST project. The next slide displays definitions of Statistical Literacy, Reasoning, and Thinking used to categorize materials and resources on the ARTIST website.You can just read or summarize the definitions.The next slide shows the ARTIST Resources page.This slide shows the RESOURCES page of the ARTIST website. Clicking the link for PROJECTS takes you to a page that lists references on using projects in a statistics course. The list is displayed on the next slide.Links are provided for all articles that are online, and the ARTIST project has obtained permission so that some articles can be downloaded as PDF documents.This page also provides links to examples of projects, which is displayed on the next slide.You can download PDF documents that provide the instructions for course projects created by several statistics instructors. The PDF documents for John Holcombs projects are quite rich in that they provide links to data sets in addition to the instructions for each project.Another resource on the ARTIST website are references to articles on conducting assessment in statistics.This slide shows one of the two links that can be clicked to view a list of the references.The next slide shows part of the reference list.This is a small subset of the references. Similar to the PROJECTS page, links are provided for online articles, and permission has been obtained to download some of the articles as PDF documents.</p><p>The next slide provides information about online tests developed by the ARTIST project.The next slide provides an example of a Test Report.Test Reports can be retrieved after all students have completed an online assessment. This slide shows information for the first item on the DATA REPRESENTATION test. The percent of students who chose each answer option is displayed. The percent for the correct response is in Boldface type.A second report that displays each students percent correct, and that can be opened with Excel, is also sent to the instructor.The next slide provides information on the CAOS test (Comprehensive Assessment of Outcomes in Statistics).Similar to the topic tests, test reports can be retrieved after all students have completed the CAOS test.The next slide shows how to access the ARTIST online Item Database.The ARTIST website also has an online Item Database of over 1100 items. This slide shows the link to click to access the Item Database.A tool called the Assessment Builder is used to search and retrieve items from the database. Each item is coded for item format, the topic it represents, and the type of learning outcome that is assessed. The Assessment Builder lets the user select choices for each of these three item characteristics. A subset of items is displayed when the search criteria are submitted.Here, the search resulted in 152 items that met the search criteria. All 152 items are displayed on a single, scrollable page. Information such as the topic and learning outcome is displayed for each item. The user selects an item by clicking the INCLUDE checkbox for an item.All items that are selected by the user are collected into a file. The user can add more items to the collection at any time (by clicking the VIEW button for a collection). A document of the items in Rich Text Format (RTF) can be downloaded and edited by the user (by clicking the GET button).An evaluation of the ARTIST website was conducted Ann Ooms, who was a graduate research assistant working on the project at the time.</p></li></ul>


View more >