nh alps mathematics scoring training july 2012. training outline introductions general scoring...

NH ALPs Mathematics Scoring Training July 2012

Training Outline Introductions General Scoring Guidelines Mathematics Learning Progressions Levels of Scoring Clarifications Scoring Forms, Template, Evidence Video Logistics Question & Answer Period

Introductions Department of Education Gaye Fedorchak NHDOE Director of Alternate Assessments & Access Services Allyson Vignola Reading and Writing Scoring Leader and Content Expert Marie Cote Mathematics and Science Scoring Leader and Content Expert Measured Progress Staff Tina Haley - Program Manager Sarah Greene Program Assistant Sharman Lyons Program Support Susan Izard Measured Progress SPED Director

General Scoring Guidelines Score only what you see in the portfolio Avoid biases - (See: Scoring Manual, Gen. Guidelines Detail) Score each task entry independently, separately Respect student confidentiality Test breach considerations Keep portfolio in order it was submitted Do not write or leave notes in portfolios

1. What are we assessing? The NH Curriculum Frameworks Its the same for ALL. The Learning Progressions Standards as Written Access-based definitions of the standards as written NH ALPs: 3 Big Concepts 2. How are we observing what our students know and are able to do? An Effective Communication System Access to Academic Learning & Performance Student as author of own work! Active agent

3. What is the Evidence of performance? Constructivism: Authentic Tasks 3 Big Concepts, continued... Making sense of mathematics ideas... to reason & solve problems. Show the student actively engaging with ideas about quantity, numbers, equality, patterns, change, measurement, & data, Show us your student

Learning Progressions describe how we expect students knowledge and skills in one content area will develop over time. Each learning progression represents a cluster of knowledge and skills (GLEs) that develop together based on years of professional experience and research. Learning progressions contain many sequential steps that are needed for certain ideas and concepts to develop and build upon one another before full, more comprehensive understanding of the topic is gained. Sometimes students grow directly upward in learning new, more challenging content, but sometimes they grow sideways learning new content at the same level of challenge before they are ready to move upward toward higher skill clusters. What is a Learning Progression?

What we expect can be wrong. Students can (and often DO) surprise us. NH Alternate Learning Progressions are built in a way that allows us to observe our students as they grow We can map current levels of performance based on NH Curriculum Standards AND we can map year-to-year growth in academic skills, last years base is this years starting point. Learning progressions offer a clear, concise, visual way of presenting information to families and to educational teams when making instructional planning decisions. Learning Progressions continued

4 Progressions 1. Understanding Rational Numbers 2. Solving Problems with Operations 3. Equality 4. Patterns and Change Math Learning Progressions

How to Read the Progressions The Mathematics progressions are meant to be read as one complete sentence. When you read the Mathematics progressions in this way, you find that every Mathematics sub-level becomes one defined assessment task.

How to Read the Progressions continued... In this case there are 5 scorable units (A1, A2, A3, A4, B1). Scorable units within the Mathematics progressions are always indicated by a number (1, 2, 3, etc..) or alpha numeric code (A1, B2, C1, etc.).

Scorable units A1, A2, A3, A4 need to be demonstrated using whole numbers 0 20 as indicated by letter A in the first column. Scorable unit B1 is linked to letter B (1/2 as fair share) also stated in the first column, so the student would have to demonstrate the concept of fair share using a quantity between 1 20. How to Read the Progressions continued...

S TUDENT AS A UTHOR T HE C RITICAL R OLE OF A CCESS TO L EARNING, C OMMUNICATION & P ERFORMANCE What does independent performance mean for an alternately assessed student? Directive prompts violate the learning construct that we are trying to measure. No credit can be earned when directive prompts are used. Supportive prompts allow students to show personal authorship of standards-based tasks. Students can earn credit when supportive prompts are used because the constructs being tested are honored and demonstrated.

Two Levels of Score Review 1. Task Level Review Content Fidelity 2. Item Level Review

Task Level Review Task Level Review determines whether or not the student performance sample can advance to the Item Level Score Review, based on sufficiency of match to the required academic content standards (Content Fidelity). If a student performance work sample is found not to sufficiently match the required content, the sample does not pass task review.

TASK LEVEL REVIEW During this review, a scorer must ask the following two (2) questions: 1. Is the student attempting to apply knowledge or skills that are clearly within the content area? AND 2. Is the type of skills and knowledge clearly related to the standards of the progression?

Task Level Review continued If the answer to both of these questions is YES, then sufficient evidence of content knowledge exists and scoring may continue to the Item Level Review. If the answer to either or both of these questions is NO, then sufficient evidence of content knowledge does not exist and scoring may not continue. If student performance does not meet this criterion, then the task sample cannot continue to the Item Level Review and the applicable comment code must be documented.

Task Level Review Comment Codes 1. Student method of communication could not be determined. Content Assessment Cover Sheet information was not submitted and Summary Description of Student Access to Learning and Performance was not submitted. 2. Challenge Level and/or Sub-Level was not identified. 3. Performance task evidence provided, as shown by student work sample, does not match content progression. (i.e.: Required Reading text type was not submitted, required Writing genre not submitted, required Mathematics progression not submitted, or required Science domain not submitted). 4. Copy or summary of text was not submitted for Response to Text writing sample. 5. The Science column could not be determined (last letter in the Science Content ALPs code).

Item Level Review An item is referred to as a scorable unit within a content standard. There may be one or multiple scorable units within a content standard. Each scorable unit is considered to be worth one ( 1 ) point.

ITEM LEVEL REVIEW 1. Is the performance criteria met as stated in the standard (or in the clarifications)? AND 2. Is the student clearly the author of this academic performance? If the answer to both of these questions is YES, then one ( 1 ) point is credited. If the answer to either or both of these questions is NO, then no point is credited. If student did not earn credit for any scorable units within the chosen standard then it is considered Not Demonstrated (ND) and the applicable comment code must be documented.

Item Level Review Comment Codes 6. Quality of video impacted score (i.e., sound quality, camera angle, clarity) 7.Video exceeded maximum allowable length. Score review ended at maximum time. 8.Student did not demonstrate standard(s), as written, in identified progression or sub-progression. 9. Directive prompts prevented observation of student authorship. 10. Directive prompts impacted observation of student authorship. 11. Science Process Skill was not indicated. 12. Challenge level of Science Process Skill did not match Science content challenge level in this entry. 13. Science Process Skill was duplicated; only 1 Process Skill qualified for score review.

Question 1 Is the Performance Criterion Met? Mathematics success at the item level is defined as one clear demonstration of a mathematics scorable unit - as written that also includes clear evidence of student authorship. One clear demonstration means that student performance must meet the scoring criterion defined by the scorable unit(s) with the standard- as written.

Question 2 Is the student clearly the author of this academic performance? Credit or non-credit of the scorable unit is also determined by the demonstration of student authorship. There must be compelling evidence that the student is clearly the author, as demonstrated in his/her performance of the scorable units (or item responses) within the standard.

The following questions may serve as guidance to a scorer judgment in making the determination regarding the presence or absence of student authorship: Does evidence of student access to learning, communication, and performance show student engaging in the task as a willful agent who is effectively communicating his/her decision-making to indicate personal choice and authorship of performance of this scorable unit/item? Did the teacher provide supportive prompts and not directive prompts? (See Training Manual) Is there a biasing response context? Student as Author

Student As Author continued Is the student able to communicate choices effectively? If a student has a severe motor impairment and some responses appear to be a result of the clear willful intention of the student but some responses seem possibly accidental or random, is evidence of student response control sufficiently clear to make authorship compelling? (Requires scorer judgment)

Terminology Used in Mathematics Progressions The words used in the content standard as written are very important. In addition to the wording interpretation guide below, the definitions provided in NH ALPs Master Glossary provide an additional reference to help clarify the meaning of the words used in the standard. Understanding the specificity of the Mathematics scorable units as they are written. Multiple requires demonstration of three (3) or more unless standard permits less. Quantity requires demonstration of how many or an amount. Magnitude means demonstration of quantity NOT size. Or requires demonstration of either one or the other connected by the word or (minimum of one is required).

Terminology continued And or the use of & when shown within the Stem concept or when shown between standards (scorable units) is meant to be read as a complete mathematical sentence. Students should demonstrate as many standards (scorable units) as possible within the mathematical sentence. And or the use of & within a scorable unit requires demonstration of both elements connected by the word and for credit to be earned.

Terminology continued / is treated as an indication that each choice separated by the / must be offered to the child. (i.e. identify objects which are bigger, /smaller, /about the same. Point is to see child making the comparison and choice among all of these specified alternatives. The / in Patterns and Change, Emergent, sub-levels 1, 2, & 3 will be an exception. For these three sub-levels the / will mean or as it refers to sounds/movement and colors/textures. Commas in a series with no other connecting words are treated as an OR statement, as follows: bigger, smaller, same- the commas will be treated as bigger or smaller or same Commas in a series with other connecting words are treated as the connecting word implies, as follows: bigger, smaller, and same- the commas will be treated as bigger and smaller and same

Clarifications Compare and Order Making Estimates Number Sentence and Expression (Expression vs. Equation) Verbal Explanation Finding a Missing Element Repeating (3 iterations) and Growing Patterns Models (Objects, Materials, Models) Tables, Graphs, Charts, Etc Number Parameters at different levels. Area, Set, and Linear Models Ordering by Attributes

Compare and Order Understanding Rational Numbers Comparing Numbers: Comparing numbers, a and b, means to determine if a is less than b, if a is greater than b, or if a is equal to b. At level 17, students are using equality and inequality symbols to express the comparison (=, ,, ) Compare and Order: These are very solid examples. Teacher stimulus presents an unordered group of numbers; credit given if the student puts them in order and describes why a comes before b. Teacher stimulus presents an unordered group of numbers; credit given if the student puts them in order smallest to largest or largest to smallest. Teacher stimulus presents an unordered set of numbers; credit given if the student takes just two numbers and finds bigger/smaller (compares), and then puts them in order (sequence). Teacher stimulus presents an unordered group of numbers; credit given if the student puts them in order and demonstrates understanding of one greater than the stimulus sample.

Making Estimates Operations Makes estimates: Credit is given if student demonstrates scorable unit A and B congruent or independent of each other. Teacher stimulus may present A and B together using the same objects OR present A and B separate using two different objects on two different days. Students who are nonverbal may be given a choice. Teacher stimulus presents 7 cubes. Teacher shows student number cards, one with the number 10 and one with the number 20; credit given if the student demonstrates estimation by choosing (per individual means of communication) the number card with the number 10.

Estimation continued Method of Estimation: An estimate is an approximation or rough calculation. Estimation requires good intuition about numbers, good understanding of problem situations, and a flexible repertoire of techniques. Methods include: Rounding: replacing a number or numbers in a computation with the closest multiple of 10, 100, 1000, etc. to make the computation easier to do Substituting compatible numbers: replacing some or all of the numbers in a computation with numbers that are easier to work with Front end estimation: calculating with the leftmost, or front- end, digit of each number as if the remaining digits were all zeros

Number Sentence and Expression (Expression vs. Equation) Number Sentence: A number sentence is an equation or inequality such as 10=8+2, 12 n = 4, or 14 > 5. A number sentence has a left hand side, a relation symbol, and a right hand side. Each side of a number sentence is a numerical expression. Number sentences can be true, false, or neither true nor false. A number sentence that is neither true nor false is called an open sentence (see Equality Progression).

Verbal Explanation Equality Credit is given for standard A2 describe/describing, if student provides a verbal response, but not necessarily a spoken response, per individual means of communication. Starting at standard B3A4, a verbal explanation is required, (this permits other communication methods consistent with definition of explanation in glossary) If student provides a verbal explanation using words, they also receive credit for standard A4.

Finding a Missing Element Patterns & Change Starting at standard B1A2 - Finds a missing element - Credit is given if student identifies a missing element from some pattern. Missing element can be at the beginning, middle, or end of the pattern (i.e.: 2 2 3 1 2 2 3 _)

Repeating and Growing Patterns Patterns & Change Concepts B3 & B4 Growing Numeric Patterns- Credit is given if student identifies an extension of a pattern, or an extension of a decreasing pattern. Increasing is defined as growing or extending, as related to these standards Decreasing is defined as extending a decreasing linear pattern, as related to these standards

Models (Objects, Materials, Models) Equality Models: At the Emergent level only, models will be synonymous with demonstrate. Using Models: A student uses models when he/she uses manipulatives, pictures, diagrams, graphs, or mathematical symbolism to simulate real-life situations and understand quantitative relationships.

Tables, Graphs, Charts, Etc Patterns & Change Tables: Numbers or quantities arranged in labeled rows and/or columns. Graphs: A diagram of values, usually shown as lines or bars with consistent and appropriate scales. Each dimension of the graph should have its own scale. Graphed values can also be represented in pie chart, box and whiskers data plot, scatter plot, bubble plot, or in other graphic forms. Graphed values are diagrammed along scales that are labeled, consistent, and appropriate to the data being presented. If there is only one type of value (variable), the graph is on a single number line (presented in one dimension). If there are two variables, the graph is on the coordinate plane. (Shown in two dimensions). If there are three variables, the graph is shown in three-dimensional coordinates. In general, for n variables, the graph is in n dimensions. Each dimension (or axis) of a graph should have its own labeled, consistent, and appropriate scale.

Charts, Graphs, Tables continued If the scorable unit, as written, requires a graph, but only a table is presented, then no credit can be given. If the scorable unit, as written specifies a certain type of graph, then that type of graph must be shown to earn credit. If there is a graph in the student performance evidence, per condition of the scorable unit, and if the graph shows the coordinates without the values labeled, credit cannot be given. The graphed values must be represented & appropriately labeled. One way the graphed coordinate values may be represented (but not the only way) could be in a table format in addition to (paired with) the graph. If the scorable unit, as written specifies a table, but only a graph is presented, then no credit can be given.

Number Parameters at Different Levels When number parameters are given, students must demonstrate numbers beyond the previous level in order to receive credit for scorable units. Rational Numbers

Area, Set, and Linear Models Area Models: Students may use many types of manipulatives and/or pictures to develop area models (e.g., pattern blocks, circular fraction regions, pizzas, geoboards, spinners). An area model can be used to represent part to whole relationships for fractions, decimals, and percents. The entire model may represent the whole where the model is divided into parts of equal area, the model given may represent a part where the whole is to be determined, or the model given may represent a part where another part is to be determined. Shade one square, partitioned vertically, to represent 3/8 (shown below in pink): Shade another square, partitioned horizontally, to represent 2/3 (shown below in blue): Superimpose the two squares. The product is the area that is double-shaded (shown below in purple):

Models continued Linear/Measurement Models: Linear models include number lines, scales (temperature), and linear measurements. Linear models can be used in a similar fashion as area and set models. Students use the number line model found on rulers or divide fraction strips into the appropriate sections by length. Teachers may use any of these manipulatives to develop the measurement or linear model: number line rulers linking cube trains Cuisenaire rods fraction bars fraction strips

Models continued Set Models: Since a set is a collection of objects, demonstrating understanding of part to whole relationship in a set model means to identify a fractional part of a set, or identify the fraction represented. Additionally, as with area models, the set given may represent a part where the whole is to be determined, or the set given may represent a part where another part is to be determined. Students may use many types of manipulatives and/or pictures to develop a set model (e.g., pictures, cubes, foam shapes, etc.). See glossary for examples.

Patterns Using cards student orders cards 1(ace) through 10 and then teacher questions to have student explain why they ordered that way (this is ordering by attribute of number) 5 triangles of the same shape but of different sizes could be used to demonstrate understanding of ordering by size These activities could easily be turned into patterning by asking the student to continue the pattern that was already demonstrated through ordering.

Problem Type Operations E1, E2 Joining action is permitted at levels 1 and 2 since part-part-whole is a simpler version of a joining action.

Problem Type Chart

Scoring Sample

Template Components Portfolio Validation Form Decision Making Worksheet Parent Review Statement Informed Consent Communication Inventory (Must be present. If not, check Content Cover Sheet. If both are missing, content area cannot be scored.)

Required: Mathematics Assessment Cover Sheet Required: Video for 4 Tasks Required: Concepts of Rational Numbers Cover Sheet Required: Solving Problems with Operations Cover Sheet Required: Equality Cover Sheet Required: Patterns and Change Cover Sheet Supplemental: A copy of any paper products used/produced (tests, worksheets, cards, etc) Supplemental: Written Transcript of audio portions of video that are difficult to understand Math Evidence Collection Documentation

Scoring Materials Student Score Form Step by Step Scoring Procedures for Mathematics Scoring Comment Codes Training Manual: Learning Progressions Glossary

How to View the Video Insert thumb drive Go to My Computer Select removable disc Choose appropriate content area submission Double click to view If video doesnt play please notify table leader

Removing Thumb Drive Close video screen Remove Thumb Drive You do not need to do a complete safe removal of the thumb drive Place thumb drive back into envelope as soon as it is removed so it does not get misplaced among other scoring materials.

Scoring Samples

Scoring Logistics 8:00 am - Start 10:00 - 10:15am Break 12:00 -12:30 - Lunch 2:00 2:15 Break 4:00 End

Organizing Your Space Keep all portfolio components together Leave unneeded documents in the Tyvek envelope Take only the paperwork you need for a particular entry

Protocol for Screening/Scoring Questions: Scorer Table Leader Scoring Leader and Content Expert Marie MP Staff Tina

Finding Samples Videos that clearly and fully show the demonstration of the chosen standard. Creative activities used to demonstrate standards. If you find evidence that you feel could be used as a sample, please fill out an Excellent Example form and give to your table leader

Questions

nh alps mathematics scoring training july 2012. training outline introductions general scoring...

Documents

learning new

student slide

active agent slide

progress sped director

writing scoring leader

science scoring leader

new content

students knowledge