june 18, 2015 (salem, or) june 30, 2015 (eugene, or) oregon rubric & quality review training...

June 18, 2015 (Salem, OR)June 30, 2015 (Eugene, OR)

Oregon Rubric & Quality Review Training Session: Mathematics

Welcome and Warm Up

• Google Me!– Google your name, and share a few details about

what you found with your table group.– Be prepared to share an interesting persona with

the rest of the room.

Reasons for Review

• Chapter 337 of ORS– State Board of Education shall adopt a list of instructional materials for

each grade level and subject area in which textbooks are required (337.050).

• Division 11 of OAR – Instructional materials to be evaluated by committees composed of

teachers and other stakeholders who are knowledgeable of the grade level and subject under evaluation (581-011-0066).

– Only basal materials may be evaluated and adopted (581-011-0050).

Common assumptions

• Instructional Material vs. Curriculum– What do you see as the difference?

• Our job is to review instructional materials– Materials are the raw materials– Curriculum includes everything a teacher pulls

together to support instruction• No instructional material is going to be perfect

(100%), or imperfect (0%)

Session Goals

Develop a cadre of professionals who can guide the use of the Oregon quality review process (based on EQuIP and IMET) to determine the quality and alignment of lessons and units to the Common Core State Standards (CCSS) in mathematics

• Develop their abilities to use Oregon math materials criteria to provide observations about CCSS-aligned instructional materials and make suggestions for improvement

• Develop a common understanding of the Oregon quality review process

• Develop a common understanding of the rating scale and descriptors for the four rubric dimensions and the rating categories and descriptors for overall ratings

• Develop their abilities to consistently use the criteria, rating scales and rating descriptors to accurately rate instructional materials

Oregon Quality Review: Principles & Agreements

1. CCSS: Before beginning a review, all reviewers are familiar with the CCSS. 2. Inquiry: Inquiry rather than advocacy and organized in steps around guiding

questions.3. Respect & Commitment: Each member of a review team is respected 4. Criteria & Evidence: All decisions are evidence based. 5. Constructive: Lessons/units to be reviewed are seen as “works in progress.”6. Individual to Collective: Each member of a review team independently

records his/her observations prior to discussion. Discussions focus on understanding all reviewers’ interpretations of the criteria and the evidence they have found.

7. Understanding & Agreement: The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to the CCSS.

Criteria Overview: Content and Mathematical Practices

Alignment to Mathematical Content

Alignment to Mathematical Practices

1. Focus 6. Appropriate mathematical practices

2. Coherence 7. Overarching habits of mind of a productive mathematical thinker

Rigor 3. Application4. Conceptual Understanding5. Procedural Skill and Fluency

8. Reasoning and explaining

9. Modeling and using tools10. Seeing structure and generalizing

Criteria Overview: Instructional Supports and Assessment for K-8

Criteria Overview: Instructional Supports and Assessment for High School

Focus & Coherence

Focus

• What is focus?• Why is this important?

Major Clusters

Finding Focus in the CCSS

13

HS Focus documents

14

Focus Activity

• Go To Activity 1– Looking at the focus document, which 2 or 3

Major clusters would want to look at first when reviewing instructional materials• repeat for each grade band in the submission category

– What major clusters rise to top for each grade/course?

– What will you look for first when given a new instructional material?

15

Coherence

• What is coherence?• Why is this important?

Looking For Coherence Within Grades

• Examples: 1st grade – 5th grade: Represent and Interpret

Data 3rd grade & 5th grade: “Relate area (volume) to

multiplication and to addition.” 6th grade: Solve problems by graphing in all 4

quadrants. (1st year of rational numbers) 8th grade: “Understand the connections between

proportional relationships, lines and linear equations.”

Coherence Within A Grade

Use addition and subtraction within 100 to solve word problems involving lengths that are given in the same units, e.g., by using drawings (such as drawings of rulers) and equations with a symbol for the unknown number to represent the problem.

2.MD.5

18

Coherence Within A Grade

Make a line plot to display a data set of measurements in fractions of a unit ( ½, ¼, 1/8). Solve problems involving addition and subtraction of fractions by using information presented in line plots. For example, from a line plot find and interpret the difference in length between the longest and shortest specimens in an insect collection.

4.MD.4

19

Looking for Coherence Across Grades

• Coherence is an important design element of the standards.

• “The Standards are not so much built from topics as they are woven out of progressions.”

Structure is the Standards, Publishers’ Criteria for Mathematics, Appendix

• http://tinyurl.com/orimreview

Coherence: Activity 2

• Go to Activity 2 Google Doc• Take out example lesson for grade band– Questions to ask:• Does the material use CCSS language?• What is the on-grade level content for this lesson?• What content is from either prior or subsequent

grades?– How does the lesson connect on-grade content to prior

knowledge?

• What connections to within grade-level content are evident within the lesson?

Focus/Coherence Reflection

• Open Results from Activity 1• After completing the activity:

1. How will you use the major work of each grade to streamline your review process?

2. How could you evaluate for coherence across your grade band as you look at individual lessons?

22

Rigor

Rigor

• CCSSM require students to engage with and demonstrate challenging mathematics with appropriate balance among:– Application– Conceptual understanding– Procedural skill and fluency

How do the Standards signal Rigor?

• Application:– Provide opportunities for students to independently apply

mathematical concepts in real-world situations.

• Conceptual Understanding:– Develops understanding through conceptual problems and

questions, multiple representations and opportunities for students to write an speak mathematically.

• Procedural Skill and Fluency:– Expects, supports, and provides guidelines for procedural skill and

fluency with core calculations and mathematical procedures to be performed quickly and accurately.

The three legged stool

Application

• Real-world problems (single- and multi-step)

• Non-routine problems

• Varied problem types (see Tables in CC.OA progressions)

• Enhance major work of the grade

• Constructing models (6-12)

Conceptual Understanding

• Problems can (and should sometimes) be brief

• Explaining reasoning is one way to address conceptual understanding

• Problems and exercises should be grade-level appropriate

• Connections between representations are emphasized

Procedural Skill and Fluency

• Purely procedural problems

• Opportunistic strategies; writers are thoughtful about numbers used

• Repeated practice

• Procedures are built from conceptual understanding

How do the Standards signal Rigor?

• Conceptual Understanding:3.NF.1 Understand a fraction 1/b as the quantity formed by 1 part when a whole is partitioned into b equal parts; understand a fraction a/b as the quantity formed by a parts of size 1/b.

• Procedural Skill and Fluency:5.NBT.5 Fluently multiply multi-digit whole numbers using the standard algorithm.

• Application:7.NS.3 Solve real-world and mathematical problems involving the four operations with rational numbers.

Activity 3: Rigor

• Form groups of 2-3 within your Category; each group will look at a different rigor strand.

• CONCEPTUAL UNDERSTANDING• PROCEDURAL SKILLS, and FLUENCY• APPLICATION

– In your rigor strand:• Look through the standards identified• Underline key words related to that aspect of rigor• Identify grade level clusters associated with the standard

• Move into K-12 groups by three strands of rigor.– By grade level (or course), discuss the standards with your group to

identify key aspects in the progression that specifically address a particular type of rigor in that grade (or course).

– Note the key aspects of rigor on your copy of the standards for each grade level.

31

Rigor

• Organize back into grade-band groups• Go to Activity 3 Google Doc• Examine example unit for Rigor• What examples did you find? Note these on

your grade band’s Activity 3 document.

• CONCEPTUAL UNDERSTANDING• PROCEDURAL, SKILL, and FLUENCY• APPLICATION

32

Math Practices

33

Grouping of Math Practices

Reasoning and Explaining2. Reason abstractly and quantitatively3. Construct viable arguments and critique the reasoning of others

Modeling and Using Tools4. Model with mathematics5. Use appropriate tools strategically

Seeing Structure and Generalizing7. Look for and make use of structure8. Look for and express regularity in repeated reasoning

Overarching Habits of Mind of a Productive Mathematical Thinker1. Make sense of problems and persevere in solving them6. Attend to precision

Adapted from (McCallum, 2011)

34

Small Group Discussion

• Go to Activity 5 Google Doc– http://www.corestandards.org/Math/Practice/ – Math Practice Elaborations in folder

• Form groups around pairs of math practices• Evaluate your given unit for your given math

practices

• Questions:– What is this practice standard asking of students?– What would this practice look like in your classroom?

35

Closer look at the OR-IMET

36

Using the Quality Review Rubric (Adopted Criteria)

OR-IMET: Oregon Instructional Materials Evaluation Tool

38

Closer look at the OR-IMET

• At a high level, connect the supporting worksheets to the adopted criteria

• Take a closer look at read the wording of each “Quality Indicator”– Does each of the Quality Indicators make sense?– What may be missing?– What do you need clarification on?

39

Evaluating Instructional Materials

Quality Review Steps

• Focus/Coherence:– Review discrete sampling of lessons across the grade band addressing

major work of interest– Focusing on a particular domain progression may be helpful

• Rigor– Review sampling of units across the grade band addressing major work of

interest• Math Practices

– Use the same sample of units and lessons you looked at to find evidence of math practices

• Instructional Supports & Assessments– Look for evidence additional criteria after you looked at a critical mass of

sample lessons and units

Reviewing Materials

• Individually review sample curriculum – Within each grade band, decide who will be

looking at:• Focus/Coherence• Rigor• Math Practices

– Score your given set of indicators, and be prepared to discuss

– Once finished with your indicator, move on to Instructional Supports and Assessments

42

Coming to Consensus

43

How to reach consensus.

• What is consensus?– Is it an average?– Is it the most common rating?– Does each group member have to agree?

44

What is Consensus?

• Consensus means overwhelming agreement

• NOT unanimity

45

What Consensus IS and is NOT

• See handout in your folder:

46

Consensus Is1 Consensus is NOTAll group members contribute All group members agree

Everyone’s opinion is heard The result is everyone’s first choice

Differences are viewed as helpful There are no differences of opinion

Everyone can paraphrase the pros and cons Everyone possesses a complete understanding

Everyone agrees they can live with the final decision

Conflict and resistance will be overcome immediately

All members share the final decision All members lend their full-fledged support for the final decision

Facilitating Consensus

• Make sure everyone feels comfortable participating

• Make sure the group feels that the ideas and decisions are theirs, not just the leader's.

• Identify areas of agreement and disagreement

47

Consensus is Reached

• When all members can say: – I believe that you understand my point of view

and that I understand yours.

– Whether or not I prefer this decision, I support it because • it was reached fairly and openly, and • it is the best solution for us at this time.

48

Consensus Test

1. Do we agree?

2. Does anyone disagree?

3. Do we have a consensus?

49

Making Comments

50

Giving Feedback

Writing effective feedback is vital to the Quality Review Process. Below are the four qualities of effective feedback. • Criteria-based: Written comments are based on the criteria used for review

in each dimension. No extraneous or personal comments are included. • Evidence Cited: Written comments suggest that the reviewer looked for

evidence in the lesson or unit that address each criterion of a given dimension. Examples are provided that cite where and how the criteria are met or not met.

• Improvement Suggested: When improvements are identified to meet criteria or strengthen the lesson or unit, specific information is provided about how and where such improvement should be added to the material.

• Clarity Provided: Written comment are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions.

Feedback Example #1: Mathematics

This unit clearly targets three CCSS, which are noted in the overview. The overview also indicates which Standards for Mathematical Practice are central to the lesson. The activities throughout the unit present a balance of mathematical procedures and deeper conceptual understanding of the standards. The activities reinforce the standards and are well-connected to the content. I think the activities might be challenging with a large class with classroom management issues.

Is this feedback criteria-based?Was evidence cited?Was there an improvement suggested?Is clarity provided?

Example 1: Math Feedback

Criteria-based: YesEvidence Cited: PartialImprovement suggested: NoClarity Provided: Yes• This feedback could be more effective. The reviewer mentions three standards and cites

evidence: “which are noted in the overview.”• The reviewer states, “ The overview also indicates which Standards for Mathematical Practice

are central to the lesson.” The reviewer does not provided any evidence to support the assertion.

• The reviewer comments, “the activities through the unit present a balance of mathematical procedures and deeper conceptual understanding of the standards,” but does not cite specific details about why or how they are grade appropriate and well connected to the content being addressed.

• No improvements are suggested. The reviewer does insert a personal opinion when saying, “ I think the activities might be challenging with a large class with classroom management issues. Personal opinions and comments unrelated to the criteria should be avoided when providing feedback; criteria-based feedback is more effective.

• The written comments are constructed in a manner in keeping with basic grammar, spelling, sentence structure and conventions.

Feedback Example #2: Mathematics

The lesson targets two CCSS, which are highlighted in the lesson cover page. Although the lesson does integrate Standards for Mathematical Practice, including appropriate tools strategically, given the CCSS that are targets, modeling may be a better fit. The lesson does present a balance of mathematical procedures and deep content knowledge.

Is this feedback criteria-based?Was evidence cited?Was there an improvement suggested?Is clarity provided?

Example 2: Math Feedback

Criteria-based: YesEvidence Cited: PartialImprovement suggested: PartialClarity Provided: No• This feedback could be more effective. • Some specific evidence is cited to support the claims that criteria are present in

the lesson.• There is an improvement suggested with, “modeling may be a better fit,”

however, it would be beneficial to explain why.• The written comments are not constructed in a manner in keeping with basic

grammar, sentence structure and conventions.

Practice Consensus & Comments

56

Activity: Small group ratings

• Practice coming to consensus scores within your grade band subgroups (Focus/Coherence, Rigor, Math Practices)

• After evidence has been shared, other pair shares what they believe the rating should be based on the evidence presented

• Switch roles

• Reach a consensus score for your table.

• With your entire grade-band team, discuss:– Which evidence was most helpful in determining the rating?

– What was most difficult?

– What disagreements came up? Were they resolved? If so, how?

Activity: Grade band consensus

• Regroup in full grade band groups• Take turns by subgroup to – Share consensus rating– Share evidence of rating– Answer questions from other members– Document any additional comments from the

larger group• Switch to next subgroup. Make sure each

group has time to share ratings.

58

Debrief

• Whole group discussion:– Which evidence was most helpful in determining

the rating?

–What was most difficult?– What disagreements came up? Were they

resolved? If so, how?

59

THANK YOU!