WELCOME!!Using Assessments and Data to

Improve Student LearningDay 1

1

Introductions

• Notre Dame PD Team– Matt Kloser: Assistant Professor; Director, Notre Dame

Center for STEM Education

– Matt Wilsey: Associate Director, Notre Dame Center for STEM Education

• Stanford PD Team– Hilda Borko: Professor, Curriculum & Teacher Education;

Center to Support Excellence in Teaching

– Stephanie Rafanelli: Doctoral Student, CTE Science Education

2

Introductory Activity

Conceptual Models of Assessment

Task: Develop a visual representation to show how you think about assessment in your science classroom and how different assessment practices are related.

3

Conceptual Models: Directions and Examples

“School science:”

4

Task

Develop a visual representation to show how you think about assessment in your science classroom and how different assessment practices are related.

5

Pair Discussion

• What common themes of assessment did you share?

• What were some differences in your representations of assessment practices?

• What was the most obvious aspect to include?• What surprised you most about your pair’s

conceptual models?

6

Suggested Group Norms

Ancora Imparo

8

Whole Group Share Out/Introductions

Introduce your partner:• Name• At what school does he or she teach?• What grades/subjects does he or she teach?• What stood out most or resonated with you

most strongly about your partner’s conceptual model.

9

IDENTIFYING DIMENSIONS OF ASSESSMENT PRACTICE

10

Norms for Examining Practice

When examining teaching artifacts or practice, artifacts are seen as

examples, not exemplarsto be approached in a spirit of

investigation, not evaluation emphasizing value in

expertise from the inside, not the outside

(Sherin, 2004)

11

Pair Work:Examine One Assessment Artifact

• How would you describe the artifact?• How is the teacher trying to learn about

student thinking?• What can the teacher tell about student

thinking from the artifact?• What does the assessment not capture or

represent?• What else would you like to know?

12

Discussion

• What is one feature of assessment you noticed from the introductory activity?

• What else did you notice?

13

Examine Five Days of Artifacts

Part I: For materials in each day, use the worksheet to consider the following:o Describe:

1.The assessment artifacts.2.The teacher’s annotations

o Analyze1.What can the teacher learn from this assessment?2.What can the students learn from this assessment?

14

Examine Five Days of Artifacts

Part 2: Integration: After you have gone through the materials for each day, consider the notebook as a whole:o What features of science assessment practice do you

notice in the notebooks?o Are these features connected with what you know or

believe about science assessment practice? How so?

15

Share your reflections

• What did you and your partner notice?• What patterns do you see? • What dimensions did the other group find that

echo ones your group identified?

16

AN ASSESSMENT FRAMEWORK FOR SCIENCE CLASSROOMS

Why Does Assessment Matter?

17

A Thought Exercise

Hattie, 2014

18

A Thought Exercise

Hattie, 2014

SES: 22%

19

An Assessment Framework for Science Classrooms

1. Where are you going?2. Where am I now?3. How are you going to get there?

Assessment Types

Formative BenchmarkSummative

20

Formative Assessment DimensionsWhere are you going?1. Setting Clear

Learning Goals2. Alignment of

Assessments with Goals & Science Standards

Where are you now?3. Frequency4. Variety5. Cognitive

Complexity6. Reflection of

Scientific Practices

7. Students’ Use of Assessment Data

How do you get there?8. Feedback to

Students9. Adapting

Instruction Based on Assessment Data

21

WHERE ARE YOU GOING?DIMENSIONS 1 & 2

22

Dimension 1: Setting Clear Learning Goals

From the following pairs, choose what you think is a ‘better’ goal for maximizing student motivation and achievement and talk with your partner about the rationale for your decision.

23

Pair 1

A. Do your best.

B. Answer these five questions before you leave.

Locke & Latham, 2002

24

Pair 2: Task Goal

A. Balance 2 chemical equations

B. Balance 7 chemical equations

Locke & Latham, 2002

25

Pair 3

A. With a partner, develop the procedures for the lab questions we developed today. We will use these procedures for our lab tomorrow.

B. By the end of this year, develop an entire lab report that includes the research questions, procedures, methods, results, and discussion.

Bandura, 1986

26

Where might one find evidence of setting clear learning goals:

- In the classroom?- In the QAS Notebook?

27

Dimension 1 Rubric28

If you had to provide feedback on this assessment to a new teacher, what would you say?

Lesson Goal: Construct an argument from evidence that changes to the physical components of an ecosystem can affect populations

End of Lesson Quiz Questions:1. Define the term ‘population’2. Which of the following can affect a population?

a. Changes to the physical environmentb. Changes to organisms in an environmentc. Both a & b

3. Name an ecosystem and one living and one non-living part of the ecosystem

29

Dimension 2: Alignment of Assessments with Learning Goals & Science Standards

Core Science Ideas• Physical Science

– Matter & its interactions– Forces & interactions– Energy– Waves & their applications

• Life Science– Structures & Processes– Ecosystem interactions– Inheritance/variation of traits– Biological Evolution

• Earth & Space Sciences– Earth’s place in the universe– Earth’s systems– Earth & human activity

Crosscutting Concepts• Patterns• Cause and effect• Scale, proportion, &

quantity• Systems and system models• Energy and matter: flows &

cycles• Structure & function• Stability & change

30

Dimension 2: Standard Examples

MS-LS2-2: Construct an explanation that predicts patterns of interactions among organisms across multiple ecosystems, such as competition, predation, mutual benefit.

IN-8.2.6: Identify, explain, and discuss some effects human activities have on the biosphere.

31

Dimension 2 Rubric32

WHERE ARE YOU NOW?DIMENSIONS 3-7

33

Dimension 3:Frequency of Assessment

Number of Assessments Percentile Gain

1 13.5

5 20.0

10 22.5

15 24.5

20 26.0

25 28.5

30 29.0

Achieved Gain Associated with Number of Assessments over 15 Weeks

Bangert-Drowns, Kulik, & Kulik (1991)

34

Dimension 3 Rubric35

Dimension 4:Variety of Assessment

Pair Discussion• What are different ways in which you could

gain understanding (assess) about student thinking or skills in science?

• Why might variety be important and for whom?

36

Brief Pair Work

• Look at the two excerpts from actual middle school science tests. Compare the question types and what those types may or may not afford students and teachers about student understanding.

37

Dimension 4 Rubric38

Why might setting goals and alignment to those goals not be sufficient to ensure quality science assessment?

39

Bloom’s Revised Taxonomy

Creating

Evaluating

Analyzing

Applying

Understanding

Remembering

Anderson & Krathwohl, 2001

40

Dimension 5:Cognitive Complexity

Read over the two sets of questions on the handout.Think about the types of answers that student might give.

– How do the two sets of questions differ? – What can you tell about student thinking from the

two sets of questions?

41

• Creating• Evaluating• Analyzing• Applying• Understanding• Remembering

MS-PS1-2. Analyze and interpret data on the properties of substances before and after the substances interact to determine if a chemical reaction has occurred. MS-PS1-5. Develop and use a model to describe how the total number of atoms does not change in a chemical reaction and thus mass is conserved. MS-PS1-6. Undertake a design project to construct, test, and modify a device that either releases or absorbs thermal energy by chemical processes.

42

Dimension 5 Rubric43

Dimension 6:Reflection of Scientific Practices

44

Example 1:Explain why Seattle, WA receives double the yearly rainfall compared to Spokane, WA.

Example 2:What are the main factors that affect climate and weather?

45

Example 1:Devise a fair test and create an argument from the resulting data about whether a steel ball bearing or a Styrofoam ball of the same size hits the ground first when dropped from 2 meters.

Example 2:Follow the procedures in your lab manual to confirm that the acceleration due to gravity equals 9.8 m/s2.

46

Dimension 6 Rubric47

Dimension 7: Students Involvement in their Own Assessment

1. Build on Prior

Knowledge

2. Organize Deep Factual Knowledge in Conceptual Frameworks

3. Use a Meta-Cognitive Approach

48

Helping Students Monitor Their Own Learning

• Self-evaluation must be formative – it shouldn’t be used to determine one’s own grade. (Andrade & Valtcheva, 2009).– Research shows that when tied to a

grade, students self-inflate.

49

Strategies & Models

• Self-evaluation must be formative; research shows that when tied to a grade, students self-inflate (Andrade & Valtcheva, 2009).

• Have students underline parts of the rubric and the artifact with the same color

• Have students say why wrong answers are wrong

• Use existing artifacts to have students create criteria for performance

50

Q. What idea or skill was tested? Right? Wrong? Careless Mistake

Don’t Understand

1.

2.

3.

4.

5.

51

Q. What idea or skill was tested? Right? Wrong? Careless Mistake

Don’t Understand

1.Reading a pedigree

X X

2.Dihybrid cross

X

3.Defining heterozygous and homozygous X X

4.Identifying abnormalities in a karyotype X

5.Identifying phenotype and genotype

X X

52

Dimension 7 Rubric53

Reflecting on Dimensions 1-7

What Dimension do you currently see as your greatest area of strength? Weakness?

–Why?

54

ANALYZING PRACTICE: REVIEWING A NOTEBOOK

55

Rate a Notebook

• To get an understanding of what the dimensions look like in practice, you will rate a notebook that shows all of the dimensions we just discussed.

• Activity Overview– Read and review– Rate– Discuss

56

Notebook Rating -Task

1. Get a sense of what was done (planned and assessed) throughout the collection period

a. Take notes as you skim through the Notebook

2. Rate the notebook, using the dimension rubric

3. Explain your rationale4. Reflect on the rating activity

- Questions will be posted once you get started

57

Dimension Rating Form58

Reflection Questions

(Feel free to answer these on the back of the dimension rubric.)1. What are some of the strengths of this notebook?2. Where do you see opportunity for growth?3. What you are overall thoughts/comments on the notebook

59

Rating Discussion

• Discuss individual ratings for each dimension– If disagreement, make a case for ratings– If possible, come to consensus– If not, understand reasons for disagreements

• If time, discuss reflection questions

60

CLOSING DAY 1

61

Where Are We Now and Where Are We Going?

• Where are we now?– Initial assessment conceptual model– Overview: the assessment framework– Dimensions of science assessment practice– Rate and discuss example Notebook

• Where are we going?– Dimensions of science assessment practice &

Notebook rating (continued)– Rate and discuss own Notebook– Adapting instruction based on formative assessment

62

EXIT CARD

In thinking about the Dimensions of Science Assessment Practices and reviewing the example Notebook:1. What Dimension(s) do you currently see as your greatest area of strength? 2. Weakness?

–Why?

63

Exit Card

In thinking about the Dimensions of Science Assessment Practices and reviewing the example Notebook:• What was easy?• What was challenging?• What surprised you? • What initial ideas do you have for using

assessment data to modify your instruction?

64