THE Center for IDEAEarly Childhood Data Systems

2013 NAECS-SDE Annual MeetingSan Francisco, CA June 2013

Formative Assessment: So What Happens Next?

Formative Assessment: So What Happens Next?

Kathy HebbelerSRI International

1. What level of investment has your state made in promoting the implementation of formative assessment statewide in early childhood programs?

a. Limited or short term investment b. Some investment c. Substantial investment

2. Where are your early childhood teachers with regard to implementing formative assessment?

a. Not sure where they areb. Few or some teachers across the state are

implementing formative assessmentc. Substantial numbers of teachers are implementing

formative assessment

The purpose of formative (any) assessment is….

…to take some action.

4

Validity

• Is not a characteristic of a tool.• It is not accurate to say “assessment X is valid.”

• Is a characteristic of the use that follows from the results.

• The same tool can be valid for one use (decision) and not valid for another.

• Users should continue to collect and examine the validity evidence for their uses.

5

• There are not different standards of validity for different purposes

• “It is only being used by the teacher in the classroom.”

Myth busting

Findings – Meaning – Action (FMA)

• Findings • The score, the results, the report, the output• If the assessment is administered properly, not

debatable.

• Meaning• The interpretation one puts on the findings.• Reasonable people can differ but…

• Action• The action to be undertaken based on the F and the M.• Reasonable people can differ but...

7

Example

• Finding:• Of the entering kindergarteners,

• 20% can recognize all of the letters of the alphabet.• 25% can count from 1 to 10.

• Meaning• Is this good news or bad news?• What determines whether you see it as good or bad

news?

• Action• Teacher?• Principal?• State Early Childhood Specialist?

8

Name

Domain 11

Plays well with others

2Cooperates with peers

by marching in line

3Stops at

transitions

4Takes

directions well from

adults

5Is a team

player

Dasher A E NY A ADancer NY NY NY NY NYComet A A A A ABlitzen E E E NY ARudolph E E NY E A

A = AchievedE = EmergingNY = Not Yet

Teacher takes action…

Finding Meaning Action

(By student report by objective report for one class)

Individual- Student A making good

progress; Student B not making good progress

- Student A making good progress in literacy, not good progress in math

- Adjust curriculum/ teaching practices for some students

- Get extra help for struggling students

- Nothing

Group- Literacy looks good; math

does not look good

- Try a new math curriculum

- Spend more time on math

- Seek professional development related to math

10

Building level administrator takes action…

Finding Meaning Action

(Percent of students who gained 3 points on at least 80% of objectives in each domain by the end of the year for each EC classroom).

- Some teachers are better than others??

- Some teachers were working with more challenging students?

- Teachers need PD related to early math?

- Teachers aren’t spending enough time on early math?

- Talk with the teachers to try to understand the results

- Give bonuses- Fire some

teachers- Arrange for

professional development

- Develop an action plan

- Nothing

11

Regional/state level administrator takes action…

Finding Meaning Action

(Percent of students who gained 3 points on at least 80% of objectives in each domain by the end of the year for each EC program/district).

- Some schools/districts are better than others??

- Some schools/districts were working with more challenging students?

- Teachers need PD related to early math??

- Teachers aren’t spending enough time on early math?

- Talk with the principals to try to understand the results

- Give bonuses- Institute sanctions- Arrange for

professional development

- ??- Nothing

12

• Findings – What are they?• Meaning – • Good, bad, neutral?• What was expected? • Does it merit an action?

• Action –• Change the course• Stay the course• Reward• Punishment/sanction• Ignore the finding

1. What are the kinds of decisions that will be made based on the findings?

2. Is there a body of evidence to support that the findings from this tool can be used to make these kind of decisions?• Does that body of evidence support the use of the tool

with the populations with whom we are using it?3. What evidence will we collect to substantiate that the tool is

providing valid information for the decisions we are making based on the results?

Using findings from formative assessment

1. Formative assessments should be used for taking action.• Action centers around instructional practices.

2. Validity relates to the use of the findings, not to the tool itself.

3. The obligation rests with the users to:• Identify tools valid for their intended uses.• Continue to verify that the resulting information is valid

for the decisions being made.

Take Away Messages