outcomes and evidence statements
DESCRIPTION
Outcomes and Evidence Statements. Kathy Hebbeler SRI International. Prepared for the Michigan Results Group Lansing, MI July 2005. Objectives for this presentation. Review the ECO outcomes - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/1.jpg)
Outcomes and Evidence Outcomes and Evidence Statements Statements
Outcomes and Evidence Outcomes and Evidence Statements Statements
Prepared for the Michigan Results Group Lansing, MI July 2005
Kathy HebbelerSRI International
![Page 2: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/2.jpg)
2
Objectives for this presentationObjectives for this presentation
Review the ECO outcomes Introduce some concepts related to
evidence statements to assist Michigan in formulating its evidence statements
Share ECO’s recommendations for evidence statements
Describe what OSEP will be requiring Introduce the measurement approach
ECO is developing
![Page 3: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/3.jpg)
3
Early Childhood Outcomes (ECO) Early Childhood Outcomes (ECO) Center: MissionCenter: Mission
Promote the development and implementation of child and family outcome measures for infants, toddlers and preschoolers with disabilities that can be used in national and state accountability systems
![Page 4: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/4.jpg)
4
ECO CenterECO Center
5-year project funded by OSEP in October 2003Collaboration among: SRI International Frank Porter Graham Child Development
Institute (U of North Carolina, Chapel Hill) Juniper Gardens Children’s Project (U of
Kansas) National Association of State Directors of
Special Education University of Connecticut
![Page 5: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/5.jpg)
5
Public Policy ContextPublic Policy Context
Age of accountability Accountability increasingly means
looking at results – not just process Applies across all private and
public human service and education programs
![Page 6: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/6.jpg)
6
President’s Commission on President’s Commission on Excellence in Special Education Excellence in Special Education (2002)(2002)
Major Recommendation 1: Major Recommendation 1: Focus on results – not processFocus on results – not process
“IDEA will only fulfill its intended purpose if it raises expectations for students and becomes result-oriented—not driven by process, litigation, regulation and confrontation. In short, the system must be judged by the opportunities it provides and the outcomes achieved for each child.”
![Page 7: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/7.jpg)
7
Federal push for outcome data: Federal push for outcome data: Intro to AcronymsIntro to Acronyms
GPRA =GPRA =Government Performance Government Performance
and Results Actand Results Act PART = Program Assessment Rating PART = Program Assessment Rating
ToolTool OMB = Office of Management and OMB = Office of Management and
BudgetBudget IDEA = Individuals with Disabilities IDEA = Individuals with Disabilities
Education ActEducation Act
![Page 8: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/8.jpg)
8
GPRA (Government Performance GPRA (Government Performance and Results Act)and Results Act)
IDEA goals and indicators established IDEA goals and indicators established Indicators and data collection further along for Indicators and data collection further along for
school age population than for ECschool age population than for EC For early childhood, data have been collected For early childhood, data have been collected
on: on:
• Number of children served (Part C)Number of children served (Part C)
• Settings (both Part C and 619)Settings (both Part C and 619) Note: Part C = Programs for 0-3 year olds; 619 = Programs for 3-5
![Page 9: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/9.jpg)
9
PART (Program Assessment PART (Program Assessment Rating Tool)Rating Tool)
Tool used to review federal programsTool used to review federal programs Four critical Assessment Areas Four critical Assessment Areas
includingincluding
““results and accountability”results and accountability” Programs given ratings from ineffective Programs given ratings from ineffective
to effectiveto effective Purpose – “to enhance budget analysis”Purpose – “to enhance budget analysis”
![Page 10: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/10.jpg)
10
PART Evaluation: Findings and PART Evaluation: Findings and RecommendationsRecommendations
Part C and 619 Findings: No long-term child outcome goals or data
The PART required the Department to: 1. Establish long-term outcome-oriented
objectives 2. Develop a strategy to collect performance
data Office of Special Education Programs (OSEP)
asked states to provide outcome data in their Annual Performance Reports (APRs)
![Page 11: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/11.jpg)
11
Reality Check (aka Challenges)Reality Check (aka Challenges) Michigan will be required to submit data to
OSEP OSEP has recently released the indicators on
which it will be requiring data and the timeline is impossible
ECO is simultaneously working on many of the same issues the states are. We have a measurement approach in development but it is far from complete.
We have no choice but to live with the uncertainties and move forward as best we can
![Page 12: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/12.jpg)
12
Steps in Development of an Steps in Development of an Outcomes SystemOutcomes System
1. Identify overall purpose for the system (including priorities)
2. Identify outcome areas for children and families
3. Develop outcome statements4. Formulate the evidence statements 5. Identify measurement approaches (e.g.,
select instruments) ***Steps 6 to 11 after that
![Page 13: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/13.jpg)
13
Need to acknowledge the range of Need to acknowledge the range of emotional responses to outcomes emotional responses to outcomes systemssystems
Hate it, have to do it so I will Hate it, have to do it but I won’t do a very
good job of it because it is a waste of my time Neutral, just another fad that will pass Not sure, might be good but not likely and
probably will be useless to harmful Worried that this will take time away from
children and families Hopeful that this will live up to the potential Excited about the prospect of having data on
child outcomes
![Page 14: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/14.jpg)
14
Identify outcomes areasIdentify outcomes areas
What are the important outcomes areas? Child Family
Everyone wants to know “How are children doing?” but need to decide “with regard to what?”
![Page 15: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/15.jpg)
15
DefinitionsDefinitions
Outcome—a statement of a measurable condition(s) desired for the population of children or their families Children take appropriate action to
meet their needs Families know how to advocate for
the services they need
![Page 16: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/16.jpg)
16
Identify outcome areasIdentify outcome areas
Outcome statements are the system’s overall vision for children and for families
Not the same as outcomes on an IFSP or IEP which are very specific outcomes for an individual child or family
Outcomes in an accountability system are global statements of what we are trying to do for children and families
![Page 17: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/17.jpg)
17
Identify outcome areasIdentify outcome areas
How should we think about child outcomes? “…for more than three decades,
researchers and service providers have struggled with both the identification of significant child outcomes and their valid and reliable measurement.”
National Research Council and Institute of Medicine, 2000
![Page 18: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/18.jpg)
18
Identify outcome areasIdentify outcome areas
How should we think about child outcomes? Domains (language, cognitive, adaptive,
etc.) Functional skills (feeding, dressing, mobility,
etc.) Underlying functional capacities (Self-
regulation, engagement, knowledge acquisition skills, etc.)
Pre-academic skills (Early reading, math, science, etc.)
![Page 19: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/19.jpg)
19
Pitfall Alert!Pitfall Alert! What we want for children and families
(desired outcomes) reflects our values What we want for young children may not be
the same as what can be easily measured When what is valued and what can be
measured easily or well do not line up, EITHER: Outcomes will be determined by what can be
measured easily or well, OR Some of the outcomes will be more easily measured
or more adequately measured than others
![Page 20: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/20.jpg)
20
ECO Family OutcomesECO Family Outcomes
1. Families know their rights and advocate effectively for their children.
2. Families understand their children's abilities and special needs
3. Families help their children develop and learn.
4. Families have social supports. 5. Families are able to gain access to desired
services, programs, and activities in their community.
![Page 21: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/21.jpg)
21
Need for an overarching goal for Need for an overarching goal for childrenchildren
The ultimate goal is for young children to be active and successful participants now and in the future in a variety of settings– in their homes, in their child care, preschool or school programs, and in the community
![Page 22: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/22.jpg)
22
ECO Child OutcomesECO Child Outcomes
1. Children have positive social relationships
2. Children acquire and use knowledge and skills
3. Children take appropriate action to meet their needs
![Page 23: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/23.jpg)
23
Issues from stakeholder Issues from stakeholder discussionsdiscussions
Functional outcomes Best practice – kind of outcomes
recommended for IFSPs and IEPs Consistent with transdisciplinary
service delivery Not reflected well in a pure domains
framework Not captured well in current
assessment tools
![Page 24: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/24.jpg)
Children have positive social relationships
Children acquire and use knowledge and skills
Children take appropriate action to
meet their needs
Relation-ships with
adultsRelation-
ships with
peers
Follows group rules
Masters the environ-ment
Symbol use, abstract thinking
Appliesknowledge
Listening PlayExploring
Playing
Being curious
Practicing
Touching
Attending
EngagingPersisting
Knowledge of physical world & culture
Self-care, health and safety
Elaboration of the ECO OutcomesElaboration of the ECO Outcomes
![Page 25: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/25.jpg)
Children will be active and successful participants now and in the future in a variety of settings.
ECO Outcomes:
Overarching Goal:
Children have positive social relationships
Children acquire and use knowledge and skills
Children take appropriate action to meet their needs
Domains: Communication Cognition
Motor
Social-Emotional
Approaches to Learning
Content Areas:
Language Arts Math
Science Social Studies Art
Music
Note: Each of these can be broken down further into sub-areas
Processes: Memory Self-regulation Recognizing and interpreting sensory input
Listening Attending
Body Functions: Vision Seeing
Movement-flexibility-strength-postural response
Etc.
Speech production Etc.
Self Help
Example of how movement can be sub-divided
Alternative Ways of Thinking about Child OutcomesAlternative Ways of Thinking about Child Outcomes
![Page 26: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/26.jpg)
26
What is happening with these What is happening with these outcome areas and statements?outcome areas and statements?
ECO’s work is independent of OSEP OSEP has been involved in ECO’s work
from the beginning ECO submitted these outcomes to OSEP
as its recommendations ECO submitted recommended indicators
based on these outcomes to OSEP for the APR/SPP instructions that were available for public comment
![Page 27: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/27.jpg)
27
Evidence Statements(or what do you want to know about those
outcomes)
![Page 28: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/28.jpg)
28
The Importance of Evidence The Importance of Evidence Statements Statements
What constitutes evidence of good outcomes?
If you were asked to testify before your state legislature to demonstrate the effectiveness of EI, what kind of evidence do you want to have? (Purpose = accountability)
What kind of evidence about outcomes is needed by states and local programs to improve services for children and families? (Purpose = program improvement)
![Page 29: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/29.jpg)
29
Definition of Evidence StatementDefinition of Evidence Statement
Evidence Statement - a statement that incorporates a statistic and provides evidence as to whether not an outcome has been achieved
% of children who have positive social relationships
% of children who show progress toward acquiring knowledge and skills
Note: States will be asked to submit a specified kind of evidence to OSEP. Each state will need to be able to produce this kind of evidence but might want other kinds of evidence as well.
![Page 30: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/30.jpg)
30
Features of a Good Evidence Features of a Good Evidence StatementStatement
Credible: Based on valid data Meaningful: The evidence can be
interpreted Powerful: The evidence is convincing
to those who will be receiving it (purpose = accountability)
Useful: The evidence is helpful to those who will use it (purpose = program improvement)
![Page 31: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/31.jpg)
31
MeasurementMeasurement Evidence Evidence Statements Statements
Evidence statements are the ultimate product in an outcomes-based accountability system
The nature of the evidence statement that can be produced will depend on what is measured and how often
![Page 32: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/32.jpg)
32
Possible Categories of Evidence Possible Categories of Evidence Statements for Outcome DataStatements for Outcome Data
1. Status (achievement at one point in time)
Example: “% of children who [achieved this..]
2. Progress (change relative to earlier status)
Example: “% of children who made progress in..]
![Page 33: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/33.jpg)
33
Two Ways to Think about Two Ways to Think about Progress with Young ChildrenProgress with Young Children
Continuous Progress = acquisition of new skills and behaviors (almost all children show this)
Or Age-anchored Progress = change in
developmental trajectory (i.e., narrowing the gap between a child with a delay and typical development; “less delayed” after intervention)
![Page 34: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/34.jpg)
34
Examples of Two Kinds of Examples of Two Kinds of ProgressProgress
Continuous– increase in vocabulary Age-anchored– change from being
1.5 standard deviations below norm to .5 standard deviation
![Page 35: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/35.jpg)
Hypothetical Language Acquisition Rates for Three Groups of Children: Change in
Developmental Trajectory (Progress toward Closing the Gap)
0
10
20
30
40
50
60
70
1 6 11 16 21 26 31 36 41 46 51 56Age in Months
La
ng
au
ge
Sc
ore
Typically developing children
Children with delays without intervention
Children with delays after intervention
![Page 36: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/36.jpg)
36
Advantages and Disadvantages of Different Evidence Statements
![Page 37: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/37.jpg)
37
Status (where children are)Status (where children are)
Reports the percentage of children who achieved or could do X
Would likely involve some kind of comparison to expectations for children of a given age
Examples: 53% of early intervention graduates were
rated as “typical and proficient” with regard to having positive social relationships
![Page 38: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/38.jpg)
38
Status Evidence StatementStatus Evidence Statement
Pro Requires one measurement point Could be strong evidence Nearly all K-12 evidence statements are of this
form (grad. rates, “proficient” reading)Con What is the benchmark that makes sense for
all children with disabilities? How does one interpret the data (is this finding
good news or bad news?)
![Page 39: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/39.jpg)
39
Continuous Progress (change Continuous Progress (change compared to earlier status)compared to earlier status)
Reports % of children who improved Progress on a curriculum-based
assessmentExamples:
% of preschool children who made progress toward having positive social relationships
% of preschool children who showed an increase of 3 of more objectives related to positive social relationships scale after one year
![Page 40: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/40.jpg)
40
Continuous ProgressContinuous Progress
Pro Straightforward Could quantify extent of child growth Incorporates all levels of gain, even the very
small gains made by children with severe impairments
Con Requires at least 2 points of data Nearly all children will improve over time Weak evidence. What constitutes “good
news”?
![Page 41: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/41.jpg)
41
Age-Anchored Progress (closing the Age-Anchored Progress (closing the gap toward typical development)gap toward typical development)
Captures progress relative to same age peers
Examples: 23% of children moved from low performing
to age appropriate in positive social relationships by kindergarten entry
With regard to acquiring and using knowledge and skills, 75% of children made progress sufficient to maintain their functioning at an age appropriate level
![Page 42: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/42.jpg)
42
Age-Anchored Progress (closing the Age-Anchored Progress (closing the gap toward typical development)gap toward typical development)
Pro Consistent with the intent of intervening Could produce strong evidence
Con Requires at least 2 time points of data Closing the gap is not an expectation for all
children with disabilities. How is the evidence to be interpreted?
If some children don’t close the gap, is this interpreted as lack of success for the program?
![Page 43: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/43.jpg)
43
Deciding on desired evidenceDeciding on desired evidence
Not an either-or Different levels will want different
kinds of evidence Many different kinds of evidence
can be produced from the same set of data If the desired evidence statements
are identified as part of the planning
![Page 44: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/44.jpg)
44
What OSEP will be requiring from states for Part C and Preschool
Special Education
![Page 45: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/45.jpg)
45
Reporting Requirement: Part CReporting Requirement: Part C
Percent of infants and toddlers with IFSPs who demonstrate improved:
A. Positive social-emotional skills (including social relationships)
B. Acquisition and use of knowledge and skills (including early language/communication)
C. Use of appropriate behaviors to meet their needs
![Page 46: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/46.jpg)
46
The detailsThe details
a. % of infants and toddlers who reach or maintain functioning at a level comparable to same-aged peers
b. % of infants and toddlers who improve functioning
c. % of infants and toddlers who did not improve functioning.
If children meet the criteria for a, report them in a. Do not include children reported in a in b or c.
![Page 47: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/47.jpg)
Illustration of 5 Possible Paths
0
10
20
30
40
50
60
70
1 6 11 16 21 26 31 36 41 46 51 56Age in Months
Sco
re
Maintained functioning comparable to age peers
Achieved functioning comparable to age peers
Moved nearer functioning comparable to age peersMade progress; no change in trajectory
Did not make progress
![Page 48: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/48.jpg)
48
3 OSEP Evidence Statements3 OSEP Evidence Statements
OSEP a Maintained functioning comparable to age
peers Achieved functioning comparable to age peersOSEP b Moved nearer functioning comparable to age
peers Made progress; no change in trajectoryOSEP c Did not make progress
![Page 49: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/49.jpg)
49
Two kinds of outcomes questions for Two kinds of outcomes questions for each child with a disabilityeach child with a disability
1. How is this child doing relative to the individualized outcomes that were established through the IFSP or IEP process?
2. How is this child doing relative to same-aged peers?
![Page 50: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/50.jpg)
50
Part C Indicators Part C Indicators Family OutcomesFamily Outcomes
Percent of families participating in Part C who report that early intervention services have helped the family Know their rights; Effectively communicate their
children’s needs; and Help their children develop and learn.
![Page 51: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/51.jpg)
51
Measurement guidelinesMeasurement guidelines
State selected data source, sampling permitted
% of families who report that early intervention services have helped their family in each of the three areas
![Page 52: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/52.jpg)
52
SPP/APRSPP/APRMeasurement implications Measurement implications Requires data collection from family
members/caregivers Data at one time period
Asks families to recognize that early intervention services contributed to the change or status listed.
Does not require family data to be linked with child data
![Page 53: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/53.jpg)
53
Key Questions Key Questions
How can Michigan provide the indicators being requested by OSEP?
What kinds of evidence statements does Michigan want to be able to produce for its own use?
What do the answers mean for measurement?
![Page 54: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/54.jpg)
54
Producing Data on the Outcomes
![Page 55: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/55.jpg)
55
Rolling up assessment data to Rolling up assessment data to the outcomesthe outcomes
Given: Different children will have different kinds of
assessment data. Also:
The same child may have several different kinds of assessment data.
Challenge: How to roll up multiple pieces of information (e.g., data from different assessment tools) to a single score?
![Page 56: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/56.jpg)
56
Children will be active and successful participants now and in the future in a variety of settings.
ECO Outcomes:
Overarching Goal:
Children have positive social relationships
Children acquire and use knowledge and skills
Children take appropriate action to meet their needs
Domains: Communication Cognition
Motor
Social-Emotional
Approaches to Learning
Content Areas:
Language Arts Math
Science Social Studies Art
Music
Note: Each of these can be broken down further into sub-areas
Processes: Memory Self-regulation Recognizing and interpreting sensory input
Listening Attending
Body Functions: Vision Seeing
Movement-flexibility-strength-postural response
Etc.
Speech production Etc.
Self Help
Example of how movement can be sub-divided
![Page 57: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/57.jpg)
Different assessments on different childrenDifferent assessments on different children
Roll up Process
Assessment 1
Assessment 2
Assessment 3
Assessment 4
Single Score
![Page 58: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/58.jpg)
Roll up Process
Assessment 1
Assessment 2
Parent Report
Informed Clinical Opinion
Single Score
Different pieces of information on the same childDifferent pieces of information on the same child
![Page 59: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/59.jpg)
59
Rolling up from the different Rolling up from the different assessmentsassessments
ECO is working on a scale that will provide a way to give a child a numerical rating on each outcome It is NOT an assessment tool. It assumes teachers and providers are
already using an assessment tool to collect information about how children are doing.
The scale is a way to take what those closest to the child know and put a number on it.
![Page 60: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/60.jpg)
60
What does the outcomes scale What does the outcomes scale look like?look like?
5 pt and a 7 pt version The precision question….
The high point (5 or 7) indicates outcome achieved at an age-expected level
The lowest point (1) indicates the farthest distance from age-expectations
![Page 61: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/61.jpg)
61
What does the outcomes scale What does the outcomes scale look like?look like?
Challenge is in how to anchor and provide the descriptors for the middle points on the scale.
Progress is moving up the scale between time 1 and time 2.
![Page 62: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/62.jpg)
62
Measuring the Family OutcomesMeasuring the Family Outcomes
ECO working on a tool ECO working on a document with
guidance for states that reviews key decisions and includes several options
![Page 63: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/63.jpg)
63
Staying in touchStaying in touch
Web site: the-eco-center.org Follow developments related to work of the Center Obtain ECO resource documents Obtain other related resources
Email: [email protected] ECO sends to existing listservs Can join the ECO mailing list
![Page 64: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/64.jpg)
64
VisionVision
To build a system where all the pieces will fit together smoothly
Meaningful outcomes data collected regularly that can used in programs by programs AND reported to the state (and feds)
![Page 65: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/65.jpg)
![Page 66: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/66.jpg)
Local data
State needs
![Page 67: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/67.jpg)
![Page 68: Outcomes and Evidence Statements](https://reader038.vdocuments.mx/reader038/viewer/2022103007/568146fb550346895db4362e/html5/thumbnails/68.jpg)
68
When Michigan can realize this vision, children and families will
reap the benefits.