critical markers of high quality child outcomes data eco advisory board march, 2012
TRANSCRIPT
![Page 1: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/1.jpg)
Critical Markers of High Quality Child Outcomes Data
ECO Advisory BoardMarch, 2012
![Page 2: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/2.jpg)
Topics
• Why identify critical markers of high quality data?
• Anticipated use• 3 areas of focus• Draft critical markers• Where these might be
discussed in the APR• Reactions?
2Early Childhood Outcomes Center
![Page 3: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/3.jpg)
Looking for…
Proposed markers that suggest a state’s EC outcomes systems may produce high quality data
Use markers to:
- systematically track over time - track internally (within states)
- to produce a national picture
3Early Childhood Outcomes Center
![Page 4: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/4.jpg)
Anticipated Use
• Tracking state progress (based on APR info)• Compare data from each state to a series of
critical markers for summaries– X % of states met standard on each marker nationally– X% of states met standard on 7 out of 10 markers
• NOT a state by state report card• Share detailed info with each state, upon
request
4Early Childhood Outcomes Center
![Page 5: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/5.jpg)
3 Areas
• Completeness of data
5Early Childhood Outcomes Center
![Page 6: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/6.jpg)
Completeness of data
• Missing data concepts– Are whole forms missing? (Have data
from all kids expect to have data from)
– Are the forms that you have complete?
6Early Childhood Outcomes Center
![Page 7: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/7.jpg)
3 Areas
• Completeness of data
• Accuracy of data
7Early Childhood Outcomes Center
![Page 8: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/8.jpg)
Accuracy of Data
• Do the data reflect the “true” performance of children in the program– Unknowable, so….
• Are the patterns in data what you would expect to see if the data were accurate?
• Do they make sense? Or, are there red flags that raise more questions?
8Early Childhood Outcomes Center
![Page 9: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/9.jpg)
3 Areas
• Completeness of data
• Accuracy of data
• State efforts related to tracking quality of data
9Early Childhood Outcomes Center
![Page 10: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/10.jpg)
Criteria for Selection of Markers
• Important – necessary for or indicative of
high quality data• Accessible to ECO
– Information to determine presence or absence of marker is available
10Early Childhood Outcomes Center
![Page 11: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/11.jpg)
Draft Markers: Completeness
• State calculates and publicly reports number of children missing outcomes data
• Percent of missing outcomes data is less than 5%. – Is this reasonable?
• Percent of missing data by proxy calculation is less than. – 40% of exiters for Part C– 20% of child count for 619
11Early Childhood Outcomes Center
![Page 12: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/12.jpg)
Part C percent of exiters 2009-10
12Early Childhood Outcomes Center
![Page 13: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/13.jpg)
Draft Markers: Accuracy/Patterning
• State % in a is not overly high (GT 5%)• State % in b is not overly low (LT 5%) or high (GT 50%)• State % in c is not overly low (LT 5%) or high (GT 50%)• State % in d is not overly low (LT 5%) or high (GT 50%)• State % in e is not overly low (LT 5%) or high (GT 65%)
13Early Childhood OutcGTomes Center
![Page 14: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/14.jpg)
Knowledge and SkillsPart B 619 proportion of children that make no
progress (progress category a)
14Early Childhood Outcomes Center
![Page 15: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/15.jpg)
Knowledge and SkillsPart B 619 proportion of children who improved functioning but not sufficiently to move nearer to functioning comparable to same age
peers(progress category b)
15Early Childhood Outcomes Center
![Page 16: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/16.jpg)
Knowledge and SkillsPart B 619 proportion of children who improved functioning to a
level nearer to same age peers but did not reach it(progress category c)
16Early Childhood Outcomes Center
![Page 17: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/17.jpg)
Knowledge and SkillsPart B 619 proportion of children who improved functioning to
reach a level comparable to same aged peers(progress category d)
17Early Childhood Outcomes Center
![Page 18: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/18.jpg)
Knowledge and SkillsPart B 619 proportion of children who maintained
functioning at a level comparable to same aged peers(progress category e)
18Early Childhood Outcomes Center
![Page 19: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/19.jpg)
Percent of states meeting the Critical Markers for 2009 - 10
19Early Childhood Outcomes Center
Accuracy of data Completeness of data
Both
Part C 66 59 47
Part B 619 63 66 41
![Page 20: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/20.jpg)
Draft Markers: State Quality Review of Data
• State conducts data quality checks• State’s own analyses provide evidence of
high quality data
20Early Childhood Outcomes Center
![Page 21: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/21.jpg)
21Early Childhood Outcomes Center
Completeness/ Missing Data
Accuracy/ Patterns
Location in Suggested APR Templatehttp://www.fpg.unc.edu/~eco/pages/fed_req.cfm#ECOSuggestedFormats
![Page 22: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/22.jpg)
22Early Childhood Outcomes Center
State Quality Review of Data
Location in Suggested APR Templatehttp://www.fpg.unc.edu/~eco/pages/fed_req.cfm#ECOSuggestedFormats
![Page 23: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/23.jpg)
Questions for you
• Are these markers important?– Related to high quality data– Things you value and might track or are already tracking
them?– Would tracking them help you improve your system?
• Is there anything else you can think of as a good marker of high quality data
23Early Childhood Outcomes Center
![Page 24: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/24.jpg)
Summary – Draft Critical Markers
• State calculates and publicly reports number of children missing outcomes data
• Percent of missing outcomes data is less than 5%• Percent of missing data by proxy calculation is less than X • State % in a is not overly high (GT 5%)• State % in b is not overly low (LT 5%) or high (GT 50%)• State % in c is not overly low (LT 5%) or high (GT 50%)• State % in d is not overly low (LT 5%) or high (GT 50%)• State % in e is not overly low (LT 5%) or high (GT 65%) • State conducts data quality checks• State’s own analyses provide evidence of high quality data
24
![Page 25: Critical Markers of High Quality Child Outcomes Data ECO Advisory Board March, 2012](https://reader035.vdocuments.mx/reader035/viewer/2022062409/5697c0011a28abf838cc2811/html5/thumbnails/25.jpg)
Comments, Reactions, Questions?
25