Christina KasprzakECTA/ECO/DaSy

Lauren BartonECO/DaSy

Ruth Chvojicek WI Statewide Part B Indicator 7 Child Outcomes Coordinator

March 17, 2014

Encore Webinar: Improving Data, Improving Outcomes Conference

Where the Rubber Hits the Road:

Tools and Strategies for Using

Child Outcomes Data for

Program Improvement

Purposes

• To describe national resources for promoting data quality and supporting program improvement

• To share Wisconsin 619 experience and strategies to promote data quality and program improvement

2

Available Resources

Question Resource

Can I trust the quality of my data? Where are there red flags?

Pattern Checking Table

3

See the “using data” page under Outcomes Measurement on the ECTA website: http://www.ectacenter.org/eco/pages/usingdata.asp

Pattern Checking for Data Quality

Strategies for using data analysis to improve the quality of state data by looking for patterns that indicate potential issues for further investigation.

http://www.ectacenter.org/eco/assets/pdfs/Pattern_Checking_Table.pdf

Questions to Ask

• Do the data make sense?– Am I surprised? Do I believe the data?

Believe some of the data? All of the data?

• If the data are reasonable (or when they become reasonable), what might they tell us?

5

Available Resources

Question Resource

Can I trust the quality of my data? Where are there red flags?

Pattern Checking Table

What are your internal systems that support quality data?

Local Contributing Factors Tool

6

Local Contributing Factors Tool

Provides ideas about questions a local team would consider in identifying factors impacting performance.

Introductory video and document at: http://www.ectacenter.org/eco/pages/usingdata.asp

Available Resources

Question ResourceCan I trust the quality of my data? Where are there red flags?

Pattern Checking Table

What are your internal systems that support quality data?

Local Contributing Factors Tool

What strategies might support improvement in different outcomes for Part C?

Relationship of Quality Practices to Child and Family Outcome Measurement Results

8

Relationship of Quality Practices to Child and Family Outcome

Measurement Results

Designed to assist states in identifying ways to improve results for children and families through implementation of quality practices.

9

http://www.ectacenter.org/~pdfs/QualityPracticesOutcomes_2012-04-17.pdf

Available Resources

Question ResourceCan I trust the quality of my data? Where are there red flags?

Pattern Checking Table

What are your internal systems that support quality data?

Local Contributing Factors Tool

What strategies might support improvement in different outcomes for Part C?

Relationship of Quality Practices to Child and Family Outcome Measurement Results

How can I use my data to analyze and improve program practices?

Analyzing Child Outcomes Data for Program Improvement: A Guidance Table

10

Analyzing Child Outcomes Data for Program Improvement

• Quick reference tool• Consider key issues,

questions, and approaches for analyzing and interpreting child outcomes data.

http://www.ectacenter.org/eco/assets/pdfs/AnalyzingChildOutcomesData-GuidanceTable.pdf

Describes Steps in Process of Using Data for Program Improvement

• Defining analysis questions• Clarifying expectations• Analyzing data• Testing inferences• Data-based program improvement

planning

12

Guidance Table

13

14

USING DATA FOR STATE & LOCAL IMPROVEMENT WISCONSIN’S PART B

Ruth Chvojicek – WI Statewide Part B Indicator 7 Child Outcomes Coordinator

WISCONSIN T/TA SYSTEM

Department of Public Instruction

Indicator 6/7/12 Coordinators

CESA Program Support Teacher

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

State 11-12 Entry Rating Distribution

EXAMPLES FROM 2011-2012

WISCONSIN DATA REVIEW PROGRESSION

2011-2012

• Pilot – 20 LEAs• Delivery via technology• Pattern common discussion points informed 12-13 statewide

PD

2012-2013

• Offered group data reviews within each CESA region and several large LEA’s

• Focus on enhancing data quality (accuracy of rating)• Individual District Data Reports• Process for development of reports• District action steps: Assessment process, rating practice +

SAMPLE GRAPHS – ENTRY OR EXIT RATING DISTRIBUTION

SAMPLE – ENTRY RATING BY ELIGIBILITY

Outcome One 1 2 3 4 5 6 7Other Health Impairment 50.0% 0.0% 50.0% 0.0% 0.0% 0.0% 0.0%Significant Developmental Delay 30.0% 30.0% 40.0% 0.0% 0.0% 0.0% 0.0%Speech or Language Impairment 0.0% 0.0% 8.3% 4.2% 16.7% 16.7% 54.2%

Outcome Two 1 2 3 4 5 6Other Health Impairment 50.0% 0.0% 50.0% 0.0% 0.0% 0.0%Significant Developmental Delay 40.0% 20.0% 30.0% 10.0% 0.0% 0.0%Speech or Language Impairment 0.0% 12.5% 20.8% 20.8% 41.7% 4.2%

Outcome Three 1 2 3 4 5 6 7Other Health Impairment 50.0% 0.0% 50.0% 0.0% 0.0% 0.0% 0.0%Significant Developmental Delay 20.0% 30.0% 0.0% 10.0% 20.0% 20.0% 0.0%Speech or Language Impairment 0.0% 0.0% 8.3% 4.2% 12.5% 8.3% 66.7%

SAMPLE – ENTRY RATING COMPARISON

CESA #Outcome Two 1 2 3 4 5 6 71 28 24 7 3 5 2 12 4 57 26 25 26 14 133 1 10 33 39 28 40 184 1 5 6 42 75 32 315 5 6 27 86 77 966 4 3 4 25 50 557 1 9 31

Outcome Three

EXAMPLE RACE/ETHNICITYOutcome 1 1 2 3 4 5 6 7Asian 4.3% 18.4% 9.9% 9.2% 23.4% 15.6% 19.1%Black 6.4% 15.3% 11.3% 16.7% 20.1% 15.2% 15.2%Hispanic 4.9% 11.9% 12.1% 13.5% 21.5% 21.7% 14.5%American Indian Alaskan 2.7% 10.0% 15.5% 19.1% 24.5% 17.3% 10.9%Hawaiian Other Pacific Islander 0.0% 31.6% 21.1% 10.5% 31.6% 0.0% 5.3%Two or More Races 4.5% 7.2% 11.7% 18.9% 26.1% 18.9% 12.6%White 2.8% 9.5% 8.1% 13.2% 21.4% 22.2% 22.8%

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

Outcome 1 State 11-12 Entry

State BlackState HispanicState White

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

Outcome 1 CESA 1

CESA 1 BlackCESA 1 HispanicCESA 1 White

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

Outcome 1 CESA 2

CESA 2 BlackCESA 2 HispanicCESA 2 White

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

Outcome 1 District M 11-12 Entry

District M BlackDistrict M HispanicDistrict M White

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

Outcome 1 District B 11-12 Entry

District B BlackDistrict B HispanicDistrict B White

1 2 3 4 5 6 70

5

10

15

20

25

30

Outcome One 2011-2012/2012-2013Entry Rating Distribution Comparison

1 2 3 4 5 6 70

5

10

15

20

25

30

Outcome Two 2011-2012/2012-2013Entry Rating Distribution Comparison

1 2 3 4 5 6 70

5

10

15

20

25

30

35

Outcome Three 2011-2012/2012-2013 Entry Rating Distri-bution Comparison

BUT … OUTCOME ONE EXIT RATING

1 2 3 4 5 6 70.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

Outcome 1 Exit Rating Comparison

OUTCOME THREE

1 2 3 4 5 6 70.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Outcome 3 Exit RatingComparison

WISCONSIN DATA REVIEW PROGRESSION

2011-2012

• Pilot – 20 LEAs• Via technology• Pattern common discussion points informed 12-13 statewide PD

2012-2013

• Offered group data reviews within each CESA region and several large LEA’s• Focus on enhancing data quality (accuracy of rating)• Individual District Data Reports• Process for development of reports• District action steps: Assessment process, rating practice +

2013-2014

• Continue group data reviews all CESA regions• Continue working on data quality – plus…• Piloting conversation around settings and Indicator 7 progress• Statewide Stakeholder Workgroup – Discussion using Analyzing Child Outcomes

Data tool

Next Steps?

• Try out using these resources• Send feedback to ECO Center about the

new Analysis tool • What are your ‘take aways’ and next

steps related to analyzing your data for data quality and/or program improvement? (notes for State Team time)

35

36

Find more resources at: http://www.ectacenter.org

Questions? Comments?

• #6 to un-mute

37