broad data analysis
DESCRIPTION
Broad Data Analysis. Child Outcome Data. Broad Analysis: Child Outcomes. Does our state’s data look different than the national data? Are our state child outcomes trends stable over time? Is the data trending upwards? Is the data trending downwards? - PowerPoint PPT PresentationTRANSCRIPT
Child Outcome Data
Broad Data Analysis
Broad Analysis: Child Outcomes
Does our state’s data look different than the national data?
Are our state child outcomes trends stable over time?Is the data trending upwards?Is the data trending downwards?
Is our state performing more poorly in some outcomes than others?
Are the child outcomes similar across programs?
What about data quality? Can we be confident in our data?
Child Outcomes: State vs. National
Child Outcomes: State vs. National
National Vs. State Meaningful Differences
Child Outcomes: National vs. State FFY11 and State FFY12
Data Quality – Trends
Virginia Trends
Virginia Trends
Outcome 3:Use of Appropriate Behaviors to Meet
Needs
Outcome 2: Acquisition and
Use of Knowledge and Skills
Virginia Trends
Child Outcome Results FFY11/FFY12
Meaningful Differences Calculation
Child Outcomes: National vs. State FFY11 and State FFY12
Child Outcomes: Local vs. State
Child Outcomes: Local vs. State
Child Outcomes: Local vs. State
Child Outcomes: Local vs. State
Child Outcomes: Local vs. State
Child Outcomes: Local vs. State
Data Quality ElementsCompleteness of data
number of children reported for the outcome/number who exited
Virginia’s results: average= 65%; range for Local Systems = 17% - 100%
Expected Patterns for Progress Categories
Virginia’s state date is within these parameters for all three outcomes
Child Outcomes State Trends Over TimeAs noted on previous slides, Virginia’s results do not show
wide variations which would trigger concerns about data quality
Category a Category e
0 <5%
>10% >65%
Family Outcome Data
Broad Data Analysis
Broad Analysis: Family OutcomesDoes our state’s data look different than the
national data?Are our state family outcomes trends stable
over time?Is the data trending upwards?Is the data trending downwards?
Is our state performing more poorly in some outcomes than others?
Are the family outcomes similar across programs?
What about data quality? Can we be confident in our data?
Family Outcomes: State Trends over Time
Family Outcomes: Local vs. State 2012-20134A: EI has helped the family know their rights
Family Outcomes: Local vs. State 2012-20134B: EI has helped the family communicate their children’s needs
Family Outcomes: Local vs. State 2012-20134C: EI has helped the family help their child develop and learn
Data Quality
The data analysis for Virginia’ survey is quite extensive, using rigorous data analysis standards.
Virginia’s response rate, like other states that use mailing as the means to disseminate the family survey, is lower than states who use other mechanisms.
Broad Infrastructure Analysis
Analysis MechanismsUse of federal monitoring tools and
proceduresCommunication with Local Systems through:
Ongoing TARegional MeetingsTrainingsSystem Manager Meetings
Monitoring ResultsRecord Reviews
QMRsIndividualized TA
Local System Contract Deliverables
Infrastructure Analysis:Information Sources
Local self-reportingStakeholder inputLSM surveysTraining/meeting evaluationsMonitoring/QMRDispute ResolutionsObservations through TA and
other interactions
Broad Infrastructure Analysis
Governance - MixedStrengths
Code of Virginia establishes infrastructureVICCState Interagency Agreement; interagency
partnershipsDBHDS local contract with Local Lead Agencies
specifies LS infrastructure requirementsWeaknesses
Variable job roles and skill sets of LSMsVariable strength and specificity of LLA/provider
contracts Variable support from LLA leadership
Broad Infrastructure AnalysisFiscal - Mixed
StrengthsAdditional state fundingMedicaid EI Services programTrainings on fiscal issuesStrong working relationship with fiscal office at the
state lead agency (DBHDS)Weaknesses
Variable fiscal skill set of LSMsCode of Virginia does not require fiscal commitment at
local levelVariety of Local Lead Agencies Inconsistent reporting of fiscal data at local levelLack of fiscal data in state data systemPerception/reality that there is not enough money
Broad Infrastructure Analysis
Quality Standards – Not sure Practice Manual articulates expected
practices, but these are not labeled quality practices
Monitoring and Accountability – StrengthLocal self-monitoring and supervision for
continuous improvement – variableProfessional Development – StrengthTechnical Assistance – StrengthData – Weakness
Closing Thoughts
Based on this broad review, did anything
strike you as an area of focus for our Systemic
Improvement Plan?