PVAAS – Pennsylvania PVAAS – Pennsylvania Value-Added Value-Added
Assessment SystemAssessment System
http://www.portal.state.pa.us/portal/server.pt/community/pa_value-added_assessment_system_(pvaas)/8751
The TWO PVAAS MethodologiesThe TWO PVAAS Methodologies
Today
Looking Back/Evaluation…
Value-added Growth Reports For Cohorts of Students
Looking Forward/Planning…
PVAAS Projection Reports For Individual Students
and Cohorts of Students
NO individual student growth measure
PVAAS IS:
• A statistical analysis of existing assessment data (PSSA).
• A realistic measurement of a school’s progress/growth over time.
• Progress data to add to achievement data as schools are analyzing their work with students and making improvement plans.
PVAAS IS NOT:
• Another test.
• A method for providing a measure of growth for an individual student.
High Achievement
Average Achievement
Low Achievement
Current Data Collection Current Data Collection ApproachesApproaches
Achievement is based Achievement is based on each student’s ending point.on each student’s ending point.
PVAAS Adds Another Variable To Achievement…Progress
Below Growth
Standard
At or AboveGrowth
Standard
Progress is based Progress is based on each student’s starting pointon each student’s starting point..
Viewing Achievement Viewing Achievement and Progress Togetherand Progress Together
Scatter PlotsScatter Plots
Achievement and Growth - 4th Math
0
56
-20 0 20
PVAAS Growth Measure
% P
rofi
cie
nt
or
Ad
van
ced
Scatter Plot Quadrants Scatter Plot Quadrants and Their Meaningsand Their Meanings
Excelling
Underperforming but Improving
Achieving but Slipping
Underperforming and Slipping
Please review this document in Please review this document in your folder…your folder…
Example: Regional Scatter PlotExample: Regional Scatter Plot
PVAAS Growth Value
% P
rofici
ent
or
Advance
d -
PSSA
56
0
Achievement vs. Growth - 4th Grade Math
Scatter Plot ComparisonScatter Plot Comparison
State vs. IU 12 vs. YCSDState vs. IU 12 vs. YCSD
Math and ReadingMath and Reading
Grades 4 – 8 Grades 4 – 8
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 4 MathGrade 4 Math
Statewide Grade 4 Math
IU 12Grade 4 Math
York City SDGrade 4 Math
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 4 ReadingGrade 4 Reading
Statewide Grade 4 Reading
IU 12Grade 4 Reading
York City SDGrade 4 Reading
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 5 MathGrade 5 Math
Statewide Grade 5 Math
IU 12Grade 5 Math
York City SDGrade 5 Math
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 5 ReadingGrade 5 Reading
Statewide Grade 5 Reading
IU 12Grade 5 Reading
York City SDGrade 5 Reading
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 6 MathGrade 6 Math
Statewide Grade 6 Math
IU 12Grade 6 Math
York City SDGrade 6 Math
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 6 ReadingGrade 6 Reading
Statewide Grade 6 Reading
IU 12Grade 6 Reading
York City SDGrade 6 Reading
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 7 MathGrade 7 Math
Statewide Grade 7 Math
IU 12Grade 7 Math
York City SDGrade 7 Math
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 7 ReadingGrade 7 Reading
Statewide Grade 7 Reading
IU 12Grade 7 Reading
York City SDGrade 7 Reading
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 8 MathGrade 8 Math
Statewide Grade 8 Math
IU 12Grade 8 Math
York City SDGrade 8 Math
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 8 ReadingGrade 8 Reading
Statewide Grade 8 Reading
IU 12Grade 8 Reading
York City SDGrade 8 Reading
1111thth Grade Scatterplots Grade ScatterplotsThe PSSA-PVAAS Scatter Plots visually display the The PSSA-PVAAS Scatter Plots visually display the following data from the statewide distribution of following data from the statewide distribution of PSSA results for 2009-10:PSSA results for 2009-10:• AchievementAchievement
– The mean The mean predictedpredicted performance of the cohort. performance of the cohort. – The actual mean performance of all students in the state is The actual mean performance of all students in the state is
displayed for referencedisplayed for reference
•GrowthGrowth– School Effect School Effect ≈ Observed Mean – Predicted Mean≈ Observed Mean – Predicted Mean– Standardized School Effect = School Effect relative to its error – Standardized School Effect = School Effect relative to its error –
displays degree of confidencedisplays degree of confidence
The following two slides explain the meaning of The following two slides explain the meaning of each of the regions of an 11each of the regions of an 11thth grade scatter plot. grade scatter plot.
The Scatter Plot The Scatter Plot Coordinate System for Grade 11Coordinate System for Grade 11
Horizontal AxisGrowth Measure in
Standard Errors
Vertical AxisMean
Predicted Scaled Score Growth Descriptor
Boundaries (±2SEs)
2010 Mean Scaled Score
Scatter Plot Regions Scatter Plot Regions and Their Meanings for Grade 11and Their Meanings for Grade 11
Higher Performing
Above Expectations
Lower PerformingAbove
Expectations
Higher Performing
Met Expectations
Lower PerformingMet Expectations
Higher Performing
Below Expectations
Lower PerformingBelow
Expectations
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 11 MathGrade 11 Math
Statewide Grade 11 Math
IU 12Grade 11 Math
York City SDGrade 11 Math
Statewide to IU Region to DistrictStatewide to IU Region to DistrictGrade 11 ReadingGrade 11 Reading
Statewide Grade 11 Reading
IU 12Grade 11 Reading
York City SDGrade 11 Reading
Overview of PVAAS Overview of PVAAS ReportingReporting
PVAAS ReportingPVAAS Reporting District Level ReportingDistrict Level Reporting School Level ReportingSchool Level Reporting Grade Level ReportingGrade Level Reporting
• Reading (Grades 4-8, 11)Reading (Grades 4-8, 11)• Math (Grades 4-8, 11)Math (Grades 4-8, 11)• Science (Grades 4, 8, 11)Science (Grades 4, 8, 11)• Writing (Grades 5, 8, 11)Writing (Grades 5, 8, 11)
Subgroups Reporting (ex. IEP, ED, ELL)Subgroups Reporting (ex. IEP, ED, ELL) Performance Levels (Advanced, Proficient, Basic, Performance Levels (Advanced, Proficient, Basic,
Below Basic)Below Basic) Student Level ReportingStudent Level Reporting
• Reading, Math, Science, Writing (grades vary based on Reading, Math, Science, Writing (grades vary based on content area)content area)
Have all of our groups of Have all of our groups of students experienced one students experienced one year of academic growth year of academic growth
during the last school year?during the last school year?
Value-added Growth Reports
PVAAS Value-added CohortGrowth Descriptors – Grade 4 – 8
Math and Reading
Favorable Indicator
Estimated gain at or above growth standard. Students in this cohort have made at least one year growth.All schools can achieve this rating.
Caution Indicator
Estimated gain below growth standard but by less than one standard error.Students in this cohort have grown less than the standard.
Stronger Caution
Estimated gain below growth standard by more than one but less than two standard errors.Students in this cohort have fallen behind their peers.
Strongest Warning
Estimated gain well below growth standard by more than two standard errors.Students in this cohort have made little progress.
District/School Report ExampleDistrict/School Report Example
PVAAS Predictive School Report
How are our best students, our How are our best students, our average students, our average students, our
struggling students growing?struggling students growing?
Performance Diagnostic Performance Diagnostic ReportReport
Growth Standard MethodologyGrowth Standard MethodologyWhat the whiskers tell us…What the whiskers tell us…
Growth Standard
(One Year’s Growth)
Met the Growth Standard;Made One Year’s Growth
Yellow
Below Growth Standard;Less than One Year’s Growth
Rose
Exceeded Growth Standard;More than One Year’s Growth
Green
Predictive MethodologyPredictive MethodologyWhat the whiskers tell us…What the whiskers tell us…
Observed – Predicted
(One Year’s Growth)
NDD – No Detectable Difference Between Observed Score and
Predicted Score
BELOW Expectations -- Observed Score Was Less
Than Predicted Score
ABOVE Expectations -- Observed Score Was Greater
Than Predicted Score
Performance Diagnostic ReportPerformance Diagnostic Report
Performance Diagnostic ReportPerformance Diagnostic Report
What is the likelihood that a What is the likelihood that a student will be proficient on a student will be proficient on a
future PSSA exam?future PSSA exam?
PVAAS ProjectionsPVAAS Projections
Student Projection Methodology
• A separate longitudinal modeling process focused on the performance scores of individual students.
• The model uses all of the available data (both math and reading) to project future PSSA performances in Math, Reading, Science and Writing.
• Projections can be calculated for an individual student or groups of students.
PVAAS Student Projection ReportPVAAS Student Projection Report
Identified Students Meeting Search Requirements
District (Single Grade)District (Single Grade)Projection Summary ReportProjection Summary Report
PVAAS Fall 2009 Reporting: PVAAS Fall 2009 Reporting: Projections to Basic, Proficient, Advanced Projections to Basic, Proficient, Advanced
Grade PSSA
Last Taken
Math and Reading:Math and Reading: 3 3 4 or 5 4 or 5 4 4 5 or 6 5 or 6 5 5 6 or 7 6 or 7 6 6 7 or 8 7 or 8 7 7 8 8 8 8 11 11
Writing:Writing: 3 3 5 5 4 4 5 5 5 5 8 8 6 6 8 8 7 7 8 8 8 8 11 11
Science:Science: 3 3 4 4 4 4 NA NA 5 5 8 8 6 6 8 8 7 7 8 8 8 8 11 11
Grade PSSA
Last Taken
Grade PSSA
Last Taken
Approved:Approved: PA’s Proposal for Growth PA’s Proposal for Growth Model as Means to Make Model as Means to Make
AYPAYP