measures that matter achieve education trust adp leadership team meeting september 11, 2008 making...
TRANSCRIPT
Measures that MatterMeasures that Matter
AchieveEducation Trust
ADP Leadership Team MeetingSeptember 11, 2008
Making College and Career Readiness the Mission of High Schools
Measures that MatterMeasures that Matter
What we set out to do• Develop a framework for states looking to align their
assessment and accountability systems with the goal of college- and career-readiness for all
What we have created• Policymaker’s guide identifying key questions states need to
address along with options and promising practices• Additional tools and papers going deeper in key areas
How we did it• Working with an advisory group of state and national
policymakers, practitioners, and researchers and learning from you
Measures that MatterMeasures that Matter
Align High School Standards with the Demands of College and Careers
Measures that MatterMeasures that Matter
K-12 standards must be anchored in real world expectations
College ready = career ready Standards must be globally competitive and reflect 21st
century skills Vertical alignment through the grades Standards must reflect tough choices about what’s most
important—fewer, clearer, higher
Measures that MatterMeasures that Matter
Assure that Students Enroll in a Course of Study Aligned with College and
Career Readiness Standards
Measures that MatterMeasures that Matter
Rigorous course taking matters, especially for low-income and minority students
Graduation course requirements are critical lever for ensuring all students take a rigorous curriculum
Mandatory vs. “opt-out” strategies Monitor course taking patterns carefully Ensure sufficient number of teachers with knowledge and
skills to teach all students more rigorous courses Encourage all students to earn college credits while in
high school
Measures that MatterMeasures that Matter
What courses should students be required to take? Will the requirements be mandatory for all students or will
there be an opt-out provision? How will the state ensure consistent rigor across the state? Can the state create multiple but equally rigorous
pathways? How can the state invest in sufficient teacher and student
supports? Don’t let floor become the ceiling—encourage student to
aim higher (e.g., earning college credit in high school)
Measures that MatterMeasures that Matter
Measuring Student Learning: A College- and Career-Ready
Assessment System
Measures that MatterMeasures that Matter
The Problems: Too many tests and the purposes are disconnected State high school assessments not challenging enough to measure
college and career readiness…so no currency with higher education and the business community
Assessments do not measure the full range of college- and career-ready knowledge and skills (such as research, analysis, critical thinking, and collaboration)
Tests are not generating diagnostic information for educators and students
Measures that MatterMeasures that Matter
The Vision: Testing should be streamlined Proficient should mean prepared – anchored in college and
career readiness Tests should measure the full range of college and career
readiness skills –better summative and more performance based assessments
High school tests should open the door for students to higher education and good jobs
Testing should support good teaching
Measures that MatterMeasures that Matter
What tests do high school students in your state take right now?
• State? District? Other?
What are the purposes of each?• Instructional improvement• Student credentialing• School accountability
Where could tests be subtracted as new ones are added?
Measures that MatterMeasures that Matter
Does your state have a college and career readiness “anchor assessment” in high school?
• If so, is there a statewide readiness score used for placement into credit-bearing courses? Careers?
• If not, what is the strategy for building such a measure?
Measures that MatterMeasures that Matter
Benefits Designed to measure state
standards Known quantity in K-12
system Won’t dramatically
increase testing time
Challenges Not designed to measure college and
career readiness Limited credibility with higher
education/employers Most feasible if tests given in 11th
grade
End-of-Grade Tests
Measures that MatterMeasures that Matter
Benefits Currency with higher
education and students May not increase testing
burden for many students Common and comparable
results across states
Challenges Not designed to align with state
standards, school curriculum Lack of coherence with rest of
assessment system Require augmentation
College Admissions Tests
Measures that MatterMeasures that Matter
Benefits More useful to educators—
aligned to course content Can be used to monitor quality
and consistency of course rigor More flexibility—students
take test when they take course
Challenges May increase overall testing time To assess college/career readiness,
need EOCs in higher level courses Timing of when students take test
can be challenge for postsecondary use
End-of-Course Tests
Measures that MatterMeasures that Matter
How does the state ensure that the anchor assessment has credibility with higher education and employers?
• How does test development process need to change?• What studies need to be done to demonstrate a connection
between performance on the test and success in postsecondary?
What type of stakes or incentives can or should be attached to the anchor assessment to make it meaningful for students?
Measures that MatterMeasures that Matter
Collaboration to develop and use the tests – in higher education
• California Department of Education (CDE) and California State University (CSU) modified 11th grade assessment
• Placement tests waived for students who score college-ready level and continue to take challenging courses
• Students who do not score at readiness level can get targeted support
Measures that MatterMeasures that Matter
Collaboration to develop and use the tests – with employers
• Hawaii K-12, higher education and business leaders working together
• Encouraging students to “opt-in” into college and-career ready course of study
• Tying employer incentives to “opt-in” diploma, which requires Algebra II assessment
Secured commitments to access apprenticeship programs (state carpenters’ union)
Measures that MatterMeasures that Matter
Are traditional tests able to measure full set of college and career readiness standards?
Yes and no We can make summative tests better But some important skills better using performance-based
measures (team projects, essays, portfolios, demonstrations, presentations and exhibitions)
Measures that MatterMeasures that Matter
Benefits Limits of paper and pencil
test to measure full range of college and career readiness skills
Support richer instruction and deeper student engagement
Challenges Cost Workload burden on
teachers Consistency of scoring
Performance Measures
Measures that MatterMeasures that Matter
Is testing at the end of the year or grade sufficient? No…not if goal is to improve teaching and learning How can the state ensure local schools and districts have
access to high quality interim assessments throughout the school year?
Measures that MatterMeasures that Matter
Provide High Quality Curriculum and Teacher Support Materials
Measures that MatterMeasures that Matter
The problem: Curriculum development has been left to individual
districts, schools, and teachers, leading to uneven—and inequitable—course content and instruction.
Source: Prospects (ABT Associates, 1993), in “Prospects: Final Report on Student Outcomes”, PES, DOE, 1997.
Students in Poor Schools Receive ‘A’s for Work That Would Earn ‘Cs’ in Affluent Schools
The Odyssey Ninth GradeHigh-level Assignment
Comparison/Contrast Paper Between Homer's Epic Poem, The Odyssey and the Movie "0 Brother Where Art Thou"
By nature, humans compare and contrast all elements of their world. Why? Because in the juxtaposition of two different things, one can
learn more about each individual thing as well as something about the universal nature of the things being compared.
For this 2-3 page paper you will want to ask yourself the following questions: what larger ideas do you see working in The Odyssey and "0
Brother Where Art Thou"? Do both works treat these issues in the same way? What do the similarities and differences between the works
reveal about the underlying nature of the larger idea?
The Odyssey Ninth GradeLow-level Assignment
Divide class into 3 groups:
Group 1 designs a brochure titled "Odyssey Cruises". The students listen to the story and write down all the places Odysseus visited in his adventures, and list the cost to travel from place to place.
Group 2 draws pictures of each adventure.
Group 3 takes the names of the characters in the story and gods and goddesses in the story and designs a crossword puzzle.
Chicago Public Schools: Many teachers report spending huge amounts of instructional time on ACT practice during 11th-grade core courses
23 23 21
40 3958
0
20
40
60
80
100
Science Math English
Per
cen
t o
f T
each
ers
20 or more hours
12 to 20 hours
Source: Consortium for Chicago School Research, From High School to the Future: ACT Preparation – Too Much, Too Late, Why ACT Scores are Low in Chicago and What It Means for
Schools, May 2008.
“Exams are being used as instructional tools, although they are not designed
for learning. These practices steal instructional time that could be used
for deep, challenging course work that actually would prepare students for
the ACT and for college study.”
Source: Consortium for Chicago School Research, From High School to the Future: ACT Preparation – Too Much, Too Late, Why ACT Scores are Low in Chicago and What It Means for
Schools, May 2008.
Measures that MatterMeasures that Matter
The Vision: Getting students into courses with the right names is just
the beginning State policymakers need to provide greater leadership on
the issues of curriculum and instructional supports States should invest in course redesign strategies to
deliver the same content in more accessible and applied ways
Measures that MatterMeasures that Matter
Who provides detailed guidance to teachers on what they are expected to teach?
• Who devises the curriculum? Do ALL teachers have access to detailed curriculum frameworks?
• What about well-designed units, lessons and assignments, and links to supplementary resources?
Are teachers in your state expected to “compose” and “conduct”?
Measures that MatterMeasures that Matter
How do your teachers in your state know what “good enough” student work looks like?
• Has your state created exemplar or “anchor” assignments, along with scoring rubrics and examples of scored student work? Does EVERY teacher have access to model assignments?
• If these are local responsibilities in your state, how does the state ensure adequate guidance and resources for teachers in districts that lack the capacity to develop high-quality materials?
Measures that MatterMeasures that Matter
Is your state prioritizing the “redesign” of priority courses for organizing the same content into different packages?
• A LOT of energy and resources on this effort in higher education.• If CTE is part of the plan, how is the state ensuring these programs
include the academic rigor that students need? • Do CTE programs meet the Perkins “Program of Study”
designation. If not, what’s the plan for assuring alignment?
Measures that MatterMeasures that Matter
Getting Everybody Pulling in the Same Direction: An Information and
Accountability System Focused on College- and Career-Readiness
Measures that MatterMeasures that Matter
The Problems: Stakeholders do not have access to critical information about
college- and career-readiness. Accountability systems are not based on what it means to be
college- and career-ready, nor are they based on what happens throughout high school.
Accountability systems do not establish expectations for performance that reflect where we need our schools and students to be.
Accountability goals are perceived as something to meet to avoid state interference rather than something meaningful to work toward.
Measures that MatterMeasures that Matter
The Vision: Stakeholders need real-time information to support decision-
making. Accountability systems need to reflect key indicators of
progressing toward, achieving, and exceeding college- and career-readiness.
States should set stretch goals that are ambitious and achievable.
Accountability systems should provide incentives for students and schools that meet expectations – and real consequences for continually failing to meet expectations.
Measures that MatterMeasures that Matter
What information do stakeholders need to support college- and career-readiness? Are data available to meet these information needs?
• How many students are taking and successfully completing a college- and career-ready course of study? Where are there gaps between groups and districts?
• How are students performing on assessments of college- and career-readiness?
• How many students are graduating, and what happens after they leave high school?
Measures that MatterMeasures that Matter
Do accountability indicators reflect the goal of college- and career-readiness?
Measures that MatterMeasures that MatterAlong the way toward college and career readiness
Along the way toward college and career readiness
Exceeding college- and career readiness
Course completion and success
-Timely credit accumulation-Credit recovery
-Successful completion of college- and career-ready course of study
-Participation in AP, IB and dual enrollment
Achievement -Performance on aligned assessments early in high school
-Meeting standards on anchor assessment-Postsecondary remediation rates
-College-level performance on AP and/or IB exams
Attainment -Graduation -Earning a college- and career-ready diploma
-Earning credits in dual enrollment courses-Application to and enrollment in postsecondary
Measures that MatterMeasures that Matter
Does the accountability system include stretch goals and progress targets that communicate a clear path from where students, schools, and districts are to where we need them to be?
• Are all schools and students, regardless of where they stand relative to goal of college- and career-readiness, expected to make gains?
Measures that MatterMeasures that Matter
80.6% 81.5% 82.4% 83.3% 84.2% 85.1% 85.9% 86.8% 87.7% 88.6% 89.5%
52.9%56.6%
60.2%63.9%
67.5%71.2%
74.9%78.5%
82.2%85.8%
89.5%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Gra
du
ati
on
-Ra
te T
arg
et
Graduation-Rate Progress Targets for Two Tennessee Schools
Data Source: Tennessee Department of Education – Office of Innovation, Improvement and Accountability
Measures that MatterMeasures that Matter
Does the accountability system set expectations for multiple indicators while maintaining focus on college- and career-ready graduation?
• This can be accomplished through performance indices that weight the most important indicators and most important levels of performance most heavily, but credit other indicators and other performance levels as well.
Measures that MatterMeasures that Matter
Weights for different student outcomes in Louisiana’s accountability system
Measures that MatterMeasures that Matter
Do the metrics used in the accountability system evaluate both current-year performance and progress?
• Status: What percentage of students is meeting college- and career-ready standards this year?
• Growth-to-standards: What percentage of students is on trajectory to meet standards by a certain date?
• Improvement: Are schools making sufficient year-to-year improvement in the percentage of students meeting standards to meet the goal by a set date?
Measures that MatterMeasures that Matter
What are the positive incentives– for schools and students—to work hard and reach the college- and career-ready level and beyond?
• We cannot motivate excellence only by avoiding sanctions.