success for all: multiple measures for college & career readiness

Success for All:Multiple Measures for College & Career Readiness

Presented at the California Department of Education’s Public Schools Accountability Act Advisory Committee Meeting April 15th, 2014

The Linked Learning Alliance

A statewide coalition dedicated to improving California’s high schools and preparing students for success in college, career, and life.

State & Local

Agencies

Employers

Industry & TradeAssoc.

Community Org.

K-12

Higher Education

The Linked Learning ApproachLinked Learning integrates:

• rigorous academics• technical training• real-world learning opportunities• student supports

College and Career Readiness

Most skilled jobs with living wages require training beyond high school.

Neither a college nor career technical training may be appropriate for all students, but both are relevant to the aspirations of most students.

CareerHigh

SchoolCommunity

College

On-the-Job

Training

Apprentice

University

CollegeReady

CareerReady

Eligibility for postsecondary studies without remediation.Academic content knowledgeAcademic behaviorsCognitive strategies

Eligibility for gainful, skilled employment or further training.

Technical skills and knowledgeWorkplace behaviors

Contextual application of academic and technical skills/knowledge.21st Century skills and ‘soft’ skills

College and career are neither synonymous nor mutually exclusive.

Common Core Implementation

$1.25 Billion

Local Control & Accountability Planning

$2.1 Billion

SB 1070 CTE Pathway Program

$48 Million

Career Technical Programs(CPA, SSP, AgTech, CleanTech)

$40 Million+$30 Million

(Proposed for 2014-15)

+$52 Million(Proposed for 2014-15)

$250Million

$384Million

Regional Occupation Centers & Programs

Career Pathways Trust

Career Technical Funding Statewide Implementation Grants

+$48 Million(Proposed for 2014-15)

State policy has begun to acknowledge this reality.

This convergence builds upon several years of legislative activity working to better align career and college preparation.

SB 611 Curriculum Integration & Articulation

AB 790 Local Exemplars & Scale

SB 612 AB

1304

Teacher Training

AB 2648

Regulatory Alignment

SB 1458

Assessment & Accountability

SB 1X1SB

1354

Building Block Academies

AB 2211AB 86

Work Based Learning

College Readiness: One size does not fit all

In September, the committee heard about several potential indicators:• A-G Completion• Early Assessment Program (EAP)• Completion of Algebra II or above

Variable observation• None were stable or reliable across school sites

Other discussed indicators:• SAT/ACT, AP/IB and dual enrollment • Are not universally available and/or come at a cost• Likely variation in reliability across school sites

A multiple measures approach is the most realistic, equitable, and feasible.

Career readiness is no different.

Demonstrate workforce readiness through acquisition of sector specific competencies and 21st century skills.

Many career-relevant indicators are reported to the state:• Perkins• CALPADS

More are gaining momentum in districts through:• AB 790 Linked Learning Pilot Program (IEBC)• Career Pathways Trust (IEBC and Cal-PASS+)• CORE districts (School Quality Improvement System)• Local Control and Accountability Plan development

Career Indicators

Career Pathways Trust• # who have successfully participated in the career pathways program

(successfully participated is defined by a “C” grade or better in all pathway courses).

• # participating in internships, work-based learning, mentoring, job shadowing opportunities, work experience, and student leadership organizations as part of the career pathways program.

• # who received a high school diploma.• # who received a nationally recognized or state approved career technical

education (CTE) certificate.

Career Indicators

Career technical education sequences or programs of study that align with state board-approved career technical educational standards and frameworks, including:• Aspirational ROC/ROP

Each occupational sequence shall do all of the following: (1) Result in an occupational skill certificate developed in cooperation with the

appropriate employer advisory board. (2) (2) Provide prerequisite courses that are needed to enter apprenticeship or

postsecondary vocational certificate or degree programs. Where possible, sequenced courses shall be linked to certificate and degree programs in the region.

(3) (3) Focus on occupations requiring comprehensive skills leading to high entry-level wages or the possibility of significant wage increases after a few years on the job, or both.

(4) (4) Offer as many courses as possible that have been approved by the University of California as courses meeting the "A-G" admissions requirements.

Local Control Accountability Plan

• California Partnership AcademiesCareer technical education courses offered at each grade level at the academy that are part of an occupational course sequence that targets comprehensive skills and that does the following: (A) Provides career technical education in high skill occupations of regional and local

economic need. (B) Focuses on occupations requiring comprehensive skills leading to higher than entry-

level wages, or the possibility of significant wage increases after a few years on the job, or both.

(C) Provides a sequence of courses that build upon each other in knowledge, skill development, and experience, and ends in a capstone course that includes an internship component.

(D) Prepares pupils for employment and postsecondary education. Sequenced courses shall be linked to certificate and degree programs in the region, where possible.

(E) Whenever possible, prepares pupils for industry-recognized certifications. (F) Whenever possible and appropriate, offers career technical education courses that

also meet the subject requirements for admission to the California State University and the University of California.

Local Control Accountability Plan

• Linked Learning(1) A multiyear, comprehensive high school program of integrated academic and

technical study that is organized around a broad theme, interest area, or industry sector, including, but not necessarily limited to, the industry sectors identified in the model standards adopted by the state board.

(2) A program that ensures that all pupils have curriculum choices that will prepare them for career entry and a full range of postsecondary options, including two- and four-year college, apprenticeship, and formal employment training.

Local Control Accountability Plan

• Linked Learning(1) An integrated core curriculum that meets the eligibility requirements for

admission to the University of California and the California State University and is delivered through project-based learning and other engaging instructional strategies that intentionally bring real-world context and relevance to the curriculum where broad themes, interest areas, and career technical education are emphasized.

(2) An integrated technical core of a sequence of at least four related courses, that may reflect career technical education standards-based courses, that provide pupils with career skills, that are aligned to and underscore academic principles, and to the extent possible fulfill the academic core requirements.

(3) A series of work-based learning opportunities that begin with mentoring and job shadowing and evolve into intensive internships, school-based enterprises, or virtual apprenticeships.

(4) Support services, including supplemental instruction in reading and mathematics, that help pupils master the advanced academic and technical content that is necessary for success in college and career.

Local Control Accountability Plan

Data Constraints & Opportunities

The State Board of Education can only require currently reported data

• CALPADS• Perkins

But this doesn’t preclude voluntary reporting of additional data.

New, non-state data systems can offer valuable indicators of readiness among participating schools and districts:

• IEBC• Cal-PASS+• School Quality Improvement System

(CORE/Gardner Center)

the API!!

Categorizing Potential Indicators

Potential Voluntary Indicators

Course & Program Indicators

Demonstration Indicators

Postsecondary Experience Indicators

Guiding Principles

College and career readiness• Not either/or

Subsidiarity• Aligned with LCFF/LCAP• Encourage innovation to improve practices and programs statewide• Responsiveness to unique local conditions and realities.

Transparency and relevance• Clear and meaningful to students, families, educators, employers, etc.

Emphasis on student performance over student participation.

Align with, and exceed, readiness standards in other states.

Course & Program Indicators

Reflect satisfactory performance in a specific program of study and the type of knowledge acquired.

May reflect minimum eligibility requirements for postsecondary programs (college, on-the-job training, apprenticeships, etc.).

‘College-Ready’ Course & Program Indicators

A-G Completion• Indicates preliminary eligibility

AP/IB Course Completion• Signal of more rigorous course of study• Research has linked course participation to improved performance in

similar courses at college level.

CTE Pathway CompletionSequence of 3 or 4 courses within an industry theme

• Central component of career academies• Used in 6 states’ accountability systems as a career readiness option

Decades of research have shown connection between career academy completion and improved student outcomes:

• Graduation and postsecondary enrollment• Earnings• Credit completion and ‘a-g’ course completion

Could meet basic Local Control & Accountability Plan requirement.

Introduction to Engineering Design

Digital Electronics

Computer Integrated Manufacturing

Sample Engineering & Design Course Sequence

CTE Course Completion (within a pathway)

Whereas the a-g course sequence is largely ‘all or nothing’ for UC eligibility purposes, discrete CTE courses aligned in a pathway sequence reflect incremental career preparation via technical skill development.

Course 1 Course 2

Introduction to Engineering Design• 3D Modelling• Technical

Documentation

Digital Electronics• Circuitry• Simulation design

& testing• Electronics

troubleshooting

CTEPathway

College Career

2 CTE courses

CTE course

None of the above

None of the above

800

600

400

200

A-G

&

Distribution of points on the index (conceptual example for illustrative purposes)

In this scheme, an algorithm could average college and career indicators, with calibration as necessary.

= 700 points

+800A-G -2 600

2

Demonstration Indicators

Completion of activities that establish a student’s level of experience or competence.

• May meet minimum eligibility requirements for postsecondary training programs or employment.

College• EAP Ready or EAP Conditional

Career • Work-based learning experiences

Work-Based Learning Experiences

Apprenticeships• Standards-based,

industry recognized, and often prerequisite for skilled jobs

• Includes clear sequence of experiences aligned to curriculum

• Supervisor assessment of readiness

WorkAbility• Aligned to Individual

Education Plan (IEP) for students with disabilities

• Supervisors assessment of job readiness

Work Experience Education• Inconsistent standards and quality• Not all are curriculum-aligned or include employer/supervisor

assessment• Could be strengthened in statute and regulations

Only a handful of states currently use work-based learning as an option for career readiness in their accountability systems.

LCAP Implications (when standards-aligned):• Could supplement college- and career-ready baseline• Could be an additional pupil outcome reported in the plan• Career-relevance and connection to curriculum reinforce student

engagement and content mastery.• Contextualized learning can reinforce English language acquisition

Industry Certification

Demonstrates technical competency• Employer recognized, but not applicable in all sectors or industries

Schools report acquisition of certifications, but no official list. CDE has been surveying districts to learn which certifications are reported

• Ex. Certified Organic Farmer, Veterinary Technician, Certified Nurse Assistant, Microsoft Certified, Collision Repair & Refinishing

Requires creation of authoritative list• Ensures level of quality and rigor• Common practice in other states

LCAP Implications:• Could be additional pupil outcome

included in reporting.

CTEPathway

College Career

2 CTE courses

CTE course

None of the above

None of the above

800

600

400

200

A-G+ EAP Ready

&

CTEP + Ind. Cert or WBL

WBL EmployerAssessed

Industry Cert.

CTE course

EAP Conditional

1000

Raising the bar on readiness.

A-G

A-G -2

A-G or EAP

Ready

Postsecondary Experience Indicators

Success in college-level courses and/or advanced standing upon college entry

• If college-level CTE coursework, also demonstrates career preparation

Some version of these types of indicators are used in over 20 states as a reporting option for accountability purposes.

Degree and Transfer-eligible Units (DTU)

Courses that require or demonstrate college-level math or English.

Includes credit earned from IB/AP exams and college coursework completed through dual/concurrent enrollment.

Place students at advanced standing upon entry to college

Provides early exposure to college setting and/or college-level performance expectations

Can lead to increased rates of colleges persistence and completion among all students

College-level CTE course completion

Emerging research shows promise for CTE dual enrollment and students’ postsecondary outcomes.

Demonstrates college and career readiness • Shows both college-level competency and employment training

LCAP Implications:• Could supplement college- and career-ready baseline• Could be an additional pupil outcome reported in the plan• Rigor and career-relevance can reinforce student engagement, school

climate perceptions, and content mastery.• Contextualized learning can reinforce English language acquisition.

Diversifying reporting options

In this scheme, some indicators could combine college and career, reducing or eliminating the need to average and calibrate.

A-G or EAP

Ready

CTEPathway

College Career

2 CTE courses

CTE course

None of the above

800

600

400

200

A-G+ EAP Ready

&

CTEP + Ind. Cert or WBL

WBL EmployerAssessed

Industry Cert.

CTE course

EAP Conditional

1000A-G

+3 DTU

6 DTU

30 DTU

12 DTU

3 DTU

1 College CTE

Course

College & Career

WBL + 1 College

CTE

WBL + 1 College

CTE (Math or ELA pre-req.)

Diversifying reporting options

In this scheme, some indicators could combine college and career, reducing or eliminating the need to average and calibrate.

A-G or EAP

Ready

CTEPathway

College Career

2 CTE courses

None of the above

800

600

400

200

A-G+ EAP Ready

&

CTEP + Ind. Cert or WBL

WBL EmployerAssessed

Industry Cert.

CTE course

EAP Conditional

1000A-G

+3 DTU

6 DTU

30 DTU

12 DTU

3 DTU

College &

Career

WBL + 1 College

CTE1000 800

= 900 pts

+

= 800 pts.2

Potential Voluntary Indicators

Supports local control and accountability, through recognition of alternative, indicators or measures.

• Incentivizes innovation – drives state towards better measures • A number of promising indicators are already emerging• Alternative reporting and accountability systems could be integrated

What would be required?• Evidence of effectiveness• Development of evaluation criteria and review process to obtain

PSAA/SBE approval, as well as determine point values.

National Career Readiness Certificate (NCRC)Can be earned through successful completion of 3 WorkKeys exams (offered by ACT):

• Applied Mathematics• Locating Information• Reading for Information

Jobs are profiled to determine the proper skill levels necessary for successful execution of a job or internship.

Four skill levels are possible:• Bronze 35% of jobs profiled• Silver 65%• Gold 90%• Platinum 100%

Used as an option for career readiness in 13 state accountability systems.

Integrated Pathway Completion

College and Career Pathway• A-G and CTE• At least two CTE courses meet A-G requirements

Increases access to college-ready curriculum for those traditionally tracked into career tech programs.

Increases job readiness and soft/tech skill development for college-bound students.

LCAP Implications:• Clearly articulates and integrates college and career ready ideals • Career-relevance and connection to curriculum reinforce student

engagement and content mastery, improving academic performance outcomes.

National Academy Foundation Certification

Student-level certification used in almost 100 programs in California.

Includes satisfactory completion of three standardized elements:1. End-of-course exams in four pathway courses2. Four project-based assessments3. Supervisor-assessed internship

• Could potentially function as an independent indicator due to employer validation, standardization of assessment, and alignment to academic content standards.

Completion of Certified Linked Learning Pathway

Includes completion of an college and career ready curriculum sequence, in addition to a work-based learning experience.

Speaks to the rigor and relevance of a program, as well as the curricular integration across academic, technical and workplace learning.

Encourages teaching practices that are effective and evidence-based, as well as equity of access and continuous program improvement.

School Quality Improvement System (SQIS)

School accountability system for districts participating in the ESEA waiver through the California Office to Reform Education (CORE).

Data-driven accountability and continuous improvement system:• Includes academic, socio-emotional and culture/climate indicators• Indicators are evidence-based in their importance for academic

achievement and life success (ex. employment, wages, avoidance of risky behavior)

Data aggregation performed by third-party.• Gardner Center at Stanford University

Schools must meet one of the following benchmarks:• 90% on SQI Index; or• Improvement along index: 2% in 2 years, 4% in 4 years

Program EnrollmentDual Enrollment EmploymentProgram Completion Internships

Work-Based Learning Mentoring Job Shadowing Associate Degrees

Pathway-enrolled GraduatesWork Experience Industry Certification

Student Leadership Organizations Training Program Entry

4-year Transfer

Data Reporting for the Career Pathways Trust

Career Pathways Trust: Data Reporting Systems

Individual reporting elements, or aggregates thereof, could be drawn from one of the two reporting systems to meet college and career-readiness accountability requirements.

Institute for Evidence-Based Change • Nine districts currently report to this system.• 58 more LEAs/COEs will utilize as participants in the AB 790 Linked

Learning Pilot Program

Cal-PASS+• Many LEAs already provide information to this longitudinal system

Diversifying options with alternative data reporting.

A-G or EAP

Ready

CTEPathway

College Career

2 CTE courses

None of the above

800

600

400

200

A-G+ EAP Ready

&

CTEP + Ind. Cert or WBL

WBL EmployerAssessed

Industry Cert.

CTE course

EAP Conditional

1000A-G

+3 DTU

6 DTU

30 DTU

12 DTU

3 DTU

1 College CTE

Course

College & Career

WBL + 1 College

CTE

NCRC Gold

NCRC Platinum

NCRC Silver

NAF Certified

Integrated Pathway +

WBL

Integrated Pathway

90% SQIS(CORE)

IEBC/Cal-PASS+ Indicator

Linked Learning Pathway

IEBC/Cal-PASS+ Indicator

LL Pathway

- WBL

WBL + 1 College

CTE (Math or ELA pre-req.)

Additional Considerations

Bonus points to incentivize equitable opportunities and outcomes? • English language learners• Students with special needs• Socio-economically disadvantaged students

Algorithm for blending C&CR indicators • Average of each or would additional algebra be required?

Calibration of voluntary measures • How might other data systems be validated/integrated?

• Cal-PASS+, IEBC, School Quality Improvement System (CORE)• Review process and evaluative criteria

All students should have:• Access to an education that prepares them for college and career.• A choice of paths, not a single track.• The ability and support to make informed choices about the future.

The Academic Performance Index:• Should recognize the many paths that students may pursue.• Can incentivize schools to offer educational opportunities that provide

students with options after high school.

Preparing Students for College, Career, and Life

Questions?

Contact us:policy@linkedlearning.org(916) 248-4848

Visit us:www.linkedlearning.org