Performance Management Performance Management Policy Highlights Policy Highlights

Workforce Innovations 2005 Workforce Innovations 2005

July 12 and 13, 2005 July 12 and 13, 2005

2

Reporting performance is a fundamental element of improving customer services and good public administration

Performance information should be easily understood by all customers, stakeholders, and operators of the workforce investment system

Establishing common measures and standardizing customer data collection improves the comparability and understanding of performance results

Ensuring the accuracy and timeliness of performance results is necessary for demonstrating system integrity and value

Workforce System Performance Improving Performance Results

3

Focus on the core purposes of the workforce system– employment for adults and skill gain for youth

Break down barriers to integration resulting from different definitions, data and reports for each workforce program

Resolve questions raised by GAO and other oversight agencies regarding the consistency and reliability of data

Reduce confusion among our customers and stakeholders who want to know about results

Workforce System Performance Benefits of Common Performance Measures

4

Common measures focus on key results for the adult and youth programs; other management information collected will continue to be valuable

Management and oversight of programs will continue to require information that is relevant to each program

Information on statutorily-required performance measures will continue to be collected and reported

Additional information about participants, services and outcomes must supplement the story told by the common measures

Workforce System Performance Common Performance Measures

5

Three common performance measures apply to programs serving adults:

Entered Employment Employment Retention Earnings Increase

Three common measures apply to programs serving youth:

Placement in Employment or Education/Advanced Training

Attainment of a Degree or Certificate Literacy and Numeracy Gains

Common Performance Measures

6

States have negotiated performance goals for WIA programs for PY 2005 and PY 2006

WIA AdultsEntered employmentEmployment retentionEarnings changeEmployment and credential rate

WIA Dislocated WorkersEntered employmentEmployment retentionEarnings changeEmployment and credential rate

Note:

In addition, states have negotiated performance goals for WIA for Employer customer satisfaction and Job seeker customer satisfaction

AccountabilityWIA Adult and Dislocated Worker,

Wagner-Peyser and Trade Programs

PY 2005 will be a baseline year for these performance measures for W-P employment services. Goals will be negotiated beginning with PY 2006.

W-P employment services

Entered employment

Employment retention

Earnings change

States are responsible for achieving National Goals for the Trade program.

Trade

Entered employment

Employment retention

Earnings change

7

States have negotiated performance goals for WIA programs for PY 2005

and PY 2006

Older Youth

Entered employment

Employment retention

Earnings change

Employment and credential rate

Younger Youth

Skill attainment rate

Diploma attainment rate

Retention rate

AccountabilityWIA Youth Programs

States have not negotiated performance goals for the youth common measures, however states will be expected to begin reporting on these measures

All Youth- Common MeasuresPlacement in employment or education/

advanced training (PY05)Attainment of a degree or certificate

(PY05)Literacy and numeracy gains (PY05/06)

States must collect and report the first two measures beginning in PY05. States are encouraged to begin data collection and reporting for the literacy/numeracy gains measure, although it will not be required until PY06.

8

Key Definitions:

Participant – An individual who is determined eligible to participate in the program and receives a service funded by the program in a physical location.*

Exit – Participant does not receive a service funded by the program or a partner program for 90 consecutive calendar days and is not scheduled for future services.

* States and grantees are encouraged to consider as participants those individuals who receive services that are available through the Internet and are not accessed through a physical location.

Common Performance Measures

9

Adult Common MeasuresSummary of Major Policy Changes

TEGL 15-03Old Policy

TEGL 28-04New Policy

ADULT MEASURES

Entered Employment Rate No change in methodology from WIA or TEGL 15-03Permits grantees to use wage records and supplemental data

Employment Retention RateMeasurement at 1st, 2nd, and 3rd quarters

No change in methodology from TEGL 15-03Permits grantees to use wage records and supplemental data

Earnings Increase Measure

Separated into two distinct indicators of performance

1.Percent change pre- to post-program Qtr 1

2.Percent change post-program Qtr 1 to Qtr 3

Replaces the two-part earnings increase measure identified in TEGL 15-03 with a single Six Month Earnings Increase measure (pre- to post-program)Wage records are the only data sourceGrantees that do not have access to wage records can use supplemental data as an interim means of reporting

Program Efficiency identified as a core common measure

Eliminates program efficiency as a reportable measure at the grantee level

10

Measure:

Of those who are not employed at the date of participation:

Number of participants employed in the 1st quarter after the exit quarter divided by the number of participants who exit during the quarter

Same as current WIA Adult and Dislocated Worker entered employment definitions

Includes only those not employed at entry Employment status at participation is based on information from the

individual, not wage record data

Adult Common Measures Entered Employment

11

Measure:

Of those who are employed in the 1st quarter after the exit quarter:Number of participants employed in both the 2nd and 3rd quarters after the exit quarter divided by the number of participants who exit during the quarter

Additional measurement point (2nd quarter) compared to current WIA Adult, Dislocated Worker, and TAA retention rates

Includes all participants employed in the 1st quarter regardless of their employment status at participation

Employment in the 1st, 2nd, and 3rd quarters does not have to be with the same employer

Adult Common MeasuresEmployment Retention

12

Measure:

Of those who are employed in the 1st quarter after the exit quarter:[Total earnings in the 2nd and 3rd quarters after exit minus total earnings in the 2nd and 3rd quarters prior to participation] divided by the number of participants who exit during the quarter

Same as current WIA Adult Earnings Gain measure, but different than current wage replacement rate for WIA Dislocated Worker and TAA programs

Completely new measure for W-P & VETS programs Designed as a six month pre- and post-program look at

earnings change

Adult Common MeasuresEarnings Increase

13

Calculation yields a dollar amount, not a rate Negative dollar amounts are likely for WIA Dislocated Worker

and TAA customers Wage records will be the only data source for this measure

• Grantees that do not have access to wage records may use supplemental data as an interim means of reporting on the measure

States and grantees may exclude participant earnings from each of the total quarterly amounts where the following occurs:

• False or erroneous wage data has been reported to the grantee• Individual received severance pay or other earnings attributable

to a termination from employment

Adult Common MeasuresEarnings Increase, Cont.

14

Youth Common MeasuresSummary of Major Policy Changes

TEGL 15-03Old Policy

TEGL 28-04New Policy

YOUTH MEASURES

Placement in Employment or Education Excluded individuals in secondary school at

exit from performance calculations

Individuals in secondary school at exit are now included in the measure

Permits grantees to use wage records and supplemental data

Attainment of Degree or Certificate Excluded individuals in secondary school at

exit from performance calculations

Individuals in secondary school at exit are now included in the measure

Clarifies the definition of Diploma Includes degrees/certificates awarded by tribally-

controlled colleges and universities

Literacy and Numeracy Gains Included all in-school and out-of-school youth

who were basic skills deficient

In-school youth are now excluded from the measure

Provides additional guidance on providing reasonable accommodations for testing youth with disabilities

Program Efficiency identified as a core common measure

Eliminates Program Efficiency as a reportable measure at the grantee level

15

Measure:Of those who are not in post-secondary education, employed, or the military at the date of participation:Number of participants in employment, military, or enrolled in post-secondary education and/or advanced training/occupational skills training in the 1st quarter after the exit quarter divided by the number of participants who exit during the quarter

Includes individuals in secondary school at participation and exit

Similar to current Younger Youth Retention measure in that it recognizes the same outcomes – improvement from Older Youth Entered Employment

Tighter definition of advanced training

Youth Common MeasuresPlacement in Employment or Education

16

Measure:Of those who are enrolled in education at participation or any time during the program:Number of participants who attain a diploma, GED, or certificate by the end of the 3rd quarter after the exit quarter divided by the number of participants who exit during the quarter

Similar to a combination of the Younger Youth Diploma Rate and Older Youth Credential Rate

Tighter definition of certificate; does not have employment as a component like the current credential measure

Diplomas, GEDs, or certificates can be obtained while a person is still receiving services or at any point prior to the end of the 3rd quarter after exit Individuals in secondary school at exit will be included in the measure

Education refers to those in secondary, post-secondary, adult education, or any other organized program of study

Individuals in secondary school at participation and exit will be included in the measure

Youth Common MeasuresAttainment of Degree or Certificate

17

Diploma – any credential that the state education agency accepts as equivalent to a high school diploma

Certificate – awarded in recognition of attainment of measurable technical or occupational skills necessary to gain employment or advance within an occupation

• Based on standards developed or endorsed by employers

• Certificates awarded by workforce investment boards or in recognition of attaining generic pre-employment or work readiness skills are not included in this definition

Youth Common MeasuresAttainment of Degree or Certificate

18

Certificates can be awarded by the following institutions:• State educational agency• Institutions of higher education (sec. 102 of Higher

Education Act)• Professional, industry, or employer organization or a

product manufacturer or developer• Registered apprenticeship program• Public regulatory agency• Department of Veterans Affairs• Office of Job Corps• Tribally-controlled colleges and universities

Youth Common MeasuresAttainment of Degree or Certificate

19

Measure:Of those out-of-school youth who are basic skills deficient:Number of participants who increase one or more educational functioning levels divided by [the number of participants who have completed a year in the program plus the number of participants who exit before completing a year in the program]

In-school youth are excluded from this measure Individuals who are not basic skills deficient are excluded from this measure Includes individuals with learning disabilities, with accommodations and

alternate testing options Creates specific focus on improving basic skills rather than including basic skills

among work readiness and occupational skills training Does not mean work readiness and occupational skills no longer important

(impact of those skills will be seen in the Placement in Employment and Education Rate)

Standard for success much higher than with less-defined skill attainment rate

Youth Common MeasuresLiteracy and Numeracy Gains

20

To be included in the numerator, an individual must advance one or more Adult Basic Education (ABE) or English as a Second Language (ESL) functioning levels

Gain can occur in literacy or numeracy—can pre-test at different levels in each category

Educational functioning levels are consistent with Adult Basic Education’s National Reporting System (NRS)

Initial assessment test must occur within 60 days following the date of participation in the program

Individuals should be post-tested by the end of one year of participation and compared to pre-test results obtained during the initial assessment

Participants only included in measure a second time if they complete a second full year of participation

Youth Common MeasuresLiteracy and Numeracy Gains

21

Must use one of NRS cross-walked assessment tests or equate alternate test to NRS scale

Tests cross-walked with ABE and ESL levels include:• Comprehensive Adult Student Assessment

Instrument (CASAS)• Test of Adult Basic Education (TABE)• Adult Basic Learning Examination (ABLE)• Student Performance Levels for ESL• Basic English Skills Test (BEST) for ESL• WorkKeys (for the top 3 ABE levels)

Youth Common MeasuresLiteracy and Numeracy Gains

22

ETA received emergency approval in April 2005 to incorporate common measures into the WIA, W-P, VETS, and TAA program reporting systems

• Remember, where possible the definitions for common performance measures are replacing “old” definitions

These changes are effective on July 1, 2005 for WIA, W-P, and VETS programs, and October 1, 2005, for the TAA program

ETA will be publishing three 60-day notices on these reporting changes in July to provide opportunity for public comment as part of the full-approval process of these revisions

ETA is considering alternative definitions for the earnings measure

Implementation

23

Highlights of Changes to Reporting

WIA Performance Reporting System Quarterly Report

• Collects aggregate counts of self-service participants, the number of participants in training, and youth (by age and school status)

• Collects current WIA youth measures plus common measures• Separate reporting of common measures for Adults, Dislocated Workers,

and NEG participants • Rolling four-quarter reporting methodology to align with W-P employment

services Reporting System• Eliminated customer satisfaction reporting on a quarterly basis

Annual Report• Collects similar aggregate counts of participants and exiters as the

Quarterly Report• New report table to collect youth common measures

WIASRD• Collects additional data on participant services and outcomes to calculate

common measures• States do not submit WIASRD files on participants who receive only self-

services and informational activities• States must report co-enrollment of individuals in Wagner-Peyser or TAA

programs

24

Highlights of Changes to Reporting

Wagner-Peyser Reporting System (ES 9002 and VETS 200) Aligns performance reporting cohorts with WIA reporting cohorts Eliminates all customer satisfaction reporting Collects aggregate counts on participants, exiters, and those

receiving workforce information services Collects aggregate outcomes for the adult common measures Collects aggregate data on services to Homeless Veterans (VETS

200 C report only) Eliminates reporting on each category of veteran (e.g., campaign,

disabled) by age grouping

Trade Act Participant Report (TAPR) Collects data elements to calculate common measures Collects information on participant services as authorized under the

TAA Reform Act of 2002

25

Questions about performance issues? Want updates on performance?

Please contact us at: ETAperforms@dol.gov or 202-693-3031www.doleta.gov/performance

Office of Performance and TechnologyKaren Staha, staha.karen@dol.gov

Office of Workforce Investment Evan Rosenberg, rosenberg.evan@dol.gov (Youth)Gail Eulenstein, eulenstein.gail@dol.gov (Adult/Dislocated Workers)Tony Dais, dais.anthony@dol.gov (W-P employment services)

Office of National Response Terry Clark, clark.terry@dol.gov (TAA) VETS Pamela Langley, langley.pamela@dol.gov

Questions or Comments?