Program Reporting Changes and Requirements for the WIA Quarterly and Annual Reports ETA 9090 / 9091

2

Agenda

• Overview

• Highlights

– What’s New in ETA 9090 / 9091

• Common Measures Key Concepts & Considerations

2

3

Agenda (con’t)

• ETA 9090 Reporting Instructions

• ETA 9091 Reporting Instructions

• Quiz - time permitting

4

Overview

5

• Common Performance Measures Policy for ETA’s Performance Accountability System -TEGL 17-05

• Common Measures Policy - TEGL 28-04, April 15, 2004

• Revised Reporting Instructions for PY05 & PY06

• 1st (2nd?) Quarter PY05 results

Background

6

Recent Revisions to Common Measures Policy

• TEGL 17-05 Rescinds and replaces:– TEGL 28-04– TEGL 7-99 core performance measures – Customer Satisfaction measures in

TEGL 6-00 and TEGL 6-00 Ch. 1

7

Current Reporting Requirements

• TEGL 28-04 – Common Measures

– Participation & Exit

– Implementation began 7/1/05

• Revised Reporting Instructions– WI/A Quarterly Report (ETA 9090)

– WIA Annual Report (ETA 9091)

8

Illustration: Participation and Exit

Participation No Services

Exit DateParticipation

Date

Eligible and Receives Service

End of 90-Day Period

Last Service

9

Key changes in TEGL 17-05

• Program Participation

• Adult Earnings Measure

• Literacy and Numeracy Gain – Testing Interval

• CM definition of Certificate replaces Credential beginning July 1, 2006

10

Program Participation

• Participant includes:– Those who receive services remotely or

through electronic technologies

– Individuals that use self-directed tools for job search – Does not apply to WIA Youth program

• No longer exclude self-directed job search that does not result in referral to a job

11

(PY 2006) AVERAGE EARNINGS(PY 2006) AVERAGE EARNINGS

• Definition: For those employed in the 1st, 2nd and 3rd quarter after exit, average six month earnings in the 2nd and 3rd quarter after exit

• Calculation:

Of those employed in the 1st 2nd and 3rd quarters after exit…

Earnings from the 2nd and 3rd quarter after exit

Number who exit during the quarter

12

Literacy Numeracy

• Based on date of first youth service– No longer tied to date of participation

• Testing Interval– Youth participants may be included for up to 3 years

if they remain Basic Skills Deficient

13

1st (2nd?) Quarter PY05 results

– Failure to report

– Inaccurate data

– More work to be done

14

1st Quarter PY05 observations

• Total Adults (self services only) > Total Adult Participants– Should be smaller number

• Total Youth (14-21) Participants and Exiters– Should equal sum of Younger Youth and Older Youth– Should equal sum of In-School and Out-of-School

• Current Quarter (most recent) roughly equal to ¼ Cumulative 4-Quarter

15

Highlights

16

What's New in the WIA Quarterly Report - ETA 9090

17

• Uses Common Measures concepts– Participant– Exit

• Common Measures Definitions– Adult, DW, NEG

• Entered Employment• Employment Retention• Earnings Increase (PY05)• Average Earnings (PY06)

– Youth (14-21)• Placement in Employment or Education• Attainment of Degree or Certificate• Literacy and Numeracy Gains

– (opt. PY05, req. PY06)

Performance Measures

18

More changes

• Total Participants / Total Exiters breakouts– WIA Adults (self services only)– NEG– Received Training Services

• Breakout for Total Participants only

– In-School / Out-of-School

• Rolling four quarters• Customer Satisfaction not reported

19

What's New in the WIA Annual Report ETA 9091

20

Performance Measures

• Uses Common Measures concepts– Participant– Exit

• Common Measures Definitions– Adult, DW

• Entered Employment• Employment Retention• Earnings Increase (PY05)• Average Earnings (PY06)

21

Tables• Additional table H 1 for Youth

– Common Measures• Placement in Employment or Education• Attainment of Degree or Certificate• Literacy or Numeracy Gains

• Table M– Total Participants and Total Exiters breakouts

• WIA Adult (self services only)• In-school/out-of-school

• Table O– Additional lines for 3 Youth Common Measures

Common Measures Key Concepts & Considerations

22

23

Who’s reported in participant counts?

All individuals who have been determined eligible and receive a service, including self-service and informational activities, in either a physical location (One-Stop career center or affiliate site) or remotely through electronic technologies

24

Who’s included in WIA performance measures calculations?

All participants who receive a core, intensive, or training service who exit the program are to be included in the performance measures calculations, except that WIA Section 136 excludes WIA adult and dislocated worker program participants who only receive self-service or informational activities

25

What are Self-Service and Informational Activities?

Core services that are made available and accessible to the general public; that are designed to inform and educate individuals about the labor market, their employment strengths and weaknesses, and the range of services appropriate to their situation; and that do not require significant staff involvement with the individual in terms of resources or time.

Ref. 20 CFR 666.140(a)(2)

26

ETA’s Interpretation of Self-Service and Informational Activities

• Self-Service = participants serve themselves in accessing workforce investment system information in either a physical location or remotely through electronic technologies

• Informational activities = may include self-services and staff assisted core services that are designed to inform and educate a participant about the labor market and to enable a participant to identify his or her individual employment strengths, weaknesses, and the range of services appropriate for the individual.

27

Significant Staff Involvement• Any assistance provided by staff beyond the

informational activities regardless of the length of time involved in providing such assistance

• Includes a staff member’s assessment of a participant’s skills, education, or career objectives in order to assist participants in any of the following:– Decide on appropriate next steps in search of employment,

training, and related services– Assess their personal barriers to employment– Access other related services necessary to enhance their

employability and individual employment related needs.

28

Informational Activities

• When a staff member provides a participant with readily available information that does not require an assessment of the participant’s skills, education, or career objectives, the participant is a recipient of informational activities. This includes:– Labor market trends– Unemployment rate information– Businesses that are hiring or reducing workforce– Info on high growth industries– Info on demand occupations

29

Participant vs. WIA PerformanceService Participant WIA Perf

C

O

R

E

Self Service Yes No

Informational Activities

Self Service Yes No

Staff Assisted

Readily Available Information Yes No

Requires Staff Assessment Yes Yes

Intensive Yes Yes

Training Yes Yes

Follow-up No No

30

Participants

Core Intensive Training

Self-Service

Info. Activities

31

Self-Service

Info. Activities

Self-Service

Significant Staff Involve

Core

Training

Intensive

WIA

PERF

CALC

Core

34

Scenarios

The following scenarios could occur in the delivery of services. While these scenarios do not cover every situation states and local workforce investment areas may encounter, they can be used to determine the level of staff involvement that will trigger a period of participation and a participant’s inclusion in WIA performance measures calculations.

35

An individual comes into the One-Stop Career Center believing that he can access all government services. He asks for directions to the Motor Vehicle Administration. The receptionist gives the information and the individual leaves.

This person is not a participant and will not appear in any counts for performance measures. The information that he sought did not directly relate to the services provided by the One-Stop Career Center or the workforce investment system

Scenario 1

36

An individual comes into the One-Stop Career Center and accesses the computers in the self-service area, using a system user account identification. The self-service area is jointly funded by WIA and Wagner-Peyser Act funds. She takes some notes, scribbles the notes on a sheet of paper near the computer, and leaves the facility.

Scenario 2

37

This person may be a participant but it is difficult to determine the nature of the information that the individual has accessed or written down, as there had been no staff interaction with this individual. In this case the individual is a participant because she used her system user account identification to access a saved job search on the state’s Internet job bank system and copied employer contact information. This participant’s self- referral activity was captured by the state’s datasystem and a service transaction was recorded. Because the state job bank is funded by both WIA and the Wagner-Peyser Act, this individual is included in the WIA and Wagner-Peyser Act participant counts, but not included in the WIA performance measures calculation because the participant received only self-services. This person is included in the Wagner-Peyser Act performance calculations.

38

Scenario 3 An individual accesses job listings from his home computer and uses the resume writer software that is on the local One-Stop Web site. The Web site is jointly funded by the Wagner-Peyser Act and WIA. The individual sends his resume in response to a job listing.

This person is a participant as he has used workforce investment system information and services to assist his job search. This individual is included in Wagner-Peyser Act and WIA participant counts, but not included in the WIA performance measures calculation because the participant received only self-services. This person is counted in the Wagner-Peyser Act performance calculations.

39

Scenario 4An individual accesses job listings from a computer at the local library and uses her account identification that she established with the workforce investment system to log on to the computer. The individual sends her resume in response to a job listing for which she believes she will be a suitable candidate.

This person is a participant as she has used workforce investment system services to assist her job search. This individual is included as a Wagner-Peyser Act and WIA participant, but not included in the WIA performance measures calculation because the participant received only self-services. This person is counted in the Wagner-Peyser Act performance calculations.

40

Scenario 5An individual has figured out how to access workforce investment system information from his cell phone that includes a personal data assistant. He thinks that he has found the job of his dreams on the state job bank and posts his resume with the employer.

This person is a participant as he has used workforce investment system services to assist his job search. This individual is included as a Wagner-Peyser Act and WIA participant, but not included in the WIA performance calculations because the individual received only self-services. This person is counted in the Wagner-Peyser Act performance calculations

41

Scenario 6An individual arrives at the One-Stop Career Center and, familiar with computers, she decides to go to the self-service section and search the job listings. Later, she wants to review occupational trends information. She pulls up some information, but is not quite sure how the trends were calculated. She questions the staff about the statistics and asks where she might find additional information on the need for workers in this field. The staff member answers the question and directs the individual to the Career Voyages, another self-service Web site where more information and other helpful links are available.

42

Scenario 6 con’t.This person is a participant as she has used workforce investment system information and self-service electronic tools jointly funded by WIA and Wagner-Peyser Act funds to assist her job search. This individual is included as a Wagner-Peyser Act and WIA participant, but not included in the WIA performance calculations because the participant received only self-service and informational activities. This individual did not receive services requiring significant staff involvement because the staff person provided readily available information and instructions about tools and services in the research room without having to assess the individual’s skills, education, or career objectives. This person is counted in the Wagner-Peyser Act performance calculations.

43

Scenario 7An individual arrives at a One-Stop Career Center looking for information on the availability of jobs in a specific field. The staff member asks the individual several questions to determine the individual’s previous work experience, education, and training experience in the specific industry and occupation. The staff member assists the individual to search for employment on a resource room computer and provides him with the local occupational and economic trend information.

44

Scenario 7 con’t.This person is a participant as he has used workforce investment system staff-assisted service to assist his job search. This individual is included as a Wagner-Peyser Act and WIA participant, and must be included in the WIA performance measures calculation because the participant received more than self-service and informational activities. The staff member performed an assessment of the participant’s work readiness for employment in the local area and assisted in the participant’s job search. This person is also counted in the Wagner-Peyser Act performance calculations.

45

Scenario 8

An individual arrives at the One-Stop Career Center and wants to know the various services that are available at the One-Stop Career Center. The One-Stop Career Center specialist gives the individual an orientation to the One-Stop Career Center and advises the individual of what services are available. The staff member sets the individual up at a computer and shows him how to use the equipment and to access the job search tools. Later, the individual participates in an assessment interview and begins to develop an employment plan.

46

Scenario 8 con’t.

This individual is a participant as she has used workforce investment system services to assist her job search. This individual is included as a Wagner-Peyser Act and WIA participant. She is included in the WIA performance measures calculations because her assessment interview and the development of an employment plan required significant staff involvement. This individual should also be included in the Wagner-Peyser Act performance calculations.

47

Workforce Investment Act Quarterly Report:General Reporting Instructions and ETA Form 9090

48

ETA 9090 Reporting Instructions General Instructions

• Aggregate Performance data– Determine levels of participation and

accomplishments

• Time Periods– Most recent quarter– Most recent four quarter period

49

• WIA Adult

• WIA Dislocated Worker Program

• WIA Youth Program

• National Emergency Grant

Covered Programs

50

• Separate Breakout on ETA 9090• Total Participant • Total Exiters

• Entered Employment Rate• Employment Retention Rate• Six Months Earnings Increase

National Emergency Grants

51

• Statewide activities– Include activities that involve enrollment of

individuals– Exclude activities that do not involve enrollment

of Individuals– Indirect services (e.g. research or evaluation) – Highly specialized services (incumbent worker

training)

Covered Programs (con’t)

52

Due Date

Report Quarter Due DatesJanuary – March May 15th

April – June August 14th

July-September November 14th

October – December February 14th

Note: When due date falls on a Saturday or Sunday the quarterly report is due the Friday before.

53

WIA Quarterly Report FormatETA 9090

• Four Columns– Performance Items

– Program Group• Performance Items broken into subpopulations

– Current Quarter (most recent)

– Cumulative 4-Quarter Reporting Period

54

WIA Quarterly Report Format(con’t)

• Performance Items– Total Participants

– Total Exiters

– Performance Measures

55

Total Participants• Total Adult Participants

– Total Adult Participants (self services only)– WIA Adults– WIA Dislocated Workers– National Emergency Grant

• Received Training Services– WIA Adults– WIA Dislocated Workers

• Total Youth (14-21) Participants– Younger Youth (14-18)– Older Youth (19-21)– Out-of-School– In-School

56

Total Exiters

• Total Adult Participants– Total Adult Participants (self services only)– WIA Adults– WIA Dislocated Workers– National Emergency Grant

• Total Youth (14-21) Participants– Younger Youth (14-18)– Older Youth (19-21)– Out-of-School– In-School

57

Performance Measures

• 3 Youth Common Measures– Placement in Employment or Education– Attainment of Degree or Certificate– Literacy or Numeracy Gains

• 2 WIA Younger Youth Measures– Diploma or Equivalent– Skill Attainment

58

Performance Measures (con’t)

• Entered Employment Rate (Ad, DW, OY,NEG)

• Employment & Credential (Ad, DW)

or Credential Rates (OY) • Retention Rate (Ad, DW, OY, YY, NEG)

• Six-Months Earnings (Ad, DW, OY,NEG)

59

Rationale for Rolling Quarter Methodology

• Issue: We have program years, state and federal fiscal years, and calendar years, making communication of "annual results" to customers and stakeholders seem convoluted

• This methodology allows us to speak in terms of four quarters of data (i.e., annual data) for any workforce program, whether the program is based on a program year or fiscal year

60

The Rolling Four-Quarter Methodology Explained

• Take a look at the following, which could be considered a “rolling four-letter alphabet”

• Notice that the first letter “drops off” so another letter can be added, creating the perception that the letters are “rolling”

A B C D E F G H I

A B C D E F G H I

61

Rolling Four-Quarter Methodology (cont’d)• Rolling quarters operate much the same way:

– We must have four quarters at any given time so we can communicate in terms of “annual” results

– The earliest quarter drops off so the next (most recent) quarter can be added

– As this process continues, it creates a “rolling quarter” perception

Jan – MarApr – June

July – Sept

Oct- Dec

Jan – Mar

Apr – June

July – Sept

Oct – Dec

Jan - Mar

Jan – MarApr – June

July – Sept

Oct- Dec

Jan – Mar

Apr – June

July – Sept

Oct – Dec

Jan - Mar

Jan – Mar

Apr – June

July – Sept

Oct- Dec

Jan – Mar

Apr – June

July – Sept

Oct – Dec

Jan - Mar

62

Rolling 4 Quarters

• Total Participants PY05

• 1st Qtr PY06 shift last row one column to right

2004 2005 2006

Jan-Mar Apr-June July-Sept Oct-Dec Jan-Mar Apr-June July-Sept Oct-Dec Jan-Mar Apr-June July-Sept

      Total Participants 14-Nov    

        Total Participants 14-Feb  

          Total Participants 15-May  

            Total Participants 14-Aug

63

Rolling 4 Quarters

• Total Exiters PY05

• 1st Qtr PY06 shift last row one column to right

 14-Aug Total Exiters     

  15-May Total Exiters    

   14-Feb Total Exiters   

    14-Nov Total Exiters  

Oct-DecJuly-SeptApr-JuneJan-MarOct-DecJuly-SeptApr-JuneJan-MarOct-DecJuly-SeptApr-JuneJan-Mar

200620052004

64

Placement in Employment or Education Time Periods

For the Quarter ending Sept 2005

Each following quarter shift 1 column to the right

2004 2005 2006

Jan-Mar Apr-June July-Sept Oct-Dec Jan-Mar Apr-June July-Sept Oct-Dec Jan-Mar Apr-June July-Sept Oct-Dec

Exit Cohort     

        

  Employment or Education     

      

     Wage Delay for Employment

outcome14-Nov

    

 

65

For the Quarter ending Sept 2005

Each following quarter - shift one quarter to right

2003 2004 2005 2006

July-Sept Oct-Dec Jan-Mar Apr-June July-Sept Oct-Dec Jan-Mar Apr-June July-Sept Oct-Dec Jan-Mar Apr-June

Exit Cohort  

  Employment Qtr              

    Retention & Earnings  

  2 Qtr Wage Record Delay      

  14-Nov5    

Retention & Earning IncreaseTime periods

66

ETA 9091 Reporting Instructions

67

Workforce Investment Act Annual Report:General Reporting Instructions and ETA Form 9091

General Instructions

• Emulate Private Sector’s Report to Stockholders– Meets federal reporting requirements

• WIA 136(d)(1) and WIA 185(d)

– Allows flexibility to showcase unique programs, strategies, and accomplishments.

68

General Instructions (con’t)

• Three required elements (WIA Sec. 136)– State & Local performance data

• Perf data must be based on WIASRD data

– Status of state evaluation activities– Cost of WIA activities relative to the effect of those

activities on the performance of participants

69

Covered Programs

• WIA Adult Program

• WIA Dislocated Worker Program

• WIA Youth Program

70

Covered Programs (con’t)• Performance information on NEG participants

will not be included – Reported on Quarterly Report

• Statewide activities– Include activities that involve enrollment of

individuals– Exclude activities that do not involve enrollment of

Individuals– Indirect services (e.g. research or evaluation) – Highly specialized services (e.g. incumbent worker

training)

71

Annual Report Specifications

Two Required Sections

• Narrative Section

• Tables A - O

72

Narrative Section

• Narrative Section - must include:– Cost of WIA activities relative to the effect on

the performance of participants– Description of state evaluations of WIA

activities

• Questions the evaluation will/did address• Description of methodology• Info on timing of feedback and deliverables

73

Narrative (con’t)

Narrative Section may include:– success stories, – messages from Governor,– info on State WIB members, – market analysis, strategies for improvement,

program’s effects on major industries, etc.

74

Table Section

• State Wide Performance data• Tables A-N

• Local Area Performance date• Table 0, one for each local area

Don’t forget Customer Satisfaction!!

Results are reported in Table A

75

American Customer Satisfaction Index (ASCI)Required Methodology for Statewide Results

Who Will be surveyed?• Participants – WIA Title IB exiters• Employers who received a substantial service that involved personal contact by WIA staffTime frame for surveys• Participants – within 60 days of expected last date of

service• Employers – ASAP after completion of service, but no

later than 60 days after completion of service

76

Required Methodology for Statewide Results (con’t)

• Telephone Surveys• Random Samples

– 500 completed surveys– 70% response rate

• 3 required questions– 10 point scale

• 1= Very Dissatisfied, 10 = Very Satisfied

– Weighted Average results

77

Parameters for Collecting Customer Satisfaction information to Measure Local WIB Performance

• Purchase license and use ASCI methodology• Develop alternative measurement instruments and

methodology– objective, quantifiable, and measurable results– Consistently applied among all local WIBs– Collected in uniform and Equitable manner– Conform to widely accepted methodological and statistical

criteria for measuring customer satisfaction

Additional Info on Customer Satisfaction requirements http://www.doleta.gov/Performance/

78

Due Date

Program Year Due Dates

PY 2005 October 1, 2006

PY 2006 October 1, 2007

PY 2007 October 1, 2008

Note: When due date falls on a Saturday or Sunday the quarterly report is due the Friday before.

79

Submission Procedures

• Tables A-O submitted through Enterprise Business Support System (EBSS)

• 3 copies of narrative mailed to ETA and/or• Additional electronic copy for posting on ETA’s

website (WIA.AR@dol.gov)– WordPerfect, Microsoft Word, Acrobat or other

commonly used format.

• Must complete Report Validation prior to submitting the WIA Annual Report