Goals, Cohorts, and New Reports:

Oh My!

Classroom reportHow many of you have been keeping track of how you are doing on Table 4 this year?

• Statewide information

Classroom report: This report can help you locate students who can or need to be tested.• Examples

o Instructional Year Start testo Post tests

GoalsProgram Year 2012 Table 5 Outcomes • During Program Year 2012, programs were responsible

for assisting students in meeting their goals. • Programs had the opportunity to select goals that

were reasonably attainable.• The goals that programs were held accountable for

were:o GED attainmento Postsecondary Ed/trainingo Enter employmento Retain employment.

• How did we do?

Statewide DataGoal Description Participants

Enter Employment 1115

Enter Postsecondary Education or Training 501

Receipt of Secondary School Diploma or GED 1145

Retain Employment 1059

Completed

510

318

917

834

% Achieving

45.86

63.6

80.16

79.66

Targets

47%

62%

62%

66%

As a state, we hit the targets in three out of four goals. We missed Entered Employment by one percentage point.

Programs by Size• Small Programs (50 or

fewer)AnacondaDeer LodgeForsythGlendive - Dawson Community College Havre

• Medium Programs (51-200)LVA BitterrootButteHardinLewistownLivingstonMiles Community CollegeMontana State Prison

• Large Programs (201-700)

Billings Bozeman FVCC Great Falls Helena Missoula

Students Served• Small Programs

144 Students with 12 or more hours

• Medium Programs661 Students with 12 or more hours

• Large Programs2367 Students with 12 or more hours

Small Program Outcomes

Goal Total with Goal

Number Completed Goal

% Completed Goal

State Targets

# Programs to Hit Target

Enter Employment

58 20 34% 47% 2/5

Enter Post secondary Ed or Training

14 11 79% 62% 4/4

Receipt of Secondary School Diploma or GED

50 29 58% 62% 1/5

Retain Employment

48 35 73% 66% 4/5

Total (144 students with 12 or more hours)

170 95 56%Small programs hit the targets in two out of the four goal areas. GED and Enter Employment are the two areas that appear to be the most challenging.

Medium Program Outcomes

Goal Total with Goal

Number Completed Goal

% Completed Goal

State Targets

# Programs to Hit Target

Enter Employment

234 106 45% 47% 3/6

Enter Post Secondary Ed or Training

53 28 53% 62% 3/6

Receipt of Secondary School Diploma or GED

281 234 83% 62% 6/7

Retain Employment

164 127 77% 66% 5/6

Total (661 students with 12 or more hrs.)

732 495 68%Medium programs hit the targets in two out of the four goal areas. Post Secondary Ed and Training and Enter Employment are the two areas that appear to be the most challenging.

Large Program Outcomes

Goal Total with Goal

Number Completed Goal

% Completed Goal

State Targets

# Programs to Hit Target

Enter Employment

728 354 49% 47% 3/6

Enter Post secondary Ed or training

408 278 68% 62% 3/6

Receipt of Secondary School diploma or GED

766 618 81% 62% 5/6

Retain Employment

785 620 79% 66% 6/6

Total (2,367 students with 12 or more hrs)

2,687 1,870 70%Large programs over all hit the targets in all 4 goal areas. Half of the programs were challenged with hitting targets in Enter Employment and Post Secondary Ed or Training.

Retention Strategies• What’s your plan?• How’s it working?

CohortsBeginning July 1, 2012, programs are no longer responsible for hitting outcomes for student goals that are reasonably attainable. Instead, programs are responsible for hitting targets in cohorts based upon a student’s status at entry.• Cohorts are:

o Obtain GEDo Enter Postsecondary Ed/trainingo Enter Employmento Retain Employment

• What is the difference between a goal and a cohort?• Why are goals still important?

Cohorts• GED Cohort

o Definition: When a student has 12 or more hours during a program year, has taken all 5 GED tests, and is exited during the program year, he/she falls into this cohort.

o Examples:• A student who has previously taken any tests, finishes

testing, and is exited.• A student who previously didn’t pass for any reason and

who returns, either passes or doesn’t and is exited.• A student who has taken all 5 tests, passed, and exited.• A student who has taken all 5 tests, not passed, and

exited.

Cohorts• GED Cohort

o Considerations for practice:1. A student must have taken all 5 GED tests at least once to be

considered for this cohort.2. It is important to exit a student with the last day he/she attended

if student obtains a GED3. Best practice for following up and exiting these students

• Stay current with these students so you can encourage them to continue on with postsecondary ed/training.

• Exit as soon as you know they are done participating. 4. If a student doesn’t exit during the program year, it doesn’t count

for or against you in this cohort.5. How many students currently fall into this cohort?

GED cohortProgram size

Total students

# in GED cohort

% in GED cohort

# passed

% passed

Small 62 12 19% 7 58%Medium 227 90 40% 75 83%Large 1099 232 21% 165 71%MSP 109 36 33% 23 64%Statewide

1492 371 25% 270 73%

*It should be noted that the students in the GED cohort could have taken all 5 tests in a previous year and just be returning for instruction and not have tested this year.

Using Your Report

• Using the cohort report to look at your numbers.

• Using it for follow up for those who have taken all 5 tests and not passed.

• Using it to verify that students are exited at the end of the year.

Cohorts• Postsecondary Ed or Training Cohort

o Definition: Any student with 12 or more hours during a program year who has a high school diploma or GED and does not have a degree or certificate and who exits during the year. (the definition of this cohort is currently under discussion and may change.)

o Examples:• A student who enters the program with a high school diploma

or GED who has not completed a college degree or certificate.• Students who have previously been enrolled in postsecondary

ed and are preparing to return and those who are co-enrolled in ABE and postsecondary ed are counted here.

• This may include a student who passes the GED during the program year who intends to go on to postsecondary ed or training.

Cohorts• Postsecondary Ed or Training Cohort

o Considerations for practice:1. This encourages us to think beyond just being GED

programs.2. Discuss future plans when a student initially comes in.

Don’t wait until after a student completes the GED.3. Community resources/partners that can be helpful.4. What do students need to be successful with

postsecondary ed/training and what can you provide.5. What training opportunities are there in your community

and at a distance that lead to a student’s career goals?6. Using post secondary ed/training telephone and

enrollment logs.7. How does data matching work?8. How many students currently fall into this cohort?

Postsecondary ed/training cohort

Program size

Total students

# in cohort

% in cohort

# of GED completers

% of GED completers

Small 62 20 32% 8 40%Medium 227 91 40% 75 82%Large 1099 439 40% 165 38%MSP 109 23 21% 23 100%Statewide

1492 573 38% 270 47%*For medium programs, 82% of the students potentially in the post secondary ed cohort are GED completers. It could be important for them to make a plan to get GED students thinking beyond the GED, determine what post secondary ed/training partners and opportunities are available and how to do follow up so the enrollment can be captured.

Using Your Report• Using the cohort report to look at your numbers.• Using the cohort report to do follow up for postsecondary

ed/training logs.• Checking for errors – Students who don’t have a GED

shouldn’t be in this cohort. If they are, there is likely a data entry error.

• Using the cohort report to verify that students are exited at the end of the program year.o Best practices for exiting students.

• What does it mean that students have two years to enter postsecondary ed/training? o what does that mean in practice? o How can you use the cohort report to help?

Postsecondary Data Match

Students now have until the end of the year following the year they exit to be counted as completing this measure.• If a 2011-2012 student with the Enter Postsecondary

goal, who exited by June 30, 2012, is enrolled in college or a training program by June 30, 2013 you will get credit for that measure

• First data match was done November 2012 and reported on the December 2012 federal report

• Second data match will be done July 2013 and reported on the December 2013 report

• The second report will contain the combined total of the two years’ worth of data match results

• We will report on each year’s cohort separately – we will not combine totals for 2011-12 and 2012-13

Retention Strategies• What’s your plan?• How’s it working?

Changes to Employment Status

Definition***Note – Learners who are employed in the following types of jobs should be marked as Not in the Labor Force because we cannot do data match on them.• Military• Farm workers• Anyone paid under the table• Anyone working out of state with no Montana address• People who are self-employed and not reporting payroll taxes• Anyone on TANF who is doing WEX hours • Students in youth employment/training programs who are being paid by the youth employment and not by an employer• Individuals in on-the-job training programs who are not being paid by employers• Individuals who are incarcerated and unable to leave the facility• Individuals employed by the Fort Belknap Indian Reservation*• Individuals employed by the railroad.*

Employed: Learners who work as paid employees. (Retain employment cohort.)

Other Employment Status Definitions

• Unemployed: Learners who are not working but are seeking employment, make specific efforts to find a job, and are available for work. (Gain employment and potentially retain employment cohort)

Not in the Labor Force: Learners who are not employed and are not seeking employment. (No employment cohort.)

• How many students currently fall into the gain employment cohort?

Cohorts• Gain Employment Cohort

o Definition: Any student with 12 or more hours during a program year who enters the program year with an employment status as unemployed and the student exits during the program year.

o Examples:• Student must have an employment status of

unemployed. • To succeed in this outcome, students must

have been employed anytime in the first quarter after exit in a job that reports income.

Cohorts• Gain Employment Cohort

o Considerations for practice:• If you choose unemployed as the employment status, be sure that the

student is actually looking for work and employable. Suggestion have you been putting applications in?

• If the student wants to find work, consider what you can provide in the classroom to help him/her.

• Make sure that the type of work he/she is pursuing is able to be data matched.

• Refer students to community resources/partners that can assist them in obtaining employment.

• Consider the questions you ask at intake to insure that you choose the employment status correctly. Employment status is set at the beginning of the program year.

• Change the employment status for rollover students at the beginning of the new program year.

• Opportunity to correct employment status this year only; however, beware! if a student entered the program unemployed and has since gained employment, you do not want to change the status to employed!

• Deadline for changes

Gain Employment cohort

Program size

Total students

# in cohort

% in cohort

Small 62 23 37%Medium 227 99 44%Large 1099 382 35%MSP 109 1 1%Statewide

1492 503 34%

*Consider what you can do to connect students to community resources to assist them with finding employment and what you can do in the classroom to help!

Using Your Report• Using the cohort report to look at your numbers.• Using the cohort report to check in with students

and refer them to resources and determine which students would benefit from instruction related to gaining employment.

• Using the cohort report to verify students who should be exited at the end of the year.o Best practices for exiting students.

Cohorts• Retain Employment Cohort

o Definition: Any student who enters the program year with an employment status as employed and the student exits during the program year and any student who was unemployed when they entered the program, exited during the program year, and data matched as employed any time during the first quarter after exit.

o Examples: • A student who has an employment status of employed. To

succeed, the student needs to be employed sometime during the third quarter after exit. It does not need to be in the same job, just in a job where they receive wages that are reported.

• Don’t forget that this cohort grows to include anyone who entered as unemployed and was employed in a job where he/she received wages that were reported anytime during the first quarter after exit.

Cohorts• Retain Employment Cohort

o Considerations for practice:• If you choose employed as the employment status, be sure that

the student is employed in a job that can be data matched.• Consider the questions you ask at intake to insure that you

choose the employment status correctly. Employment status is set at the beginning of the program year.

• Change the employment status for rollover students at the beginning of the new program year.

• Opportunity to correct employment status this year only. Double check that you haven’t changed any students who came in unemployed to employed, and make sure the student is in a job that can be data matched.

Deadline - • How many students currently fall in this cohort?

Retain Employment cohort

Program size

Total students

# in cohort

% in cohort

Small 62 18 29%Medium 227 74 33%Large 1099 395 36%MSP 109 0 0%Statewide

1492 485 33%

*Remember the number in this cohort will grow when the gain employment cohort has been data matched at the end of the first quarter after exit. Those who have gained employment will be added to the number in the retain employment cohort. They will be data matched in the third quarter after exit.

Retain Employment Cohort

• Using the cohort report to look at your numbers.o This will likely be deceiving because you will not be able to tell which of

your unemployed students will be employed during the first quarter after exit. What can you do? Teach soft skills, other ideas?

• Using the cohort report to verify that you have exited students at the end of the program year.o Best practices for exiting students

Employment Data Match

Always done for students in prior program year – (more or less).• For the 2011-12 report that we just submitted:

o For Enter employment, we matched students who exited during the 2nd, 3rd and 4th quarters of 2010-11 and the 1st quarter of 2011-12

o For Retain employment, we matched students who exited during the 4th quarter of 2009-10, and the 1st, 2nd, and 3rd quarters of 2010-11

• Students with employment goals who exited during 2011-12 will be data matched and reported on the 2012-13 report.

Retention Strategies• What’s your plan?• How’s it working?

Default classes• Default classes

o State perspective

Wrap up for the day• Outstanding Questions• Things that still need to be resolved.• Travel reimbursement• Evaluations

Safe Travels.