Increasing Student Retention and Graduation: Findings from the ASAP Evaluation

Innovations ConferencePhiladelphia, PA March 6, 2012

ASAP History» 2007: Admit 1,132 students at six CUNY community colleges

with support from the Mayor’s Center for Economic Opportunity (CEO)

» 2010: Exceed 3-year graduation goal and funding made permanent allocation to CUNY by the Mayor’s Office

» 2011: CUNY announces plans to expand ASAP to more students at CUNY community colleges

ASAP History» Fall 2007 enrollment across all six CUNY community colleges:

˃ Borough of Manhattan: 249˃ Bronx: 118˃ Hostos: 82˃ Kingsborough: 247˃ LaGuardia: 208˃ Queensborough: 228

» Fall 2007 cohort was fully skills proficient at time of entry; 28% had developmental need when recruited and addressed over summer

» Since Fall 2009, began to admit students with some developmental need at time of entry

ASAP Goal» At least 50% of ASAP students will complete an

Associate’s degree in no more than three years ready to transfer to a baccalaureate program and/or enter the workforce.

ASAP Theory of ActionSupport students with:

1. Gaining and maintaining academic momentum

2. Developing a connection to the college

3. Accessing timely and relevant resources and services

Students are better engaged and graduate in a timely manner

ASAP Program Design

˃Required full-time study ˃Consolidated class schedule ˃Cohort design by major ˃Winter and summer course taking˃Dedicated full-time staff at each college

ASAP Resources & Services» ASAP Financial Resources:

˃ Tuition waivers for financial aid-eligible students˃ Free Use of Textbooks ˃ Monthly MTA MetroCards

» ASAP Services:˃ Case management advisement˃ Faculty engagement˃ ASAP Seminar˃ Academic support services˃ Career development services˃ Special programs

ASAP Evaluation and Key Outcomes

ASAP Data AnalysisQUANTITATIVE:» Student-level data from the CUNY Office of Institutional

Research and Assessment and colleges» Student Surveys

QUALITATIVE:» Annual Student Focus Groups

ASAP EVALUATION» Constructed Comparison Group (using CUNY data)˃ Entered one year prior to ASAP Cohort

˃ Met same criteria as ASAP students• Full-time Associate students• Begin program with 12 or fewer credits• Not enrolled in College Discovery (program offering similar services as

ASAP)• New York City residents• Enrolled in majors offered to ASAP students• Developmental Education

Cohort 1: Fully skills proficient in reading, writing and mathCohort 2: Allowed up to two remedial courses (in any subject)

ASAP Cohort One Three-Year Outcomes

Summary Profile: Cohort One ASAP and Comparison Group Students

Fall 2007 ASAP Fall 2006

Comparison Group

Total Enrollment N 1,132 1,791Gender

Male % 45.6 46.8Female % 54.4 53.2

Race/EthnicityAmerican Indian/Native Alaskan % 0.2 0.2Asian/Pacific Islander % 10.2 15.6Black % 31.5 26.8Hispanic % 37.2 29.2White % 20.8 28.2

Age Group18 or younger % 58.3 35.119 or 20 % 22.2 36.121 to 23 % 8.5 16.524 or older % 11.0 12.3

Median Age median 19 20Mean Age mean 20 21

Receiving a Pell Grant* % 57.9 67.1Income median $35,008 $29,231**

Admission TypeFirst-Time Freshmen % 75.2 36.9Transfer Students % 9.5 22.3Continuing Students % 15.3 40.8

*Based on dependent students only. **Based on 2007 comparison group's income because income data for 2006 comparison group was not available. Fall 2007 comparison group students met the same criteria as ASAP students but chose not to enroll in ASAP.

Regression Matching Techniques

» Used one-to-one propensity score matching˃ Led to large sample loss

» Used optimal full matching as final procedure˃ One-to-many matching procedure˃ Maintained most of ASAP sample˃ Conducted in partnership with Metis Associates (external evaluators)

ASAP Cohort One Three-Year Graduation Rate: Before and After Optimal Full Matching

Statistical Mean Difference = 28.4 Percentage Points, p < 0.05

0%

10%

20%

30%

40%

50%

60%

Before Matching After Matching

55.0% 54.6%

24.7% 26.9%

ASAP (Fall 2007) Comparison group (Fall 2006)

N=1,132 N=1,791 N=1,104 N=1,242

ASAP Cohort One Three-Year Cumulative Credits and GPA: After Optimal Full Matching

Statistical Mean Difference = 6.6 Percentage Points, p < 0.05

N=1,100 N=1,247

25

30

35

40

45

50

ASAP(Fall 2007)

Comparison group(Fall 2006)

47.3

40.7

˃ GPA Outcome – no statistically significant difference between groups (ASAP GPA: 2.50 , Comparison GPA: 2.46)

Graduation Predictors Study» Research Question

˃ Which variables are good predictors of two-year graduation • studied cumulative GPA, credits earned in 1st semester, and advisement

meetings in 2nd year˃ OUTCOME: Two-Year Graduation

» Sample˃ ASAP students enrolled for four consecutive semesters (N=761)

» Control Variables˃ Gender, Race, Age, Admission type, Parental education, Household income, High

school average, Regents’ scores

» Results˃ All three predictor variables significant

• as predictor variable increases so does probability of graduating in two years

16 meet

1 SD = 7 meetings

RESULT: probability of graduating in 2 years increases by 10 percentage points.

23 meet9 meet

2nd Year Advisement Meetings Outcome:

For every standard deviation increase in advisement meetings during the second year, approximately 7 meetings above the mean of 16.3 meetings, the probability of graduating increases by 10 percentage points.

ASAP Cohort Two Two-Year Outcomes

Summary Profile: Cohort Two ASAP and Comparison Group StudentsCohort 2 ASAP

Students

Fall 2008 Comparison

Group Students*

Total Enrollment N 429 1,510Gender

Male % 40.1 38.1Female % 59.9 61.9

Ethnicity

American Indian/Native Alaskan % 0.2 0.5Asian/Pacific Islander % 9.8 16.8Black % 35.4 27.6Hispanic % 37.5 41.2White % 17.0 13.9

Age Group

18 or younger % 41.0 33.219 % 18.2 17.820 to 22 % 18.4 25.823 to 29 % 13.1 15.730 or older % 9.3 7.5

Mean Age mean 22 21Admission Type

First-time Freshmen % 69.5 50.5Transfer Students % 7.0 17.5Continuing Students % 23.5 32.0

Developmental Students1 % 76.7 75.8Pell Receipt % 84.6 90.5Household Income

Dependent Students2 mean 26,103 26,042Independent Students mean 13,050 11,832

Retention of Cohort Two ASAP Students: Re-enrolled in College of Entry

50%

60%

70%

80%

90%

100%

2nd Semester 3rd Semester 4th Semester

93.9%

84.4%

77.6%

85.8%

66.9%

59.1%

ASAP (Fall 2009) Comparison group (Fall 2008)

One-Year Basic Skills Proficiency Change:ASAP Cohort Two and Comparison Group Students

0%

10%

20%

30%

40%

50%

60%

70%

80%

Dev. Need(s)1st Sem

Dev. Need(s)2nd Sem

Dev. Need(s)3rd Sem

76.0%

43.5%

16.4%

75.1%

59.9%

39.7%

ASAP (Fall 2009) Comparison group (Fall 2008)

Two-Year Graduation Rate of ASAP Cohort Two Students

0%

5%

10%

15%

20%

25%

30%

ASAP(Fall 2009)

Comparison group(Fall 2008)

27.5%

7.2%

ASAP Student Survey and Focus Group Results

ASAP Students’ Educational Aspirations

“I wouldn’t be in school (without ASAP)…I learned the sky is the limit.”

~ Cohort 2 student

0%

20%

40%

60%

80%

100%

Graduate/professional degree

4-year degree 2-year degree

69%

29%

2%

63%

34%

3%

ASAP cohort 1 (Fall 2007)

ASAP cohort 2 (Fall 2009)

Graduating Two Year ASAP Students :Most important ASAP Resource/Service to College Success

“Tuition and books being paid for really helps because there is less to worry about.”

~ Cohort 2 student

0%

20%

40%

60%

80%

100%

FinancialResources

Services - ASAPAdvisor

EarlyRegistration

All Other ASAPServices

70%

20%

2%7%

72%

19%

3% 6%

ASAP cohort 1 (Fall 2007)

ASAP cohort 2 (Fall 2009)

Graduating Cohort 2 Students: ASAP Resources “Very Helpful” to Student Success in College.

“Advisors track students academics; they play a huge role. If you have a problem, you go to your advisor.”

~ Cohort 2 student

0% 20% 40% 60% 80% 100%

Financial resources

Services - ASAPAdvisor

Early registration

Services - ASAP CES

Small class size

ASAP tutoring

Block programming

98%

92%

85%

64%

51%

49%

42%

Multiple responses allowed

Graduating Cohort 2 Students: Advisor Services

“It would be dramatically different without advisors. They know what you need. Without an advisor, the load would be on you to get what you need. It would not feel as personal.”

~ Cohort 2 student

0% 20% 40% 60% 80% 100%

Emphasized the importance of timemanagement

Made sure they were attending classregularly

Provided strategies to help deal withacademic problems

70%

66%

63%

28%

30%

32%

My ASAP Advisor Has:

Strongly Agree Agree

Graduating Cohort 2 Students: How ASAP Has Helped Me

“ASAP has done so many things to change my life…I am overwhelmed by the experience...”

“I’m involved more in college because of ASAP activities. It’s been a great experience.”

~ Cohort 2 students

0% 20% 40% 60% 80% 100%

Come closer to achieving my goals

Inspired by at least one of myprofessors

Learned to ask for help

75%

67%

65%

23%

28%

30%

As a Result of Being in the ASAP Program, I Have:

Strongly Agree Agree

Graduating Two Year ASAP Students :Post-Graduation Plans

“ASAP plays a major role by helping me stay focused”.

~ Cohort 2 student

0%

20%

40%

60%

80%

100%

Attend4-yr college

Work Other Plans

92%

32%

5%

94%

39%

3%

ASAP cohort 1 (Fall 2007)

ASAP cohort 2 (Fall 2009)

Multiple responses allowed

ASAP Focus Groups: Recurring Themes

» ASAP provides connection to the program and between classmates.

“ASAP has helped me to learn how to speak and interact with people. It has opened doors for me to be free.”

» ASAP encourages success, provides direction, and keeps students on track academically.

“ASAP provides a roadmap for what you should focus on.”

» ASAP gives students the skills to navigate the college experience and utilize available resources.

“ASAP tells you where to go, how to contact them, and who you need to talk to.”

» $5 million in additional foundation funds raised to expand ASAP program and evaluation capacity

˃ Transfer scholarship program for ASAP graduates at six CUNY senior colleges

˃ Random assignment study by MDRC at BMCC, Kingsborough, and LaGuardia

» Fall 2011: CUNY announce plans to expand ASAP to more students at CUNY community colleges

ASAP Updates

ASAP Expansion» Expanding ASAP over the next three years at all

community colleges to 4,000 + students by fall 2014» 1,500 will be recruited for fall 2012» Central & college planning teams developing

expansion plans» Citywide outreach & marketing campaign includes

ASAP graduates

Contact Us

www.cuny.edu/asap