evaluating tulsa’s two-generation careeradvance program ......two-generation programs !...
TRANSCRIPT
Evaluating Tulsa’s Two-Generation CareerAdvance® Program:
Early Indications
Christopher T. King Ray Marshall Center, LBJ School of Public Affairs
The University of Texas at Austin
Administration for Children and Families Welfare Research & Evaluation Conference
Grand Hyatt Hotel, Washington, D.C.
May 28, 2014
Acknowledgements: Collaborators
• Lindsay Chase-Lansdale, Teresa Eckrich Sommer, Terri J. Sabol, Mumbe Kithakye, Rayane Alamuddin, Emily Ross, Allison Frost & Katie Dahlke, Northwestern University
• Jeanne Brooks-Gunn, Columbia University • Tara C. Smith, Kristin Christensen, The
University of Texas at Austin • Hirokazu Yoshikawa, New York University • Celia Gomez, Harvard University • Steven Dow & Monica Barczak, Community
Action Project-Tulsa County
Acknowledgements: Funders
Funding for this project was provided by: • The U.S. Department of Health and Human
Services, Administration for Children and Families, Grant #90PH0020 & Grant #90FX00100
• W.K. Kellogg Foundation • Bill and Melinda Gates Foundation • George Kaiser Family Foundation
Ø Two-Generation Programs Ø CareerAdvance®: A Model 2-Gen Program Ø Evaluating CareerAdvance®
l The CareerAdvance® Implementation Study l The CAP Family Life Study
Ø Early Indications l Program Changes, Lessons Learned l Persistence Findings l Qualitative Findings via Participant Voices
Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views or policies of the Office of Planning, Research and Evaluation, the Administration for Children and Families, or the U.S. Department of Health and Human Services.
Presentation Overview
Ø Wide variation in two-generation program types Ø Our focus is on variant emphasizing human
capital investments, programs that intentionally and simultaneously connect: l High-quality education and job training
programs for parents and l High-quality early education for their young
children
Two-Generation Programs Defined
Theory of Change
• Improved cognitive and social development
• Higher attendance • Readiness for kindergarten
• Motivation to pursue education and careers
• Defined education and career goals
• Higher rates of adult basic education • Higher rates of education and career
training enrollment
Child
Parent
Two Generation Programs
Early Childhood Centers
• High-Quality Classrooms
• Family Support Services
PSE/Workforce Development
• High-Quality Community Colleges
• High-Quality Job Training Programs
• Employers
Theory of Change
• Higher rates of persistence in education and job training
• Improved job training skills and career development
• Higher rates of employment
• Higher wage growth
• Higher motivation and engagement in school
• Academic success in elementary school
• Social competence Child
Parent
Two Generation Programs
Early Childhood Centers
• High-Quality Classrooms
• Family Support Services
PSE/Workforce Development
• High-Quality Community Colleges
• High-Quality Job Training Programs
• Employers
Theory of Change
• Increased high school graduation rates
• Increased training and postsecondary education attainment
• High expectations and positive future orientation
• Stable career • Family supporting wage
• Greater life stability
• Better functioning family system
Child
Parent
Two Generation Programs
Early Childhood Centers
• High-Quality Classrooms
• Family Support Services
PSE/Workforce Development
• High-Quality Community Colleges
• High-Quality Job Training Programs
• Employers
CareerAdvance®: A Model 2-Generation Education and Workforce Program
Target Population: Parents of Head Start and Early Head Start children, mainly single mothers, though some fathers as well
Early Childhood Program
Career Coaches
Peer Support
Incentives & Support Services
Sectoral Training/
Career Paths Employers
Elementary Schools
Local Colleges
Basic Education &
ESL
CareerAdvance®
Evidence-Based Components Ø Stackable skills training in growth sectors offered
via structured career pathways (e.g., healthcare, healthcare IT, allied health)
Ø Contextually-based adult ed, GED, & ESL instruction
Ø Peer supports Ø Weekly career coach meetings Ø Mutual obligations agreements Ø Workforce intermediary services, connecting
participants, providers, and employers Ø Conditional cash (in-kind) incentives
Evaluating CareerAdvance® Implementation Study • Focus on program implementation and progress
for all participant cohorts (2009-2015) • Staff & partner interviews, focus groups, program
data analysis
CAP Family Life Study • Mixed-methods Approach: Quantitative &
Qualitative • Quasi-experimental • Longitudinal outcomes for Cohorts 4-10
(2011-2015)
Implementation Study Research Questions
1. How has CareerAdvance® changed over time and why?
2. What progress have CareerAdvance®
participants made?
3. What program and institutional factors contribute to or impede participant progress?
8 wks 4-8 wks 16 wks 15-17 mos. 3 years
Certified Nurse Assistant
$9-$12
CNA I
Geriatric Technician
$8.51-$14.13
Advanced Unlicensed Technician
$11-$15
Licensed Practical Nurse
$16-$20
Registered Nurse
$20-$30
Registered Nurse
CNA II
Patient Care Technician
Licensed Practical
Nurse
LPN to RN
Bridge
Program Changes: Nursing Career Ladder
Certified Medical Coder
$11-$20
Health Information Technology Associate $13-$30
Health Information Technology
Program
Medical Coding
Program
3 years 7 mos.
Program Changes: Health Information Technology (HIT) Ladder
Certified Medical Assistant
$10-$18
Medical Assistant Program
9 mos.
Program Changes: Medical Assisting
Changes in Basic Skills Preparation
Ø Participants’ basic skills measured by TABE® and COMPASS® tests
Ø Higher baseline basic skills in reading than math l Mean TABE® reading scores at 10th-12th grade level
l Mean TABE® math scores at 7th- 8th grade level
Ø Also, wide range of basic skills within cohorts and between pathways
l Some participants test at 1st & 2nd grade levels, others at 12th grade levels
Ø Group and individual tutoring Ø Advanced Nursing Skills course Ø Individual instruction at Sylvan Learning Ø Added new Educational Pathways Program (Jan.
2013) to build CAP parents’ basic skills for future program entry. Parents sorted into three groups: 1. ‘Skill-ready,’ testing at 6th-8th grade level
2. ‘College-bound,’ testing at 9th-12th grade level
3. ‘Career-ready,’ testing at college level
How CareerAdvance® Partners Strengthened Basic Skills Preparation
Factors Impeding Program Progress
Ø Time management (e.g., work/school/family balance) and perceived value of components
Ø Partner meeting concerns for participants further into their pathway
• Meetings frequency reduced
Ø New pathway start-up problems
Factors Supporting Program Progress Ø High parent motivation for obtaining a career, not a
job
Ø Peer supports and cohort approach
Ø Career coaches
Ø Performance-based financial incentives
Ø Customized program schedule
Ø CAP connection: “CAP treats you like family.”
Family Life Study Research Questions
1. Does participation in CareerAdvance ® relate to positive short- and long-term outcomes for parents and children?
2. Are there any negative outcomes?
3. To what extent do parent and child outcomes vary as a function of CareerAdvance® dosage and persistence?
Mixed Methods: Quantitative Ø Computer-assisted parent survey (90 mins.)
Ø Direct child assessments
Ø Teacher survey (i.e., basic/professional demographics; classroom characteristics; child’s social skills, executive functioning, temperament; quality of parent relationship)
Ø Oklahoma administrative data (e.g., UI employment and earnings, TANF, SNAP records)
Ø Head Start Agency (CAP Child-Plus) data (e.g., dosage, persistence, baseline characteristics)
Overview of Constructs: Parent Survey
• Demographic characteristics • Education, employment, & income • Mobility • Self-confidence and self-efficacy • Academic expectations for child • Parenting attitudes and practices • Mental and physical health • Moving from job to career
Parent Survey: Overview of Constructs
Psychological Measures l Conscientiousness l Optimism l Self-Esteem l Impulsivity l Financial Worry l Psychological Distress (K6, Kessler et al. 2002): 6%
of CAP Family Life Study sample reported “severe psychological distress” v. 3% in large NHIS sample
l Material Hardship (6 item scale adapted from New Hope)
Overview of Measures: Child Assessments
Basic Academic Skills
Bracken School Readiness Scale
Math
Woodcock Johnson Achievement Test- Applied Problems
Executive Functioning
Pencil Tap
Language
Peabody Picture Vocabulary Test (PPVT-III)
Gift task
Internal Representations Adapted Berkeley Puppet Interview
CAP Family Life Study: Sample (Cohort 4-6; n=163)
Parent Demographics M(SD) Age 29.47(6.64) Male 3% Adult race/ethnicity (%) White 29% Black/African American 40% American Indian 7% Latino 10% Other/Two or more races/ethnicities 13% Country of origin (%) United States 94% Mexico 5% European Country 1%
CAP Family Life Study: Sample (Cohort 4-6; n=163)
Household Characteristics and Family Structure M(SD) Household Size 4.31(1.40) Number of children in home <18 2.27(1.06) Income-to-needs ratio 1.13(0.73) Relationship Status Married 34% Steady romantic relationship 31% On-again/off-again relationship 8% Single, not in a relationship 27%
0 5
10 15 20 25 30 35 40
Perc
ent
Baseline Parent Education Level Cohort 4- 6 (n=163)
Parent Interview and Child Assessment Year 1 Year 2 Year 3 Year 4 Year 5
Cohort 2011 2012 2013 2014 2015
S F S F S F S F S F 4 60 60 60 60 5 60 60 60 60 6 60 60 60 60 7 60 60 60 8 60 60 60 9 60 60
10 60 60 Total Interviews/
Assessments Per Year
60 180 300 420 420
* S=Spring; F=Fall
Data Collection: Quantitative
Preliminary Persistence Findings
Program /Study Sample Results
Opening Doors Program (2003-2004)
537 low-income students across two community colleges
32% enrolled one year later
New Chance Demonstration (1989-1992)
1,401 low-income, young mothers
52% attained GED or diploma after 42 months
Learning, Earning, and Parenting Program (LEAP) (1987-1997)
267 participants enrolled in school
46% attained GED or diploma after 36 months
Teenage Parent Demonstration (TPD) (1987-1991)
1,739 participants 56% attained GED or high school diploma
Targeted Educational Programs
Program /Study Sample Results
Year-Up (2007-2008)
120 participants (18-26 years)
64% participants completed year-long program
Project QUEST, Inc. (1998-1999)
1,059 participants over 3 years
72% average graduation rate across 3 years
Job Training Partnership Act Title II-A Programs (1987-1989)
6,102 adult women across 16 sites
32% of women who needed a GED or diploma received one within 30 months
Past Workforce Training Programs
Cohort 4 Weeks 0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 136 144 152 160 168 176 184 192 200
ANS
CNA 1
CNA2
CN
A3
PCT RN Pre Req RN
LPN RN Pre Req RN Cohort 5 Weeks 0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 136 144 152 160 168 176 184 192 200
ANS
CNA 1
CNA2
PCT RN Pre Req RN
LPN RN Pre Req RN
KEY
Cer%fica%on
Pink = CNA Courses
Blue= PCT Track
Orange = LPN Track
Green = RN Courses
(1 semester = 16 weeks)
Certification
Cohort 6 Weeks 0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 136 144 152 160 168 176 184 192 200
ANS
CNA 1
CNA2
PCT RN Pre Req RN
LPN RN Pre Req RN Cohort 7 Weeks 0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 136 144 152 160 168 176 184 192 200
ANS
CNA1
CNA2
PCT RN Pre Req RN
LPN RN Pre Req RN
CareerAdvance® Nursing Cohorts 4-7 (n=99)
Defining Educational Success in CareerAdvance®
Ø Persistence l Enrolled at end of year one l Number of days enrolled
Ø Transitions l Exit->re-enter
Ø Educational advancement l Certificates (number, proportion, y/n) l Classes (number, proportion)
Ø Program participation l Number of partner meetings
Enrollment & Certification in CareerAdvance® at One Year (Cohorts 4-7)
CareerAdvance® Participants
(n=99)
Exited (n=32)
Attained certificate
(n=15)
Did not attain certificate
(n=17)
Enrolled (n=67)
Attained certificate
(n=51)
Did not attain certificate
(n=16) 67% enrolled at end of Yr 1 66% attained certificate by end of Yr 1
Predictors of Persistence in CareerAdvance® (Cohorts 4-7; n=99)
Probability of educational success after one year
(odds ratios)
Financial Circumstances Enrollment Certification
Material Hardship 1.88* 1.69*
Income to needs 1.41 2.08* Employed adult in home 0.95 0.91
Psychological wellbeing Psychological distress 0.77 0.53* Optimism 1.20 0.66
Includes controls for baseline demographic characteristics *p<.05
Predictors of Persistence in CareerAdvance®
In-depth interviews at baseline, 2 Cohorts (n=21)
• Among parents who expressed financial worry at baseline, many completed certification in first year
• Among parents who did not advance academically in year one, most described psychological distress
Predictors of Persistence in CareerAdvance®
In-depth interviews (n=21)
• Almost all parents expressed high levels of educational motivation
• Among parents who completed certification
• Describe plans for financial impact of program
• Identify potential barriers to or worries about potential to succeed
Preliminary Qualitative Findings
Types of Annual Data Collection
PARENT INTERVIEWS
FOCUS GROUPS, PARENTS
FOCUS GROUPS,
STAFF
CA Parents
MC Parents
CA Exits
CA Coaches & Program Manager
Family Support Staff
Once-in-a-Lifetime Opportunity
It’s amazing. I mean, you can’t go anywhere to where any program will literally pay for anything and support you all the way. That’s what amazed me the most - the before and after care, the tuition, the scrubs, stethoscopes, books, dictionaries, shoes, watches, gas money.... You’d seem ungrateful if you dropped out and said ‘No I’m not doing that.’ You really wouldn’t have an explanation.
Increased Confidence
It’s been so long since I’ve been in school. I spent almost the last 5 years at home being a mom, and you start doubting yourself and you don’t think you can do it. Once you’re back in the routine of it, and you’re like, ‘Yeah, it’s still in there. Just clean off the cobwebs.’
Coordinated Parent-Child Schedule
I like how they’ve made the program fit around the youngest child’s schedule... how they’ve tailored it to fit around those hours, which really would tailor around all school-age children’s hours. So only during clinical times do you have to really worry about before and after care. But for the most part, all of us can still take the kids, kiss them goodbye, do our thing, and then be there to pick them up.
Extraordinary Peer Support
I know if I tried to leave this program, I would have some people on my phone. And that’s the good thing about us being, that’s the one good thing about us being a small group of people. If one of us tried to leave it, oh, we gonna be on that phone quick, ‘Wait a minute what are you doing?’
Intensive Support from Career Coaches and Center Staff
My favorite part is so much support we’re getting. We can pretty much call her [the coach] anytime and be like, you know.... We constantly have the support not only from our classmates but also from our teachers and our coach. You know, and when I was in college before, it was just me against the world basically you know. So if I dropped out, nobody cared. It was just, I was only just disappointing myself. Now if anybody is missing too much class we’d call them and are like, you know ‘Where are you at? Come to class’.
Stress and Strain
It’s just harder for me because now I’m having to work, like, with any free time that I have, on the weekends or at nights or anything else. You know, me and my husband, If it comes down to choosing between school and work, I have to pick work.
Engagement and Learning
I have found on a positive note, what school has done in our house is…Like my 9-year old has always struggled in math. And I have always struggled in math. It’s never been a strong suit. I’ve always told her that, you know, sorry I can’t really help you. And she’s relied on that, “Well mommy can’t help me. She doesn’t get numbers…”
Engagement and Learning …
Well when I got put in this math tutoring class, I felt like I could then relate to her more, and I felt like it was empowering me because it was giving me those skills that I left behind somewhere in high school and junior high. And so when I would get home, for the first couple of weeks, I’d be like, “I can help you.” She’s like, “No you can’t, you don’t know how to do this” and I was like, “No, really, I know how to do it now.”
Engagement and Learning …
So I feel like, I wasn’t getting so upset with her because now I know the material and understand it and I’m getting it so it’s helping her to feel better about herself, and I feel better about myself because for all those years, it was embarrassing to tell your 9-year old, “Sorry I can’t help you with this because I don’t know it myself…So I feel like that’s been a positive is that I can guide them better now, that I have the information, I can help them better.
Role Modeling
I’m the first person to even go to school. So it feels good to me to just know that I’m gonna make a better, like pave a better path for my son. The chances of him going to school if I complete school are so much higher. And that’s you know, not only will I create a better life for him as a child, but it’ll give him some encouragement and motivation, and I can be a better role model for him to go to school when he’s older. So it makes me feel a lot better I think.
Role Modeling …
My son knows at 4 years old that he’s not stopping his education after high school, he’s gonna keep going. And he knows that now, and he’s, you know...I think he’s gonna be much more prepared than I was when I was in high school.
Less Time with Children
So it’s really hard cause you want to spend time with your family but you don’t have the time to spend with them you want to. Or you have your, you know, your toddler coming up to you, “Mama, mama” and you’re like, oh, I have to write a paper, you know, that’s really hard.
“This program has changed my life; it’s changed my future, my family’s future definitely. I mean, this has opened up so many opportunities for me and my family.”
Hope for the Future
Ø Long-term follow-up of parents and children in CAP Family Life Study
Ø Deepen understanding of two-gen programs Ø Scale-up, sustain CareerAdvance® and expand
beyond Tulsa Ø Conduct RCTs with varying 2-gen program
models Ø Develop strategies for informing broad policy and
practitioner communities Ø Address tension between fostering political will
and building research evidence
Future Goals