national council for workforce education (ncwe) darlene g. miller, ed.d., executive director...
TRANSCRIPT
National Council for Workforce Education (NCWE)Darlene G. Miller, Ed.D., Executive Director
Heldrich Center for Workforce Development, Rutgers UniversityMichelle Van Noy, Ph.D., ResearcherJennifer Cleary, Ph.D., Researcher
Burning Glass TechnologiesKelly R. Bailey, Director of Partnerships and Alliances
Aligning Workforce Programs with the Labor Market: Strategies and Tools for TAACCCT Applications
What this presentation can do for you
Provide a framework for thinking about how to build TAACCCT-funded workforce programs that align with the labor market, as well as the needs of students and other stakeholders.
Offer tools and information to aid in building strong programs.
Provide tips on how to build an evaluation-friendly workforce program.
Overview of SGA and Labor Market Alignment
USDOL Priorities Challenge Colleges to Create New Program Models
Key Goals: Align programs with the needs of the labor
market and students Use sector-based & evidence-based approachesEvaluate program implementation and
outcomes
Data systems and administration goalsOpen sharing of curriculaCoordinated data collection across programs
Evidence-Based Design
Three levels of evidence:
Strong
Moderate
Evidence of Promise
Strong Theory
Priority program approaches:
• Sector-based• Career Pathways & Stackable
Credentials• Employers involved in ALL PARTS OF
PROGRAM• Use Real-time and other LMI (program
selection, student advising)• On-line learning• Accelerated, contextualized remedial
education• Competency-based & Work-based
learning (including entrepreneurship)• Share outcomes and LMI with students • Credit granting for non-credit work
Higher Education Labor Market Program-level Alignment Model
Program Selection
Recruitment &
Enrollment
Curriculum, Teaching,
& Assessmen
t
Support Services
Employer Needs
Student Needs
Data Collection/Validation
Incorporate
Relationship-building
Outcomes
Credential Attainment
Employment outcomes
Employer/Student Satisfaction
Incorporating data collected into the program is a multifaceted, complex process. It is more than program selection!
Employer & Student Needs
Contextual Factors
•Institutional Factors
•Program/ Institutional Structure
•Labor Market Dynamics
The SGA requires alignment activities occur at many organizational levels
System
Class
- Data systems to share student outcomes- Policies for open sharing curriculaPolicies/systems/facilities to
allow:- Credit for non-credit work- Flexible schedule /enrollment- IT/On-line instruction- Accelerated & competency-based learning - Work-based/hands-on learning
- Policies to allow for sharing of faculty for contextualized ed.
Systems and skilled staff for: -Data collection and analysis-Incorporating data into program selection and all program activities-Measure interim goals-Build relationships with stakeholders
- Effective pedagogy for contextualized and accelerated learning
Strategies for Successful Program Design
Alignment Activities: Systemic Change
Grant Leads to System Change within the Institution (not a boutique program)
Embedded into the culture of the institution Difficulty of navigating internal barriers
College level commitment Administrative Engagement Cross-functional Leadership Team
For Consortiums Consortia Leadership Team Consortium Director
Alignment Activities: Relationship Building
Relationships to Improve Outcomes andMeet Grant Priorities
DOL Priority on Long Term Unemployed Relationship with Workforce Investment Boards
Strong Employer Engagement and Commitment Career pathways and stackable credentials
Employer validation and data validation Employer validation of supply/demand Role of Industry Associations Employer commitment to work-based learning
Alignment Activities: Relationship Building
Relationships to Improve Outcomes andMeet Grant Priorities
For Consortium Grants – Peer Learning Networks Learning best practices from each other Sharing curriculum: Don’t reinvent the wheel Online and face-to-face
Platform and compatibility Data collection and sharing
Data systems in synch
Alignment Activities: Data Collection and Validation
Alignment with Needs of Labor Market and Students
Real Time Jobs Analysis* Allows understanding of job vacancy and skill needs of
local labor market Utilizing multiple “reliable” sources
LMI Data, Local WIB data, state data, industry association data
Gap analysis of competing programs What are other colleges in your area doing What about local CBOs or apprenticeship programs Understand available supply to fill job vacancies
Alignment Activities: Data Collection and Validation
Alignment with Needs of Labor Market and Students
Convening stakeholder groups to collect information on job vacancy and skill demand Multiple relationship building activities (WIBs, employers, etc) Ensures multiple sources of evidence for demand and supply
Validation by the right person (HR manager v CEO) Allows for collection of skills information for curriculum
development Locally recognized industry-based credentials
Aligning enrollment targets with job vacancies and gaps Employment data over time
Alignment Activities: Program Implementation
Curriculum Development Open sharing of curriculum
Curriculum is no long proprietary or owned by the faculty Open Sharing On-line using DOL platform
• Faculty support in budget for making the curriculum public and accessible
Acceleration strategies Evidence-based design Integration and contextualization for low-skilled adult
learners
Alignment Activities: Program Implementation
Curriculum Development Credit vs Noncredit
Noncredit must be linked to industry-recognized credentials (NIMS, NCCER, etc)
Articulation between noncredit and credit Acceleration and Credit for Prior Learning
BE CLEAR WITH THE FACULTY UPFRONT REGARDING PLA College policy Accreditation Transparency
“Real Time” Jobs Data
Evaluation Approaches and Considerations
Evaluation Consideration : Outcomes Measurement and Program Design
Randomized Controlled Trials Plan for recruitment and assignment?
Regression Discontinuity Assessment tests with cut-off scores? Common across
colleges?Comparison Group with Controls
Drawn from similar programs at college either from prior or current years
Evaluation Consideration : Outcomes Measurement & Data Collection
Develop a plan to collect key student data Including Student Name, SSN, DOB
Labor market outcomes data Partnerships with state to link with UI wage data
Other important data also need to be collected Courses, grades, services received, assessments, scores, #
attempts, activity in on-line courses, attendance
Plan to store data & ensure consistent data entry
Evaluation Considerations: Program Implementation Goals
Document implementation to inform outcomes analysis Provide detail on how program was implemented
including strengths and weaknesses
Uncover opportunities to strengthen implementation Evaluation can be an opportunity for on-going
learning, reflection, and feedback
Evaluation Considerations: Program Implementation Components
Multiple data collection efforts to provide a full understanding of implementation may include: Focus groups Surveys Interviews
Include the perspectives from a variety of stakeholders including: College staff, students, employers, other partners
Evaluation Considerations: Program Implementation Issues
How did you choose program design? Employer involvement? Evidence?
Does the delivery method fit student and employer needs? Are the assessments sufficient and well implemented?
Abilities, skills and interests What ? Who? How implemented? How used?
Is the program structure sufficient to ensure data collected on skills, etc. is incorporated in all program components? Is there a communication or quality control system in place to ensure all parts
of the program are coordinated around stakeholder priorities. Are your supportive services directly addressing employer, as well
as, student needs? Career guidance for students, support of employer skill priorities Are your partners involved in all aspects of the program
What relationship building activities have you created to recruit, retain, and engage partners?
Final Thoughts
Final Thoughts
Create a dashboard of metrics right up front along with a workplan for all of the partners Make sure all of your partners can live with the plan and
understand their role in accountability Don’t bite off more than you can chew
Be realistic in your outcomes Just because you think a reviewer wants to see this,
don’t put things into the grant that you cannot accomplish
Increased scrutiny by DOL Congress wants to know that their money was well spent Importance of a strong evaluation plan Take audits seriously
NCWE Team of TechnicalAssistance Providers
Darlene G. Miller, Ed.D., NCWE Executive Director
Nan Poppe, Ed.D., Campus President Emeritus, Portland Community College
Rebecca Nickoli, Ed.D., former Vice President of Corporate College (Workforce and Economic Development), Ivy Tech Community College
Nick Kremer, Chair, CA Vocational Research and Accountability Committee and former Dean of Career and Technical Education, Cerritos College
For More Information…
Darlene Miller, [email protected] 603-714-1918
Michelle Van Noy, [email protected] 732-932-4100 x6213
Jennifer Cleary, [email protected] 732-932-4100 x6215
Kelly R. Bailey, [email protected] 732-800-2484