partnership design and evaluation...partnership design and evaluation mnscu webinar june 17, 2015...
TRANSCRIPT
![Page 1: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/1.jpg)
![Page 2: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/2.jpg)
Partnership Design and Evaluation
MNSCU Webinar June 17, 2015
Presented by
Brett Pawlowski
![Page 3: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/3.jpg)
Every Teen with a Dream and a Plan
Every Community with a
Capable, Ready Workforce
MISSION
![Page 4: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/4.jpg)
Resources from NC3T
![Page 5: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/5.jpg)
www.Amazon.com
![Page 6: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/6.jpg)
www.ACTEOnline.org
![Page 7: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/7.jpg)
www.ACTEOnline.com
• Partnership essentials
• Partnership models
• How to recruit
• Measurement
• Sustainability
• Advisory boards
• Case studies
![Page 8: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/8.jpg)
Today’s Agenda
• Goal Setting
• Partnership Design
• Evaluation
![Page 9: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/9.jpg)
I. Goal Setting
![Page 10: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/10.jpg)
Why Set/Measure Goals?
• Make sure partnership is
accomplishing something real:
Not just “feel good”
• Prove the value to stakeholders
• Benchmark/Improve
• Encourage future partnerships
![Page 11: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/11.jpg)
Why Set/Measure Goals?
• Partnership Selection Criteria–Willingness to collaborate: 84.7%
– Interest in measurable outcomes: 67.6%
– Commitment to the project: 64.0%
–Needs of the student body: 48.6%
– Existing relationship with coalition: 33.3%
–Accessibility to location: 21.6%
Source: Coalition Leaders Speak Out on Education, DeHavilland Associates, 2007
![Page 12: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/12.jpg)
Case Study: DARE
• Common sense didn’t lead to
real-world outcomes
•Measurement provided a
chance to
fix things
![Page 13: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/13.jpg)
SMART Goals
![Page 14: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/14.jpg)
Goals for Each Stakeholder
• Student outcomes: first and
foremost
• Also … business partner goals
• Also … school/program goals
![Page 15: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/15.jpg)
Sample Student Goals
• Increase graduation rate to 80%
• Increase number of students
receiving industry certifications
to 40%
• Increase student understanding
of an industry/occupation
![Page 16: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/16.jpg)
More on Student Goals
• Research and
case studies at
NC3T.com
![Page 17: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/17.jpg)
Sample Business Goals
• Hire five qualified machinists from the program each year
• Improve soft skills of the local labor pool
• Boost employee morale
![Page 18: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/18.jpg)
Sample Education Goals
• Improve measured education statistics (grad rates, state assessments, etc.)
• Keep teachers/professors current on industry expectations
• Improve student engagement (attendance, participation, etc.)
![Page 19: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/19.jpg)
Interim Goals
• Improving postsecondary completion rates = long term
• Points along the path?–Student/parent awareness
–FAFSA application
–Earning postsecondary credits
–Industry connections
![Page 20: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/20.jpg)
Final Thought
There is a difference between
what is easy to
measure and what
is important
to measure
![Page 21: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/21.jpg)
II. Partnership Design
![Page 22: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/22.jpg)
The Logic Model
Lays out the thinking that drives your partnership
1. Where you’re starting
2. What is your
intervention
3. What is your target
![Page 23: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/23.jpg)
The Logic Model
Development Order:
3, 1, 2
1. Where you’re starting
2. What is your
intervention
3. What is your target
![Page 24: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/24.jpg)
End Point, Start Point
• End point: already set through goal
setting process (for all stakeholders)
• Starting Point:
Where are you
starting from?
![Page 25: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/25.jpg)
Intervention
• How are you going to get from here to
there?
![Page 26: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/26.jpg)
Intervention
Three components:
• Resources
• Activities
• Participants
![Page 27: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/27.jpg)
Resources
What it takes to fuel your partnership:
• Administration
• Facilities/transportation
• Funding
• Expertise
![Page 28: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/28.jpg)
Activities
Nine partnership models:
• Advisory boards
• Expertise
• Mentors
• Real-world challenges
![Page 29: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/29.jpg)
Activities
• Student work-based learning
• Teacher learning experiences
• Advocacy
• Resource support
• Program start-up or retrofit
![Page 30: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/30.jpg)
Participants
Stakeholders and Roles
• Students
• Volunteers/mentors
• Administrators
• Parents
![Page 31: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/31.jpg)
The Full Logic Model
![Page 32: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/32.jpg)
Logic Model: An Example
You and your partner(s) decide:
• Grad rate to 85%
• 70% of grads pursue or
complete postsecondary
education
• 35% go into manufacturing
• Partners hire 15 students next
year, 20 the year after, and 25
the year after that
![Page 33: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/33.jpg)
Logic Model: An Example
Where are you starting?
• Grad rate at 80%
• 60% of grads pursue
postsecondary education
• 25% go into manufacturing
• Partners hire 10 students/year
![Page 34: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/34.jpg)
Logic Model: An Example
You look at your resources:
• Administrative support
• Facilities/transportation
• Business partners who can
offer volunteers, mentors, and
site visits
• Partner funds for incidentals
![Page 35: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/35.jpg)
Logic Model: An Example
You decide on activities:
• Career exploration for all
students – guest speakers, site
tours, career fairs, online
exploration
• Career mentoring for select
students in upper grades
![Page 36: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/36.jpg)
Logic Model: An Example
You identify participants:
• All students grades 10/11
• Select students grade 12
• 30 company employees (10 for
exploration, 20 for mentoring)
• Administrators (business/ed)
• Parents
![Page 37: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/37.jpg)
Logic Model: An Example
![Page 38: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/38.jpg)
III. Measurement/Evaluation
![Page 39: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/39.jpg)
ROI: Return on Investment
What do your stakeholders
put in?
What do they
get out?
![Page 40: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/40.jpg)
Two Types of Data
Qualitative = “Words”
• Focus groups
• Interviews
• Observations
• Journals
![Page 41: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/41.jpg)
Two Types of Data
Quantitative = “Numbers”
• Number of students hired
• Graduation rate
• Attendance figures
• Test scores
• Survey results
![Page 42: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/42.jpg)
Student Outcomes
• School-based statistics
– Graduation rates
– Attendance rates
– Incident rates
– Scores on state assessments
– Other official assessments (WorkKeys, SAT,
ACT, etc.)
– Student grades
![Page 43: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/43.jpg)
Student Outcomes
• Employability skills
• Industry certifications
• Work experience
• Work product (projects, portfolios)
• Extracurriculars
• Course selection
• Education/career plans
![Page 44: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/44.jpg)
Business Outcomes
• Hiring
• Labor pool measurements – quality of
graduates
• Employee numbers
– Morale
– Retention
– Skill development
• Branding/Awareness
![Page 45: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/45.jpg)
Education Outcomes
• Student outcomes
• Teacher measures
– Quality (certifications?)
– Morale
– Retention
• Resources/Volunteer hours
• Program performance
![Page 46: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/46.jpg)
What Do You Do with This Data?
• Share it!
• Keep getting
better!
![Page 47: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/47.jpg)
Effective Practices
• Don’t overreach
• Get a baseline
• Measure consistently
• Train your people
• Review and modify
your program based
on data
![Page 48: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/48.jpg)
Effective Practices
For larger-scale initiatives:
• Independent evaluation
• Random assignment
• Test/control model
![Page 49: Partnership Design and Evaluation...Partnership Design and Evaluation MNSCU Webinar June 17, 2015 Presentedby Brett Pawlowski. Every Teen with a Dream and a Plan Every Community with](https://reader034.vdocuments.mx/reader034/viewer/2022042415/5f3009ab91e8026b402bf9fc/html5/thumbnails/49.jpg)
Questions?
Type questions directly into panel box