building strong programs: lessons from working in the schools (wits)
DESCRIPTION
Building Strong Programs: Lessons from Working in the Schools (WITS). January 31 st , 2014. Discussion Topics. Introductions Overview of WITS Training, Expectations, & Strategies Evaluation and Assessments Small Group Discussions. WITS Video. Bridging the Gap. - PowerPoint PPT PresentationTRANSCRIPT
BUILDING STRONG PROGRAMS: LESSONS FROM WORKING IN THE SCHOOLS (WITS)January 31st, 2014
DISCUSSION TOPICS• Introductions
• Overview of WITS
• Training, Expectations, & Strategies
• Evaluation and Assessments
• Small Group Discussions
WITS VIDEO
WITS believes that when mentors focus their energy and time on the success of
our city’s students, their volunteer service directly impacts the lives of young people and helps support a
better public school system.
BRIDGING THE GAP
WITS PORTFOLIO OF PROGRAMS
Programs:• Mid-Day Mentoring• Workplace Mentoring • Early Childhood• Classroom Reading Tutors• Saturday Tutoring• Early Childhood Summer Program• WITSummer in the Parks
STAKEHOLDERS
*
Volunteers (community,
corporate groups,
universities, churches)
StudentsSchools
POSITIONING VOLUNTEERS FOR SUCCESS
Frequency and
Quality of Support
SettingAppropriat
e Expectatio
ns
Modeling,Coaching,Reflection
STRATEGIES FOR WORKING WITH YOUTH
OVERCOMING OBSTACLESModel Problem Solving• Name the problem• Talk through possible solutions• Model appropriate problem solving
Encourage the Student • Praise their process• They aren’t alone and—neither are
you!The Goal: Students to see themselves as capable problem solvers and valuable individuals
BUILDING STUDENT FLUENCY Model Fluency• Reading expressively• Use character voices• Modulating speed and
volume
Encourage the Student• Help student choose books• Re-read to build confidence
The Goal: students see reading as an opportunity to explore their interests—not a task to complete.
THE FIVE C’S OF POSITIVE YOUTH DEVELOPMENT (PYD)
PYD: focus on the strengths of youth and the positive qualities and outcomes we wish youth to develop
Competence Confidence Connection Character Caring
*
USING EVALUATION TO STRENGTHEN PROGRAMS
Types:• Formative• Summative
Methods:• experimental• quasi-experimental• observational
Considerations:• What is the purpose of the evaluation?• Who will use the evaluation results?• How will they use the evaluation results?• What do stakeholders need from the
evaluation?
*
THE POWER OF OBSERVATION (FORMATIVE EVALUATION)
Questions:• Are we meeting our program goals?
• Is there consistency across programs?
• What immediate improvements can we make?
• What changes can/should we make in the future?
• Use observation forms/standardize
WORKPLACE MENTORING LOGIC MODEL
Priorities
Inputs Outputs Outcomes
-need for consistent and caring mentors
-need for literacy enrichment programming in the afterschool space
-scaffold student self-efficacy
-scaffold student attitude
Assumptions
-all students are capable of becoming better readers
-students are motivated to participate in WITS programs
-volunteers are competent tutors/mentors
-students and volunteers form a positive relationship over time
-human resources: staff; corporate + university volunteer time investment
-partners: teacher/principal time investment; experts
-WITS Board; Associates Board
-financial resources
-build knowledge base
-other: transportation; bus chaperones; snacks; maintain program materials
identify external factors
SituationProgram Goals
EVALUATION
-effective 1:1 literacy-focused mentoring programs
-300 students from 13 schools participate
-19 corporate and university groups provide 600 volunteers
-curriculum implemented
-volunteers coached to provide student with effective support
Student:
-improved attitude toward reading
-increased self-efficacy
-improved reading fluency
Volunteer:
- sustained school/community partnership
-high volunteer satisfaction
-high volunteer retention
WITS EVALUATION GOALS
• mixed method approach
• multiple data sources
• triangulate data
• balance formative/summative
• make decisions on multiple findings
BEST PRACTICES• align data collection with program goals
• think before you collect
• weigh pros and cons of types of questions
• balance formative and summative
• pilot a new assessment
• ask the hard/unpopular questions
• consult peers
• know your research domain
EXAMPLE: LEVERAGE EXTERNAL DATA
BENEFITS OF WORKPLACE VOLUNTEERISM
2011 Deloitte Volunteer IMPACT Survey
EXAMPLE: LEVERAGE INTERNAL DATA
STUDENT-REPORTED SATISFACTION
Youth Emotional Engagement Benchmark Survey adapted from: Herrera, C. (2004). School-Based Mentoring. A Closer Look. Public/Private Ventures, 42.
EXAMPLE: LEVERAGE INTERNAL DATALENGTH OF PARTICIPATION IN WORKPLACE MENTORING PROGRAM
Enthusiasm Toward Read-
ing
Positive Attitude - Silent Reading
Positive Attitude - Oral Reading
Positive Attitude - School in General
0%
20%
40%
60%
80%
100%
Teacher Ratings of Student Motivation/Attitude Toward School and Reading
0 years minimum of 2 years
COMMON EVALUATION ERRORS
Avoid:• failing to leverage the data you already have• collecting more data than you really need/can
handle• using the same data points when a project has
changed• poor survey/question design
o wrong question typeo vague wording/jargono incorrect assumptionso leading questionso double-barrelingo missing response options o incorrect use of scales
SMALL GROUP DISCUSSION TOPICS
1. Discuss some challenges you have had when working with youth. What changes did you make to your program structure, training or curriculum due to those challenges?
2. How do you gain feedback from your volunteers/staff? Provide an example of feedback your organization has received and programmatic changes you have made due to the feedback?
3. Discuss manageable formative evaluation activities that you and your organization can add to improve program quality.
4. Discuss realistic summative evaluation activities that you and your organization can add to improve program quality.
5. How should an organization respond to negative evaluation results? How should an organization respond to positive results?
KEY IDEAS• Set clear and appropriate expectations!
• Constantly observe and troubleshoot
• Think critically (but constructively)
• Small organizations can easily take big steps toward evaluating program quality