circle program presentation
TRANSCRIPT
The New Hampshire Circle Program Final Presentation
Derek Gower, Stacy Martinez, and Phillip Pless
2017 M.P.P. Candidates at The College of William & Mary
December 2016
“I met my second mom through the program. She forever changed my life. I owe so much to the camp.”
-Circle Program Alumna
Project Deliverables
Entrance and Exit Survey
InstrumentsLiterature Review Analysis of Survey
Results
SWOT Analysis Program
Recommendations
Catalano et al. Meta-Analysis of PYD Efficacy
Studies included review of programs of one, two, or two or more social domains
School
Family
Community
Experimental or quasi-experimental design studies
Improvements in school attendance, academic performance, and peer emotional support
Increased communication and cognitive competence
Program findings not generalizable
Successful Characteristics of PYD Programs
Include at least five PYD constructs
Measure positive and problematic outcomes
Structured curriculum
At least 9 months or greater in length
Dubois et al. (Meta-Analysis)
Meta-analysis reviewed 55 studies on the positive effects of mentoring programs on youth
Results of a meta-analysis provides support for effectiveness of youth mentoring programs
Results of a fixed-effects model analysis indicate an overall or modest positive effect for specific mentoring programs
Theory-based mentoring programs (i.e. those that provided ongoing training to mentors and structured activities with mentors and youth) tended to higher effect sizes than others
Having ongoing support for mentoring relationships is an important predictor for larger effect sizes
Intensity and quality of relationships is also an important influencer
Meta-Analysis Limitations
Findings do not necessarily reflect the causal effects of either mentoring or different moderator variables examined.
Methodologies of experimental design differed (some random assignment versus others)
Generalizability of findings are also a significant issue of concern
This study doesn’t include potential changes in intervention design that could impact the findings
Other studies (e.g. Dubois and Silverthorn) suggest that the overall effects of mentoring is small
Research Gaps
Some studies (4-H) under-represent minorities in sample
Effectiveness of PYD on substance use not definitive
No common mentoring theoretical framework may impact program evaluations
“The Circle Program forced me out of my comfort zone…it taught me about responsibility and reward, and my experiences there are now among my most cherished memories”
-Circle Program Alumna
Analysis of Teen Survey Results
Demographics (11 Questions)
Program Involvement (4 Questions)
Positive Youth Development (PYD)
Confidence (6 Questions)
Connections (3 Questions)
Character (8 Questions)
Caring (3 Questions)
Competency (3 Questions)
Contribution (3 Questions)
Teen Survey: Demographics
Eleven questions developed measured demographic information
*n=25; Teen Survey Question #1
0
1
2
3
4
5
6
7
8
9
14 15 16 17
Nu
mb
er o
f R
esp
on
den
ts
What is your age?
0
1
2
3
4
5
6
7
8
9
10
Freshman (9thGrade)
Sophomore(10th Grade)
Junior (11thGrade)
Senior (12thGrade)
Nu
mb
er o
f R
esp
on
den
ts
What grade are you currently in?
Teen Survey: Competence
Three questions were developed to investigate enhancement in social, emotional, cognitive, and vocational ability.
91.7%
8.3%
Since Circle Program: Do you feel that you think more about what you'd like
to do after High School?
Yes
No
*n=24; Teen Survey Question #11b
64.0%
54.2%
36.0%
45.8%
0%
10%
20%
30%
40%
50%
60%
70%
Before Circle Program After Circle Program
Perc
ent
of
Resp
ondents
Did you generally feel that you had trouble focusing or concentrating?
Yes No
Teen Survey: Confidence
Six questions were developed to measure improvements in self-esteem, self-confidence, and self-efficacy.
40%
92%
60%
8%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Before Circle Program Since Circle Program
Pe
rce
nt
of
Re
spo
nd
en
ts
Before & Since Circle Program: Do you generally feel confident?
Yes No
75%
96%
25%
4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Before Circle Program Since Circle Program
Pe
rce
nt
of
Re
spo
nd
en
ts
Before & Since Cicle Program: Do you generally feel proud of your accomplishments?
Yes No
*n=25; Teen Survey Question #8a+14a *n=25; Teen Survey Question #8b+15b
Teen Survey: Connections
Six questions were developed that aim to measure the building or strengthening in connections with people and institutions.
79%
100%
21%
0%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Before Circle Program Since Circle Program
Pe
rce
nt
of
Re
spo
nd
en
ts
Before & Since Circle Program: Did you generally feel that you had people you could
talk to about your problems?
Yes No
*n=24; Teen Survey Question #9a+15a
Teen Survey: Character
Eight questions were developed to measure enhancement in character development through increased self-control, and increased respect for society rules.
96%
4%
Since Circle Program: Did you generally feel like you have a better understanding of right
and wrong?Yes
No
55%
29%
45%
71%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Before Circle Program Since Circle Program
Pe
rce
nt
of
Re
spo
nd
en
ts
Before & Since Circle Program: Did you generally feel that you didn't want to go to school?
Yes No
*n=24; Teen Survey Question #9d+15d *n=25; Teen Survey Question #11e
Teen Survey: Caring
Three questions assess an increase capacity in girls for caring through empathy and identification with others.
92%100%
8%0%
0%
20%
40%
60%
80%
100%
Before Circle Program Since Circle Program
Pe
rce
nt
of
Re
spo
nd
en
ts
Before & Since Circle Program: Did you generally feel that you listened to and cared
about other people?
Yes
No
92%
8%
0%
20%
40%
60%
80%
100%
Yes No
Pe
rce
nt
of
Re
spo
nd
en
ts
Since the Circle Program: Do you feel like you care more about your friends
in general?
*n=25; Teen Survey Question #9b&15b *n=25; Teen Survey Question #11d
Teen Survey: Contribution
33%
8%
58%
0%
10%
20%
30%
40%
50%
60%
70%
Yes No No, but I would liketo in the future
Pe
rce
nt
of
Re
spo
nd
en
ts
Have you served as a peer-mentor for other girls in summer camp?
0
1
2
3
4
5
6
7
8
9
10
None 1 2 3 4 5
Nu
mb
er
of
Re
spo
nd
en
ts
Number of Extra-Curricular Activities
How many extra-curricular activities are girls involved in outside of school?
*n=24; Teen Survey Question #18 *n=25; Teen Survey Question #4
Teen Survey: Program Involvement
Four questions aimed to measure program involvement
25%
21%
38%
13%
4%
0%
5%
10%
15%
20%
25%
30%
35%
40%
More thanonce aweek
Once aweek
Two ormore times
a month
Once amonth
Less thanonce permonth
Pe
rce
nt
of
Re
spo
nd
en
ts
How often are you in contact with your Circle Program mentor?
87.5%
12.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes No
Pe
rce
nt
of
Re
spo
nd
en
ts
Are you still in contact with friends you made at camp during the school
year?
*n=25; Teen Survey Question #19 *n=25; Teen Survey Question #17
Entrance and Exit Survey:
Lessons Learned from Teen Survey Assessment
More Likert Scale Questions
Simplify wording (avoid double negatives)
Need for open-ended questions
“The Circle Program gave me a sense of home when I did not feel like I had one.”
-Circle Program Alumna
Survey Design- Alumni PYD
Research and Teen Survey
Build off Previous
Work
What do we want to know?
Tailor Survey to
Adults
Audience
Readability
Simplicity
Recall Bias
Original Program
Questions
6 C’s
Personal and Financial Security
Anecdotal Free Response
Demographic
Survey Design- Alumni
Total of 35 Questions
11 Demographic
14 relating to 6 C’s
3 Financial Stability
3 Personal Stability
4 Free Response
Survey Analysis- Alumni
“The lessons learned in Circle have enabled me to go through day to day experiences and identify the positives from each day. Circle has taught me to trust others, to create friendships, to work hard, and to appreciate everything in life. Thank you.”
– Circle Alumna
Survey Analysis- Alumni
0
2
4
6
8
10
12
4th Grade 5th Grade 6th Grade
# o
f Resp
ondents
What Grade were you in when you entered the Circle Program?
0
2
4
6
8
10
12
14
6th Grade 7th Grade 8th Grade 9th Grade 10th Grade 11th Grade 12th Grade
# o
f Resp
ondents
What grade were you in when you left the Circle Program?
Survey Analysis- Alumni
50% Yes
4
3 3
2
33% without children want them someday
0
2
4
6
8
10
12
14
Do you havechildren?
1 Child 2 Chidren 3 Children 4 Children If none now,in the future?
Children
0
2
4
6
8
10
12
14
Strongly Agree Agree Neutral Disagree StronglyDisagree
# o
f Resp
ondents
Generally, I would say I am happy and healthy.
Survey Analysis- Alumni
0
1
2
3
4
5
6
7
More than oncea week
Once a week Two more timesa month
Once a month Less than once amonth
# o
f Resp
ondents
Fequency of Contact
Connection - 45% of Aumni surveyed are still in contact with a mentor
0
1
2
3
4
5
6
7
8
9
More than oncea week
Once a week Two more timesa month
Once a month Less than once amonth
# o
f Resp
ondents
Fequency of Contact
Connection - 83% of Aumni surveyed are still in contact with friends made at camp
Survey Analysis- Alumni
75%
42%
0
2
4
6
8
10
12
14
16
18
20
Do you see yourself as a mentor to someoneelse?
Do you currently have a mentor (in or out ofthe Circle Program)?
# o
f YES R
esp
onse
s
Contribution and Connection
0 2 4 6 8 10 12 14 16 18 20
Doing what I believe is right, even if myfriends make fun of me.
Standing up for what I believe, even when it isunpopular for me to do so.
Telling the truth, even when it's not easy.
Accepting responsibility for my actions when Imake a mistake or get into trouble.
Doing my best, even when I have a job I do notlike.
Lerner's Character EvaluationHow important are the following to you?
Extremely Important Very Important Somewhat Important Not Important
Survey Analysis- Alumni
0
1
2
3
4
5
6
7
8
9
10
Not at all Slightly Moderately Quite a bit Completely
# o
f R
esp
on
den
ts
Competence and Connection - Do you feel that you developed skills and relationships of value from the
Circle Program?
92%
62%67%
87%
62%
0
5
10
15
20
25
Working with otherpeople
Expressing myfeelings
Understandingother people's
feelings
Being confident in agroup of people
Being able tomanage everyday
life stressors better
Competence, Confidence, Caring - Did Circle Participation help you feel more capable of the
following?
Survey Analysis- Alumni
0
2
4
6
8
10
12
14
16
Anxiety Stress Anger Uncertainty Sadness Frustration
Currently do you feel you cope well with the following emotions?
Yes Somewhat No
0
2
4
6
8
10
12
14
16
Just me One other adult Two other adults Three other adults Four or more otheradults
How many adults live in your home?
Survey Analysis- Alumni
0
1
2
3
4
5
6
7
8
9
10
Some highschool
Highschool
graduateor GED
SomeCollege
Currentlyenrolled
in college
Collegedegree
Tradeschool or
vocationaltraining
AttendingGraduate
School
# o
f R
esp
on
den
ts
Highest Level of Education
0
2
4
6
8
10
12
14
16
Full-time Part-time Seasonal Work fromhome
Unemployed Other
Number of responses more than number of individuals surveyed. Question worded as "Check all that apply".
Employment Status of Alumni Surveyed
Survey Analysis- Alumni
25%
50%
12.5% 12.5%
00
2
4
6
8
10
12
14
Completely Quite a bit Moderately Slightly Not at all
# o
f R
esp
on
den
ts
Do you feel participation in Circle Program aided in your personal and/or social achievements?
50%
25%
4%
21%
0
2
4
6
8
10
12
14
Never 1-2 times a year 3-6 times a year Almost every month
# o
f R
esp
on
den
ts
Frequency
How often do you worry about paying your electric or gas bill?
Survey Analysis- Alumni
Positive Results, 6 C’s appear to have positively influenced Alumni
Recall Bias
Self Reporting
Response Bias
SWOT Analysis:
Strength:
Combination of year long mentoring and residential summer camp
Survey analysis suggests program effectiveness
Weaknesses:
Lack of resource diversification
Anecdotal evidence for program effectiveness
Opportunity:
Entrance/Exit Survey Circle Program’s impact
Program expansion opportunities
Threats:
Downturn in economy
Recommendations:
Administer Entrance and Exit Survey Electronically
Measure improvements in school and/or reductions in risky behavior
Assess survey results and methodology annually
Alumni are interested in participating
Consider developing a mentor survey
“Being a middle child of five children, the Circle Program helped me to establish my own personal identity through the many experiences that the program offered to me.”
-Circle Program Alumna
Sources
“Mission and Philosophy Quote.” The Circle Program’s Mission and Philosophy, 01 Jan. 2016. Web. 09 Sept. 2016. http://www.circleprogram.org/mission1
“Circle History.” The Circle Program’s History, 01 Jan. 2016. Web. 09 Sept. 2016. http://www.circleprogram.org/about1-c1xmq
“Annual Newsletter.” The Circle Program’s Newsletter, 01 Jan. 2016. Web. 09 Sept. 2016. http://www.circleprogram.org/newsletters
Ramer, H. 28 April 2014.”New Hampshire ranks 8th in high school graduation rates.” http://www.washingtontimes.com/news/2014/apr/28/new-hampshire-ranks-8th-in-high-school-graduation/ Last accessed 28 Nov 2016