1

Resiliency in Communities After Stress and Trauma (ReCAST) Evaluation Report Reporting Period Ending September 30th 2020

Saint Louis University Evaluation Team

Elizabeth Baker, PhD, Department of Behavioral Science and Health Education

Rhonda BeLue, PhD, Department of Health Management and Policy, Co-PI

Michaila Dix, MHA Student, Department of Health Management and Policy

Kimberly Enard, PhD, Department of Health Management and Policy

Keon L. Gilbert, DrPH, Department of Behavioral Science and Health Education, Co-PI

Michael Mancini, PhD, School of Social Work

Katie Stamatakis, PhD, Department of Epidemiology and Biostatistics

Clare Vogt, MPH Student, Department of Health Management and Policy

2

Table of Contents

Executive Summary...............................................................................................................................................................3

Background............................................................................................................................................................................4

Intended Use..........................................................................................................................................................................5

Scope of Work........................................................................................................................................................................6

Purpose

Evaluation Questions

Covid-19 Considerations

Methods..................................................................................................................................................................................6

Data Sources..........................................................................................................................................................................7

Primary Data

Secondary Data

Data Analysis..........................................................................................................................................................................7

Results....................................................................................................................................................................................7

Phase II Conclusions............................................................................................................................................................13

Phase III Considerations.......................................................................................................................................................13

References............................................................................................................................................................................1

6

Appendices...........................................................................................................................................................................16

Attached

3

Executive Summary The Saint Louis County Department of Public Health (DPH) received $4.7 million from SAMHSA to fund the St. Louis Resiliency in Communities After Stress and Trauma (ReCAST) project for five years, focused on promoting community well-being in the Promise Zone (Bamberger, 2016). In St. Louis, the primary goals of the ReCAST project are to: 1) build a foundation to promote well-being, resiliency, and community healing through service integration advised by community-based participatory approaches and the Trauma-Informed Community Building Model; 2) Improve access to trauma-informed community behavioral health resources and youth peer support; and 3) Create community change through community-based participatory approaches that promote community and youth engagement, leadership development, improved governance, and capacity building (Schmidt, 2017).

The ReCAST evaluation team employed a mixed qualitative-quantitative methodological approach to examine processes and outcomes related to: contractor program experience, delegate experiences, Core Advisory Board experience and to gain an understanding of how ReCAST programming imparts resilience into the Promise Zone community.

Key findings:Despite challenges created by COVID, ReCAST stakeholders were able to implement a modified version of their programming. Participatory budgeting continues to be a positive and helpful game-changer for ReCAST stakeholders who suggest implementing the process to additional community organizations and processes. Additional youth involvement and a focus on economic mobility and feedback is necessary to build sustainable resilience strategies.

Pandemic Notes. Planned evaluation activities were updated and rescheduled. Planned activities including photovoice with ReCAST youth stakeholders was rescheduled for 2021 and will focus on web-based images that represent youth experience with factors related to resilience as opposed to youth taking photos in their communities. Evaluation and contractor interviews were also delayed as we transitioned to online interactions and contractors retooled their services to accommodate COVID-19 safety precautions.

Action steps. Future evaluative services will include, contractor interviews, delegate surveys and various forms of youth engagement.

4

Background In 2013, the Obama Administration launched the Promise Zone program, identifying communities experiencing high poverty and poor opportunity in order to create federal partnerships with local government and community organizations to improve economic investment, access to affordable housing, and improved education (Obama, 2016). In 2015, St. Louis and St. Louis County neighborhoods were identified as the St. Louis Promise Zone Twenty-five zip codes were designated, including 11 wards in St. Louis City and the neighborhoods of Bellerive Acres, Bel-Nor, Bel-Ridge, Berkeley, Beverly Hills, Cool Valley, Country Club Hills, Dellwood, Ferguson, Flordell Hills, Glen Echo Park, Greendale, Hazelwood, Hillsdale, Jennings, Kinloch, Moline Acres, Normandy, Northwoods, Pagedale, Pine Lawn, Riverview, University City, Uplands Park, Velda City, Velda Village Hills, and Wellston (“Promise Zones—HUD Exchange,” n.d.). (see Figure 1).

Figure 1. HUD Map of St. Louis Promise Zone

At the time of the designation, the following applied Population - 199,792 Poverty Rate - 35.48% Median Income - $26,546 Unemployment - 20.91% Twenty-eight municipalities in the Promise Zone are served by seven school districts. Two of the seven school districts are currently not accredited by the state, with one being provisionally accredited.

5

This designation became a catalyst for funding change in the area, and in 2016, the Substance Abuse and Mental Health Services Agency (SAMHSA) began awarding funds to areas where youth and adults were impacted by trauma, violence, and conflict between communities and law enforcement (“Grants & Awards, 2016,” 2016).

The Saint Louis County Department of Public Health (DPH) received $4.7 million from SAMHSA to fund the St. Louis Resiliency in Communities After Stress and Trauma (ReCAST) project for five years, focused on promoting community well-being in the Promise Zone (Bamberger, 2016). Formally titled Project CATALYST (Community Action for Trauma-informed Access, Leadership, Youth Support and Training), the award was one of eight projects funded by SAMHSA to address resilience and equity, and the only one funded in 2016 to specifically include a community-based participatory approach. ReCAST awards were designed “to assist high-risk youth and families and promote resilience and equity in communities that have recently faced civil unrest through implementation of evidence-based, violence prevention, and community youth engagement programs, as well as linkages to trauma-informed behavioral health services (Carr, 2017).”

In St. Louis, the primary goals of the ReCAST project are to: 1) build a foundation to promote well-being, resiliency, and community healing through service integration advised by community-based participatory approaches and the Trauma-Informed Community Building Model; 2) Improve access to trauma-informed community behavioral health resources and youth peer support; and 3) Create community change through community-based participatory approaches that promote community and youth engagement, leadership development, improved governance, and capacity building (Schmidt, 2017).

After completing a formal needs assessment and strategic plan, a community engagement plan was designed, and community outreach was used to recruit delegates to the participatory budgeting process. In this process, Promise Zone residents volunteered their time as Community Delegates to define community priorities, determine the scope of work to be included in requests for proposals (RFPs), receive training on reviewing and evaluating proposals, and vote on projects to receive grant awards.:

Intended Use In 2020, DPH contracted with Saint Louis University’s College for Public Health and Social Justice to conduct a process and outcome evaluation for each grant recipient in two phases. In the first phase, service providers will be evaluated for their program fidelity to contracted services, efficacy of resource use, and outcomes of program participants. Future evaluative services will involve the Core Advisory Board, Coalition of Stakeholders, Community Delegates and youth served by the program providers, and the community at large.

6

This report is phase II, which includes an assessment of how ReCAST contractor services build resilience in their communities, measuring resilience in contractor clients, and stakeholder interviews with 1) delegates, 2) 2020 contractors and 3) The Core Advisory Board. It is intended for access by SAMHSA, DPH, and all stakeholders.

Scope of Work Purpose The evaluation team contracted to provide analysis and evaluative services to the Department of Public Health during the Year 4 Grant Budget Period, which concluded September 30, 2020. The program evaluation report is intended to fulfill the requirements of the SAMHSA grant reporting, the contract enacted between DPH and SLU, and the Promise Zone community at large. The ReCAST DPH team engaged SLU to address additional evaluation questions:

Evaluation Questions 1. How do ReCAST contractors promote resilience in the community? 2. How well did community delegates and the Core Advisory Board feel they were able to represent their own interests, needs, and desires in what they wanted ReCAST to fund? 3. How to measure resilience in ReCAST participants.

COVID-19 Considerations Given the COVID-19 pandemic, contractors, the evaluation team and other stakeholders had to make adaptations to a primarily online format to adhere to COVID-19 restrictions. Contractors had to make the following changes 1) suspend or delay programming due to COVID shutdowns 2) reducing intended number of program participants 3) Changing programming to accommodate smaller socially distanced groups, online only interactions or outside activities. Several evaluation activities planned for 2020 were also delayed and/altered given COVID restrictions. All evaluation activities were done via ZOOM. Other activities were delayed until 2021 pending IRB approval to accommodate updates to protocols to include moving activities to an online format.

Methods We employed a mixed methods approach that includes 1) Semi-structured interviews 2) Use of valid and reliable survey scales and other survey items. Note that site visits to contractor organizations were suspended due to COVID.

7

Data Sources Primary Data The evaluation team contacted each service provider to set up either individual interviews or small group interviews with executive leadership, ReCAST program managers and other ReCAST program staff. Interviews were held via Zoom, in small-groups with members of management and service providers. Interviews were conducted with delegates (N=4) the Core Advisory Board members (N= 10), lead contractors (n=4) and PRAB (N=1). Interviewers used open-ended questions to elicit perspectives based-on the evaluation and ReCAST team questions. Each interview was recorded and transcribed verbatim by an independent transcription service. Secondary Data We reviewed all ReCAST grants to identify programming components related to community resilience.

Data Analysis The evaluation team identified preliminary codes to establish broader themes across the four organizational interviews. We used an a priori coding strategy to provide a preliminary analysis of these data for reporting purposes. These a priori codes and themes emerged from the evaluation questions, the initial ReCAST Evaluation RFP which guided our focus on services provided, resources and budget allocation, operational impact, capacity building, process fidelity, and client outcomes.

Results Results Part 1: Building a Resilience in Collaboration with the Promise Zone The Community Resilience Framework (CFR) (Cafer et al 2019), draws on the Sustainable Livelihoods Approach and Community Capitals Framework (Gutierrez-Monte et al 2009) to address the drivers of resilience within communities in consideration of their everyday challenges and needs. The CFR facilitates assessment of the individual needs and community assets. The elements of the framework reflect factors that relate to a community’s resilience. The effectiveness of community-based interventions meant to improve community resilience can be assessed based on these factors hazards, assets, and vulnerabilities including: human (community member skills and abilities), social (connections among community members), cultural (racial-ethnic and diversity and inclusion), political (community influence over laws and enforcement), financial (financial resources to facilitate community development), natural (resources and natural beauty) and built (e.g. parks and roads).

8

The following diagram displays how ReCAST’s funded projects address different aspects of community resilience. ReCAST has funded a total of 17 community projects between 2017-2020. Funded projects focused on mental health, violence prevention, and youth engagement. Over 8000 Promise Zone community members have been reached by funded projects over the past four years.

Organizations funded include community health centers, youth/family mental health service organizations, social service organizations, youth arts/performance organizations, and youth and family services organizations.

CRF Factor How ReCAST Initiatives Address Resilience in their Programming

Mental Health Violence Prevention Youth Engagement Human Factors -Coping and Self-

care education -Counseling

services

-Financial Literacy -Trade/educational support

-Employment training -Gun shot wound care

-Employment Training -Empowerment education

-Counseling Academic Tutoring

-Peer education training Social Factors -Community

Advisory Boards -Community

Mental Health Awareness

Training

-Group Community-Safety Activities

-Group Community-Safety Activities

Cultural factors -Arts and theater training

-Cultural Diversity education -Arts and theater training - Cultural Diversity education

Financial Factors

Employment support

Natural resources

Park and Community Beatification

Park and Community Beatification

-Park and Community Beatification-youth engaged

Built Environment

Community Gardening-youth engaged

-Community Gardening-youth engaged

9

Results Part 2: Stakeholder Interviews

Community Participation

Stakeholders had overwhelmingly positive things to say about the effect of the ReCAST program. Interviews were conducted with community delegates who were engaged in the participatory budgeting and community voting on programs to be funded; as well as contractors, whose programs had received grants from ReCAST. Their specific subject areas of involvement also varied, as one delegate was very involved in a youth engagement and violence prevention program, while another contractor ran a program to teach community members about mental health and the issues of self-care and resilience.

Building Resilience: Definitions and Examples

We asked stakeholders to reflect on the role of resilience a major topic of discussion during the interviews, as a fundamental goal of the ReCAST program is building resilience in the communities of the Promise Zone. One of the delegates was unaware of the concept of resilience as a fundamental principle of the program, but stated that it was probably in the back of her mind as she was voting and thinking about the program and its intended outcomes. She expanded on this and stated that she believed that the youth engagement programs were likely intended to do that, and that the capacity building workshops were very good for building confidence and skills in participants, which she considered an aspect of resilience. Another delegate defined resilience as “understanding what we need to do to move forward. Staying focused and strong and work at what we need to do for progress.” He stated that the youth programs were well-received and “forced a conversation with residents and the city…different parts of the city.” Resilience was indirectly discussed with one of the contractors, who led a program focused on self-care and mental health education. She spoke at length about self-care and mental health, and the importance of these concepts for the community, implying that they foster resilience. She mentioned prevention of mental health issues as a specific goal of the program, something she referred to as “getting upstream” which also seemed to imply a goal of resilience as a preventative mechanism for mental health issues and their attendant consequences.

10

As far as room for improvement, one delegate stressed the need for more education about the participatory budgeting process, and more information about the contractors before community voting. She expressed that she, in particular, had not received information about the programs to be voted on until immediately before the vote, and that some information to review beforehand would be very helpful in making an informed decision based on her priorities for the community. The need for more outreach and education was echoed by the other delegate as well. This was discussed in terms of educating people about the concept of budgeting and voting, but also about the ReCAST program in general and what it was intended to for the community. One delegate in particular spoke of his desire to see the program expanded beyond the area of the Promise Zone, because he felt very strongly that the principles were so important to bring to other communities as well.

Overall, there was strong approval of the ReCAST program and its contractors by the delegates,emphasized a need for increased outreach and education for the community, and in some cases more information for the delegates as well. Contractors also expressed very positive views. The participatory budgeting process and community voting received the most positive input from delegates, and support for contractors and availability of the ReCAST staff was emphasized by the contractors. The ReCAST program certainly appears to have made a positive impact towards resilience in the Promise Zone, with all interviewees wishing to see the programs continue and expand.

Results Part 3: Measuring Resilience Given the ReCAST program’s goal is to build resilience in Promise Zone communities, in 2020 we launched a pilot study to measure resilience in ReCAST program participants by partnering with ReCAST contractors to administer both adult and child resilience scales to their clients.

Participants included 2019 contractor -CHiPS Health and Wellness Center; 2020 contractor Gentlemen of Vision (GoV).

The evaluation team administered both the Adult Resilience Measure-Revised (ARM-R), as well as the Child and Youth Resilience Measure (CYRM). The ARM-R scale is intended for adults 18 years or older, and the CYRM-R is intended for children ages 5-9 and youth ages 10-23. Both questionnaires were provided by the Resilience Research Centre of Dalhousie University (Halifax, NS), and administered via Qualtrics Survey. ReCAST grantees were provided anonymous Qualtrics links, which were given to their adult and child populations to access and complete the survey.

11

Students and parents were instructed to assess how each question applies to their own levels of awareness and personal situations. These questions measured sociability, value of education, ability to behave in particular situations, connection towards family/friends, availability of food, etc. The ARM-R and CYRM-R differed slightly in the way in which questions were presented to participants.

Below is the numbered list of questions in the CYRM-R (Caregiver Resilience): 1. I cooperate with people around me. 2. Getting an education is important to me. 3. I know how to behave in different social situations. 4. My parent(s)/caregiver(s) really look out for me. 5. My parent(s)/caregiver(s) know a lot about me. 6. If I am hungry, there is enough to eat. 7. People like to spend time with me. 8. I talk to my family/caregiver(s) about how I feel. 9. I feel supported by my friends. 10. I feel that I belong/belonged at my school 11. My family/caregiver(s) stand by me during difficult times. 12. My friends stand by me during difficult times. 13. I am treated fairly in my community. 14. I have opportunities to show others that I am becoming an adult and can act responsibly. 15. I feel safe when I am with my family/caregivers. 16. I have opportunities to develop skills that will be useful later in life. 17. I enjoy my family’s/caregivers’ cultural and family traditions.

Below is the numbered list of questions in the ARM-R (Relational Resilience):

1. I get along with people around me 2. Getting and improving qualifications or skills is important to me 3. I know how to behave in different social situations (such as at work, home, or other public places) 4. My family is supportive towards me 5. My family knows a lot about me (for example, who my friends are, what I like to do) 6. If I am hungry, I can usually get enough food to eat

12

7. People like to spend time with me 8. I talk to my family/partner about how I feel (for example, when I am sad or concerned) 9. I feel supported by my friends 10. I feel that I belong in my community 11. My family/partner stands by me when times are hard (for example, when I am ill or in trouble) 12. My friends care about me when times are hard (for example, when I am ill or in trouble) 13. I am treated fairly in my community 14. I have opportunities to show others that I can act responsibly 15. I feel secure when I am with my family/partner 16. I have opportunities to apply my abilities in life (like using skills, working at a job, or caring for others) 17. I like my family’s/partner's culture and the way my family celebrates things (like holidays or learning about my

culture) Scoring Provided by the Resilience Research Center, to derive caregiver and relational scores, sum the following 7 questions: 4, 5, 6, 8, 11, 15, 17. Given that a 5-point measure was used (Not at all, A little, Somewhat, Quite a bit, A lot), the minimum subscale score is 7, and the maximum score is 35. Please refer to appendix A for the CHiPS and GoV detailed analysis.

Validation and Reliability Internal Reliability/Consistency (Derived from Jefferies et al. 2018) Cronbach’s alpha = .82 (Personal resilience), .82 (Caregiver – CYRM, and Relational – ARM-R), .87 (overall resilience). These alphas were determined by the above mentioned authors from multiple study analyses, but the results were never published.

Rasch Validation The subscale scores of these measures were validated using Rasch analyses, which concluded the following: unidimensionality requirements were satisfied, measures had good fit statistics and targeting properties, lacked item bias and problematic local dependency (Jefferies et al., 2018).

GoV results (Calculated from mean scores) Caregiver (Child/Youth) (CYRM-R) Resilience Subscale (n= 17 youth participants)

● Sum 7 items: 4, 5, 6, 8, 11, 15, 17.

13

● Maximum Score is 35. ● Actual Score: 28.24

Relational (Adult) (ARM-R) Resilience Subscale (n= 16 adult participants)

● Sum 7 items: 4, 5, 6, 8, 11, 15, 17. ● Maximum Score is 35 ● Actual Score: 31.31

CHiPS results (Calculated from mean scores)

● Caregiver (Child/Youth) (CYRM-R) Resilience Subscale (n= 10 youth participants) ● Sum 7 items: 4, 5, 6, 8, 11, 15, 17. ● Maximum Score is 35. ● Actual Score: 27.50

Phase II Conclusions Despite challenges created by COVID, ReCAST stakeholders were able to implement a modified version of their programming. Participatory budgeting continues to be a positive and helpful game-changer for ReCAST stakeholders who suggest implementing the process to additional community organizations and processes. Additional youth involvement and a focus on economic mobility and feedback is necessary to build sustainable resilience strategies.

Phase III Considerations The evaluation team has discussed next steps for evaluation activities with the ReCAST management team at the St Louis County Health Department and the ReCAST Core Advisory Board.

Figure 2 provides a Gant chart of all 2021 evaluation activities

Concept Mapping (CM): The concept mapping procedure will begin in January 2021. Recruitment of Core Advisory Board, Coalition of Stakeholders, and Community Delegates for CM participation has already begun. ReCAST participants age 18 and over Concept mapping will be used to identify 1) Stakeholder perspectives on how communities can continue to build resilience 2) Sustainability of ReCAST initiatives 3) future directions

14

Resilience Questionnaires: Resilience questionnaires will be administered in collaboration with all new contractors at baseline (within one month of starting programming) and before the contract ends. Contractors will work with the evaluation team to identify clients to complete the resilience questionnaires.

Youth Participants and Photovoice: Will be conducted with ReCAST stakeholders age 12-18. Due to COVID, we will take an online approach to photovoice. Participants will identify images online that represent resilience and challenges in their communities. Youth participants will process the images they chose with the evaluation team and community moderators at the 1st annual ReCAST youth conference.

Youth Conference: The St Louis County Health Department will host a virtual youth conference in approximately April of 2021 to provide youth an opportunity to provide feedback to ReCAST stakeholders on youth needs.

Delegate Surveys: Delegates will be asked about their experience with the ReCAST project. See appendix for questionnaire.

Contractor Interviews: Similar to 2019 and 2020, we will repeat contractor interview questions with the new contractors.

Participatory Budgeting Process: We will conduct interviews with ReCAST staff, Coalition of Stakeholders and proposal review participants.

15

References

Bamberger, A. (2016, September 13). St. Louis County Secures $4.7 Million Federal Grant to Fund Youth Violence Prevention and Mental Health Programs. Retrieved September 18, 2019, from St. Louis County Government website: https://www.stlouisco.com/Your-Government/Revenue/News/Article/1540/St-Louis-County-Secures-47-Million-Federal-Grant-to-Fund-Youth-Violence-Prevent

Carr, C. (2017, March 16). Resiliency in Communities After Stress and Trauma [Grant Announcements]. Retrieved from https://www.samhsa.gov/grants/grant-announcements/sm-17-009

Cafer, A., Green, J., & Goreham, G. (2019) “A Community Resilience Framework for Community Development Practitioners Building Equity and Adaptive Capacity.” Community Development, 50(2): 201-216. https://www.tandfonline.com/doi/abs/10.1080/15575330.2019.1575442

Grants & Awards (2016). Retrieved September 18, 2019, from SAMHSA - Substance Abuse and Mental Health Services Administration website: https://www.samhsa.gov/grants-awards-by-state/MO/discretionary/2017/details

Gutierrez-Montes, I., Fernandez-Baca, M.E., & Fernandez-Baca, E. (2009). The Sustainable Livelihoods Approach and the Community Capitals Framework: The Importance of System-Level Approaches to Community Change Efforts, Community Development, 40:2, 106-113. https://doi.org/10.1080/15575330903011785

Obama, B. (2016). Office of the Press Secretary. Presidential Studies Quarterly, 39(3), 429–430. https://doi.org/10.1111/j.1741-5705.2009.03698.x

Promise Zones—HUD Exchange. (n.d.). Retrieved September 18, 2019, from https://www.hudexchange.info/programs/promise-zones/

Schmidt, S. (2017). St. Louis ReCAST Community Strategic Plan (p. 22). Saint Louis County Department of Public Health.

16

Appendix A

Figure 1. ReCAST Logic Model

17

Appendix B

Figure 2. GoV ARM-R Resilience Survey Frequencies

18

Appendix C

Figure 3. GoV ARM-R Resilience Survey Frequencies

19

Appendix D

Figure 4. GoV CYRM-R Resilience Survey Frequencies

20

Appendix E

Figure 5. GoV CYRM-R Resilience Survey Frequencies