comprehensive prevention program...

Comprehensive Prevention

Program Evaluation

Annual Report Prepared By: Iowa Consortium for Substance Abuse Research and Evaluation University of Iowa, Iowa City, Iowa 52242-5000 With Funds Provided By: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Prevention, Iowa Department of Public Health, Bureau of Substance Abuse Prevention and Treatment

Citation of references related to this report is appreciated. Suggested citation: White, K., Clayton, R., & Arndt, S. (2009). Comprehensive prevention program evaluation: Annual report. (Iowa Department of Public Health contract #5889UI01). Iowa City, IA: Iowa Consortium for Substance Abuse Research and Evaluation. http://iconsortium.subst-abuse.uiowa.edu/

Comprehensive Prevention Program Evaluation

Annual Report

July 2008 – June 2009

Prepared By:

Kristin L. White, MA Evaluation Coordinator

Rebecca Clayton, BS Associate Director

Stephan Arndt, PhD

Director

© 2009 University of Iowa

Table of Contents Introduction 1 Evaluation Design 1 Evaluation Results 1 Demographics 1 Outcome Data 2

Attrition Analysis 2 Past 30-Day Use 3 Table 1: Elementary and Middle School Age Youth in Single-Year Programs 5 Graph 1: Elementary and Middle School Age Youth in Single-Year Programs 6 Table 2: Elementary and Middle School Age Youth in Multi-Year Programs 8 Graph 2: Elementary and Middle School Age Youth in Multi-Year Programs 9

Table 3: High School Age Youth 11 Graph 3: High School Age Youth 12

Attitudes Toward Alcohol Use 13

Graph 4: Elementary and Middle School Age Youth in Single-Year Programs 13 Graph 5: Elementary and Middle School Age Youth in Multi-Year Programs 14 Graph 6: High School Age Youth 15

Attitudes Toward Cigarette Use 16


Attitudes Toward Marijuana Use 19

Graph 10: Elementary and Middle School Age Youth in Single-Year Programs 19 Graph 11: Elementary and Middle School Age Youth in Multi-Year Programs 20 Graph 12: High School Age Youth 21 Table 4: Positive Outcome Percentage Summary for Attitudes 22

Perceived Risk of Harm from Alcohol Use 23


Perceived Risk of Harm from Cigarette Use 26


Perceived Risk of Harm from Marijuana Use 29


Table 5: Positive Outcome Percentage Summary for Perceived Risk 32

Conclusion 33

Appendix A: Institute of Medicine Categories of Populations Served by Agency and Program

1

Introduction The purpose of the Comprehensive Substance Abuse Prevention (Comprehensive) project is to provide primary alcohol, tobacco, and other drug (ATOD) prevention services in Iowa. Twenty-two providers covering twenty three service areas implemented a variety of evidence-based prevention programming, all of which met State guidelines for inclusion in the project. A list of programs implemented by each agency and the Institute of Medicine (IOM) category of populations served appears in Appendix A. Evaluation Design This evaluation of the Comprehensive project includes only recurring educational services for youth; other services are monitored outside of this evaluation. All services have the performance measure target of increasing participants’ perception of risk/harm from alcohol, tobacco, or other drugs. The evaluation involves a matched pre-post design whereby a survey is administered to the target population at the beginning and at the conclusion of the prevention program. The post-test survey is also administered after each year of programming for programs spanning multiple years. Post-test survey data from the previous program year is used as a baseline for reporting current-year outcomes for multi-year programs. Agencies submitted 5666 pre-tests, 5151 first-year post-tests, 956 second-year post-tests, and 66 third-year post-tests during State Fiscal Year 2009 (July 1, 2008 – June 30, 2009). This yielded 5610 total matched pre- and post-test surveys; 4829 of these were pre- to first-year post matches. Second-year post-tests were matched with those participants’ first-year post-tests, yielding 741 matches. The number of third-year post-test to second-year post-test matches totaled fewer than 50, and those surveys are omitted from this report to protect respondents’ confidentiality. The matched data sets are used to answer the following evaluation questions:

Has alcohol/tobacco/marijuana usage changed in the target population? Has the percentage of the target population who indicate positive attitudes at

baseline (pre-test) maintained or increased after the intervention (post-test)? Has perception of risk of harm from alcohol/tobacco/marijuana use maintained a

positive response or increased from pre-test to post-test? Evaluation Results Demographics Demographic data include participants completing the Comprehensive, Younger Youth, and Juvenile Alcohol and Drug Education (JADE) survey instruments. Participants taking the Life Skills survey are not included in these data, as no post-tests were submitted during State Fiscal Year 2009. There are some instances where individual responses on demographic data varied from pre-test to post-test. Some differences are naturally occurring, such as participants’ age or grade increasing by one year. For the Comprehensive and Younger Youth surveys, post-test responses were used in those cases, as those data most closely represent the matched group included in this report. The JADE post-test survey does not include demographic data, so the demographic data were

2

taken from the pre-tests for those participants. In addition, there were numerous mismatches in individual participant responses from pre-test to post-test for gender, race, and ethnicity on the Comprehensive and Younger Youth surveys. Upon investigation, agency and Consortium staff discovered that the majority of these discrepancies were due to participants giving different responses and not due to data entry error. Therefore, evaluation staff created a standard rule for addressing discrepancies in demographic data, which was to use the responses on the post-tests. The median age (at post-test) of participants included in this evaluation is 13. Half (49.95%) of the participants are 7th and 8th grade students. Males comprise 50.6% of respondents, and 7.4% of respondents are Hispanic or Latino. Participant racial groups are delineated below:

81.68% White 9.72% More than one race 4.16% Black/African American 2.12% Asian 1.69% American Indian/Alaska Native 0.33% Native Hawaiian/Other Pacific Islander 0.30% Arab American

Outcome Data The figures included in the outcome data detail past 30-day substance use, attitudes about substance use, and perceived risk of harm from substance use. Attrition Analysis Due to a large number of unmatched pre- and post-tests (possibly caused by survey administration anomalies, data entry error, conflicting timelines between project reporting and program implementation, or ID assignment mistakes) an attrition analysis was performed on these data, which examined demographics, attitudes toward alcohol, tobacco, and marijuana use, and perception of risk of harm from alcohol, tobacco, and marijuana use. This analysis examined individual level attrition; if an agency terminated a program early, those groups were not included in the analysis. Statistically significant differences between participants who completed a post-test and those who did not were found in age, grade, race, ethnicity, attitude toward tobacco use, and perceived risk of harm from tobacco use. Attrition was higher among twelve and thirteen year olds than among participants of other ages; twelve year olds comprised forty-three percent (43.05%) of the drop outs and thirteen year olds comprised twenty-seven percent (27.04%) of the drop outs (p < 0.0001). Correspondingly, attrition among seventh graders was higher than among participants in other grades, with these students comprising sixty-two percent (62.06%) of the drop outs compared to forty-three (43.08%) percent of those who did not drop out (p < 0.0001). Participants of Hispanic or Latino ethnicity tended to drop out at higher rates than non-Hispanic or Latino participants; forty-one percent (40.65%) of Hispanic/Latino participants dropped out compared to twenty-seven percent (26.75%) of participants whose ethnicity was not Hispanic/Latino (p < 0.0001). Participants of racial minority groups tended to drop out more than did whites; non-whites comprised nineteen percent (18.67%) of participants completing the program but comprised twenty-four percent (24.06%) of those not completing the program (p < 0.0001). In addition, the analysis showed small but statistically

3

significant differences between program completers and non-completers on attitudes toward tobacco use (p < 0.04) and perceived risk of harm from tobacco use (p < 0.05). These effects of attrition indicate that participants who did not complete prevention programming differed in some respects from those who completed programming. Thus, the participants represented by these outcome data are in some ways different from those who initiated the program. This should be taken into consideration when interpreting the outcome data. Past 30-Day Use Tables 1, 2, and 3 on pages 5, 8, and 11 present data on change in past 30-day alcohol, tobacco, and marijuana use. Program-specific data are provided for evidence-based prevention programs with at least 50 matched pre- and post-tests. A Comprehensive Prevention project total is also provided so that individual program results can be viewed in relation to the overall totals for the project. The Comprehensive (All) figures include all participants in the Comprehensive Prevention Project with matched pre- and post-tests, including participants in programs for which there were not enough matches to report on separately. The 30-day use data do not include participants taking the Younger Youth Survey because that survey does not contain questions regarding respondent substance use. Graphs 1, 2, and 3 on pages 6, 9, and 12 graphically present the information shown in Tables 1, 2, and 3. The programs are grouped according to the age of participants at post-test (elementary and middle school versus high-school) and according to the duration of the program (less than a year versus multi-year programs). The Iowa Youth Survey (IYS) data are provided as a reference for interpreting the outcome data in this report. The Iowa Youth Survey is a triennial assessment of Iowa’s school-age (grades 6, 8, and 11) students’ attitudes toward substance use and actual use of substances. The IYS data reflect changes due to maturation of the youth through the different grade levels. The 2008 IYS data provided here represent an estimate of the change one might see among youth in the general population over the course of one year. Thus, this shows the estimated annual change one might expect in Iowa’s general youth population versus the outcomes of youth who complete specific prevention programming under the Comprehensive Prevention project (Note: Youth who received Comprehensive programming may also have completed the IYS). The average yearly change was calculated by dividing the difference between grades by the number of years between grades. This was done using 6th and 8th grade IYS data to provide a reference for programs implemented in the elementary and middle schools, and using 8th and 11th grade IYS data for programs in the high schools. A single average yearly change figure is given in the tables below to simplify interpretation. True yearly change rates, however, would increase each successive year (i.e., past 30-day use between 6th and 7th grade may increase less than 3%, but may increase more than 3% between 7th and 8th grade). While the time span between pre-test and post-test for some prevention programs presented here is less than one year, the IYS average yearly change serves as a general point of reference when examining the program outcomes rather than comparing to zero, or no change. Table 1 on page 5 and Graph 1 on page 6 display the change in past 30-day use of alcohol, tobacco, and marijuana reported by elementary and middle school age youth in single-year programs. A positive (+) figure indicates an increase in use, whereas a negative figure (-) indicates a decrease in use. Iowa Youth Survey data show increases in past 30-day use of

4

all three substances. Only one single-year program, Project Towards No Tobacco Use (TNT), had enough pre-post matches to be included this year. Project Towards No Tobacco Use data show a decrease in tobacco use from pre-test to post-test. No participants had reported marijuana use at pre-test, and this remained the case at post-test. Project TNT participants show an increase in past 30-day alcohol use; however, the increase is less than that seen in the IYS group.

5

Table 1. Change in Past 30-Day Use: Elementary and Middle School Age Youth in Single-Year Programs

Percentage of Youth Reporting Past 30-Day Use at the Pre-Test and Change at Post-Test: Elementary and Middle School Age Youth in Single-Year Programs

Group N Median

Age

Alcohol Tobacco Marijuana

Pre-Test % Change Pre-Test % Change Pre-Test % Change

IYS1 64,9372 13 – +5.00 – +2.50 – +1.00

Comprehensive (All)3 4406 13 12.38 +0.32 7.04 - 0.30 3.36 +0.35

TNT 156 11 4.49 +3.84 1.28 - 0.64 0 0

1 IYS entries indicate the yearly average change in 30-day use between all Iowa students in grades 6 and 8. The majority of 6th graders completing the IYS were 11 years old; the majority of 8th graders were 13. Data were from the 2008 Iowa Youth Survey, State of Iowa report (pgs.12, 29-30, 90-91). 2The total number of 6th graders completing the Iowa Youth Survey was 32,264; the total number of 8th graders was 32,673. 3The Comprehensive row includes all current year Comprehensive Prevention project participants with matching pre- and post-tests, regardless of their age or the program in which they participated.

Substance Abuse Prevention Program Key

TNT Project Towards No Tobacco Use IYS Iowa Youth Survey

6

Graph 1. Change in Past 30-Day Use: Elementary and Middle School Age Youth in Single-Year Programs


TNT Project Towards No Tobacco Use IYS Iowa Youth Survey

7

Table 2 on page 8 and Graph 2 on page 9 display the change in past 30-day use of alcohol, tobacco, and marijuana reported by elementary and middle school age youth who participated in multi-year programs. Data for each year of these programs are reported separately. For example, Project ALERT is implemented with seventh graders and has booster lessons that are implemented with those students the following year (in eighth grade). For this report, “First year of Project ALERT” refers to the programming implemented with seventh graders, and “Second year of Project ALERT” refers to the booster sessions implemented with eighth graders. Only data from the first program year of All Stars, Peer Helping1, and Too Good for Drugs are presented here, due to the low number of subsequent year pre- to post-test matches. Students in the first year of LifeSkills Training (LST) showed a decrease in alcohol and tobacco use from pre-test to post-test. Students in the second year of LST also showed a decrease in tobacco use. Students in the first year of LST showed a slight increase in marijuana use, though less than in the IYS group, but those in the second year of LST showed no increase. Project ALERT (PA) data for both program years show increases in past 30-day use of all substances. However, only past 30-day use of marijuana in the second year group increased more than in the IYS group. All Stars students showed no change (thus no increase) in alcohol use from pre-test to post-test. However, All Stars students showed a greater increase in past 30 day marijuana use than the IYS group. Peer Helping program results are notable, showing either no change or a decrease in use of alcohol, tobacco, and marijuana from pre-test to post-test. It should be noted that data for the Peer Helping program differ from data for the other Comprehensive Prevention funded programs. The contractor conducts the training but most of the peer mentors are supervised by the school in which the program is implemented. Results for Too Good for Drugs showed increases in use of all three substances, although the increases for alcohol and tobacco were less than in the IYS group. 1 Peer Helping is run as a single-year program in some locations, however the majority of participants included in the data reported here will continue with programming next year.

8

Table 2. Change in Past 30-Day Use: Elementary and Middle School Age Youth in Multi-Year Programs

Percentage of Youth Reporting Past 30-Day Use at the Pre-Test and Change at Post-Test:

Elementary and Middle School Age Youth in Multi-Year Programs

Group N Median

Age



IYS1 64,9372 13 – +5.00 – +2.50 – +1.00


First Year of LST 1179 12 6.45 - 0.77 2.63 - 1.27 1.02 +0.08

Second Year of LST 69 14 10.14 +4.35 2.90 - 1.45 1.45 0

First Year of PA 937 13 5.78 +0.43 2.24 +0.32 0.96 +0.65

Second Year of PA 594 13 4.21 +4.21 2.02 +1.01 1.35 +1.18

First Year of AS4 111 12 4.50 0 0.90 +1.80 0 +2.70

First Year of PH4 60 14 21.67 0 6.67 - 1.67 3.33 - 3.33

First Year of TGFD4 193 10 7.37 +3.68 4.66 +1.04 3.65 +1.04

1IYS entries indicate the yearly average change in 30-day use between all Iowa students in grades 6 and 8. The majority of 6th graders completing the IYS were 11 years old; the majority of 8th graders were 13. Data were from the 2008 Iowa Youth Survey, State of Iowa report (pgs.12, 29-30, 90-91). 2The total number of 6th graders completing the Iowa Youth Survey was 32,264; the total number of 8th graders was 32,673. 3The Comprehensive row includes all current year Comprehensive Prevention project participants with matching pre- and post-tests, regardless of their age or the program in which they participated. 4 The numbers of matched surveys from the 2nd year of AS, PH, and TGFG were less than 50, so those data are not reported.


LST LifeSkills Training AS All Stars TGFD Too Good for Drugs PA Project ALERT PH Peer Helping IYS Iowa Youth Survey

9

Graph 2. Change in Past 30-Day Use: Elementary and Middle School Age Youth in Multi-Year Programs


LST LifeSkills Training AS All Stars TGFD Too Good for Drugs PA Project ALERT PH Peer Helping IYS Iowa Youth Survey

10

Table 3 on page 11 and Graph 3 on page 12 display the change in past 30-day use of alcohol, tobacco, and marijuana reported by high school age youth (all high school programs represented are single-year programs). Several programs show a decrease in use of one or more substances between the pre- and post-test. Juvenile Education Groups’ (JEG) data show decreases in alcohol and marijuana use, and no change (thus, no increase) in tobacco use. Juvenile Alcohol and Drug Education (JADE) data show no increase in alcohol use and a decrease in tobacco use. Prime for Life (PFL) data show decreases in alcohol and tobacco use. While JADE and PFL show increases in marijuana use, the increase for PFL is less than in the Iowa Youth Survey (IYS) group and the increase for JADE is only slightly greater than in the IYS group. These outcomes for JEG, JADE, and PFL are particularly notable because those programs serve indicated populations – youth who have already experienced consequences from their substance use. Project Northland Class Action (PNCA) data also show a decrease in alcohol use, and the increases in tobacco and marijuana use are less than in the IYS group.

11

Table 3. Change in Past 30-Day Use: High School Age Youth

Percentage of Youth Reporting Past 30-Day Use at the Pre-Test and Change at Post-Test:

High School Age Youth

Group N Median

Age



IYS1 63,88032 14 – +7.33 – +5.66 – +3.33


JEG 55 16 36.36 - 3.63 27.27 0 10.91 - 3.64

JADE 101 17 52.00 0 45.54 - 0.99 9.90 +3.96

PFL 615 15 30.57 - 3.09 19.02 - 1.30 11.54 +1.46

PNCA 132 16 33.33 - 2.27 16.67 +0.75 12.21 +3.06

1IYS entries indicate the yearly average change in 30-day use between all Iowa students in grades 8 and 11. The majority of 8th graders completing the IYS were 13 years old; the majority of 11th graders were 16. Data were from the 2008 Iowa Youth Survey, State of Iowa report (pgs.12, 29-30, 90-91). 2The total number of 8th graders completing the Iowa Youth Survey was 32,673; the total number of 11th graders was 31,130. 3The Comprehensive row includes all current year Comprehensive Prevention project participants with matching pre- and post-tests, regardless of their age or the program in which they participated.


JEG Juvenile Education Groups PFL Prime For Life Under 21 IYS Iowa Youth Survey JADE Juvenile Alcohol and Drug Education PNCA Project Northland Class Action

12

Graph 3. Change in Past 30-Day Use: High School Age Youth


JEG Juvenile Education Groups PFL Prime For Life Under 21 IYS Iowa Youth Survey JADE Juvenile Alcohol and Drug Education PNCA Project Northland Class Action

13

Attitudes Toward Alcohol Use Graphs 4 through 12 on pages 13 through 21 show change in individual attitudes toward substance use from the pre-test to the post-test, by program. Programs are grouped according to the grade of the participants at post-test and by program duration. Attitude data do not include participants taking the JADE survey because that survey does not contain questions regarding attitudes. Individual attitudes either: 1) improved, which means that attitudes grew more unfavorable toward alcohol, tobacco, or marijuana use (e.g., respondent felt alcohol use was wrong at pre-test and very wrong at post-test); 2) maintained +, which means that the pre- and post-test responses remained the same and were unfavorable toward alcohol, tobacco, or marijuana use; 3) maintained -, which means that the pre- and post-test responses remained the same and were favorable toward alcohol, tobacco, or marijuana use; or 4) worsened, meaning that attitudes grew more favorable toward alcohol, tobacco, or marijuana use from pre-test to post-test (e.g., respondent felt marijuana use was very wrong at pre-test and a little bit wrong at post-test). Desired outcomes for these questions are improved and positive maintenance (maintained +) in attitudes. The number in parentheses after each program name is the number of respondents answering the question on both the pre-test and the post-test. Graph 4 shows the change in individual attitudes toward alcohol use from pre- to post-test for single-year programs which served elementary and middle school age youth. Graph 4. Change in Attitudes Toward Alcohol Use by Program: Elementary

and Middle School Age Youth in Single-Year Programs

Eighty-seven percent (87.37%) of Project TNT participants showed positive outcomes.

0

10

20

30

40

50

60

70

80

90

100

Improved Maintained + Maintained - Worsened

Per

cen

tage

of

Res

pon

den

ts

Attitude Change from Pre- to Post-Test

How wrong do you think it is for someone your age to drink beer, wine, or hard liquor regularly?

Project Towards No Tobacco Use (657)

14

Graph 5 shows the change in individual attitudes toward alcohol use from pre- to post-test for multi-year programs serving elementary and middle school age youth. Graph 5. Change in Attitudes Toward Alcohol Use by Program: Elementary

and Middle School Age Youth in Multi-Year Programs

Multi-year program participants showed positive outcomes as follows:

First year of LifeSkills Training – 87% Second year of LifeSkills Training – 73% First year of Project ALERT – 85% Second year of Project ALERT – 73% First year of All Stars – 91% First year of Peer Helping – 87% First year of Too Good for Drugs – 92%

0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts


How wrong do you think it is for someone your age to drink beer,wine, or hard liquor regularly?

First Year of LifeSkills Training (1168)

Second Year of LifeSkills Training (63)

First Year of Project Alert (970)

Second Year of Project Alert (559)

First Year of All Stars (111)

First Year of Peer Helping (60)

First Year of Too Good for Drugs (720)

15

Graph 6 shows the change in individual attitudes toward alcohol use from pre- to post-test for programs serving high school age youth. Graph 6. Change in Attitudes Toward Alcohol Use by Program: High School

Age Youth

High school program participants showed positive outcomes as follows:

Juvenile Education Groups – 73% Prime For Life Under 21 – 74% Project Northland Class Action – 63%

0

10

20

30

40

50

60

70

80

90

100


Perc

enta

ge o

f Res

pond

ents


How wrong do you think it is for someone your age to drink beer,wine, or hard liquor regularly?

Juvenile Education Groups (51)

Prime For Life Under 21 (607)

Project Northland Class Action (131)

16

Attitudes Toward Cigarette Use Graph 7 shows the change in individual attitudes toward cigarette use from pre- to post-test for single-year programs which served elementary and middle school age youth. Graph 7. Change in Attitudes Toward Cigarette Use by Program: Elementary

and Middle School Age Youth in Single-Year Programs

Ninety-four percent (94.41%) of Project TNT participants showed positive outcomes.

0102030405060708090

100


Per

cen

tage

of

Res

pon

den

ts


How wrong do you think it is for someone your age to smoke one or more packs of cigarettes a day?


17

Graph 8 shows the change in individual attitudes toward cigarette use from pre- to post-test for multi-year programs which served elementary and middle school age youth. Graph 8. Change in Attitudes Toward Cigarette Use by Program: Elementary

and Middle School Age Youth in Multi-Year Programs



0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts










18

Graph 9 shows the change in individual attitudes toward cigarette use from pre- to post-test for programs that served high school age youth. Graph 9. Change in Attitudes Toward Cigarette Use by Program: High School

Age Youth



0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts






19

Attitudes Toward Marijuana Use Graph 10 shows the change in individual attitudes toward marijuana use from pre- to post-test for single-year programs serving elementary and middle school age youth. Graph 10. Change in Attitudes Toward Marijuana Use by Program:

Elementary and Middle School Age Youth in Single-Year Programs


0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts


How wrong do you think it is for someone your age to smoke marijuana or hashish once or twice?


20

Graph 11 shows the change in individual attitudes toward marijuana use from pre- to post-test for multi-year programs serving elementary and middle school age youth. Graph 11. Change in Attitudes Toward Marijuana Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts










21

Graph 12 shows the change in individual attitudes toward marijuana use from pre- to post-test for programs that serve high school age youth. Graph 12. Change in Attitudes Toward Marijuana Use by Program: High

School Age Youth



0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts






22

Summary of Positive Outcomes for Attitudes Toward Substance Use Table 4 shows the average positive outcome (improved or maintained+) percentage for each substance by school age group. Table 4. Positive Outcome Percentages for Attitudes Toward Substance Use

by School Age Group

Positive Outcome Percentages for Attitudes Toward Substance Use

School Age Group Alcohol Tobacco Marijuana

Elementary-Middle School Age Youth in 1 Year Programs

87.37 94.41 86.55

Elementary-Middle School Age Youth in Multi-Year Programs

83.96 92.25 83.42

High School Age Youth 69.87 80.92 74.47

All three groups show higher percentages of positive outcomes for tobacco than for alcohol or marijuana.

23

Perceived Risk of Harm from Alcohol Use Graphs 13 through 21 on pages 23 through 31 show change from pre- to post-test, by program, in individuals’ perceptions of risk of harm from using substances. Programs are grouped according to the grade of the participants at post-test and by program duration. The perceived risk data do not include participants taking the JADE survey because that survey does not contain questions regarding perceived risk. Individual perceptions either: 1) increased, which means that their reported perception of risk of harm from using alcohol, tobacco, or marijuana use increased from pre-test to post-test (e.g., respondent felt alcohol use was a moderate risk at pre-test and a great risk at post-test); 2) maintained (+), which means that the pre- and post-test responses remained the same and were unfavorable toward alcohol, tobacco, or marijuana use; 3) maintained (-), which means that the pre- and post-test responses remained the same and were favorable toward alcohol, tobacco, or marijuana use; or 4) decreased, meaning that their reported perception of risk of harm decreased from pre-test to post-test (e.g., respondent reported that marijuana use was a moderate risk of self harm at pre-test and no risk at post-test). Desired outcomes for these questions are an increase in or positive maintenance (maintained +) of perceived risk. The number in parentheses after each program is the number of respondents answering the question on the pre-test and the post-test. Graph 13 shows the change in individuals’ perception of risk of harm from alcohol use from pre- to post-test for single-year programs serving elementary and middle school age youth. Graph 13. Change in Perceived Risk of Harm from Alcohol Use by Program:


0102030405060708090

100

Increased Maintained + Maintained - Decreased

Per

cen

tage

of

Res

pon

den

ts

Change in Perceived Risk from Pre- to Post-Test

How much do you think people risk harming themselves if they have four or five drinks of an

alcoholic beverage nearly every day?


24

Eighty-three percent (82.98%) of Project TNT participants showed positive outcomes. Graph 14 shows the change in individuals’ perception of risk of harm from alcohol use from pre- to post-test for multi-year programs serving elementary and middle school age youth. Graph 14. Change in Perceived Risk of Harm from Alcohol Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts



alcoholic beverage nearly every day?First Year of LifeSkills Training (1176)







25

Graph 15 shows the change in individuals’ perception of risk of harm from alcohol use from pre- to post-test for programs serving high school age youth. Graph 15. Change in Perceived Risk of Harm from Alcohol Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cent

age

of R

esp

ond

ents



alcoholic beverage nearly every day?




26

Perceived Risk of Harm from Cigarette Use Graph 16 shows the change in individuals’ perception of risk of harm from cigarette use from pre- to post-test for single-year programs serving elementary and middle school age youth. Graph 16. Change in Perceived Risk of Harm from Cigarette Use by Program:



0102030405060708090

100


Per

cen

tage

of

Res

pon

den

ts


How much do you think people risk harming themselves if they smoke one or more packs of cigarettes per day?


27

Graph 17 shows the change in individuals’ perception of risk of harm from cigarette use from pre- to post-test for multi-year programs serving elementary and middle school age youth. Graph 17. Change in Perceived Risk of Harm from Cigarette Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts










28

Graph 18 shows the change in individuals’ perception of risk of harm from cigarette use from pre- to post-test for programs serving high school age youth. Graph 18. Change in Perceived Risk of Harm from Cigarette Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts






29

Perceived Risk of Harm from Marijuana Use Graph 19 shows the change in individuals’ perception of risk of harm from marijuana use from pre- to post-test for single-year programs serving elementary and middle school age youth. Graph 19. Change in Perceived Risk of Harm from Marijuana Use by Program:


Ninety-three percent (93.22%) of Project TNT participants showed positive outcomes.

0102030405060708090

100


Per

cen

tage

of

Res

pon

den

ts


How much do you think people risk harming themselves if they smoke marijuana regularly?


30

Graph 20 shows the change in individuals’ perception of risk of harm from marijuana use from pre- to post-test for multi-year programs serving elementary and middle school age youth. Graph 20. Change in Perceived Risk of Harm from Marijuana Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts










31

Graph 21 shows the change in individuals’ perception of risk of harm from marijuana use from pre- to post-test for programs serving high school age youth. Graph 21. Change in Perceived Risk of Harm from Marijuana Use by Program:




0

10

20

30

40

50

60

70

80

90

100


Per

cen

tage

of

Res

pon

den

ts






32

Summary of Positive Outcomes for Perceived Risk of Harm from Substance Use Table 5 shows the average positive outcome (increased or maintained+) percentage for each substance by school age group. Table 5. Positive Outcome Percentages for Perceived Risk of Harm from

Substance Use by School Age Group

Positive Outcome Percentages for Perceived Risk of Harm from Substance Use

School Age Group Alcohol Tobacco Marijuana

Elementary-Middle School Age Youth in 1 Year Programs

82.98 86.53 93.22

Elementary-Middle School Age Youth in Multi-Year Programs

83.53 84.61 91.59

High School Age Youth 82.27 84.48 82.74

All three groups show lower percentages of positive outcomes for alcohol than for tobacco or marijuana.

33

Conclusion This evaluation of the Comprehensive Prevention project answers the following questions:

Has alcohol/tobacco/marijuana usage changed in the target population? Almost all of the program-level percentages of change for past 30-day use of alcohol, tobacco, and marijuana are lower than the Iowa Youth Survey percentages. Exceptions occur in three elementary/middle-school level programs and one high school level program with past 30-day marijuana use. The elementary/middle school program exceptions are the second year of Project ALERT (PA, a 1.18% increase in marijuana use for program participants versus 1% for IYS respondents), the first year of All Stars (AS, a 2.7% increase in marijuana use for program participants versus. 1% for IYS respondents), and the first year of Too Good for Drugs (TGFD, a 1.04% increase in marijuana use for program participants versus 1% for IYS respondents). The high school program exception is Juvenile Alcohol and Drug Education (JADE, a 3.96% increase in marijuana use for program participants versus 3.33% for IYS respondents). Two programs showed either no change or a decrease in use of all substances: Peer Helping (PH) and Juvenile Education Groups (JEG). This outcome for Juvenile Education Groups is particularly notable because that program served an indicated population. Similarly, Prime for Life (PFL) and Juvenile Alcohol and Drug Education (JADE) served indicated populations; PFL showed decreases in use of two substances (alcohol and tobacco) and JADE showed a decrease in use of tobacco and no change (therefore no increase) in use of alcohol. As mentioned in the Outcomes Data: Past 30-Day Use” section, data for the Peer Helping program differ from data for the other Comprehensive Prevention funded programs. The contractor conducts the training but most of the peer mentors are supervised by the school in which the program is implemented. The first year of LifeSkills Training (LST) also showed decreases in use of two substances: alcohol and tobacco. The second year of LifeSkills Training and Project Towards No Tobacco Use showed no change in marijuana use and decreases in tobacco use.

Has the percentage of the target population who indicate positive attitudes (i.e., that it is wrong to use substances) at baseline (pre-test) maintained or increased after the intervention (post-test)?

All single year programs and the first years of multi-year programs for elementary and middle school age youth had 85% or more of participants show positive outcomes for attitudes toward each substance. The second year of multi-year programs did not perform as well on alcohol (73% showed positive outcomes for the second years of LST and PA) and marijuana (63% showed positive outcomes for the second year of LST; 80% for PA). Programs serving indicated high school populations (JEG and PFL) had 73% or more show positive outcomes for all substances. The other high school program, Project Northland Class Action (PNCA), did not perform as well on two substances, with 63% showing positive outcomes for alcohol and marijuana. Regardless of the grade level of participants, all programs had higher positive outcome percentages for tobacco than for alcohol or marijuana, indicating that youth may see tobacco use as more wrong than alcohol or marijuana use.

34

Has perception of risk of harm from alcohol/tobacco/marijuana use maintained a positive response or increased from pre-test to post-test?

All single year programs and the first years of multi-year programs for elementary and middle school age youth had 83% or more of participants show positive outcomes for perceived risk of harm from use of each substance. The second year of multi-year programs did not perform as well on alcohol (77% showed positive outcomes for the second year of LST and 78% showed positive outcomes for PA). The second year of LifeSkills Training also did not perform as well on tobacco, with 72% of participants showing positive outcomes. Regardless of the grade level of participants, most programs had higher positive outcome percentages for perceived risk of harm from tobacco and marijuana use than for alcohol, indicating that youth may see alcohol use as less dangerous than tobacco or marijuana use. Attrition analysis results indicate that participants who are in the 7th grade, participants of racial minority groups, and Hispanic/Latino participants are dropping out of prevention programs at higher rates than others. It may be beneficial for prevention staff to assess the reasons for attrition of these participants and seek to identify and implement methods of increasing retention in these populations.

Appendix A

Institute of Medicine Categories of Populations Served

by Agency and Program

36

Institute of Medicine Population Categories by Agency and Program (continued on following page)

AGENCY PROGRAM POPULATION SERVED (IOM CATEGORY)

ADDS LifeSkills Training

Universal-Direct Project ALERT

ASAC

Project Toward No Drug Abuse Universal-Direct

LifeSkills Training

Boone Prime for Life Indicated

CADS Project ALERT

Universal-Direct Too Good For Drugs

Capstone Diversion Indicated LifeSkills Training Universal-Direct

CFR Too Good for Drugs

Universal-Direct Project ALERT

Clear Lake AEA 267 JCS Diversion Indicated

Compass Pointe Juvenile Alcohol & Drug Education

Indicated

EFR

Project Northland Class Action

Universal-Direct LifeSkills Training Project ALERT Project Towards No Tobacco UseSafe Dates

Helping Services Project Towards No Tobacco Use Universal-Direct

Jackson Recovery All Stars

Selective Girl Power Girl Power Universal-Direct

MECCA LifeSkills Training Universal-Direct

Prime for Life Selective/Indicated

New (View) Opportunities LifeSkills Training Universal-Direct

New Directions All Stars Universal-Direct

New Horizons Project ALERT

Universal-Direct Project Toward No Drug Abuse Diversion Indicated

Pathways

Prime for Life Indicated

Effective Black Parenting Selective

Women’s Empowerment

Prevention Concepts (10) Project Towards No Tobacco Use Universal-Direct Girls Circle

Indicated Diversion

37

Institute of Medicine Population Categories by Agency and Program (continued from previous page)

AGENCY PROGRAM POPULATION SERVED (IOM CATEGORY)

Prevention Concepts (16) Diversion Indicated Girls Circle

Universal-Direct Project Towards No Tobacco Use

SASCC Peer Helping

Universal-Direct All Stars

SATUCI Juvenile Education Groups Indicated

SIEDA Project ALERT Selective

Project Towards No Tobacco Use

YSS Project ALERT

Universal-Direct Too Good For Drugs

comprehensive prevention program...

Documents