experimental evaluation of anti- tobacco psas · broad objective to conduct experimental research...

Experimental evaluation of anti-tobacco PSAs

Supported by: NCI Center of Excellence in Cancer Communication Research Grant at the ASC/APPC University of Pennsylvania

Andrew Strasser, Joseph Cappella, Christopher Jepson, Robert Hornik, Martin Fishbein, Caryn Lerman

©2011 Caryn Lerman, DO NOT COPY OR USE

Improving Anti-Tobacco Public Service Announcements

•Mass media anti-smoking campaigns have had some promising results, but outcomes have been inconsistent

•Self-report data on message effectiveness have limitations

•Experimental research studies can be used for PSA design and evaluation


Broad ObjectiveTo conduct experimental research on features of anti-tobacco PSAs that influence persuasiveness, and translate this research to the development of more effective media campaigns

To test the main and interacting effects of PSA message sensation value (MSV) and argument strength (AS) on physiological responses, persuasion, and behavior change.

Specific Objective


Phase I: PSA Rating and Selection

1. Categorizing Topics

2. Evaluating Message Sensation Value

3. Evaluating Arguments


•569 smoking-related PSAs were categorized.

•Exclusion Criteria included:Target – ChildrenTarget – AdolescentsPSAs other than 30-second durationnon-English speakingsecond hand smoke-focus

1. PSA Categorization


Reliability of Categorization

2 Annenberg and 2 TTURC raters categorized the PSAs:

- TTURC raters were >90% in agreementwith Annenberg ratings

- TTURC raters were >95% in agreement with each other

- Overall Kappa = 0.89, p<.001


Target Audience (adults)

155942

010203040506070

Promote Tx-Seeking Neg. Conseq. (any) Knowledge about smoking

# of PSAs (n=99)


2. Evaluating Message Sensation Value PSAs differ in format, content and audiovisual features. Message Sensation Value (MSV) is a measure of these attributes.

Perceived Message Sensation Value (PMSV) is the subjective report of the MSV features.

MSV is coded for visual, audio and content dimensions.

Morgan, Palmgreen, Stephenson, Hoyle, Lorch: J of Comm, 2003.©2011 Caryn Lerman, DO NOT COPY OR USE

MSV Scoring (Kentucky/rPMSV Morgan et al)Category Item Scoring

Visual

Length Time (seconds)

Animation 0/1

Cuts 0=1-6; 1=7-14; 2=15+

Edits 0=1-6; 1=7-14; 2=15+

Faces count

Special Visual Effect 0/1

Slow Motion 0/1

Fast Motion 0/1

Unusual Colors 0/1

Intense Moments 0/1


MSV Scoring (Kentucky/rPMSV Morgan et al)

Category Item Scoring

Audio

Sound Saturation 0/1

Music 0/1

Sound Effects 0/1

Slow Voice 0/1

Fast Voice 0/1

Content

Acted out/ Talking Head 1 vs 0

Narrative 0/1

Unexpected Format 0/1

Surprise/Twist Ending 0/1

Celebrity 0/1


Reliability of MSV scoresInter-rater reliability of MSV scoring (2 raters):

MSV scores from subset of 45 of 99 PSAs:Kendall’s Tau = 0.906, p<0.001.

Correlations

1 .961** .523** .139.000 .000 .171

99 99 99 99.961** 1 .353** -.031.000 .000 .762

99 99 99 99.523** .353** 1 -.118.000 .000 .244

99 99 99 99.139 -.031 -.118 1.171 .762 .244

99 99 99 99

Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)N

msv

visual

audio

content

msv visual audio content

Correlation is significant at the 0.01 level (2-tailed).**.

Internal Consistency (MSV, visual, audio, content dimensions)


3. Evaluating ArgumentsStep 1: Argument Extraction

Arguments were extracted in a 2-step process.1) 2 raters viewed the PSAs and typed explicit and

implicit messages from each PSA.2) 2 new raters viewed PSAs and read previous arguments to form one statement reflecting argument of PSA.

Step 2: Argument EvaluationShopping mall-intercept study of 300 participants (149 female) who were each presented 12 PSA arguments such that each argument was presented a minimum of 36 times (36-38 times) during the study.

All participants were current smokers (mean 17.5, sd = 11.7)who had smoked at least 100 cigarettes during their lifetime.


After each statement, participant was asked if the statement (rated 1 strongly disagree to 5 strongly agree):

Is a reason for quitting smoking that is BELIEVABLE

Is a reason for quitting smoking that is CONVINCING

Is a reason for quitting smoking that is NEW to me

Is a reason for quitting smoking that APPLIES to me

Gives a reason for quitting smoking that is IMPORTANT TO ME

Put THOUGHTS in my mind about quitting smoking

Put THOUGHTS in my mind about wanting to continue smoking

Helped me feel CONFIDENT about quitting smoking

Would help my friends quit smoking

Made me want to quit smoking

Overall, how much do you agree or disagree with the statement

Is the reason given for quitting smoking a strong or weak reason* Argument Evaluation Survey adapted from Zhao, Cappella et al (NIDA study)©2011 Caryn Lerman, DO NOT COPY OR USE

Descriptive Data on Argument Evaluation

Count 99

Mean 29.68

SD 2.23

Minimum 20.85

Maximum 33.03


Pearson CorrelationP-value

Arg. Eval. NFC Perc. Vul. Try to quit Will QuitInterest in Quit

Argument Evaluation Total

.061

.294.510 *.000

.424 *

.000.350 *.000

.524 *

.000

Need ForCognition

.061

.294.193 *.001

.135*

.019.056.337

.172*

.003

Perceived Vulnerability

.510 *

.000.193 *.001

.229 *

.000.182*.002

.476*

.000

Try to quit .424 *.000

.135*

.019.229 *.000

.767*

.000.662*.000

Will quit .350 *.000

.056

.337.182*.002

.767*

.000.604*.000

Interest in quit .524 *.000

.172*

.003.476*.000

.662*

.000.604*.000

Association of Argument Evaluation with Intention Items (N=300)


PSA Selection

0.00 1.00 2.00 3.00 4.00 5.00

rPMSV

-2.00

-1.00

0.00

1.00

2.00z_

scor

e

1.00

2.00

3.00

4.00

5.00

6.00 7.00

8.00

9.00

10.00

11.00

12.00

13.00

14.00

15.00

16.00

17.00

18.00

19.00

20.00

21.00

22.00

23.00

24.00

25.00

26.00

27.00

28.0029.00

30.00

31.00

32.00

33.00

34.00

35.00

36.00

37.00

38.00

39.00

40.00

41.00

42.00

43.00

44.0045.00 46.00

47.00

48.00

49.00

50.00

51.00

52.00

53.00

54.00

55.00

56.00

57.00

58.00

59.00

60.00

61.00

62.00

63.0064.00

65.00

66.00

67.00

68.00

69.00

70.00

71.00

72.00

73.00

74.0075.00

76.00

77.00

79.00

80.00

81.00

82.00 83.0084.00

85.00

86.00

87.00

88.00

89.00

90.00

91.00

92.00

93.00

94.00

95.00

97.00

98.00

99.00

-1SD Mean +1SD

+1SD -1SD


Study Design and Procedures

Baseline Questionnaire

View PSAs (as a set of 4)•Psychophysiology assessment

Outcome Measures•Beliefs, attitudes, intentions, etc

View PSAs (individually)•Processing,effectiveness

Recall Measures

N=50

N=50

N=50

N=50

Argument Strength

Weak Strong

MSV

Low

High

Calling TTURC Quit Line4 PSAs/condition


MSV

ARG

PSA•Arousal•Process•Recall

Beliefs Norms Intention

Attitude

Efficacy

Activation Theory Integrated Model

Conceptual Model


N=200

High MSV Low MSV

High Arg(N=49)

Low Arg(N=48)

High Arg(N=50)

Low Arg(N=53)

Age 46.4 (12.2) 42.4 (11.7) 44.2 (13.9) 41.3 (11.5)

Sex (% male) 47.5 44.6 40.8 50.1

Race (% White) 65.0 60.0 57.0 60.3

Education (%HS) 62.3 58.3 57.1 58.0

Cigs per day 21.7 (8.8) 20.4 (9.4) 23.3 (14.1) 18.6 (8.2)

FTND 5.2 (2.2) 5.4 (2.3) 5.5 (2.2) 5.3 (2.5)

Sensation Seeking 20.8 (4.3) 20.4 (4.4) 21.2 (3.9) 20.4 (4.8)

Descriptive Statistics by PSA Condition


MSV

ARG

PSA•Physio.•Process•Recall


Attitude

Efficacy


Conceptual Model


Zygomatic and Corrugator


http://en.wikipedia.org/wiki/File:Zygomaticus.png�

http://upload.wikimedia.org/wikipedia/commons/b/b4/Corrugator_supercilii.png�

0

0.02

0.04

0.060.08

0.1

0.12

0.14

0.16

High Low

highlow

Physiological Measures -- Corrugator (mV)

Arg

MSVAS: ns MSV: p=.012


0

0.02

0.04

0.060.08

0.1

0.12

0.14

0.16

High Low

highlow

Physiological Measures -- Zygomatic (mV)

Arg

MSVAS: nsMSV: ns


MSV

ARG



Attitude

Efficacy


Conceptual Model


4

4.2

4.4

4.6

4.8

5

5.2

5.4

High Low

highlow

Cognitive Processing

Arg

MSVAS: nsMSV: p=.013“…think about the consequences of quitting smoking as

shown in the PSAs” ©2011 Caryn Lerman, DO NOT COPY OR USE

MSV

ARG



Attitude

Efficacy


Conceptual Model


0

20

40

60

80

100

High Low

highlow

Recall -- Percent Correct

Arg

MSV

AS x MSV: p=.001


MSV

ARG



Attitude

Efficacy


Conceptual Model


3.9

4

4.1

4.2

4.3

4.4

4.5

High Low

highlow

Positive Beliefs (Mean: 5 items, 5 point scale)

Arg

MSVAS: nsMSV: ns


2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

High Low

highlow

Negative Beliefs (Mean: 7 items, 5 point scale)

Arg

MSVMSV: p=.03


MSV

ARG



Attitude

Efficacy


Conceptual Model


24

25

26

27

28

29

30

31

High Low

highlow

Efficacy (10 item; 1-4 scale)

Arg

MSVMSV: p=.035


MSV

ARG



Attitude

Efficacy


Conceptual Model

NS

But… Thanks to Michael Hennessy, we find support for the Integrated Model when using perceived PSA parameters, rather than conditions…. Stay tuned..©2011 Caryn Lerman, DO NOT COPY OR USE

This is Your Brain on PSAs….


PSA Video:“Family Reunion”

30 sec

Frame #100:01:32

Frame #200:11:56

Frame #300:21:02

Frame #400:25:77


Recognition by MSV Level

1200

1250

1300

1350

1400

Hi MSV Lo MSV

RT

RT


Brain Effects of Anti-Tobacco PSAs

Recognition of PSA frames is positively correlated with prefrontal and temporal, and negatively correlated with occipital activation

Low MSV PSAs are associated with greater prefrontal and temporal activation; High MSV produced occipital activation

Langleben et al. Neuroimage, 2008©2011 Caryn Lerman, DO NOT COPY OR USE

Effects of Smoking Cues in Anti-Tobacco PSAs

Argument Strength

(between-subject)

Smoking Cue (within-subject)

No Yes

Strong3 PSAs 3 PSAs

Weak3 PSAs 3 PSAs

Yahui KangJ. Cappella

advisor

N=82©2011 Caryn Lerman, DO NOT COPY OR USE

3.103.153.203.253.303.353.403.453.503.55

pre no-cueads

post no-cue ads

pre smk-cue ads

post smk-cue ads

Smoking Cue

Sm

okin

g U

rge

Weak ArgumentStrong Argument

Smoking Cues in PSAs with Weak Arguments Increase Urges

Kang et al., 2009©2011 Caryn Lerman, DO NOT COPY OR USE

Eye Tracking Smoking Cues

• Ashley Sanders Jackson, Cappella et al• Visual attention to smoking cues in ad segment• 84 adult smokers• 16 PSAs

EOS = eyes on screenP < .01


Smoking Cues in Anti-Tobacco PSAs

Model of Cue Effects on Persuasion

We will also explore moderating influences on cues effects, such as nicotine dependence, motivation to quit, and gender.

Argument strength

-

-

+SmokingCues

Message Processing•Recall•Ad effectiveness

Cue-ResponseManipulation

urges

physio.Persuasion•Attitudes•Self-efficacy

Intention to quit

Smokingbehavior

Manipulation

+

+

3 (no cue, central cue, peripheral cue) x 2 (weak, strong argument) factorial design (n=300)…ads balanced for MSV, themes


Implications

• “Attention-grabbing” high MSV format could impede the learning and retention of PSA message

• Smoking cues in PSAs may increase urges and distract attention from PSA message, thereby reducing effectiveness (effect may depend on argument strength and whether cue is central to PSA argument)

• Further experimental support could influence PSA design and effectiveness of future anti-tobacco media campaigns


CollaboratorsAndrew Strasser, Robert Hornik, Joseph Cappella, Martin Fishbein

Paul Wileyto, Christopher Jepson, Paul Sanborn

Daniel Langleben, James Loughead, Kosha Ruparel

FUNDING: NCI P50©2011 Caryn Lerman, DO NOT COPY OR USE

experimental evaluation of anti- tobacco psas · broad objective to conduct experimental research...

Documents