Preliminary findings from a randomized mixed-methods cross-over study to

compare the performance of DVD and web-based patient decision support

Dominick L. Frosch, PhD

Associate Investigator, PAMFRI Associate Professor of Medicine, UCLA

July 31, 2012

Knowledge gaps in decision support

n  The internet is increasingly becoming the primary medium for providing patients decision support.

n  Few data are available to guide optimal design of internet decision support and ensure comparable effectiveness compared to video.

n  A handful of studies have examined the efficacy of internet decision support, but only one study (2003) directly compared internet and video decision support.

Review of Video and Internet Presentation

n  98% of participants assigned to the Video group viewed the tape.

n  54% of those assigned to Internet viewed the

entire presentation of 47 slides. 40% never reviewed any part of the presentation.

Frosch, Kaplan & Felitti, JGIM, 2003.

PSA Knowledge

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Pre Post

VideoInternet

p<.001

PSA Knowledge: Video & Internet Completers

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Pre Post

Video I-Net Comp.

Research questions

n  How do internet and video decision support compare in terms of: –  Increases in patient knowledge –  User acceptability –  Decision quality

n  What proportion of web-based decision support do individuals review and how much time do they spend with it?

Study design

Figure  1.  Research  design  Primary care

patients (N=90)

Complete baseline quantitative survey - 1

Review DVD

Review Health

Crossroads

Complete quantitative survey - 2

Review DVD

Review Health

Crossroads

Complete quantitative survey - 3

RA probes content selection

RA probes content selection

Study measures

n  Decision/disease specific knowledge n  Quantitative user ratings of decision support n  Decision quality n  Role preferences n  Click-stream (para) data on HC use n  Think-out loud while using HC n  In-depth interviews following each review of

decision support

Study setting and sampling

n  Palo Alto Medical Foundation in Silicon Valley, CA.

n  Participants are active primary care patients at PAMF who have/are: –  Diabetes –  Benign Prostatic Hyperplasia –  Overdue to consider colon cancer screening

n  Participants are randomly selected from primary care panels and invited by mail/phone.

Participant demographics

N=30

Age 64.6 (SD=12.9)

Gender 63.3% Male

Marital status 76.7% married or living with partner

Ethnicity 83.3% Caucasian 10.0% Asian 6.7% Latino

Education 80.0% College graduate or more

Household income 85.7% >$50,000

Health literacy score (range 0-8) 7.9 (SD=.35)

How much do patients view on the Health Crossroads?

50.00% 51.00%

45.00%

54.00% 50.00%

36.00%

66.00%

46.00%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

All BPH CRC Diabetes

DVD first WWW first

How much time do patients spend on the Health Crossroads?

25.2 26.7

16.6

32.2

21.0 17.6 17.9

27.4

33

45

31

24

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

All BPH CRC Diabetes

DVD first WWW first Video length

How many videos in the Health Crossroads do patients click on?

1.5 1.4 1.8

1.4

2.5

0.8

5.0

1.8

0.0

1.0

2.0

3.0

4.0

5.0

6.0

All BPH (25) CRC (27) Diabetes (19)

DVD first WWW first

Time spent watching videos in the Health Crossroads

1.6

1.3

2.0

1.4

2.4 2.6

3.1

1.6

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

All BPH CRC Diabetes

DVD first WWW first

Reflections on using Health Crossroads

“I didn’t even look at the videos but just the presence of the videos seemed like it was getting in my way, it was just filling up space, and you know I didn’t come looking for a video…these days not knowing what’s going to happen when you click on a video, if you’re going to get an advertisement, and uh, and then you know, something you know a video, will take 10 minutes to say something that you can read in a minute.” MALE, 54, CRC "My reaction is to take the DVD only because everybody likes to watch a DVD, but the website is better. But see I was forced to read the website. If I had the DVD, I will actually put the DVD in and play it. I would go to [the website]; I don't know if I would do as intensive reading as I did here, um, um. There has to be a catch but I don't know what the catch is...There has to be follow-up. If there's no follow up, then they might just flip through [the website] and jot down and see a picture and read a line, but not really read the whole thing." MALE, 61, DIABETES

Changes in knowledge - Combined

65.0%

82.0% 79.0%

56.0%

68.0%

77.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

Baseline First review Second review

All

DVD first WWW first

Changes in knowledge - BPH

40.0%

73.0%

63.0%

43.0% 48.0%

60.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

Baseline First review Second review

BPH

DVD first WWW first

Changes in knowledge – CRC Screening

78.0%

95.0% 98.0%

55.0%

80.0%

92.0%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

Baseline First review Second review

CRC

DVD first WWW first

Changes in knowledge – Diabetes

80.0% 80.0% 80.0%

71.0%

73.0%

78.0%

66.0%

68.0%

70.0%

72.0%

74.0%

76.0%

78.0%

80.0%

82.0%

Baseline First review Second review

Diabetes

DVD first WWW first

Understanding how patients use HC

“It wasn’t too busy, you know um, I didn’t get bored, I could easily find something you know, some websites are too much you know, this was simple here’s all the different tests you can take and these are all the procedures, I liked the chart you know the summary thing cause you forget and with the summary then you don’t have to click from page to page cause you have it all right there so that’s probably what I liked best was the summary chart.” FEMALE, 52, CRC "I liked [the website] better because you could read something if weren't sure about it you can go back and look at it." "In any given amount of time, you can absorb just so much information. So the DVD is a little too fast, but that can't be helped. So the website is a very good supplement." MALE, 85, BPH

Predictors of Health Crossroads knowledge gain

Variable R2 Change Beta Age .10 -.17 Education .21 .28 Proportion of pages viewed

.05 .18

Number of videos clicked .10 .40

Cumulative R2 = .46

Which medium do patients prefer?

21.4%

64.3%

14.3%

43.8%

37.5%

18.7%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Prefer video Equal ratings Prefer Health Crossroads

DVD first WWW first

Preferences for Health Crossroads

"Maybe I shouldn't say I would never use the DVD. If [my doctor] gave me the DVD as a package and said view the DVD, you know, and then go to the website, you know, I would do that. As I was watching [the DVD], I was thinking that I would never go back to this as a tool. I would watch it once...and be done with it." MALE, 66, DIABETES "I am not an internet person. I don't sit there and look at this stuff all day long, but I can read it. I can go back to it. I can access it immediately with a mouse, and get what I want. [I] don't have to watch the whole thing again to figure out where in the picture it was and I have to go find this out. So I like that first website very much." MALE, 85, DIABETES “The DVD you didn’t have any choice you just had to watch whatever came up but this one you could choose what you wanted to see.” FEMALE, 64, DIABETES "Everybody sits at a computer and it's just so easy you know, but you get a DVD in mail and go "Oh I want to watch that" and you throw it on your desk, but you never do it. But everyone sits at a computer and it's just "click" and we're there." FEMALE, 52, CRC

Conclusions

n  Data appear to confirm that DVD may be the superior option, for ensuring that patients are exposed to all necessary information.

n  Knowledge gains appear to be greater for DVD.

n  More viewing of videos embedded in HC appears to drive greater knowledge gains.

n  Some suggestions that having both mediums is valuable for patients – start with DVD, but use HC to reinforce knowledge gains.

Limitations and next steps

n  Findings are preliminary, based on 1/3 of ultimate sample of 90 participants.

n  Study is exploratory, not designed or powered for null-hypothesis significance testing.

n  Non-naturalistic laboratory study, but qualitative data give us insight into how people might use decision support at home.

n  Participant recruitment is currently 85% complete. Analysis will be complete by year-end.

Questions and suggestions?