www.ert.com

Laura Khurana MPH,1 Michelle K. White PhD,2 Jakob B. Bjorner MD PhD,2 Sarah Gary PhD,1 Ellen Durand PhD,1 Antonio Otero,1 Susan Dallabrida PhD1

1ERT (F/K/A PHT Corporation), Boston, MA USA • 2Optum (F/K/A QualityMetric), Lincoln, RI USA

OBJECTIVESThe Short-Form 36 Health Survey version 2.0 (SF-36v2®) is a validated patient-reported outcome (PRO) instrument that measures functional health and well-being from the patient’s point of view. It consists of 36 questions that measure eight health domains: physical function (PF), role limitations due to physical health (RP), bodily pain (BP), general health (GH), vitality (VT), social function (SF), role limitations due to emotional problems (RE), and mental health (MH). The eight scales can be summarized in a physical health component (PCS) and a mental health component (MCS) (Table 1).

Clinical trials are increasingly using PRO measures,1 and the SF-36® is the most frequently used PRO instrument in clinical trials.2 Although it was originally developed to support paper and pencil administration, the increased pace of electronic data collection in clinical trials necessitates that PRO instruments be adapted to electronic formats in ways that ensure measurement comparability to the original paper-based instruments. In 2007, a single-item format (SIF) version of the SF-36v2 was developed for deployment on computer screens/tablet-sized devices. This version presents the items to subjects one at a time, and was tested in 2009.3 In response to increasing requests from trial sponsors and to accomodate smaller device screen sizes, a handheld SIF version was developed in 2012 which presents instructions on a single screen and each item text alone (without instruction) one at a time on subsequent screens. The handheld SIF version has not yet been validated, although a similar version was tested in a rheumatoid arthritis population.4

The objectives of this study were to evaluate measurement equivalence between the paper and electronic (handheld SIF) versions of the SF-36v2 administered using a handheld device or a smartphone app in four disease groups (rheumatoid arthritis, COPD, depression, and type II diabetes), and to determine patient preference for mode of administration. Results presented here are from the type II diabetes group only, and thus only a preliminary analysis is possible.

METHODSSubjects were recruited from the general public in the Boston area and screened over the phone. Eligible subjects were 18 or older, reported that they had been told by a doctor that they have diabetes, high blood sugar, or sugar in their urine, and reported taking insulin or other medication for their diabetes. 101 subjects participated in a randomized crossover study in which they completed the SF-36v2 on two modalities: paper and either the electronic handheld (PHT LogPad® LW, Figure 1A) or the smartphone App (PHT LogPad App™, Figure 1B). Subjects completed the assessments in a single session with distraction activities between completion of the fi rst and second modality (Figure 2). After the study, subjects completed an exit survey which assessed patient preference for mode of administration. Subject responses to the paper SF-36v2 were entered into an electronic database for analysis. Mean score diff erences were analyzed as electronic score minus paper & pencil score. The score diff erences were evaluated using statistical tests and published criteria for minimally important diff erence (MID).5 Agreement between modes of administration was evaluated using the intraclass correlation coeffi cient (ICC), with 95% confi dence

intervals (CI) estimated through empirical bootstrap.

RESULTSSubjects ranged in age from 30 to 79 years and were 52% female. 62% of subjects had received at least some college education. 50% reported owning a smartphone, and 67% reported using the internet on a daily or weekly basis. About a quarter of subjects (26%) reported that they never use the internet or cannot recall the last time they did (Table 2).

Most subjects found the electronic mode (either handheld or App) easy to use (83%) and easy to navigate (81%). Of those subjects who expressed a preference, 71% found the electronic mode faster to complete than paper, 62% found it more physically comfortable than paper. There were 55% of subjects who would prefer an electronic handheld mode over paper and 20% had no preference.

For subjects using electronic data capture on handheld devices, 100% of responses were captured for all items of the SF-36v2 whether subjects used the smartphone App or the LW device. In contrast, 11% of paper SF-36v2 administrations were missing at least one data point and as many as 10 responses were missing from a single subject on paper. Subject response rates on paper ranged from 72%-100% complete. Subjects most frequently omitted questions with multiple parts on paper (92% of missing responses). The domains most aff ected were MH, VT, RE, PF, and SF. 64% of the missing responses were from Question 9, which consists of nine MH and VT domain items.

Overall, average scores for the SF-36 in this population were more than 5 points below the general population for the PCS and six subscales: PF, RP, BP, GH, SF, and RE (Table 3). Some ceiling eff ects were observed for the RP, SF, and RE subscales (Table 3).

Results of comparison of handheld and paper & pencil administration

(Figure 3 and Table 4):

• Signifi cantly higher scores were found on the app administration of the RE scale and the MCS. However, diff erences were below the recommended MID.

• No signifi cant score diff erences were found for the remaining scales or for PCS

• High ICCs (>0.75) were found for six SF-36 subscales and the two component summaries. However, the confi dence intervals are wide.

• ICCs were low for RP and RE subscales.

Results of comparison of App and paper & pencil administration (Figure 4 and Table 4):

• No signifi cant score diff erences were found on any subscale or component summary.

• All score diff erences were below the MID.

• High ICCs were found for 6 SF-36 scales and the two component summaries. However, the confi dence intervals are wide.

• ICCs were low for the RP and RE subscales.

CONCLUSIONIn a study of type II diabetes patients, we used a randomized cross-over design to compare paper and pencil administration of the SF-36v2 Health Survey with administration by smartphone App or handheld device (LogPad). Out of 20 comparisons, we found no score diff erences that exceeded the recommended MID. Comparison of the LogPad device and paper & pencil showed no signifi cant score diff erences for any subscale or component summary. For comparisons of App and paper & pencil administration, we found score diff erences for the RE subscale and the MCS that were below MID but statistically signifi cant. These diff erences could be spurious (due to multiple testing), or due to transferring the RE items from paper to smartphone app. Two subscales, RP and RE, showed low ICCs for comparison across modes. However, confi dence intervals were wide. Ongoing work is evaluating mode diff erences in a larger sample and alternative display of the RP and RE items on app devices.

Overall, the results of this study demonstrate preliminary equivalence to within the specifi ed MID for the SF-36v2 as implemented on the LogPad handheld device and the smartphone App and the paper and pencil administration. Full results of the equivalence analysis will be possible with the addition of data from the three remaining cohorts, which encompass an additional 300 subjects. Most patients with diabetes found the handheld electronic versions easy to use and navigate, and would prefer to use them over paper & pencil. In addition, electronic administration eliminated missing or incomplete data.

.

MEASUREMENT EQUIVALENCE AND PATIENT PREFERENCE FOR THE SF-36V2 ON A HANDHELD DEVICE AND SMARTPHONE APP

DISCLOSURESThe authors are employees of ERT (F/K/A PHT Corporation) and Optum (F/K/A QualityMetric).

REFERENCES1 Vodicka E, Kim K, Devine EB, et al. Inclusion of patient-reported outcome measures in registered clinical trials: Evidence from ClinicalTrials.gov (2007-2013). Contemp Clin Trials. 2015 Apr 18. Epub ahead of print.

2 Scoggins JF, Patrick DL. The use of patient-reported outcomes instruments in registered clinical trials: evidence from ClinicalTrials.gov. Contemp Clin Trials. 2009;30(4):289-92.

3 Smith K, Kosinski M. Psychometric comparability of single item and drug form administration of the SF-36v2 health survey. Poster presentation at the ISPOR 16th Annual International Meeting, Baltimore, MD. 2011.

4 Tiplady B, Goodman K, Cummings G, et al. Patient-reported outcomes in rheumatoid arthritis: Assessing the equivalence of electronic and paper data collection. Patient. 2010;3(3):133-43.

5 Maruish M (Ed.). User’s manual for the SF-36v2 Health Survey (3rd ed.), Lincoln, RI:QualityMetric Inc., 2011.

This documentation and the data contained herein is the property of PHT Corporation, located at 500 Rutherford Avenue, Boston, MA 02129, and its licensors. Any use, disclosure, reproduction, modifi cation, display or transfer of the data and/or documentation is prohibited, except by the express written authorization of PHT Corporation. This document contains proprietary PHT Corporation information. Copyright © 1994-2015 by PHT Corporation. All rights reserved. PHT®, ESENDANT®, FIRSTHAND KNOWLEDGE®, LOGPAD®, SITEPAD®, NETPRO®, ESENSE™, STUDYWORKS™, CONFIDENCE AT EVERY PHASE™, LOGPAD APP™, SMARTREPORTS COMPLIANCE APP™, PHT MEDICATION MODULE™, PHT INTEGRATION ENGINE™, PATIENT CONNECT™ and ETMF OPTIMIZE™ are trademarks of PHT Corporation. All other trademarks are the property of their respective owners. The documentation and referenced inventions herein are covered by one or more of the following U.S. Patents Nos. 5,778,882; 6,095,985; 6,282,441; 6,640,134 and 7,273,454. THE INFORMATION IN THIS DOCUMENT IS SUBJECT TO CHANGE WITHOUT NOTICE. PHT CORPORATION MAKES NO WARRANTIES OF ANY KIND REGARDING THIS MATERIAL AND, ASSUMES NO RESPONSIBILITY FOR ANY ERRORS THAT MAY APPEAR IN THIS DOCUMENT.

Presented at the International Society for

Pharmacoeconomics and Outcomes Research

(ISPOR) 20th Annual International Meeting

16-20 May 2015, Philadelphia, PA, USA

Table 2. Subject Demographics and Experience with Technology (N=101)

Gender

Male 48%

Female 52%

Age

Average (range) 56 (30-79)

Highest level of education completed

8th grade or less 4%

Some high school 5%

High school graduate/GED 30%

Some college/Technical degree/AA 35%

College degree (BA/BS) 18%

Advanced degree (MA, PhD, MD) 9%

Do you have a smartphone?

Yes 50%

No 50%

How frequently do you use the internet?

Daily 54%

Weekly 13%

Monthly 7%

I have used it, but do not recall when I used it 6%

I do not use the internet 20%

Table 1: Abbreviations for SF-36v2 Subscales and Component Summaries

Health Domain or Component Summary Abbreviation

Physical Functioning PF

Role-Physical RP

Bodily Pain BP

General Health GH

Vitality VT

Social Functioning SF

Role-Emotional RE

Mental Health MH

Physical Component Summary PCS

Mental Component Summary MCS

Table 3. Distribution of SF-36 subscale and component summary scores (paper & pencil

administration) in the two substudies

LogPad App Mean Std % Min % Max Mean Std % Min % Max

PF 43.3 10.3 0 4 42.0 10.8 2 8

RP 43.4 9.8 2 16 42.8 10.1 4 16

BP 43.6 9.2 0 6 42.4 9.8 4 8

GH 45.4 10.7 0 2 43.7 10.4 0 2

VT 50.1 9.7 0 2 48.3 10.1 0 4

SF 45.4 10.7 0 32 43.0 11.6 4 25

RE 43.3 12.4 2 34 40.0 12.6 4 24

MH 48.3 11.1 0 8 46.3 11.2 0 4

PCS 43.1 9.0 0 0 42.8 8.6 0 0

MCS 48.1 12.4 0 0 45.2 11.3 0 0

Table 4. Intraclass correlations (ICC) between electronic and paper & pencil administration of SF-36

subscales and component summaries

LogPad App Scale ICC (95% CI) ICC (95% CI)

PF 0.79 (0.62-0.91) 0.85 (0.77-0.92)

RP 0.67 (0.40-0.87) 0.69 (0.48-0.85)

BP 0.78 (0.64-0.87) 0.81 (0.67-0.90)

GH 0.90 (0.82-0.95) 0.86 (0.77-0.92)

VT 0.82 (0.68-0.91) 0.82 (0.64-0.92)

SF 0.78 (0.63-0.89) 0.77 (0.64-0.88)

RE 0.69 (0.42-0.88) 0.68 (0.50-0.83)

MH 0.90 (0.82-0.95) 0.87 (0.78-0.93)

PCS 0.88 (0.81-0.93) 0.89 (0.82-0.94)

MCS 0.89 (0.78-0.96) 0.82 (0.70-0.91)

Figure 1B

SF-36v2® SF-36v2®Distraction activities

n=26

n=24

n=27

n=24

Demographic survey

Mod

e of

Adm

inis

trat

ion

Study procedures

Electronic(LogPad® LW)

Paper

Electronic(LogPad App™)

Figure 2

Figure 4

Score by App compared to paper & pencil

(mean score diff erences and 95% confi dence interval)

Grey lines indicate recommended values for the minimally important diff erence for each scale5.

-4

-2

0

2

4

6

PF RP BP GH VT SF RE MH PCS MCS

Figure 4. Score by APP compared to paper & pencil(mean score difference and 95% con�dence interval)

Figure 3

Score by LogPad compared to paper & pencil

(mean score diff erences and 95% confi dence interval)

Grey lines indicate recommended values for the minimally important diff erence for each scale5.-4

-2

0

2

4

6

PF RP BP GH VT SF RE MH PCS MCS

Figure 4. Score by LogPad compared to paper & pencil(mean score difference and 95% con�dence interval)

For more information, contact

ert.com/contact-us.

Figure 1A

1. ERT ISPOR 36x72_ Poster.indd 1 5/16/15 12:17 AM