![Page 1: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/1.jpg)
Assessing Moderated Effects
of Mobile Health
Interventions
on Behavior
Susan Murphy
09.28.16
HeartSteps
JITAI
JITAIs
JITAIs
JITAIs
![Page 2: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/2.jpg)
2
The Dream!
“Continually Learning Mobile Health Intervention”
• Help you achieve, and maintain, your desired
long term healthy behaviors
– Provide sufficient short term reinforcement to enhance
your ability to achieve your long term goal
• The ideal mobile health intervention
– will engage you when you need it and will not intrude
when you don’t need it.
– will adjust to unanticipated life events
![Page 3: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/3.jpg)
Example: Heart Steps
oGoal: Develop an mobile activity coach
for individuals who have coronary artery
disease
oThree iterative studies:
o 42 day pilot micro-randomized study with
sedentary individuals,
o 90 day micro-randomized study,
o 365 day personalized study3
![Page 4: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/4.jpg)
Heart Steps
Data from: Wearable band → activity and
sleep quality; Smartphone sensors →
busyness of calendar, location, weather;
Self-report→ stress, utility
a) In which contexts should the smartphone
provide the user with an activity suggestion?
b) On which evenings should the smartphone
ping the user to plan his/her physical activity
over the next day?4
![Page 5: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/5.jpg)
5
Data from wearable devices that
sense and provide treatments
• On each individual: ��, ��, ��, … , �� , �� , ���, …
• t: Decision point
• ��: Observations at tth decision point (high
dimensional)
• ��: Action at tth decision point (treatment)
• ���: Proximal outcome (e.g., reward, utility, cost)
![Page 6: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/6.jpg)
6
Examples
1) Decision Points (Times, t, at which a
treatment can be provided.)1) Regular intervals in time (e.g. every 10
minutes)
2) At user demand
Heart Steps: a) approximately every 2-2.5 hours
(activity suggestions) and
b) every evening (planning prompts)
![Page 7: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/7.jpg)
7
Examples
2) Observations Ot
1) Passively collected (via sensors)
2) Actively collected (via self-report)
Heart Steps: classifications of activity,
location, weather, step count, busyness of
calendar, usefulness ratings, adherence…….
![Page 8: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/8.jpg)
8
Examples
3) Actions At
1) Types of treatments that can be provided at
a decision point, t
2) Whether to provide a treatment
Heart Steps: a) tailored activity suggestion (yes/no)
and b) evening planning prompt (yes/no)
![Page 9: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/9.jpg)
9
Tailored Walking
Activity Suggestion
![Page 10: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/10.jpg)
10
Examples
4) Proximal Outcome Yt+1
Heart Steps: a) Step count over next 30 minutes
(activity suggestions),
b) Step count on following day (planning prompts)
![Page 11: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/11.jpg)
11
Continually Learning Mobile Health
Intervention
1) Trial Designs: Are there effects of the actions on the
proximal response? experimental design
2) Data Analytics for use with trial data: Do effects vary by
the user’s internal/external context,? Are there delayed
effects of the actions? causal inference
3) Learning Algorithms for use with trial data: Construct a
“warm-start” treatment policy. batch Reinforcement
Learning
4) Online Algorithms that personalize and continually update
the mHealth Intervention. online Reinforcement Learning
![Page 12: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/12.jpg)
12
Heart Steps Micro-Randomized Trial
For each of n individuals and at each of T decision
points, treatment is repeatedly randomized:
a) Activity suggestion (T=210 randomizations)
• Provide a suggestion with probability .6; do nothing
with probability .4
b) Evening planning prompt (T=42)
• Prompt activity planning for following day with
probability .5; do nothing with probability .5
![Page 13: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/13.jpg)
13
Conceptual Models
Generally data analysts fit a series of increasingly more
complex models:
Yt+1 “~” α0 + α1T Zt + β0 At
and then next,
Yt+1 “~” α0 + α1T Zt + β0 At + β1 At St
and so on…
• Yt+1 is subsequent activity over next 30 min.
• At = 1 if activity suggestion and 0 otherwise
• Zt summaries formed from t and past/present observations
• St potential moderator (e.g., current weather is good or not)
![Page 14: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/14.jpg)
14
Conceptual Models
Generally data analysts fit a series of increasingly more
complex models:
Yt+1 “~” α0 + α1T Zt + β0 At
and then next,
Yt+1 “~” α0 + α1T Zt + β0 At + β1 At St
and so on…
α0 + α1T Zt is used to reduce the noise variance in Yt+1
(Zt is sometimes called a vector of control variables)
![Page 15: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/15.jpg)
15
Causal Effects
Yt+1 “~” α0 + α1T Zt + β0 At
β0 is the effect, marginal over all observed and all
unobserved variables, of the activity suggestion on
subsequent activity
Yt+1 “~” α0 + α1T Zt + β0 At + β1 At St
β0 + β1 is the effect when the weather is good (St=1),
marginal over other observed and all unobserved variables,
of the activity suggestion on subsequent activity.
![Page 16: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/16.jpg)
16
Data Scientist’s Goal
• Challenges:
• Time-varying treatment (At , t=1,…T)
• “independent” variables: Zt, St that may be
affected by prior treatment
• Develop data analytic methods that are
consistent with the scientific understanding of
the meaning of the β regression coefficients
• Robustly facilitate noise reduction via use of
controls, Zt
![Page 17: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/17.jpg)
17
“Centered and Weighted
Least Squares Estimation”
• Simple method for complex data!
• Enables unbiased inference for a causal,
marginal, treatment effect (the β‘s)
• Inference for treatment effect is not biased by
how we use the controls to reduce the noise
variance in Yt+1
https://arxiv.org/abs/1601.00237
![Page 18: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/18.jpg)
18
Application of the “Centered and
Weighted Least Squares Estimation”
method in an initial analysis of
Heart Steps
![Page 19: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/19.jpg)
19
Heart Steps Pilot Study
On each of n=37 participants:
a) Activity suggestion, At
• Provide a suggestion with probability .6
• a tailored sedentary-reducing activity suggestion
(probability=.3)
• a tailored walking activity suggestion (probability=.3)
• Do nothing (probability=.4)
• 5 times per day * 42 days= 210 decision points
![Page 20: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/20.jpg)
20
Conceptual Models
Yt+1 “~” α0 + α1Zt + β0 At
Yt+1 “~” α0 + α1Zt + β0 At + β1 At dt
• t=1,…T=210
• Yt+1 = log-transformed step count in the 30 minutes after
to the tth decision point,
• At = 1 if an activity suggestion is delivered at the tth
decision point; At = 0, otherwise,
• Zt = log-transformed step count in the 30 minutes prior to
the tth decision point,
• dt =days in study; takes values in (0,1,….,41)
![Page 21: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/21.jpg)
21
Pilot Study Analysis
Yt+1 “~” α0 + α1Zt + β0 At , and
Yt+1 “~” α0 + α1Zt + β0 At + β1 At dt
Causal Effect Term Estimate 95% CI p-value
β0 At
(effect of an activity suggestion)
�=.13 (-0.01, 0.27) .06
β0 At + β1 At dt
(time trend in effect of an
activity suggestion)
�
= .51
�
= -.02
(.20, .81)
(-.03, -.01)
<.01
<.01
![Page 22: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/22.jpg)
22
Heart Steps Pilot Study
On each of n=37 participants:
a) Activity suggestion
• Provide a suggestion with probability .6
• a tailored walking activity suggestion
(probability=.3)
• a tailored sedentary-reducing activity suggestion
(probability=.3)
• Do nothing (probability=.4)
• 5 times per day * 42 days= 210 decision points
![Page 23: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/23.jpg)
23
Pilot Study Analysis
Yt+1 “~” α0 + α1Zt + β0 A1t + β1 A2t
• A1t = 1 if walking activity suggestion is delivered at the tth
decision point; A1t = 0, otherwise,
• A2t = 1 if sedentary-reducing activity suggestion is
delivered at the tth decision point; A2t = 0, otherwise,
![Page 24: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/24.jpg)
24
Initial Conclusions
• The data indicates that there is a causal
effect of the activity suggestion on step
count in the succeeding 30 minutes.
• This effect is primarily due to the walking
activity suggestions.
• This effect deteriorates with time
• The walking activity suggestion initially increases
step count over succeeding 30 minutes by 271
steps but by day 20 this increase is only 65 steps.
![Page 25: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/25.jpg)
25
Heart Steps Pilot Study
On each of n=37 participants:
b) Evening planning prompt, At
• Provide a prompt with probability .5• Prompt using unstructured activity planning for
following day with probability=.25
• Prompt using structured activity planning for
following day with probability=.25
• Do nothing with probability=.5
• 1 time per day * 42 days= 42 decision points
![Page 26: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/26.jpg)
26
Conceptual Models
Yt+1 “~” α0 + α1Zt + β0 At
Yt+1 “~” α0 + α1Zt + β0 At Wt + β1 At (1-Wt)
• t=1,…T=42
• Yt+1 = square root-transformed step count on the day after
the tth day,
• At = 1 if activity planning prompt on the evening of the tth
day; At = 0, otherwise,
• Zt = square-root step count on the tth day,
• Wt =1 if Sunday through Thursday; Wt =0, otherwise
![Page 27: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/27.jpg)
27
Pilot Study Analysis
Yt+1 “~” α0 + α1Zt + β0 At , and
Yt+1 “~” α0 + α1Zt + β0 At Wt + β1 At (1-Wt)
Causal Effect Term Estimate 95% CI p-value
β0 At
(effect of planning)
�= 1.7 ns ns
β0 At Wt + β1 At (1-Wt)
(effect of planning for
weekday (Wt =1) and
for weekend(Wt =0)
�
= 3.6
�
<0
(.74, 6.4)
ns
<.02
ns
![Page 28: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/28.jpg)
28
Heart Steps Pilot Data
On each of n=37 participants:
b) Evening planning prompt
• Provide a prompt with probability .5• Prompt using unstructured activity planning for
following day with probability=.25
• Prompt using structured activity planning for
following day with probability=.25
• Do nothing with probability=.5
• 1 time per day * 42 days= 42 decision points
![Page 29: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/29.jpg)
29
Conceptual Model
Yt+1 “~” α0 + α1Zt + β0 A1t + β1 A2t
• Yt+1 = square root-transformed step count on the day after
the tth day,
• A1t = 1 if unstructured activity planning prompt on the
evening of the tth day; A1t = 0, otherwise,
• A2t = 1 if structured activity planning prompt on the
evening of the tth day; A2t = 0, otherwise,
• Zt = square-root step count on the tth day,
![Page 30: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/30.jpg)
30
Pilot Study Analysis
Yt+1 “~” α0 + α1Zt + β0 A1t + β1 A2t t=0,…T=41
• A1t = 1 if unstructured activity planning prompt on the
evening of the tth day; A1t = 0, otherwise,
• A2t = 1 if structured activity planning prompt on the
evening of the tth day; A2t = 0, otherwise,
Causal Effect Term Estimate 95% CI p-value
β0 A1t + β1 A2t β��
= 3.1
�
>0
(-.22, 6.4)
ns
.07
ns
![Page 31: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/31.jpg)
31
Heart Steps Pilot Study Analysis
Yt+1 “~” α0 + α1Zt + β0 A1t Wt + β1 A1t (1-Wt)
+ β2 A2t Wt + β3 A2t (1-Wt)
A1t = 1 if unstructured activity planning prompt on the
evening of the tth day; A1t = 0, otherwise,
Wt =1 if Sunday through Thursday; Wt =0, otherwise
Causal Effect Estimate 95% CI p-value
β0 A1t Wt + β1 A1t (1-Wt)
+ β2 A2t Wt + β3 A2t (1-Wt)
�
= 5.3
�
<0, �
>0,
β�
<0
(2.2, 8.5)
all ns
<.01
all ns
![Page 32: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/32.jpg)
32
Initial Conclusions
• The data indicates that there is a causal
effect of planning the next day’s activity on
on the following day’s step count
• This effect is due to the unstructured planning
prompts.
• This effect occurs primarily on weekdays
• On weekdays the effect of an unstructured
planning prompt is to increase step count on the
following day by approximately 780 steps.
![Page 33: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/33.jpg)
33
Discussion
Problematic Analyses
• GLM & GEE analyses
• Random effects models & analyses
• Machine Learning Generalizations:
– Partially linear, single index models & analysis
– Varying coefficient models & analysis
--These analyses do not take advantage of the micro-
randomization. Can accidentally eliminate the advantages of
randomization for estimating causal effects--
![Page 34: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/34.jpg)
34
Discussion
• Randomization enhances:
– Causal inference based on minimal structural
assumptions
• Challenge:
– How to include random effects which reflect
scientific understanding (“person-specific” effects)
yet not destroy causal inference?
![Page 35: Assessing Moderated Effects of Mobile Healthpeople.seas.harvard.edu/~samurphy/seminars/FENS.9.28.2016.pdf · effect of planning the next day’s activity on on the following day’s](https://reader034.vdocuments.mx/reader034/viewer/2022042102/5e7f38c77e352c6e5e43abf8/html5/thumbnails/35.jpg)
35
It takes a Team!