conducting longitudinal mobile studies

1© User Centric, Inc., August 2012We Believe Experiences Matter™ Webinar Series

Conducting Successful Longitudinal Mobile Studies

August 21, 2012

PRESENTERS:HeatherRakauskasAssociate DirectorUser Centric, Inc.

Pamela Stoffregen-GaySr. Marketing Mgr.User Centric, Inc.

MODERATOR:

#uxlunch

@UserCentricInc

PiyatidaHaerrSr. User Experience SpecialistUser Centric, Inc.

2© User Centric, Inc., August 2012

About User Centric, Inc.

• User research and design consulting firm; founded in 1999

• Privately held firm with over 50 full time UX research and design consultants

• Headquartered in Chicago

• Approximately 40% of our work is in mobile


Why are we here today?

• 10 tips for conducting a successful longitudinal study

• How to avoid common pitfalls relating to:

• Step-by-step “how to”

• Study design

• Comparison of methodologies

• Data collection tools


What is longitudinal mobile research?

• Involves studying the same group of individuals over an extended period of time

• What are some common questions longitudinal research may answer?

• Stimuli, length of the study, sample sizes, and data collection methods can vary

• What are the benefits vs. drawbacks of longitudinal research?

FrequencyEase of Use

Discoverability

Real World UsePerformance

LearnabilityOverall Experience

Satisfaction


# 1: Plan! Plan Plan!


# 2: Be realistic in your scheduling… timing is everything!

Consider the following when scheduling your study:

• When will devices be ready for testing?

• How will the seasons and holidays impact the schedule?

• When recruiting, ask about prior commitments that may impact participant’s ability to complete part of the study

• Anticipate slippage to your schedule


# 3: Recruit based on behavior, rather than just demographics

• Who would (NOT) be a good fit for your study?

• Screen for behaviors, not just demographics• Past behavior and experiences

predict performance more accurately than straight demographic profiles

• Ask precise screener questions


# 4: Recruit motivated participants

• Why motivated participants?• Ensures that you will get good

data in a timely manner• Increases participant retention• Increases the likelihood you will

get devices back

• Motivators for participants:• $$$$• Getting to try out new technology• Sharing feedback

• Time commitments should be reasonable


# 5: Support participants to be able to use the test device as their own

• Research ahead of time what needs to be done to transfer content• Not just as simple as swapping

out the SIM card!

• Ensure you have allotted enough time to transfer content for each participant

• Make sure the device / application works before participants leave

• Have participants back up their data before receiving and returning the test device


# 6: Be prepared to troubleshoot (during off hours)!

• Expect that participants will experience some difficulties along the way• However, not all participants will

alert you to problems they are having

• Encourage participants to report anything out of the ordinary

• Monitor data

• Allow enough time for troubleshooting

• You may need to be available to troubleshoot during evenings / weekends

• Have backup devices


# 7: Conduct a trial run to catch / prevent issues early on

• Be proactive rather than reactive

• Procedure and environment should be same as real study

• Smaller sample size

• Week or more before actual launch of study

Data CollectionPulling data from device

Technical IssuesLogistics and Scheduling

Device Setup


# 8: Plan for data monitoring

• What to monitor during fieldwork• Technical issues• Task completion• Actual data• Backup

• How often to monitor data• Periodically vs. after tasks /

surveys sent out• Encourage participation, but

don’t bug them


# 9: Anticipate findings

• Ensure your study design will allow you to collect the right data to answer your objectives

• Will users discover the feature of interest?

• Will the feature be used frequently enough in the time allotted?


# 10: Think ahead to how data will be used / analyzed

• What questions are you trying to answer?

• Don’t get lost in the weeds!

• What data should you be collecting?

• What methods are you going to use for data collection?

• How often will you be collecting data?

Think now, track later!


In summary…

1. Plan! Plan! Plan!

2. Be realistic in your scheduling… timing is everything!

3. Recruit based on behavior, rather than just demographics

4. Recruit motivated participants

5. Support participants to be able to use the test device as their own

6. Be prepared to troubleshoot (during off hours)!

7. Conduct a trial run to catch / prevent issues early on

8. Plan for data monitoring

9. Anticipate findings

10.Think ahead to how data will be used / analyzed

17© User Centric, Inc., August 2012We Believe Experiences Matter™ Webinar Series

Questions?

August 21, 2012

#uxlunch

@UserCentricInc

Conducting Successful Longitudinal Mobile Studies

Up Next:Who’s Counting? Using Eye Tracking Measures to Quantify User ExperienceTuesday, September 11, 2012Noon CDT (UTC-4)http://usercentric.com/webinars/

PRESENTERS:HeatherRakauskasAssociate DirectorUser Centric, Inc.

[email protected]@heatheruc

PiyatidaHaerrSr. User Experience SpecialistUser Centric, [email protected]

mailto:[email protected]

mailto:[email protected]

conducting longitudinal mobile studies

Technology