My Healthy Weight Action Plan

Lessons LearnedNovember , 2010

Agenda

1. Why an etool?

2. Tour of the tool

3. Process steps and learning• “If I were to do it again”

4. Post launch live learning• Usability• User data and feedback

5. Enhancements

Why an Healthy Weight etool?

Weight was a strategic focus

Needed to strengthen HSF consumer information in this area

Have had immense success with three prior etools:• Blood Pressure Action Plan• Heart&Stroke Risk Assessment• E Mail Coach Program

Objective:•Incorporate most promising practices into our health information•Increase engagement and interactivity of our health information

Program Tour

Weekly Lessons

12 Lessons

1. Monitoring2. Goal setting3. Tracking4. Overcoming challenges5. Rewards6. Setbacks7. Support system8. Behaviour contract9. Time management10. Stress11. Plateaus12. Maintaining/new beginning

Lesson 2- A Key Lesson

• Feedback on week of self-monitoring• Recommended caloric intake, number of servings• Learn to set goals• Personalized recommendations re what to work on• VERY LONG

Tool Sections and Features

Process Steps and Learning

Phase 1 literature search and environmental scan

Phase 2 Design and Build•Vendor selection•Contract negotiation•Design and build process

Lit Search & Env’t Scan

Purpose:What tools are out there?What is available to buy? License?How effective are they? What is the research evidence?Process:1. RFP- to researchers and academics active in consumer eHealth tools2. Criteria:

Expertise: behavior change; online interventions; technology; Price and timingPotential for involvement in future phasesNo conflict of interest. (i.e. able to provide a unbiased review)Commitment to the project. Creativity

3. Cross Foundation Review Panel

Lit Search & Env’t Scan- Conclusions

Hundreds of diet/weight e-tools– most are highly commercials; generally either:

– simple counters & assessments or– tools that inundate with features and gadgets

– none theory based; no structure

Nothing suitable to buy

→Developed recommended practices

→Decision to proceed

If I were to do it again….

1. Would stipulate at lit search RFP stage, that if we proceed to build, the author of the lit search cannot take a lead role (researchers have vested interested)

2. Would challenge conclusions a bit more.

Design and Build- Vendor Selection ProcessRFP •to researchers, academics, commercial vendors in consumer eHealth •internal and external experts and stakeholders to assess proposals

Weighted Criteria:•Project Approach & Plan – 35%•Qualifications of the vendor- 30%•Quality, clarity and responsiveness of proposal – 10%•Financial stability of vendor – 5%•Cost quotation and IP needs – 20%

Contract: Did not give one vendor full budget including design and programming (ie. have them subcontract) due to concerns regarding contract size.Vision: Product design vendor would lead the selection of design and programming vendors, and HSFO would hold the contract.

If I were to do it again….

1. Keep the formal criteria and workgroup for vendor selection•Helped with the quality and effectiveness of decision process, as well as buy in•Decision of this size not be made by one person or one department, •Provides transparent process

2. Would give one vendor full project responsibility- from concept design, to graphic design through to implementation • having 3 separate contracts allowed for vendors to avoid/debate responsibility “ie.

that is their job” • Define deliverables and roles in detail.

3. Seek a vendor with full capability (from concept – implementation) • Particularly when client does not have internal expertise or capacity• Ensure vendor has expertise on projects of same scale

4. Look more closely at the commercial sector. •More likely able to provide the IA and technical services required; subject matter knowledge can be brought in easily.

Contract Discussions and IP Rights

Contract Discussions: •Took 4 months; $20K in legal fees; IP rights as a sticking point•Vendor perceived IP rights where we thought there was none•Agreed royalty free rights for the life of the program

If I were to do it again…•Increase clarity at the RFP stage re IP ownership•Be aware of Universities drive for IP ownership•If IP becomes barrier at contract stage: be firm, willing to walk away.

Timing and Stages

Design and Copy Phase: Fall 2007Wireframes and copySelected graphic design and programming vendors• Specs written by lead vendor• December: Handing off to Design Vendor

Graphic and interface design: December-January 2008Full detail (architecture and specs) not outlined or tested at time graphic and interface design was developed

Timing & Stages

Final Rules Development:Lead vendor was to apply “final rules” to design prior to hand off to programmer• Questions re how best to move forward lead us to engage web developer to review work

Conclusion From Review Process:•IA insufficiently defined•Architecture and navigation needs simplification•Need to pare back scope of tool

Spring-Summer 2008 spent defining IA•Programming started Fall 2008•Product tested and completed June 2009

If I were to do it again….

1. Ensure roles are exhaustively defined and scoped out. Revisit them frequently.• Role definition is that much more urgent if you work with multiple vendors.

2. Engage a dedicated Project Management expertise and capacity. PLUS, plan on extensive staff involvement for strategic guidance.

3. Do not proceed to interface design, until the wireframe and requirements are fully defined .

If I were to do it again….

4. Need detailed wireframes or designs for every unique page and every page that is interactive.

5. Invest the time and money to test a paper prototype.

6. Keep scope modest so that you can finish design and programming in that time frame.•Too large undermines the ability to generate momentum•Too overwhelms and drags down morale•Market changes

Post Launch Live Learning

User data

Usability research

Participation and Attrition

Great interest in the tool. Key attrition areas identified for attention• Landing page to risk assessment; risk assessment to registration

• Only 38% of users to landing page proceeded to RA• 64% or users finishing RA we on to register for HWAP

• Tradeoff :value of data versus the extra effort and time required of user.

Sept- Aug. 2010 ,000’s

HW Plan Landing Page 123.9

Started Risk Assessment 47.8

Finished RA 40.6

Registered for HW Plan 26.0

User Demographic

Gender: 90% are female

Age: 76% are 35-64

Ethnicity: Majority (90%) are Caucasians; 1% Chinese, 3% S Asian, 2% African Heritage, 2% Aboriginal

BMI: 30% have overweight BMI 25-29.9; 49% obese BMI 30+

Participation over time

Rapid up-front attrition. Only 113 (0.5%) completed Lesson 12, of 21,659 that started lesson 1 Most users were completing multiple lessons on the same day, rather than coming weekly. Lesson Started Completed % of Started Cumulative % 1 21,659 17,510 81% 80.8% 2 10,301 2,450 24% 11.3% 3 1,820 1,046 57% 4.8% 4 920 637 69% 2.9% 5 557 406 73% 1.9% 6 369 292 79% 1.3% 7 270 232 86% 1.1% 8 217 193 89% 0.9% 9 186 164 88% 0.8% 10 161 150 93% 0.7% 11 142 133 94% 0.6% 12 127 113 89% 0.5%

Lesson Feedback- “How did you find the lesson?”

Majority responded positively suggesting the contents of the lesson itself are positive.

Negative feedback fell into several categoriesNothing new/expected more: (25%)Concerning motivation (e.g., “didn’t do it, as not inspired,” “it doesn’t keep me motivated”): (15%)Navigation or technical issues: (14%)Program did not meet needs/unmet need: (10%)Negative comments (e.g., “boring”, “didn’t work”, “too short,” “): (8%)Weight management is a personal responsibility : (5%)

Article Feedback & Journaling

Articles: viewed an average of 1.6 times per user, •a small proportion (4%) of users who opened an article rated it •those that did rate, tended to rate the articles as being good

Less than 3% of users journalled•total of 624 journal entries. •Majority made only 1-2 entries; a few used journal function 6 times+ Themes:•Recognition of the importance of tracking •The food tracker made users feel discouraged (too many frowns) •The benefits of making healthier choices (“feeling happy and energetic”)•Frustration that “doing good” does not necessarily translate to the scale•Description of challenges to making healthier choices •Recognition of the problems of emotional eating •Health concerns “Cardiologist told me…”

Goal Setting Practices

•2600 users created goals.

•Poor understanding of what constituted a SMART goal or what was meant by a “task.”

•Did not use the goal system as intended (either due to confusion or choice)

•Many picked up on HW PLAN concepts in creating their goals

Usability Evaluation

1. The value proposition of the site is unclear.Not immediately clear what the value of using this site is, and why one should use

it rather than another online weight loss site. Recommendation:•Ensure clarity: what HW Plan offers versus other online weight loss tools. •Improve execution and communication of the program “Point of Difference”. 2. The use of the counselor (in its execution) has caused usability issues.The reason for the presence of the counselor is unclear, and her lack of identity makes her presence and lesson language uncomfortable, phony and cumbersome. Static execution of the counselor is not consistent with internet trends/capabilities Recommendation:Remove the counselor from the site entirely.

Usability Evaluation

3. The linear format of lessons does not translate well to the web.The general premise of the HW Plan, which directs users through twelve distinct online lessons in sequence, does not work well. Users are forced through content that is not interesting or relevant to them, and must make a decision: continue forward, waiting for later content up that does look interesting, or leave the site, to find more interesting content elsewhere.Recommendation:Remove the rigid approach of twelve linear sessions that restrict the user from key concepts that may be helpful at the outset.

4. Nothing is done to encourage the user to return on a weekly basis.Although users are told they should return to the site weekly, nothing is done to promote this behaviour. Recommendation: If the metaphor of twelve sessions is kept, design techniques for getting users to return to the site, e.g. a reminder email once a week.

Usability Evaluation

5. Navigating around the site is difficult.The structure of the site is confusing. Several items can be found in multiple areas of the site. Because concepts are spread out over so many pages of the site, it can be difficult to remember where to find something. The labels used in the navigation do not help. Recommendation: Revisit the information architecture of the site. Keep concepts like “goals” in specific sections, rather than having them spread out over a number of pages Assess whether labels are sufficiently intuitive. 6. There is no way to communicate with other participants of the site.Today, the internet is all about communication. In the world of “Web 2.0”, site visitors are used to creating their own content, and interacting with others. By not giving program participants any way to communicate with each other, sharing challenges, successes and tips, the site is missing out on a great opportunity. Recommendation: Build increased social networking opportunities into the program.

Usability Evaluation

9. Line lengths are extremely long, making text harder to read.In typography, the optimal length of a line of text is 66 characters

Recommendation:In redesign, ensure line length is consistent with usability best practices.

Current Status and Plans

Revising architecture since last May, working with usability experts•Iterative, including copy writing

Expert review in September with behavior change experts sent us back to the drawing board.

Revised architecture and wireframes and copy.

Just completed consumer testing.

Programming quotes and go/no go decision.

Development team and science basis•Project design lead by WMU•Team of 2 Canadian RD’s; reviewed by HSF RD•PA content reviewed by:• a certified personal trainer with experience in community

recreation, • a volunteer with post grad in Kinesiology

•Content experts:•Review by CBT and Obesity Expert• Product design consultation provided by consultant with expertise in e intervention design and research

•Existing Canadian guidelines and available research •Expert Advisory Group: High level advice

;