![Page 1: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/1.jpg)
Evaluation in an Era ofDigital Technology Innovation
Reflections for Philanthropy
Eric C. Schneider, MD, MSc, FACPSenior Vice President for Research and Policy
@ericschneidermd
![Page 2: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/2.jpg)
Outline
• Challenges in evaluating digital health technology • A human-centered measurement approach• Concluding thoughts
![Page 3: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/3.jpg)
Evaluating digital technology:Are these devices?
• Clinical trials testing • Five-phase model• Reliance on randomized, controlled trials• Highly-selected patient samples• Clinical outcomes/surrogate biological markers• 12-18 years from lab to marketing
![Page 4: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/4.jpg)
Digital health technologies: Many value propositions
• Increase patient engagement• Close communication gaps between patient and providers• Improve providers’ ability to tailor services to population
segments• Enable convenient and cost-effective care delivery• Improve decision making
4
M. Hostetter, S. Klein, and D. McCarthy, Taking Digital Health to the Next Level: Promoting Technologies That Empower Consumers and Drive Health System Transformation, The Commonwealth Fund, October 2014.
![Page 5: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/5.jpg)
Innovation on Three Different Time HorizonsUnknown
Known
Partially known
Existing Business Model: Process
InnovationExecute
New/Disruptive Business Model
Search
New Opportunities via Business Model
InnovationExecute/Search
Source: Steve Blank, Lean Innovation Management – Making Corporate Innovation Work
Digital technology innovation is unpredictable
![Page 6: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/6.jpg)
Dependencies: VIVIFY Health and CHRISTUS St. Michael Health
• Goal: Reduce 30-day readmissions and reduce Care Transition Nurse case load
• Implement a remote patient monitoring solution• Transmits patient data using personal health devices
(tablets, weight scales, blood pressure cuffs, etc.)• If patient data indicates a health concern, device alerts
Care Transition Nurse via text/email
![Page 7: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/7.jpg)
Measurement Myopia:VIVIFY Health Evaluation Results• Pilot study: patients with chronic congestive heart failure
who consented to participate in care transition program• Results
• 44 patients• Decrease in cost of care from $12,937/per patient to $1,231/per
patient• ROI of $2.44, including cost of technology
• A winner?
![Page 8: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/8.jpg)
Evaluating Smarter:VIVIFY Health’s Qualitative Insights•Connectivity issues
• Patients outside of cellular service areas
•Equipment issues• Tablets overheated
•Bugs!• Packaging promoted egg-laying
Ambar Kulshreshtha JCK, Abhinav Goyal, Elkan F. Halpern, and Alice J Watson. Use of Remote Monitoring to Improve Outcomes in Patients with Heart Failure: A Pilot Trial. International Journal of Telemedicine and Applications. 2010;2010(870959):7
![Page 9: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/9.jpg)
A New Approach: Human-Centered Designwww.sagehealthadvisor.com
EMOTIONAL NEEDS
PERSONALNEEDS
FUNCTIONAL NEEDS
MEDICALNEEDS
![Page 10: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/10.jpg)
Measuring the impact of digital health technologies
Technology•Feasibility•Functionality•Unexpected
Bugs
Workflow•Take up
•Use Patterns
•Stickiness
•Safety
•Unintended Consequences
•Costs
Clinical•Adherence to
Treatment•Use of Health
Services•Quality of
Care•Patient
Engagement•Patient
Experience•Provider
Experience
Person/Life
•Social Relationships
•Quality of Life
•Caregiver experience
•Self-Efficacy
•Functioning
•Health Status
•Emotional Status
![Page 11: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/11.jpg)
Conclusion: Evaluation Approach
• Start with a “person/life” holistic view
• Assess technology compatibility with human-centered design principles
• Beware of workflow and adjacent technologies
• Expect the unexpected!
![Page 12: Evaluation in an Era of Digital Technology Innovation: Reflections for Philanthropy](https://reader031.vdocuments.mx/reader031/viewer/2022030309/58f1c80d1a28aba16b8b4591/html5/thumbnails/12.jpg)
References1. Jones SS, Heaton PS, Rudin RS, Schneider EC. Unraveling the IT productivity
paradox--lessons for health care. N Engl J Med. 2012;366(24):2243-2245.2. HIMSS. Case Study: Decreasing Costs and Improving Outcomes Through
Community-Based Care Transitions and Care Coordination Technology Healthcare Information and Management Systems;March 2014.
3. Broderick A. Partners HealthCare: Connecting Heart Failure Patients to Providers Through Remote Monitoring. Online: The Commonwealth Fund;2013.
4. Ambar Kulshreshtha JCK, Abhinav Goyal, Elkan F. Halpern, and Alice J Watson. Use of Remote Monitoring to Improve Outcomes in Patients with Heart Failure: A Pilot Trial. International Journal of Telemedicine and Applications. 2010;2010(870959):7.
5. Rudin et al. The value of health information technology: filling the knowledge gap. Am J Managed Care. 2014; eSP1-eSP8.
12