We present the methodology for a multi-site evaluation process of minimum viable products to inform the development and dissemination of a patient-centred and patient-driven mobile application that will facilitate the use of the ICF to improve care and management for people with disabilities and chronic health conditions.

Abstract

Title

We demonstrate the methodology to evaluate a minimum viable software product (MVP) to facilitate the use of the ICF by the population. This requires careful assessment of needs and usability before a product can be rolled out to a larger group of users.

mICF work package 4 Market research methodology to evaluate the development of a mobile application of the ICF

17-23 October 2015 Manchester

United KingdomPoster Number: 000

WHO - FAMILY OF INTERNATIONAL CLASSIFICATIONS NETWORK ANNUAL MEETING 2015

Kraus de Camargo O1, Snyman S2, Anttila H3, Maribo T4, Martins A5, Weckström P6, Wagener D7, Masson C7, Paltamaa J6, on behalf of the International mICF partnership

1McMaster University, Canada; 2Stellenbosch University, South Africa; 3National Institute for Health and Welfare, Finland; 4Marselisborg Centre, Aarhus University and Central Denmark Region, Denmark; 5ESTeSC Coimbra Health School, Portugal; 6JAMK University of Applied Sciences, Finland, 7Stone Three Venture Technology (Pty) Ltd, South Africa

Title

Methods & Materials

References

This work will occur in close collaboration with both content specification (WP2) and lean MVP design (WP3) teams, which run as parallel processes. This work package addresses the question, if the application is not only usable but also if it describes the person’s level of functioning in a valid way.To inform the development team in a timely fashion, the test sites will work with relatively small user groups of about 30 service users and 2 to 4 service providers. This allows for a quick collection of relevant data, feedback to the software designers and rapid updating and bug fixing. Each iteration of the product will be tested in a new group of users without previous exposure to the application. This allows for capturing relevant data from first-time users and avoids a masking effect of potential issues through learning from subsequent exposures. The captured results will then be sent back to the content (WP2) and software design (WP3) working groups. It is estimated that in a period of 6 months, 5 iterations can be concluded in each testing site (see table for testing sites).Evaluations will be conducted in different parts of the world focusing specifically on 3 important and interrelated aspects: information quality, system quality and service quality that will be measured separately.1 Both formative and summative evaluations will be used to gather opinions of service user and service provider groups to enhance further development and dissemination. Their opinions are important in the further development and dissemination of the mICF.2

1) DeLone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J Manag Inf Syst. 2003;19(4):9–30.

2) Ammenwerth E, Duftschmid G, Gall W, Hackl WO, Hoerbst A, Janzek-Hawlat S, et al. A nationwide computerized patient medication history: Evaluation of the Austrian pilot project “e-Medikation.” Int J Med Inform. 2014;83(9):655–69.

3) Holden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. J Biomed Inform [Internet]. Elsevier Inc.; 2010;43(1):159–72. Available from: http://dx.doi.org/10.1016/j.jbi.2009.07.002

4) Davis FD. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of lnformation Technology. MIS Q [Internet]. 1989;13(3):319–40. Available from: http://links.jstor.org/sici?sici=0276-7783(198909)13:3<319:PUPEOU>2.0.CO;2-E

5) Abdekhoda M, Ahmadi M, Dehnad a, Hosseini a F. Information technology acceptance in health information management. Methods Inf Med. 2014;53(1):14–20.

6) Schulz R, Wahl H-W, Matthews JT, De Vito Dabbs A, Beach SR, Czaja SJ. Advancing the Aging and Technology Agenda in Gerontology. Gerontologist. 2014 Aug;[Epub ahead of print].

The evaluation results will contribute to a strengthened evidence base on health outcomes, quality of life and care efficiency gains from the use of the ICT (mICF solution) in integrated service provision. This will reinforce knowledge with respect to management of co-morbidities.

Countries Partners

Finland THL, North Karelia District, Barona Hoiva Oy, JAMK & PT centre (low back pain), JAMK & Onerva Centre for Learning and Consulting

Denmark MC & Spine CentrePortugal ESTeSC – Coimbra Health SchoolItaly AAS2 (adults), FINCB (children)Germany MSH & Early Intervention CentresSouth Africa SU – Rural sites: Cape Winelands Health Districts, Ukwanda Rural Clinical

SchoolSU – Urban sites: Developmental and Community-based healthcare centres

Canada MCM Children’s Hospital, ASD ServicesBrazil CIF & Functional Health Centres from Municipality of BarueriIndia DFI and Community Health Settings (elderly)Korea Silla University Hospital (low back pain)

E

Introduction

ConclusionsThe aim of the tool is to assess how users’ are interacting with mICF (users per country, frequency of contacts with different service providers, costs and net benefits). The technology acceptance model (TAM) will be used for measuring use/intention to use and user satisfaction.3 TAM evaluates:

• The perceived usefulness: “People tend to use or not use an application to the extent they believe it will help perform their job better”. Perceived ease of use refers to "the degree to which a person believes that using a particular system would be free of effort“.4,5 To guarantee the validation of the results, the evaluation will use data triangulation with regard to time, space, or persons, investigator triangulation, theory triangulation, and methods triangulation.2

• Acceptance and use of IT: information technology:

We will also assess by whom and why technology is adopted and/or abandoned. As we strive to hit the “sweet spot” of maximizing well-being and functioning through technology without undermining the future performance potential of the individual we will also assess the potential negative outcomes. The harmful effects could be e.g. compromising autonomy and independence and by promoting a false sense of security.6 @ICFmobile