challenges and best practices in ethical review of human and … · stephanie jansen-kosterink6,...
TRANSCRIPT
-
58
IMIA Yearbook of Medical Informatics 2020
© 2020 IMIA and Georg Thieme Verlag KG
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of TestimoniesA Joint Contribution from the International Medical Informatics Association’s Human Factors Engineering and the European Federation for Medicatl Informatics’ Human and Organizational Factors of Medical Informatics Working GroupsLinda W. Peute1, Valentina Lichtner2, Melissa T. Baysari3, Maria Hägglund4, Juell Homco5, Stephanie Jansen-Kosterink6, Ignacio Jauregui7, Johanna Kaipio8, Craig E. Kuziemsky9, Elin Christina Lehnbom10, Francisca Leite11, Blake Lesselroth5, Daniel Luna7, Carlos Otero7, Rune Pedersen12, Sylvia Pelayo13, Raquel Santos11, Nuno-André Silva11, Mari Tyllinen8, Lex Van Velsen6, Wu Yi Zheng3, Monique Jaspers1, Romaric Marcilly131 Centre for Human Factor Engineering of Health Information technology - Amsterdam UMC,
University of Amsterdam, department of Medical Informatics, Amsterdam, The Netherlands2 Centre for Medication Safety and Service Quality, UCL School of Pharmacy, UK3 The University of Sydney, Faculty of Medicine and Health, Sydney, Australia4 Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA; Department of
Women’s and Children’s Health, Uppsala University, Uppsala, Sweden5 Department of Medical Informatics, University of Oklahoma - Tulsa School of Community Medicine, USA6 Roessingh Research and Development, eHealth group, Enschede, The Netherlands7 Health Informatics Department, Hospital Italiano de Buenos Aires, Argentina8 Department of Computer Science, Aalto University, Finland9 MacEwan University, Edmonton, AB, Canada10 Department of Pharmacy, Faculty of Health Sciences, UiT The Arctic University of Norway,
Norway; Department of Health and Caring Sciences, Faculty of Health and Life Sciences, Linnæus University, Sweden
11 Hospital da Luz Learning Health, Portugal12 Norwegian Centre for E-health Research, University Hospital of North Norway HF, Norway; Depart-
ment of Clinical Medicine, Faculty of Health Sciences, UiT The Arctic University of Norway, Norway13 Univ. Lille, CHU Lille, ULR 2694 - METRICS: Évaluation des technologies de santé et des pratiques
médicales, INSERM-CIC-IT 1403/Evalab, Lille, France
SummaryObjective: Human and Organizational Factors (HOF) studies in health technology involve human beings and thus require Institu-tional Review Board (IRB) approval. Yet HOF studies have specific constraints and methods that may not fit standard regulations and IRB practices. Gaining IRB approval may pose difficulties for HOF researchers. This paper aims to provide a first overview of HOF study challenges to get IRB review by exploring differences and best practices across different countries. Methods: HOF researchers were contacted by email to provide a testimony about their experience with IRB review and approval. Testimonies were thematically analyzed and synthesized to
identify and discuss shared themes. Results: Researchers from seven European countries, Argentina, Canada, Australia, and the United States answered the call. Four themes emerged that indicate shared challenges in legislation, IRB inefficiencies and inconsistencies, general regulation and costs, and lack of HOF study knowledge by IRB members. We propose a model for IRB review of HOF studies based on best practices. Conclusion: International criteria are needed that define low and high-risk HOF studies, to allow identification of studies that can undergo an expedited (or exempted) process from those that need full IRB review. Enhancing IRB processes in such a way would be beneficial to the conduct of HOF studies. Greater knowl-
edge and promotion of HOF methods and evidence-based HOF study designs may support the evolving discipline. Based on these insights, training and guidance to IRB members may be developed to support them in ensuring that appropriate ethical issues for HOF studies are considered.
KeywordsEthical review, human factors, organizational factors, ergonomics, social sciences
Yearb Med Inform 2020:58-70http://dx.doi.org/10.1055/s-0040-1701979
Published online: 2020-04-17
-
IMIA Yearbook of Medical Informatics 2020
59
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies
1 IntroductionAfter the horrors of World War Two, the Dec-laration of Helsinki was widely adopted as a guide to conduct research on human beings [1]. Even if it initially explicitly targeted medical research, it is recognized that this declaration concerns also social sciences, and any research involving humans or infor-mation/data about them. Nowadays, various national and international scientific societies have adopted, extended, and updated this declaration to help researchers conduct studies in an ethical way (e.g. American Psychology Association [2]).
As for medical informatics, the American Medical Informatics Association (AMIA) acknowledges that the Declaration of Helsinki “should guide all human subject research, including research that involves users of informatics tools and interventions as human subjects (e.g., workflow analysis studies, clinical decision support systems analysis, patient care innovations analysis, etc.)” [3]. Therefore, human and organiza-tional factors (HOF)-related studies in the field of medical informatics must comply with the Declaration of Helsinki.
One of the principles of the Declaration of Helsinki is that research protocols must be reviewed by an independent committee prior to initiation. Institutional Review Boards (IRB, also known as institutional research board, ethical appraisal board, committee of protection of persons, review ethical board, (competent) ethics committee, ethical review authority, (medical) research ethics commit-tee) have been created at institutional, local, regional, or national levels. Their mission is to determine whether the benefits of a research study outweigh its risks, appropriate participant consent procedures have been included, and the research design treats all groups of individuals fairly – e.g. no one is excluded from the research. Inspired by medical scientific journals, medical infor-matics journals also increasingly require that investigators seek approval to conduct their research from an IRB for the study to be published (e.g. [4]).
Yet, there is variability across the “ethical traditions” of review boards. This variability is likely to be the result of different research fields (e.g., psychology, sociology, manage-
ment sciences), types of institutions where the research will take place (e.g. university, hospital), and regulations and laws applica-ble in the country. Therefore, researchers in HOF within the medical informatics domain face different regulatory constraints and challenges, which is even more complex when studies cross national boundaries. As for other types of research (e.g. [5]), submission of HOF studies to an IRB can be resource intensive and time consuming, resulting in delays in the conduct of the study. Synchronizing review approval pro-cesses with timelines of health information technology (HIT) implementations under study can be especially challenging – e.g., resulting in the technology being implement-ed before permission to conduct the study has been obtained.
Differences in rules and practices across IRB are often not justified. Best practices should be shared and implemented across the world, so that research benefits from the ethical review and is not negatively affected by the review process. Furthermore, if we wish to develop, implement, and evaluate transnational HIT, such as health informa-tion exchange or patient-facing applications (patient travel beyond national boundaries), it is important to understand the rules and practices in ethics reviews in the field of HOF across the world.
2 Aim of the Contribution and Collection of TestimoniesThe aim of this paper is to perform a first exploration of practices related to IRB review in the context of HOF studies, their processes, constraints, outcomes, and advan-tages in different countries and to provide an overview of best practices. With this goal in mind, IMIA’s Human Factors Engineering and Organizational Issues working group and EFMI’s Human and Organizational Fac-tors of Medical Informatics working group contacted researchers in HOF in health tech-nology with experience in involving human beings in their studies. These researchers were asked to report their experiences with IRB addressing the following topics:
• Context of their experience with IRB in their country: which types of HOF studies should be submitted to IRB and to what extent? Are all studies concerned by the same process?
• How do IRB operate? (submission pro-cess, duration of the process, administra-tive level of submission)
• What are the perceived advantages and drawbacks of the process? What is the perceived usefulness of the IRB review process for HOF studies and medical technology in general?
Researchers representing each country are not legal experts: they reported only their own experiences with IRB, which may not be an accurate representation of the regu-lation applicable in their own country. The reports of each country were not intended to be exhaustive. Once the reports of each country were collected, similarities and differences between the IRB processes of each participating country were synthesized and discussed in order to highlight the best practices that should be shared and the com-mon challenges.
3 TestimoniesResearchers from 11 countries accepted the invitation to contribute on this topic. Con-tributions are provided by country, listed in alphabetical order. Appendix 1 provides a structured overview of the organization of the IRB review process, the planning of the process, and the challenges faced country per country.
3.1 Argentina (I. Jauregui, D. Luna, C. Otero) In Argentina, every study involving human participants is required to have an IRB ap-proval prior to its initiation, and to deliver a follow-up documentation at least once a year until the study is completed. This process is regulated by a national law that although mainly focused on pharmacolog-ical, diagnostic, and therapeutic protocols, also includes HOF studies in healthcare because of their intrinsic nature of studying human beings and healthcare. This national
-
60
IMIA Yearbook of Medical Informatics 2020
Peute et al.
law, which subscribes to the Declaration of Helsinki, delegates the surveillance of the protection of participants’ rights to the jurisdictional and institutional IRBs where the studies are carried out, by controlling and supervising the studies. Local ethical boards are autonomous: they decide about the approval of the studies, and they report to the jurisdictional (city) level only for statistical purposes.
IRBs have to be composed of multidisci-plinary staff, evenly distributed across age, gender, and scientific and non-scientific members. The Hospital Italiano de Buenos Aires has an IRB that oversees all studies involving human subjects. It has a main procedure for approving protocols, in which investigators are required to present a study plan including a detailed methodology, clinical impact, data protection measures and funding sources, an informed consent of the participants to the study, a good clinical practices certificate, a letter of compliance with the Declaration of Helsinki, and the CV of the main investigator. The committee meets every 15 days, and in each session, it analyzes the new protocols and notifies the investigators of its decision, or asks for supplementary information or amendments for the protocol to be approved. This process takes around one to two months, and after approval investigators have to fill a form for the Health secretary that keeps track of all clinical protocols within its jurisdiction.
The IRB has also an expedited procedure for studies that investigate aspects related to the clinical management of the organization, or for studies in which no sensitive data (i.e. demographics, diagnosis, vital signs, procedures, etc.) are handled, no therapeu-tic or diagnostic instrument is used, and no possible harm is done to the patient. In this procedure, the statement from the commit-tee is issued more rapidly. This procedure is sometimes used to get IRB approval for HOF studies, since no possible harm is done to participants, and findings help the organization to better engage with patients and staff information needs, and interaction with computer systems.
We believe that legislation about ethics in HOF and generally in Health Informatics studies is lacking in our country. This type of studies arises different risks for participants
than pharmacological or diagnostic test studies, and they should be regulated and monitored in a more specific way.
3.2 Australia (M. Baysari, W. Y. Zheng) As human factors researchers, we undertake studies that focus on understanding how work is done in practice and how information technology supports or hinders that work. For example, we have run multiple qualita-tive observational and interview studies in hospitals where we examined how computer-ized decision support influenced medication decision-making by doctors [6–9]. Although this research is non-interventional, to col-lect and publish data of any kind requires researchers to obtain ethics approval. In Australia, this is a two-step process. Initially, ethics approval must be obtained from a Na-tional Medical and Health Research Council (NHMRC)-approved Human Research Ethics Committee (HREC). These committees, which typically meet monthly, assess research to ensure it meets the requirements of the National Statement on Ethical Conduct in Human Research and is ethically acceptable [10]. Following HREC approval, site-spe-cific governance approval must be obtained from each participating site (e.g. hospital). Site-specific assessment (SSA) allows the site to consider whether they have the capacity to conduct the research (e.g. physical resources, staff, insurance, and indemnity requirements). This can be particularly time-consuming when multi-site research is being done. If you are examining an IT system in three hospitals, you would need to obtain HREC approval once, but SSA approval three times. We see value in obtaining ethics approval to under-take research, including non-interventional HOF research, but believe there are some key problems with the process.
Our biggest concern is with the lack of standardization in the application and approval process across sites and jurisdic-tions. For example, additional forms are required for interstate researchers wishing to conduct research at sites in Queensland (Public Health Act) and Victoria (Victorian Specific Module).
Second, up until recently, the processes and application form required for ethics approval was dependent on the risk level of a project.
Projects deemed to be at “low or negligible risk” were expedited through a streamlined process and the form required to be completed by researchers was shorter and simpler than the standard ethics form. Most HOF research comprises low risk research and so was processed through this expedited process. In 2019, Australia’s national ethics form was revised and now a single application form is used for all projects, regardless of risk level.
Finally, in order to gain access to a site for research, researchers are required to go through a series of authorizations and checks including vaccinations, police checks, and employment checks. Unfortunately, once a researcher is approved to collect data at one site, this does not carry across to other sites, even if in the same Australian State. New forms and checks are required to be completed at each data collection site.
Overall, we do not dispute the value of ensuring HOF research is undertaken ethically, posing minimal risk and inconve-nience to participants (in our case, users of technology). Currently, the ethics approval process is viewed by HOF researchers as a barrier to completing research and can act as a deterrent to conducting multi-site research within public and private health organizations. This is problematic, as much can be learned from conducting research at multiple sites (i.e. context-specific factors influencing IT uptake or success). We are confident that the availability of a low effort, streamlined and consistent process for appli-cations which pose a minimal risk to users and organizations, would be welcomed by all Australian HOF researchers.
3.3 Canada (C. Kuziemsky) In Canada, the Tri-Council Policy Statement (TCPS 21) is the national policy for ethical research involving humans. The TCPS 2 pro-vides definitions for types of research studies and provides guidance for more complicated studies such as multi-jurisdictional research. There are three general categories of review. First is full board review, which is used for clinical trials or other intervention studies. Second is minimal-risk review, which is
1 http://www.pre.ethics.gc.ca/eng/policy-politique_tcps2-eptc2_2018.html
-
IMIA Yearbook of Medical Informatics 2020
61
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies
done when the risk of adverse outcomes is no more than the risk from everyday life. Minimal risk is typically done by a delegated review where a selected set of review ethical board (REB) members review the file. Third is expedited review, done in cases when a project already has ethics approval from a TCPS 2-compliant Canadian university or hospital or if a project is using anonymized non-public secondary data. An example of this would be using anonymized data from a hospital. HOF studies involving human sub-jects where data is being collected to answer a research question would always require ethics approval. Studies involving patient recruitment for observation or evaluation such as a usability study would be classified as minimal risk review and typically have turnaround times of up to six weeks. Expe-dited review would take up to three weeks.
However, while general guidelines for ethics approval are quite clear, the actual process of review can vary significantly depending on several factors. One factor is the affiliation of the researcher and where the study is being conducted. If a HOF researcher is affiliated with a University but doing research at a healthcare entity such as a hospital, ethics approval will be required from both the healthcare center (e.g. hospital) and the academic institution. The usual protocol is to first obtain ethics approval from the hospital and then submit for REB approval to the University where an expedited review of the ethics approval is done. Universities may sign a formal agreement with certain hospitals to better facilitate this two-stage application process. However, this two-stage process can lead to problems. Disputes can occur where ethics approval and conditions of it (e.g. how to recruit patients or conditions of a consent form) were approved by a hospital board but the University board requests changes to the protocol that contradicts what the hospital approval has described. Another factor is the healthcare context where the study is taking place. While hospital ethics boards should be knowledgeable of HOF studies, situations can arise where a healthcare region may have an ethics board that is not familiar with HOF studies and thus may have a hard time understanding the proposal or may be overly critical of the methods or approaches.
To make ethics approval more stream-lined some Canadian provinces have a provincial ethics approval process that is a collaboration between universities and research sites such as hospitals or health authorities. An example of such a system is in British Columbia2. This system enables one streamlined ethics submission and can prevent conflicting reviews between boards. An overall challenge is that HOF studies are not well known across many Canadian healthcare facilities and universities, which can introduce problems during the review process. To overcome this issue, we need better promotion of HOF methods and approaches.
3.4 Finland (J. Kaipio, M. Tyllinen) We have experience on two different types of studies involving ethical board review in Finland: health and social care workers’ experiences on their IT systems at a nation-al level and patients’ experiences on their illness, care, and related digital services at a local level. These studies have different ethical review processes.
In Finland, HOF studies fall under the ethical board review process if the require-ments set by the Finnish Advisory Board on Research Integrity are met. These include e.g., no informed consent of subjects, sub-jects under 15-year-old without parental consent, subjects exposed to security risks, or an intervention with impact on physical integrity. These ethical boards exist at the universities and in the Finnish national in-stitute for health and welfare (THL). When hospitals are involved or when the research project comes under the Medical Research Act (national law), hospitals district’s ethical boards are responsible for the review.
At Aalto university, the ethical commit-tee is responsible for the ethical evaluation of the university‘s non-medical research projects in human sciences. Review is re-quired in specific research configurations, but it can also be requested if the study’s publication forum, financier, or internation-al partner requests it. The hospital district is the owner and supervisor of any research that studies their patients or in which
2 https://researchethicsbc.ca/
members of their staff or data systems are utilized, or that is funded by research funds allocated by the district. In all cases, submissions are done via electronic systems and committees meet every month. At least, the following information is required when applying for an ethical review: a research plan including plan for conducting the re-search, the list of the persons taking part in the research, and the privacy notice of the research for participants.
In different fields of science and their related universities, ethical review boards have varying history and tradition, e.g., medical research has a long-established history compared to technical sciences. On the other hand, the recent changes in legis-lation, particularly the European General Data Protection Regulation (GDPR), have complicated the study planning in general, especially when best practices are still lack-ing. It seems that the organizations do not share a common understanding on what is regarded as personal data.
Based on our experience, the advantages of the ethical review process are: organiza-tions have their own review boards, which are easy to reach, and processes are relatively quick. The process supports the planning of the details of the study including the privacy aspects of data storing and sharing. However, there are few concrete examples of how to apply these considerations in HOF studies, which makes the process cumbersome, bu-reaucratic, and lengthy. It is also difficult for researchers to know in which circumstances an ethical board should be contacted and when it is the case, which one (especially when healthcare organizations are involved in the study as subjects).
In Finland, the law considers medical research and not research ethics in general. Clearer national guidelines would be needed on whether HOF studies should have an ap-proval from the ethical board or not.
3.5 France (S. Pelayo) The Jardé law is a French law that governs „research organized and carried out on volunteers (either healthy or sick) with the objective of developing biological or med-ical knowledge“ [11]. This law defines the categories of research and the functioning of
https://researchethicsbc.ca/
-
62
IMIA Yearbook of Medical Informatics 2020
Peute et al.
the network of national ethical boards called the committees of persons’ protection (CPP). Three categories of research are defined based on the level of risk to the subjects:1. Interventional studies where the interven-
tion in treatment decisions and treatments differs from usual care;
2. Interventional studies presenting minimal risks and constraints when no pharma-ceutical products are involved or only under their usual conditions of use. These studies are listed in a decision [12];
3. Observational studies when there is no intervention in treatment decisions. The researcher observes treatment and results in a systematic manner without changing, influencing, or interfering with diagnosis, treatment, or monitoring.
A decree has been published that defines the kinds of studies covered by the Jardé law [13]. “Experiments in human and social sciences in the field of health” are exclud-ed. Consequently, it seems HOF studies do not fall within the scope of the Jardé law [14]. Yet, it is generally admitted that HOF research that might put the subjects at physical or psychological risk (e.g. ques-tionnaire about suicidal ideations, stressful simulation) should require an ethical ap-proval because they may be categorized in category 2 or 3. However, criteria to know whether a HOF study has to be submitted to a CPP are still subject to interpretation and discussion among researchers, institutions, and ethics boards. Submission of a protocol to a CPP is done through the platform of the French Healthcare Delivery Authority. This web platform randomly assigns submitted protocols to a regional CPP for appraisal. Re-view times range from 45 days for research categories 2 and 3 to 60 days for category 1. CPPs base their decision on the condi-tions under which the researcher ensures the protection of persons, on the merits and relevance of the research project, and on its methodological quality.
For HOF studies not covered by the Jardé law (the large majority), there is no alternative regulation. Yet, if the researcher is affiliated to an institution (e.g., a university) that has its own local ethics board, s/he might need to get its approval. Those local boards are neither standardized in form nor in proce-
dure and have no statutory recognition (only CPPs are recognized); they “only” provide advices to their staff and support them in identifying ethical issues in their research projects and in thinking about their practices.
The Jardé law is a relevant initiative to frame research approval by proposing a common regulation and a one-stop-shop rec-ognized interlocutor. Like any new system, it takes time to operate properly. The Jardé law and related decrees were written for tradi-tional medical research and did not apply to every research case. Now, researchers need to be informed and trained to the Jardé law and related decisions.
The Jardé law also broadened CPP scope: the number of protocols to appraise has grown, increasing the response time and delaying research projects’ implementation. The large majority of CPP members (physi-cians, methodologists, pharmacists, lawyers, patients’ representatives, psychologists) are not informed about HOF studies and meth-ods. This makes it difficult for them to make standardized and unbiased appraisals. It is also difficult for applicant HOF researchers to write “understandable” protocols. For the time being, practices in the CPP network are still quite heterogeneous with decisions not always harmonized.
3.6 Norway (E. C. Lehnbom, R. Pedersen)Medical and health research conducted in Norway needs to be approved by an ethics committee if the aim is to trial new experi-mental treatments, acquire new knowledge about health and disease, or if human biological samples or identifiable personal information (either collected by research-ers or obtained from one or several central health registers) is to be used.3 Testing and evaluation of medical devices, defined as instruments or apparatus, produced to be used on people to diagnose, prevent, monitor, treat, or relieve disease, also require ethics approval. Research projects to evaluate how a new technology is being used (direct ob-servations), to explore opinions (interviews) about new technologies, and quality assur-
3 REK Regional Committees for Medical and Health Research Ethics. Available at: https://helseforskning.etikkom.no. Accessed 18 Oct. 2019.
ance projects do not require ethics approval. Before submitting a complete ethics appli-cation using the online portal, researchers can submit a request for assessment by the ethics committee. The request for assessment should contain information about the study aim, methods, analysis plan, requirement strategy, and data storage, and is also submit-ted using the online portal. The committee then assesses the project and decides whether a full review is necessary or not.
In addition to ethics assessment or approval, researchers also need to report projects to the regional PVO (Patient Protection Agent), which in turn inform the Data Protection Services (NSD)4 prior to commencing a research project if any personal information (such as a name on a consent form) is to be collected. There are no fees associated with the application for assessment or full review by the ethics committee, PVO, or NSD.
Submission processes (ethics and data protection) are straightforward. The review process for ethics assessment is relatively quick, and usually done within one month. In our opinion, it is useful that an external com-mittee reviews research projects and plans for data storage to minimize the risk of harm and breaches in privacy. It is important that the process is thorough yet quick, to achieve high compliance and to avoid unnecessary delays in starting research projects.
3.7 Portugal (R. Santos, F. Leite, N. A. Silva)In Portugal, all clinical research is regulat-ed by Law No. 21/2014. Any systematic study designed to discover or verify the distribution or the effect of health factors, health status or outcomes, health and dis-ease processes, or the performance and / or safety of interventions or services provided, is considered as clinical research. In the context of clinical research there are two main branches of work, clinical trials and clinical studies. Clinical studies may further be subdivided into studies with or without intervention. A study is considered without intervention when clinical practice is not altered by the study.
4 NSD Norwegian Centre for Research Data. Avai-lable at: https://nsd.no/. Accessed 18 October 2019.
https://nsd.no/
-
IMIA Yearbook of Medical Informatics 2020
63
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies
All clinical research studies must be ap-proved by the Competent Ethics Committee (CEC) which is responsible for ensuring the protection of patients’ rights, safety, and well-being for all patients involved in the research. Depending on the type of study, the CEC may be the nationwide Ethics Committee for Clinical Research (CEIC) for clinical trials and interventional clinical studies, or the local, hospital-specific, Ethics Committee for Health (CES) for clinical studies without intervention.
HOF studies generally fall under the classification of clinical studies without intervention, although the Law makes no special reference to these studies. The private group Luz Saúde, owner of the Hospital da Luz network where we operate, has a group-wide CES and a group-wide Clinical Research Commission (CIC), supporting its 30 hospitals and clinics. The CIC assesses the clinical relevance and scientific quality of submitted clinical studies. HOF studies are evaluated by the CIC whenever the objective of the study is within the definition of clinical research. Additionally, studies should always be evaluated by the CES, even if the study is not considered to be clinical research, unless patient data is not used.
If patients are involved in a study, informed consent may also be required, and its clarity and completeness is evaluated by the CES. If personal data of participants based on the GDPR definition are also required, a Data Pro-tection Impact Assessment (DPIA) should be performed. The responsibility for the devel-opment of DPIA relies on the study sponsor. Hospital da Luz Learning Health (HLLH), a company within the Luz Saúde group re-sponsible for all training and research, and the local Data Protection Officer (DPO) may assist the sponsor on the development of the DPIA, and will ultimately validate (HLLH) and approve (DPO) the DPIA.
The duration of these local assessments may vary. As a reference, in the Luz Saúde group, the CIC assessment will last two to four weeks, and the CES assessment will occur within a similar time frame.
If the scope of the study is by no means clinical and does not involve patient data, there is usually no need to involve the CIC or the CES, and what is required is the evalu-ation by the HLLH and final approval by the Luz Saúde Executive Committee.
In Portugal, HOF studies that are part of the improvement of the socio-technical system that supports all care delivery, and specifically HOF studies related to medical technology, are still infrequent and unknown, which may initially hinder the evaluation process. Nevertheless, the increasing sub-mission of HOF studies will improve knowl-edge and awareness of these types of studies and will help in developing the discipline. It is our belief that the increasing visibility of these studies, also at the level of Ethics Committees, will not only raise the quality requirements for these studies to a level similar to other disciplines, but will also help to create evidence of their usefulness and positive impact.
3.8 Sweden (M. Hägglund) In Sweden, there is only one ethical review process that is applied to all research that in-volves human subjects. The process is guided by the law on ethical review for research involving human subjects (2003:4605). This legislation applies to all research that involves physical interventions, methods that aim to affect the research participants, involves biological materials that can be traced back to a person, but also to research that includes handling of sensitive personal information (as defined in the European GDPR §9.1). Does this mean that HOF studies need ethical approval? Well, that would depend from case to case on whether or not they would handle sensitive personal information (e.g., on an individual’s health) or whether they may aim to somehow affect the subject physically or psychologically.
In general, this means that a study of healthcare professional’s use of IT, organiza-tional studies and usability studies involving healthcare professionals do not require ethical approval according to the current Swedish legislation. If patients are involved, we howev-er need to consider whether any health-related data would need to be gathered; if so – ethical approval would be required. This could be the case even if you only test an application with
5 https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460
mock data and you document information about the study participant’s current or past health issues, or their contact with health-care. Therefore, in most health informatics research, ethical approval is sought whenever patients will be involved in the study, as sen-sitive personal data regarding their health will likely be documented.
All ethical reviews are handled by the Swedish Ethical Review Authority6, which started on the first of January 2019. The work is still divided between six different regions (Göteborg, Linköping, Lund, Umeå, Uppsala and Stockholm). Each regional office has at least one division for reviewing medical research and one division for other research. Each division has 10 representatives with scientific background and five representa-tives of the public. The chair of a division should be or has been a judge.
When you apply for ethical approval, you send your application to the Ethical Review Authority, and it will be assigned to the region and division they consider most appropriate. The application is therefore very similar for medical research as it is for other types of research. An application for ethical review costs 5000 SEK (515 USD), or 16000 SEK (1648 USD) if more than one research institution is involved. If at a later point, an ethical approval needs to be changed (e.g. addition of a new study site, or inclusion of more study participants) a new assessment will be made for a cost of 2000 SEK (206 USD). An ethical application will usually be handled within 60 days of submission, and the applicant will have the results within two weeks of the decision.
The ethical review process in Sweden clearly takes its starting point in medical research. It is often unclear what aspects of a study focusing e.g., on the usability of an app or medical device will need ethical approval, and there exists very little guidance for researchers within this field. Therefore, we often end up applying for ethics review just to be on the safe side and are often told that ethical approval is not required for this study. Considering the costs and extensive documentation required to apply for ethical approval, it sometimes creates barriers for
6 Etikprövningsmyndigheten: https:/ /et ik-provningsmyndigheten.se
https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/lag-2003460-om-etikprovning-av-forskning-som_sfs-2003-460https://etikprovningsmyndigheten.sehttps://etikprovningsmyndigheten.se
-
64
IMIA Yearbook of Medical Informatics 2020
Peute et al.
important research that needs to be done. An expedited application process for projects that will not expose study participants to great risk would be helpful, or a pre-application process to help determine whether a project will be exempt from applying for ethical approval.
The application process is also poorly designed to meet requirements of more design-oriented research, where formative evaluations, scoping interviews, and focus groups or workshops are needed. Especially as we see an increased interest in patient par-ticipation, patients as research partners, and patient-driven research, we need to consider how these types of research formats will affect the ethical approval process. Ethical reviews will continue to be just as important but may require a different format and an understanding of these types of research in the ethical review divisions. Finally, how we guide researchers in interdisciplinary research projects that involve both needs analysis, design, formative evaluations, and clinical testing is still not clear.
3.9 The Netherlands (L. van Velsen, S. Jansen-Kosterink) In the Netherlands, ethical considerations re-garding medical research involving humans and the procedures for requesting permission to conduct these studies, are described in the Dutch Medical Research Involving Human Subjects Act (WMO)7. Research is subject to the WMO if the following criteria are met: (1) it concerns medical scientific research and (2) participants are subject to procedures or are required to follow rules of behavior. If a study is subject to the WMO, it must undergo a review by an accredited Medical Research Ethics Committee (MREC). These MRECs are typically hosted by large, aca-demic hospitals.
HOF studies are, normally, not subject to the WMO if people voluntarily participate and if the study does not infringe upon the physical and / or psychological integrity of participants. This is often the case for stud-ies such as usability tests of a new eHealth application or focus groups in which the
7 https://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-not
added value of a technology is discussed. To be sure that our HOF studies do not fall under the scope of the WMO, the authors normally ask an accredited MREC to check this assumption. If so, the MREC provides an official letter, stating that the study is not subject to the WMO. This check requires to submit the study protocol, the information letter for the subjects, and the informed consent form. After submission, this official letter is received within a week. In any case, each study will be conducted according to the principles of the Declaration of Helsinki: all subjects must provide informed consent before participation, and their data will be stored and analyzed anonymously.
If, for some reason, a HOF study does fall within the WMO, a full application for conducting the study must be sent to an MREC. For general (medical) research, that is, no research with a medicinal product, a so-called “reasonable timeline” of eight weeks applies. This means that the MREC has a maximum of eight weeks to reach a decision, unless the MREC gives notice that more time is needed, including an alternative timeline. Conducting a study that falls under the WMO without a positive decision from an accredited MREC is an offence.
The Netherlands has strict rules for re-search with human subjects and this makes it clear for researchers how to handle ethical review. Furthermore, the review of your medical or HOF study by an accredited MREC is very valuable. It makes one reflect on the benefit of your study against patient burden, so that the study protocol can be improved in order to comply with ethical principles. However, MRECs mainly review general medical research and have less ex-perience with HOF studies. In some cases, this make it difficult for them to review these studies properly. The letter of exception that is often granted for HOF studies, makes it easy for researchers to publish their work, as it serves as an official statement about a study’s ethical compliance.
3.10 United Kingdom (V. Lichtner) I have been doing research on implemen-tation and use of IT in clinical settings in England (UK) for over ten years, such as evaluations of IT implementations in doc-
tors’ clinics in the community and hospitals in the National Health Service (NHS). This HOF research was conducted with quali-tative studies aimed at understanding how technology responds to clinicians’ informa-tion needs in local sociotechnical contexts.
Any research conducted in the NHS requires approval by an NHS Research Ethics Committees (REC) and approval by local research governance (RG) bodies (e.g. hospital research units). Over the years, applications to NHS REC in England have been streamlined, leading to a centralized online submission through an Integrated Research Application System (IRAS), man-aged by the NHS Health Research Authority (HRA)8. HOF studies are usually classified as ‘low risk’ and a simplified application form is generated by the system, compared to the one suitable for clinical trials where risks of intervention to participants may be considerable. At the time of submission, IRAS suggests ‘the first available slot’ of a REC – anywhere in England - suitable for the type of study submitted. Researchers are not bound to choose ‘the nearest REC’; choosing ‘the first available slot’ may speed up the approval process.
In the context of the studies I conduct-ed, obtaining local RG approval has been much more challenging. The process was often not standardized. Some RG bodies required contracts in place between the University and the NHS organization where the research was taking place. It often involved communication with staff locally to track the status of the application, and what would be needed to move it forward, but identifying the person to speak to was difficult. Receiving RG approval at the level of the hospital, or health authority, was also not sufficient to guarantee access to single clinical wards or GP clinics – each had to be negotiated separately. This made multi-site studies incredibly difficult and time consuming to set-up, hindering research. For example, in the case of the evaluation of the electronic transmission of prescriptions in primary care clinics, we had to apply to different local authorities where the clinics were located, each taking months to re-
8 https://www.hra.nhs.uk/approvals-amendments/what-approvals-do-i-need/
https://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-nothttps://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-nothttps://english.ccmo.nl/investigators/legal-framework-for-medical-scientific-research/your-research-is-it-subject-to-the-wmo-or-not
-
IMIA Yearbook of Medical Informatics 2020
65
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies
spond. While the roll-out of the technology across England followed an agile approach, we were unable to be present on the sites at the same speed to observe activities [15].
The HRA has recently centralized the local RG applications, as part of the appli-cation to REC9. However, the researcher is still responsible for updating local sites of progress. Guidance provided on how to fill in the HRA application to satisfy local approval feels overwhelming10. There is also varia-tion in how HRA administrators interpret the GDPR – anecdotally, some are placing unrealistic burdens of patient consent on researchers in clinical areas where patient information is otherwise displayed publicly on whiteboards.
A further stumbling block is that the REC-approved study protocol may place unexpected barriers on the research as it unfolds in practice. HOF qualitative research needs to be ‘open to the unexpected’. For example, understanding technology for medication in a clinical ward may take the researcher to also investigate dispensing activities in the hospital pharmacy. If this is not foreseen in the initial application, the researcher must stop the study and submit an amendment to REC, losing valuable time and perhaps valuable opportunities for gaining insight into current work practices.
3.11 United States (B. Lesselroth, J. Homco) Our team has nearly 20 years of collective experience working on informatics projects at various US institutions. We have partici-pated on intercollegiate, governmental, and industry projects, affording us the opportu-nity to work with regulatory oversight bodies at the federal, state, and private levels. Our projects tend to be operational, addressing quality improvement (QI), HIT implemen-tations, or user experience (UX) evaluation.
Christine Grady wrote that medical research and quality improvement occupy a continuum ranging from passive obser-vation to controlled experimentation [16]. The distinctions can be murky when con-
9 https://www.hra.nhs.uk/approvals-amendments/what-approvals-do-i-need/hra-approval/
10 https://www.myresearchproject.org.uk/help/hlpsitespecific.aspx#UK-Local-Information-Pack-OID
ducting HOF studies – particularly when evaluating users interacting with HIT. This fact notwithstanding, we believe infor-maticians have an obligation to evaluate HFO impact with objectivity. This requires rigorous standards for data handling and independent review to protect patient and staff safety. Therefore, we self-govern by writing protocols for every project, ensur-ing a high level of methodologic rigor and a consistent process for data handling. As per institutional expectations, we solicit a review from the IRB (either a full review or an expedited process, as described below). Furthermore, we notify our IRB office of any projects involving human subjects that will likely result in publication of findings that may contribute to a cumulative knowl-edge base.
The University of Oklahoma rules and regulations governing research operation-alize at a local level federal regulation enforced by the US Department of Health and Human Services, the US Food and Drug Administration, the US Department of Veterans Affairs, and the US Department of Defense. The University also upholds the Federal Health Insurance and Portability and Accountability Act (HIPAA) for the protection of patient health information and the Oklahoma state laws pertaining to protections of vulnerable populations including emancipated minors and cogni-tively impaired individuals. Criteria used for defining and policing research are con-sistent with federally published standards and are similar between institutions (US Code of Federal Regulations 45FR.46.102; Bailey, Hastings Center Report, 2006).
We submit all projects – including HOF work – to our institutional IRB well in ad-vance of any data collection. Many IRBs, including at the University of Oklahoma, offer an expedited pathway for quality im-provement efforts, educational programs, and non-experimental technologies. Typi-cally, these studies do not include participant consent forms, different exposures, random-ization, or a control group. The expedited process permits IRB members to pre-screen projects and allow low-risk ones to bypass the normative IRB review and governance system. The process begins with completion of a “determination of research” checklist
and provision of a project description that in-cludes methods, types of data to be collected, and anticipated deliverables. The checklist includes criteria to help reviewers estimate risk and disambiguate quality improvement activities – including HOF work – from traditional research. At a high level, research tests hypotheses to develop generalizable knowledge, whereas quality improvement and HOF apply generalizable knowledge to “quickly” improve health delivery. The IRB at the University of Oklahoma School of Community Medicine typically responds to an application within two to four weeks.
There are several important strengths and limitations of the current IRB processes. Strengths include: (1) a single intake path-way and administrative infrastructure; (2) a clear set of criteria and definitions that can inform protocol development; (3) a stream-lined set of monitors and documentation for QI and HOF work; and (4) the presence of a regulatory framework that protects stakeholders and promotes a high level of academic rigor.
Weaknesses include: (1) lack of specific guidance for evaluation of HIT including electronic health record configurations or modules; (2) ambiguity surrounding par-ticipation of clinicians or trainees when participation is mandatory as a component of care; (3) lack of administrative stake-holders with experience conducting QI or UX work (leads to difficulty understanding proposal, methods, scope, and impact); (4) the additional time required to determine the single IRB of record when multiple agencies are participating on a project; (5) an often lengthy and bureaucratic process (frequently impacted by personnel shortages); (6) a commonly held, but false dichotomy between “research” and HOF work; (7) organizational opposition to the use of “research language” in HOF and QI applications; and (8) the need for language defining “generalizability” of f indings (point of contention when we seek to pub-lish findings or lessons learned about the design, implementation, and evaluation). To this final point, we contend that QI and HOF works make important contributions to our theoretical frameworks and under-standing of QI methods and implementation effectiveness.
https://www.hra.nhs.uk/approvals-amendments/what-approvals-do-i-need/hra-approval/https://www.hra.nhs.uk/approvals-amendments/what-approvals-do-i-need/hra-approval/
-
66
IMIA Yearbook of Medical Informatics 2020
Peute et al.
4 Synthesis and DiscussionThis study explored the differences, chal-lenges, and best practices of HOF of medical informatics (HOFMI) research projects with respect to ethical board review processes in 11 countries. From the testimonies above, key findings emerge that indicate shared challenges that researchers face in ethical board review for HOFMI studies. Table 1 synthesizes these challenges and offers an overview of proposed solutions recom-mended by participating HOFMI researchers based on the needs and experiences within
Table 1 Synthesis of the challenges faced by HOFMI studies and some proposed solutions. Countries that specifically described challenges or proposed solutions are indicated between brackets. Note that not all countries that pointed to a challenge proposed a solution. Conversely, some countries proposed solutions to challenges that they did not face. There is no one-to-one correspondence between expressed challenges and proposed solutions; some challenges require the application of several solutions and some solutions correspond to several challenges.
their research. It must be stated that the themes presented in the table are not ex-haustive; some countries might experience the same concerns, but they might not have referred to this in their testimonies.
A major concern mentioned by HOF researchers relates to ambiguous ethics legis-lation and guidelines. From a legal standpoint, the need for protecting the safety and privacy of participants in research projects is specified in the national laws of the country where the research is carried out. The majority of the testimonies however describe that knowing whether HOF studies need IRB approval
remains open to debate because these laws do not specify how to interpret them when dealing with HOF research in healthcare. To ensure that research protocols consider rele-vant ethical dimensions, HOF researchers are prone to take the safe route and submit their research protocols to the IRBs for review. In a practical sense however, not all HOF studies benefit from full IRB review. For studies that are generally considered with ‘no’ or ‘low’ risk to participants, such as usability testing of an eHealth application with healthy and willing participants, a shortened and more efficient review process would be better
Challenges for HOFMI studies
Legislation about HOFMI and ethics
Legislation is unclear, undefined, and subject to interpretation; lack of clarity with respect to what projects need to be reviewed leads to a cumbersome and disproportionate reviewing process. [Argentina, Australia, Finland, France, Sweden, United States] Review processes are the same regardless of risk level: criteria to define whether a HOFMI study is of
low or high risk regarding ethical review are lacking. [Argentina, Australia, Sweden] Ethical review processes are medical/evaluation-oriented and are not adapted to (patient-centered) IT
design studies. Specificities of HOFMI research protocols do not fit the ‘known’ ethical review process. [Sweden, United Kingdom]
IRB reviewing process
Inconsistencies in the ethical process and jurisdictional differences between sites are barriers to the performance of (multisite) HOFMI research. [Australia, Canada, United Kingdom] Repetition of the ethical process across multiple sites is inefficient and makes it difficult for researchers
to work in line with the fast developments observed in the medical/technical field. [Australia, Canada, United Kingdom, United States]
Specific issues
Application of GDPR in relation with HOFMI studies encounters difficulties in interpretation and relevance. [Finland, United Kingdom] Cost and extensive documentation represent a barrier. [Sweden]
Awareness of HOF studies
Lack of awareness and lack of knowledge about HOF studies by IRB members lead to problems and delays in the reviewing process. [Canada, France, Portugal, The Netherlands, United States] HOF researchers, specifically from the technical sciences, may be unaware of IRB approval. Little
guidance is provided. [France, Sweden, United Kingdom, United States]
Proposed solutions
Legislation to specify at a national level if HOF studies need or not approval from ethical boards. [Finland] Regulate and monitor HOFMI studies in a way more specific/suitable to these
studies. [Argentina] Elaborate principles to define the level of risk of HOF research (high risk/low risk). Define an expedited ethical review process for low risk studies, e.g., through a
pre-application process. [Argentina, Sweden]
Umbrella’ partnership agreements between universities and healthcare centers to facilitate local approval processes. [Canada, United Kingdom] A low effort, streamlined review process. [Australia]
A standardized approach and guidance on how the GDPR applies to different types of studies [United Kingdom], supported by an external committee reviewing research projects to minimize the risk of privacy breaches. [Norway] Review process free of charge. [Norway]
Better promotion of HOF methods and approaches. [Canada, France, Portugal, The Netherlands, United States] Increase HOF knowledge for ethical review board members. [France] Provide concrete examples of applying ethical considerations in HOF studies.
[Finland, France, United States] Develop guidelines for HOF researchers on how and when to apply for ethical
review. [Australia, France Norway Sweden, United States]
-
IMIA Yearbook of Medical Informatics 2020
67
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies
suited. HOF researchers from Argentina, Norway, The Netherlands and the United States have a pre-application ethical review process available in their country. This may consist of pre-screening to determine if the research protocol needs approval, which may offer researchers an exemption from ethical review. Publishing journals require a study to comment on their ethical compliance; a letter of IRB review exemption would provide researchers with a way to respond to journals’ requirements. Furthermore, in determining the risks associated with a study, an expedited review process may help bypass the extended IRB review that is needed for higher risk studies. Such an expedited process may enable HOF research to better align to the quick technical developments and tech-nology implementation processes that are the subject of the study. Having a separate shortened ethics review process would not only facilitate HOF researchers, but it would also lower IRB administrative burden. On the basis of these testimonies, we propose in Figure 1 a best practice IRB ethics review process for HOF studies which may serve as a model to improve IRB processes.
Australia however, has revised their na-tional ethics form in 2019 and instead of an expedited process now a single application form is used for all projects, regardless of risk level. This development indicates the need to define clear international criteria when full IRB review is indeed needed and
when not. A private initiative such as from a medical product development company may provide a first approach to determine the risk level of HOF studies, but this needs further scientific foundation [17].
A second main concern relates to perceived inefficiency and inconsistency in the ethics reviewing processes due to variability in ethics committees’ processes between institutional and local IRBs. Often, a two-step process in which either a national and local level board review or a university and hospital agreement is required to guarantee access to single re-search sites. A HOF researcher hence needs to liaise with local sites and is in constant need of ‘getting people on board’ while explaining the relevance and benefits of the study. This is time consuming for HOF researchers and may pose a significant hindrance to multi-side studies. To tackle this, umbrella partnerships are proposed to quicken and streamline these processes. In addition, a smaller but still important barrier is mentioned with regard to the experienced uncertainty around the interpretation and use of the EU GDPR [18]. When research organizations do not share a common understanding on what is regarded as personal data, this complicates the study planning. More guidance is therefore also needed to adequately interpret the GDPR for different types of studies.
A final central theme in the majority of tes-timonies is the unawareness of IRB members with the goals and standard practices of HOF
research. IRBs are from a historical perspec-tive more familiar with clinical trials and other quantitative study designs involving human subject research and experience variability in reviewing outcomes of protocols in other types of research areas [19]. HOF research is characterized by often smaller scale, less-strict and more qualitative research designs that need high flexibility and for which standard (bio)medical study designs are often unsuitable [20]. As a result, the reviewing time of the board may be delayed until the study approach and design can be appraised appro-priately. Reviews may be then also subject to variation depending on the expertise of the board members. Other concerns with regard to IRB review processes regard the necessity for a systemic improvement that needs to in-clude better standardization of review practic-es, enhanced training for IRB members, and accreditation of review boards [21]. With the continual rise of technical innovations imple-mented in healthcare and associated studies, the need for IRB members to receive adequate training in reviewing these types of studies is becoming imperative. Visa versa, studies in the medical technology industry also need to become more aware of the need for ethical approval and the procedures involved.
This study has some specific strengths and limitations. Our unstructured survey approach allowed contributing researchers to highlight issues they considered most relevant when facing current IRB processes
Fig. 1 Schematic representation of a proposed two-step review process. On the left, the pre-application process would inform the researcher whether her/his study must be submitted to an IRB and what is the level of risk for study participants. A letter of exemption is given to the applicant if his/her study does not have to undergo a complete review process. On the right, the researcher submits the full version of the protocol to the IRB for an expedited process (in case of low risk studies) or for a regular process (in case of high risk studies).
-
68
IMIA Yearbook of Medical Informatics 2020
Peute et al.
in their country. HOF research practices however differed in organization, e.g., from a national level to a more local level, which limited the comparability of the contribu-tions. It was however not our intention to provide an exhaustive view on (dis)similar-ities but rather to take a first step in obtain-ing insight on what impedes or facilitates good IRB practices for HOF studies across countries. Topics reported were found to be largely similar or equivalent, highlighting the generic nature of issues faced, and indicating the high validity of the synthesis.
5 ConclusionOverall, it may be stated that not all HOF research needs ethical approval and that going through a full IRB review process may result in inconvenient consequences in performing the study. For that reason, HOF research needs more guidance and greater clarity in which research protocols need IRB approval to demonstrate their compliance to AMIA’s code of professional and ethical conduct. For this, our HOF community will start with gaining consensus on criteria that define low and high-risk HOF studies at an international level. Best practices as described within this paper substantiate the proposed pre-application and expedit-ed IRB review process for ethical review of low risk HOF studies. Enhancing IRB processes in such a way would not only be beneficial to HOF studies but would also lower the administrative pressure on IRBs. In addition, by providing an overview of relevant evidence-based HOF study designs and better promoting these, knowledge and awareness for HOF studies will increase which in turn will help in developing the dis-cipline. This overview may also inform the development of a (online) training program for IRB members on HOF study approaches and methodologies, such as qualitative and mixed methods studies, iterative user design research of health technology combined with, for example, design thinking, rapid engineering, and user testing approaches. Such guidance for IRBs is needed in this age of innovative health technology so that the correct ethics dimensions are considered.
Acknowledgement and EndorsementThis paper was initiated during the discussions about a panel proposal by the Human and Organization Factors of Medical Informatics (HOFMI) working group of the European Federation for Medical Informatics (EFMI) to MIE 2020. This paper is endorsed by IMIA’s Human Factors Engineering (HFE) working groups and by EFMI’s HOFMI working group.
References1. World Medical Association. WMA - The World
Medical Association-WMA Declaration of Hel-sinki – Ethical Principles for Medical Research Involving Human Subjects [Internet]. [cited 2019 Aug 5]. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethi-cal-principles-for-medical-research-involving-hu-man-subjects/
2. American Psychology Association. Ethical Principles of Psychologists and Code of Conduct [Internet]. https://www.apa.org. [cited 2019 Aug 5]. Available from: https://www.apa.org/ethics/code/index
3. Petersen C, Berner ES, Embi PJ, Fultz Hollis K, Goodman KW, Koppel R, et al. AMIA’s code of professional and ethical conduct 2018. J Am Med Inform Assoc 2018;25:1579–82.
4. International Journal of Medical Informatics. International Journal of Medical Informatics - Author Information Pack [Internet]. Elsevier - Int J Med Inform [cited 2019 Aug 5]. Available from: https://www.elsevier.com/journals/internation-al-journal-of-medical-informatics/1386-5056/guide-for-authors#5000
5. Duplancic C, Crough T, Bell SC, for the Australian Non-tuberculous Mycobacteria in Cystic Fibrosis Study Group, Thomson R, Wainwright C, et al. Multi-centre ethics and research governance review can impede non-interventional clinical research. Intern Med J 2019;49:722–8.
6. Baysari MT, Westbrook JI, Richardson KL, Day RO. The influence of computerized decision support on prescribing during ward-rounds: are the decision-makers targeted? J Am Med Inform Assoc 2011;18:754–9.
7. Baysari MT, Westbrook JI, Richardson K, Day RO. Optimising computerised alerts within electronic medication management systems: A synthesis of four years of research. Stud Health Technol Inform 2014;204:1–6.
8. Jaensch SL, Baysari MT, Day RO, Westbrook JI. Junior doctors’ prescribing work after-hours and the impact of computerized decision support. Int J Med Inform 2013;82:980–6.
9. Santucci W, Day RO, Baysari MT. Evaluation of Hospital-Wide Computerised Decision Support in an Intensive Care Unit: An Observational Study. Anaesth Intensive Care 2016;44:507–12.
10. The National Health and Medical Research Council the Australian Research Council and Uni-versities Australia. National Statement on Ethical
Conduct in Human Research 2007 (Updated 2018). Commonwealth of Australia, Canberra; 2018.
11. Code de la Santé Publique, Livre Ier, Titre II : Re-cherches impliquant la personne humaine. Loi Jardé n° 2012-300 du 5 mars 2012 modifiée par ordonnance n° 2016-800 du 16 juin 2016, décret d’application modificatif n° 2017-884 du 9 mai 2017 et textes subséquents incluant les arrêtés du 12 avril 2018.
12. Arrêté du 12 avril 2018 fixant la liste des recherch-es mentionnées au 2° de l’article L. 1121-1 du code de la santé publique.
13. Décret n° 2017-884 du 9 mai 2017 modifiant certaines dispositions réglementaires relatives aux recherches impliquant la personne humaine.
14. Toulouse E, Masseguin C, Lafont B, McGurk G, Harbonn A, Roberts J, et al. French legal approach to clinical research. Anaesth Crit Care Pain Med 2018;37:607–14.
15. Cornford T, Hibberd R, Barber N. The evaluation of the electronic prescription service in primary care: final report on the findings from the evaluation in early implementer sites. London, UK: University College London; 2014.
16. Grady C. Quality improvement and ethical over-sight. Ann Intern Med 2007;146:680–1.
17. Shades of Gray in Human Research: 3 Steps to Determine if Human Factors & Usability Studies Need IRB Review [Internet]. Ximedica Your Med. Prod. Device Dev. Partn. 2013 [cited 2019 Dec 1]. Available from: https://www.ximedica.com/shades-of-gray-in-human-research-3-steps-to-determine-if-human-factors-usability-studies-need-irb-review/
18. The general data protection regulation [Internet]. [cited 2019 Dec 1]. Available from: http://www.consilium.europa.eu/fr/policies/data-protection-re-form/data-protection-regulation/
19. Linden JA, Schneider JI, Cotter A, Drexel S, Frosch E, Martin ND, et al. Variability in Institu-tional Board Review for a Multisite Assessment of Resident Professionalism. J Empir Res Hum Res Ethics 2019;14:117–25.
20. Marcilly R, Peute L, Beuscart-Zephir M-C. From Usability Engineering to Evidence-based Us-ability in Health IT. Stud Health Technol Inform 2016;222:126–38.
21. Page SA, Nyeboer J. Improving the process of research ethics review. Res Integr Peer Rev 2017;2:14.
Correspondence to: Linda W. Peute Centre for Human Factor Engineering of Health Information Technology Amsterdam UMCUniversity of AmsterdamDepartment of Medical InformaticsAmsterdam, The NetherlandsE-mail: [email protected]
Romaric Marcilly Univ. LilleCHU LilleULR 2694 - METRICS: Évaluation des technologies de santé et des pratiques médicales, INSERM-CIC-IT 1403/EvalabF-59000 Lille, FranceE-mail: [email protected]
-
IMIA Yearbook of Medical Informatics 2020
69
Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of TestimoniesAp
pend
ix: O
vervi
ew of
the O
rgan
izatio
n of t
he IR
B Re
view
Proc
ess,
the P
lannin
g of t
he P
roce
ss an
d the
Chall
enge
s Fa
ced C
ount
ry pe
r Cou
ntry
Not
e: G
DPR
= G
ener
al D
ata
Prot
ectio
n R
egul
atio
n; H
OF
= hu
man
and
org
aniz
atio
nal f
acto
rs; H
RA
= H
ealth
Res
earc
h A
utho
rity;
HR
EA =
hum
an re
sear
ch e
thic
s app
licat
ion;
HR
EC =
hum
an
rese
arch
eth
ics
com
mitt
ee; I
RB
= In
stitu
tiona
l Rev
iew
Boa
rds
(als
o kn
own
as, f
or e
xam
ple,
inst
itutio
nal r
esea
rch
boar
ds, e
thic
al a
ppra
isal
boa
rds,
com
mitt
ees
of p
erso
n’s
prot
ectio
n - C
PP,
ethi
cal r
evie
w a
utho
ritie
s, re
sear
ch e
thic
s com
mitt
ees,
com
pete
nt e
thic
s com
mitt
ee –
CEC
, med
ical
rese
arch
eth
ics c
omm
ittee
- M
REC
); N
HS
= N
atio
nal H
ealth
Ser
vice
s, th
e pu
blic
ly fu
nded
he
alth
care
syst
em in
Eng
land
(UK
); TC
P =
Tri-C
ounc
il Po
licy
Stat
emen
t;
Coun
try
Argen
tina
Austr
alia
Cana
da
Finlan
d
Franc
e
Orga
nizat
ion of
Ethic
al bo
ard r
eview
Any s
tudy t
hat in
volve
s hum
an pa
rticipa
nts re
quire
s ins
titutio
nal IR
B app
roval
at a l
ocal
level
prior
to its
ini
tiatio
n, an
d to d
elive
r follo
w-up
docu
menta
tion a
t leas
t onc
e a ye
ar un
til it f
inish
es.Th
e IRB
has a
lso an
expe
dited
proc
edure
: som
etime
s used
for H
FO st
udies
, sinc
e usu
ally t
here
is ha
rm to
pa
rticipa
nts.
Follo
wing
HRE
C app
roval,
site-
speci
fic go
verna
nce a
pprov
al mu
st als
o be o
btaine
d from
each
partic
ipatin
g site
. Mo
st HO
F rese
arche
s are
low ris
k rese
arche
s. Th
ey us
ed to
be pr
ocess
ed th
rough
an ex
pedit
ed pr
ocess
. Rece
ntly,
a nati
onal
HREA
form
was
intro
duced
, and
this
form
is no
w us
ed fo
r all p
roject
s, reg
ardles
s of ri
sk lev
el.
Acco
rding
to th
e TCP
S 2, H
OF st
udies
invo
lving
patie
nt rec
ruitm
ent fo
r obs
ervati
on or
evalu
ation
such
as a
usab
ility s
tudy w
ould
be cl
assif
ied as
mini
mal ri
sk rev
iew. M
inima
l risk
is typ
ically
perfo
rmed
by a
deleg
ated
review
whe
re a s
electe
d set
of eth
ical b
oard
memb
ers re
view
the ap
plica
tion.
Depe
nding
on th
e rese
arche
r’s af
filiati
on an
d on w
here
the st
udy i
s to b
e con
ducte
d, the
numb
er of
appro
vals
may
chan
ge. If
the H
OF re
search
er is
affilia
ted w
ith a
unive
rsity
but d
oing r
esearc
h at a
healt
hcare
entity
, ethi
cs ap
prova
l wi
ll be r
equir
ed fir
st fro
m ho
spital
(mini
mal ri
sk rev
iew) a
nd th
en fro
m the
unive
rsity
(expe
dited
revie
w).
HOF s
tudies
fall u
nder
the et
hical
board
revie
w pro
cess i
f the r
equir
emen
ts set
by th
e Finn
ish Ad
visory
Board
on
Resea
rch In
tegrity
are m
et (in
cl. no
infor
med c
onsen
t of s
ubjec
ts, su
bjects
unde
r 15 y
ear-o
ld wi
thout
paren
tal
cons
ent, s
ubjec
ts exp
osed
to se
curity
risks,
or an
inter
venti
on in
the p
hysic
al int
egrity
). Eth
ical b
oards
exist
at
the un
iversi
ties a
nd in
the F
innish
natio
nal in
stitut
e for
healt
h and
welf
are.
When
a ho
spita
l is in
volve
d or w
hen t
he re
search
proje
ct co
mes u
nder
the M
edica
l Rese
arch A
ct, th
e hos
pital
distric
t’s et
hical
board
is re
spon
sible
for th
e rev
iew. E
thica
l boa
rds w
ithin
these
organ
izatio
ns ar
e eas
y to r
each
.
Huma
n and
socia
l scie
nces
are ex
clude
d from
the s
cope
of th
e rese
arch e
thics
law. Y
et, it
is ad
mitte
d tha
t som
e HO
F prot
ocols
that
can p
ut the
subje
cts at
risk s
hould
be su
bmitte
d to C
ommi
ttees
of Pe
rson’s
Prote
ction
(CPP
s) for
appra
isal. T
heir d
ecisio
n is b
ased
on th
e con
dition
s und
er wh
ich th
e rese
arche
r ens
ures t
he pr
otecti
on of
pe
rsons
, on t
he m
erits
and r
eleva
nce o
f the r
esearc
h proj
ect, a
nd on
its m
ethod
ologic
al qu
ality.
Plan
ning
Bi-mo
nthly
meeti
ngs;
1-2 m
onths
revie
w pro
cess;
1-2 w
eeks
for
exped
ited r
eview
Month
ly me
eting
s of
comm
ittees
Minim
al ris
k rev
iew:
up to
6 we
eks.
Expe
dited
revie
w is
3 we
eks.
Month
ly me
eting
s; ha
ndled
with
in 30
da
ys of
subm
ission
(on
avera
ge)
Hand
led w
ithin
45
days
for ca
tegori
es 2
and 3
stud
ies
Curre
nt ch
allen
ges
• Le
gislat
ion is
lacki
ng
• HO
F stud
ies ar
ise di
fferen
t risk
s for
partic
ipants
than
pharm
acolo
gical
diagn
ostic
test
studie
s and
there
fore s
hould
be tr
eated
diffe
rently
• No
stan
dardi
zatio
n of re
quire
ments
for e
xped
ited r
eview
proc
ess
• Tw
o-ste
p proc
ess•
Incon
sisten
t proc
esses
acros
s site
s and
juris
dictio
ns•
Form
s and
proc
esses
no lo
nger
reflec
t risk
leve
l•
Onero
us ap
prova
l proc
esses
for re
search
ers to
gain
site a
ccess,
that
are no
t sh
ared a
cross
sites
• Cu
rrentl
y, the
ethic
s app
roval
proces
s is v
iewed
by m
any p
eople
as a
barrie
r to
comp
leting
resea
rch an
d can
act a
s a de
terren
t to co
nduc
ting m
ulti-s
ite
resea
rch w
ithin
publi
c and
priva
te he
alth o
rganiz
ation
s
• Tw
o-ste
p proc
ess (b
oth un
iversi
ty an
d hos
pital
agree
ment)
•
HOF s
tudies
are n
ot we
ll kno
wn ac
ross m
any C
anad
ian he
althc
are fa
cilitie
s an
d univ
ersitie
s, wh
ich ca
n intr
oduc
e prob
lems d
uring
the r
eview
proc
ess
• Me
dical
resea
rch ha
s a lo
ng-es
tablis
hed h
istory
comp
ared t
o rese
arch i
n tec
hnica
l scie
nces
• Re
cent c
hang
es in
legisl
ation
, part
icular
ly the
EU GD
PR, h
ave c
ompli
cated
the
stud
y plan
ning,
especi
ally w
hen b
est pr
actic
es are
still
lackin
g. (e.
g. res
earch
orga
nizati
ons d
o not
share
a co
mmon
unde
rstan
ding o
n wha
t is
regard
ed as
perso
nal d
ata)
• Th
e stud
y plan
ning p
rocess
is fe
lt cum
berso
me, b
ureau
cratic
and l
ength
y. It
is als
o diffi
cult f
or res
earch
ers to
know
whe
ther a
n ethi
cal b
oard
shou
ld be
co
ntacte
d and
if so
, whic
h one
(esp
eciall
y if h
ealth
care
organ
izatio
ns ar
e inv
olved
in th
e stud
y as s
ubjec
ts)
• CP
Ps do
not k
now
abou
t HOF
resea
rch an
d meth
ods w
hich m
ight le
ad to
bia
sed ap
praisa
l•
CPPs
’ deci
sions
are n
ot alw
ays h
armon
ized.
• Th
e crite
ria to
know
whe
ther a
HOF
proto
col is
conc
erned
by th
e Fren
ch
resea
rch et
hics l
aw ar
e sub
ject to
discu
ssion
and i
nterpr
etatio
n
-
70
IMIA Yearbook of Medical Informatics 2020
Peute et al.Cu
rrent
chall
enge
s
• No
obsta
cles m
entio
ned
• HO
F stud
ies of
med
ical te
chno
logy a
re sti
ll infr
eque
nt an
d unk
nown
, whic
h ma
y hind
er the
evalu
ation
proc
ess
• Pre
fer to
apply
to et
hics b
oard
to be
on th
e safe
side
• Co
st an
d exte
nsive
docu
menta
tion r
epres
ent a
barrie
r•
Proces
s poo
rly de
signe
d for
desig
n-ori
ented
resea
rch (e
.g. fo
rmati
ve ev
alua-
tions
, sco
ping i
ntervi
ews,
focus
grou
ps…
)
• MR
ECs m
ainly
review
gene
ral m
edica
l resea
rch an
d hav
e less
expe
rienc
e with
HO
F stud
ies. In
some
cases
, this
make
s it d
ifficu
lt for
them
to rev
iew th
ese
studie
s prop
erly
• Th
e rese
arche
r is re
spon
sible
for lia
ising
with
loca
l site
s, ma
king m
ulti-s
ite
studie
s tim
e con
sumi
ng to
set-u
p•
The a
pprov
al pro
cess i
s cum
berso
me fo
r (no
t prop
ortion
ate to
) sma
ll sco
ping
studie
s •
There
are v
ariati
ons i
n the
GDPR
inter
pretat
ion
• La
ck of
speci
fic gu
idanc
e for
evalu
ation
of he
alth i
nform
ation
tech
nolog
y •
Ambig
uity s
urrou
nding
inclu
sion o
f clin
ician
s, sta
ff, or
traine
es wh
en
partic
ipatio
n is m
anda
tory a
s a co
mpon
ent o
f care
•
Lack
of ad
minis
trativ
e and
regu
latory
perso
nnel
with
experi
ence
cond
uctin
g qu
ality
impro
veme
nt (Q
I) or
user
experi
ence
(UX)
work
• Ad
dition
al tim
e req
uired
to de
termi
ne th
e sing
le IRB
of re
cord
(i.e.,
lead
org
aniza
tion)
whe
n mult
iple a
genc
ies ar
e part
icipa
ting
• An
often
leng
thy an
d bure
aucra
tic pr
ocess
• Th
e nee
d for
langu
age d
efinin
g “ge
neral
izabil
ity” o
f find
ings
• A c
ommo
nly he
ld, bu
t false
dich
otomy
betw
een “
resea
rch” a
nd H
OF w
ork•
Orga
nizati
onal
oppo
sition
to th
e use
of “re
search
lang
uage
” in H
OF an
d QI
appli
catio
ns
Orga
nizat
ion of
Ethi
cal b
oard
revie
w
Resea
rch pr
ojects
to ev
aluate
how
a new
tech
nolog
y is b
eing u
sed, to
explo
re op
inion
s abo
ut ne
w tec
hnolo
gies,
and q
uality
assu
rance
projec
ts do
not r
equir
e ethi
cs ap
prova
l. Non
ethele
ss, a
reque
st for
asses
smen
t mus
t be
subm
itted t
o the
ethic
s com
mitte
e to k
now
wheth
er an
ethic
s app
roval
(full r
eview
) is n
eede
d or n
ot. In
ad
dition
, resea
rchers
also
need
to re
port
projec
ts to
the re
giona
l Pati
ent P
rotect
ion Ag
ent, w
hich i
n turn
infor
m the
Data
Prote
ction
Servi
ces pr
ior to
start
ing a
resea
rch pr
oject
if any
perso
nal in
forma
tion i
s to b
e coll
ected
.
Clinic
al res
earch
stud
ies m
ust b
e app
roved
by th
e Com
peten
t Ethi
cs Co
mmitte
e (CE
C). Th
e CEC
may
be na
tion-
wide
(for
clinic