final eho emr benefits report jan2013
TRANSCRIPT
Benefits Realization
Study
Medical Records in
Ontario
Final
eHealth Ontario
January 2013
Benefits Realization
Study for Electronic
Medical Records in
Ontario
Final Report to
eHealth Ontario
Benefits Realization
for Electronic
Medical Records in
Benefits Realization Study for EMRs in Ontario
PwC
Acknowledgements
This study would not have been possible without the support of Steering Committee members and several clinical
leaders who participated in this study and facilitated access to their clinics and colleagues. We thank them and their
teams for their commitment of time, knowledge and expertise. Their insights and first-hand experience in
pioneering and advancing the use of electronic medical records in primary care settings in Ontario provided a rich
study environment to better understand the current and potential benefits that can be realized through the use of
electronic medical records in Ontario.
Clinical Leaders Steering Committee Members
Dr. David Barber
Queen’s Family Health Team
Emmanuel Casalino
Senior Director, Physician eHealth Program
eHealth Ontario
Dr. Sonny Cejic
Commissioners West Family Health Organization
Dr. Anne Duvall
Peer Leader OntarioMD
Barrie & Community Family Health Team
Dr. Anne Duvall
Barrie & Community Family Health Team
Dennis Ferenc
Director, Funding, Reporting and Change Management
OntarioMD
Dr. Sanjeev Goel
Wise Elephant Family Health Team
Simon Hagens
Director, Benefits Realization & Quality Improvement
Canada Health Infoway
Dr. David Kaplan
North York Family Health Team
Dr. David Kaplan
Primary Care Physician Lead Central LHIN
North York Family Health Team
Dr. Stephen McLaren
Markham Family Health Team
Dr. Wei Qiu
Director, EMR Adoption and Benefit Realization
eHealth Ontario
Christine Sham
Manager, eHealth Liaison Branch
Ministry of Health and Long-Term Care
Patricia Sullivan-Taylor
Manager, Primary Health Care Information
Canadian Institute for Health Information
Benefits Realization Study for EMRs in Ontario
PwC
Table of ContentsAcknowledgements 2
1. Executive Summary 1
2. Introduction 10
3. Methodology 12
3.1 Phase 1: Validation and Evolution of EMR Benefits Realization Framework 12
3.2 Phase 2: Analysis of Benefits Realization 14
3.3 Study Limitations 19
4. Case Study Results 21
4.1 Provider Survey Results 21
4.2 Indicator and Interview Results 24
5. Modeling and Forecasting 33
5.1 Diabetes Management 33
5.2 Increased Influenza Immunization Rates 35
5.3 Increased Colon Cancer Screening Rates 37
5.4 Staff Time Reduction Spent on Administrative Tasks 39
5.4 Summary Findings 41
6. Discussion 43
6.1 Laboratory Management 43
6.2 Communication and Coordination of Care 44
6.3 Chronic Disease Management 45
6.4 Health Promotion, Screening and Prevention 47
6.5 Efficiency 48
6.6 Medication Management 49
7. Recommendations 51
8. Concluding Remarks 54
9. References 55
Benefits Realization Study for EMRs in Ontario
PwC
Appendix A: Interviewee List
Appendix B: Consultation Guide
Appendix C: Revised EMR BR Framework
Appendix D: EMR BR Framework- Case Study Indicator Subset
Appendix E: EMR Maturity Model Criteria Description
Appendix F: Organizational Survey
Appendix G: Provider Survey
Appendix H: Site Visit Interview Guide
Benefits Realization Study for EMRs in Ontario
PwC 1
1. Executive Summary
Electronic medical records (EMRs) are a key enabler of health system transformation, with the potential to achieve
widespread benefits, including: delivering improvements in patient care processes; enabling positive health
outcomes at individual and population levels; creating efficiencies; and reducing costs.
Ontario’s Action Plan for Health Care (Ministry of Health and Long-Term Care, 2012) defines “faster access to
stronger family health care” as a key imperative, and also focuses on improving the overall health of the population
while ensuring that the right care is delivered at the right time, in the right place. In addition, the government’s
transformative Excellent Care for All Act (S.O. 2010, c14) will ensure that Ontarians receive health care of the
highest possible quality and value, and puts the needs of patients first, placing greater accountability on providers
to ensure that the best available evidence is used to make decisions about patient care. Most recently in December
2012, the government announced two very impactful changes to the health system, including the signing of the
Physician Services Agreement with the Ontario Medical Association the creation of Health Links, placing primary
care providers at the centre of the health system and ensuring that patients receive faster care, spend less time
waiting for services and are supported by a team of health care providers at all levels of the health care system
(Ministry of Health and Long Term Care, 2012).
With these and other initiatives underway, it is clear that Ontario’s health system will be undergoing significant
transformation, and changes in the delivery of primary care figure prominently in the government’s strategic
initiatives. With 80% of health care encounters occurring in primary care settings (Canadian Medical Association,
2011), the vast majority of patient data is collected and managed at the primary care level, and the transformative
changes to be undertaken will be reliant on information management programs and tools. The broad and mature
adoption of electronic medical records (EMRs) by primary care providers and their staff will support the required
transformation and the realization of benefits such as improvements in patient care, positive health outcomes at
individual and population levels, efficiencies for providers and system-wide cost reductions. The strategic and
ongoing focus on advancing the availability and use of electronic medical records (EMRs) across the province is
essential.
A key lever in this strategic transformation is the Physician eHealth Program (PeHP), which is now in its fourth
year investing in community-based physician offices to support the adoption and use of EMRs. The PeHP has
achieved several outcomes in the past four years, including the enrolment of over 9,000 physicians in the EMR
Adoption Program. Having achieved a critical mass of EMR adoption across the province, the PeHP has further
developed its focus on benefits realization in 2012 and beyond, opting to undertake a study to identify the nature of
the benefits realized through program investments to date, and those that can potentially be realized with
continued investment and broader adoption of EMRs. To further understand and assess the potential benefits to
be realized, PeHP engaged PwC to conduct a benefits realization study with a focus on:
The benefits that a select number of advanced users of EMRs in primary care practice settings sites have
realized, directly and indirectly through EMR adoption and usage; and
The potential benefits that may be realized through widespread, mature adoption of EMR use across
Ontario in the future.
Methodology
The methodology for the study was designed by PwC in consultation with the PeHP team to articulate current and
potential benefits from the use of EMRs and recommendations for widespread EMR benefits realization across
Ontario. Two phases of activities from August to December 2012 were undertaken to complete this study:
Benefits Realization Study for EMRs in Ontario
PwC 2
Phase 1 entailed the validation and evolution of a comprehensive framework for assessing the benefits
realized in Ontario as a result of EMR use in primary care settings; and
Phase 2 focused on the analysis of current and potential benefits realized across Ontario through increased
and sustained use of EMRs.
The work completed during Phase 1 set the stage for successful evaluation while simultaneously building the
necessary support among stakeholders through validation of the existing benefits realization (BR) framework. The
approach emphasized the purpose of the framework and the effective positioning of potential benefits that can be
realized by patients, providers and the health system.
Of the original indicators, 21 were identified as priority indicators, laying the foundation for a longer term BR
Framework and measurements that may be used for future analysis. The priority indicators were identified,
specifically for assessment in Phase 2 of the Benefits Realization study based on known available EMR
functionality, maturity of use and feasibility of measurement at case study sites. Based on the current availability of
information and the maturity of EMR users, a set of indicators was selected to assess benefits through a case study
approach at six primary care practices across Ontario. Following the case studies, the set of indicators was further
distilled to reflect data availability and relevance, and a final selection of 11 indicators in five categories was used to
evaluate benefits:
Laboratory Management,
Communication and Coordination of Care,
Chronic Disease Management,
Health Promotion, Screening and Prevention, and
Efficiency.
Phase 2 focused on assessing both current and potential benefits realized through EMR use. Case studies were
designed to understand the benefits realized through the use of EMRs in the six selected clinics across Ontario that
were identified as high performing users of EMRs. While the emphasis of the study was to understand both
quantitative and qualitative processes and outcomes as per the indicators defined in the BR framework, due
consideration was given to identifying relevant insights, experiences and lessons learned that supported benefits
realization and demonstrate the role of EMRs (both direct and indirect) in realizing these benefits. Emerging from
the cases studies was a sixth category of study (Medication Management) and for which qualitative benefits have
been discussed.
Evidence for the case studies was collected from surveys, interviews, direct observation and data extraction from
EMRs. Following the collection of data from the Case Study sites, the PwC team undertook a modeling exercise to
forecast the potential benefits of EMR use in Ontario in five to ten years. Benefits were extrapolated from selected
indicator values collected during the site visits and further validated and substantiated with peer reviewed
literature. The modeling involved three steps:
1. A comparison of current values of indicators for case study sites considered high performing, or advanced
users of EMRs and those for other practices in Ontario.
2. An assessment of the relative benefit realized through advanced EMR use by the case study sites compared
to other practices in Ontario.
3. Extrapolation of relative potential benefits to the entire province, providing benefit estimates (such as
potential avoided costs or quality of care outcomes), if all providers in Ontario adopted an EMR and
achieved similar results to the high performing users of EMRs examined in the case studies practices.
Benefits Realization Study for EMRs in Ontario
PwC 3
Case Study Results
The surveys of providers at case study sites indicated very high support for benefit statements attributed to EMRs.
Key results of the provider surveys include the following:
Quality of Care: 93.0% of survey respondents strongly or moderately agreed with eight statements
presented for benefits of EMR use related to quality of care;
Communication and Coordination of Care: 98.0% of survey respondents strongly or moderately agreed
with three statements presented for benefits of EMR use related to communication and coordination of
care;
Efficiency of Practice: 81.5% of survey respondents strongly or moderately agreed with nine statements
presented for benefits of EMR use related to efficiency of practice; and
Patient Experience: 94.4% of survey respondents strongly or moderately agreed with two statements
presented for benefits of EMR use related to overall patient experience.
Indicator and Interview Results
As described above, the priority indicators that were identified in Phase 1 were assessed for each of the six case
study sites. Findings from indicators and interviews were reviewed together with a sixth category, Medication
Management, which emerged in discussion with several providers during the interviews.
Key results that were identified through the case studies are as follows:
Turnaround time for lab result availability to receipt in EMRs has declined from as much as 5 days to virtually
instaneously. For most study sites, a minimum of 50% decrease in turn around time was reported with results
returned into their EMR. This timely access to test results affords the opportunity to expedite referrals and improve
access to care, and to make timely decisions related to treatments. In addition, physicians perceived the EMR to
improve ordering efficiencies.
The time to receive discharge summaries after patient discharge has declined from as much as 14 days to
virtually immediately where EMRs are integrated with tools such as the Hospital Report Manager or through
direct connectivity with hospitals. These reductions in time spent waiting for discharge summaries and referrals
expedite and facilitate the coordination of care such that patient needs can be addressed in a timely fashion.
The time from referral decision to when the referral is sent to specialist has declined from as much as 7 days to
less than 1 day. Some physicians were able to complete the referral and send it to the specialist with the patient in
the room. Overall, it was observed that the EMR improves physicians’ ability to make timely referrals, expediting
the care process and facilitating inter-office communications.
100% of the care team members have remote and local access to EMRs. This access to information at any time
and in any place was noted by all as a tremendous asset to providers and their patients, improving the ability to
communicate within clinics and often with providers outside of clinics, with overall results including improvements
in the efficiency of patient care.
Up to 70% of diabetic patients, 18 years and over, have an HbA1c level of 7% or less. With the ability to identify a
target diabetic population, physicians and care teams are better able to develop care plans tailored to populations.
Overall, provider survey respondents agreed that EMRs improve the management of chronic diseases. 92.5% of
survey respondents also reported that the EMR system supports patient education. For example, the EMR provides
access to handouts, references or tools to trend patters for BMI, blood sugar, labs. Collectively, the educational
materials and tools help patients better understand and manage their chronic conditions.
Benefits Realization Study for EMRs in Ontario
PwC 4
Up to 80% of practice populations received an influenza immunization, up to 56% of practice populations had a
screening test ordered for colon cancer, and up to 82% of practice populations have had their blood pressure
measured by their primary care provider within the last 15 months. Provider survey results emphasize the role of
the EMR system in promotion, prevention and screening activities. Provider survey results report that 92.6% of
survey respondents agree that EMRs improve patient safety and the proactive monitoring of overdue tests/ exams.
It should be noted that the provincial target for colorectal cancer screening is 40% (Health Quality Ontario, 2012)
and that the values reported by all the case study sites exceeded the provincial average.
Up to 50% less time was required by high performing practices to complete clinical/administrative
documentation by using pre-populated templates, forms and stamps. Overall, 77.7% of survey respondents agreed
with the statement that EMRs “improve the efficiency (reduction in effort) of my practice” and of those 44.4%
strongly agreed with the statement while 33.3% moderately agreed. These survey responses mirror the comments
collected during the interviews on overall practice efficiencies.
Up to 89% of survey respondents agree that EMRs support patient safety through the ability to identify patients
on prescriptions that have drug recalls. Physicians and care team members reported an important patient safety
benefit of the EMR is the ability to quickly extract patient lists for specific medications that have been recalled.
Additional patient safety benefits were identified in relation to improved legibility of prescriptions.
Modeling and Forecasting
A model was developed to forecast potential benefits and “the art of the possible” in Ontario if all providers wereadopting and using EMRs at the same level as the advanced EMR users in the case study sites. Benefits wereextrapolated from indicators collected through the case studies, and supported with peer reviewed literature.
Four indicators were selected from the BR framework for forecasting based on the availability of data for eachindicator at the various sites as well as the availability of supporting evidence to extrapolate these indicators tobenefits for Ontario. These include:
Chronic Disease Management - Improved diabetes management
Health Promotion, Screening and Prevention – Increased influenza immunization rates
Health Promotion, Screening and Prevention – Increased colon cancer screening rates
Efficiency - Staff time reduction spent on administrative tasks
Each indicator along with its associated benefit and benefit estimate is provided in the table that follows. Althoughthe selected benefits forecast only a portion of benefits expected from EMRs and that a variety of contributingfactors influence the values of these indicators (e.g. financial incentives to increase screening rates), they indicatesubstantial potential benefits to be realized if all providers in the province were to become high performing users ofEMRs in 5 – 10 years.
Benefits Realization Study for EMRs in Ontario
PwC 5
Summary of Forecasted Benefits
Indicator Qualitative Annual Benefits Potential Annual FinancialImpact by 2017
Percentage of patients, 18 years andover, with diabetes mellitus in whomthe last HbA1c was 7.0% or less (orequivalent test/reference rangedepending on local laboratory) in thelast 15 months
Patient and health system benefitsfrom management of complicationsand co-morbid conditions arisingfrom diabetes, including:
- 566 fewer foot amputations- 341 fewer cases of ESRD- 17,400 fewer MI cases- 3,100 fewer stroke cases
$125 million
($17 M from reduced footamputations, $26 M fromreduced kidney disease, $44Mfrom fewer MI, $38M fromfewer strokes)
Percentage of patient population, age65 and older, who received aninfluenza immunization
Patient and health system benefitsrelated to illness prevention,avoidance of clinic and/or hospitalvisits
$40.6 M in reduced health caresystem costs
Percentage of practice population,age 50 to 74, who had a screeningtest ordered for colon cancer
Identification and diagnosis ofcancers and malignancies at earlystages for improved prognosis andquality of life for patients
- 220 fewer Ontarians diagnosedwith stage 4 cancer;
- 217 fewer Ontarians diagnosedwith stage 3 cancer;
- 267 fewer Ontarians diagnosedwith stage 2 cancer;
- 703 additional Ontarians with nocancer or stage 0/stage 1 cancers
$38.0 M in reduced colon cancertreatment costs
Percentage reduction in time spenton administrative tasks
Increased staff capacity through areduction of 1.4 million hours thatcan be allocated to other tasks,including the provision of patientcare
$40.0 M in increased staffcapacity
Discussion
Through an approach that has focused on case studies of advanced EMR users in Ontario and forecasting of further
potential benefits, this study has demonstrated that Ontario’s investments in EMR to date have yielded noteworthy,
tangible benefits. These benefits are diverse and have accrued to patients, providers and the broader health system.
Case study participants provided a unique vantage point to convey valuable insights into benefits from their first-
hand experience in pioneering and optimizing the use of a variety of EMR tools and functionalities across their
patient care teams.
While the earliest benefits have been shown to be realized by providers and their patients, modeling and forecasting
has demonstrated that continued investment in EMRs across the province can heighten the impact of EMR use on
the broader health system, with the potential for EMRs to have very significant direct and indirect impacts.
Benefits Realization Study for EMRs in Ontario
PwC 6
While it is acknowledged that the potential benefits of EMRs are broad, the current study focused on six categories
with key benefits discussed below.
1. Laboratory Management
EMRs provide clinicians with more timely access to laboratory information, aiding care decisions
and enhancing the patient experience. The ability to receive lab results through direct transmission to EMRs
has reduced the time to receive those results by 50% on average (compared to a paper-based environment).
Consequently, clinicians are now well enabled to respond to results quickly and effectively. This timely access to
test results, while either in their clinics or from remote locations, affords the opportunity to make timely decisions,
provide prompt and appropriate care, expedite referrals, and improve patients’ access to care. The value of having
comprehensive lab information in a timely manner through all potential sources of lab information provides a
comprehensive profile for patients to form a basis for clinical decision-making. In addition, the increasing
prevalence of patient portals linked to EMRs, patients and/or proxies can have online access to results, allowing
them to review lab results when and where it is convenient for them. These initiatives can improve patients’
experiences with their care, and provide them with some comfort, understanding and ownership in care processes.
2. Communication and Coordination of Care
EMRs facilitate improved scheduling and coordination of patient visits, improving access to care.
EMRs provide physicians and care team members with the improved ability to schedule patient visits, improving
patients’ access to care and efficiency of the care team. Physicians and team members reported that improvements
in scheduling and organization facilitate their ability to hold same day appointments open, improving access to
care. Ontario’s Health Action Plan (Ministry of Health and Long-Term Care, 2012) identifies improved availability
of same-day appointments as a key focus in improving access to primary care.
EMRs improve the availability and sharing of information among interdisciplinary team members
and enhances quality and efficiency of care. The ability to access patient information at any time and in any
place was noted by all as a tremendous asset to providers and their patients. EMRs also improve the ability to
communicate within clinics and often with providers outside of clinics, with overall results improving the efficiency
of patient care. Quality of care, and the patient experience overall, is improved for patients by ensuring all
providers have access to the same patient information.
EMRs facilitate the sharing of information with specialists, thereby improving the continuity and
efficiency of care. EMRs support the ability to make much more informed and efficient referrals to specialists.
With the ease and improved efficiency of making referrals (an approximate reduction in time of 85% to make a
referral), the EMR allows primary care clinicians to quickly provide the specialist with key pieces of information
needed to understand the patient’s condition.
Expedited delivery of hospital reports to EMRs facilitates timely and appropriate care. Case study
sites reported a reduction of 85% in time spent waiting for discharge summaries as a result of the transfer of this
information to EMRs. The timely access to this information can further reduce patients’ wait times for required
post-discharge care, and allow providers to put appropriate follow-up care into place in the out-patient setting.
3. Chronic Disease Management
EMRs are a necessary and effective tool to manage the health of defined patient populations. EMRs
are being used increasingly by clinicians to manage the health of patient populations, such as patients with
diabetes, chronic obstructive pulmonary disease (COPD), and others. Physicians and their care teams are
increasingly relying on their EMRs to effectively manage the care of patients with chronic conditions that are costly
Benefits Realization Study for EMRs in Ontario
PwC 7
to the system overall and have many related co-morbidities constraining health system capacity. It was evident that
case study sites were using their EMRs for these purposes as well; for example, clinicians reported the ability to
identify patients with specific conditions and plan and monitor their care, and identified this as a very significant
benefit of EMRs. Clinicians reported a high degree of willingness and enthusiasm to better manage the health of
defined patient populations in their practice through the support of their EMRs, and acknowledged that in the
absence of EMRs this undertaking would be quite costly and time consuming.
EMRs provide valuable tools to help both care providers and patients with care management and
education. Engaging patients through education and care planning activities ensures that they are active
participants in their health. Patients that are better informed on what their target values for key indicators (e.g.,
HbA1c) should be are better able to keep their conditions under control. As data capture and reporting capabilities
improve, EMRs will further allow practices to identify a baseline and trend information to ensure their diabetic
cohort of patients are monitored for those clinical needs which require careful management and are prone to
downstream co-morbid complications.
The broad and mature use of EMRs can reduce the costs and burden of illness associated with
caring for Ontario’s growing diabetic population. Through the modelling and forecasting exercise,
potential savings and a reduction in complications associated with diabetes were identified. These complications
and illnesses include foot amputations, diabetic kidney disease, stroke and myocardial infarction, all of which
typically require acute care hospitalization. While yet to be realized, EMRs are a contributing factor to potential
savings related to diabetic-related illnesses that are in the range of $125 million annually by 2017. Diabetes must
be actively managed in the community and primary care settings and with active patient participation, in order to
complications such as those forecasted above. Without EMRs and related enablers, it would be very challenging to
do so, given the support that EMRs provide to actively identify diabetic patients, keep their conditions under
control, and communicate on an ongoing basis.
4. Health Promotion, Screening and Prevention
EMRs allow clinicians to survey patients and to proactively arrange screening and prevention
activities, while concurrently improving the efficiency of preventative care. While several reporting
capabilities related to screening and prevention are in their early stages of development, advanced users of EMRs
indicated that they were able to generate information to identify and communicate with patients for preventative
purposes. Without the EMR, this type of prevention activity is much more complicated, requiring manual and
time-consuming chart reviews. Now, patients are contacted easily and in a timely fashion for preventative care.
The potential benefits of increased screening and prevention (including vaccination) activities over time can have a
tremendous impact on the health of the population as a whole, and on the sustainability of the health system.
The widespread use of EMRs can increase the rates of influenza vaccination and yield potential
related health system savings. The forecasted reductions in illness and costs are highly dependent on the
mature, proactive use of EMRs on a province-wide basis to identify those at risk, to facilitate communication to
encourage those patients to receive the flu shot, and to track compliance. The management of influenza must be
actively managed in the community and primary care settings and with active patient participation and willingness
to receive vaccination. EMRs enable this challenge, with many interviewees discussing their ability to vaccinate a
broader group of patients by leveraging information available in their EMRs.
Use of EMRs can support the prevention of colon and other cancers through improved screening
rates and other preventative care. Case studies revealed that EMRs can greatly facilitate the ease with which
clinicians manage and deliver preventative care for their patients. Although preventive care is a relatively new (and
in some cases, advanced) area of EMR use for many clinicians, the ability to identify patients requiring screening
Benefits Realization Study for EMRs in Ontario
PwC 8
and in turn receive and act upon results is possible with EMR use today, and many provincial organizations (e.g.,
Cancer Care Ontario) recognize the power and potential of EMRs to support preventive care. Modeling and
forecasting activities suggested that the potential costs that can be realized through the prevention of four stages of
colon cancer through advanced EMR use is in the range of $38 million annually by 2017. By ensuring that
screening protocols and alerts are incorporated in all EMR specifications, there will be an increased ability to
identify those patients requiring screening on a widespread basis.
5. Efficiency
EMRs facilitate clinical transformation in the primary care setting, improving the effectiveness and
efficiency of clinical and administrative activities. The introduction of EMRs into the primary care setting
has the potential to be transformative in nature. EMRs enable changes in workflow and clinical decision-making,
and can greatly improve the way that clinicians interact with their patients on a regular basis. Benefits associated
with the effectiveness and efficiency of patient encounters and the general flow of patient activity allow greater
throughput and access to care, including same day visits. These benefits have tremendous impact on patients,
providers and the system as a whole, by reducing wait times for primary care, allowing providers to see more
patients daily, and reducing the number of unnecessary visits to hospitals.
EMRs improve the productivity of administrative staff. It is evident that there is some productivity lost in
the early stages of adoption of EMRs with many clinicians and administrators, with many reporting that there is a
period of approximately one year while all grow accustomed to working with electronic records. However,
interviewees and the forecasting model both suggest that there are significant opportunities to improve the overall
capacity among clinic staff, particularly among administrative staff and/or nurses who were previously spending
time doing administrative tasks. The greatest value in this increased capacity lies in the opportunities and potential
benefits associated with redirecting time from non-value added tasks, to those that improve efficiency, allow for
direct patient interaction, and improve the overall patient experience. The reductions in the time required for
administrative tasks through the use of EMRs, estimated at 50%, was used to model the potential benefits if EMRs
were broadly and maturely used across Ontario. The exercise estimated that approximately 1.8 million hours or a
possible $40 million could be saved annually by 2017, providing increased capacity for clinic staff.
6. Medication Management
EMRs support the ability to rapidly identify impacted patients when drug warnings are issued,
improving patient safety. A tremendous benefit of EMR use identified by many providers through the course of
the study was the ability to identify large numbers of patients to whom certain drugs have been prescribed. These
patients can be very quickly identified, and alternate means of treatment can quickly be administered, preventing
any downstream implications. As such, EMRs can have a tremendous impact on quality of care, and most
importantly, patient safety. The ability to quickly target these patients and act is almost impossible in a paper-
based clinic environment.
Access to complete medication profiles in the EMR increases efficiency and improves the accuracy
of medication management. With accurate and complete medication profiles in EMRs, clinicians are able to
quickly and accurately manage care. Prescriptions are easily monitored, and patient concerns can be addressed.
There are also potential costs avoided by the health system due to unnecessary hospitalization from adverse
medication effects.
Electronic prescribing and renewals via EMRs have improved medication management efficiencies
and patient safety. The sharing of medication information across the care team has resulted in greater
efficiencies in the patient care process and improved patient safety. Perhaps the most significant change noted by
physicians has been the generation of a printed prescription, eliminating error-prone handwritten prescriptions
Benefits Realization Study for EMRs in Ontario
PwC 9
and reducing the need for delays arising from “call backs” from pharmacists to physicians seeking clarification on
prescriptions. With EMRs, this risk to patient safety and disruption in workflow is avoided.
Recommendations
Findings from this study provide compelling evidence to continue to advance EMR adoption and maturity across
Ontario. The benefits that have been demonstrated by EMR use in the selected case study settings and the
accompanying forecasting of province-wide benefits demonstrate the “art of the possible” for Ontario. With the
implementation of a number of focused recommendations, the potential for wide ranging and transformative
benefits of EMR use can be further realized by providers, patients and the health care system as a whole.
The following five recommendations are presented to policy-makers, funders, implementers and adopters of EMRsin Ontario in support of continued benefits realization.
1. As an essential enabler and one of many important health information technology tools to
improve care delivery and related patient outcomes in primary care settings and beyond,
continued investments in EMRs should be made to ensure broad adoption and realization of
benefits across Ontario. Through the course of the study it was widely acknowledged that without the use
of EMRs, the ability to realize the identified benefits is compromised. Indirectly and directly, EMRs are critical
enablers of enhanced patient care. Continued investment in EMRs and increased physician participation in the
EMR Adoption Program are essential.
2. Continue to support increased maturity of use among current and future adopters of EMRs.
The effective realization of benefits is highly supported by EMR maturity (defined as the level of adoption and
use of the EMR in the practice setting). A continued focus in advancing EMR maturity among users will
contribute to a greater diversity of benefits with system-wide impact.
3. Continue to invest in effective change management strategies and user support that extends
beyond the initial period of EMR implementation. Mature use of the EMR requires access to training
when and where it is needed by all types of users, and should be available well beyond the initial
implementation phase as users transition through the “adoption curve”. For example, following an initial
period of use that allows clinicians to master essential EMR functions, training could be further made available
to address and support more sophisticated needs associated with reporting and analytics for population-based
planning and care.
4. Improve the management of information within and across patient care settings through
focused efforts related to interoperability of systems, improved quality of data, and the flow of
data across care settings. The ability to achieve advanced use and benefits of EMRs will be supported by
increased systems integration, improved quality of data, and improved sharing of data across care providers.
Improvements in information management through initiatives such as OntarioMD’s Hospital Report Manager
and CIHI’s Voluntary Reporting System are showing promising benefits, and similar initiatives should be
encouraged.
5. Continue to invest in focused benefits realization studies. Focused studies will afford the opportunity
to measure more of the indicators that were defined in this study as part of the Benefits Realization Framework
and have a greater understanding of the full scope of current and potential benefits realized by EMRs in
Ontario.
Benefits Realization Study for EMRs in Ontario
PwC 10
2. Introduction
Health care systems across Canada and around the world are undergoing transformations to improve quality of
care, access to care, value for money, and the patient experience (Commonwealth Fund, 2011). In Ontario, there is
an increasing emphasis on ensuring that these attributes are all central to the delivery of primary care, providing
the foundation of a strong health care system.
Ontario’s Action Plan for Health Care (Ministry of Health and Long-Term Care, 2012) defines “faster access to
stronger family health care” as a key imperative, and also focuses on improving the overall health of the population
while ensuring that the right care is delivered at the right time, in the right place. In addition, the government’s
transformative Excellent Care for All Act (S.O. 2010, c14) will ensure that Ontarians receive health care of the
highest possible quality and value, and puts the needs of patients first, placing greater accountability on providers
to ensure that the best available evidence is used to make decisions about patient care. Most recently in December
2012, the government announced two very impactful changes to the health system, including:
the Physician Services Agreement reached between the Ontario Medical Association and the government in
December 2012 (Ministry of Health and Long-Term Care, 2012), supporting the realization of significant cost
savings and efficiencies across the health system and promoting the use of electronic communications and
consultations to increase access to care; and
the creation of Health Links (Ministry of Health and Long-Term Care, 2012), placing primary care providers at
the centre of the health system and ensuring that patients receive faster care, spend less time waiting for
services and are supported by a team of health care providers at all levels of the health care system.
With these and other initiatives underway, it is clear that Ontario’s health system will be undergoing significant
change, and changes in the delivery of primary care figure prominently in the government’s strategic initiatives.
With 80% of health care encounters occurring in primary care settings (Canadian Medical Association, 2011), the
vast majority of patient data is collected and managed at the primary care level, and the transformative changes to
be undertaken will be reliant on information management programs and tools. The broad and mature adoption of
electronic medical records (EMRs) by primary care providers and their staff will support the required
transformation and the realization of benefits such as improvements in patient care, positive health outcomes at
individual and population levels, efficiencies for providers and system-wide cost reductions. The strategic and
ongoing focus on advancing the availability and use of electronic medical records (EMRs) across the province is
essential.
In supporting, promoting and accelerating the adoption of EMRs across the province, eHealth Ontario’s Physician
eHealth Program (PeHP) is a key initiative in advancing the province’s transformative agenda and Ontario’s
eHealth strategy and primary goal of establishing and maintaining electronic health records (EHRs) for all of
Ontario’s 13 million residents (eHealth Ontario, 2013). EMRs are a partial and necessary component of EHRs,
which hold all relevant health information about a person over his/her lifetime (Hodge, 2011). Since 2009, the
PeHP has invested in community-based physician offices to support the adoption and use of EMRs. Together with
its delivery agent OntarioMD, the PeHP has achieved several outcomes, including the enrolment of over 9,000
physicians in the EMR Adoption Program. The program is currently focused on:
Further equipping and enabling community-based physicians with tools to enhance the use of EMRs;
Maximizing clinical and business value of EMRs;
Providing change management support and promoting the adoption of best practices; and
Fostering the evolution and sustainment of EMRs and related benefits.
Benefits Realization Study for EMRs in Ontario
PwC 11
Having achieved a critical mass of EMR adoption across the province, the PeHP has further developed its focus on
benefits realization in 2012, opting to undertake a study to identify the nature of the benefits realized through
program investments to date, and those that can potentially be realized with continued investment and broader
adoption of EMRs. Specifically, the current and potential impacts of the use of EMRs on patients, providers and
the health system are of interest.
To further understand and assess the current and potential benefits to be realized, PeHP engaged PwC to conduct a
benefits realization study with a focus on:
The benefits that a select number of advanced users of EMRs in primary care practice settings sites have
realized, directly and indirectly through EMR adoption; and
The potential benefits that may be realized through widespread, mature adoption of EMR use across
Ontario in the future.
In collaboration with the PeHP team, PwC conducted this study through a two-phased approach from August to
December 2012. This report presents the study in its entirety as follows:
Section 3: Methodology, outlining the methodology applied to Phases 1 and 2
Section 4: Results, presenting the findings arising from the studies of six primary care clinics considered
high performing adopters of EMRs
Section 5: Modeling of Benefits, identifying potential system-wide benefits from EMR use
Section 6: Discussion, reporting on the themes and insights emerging from the results
Section 7: Recommendations, outlining key actions to be taken in order to realize potential benefits across
Ontario
Section 8: Concluding Remarks
Benefits Realization Study for EMRs in Ontario
PwC 12
3. Methodology
The methodology for the study was designed by PwC in consultation with the PeHP team to articulate current and
potential benefits, and to develop recommendations for widespread EMR benefits realization across Ontario. Two
phases of activities from August to December 2012 were undertaken to complete this study:
Phase 1 entailed the validation and evolution of a comprehensive framework for assessing the benefits
realized in Ontario as a result of EMR use in primary care settings; and
Phase 2 focused on the analysis of current and potential benefits realized across Ontario through increased
and sustained use of EMRs.
3.1 Phase 1: Validation and Evolution of EMR Benefits RealizationFramework
The work completed during Phase 1 set the stage for successful evaluation while simultaneously building the
necessary support among stakeholders through validation of the existing benefits realization (BR) framework. The
approach emphasized the purpose of the framework and the effective positioning of potential benefits that can be
realized by patients, providers and the health system.
3.1.1 External Validation of the Framework
The PeHP program had developed an initial draft of a BR framework, inclusive of several categories of
measurement, hypotheses and sixty indicators. This framework was used as the basis for consultations to obtain
feedback and was the subject of further refinement and validation.
A total of 21 consultations were conducted with a variety of representatives including physicians, family health
teams, the Ministry of Health and Long Term Care, government agencies, Local Health Integration Networks
(LHINs), and other jurisdictional EMR programs (Alberta and British Columbia), (see Appendix A for a complete
list of interviewees). Interviewees were provided with a copy of the interview guide and the draft EMR BR
Framework for review prior to the interview (see Appendix B, Consultation Guide).
Stakeholders were asked to select indicators that could describe whether or not EMRs have had an overall impact
on patients, providers and the health system. A formal system for scoring indicators was not used, however
stakeholders reviewed the indicator list and based on their experience and perspective, were able to identify those
indicators they felt would be meaningful and useful to measure.
3.1.2 Development of BR Framework and Indicators
A workshop was held in September 2012 to review all feedback and define the preferred BR framework and
indicators for the case studies and any future benefits realization studies. Workshop discussions were informed by
insights from consultations, a preliminary BR framework assessment, benefits literature and PwC’s experience with
related benefits realization studies.
At the workshop, project team members from the PeHP, Ontario MD and PwC assessed potential indicators based
on the following criteria:
Benefits Realization Study for EMRs in Ontario
PwC 13
Relevance: Assessment of the importance of the indicator for the objective/hypothesis question e.g. Does
the indicator support an understanding of the impact of the EMR on quality of care from the patient’s
perspective?
Feasibility: Assessment of the ease and cost of measurement e.g. is the data currently available?
Ease of interpretation: Assessment of the ease of interpreting the indicator, e.g. Is the indicator well
understood by multiple stakeholders, without requiring extensive explanation?
Traceability to the EMR: Assessment of whether the indicator can be directly attributed to EMR use e.g.
is there a clear and demonstrable correlation between the use of EMRs and the indicator?
Of the original indicators, 21 were identified as priority indicators, laying the foundation for a longer term BR
Framework and measurements that may be used for future analysis. The indicators were also classified into nine
categories of benefits (see Appendix C, Revised EMR Benefits Realization Framework). Priority indicators were
identified, specifically for assessment in Phase 2 of the Benefits Realization study based on known available EMR
functionality, maturity of use and feasibility of measurement at case study sites. A total of 14 indicators were
initially selected for the EMR BR Framework - Case Study Indicator Subset (see Appendix D), with study questions
in related categories including the following:
Table 1 : Case Study Questions
Category Study Questions
Laboratory
ManagementDoes EMR use reduce lab result turnaround time?
Communication and
Coordination of CareDoes EMR use improve access to information between settings?
Does EMR use facilitate referral to specialists?
Does EMR use facilitate interdisciplinary/team care?
Do physicians access the EMR remotely to provide patient care?
Chronic Disease
ManagementDoes EMR use improve chronic disease management?
Health Promotion,
Screening and
Prevention
Does EMR use improve preventative services provided?
Efficiency Does EMR use improve efficiency of care?
Benefits Realization Study for EMRs in Ontario
PwC 14
3.2 Phase 2: Analysis of Benefits Realization
Phase 2 focused on assessing both current and potential benefits realized through EMR use. In order to undertake
this assessment, a series of case studies were undertaken within six primary care settings. The information
collected within these sites together with peer reviewed studies served as inputs to a modelling exercise that defined
potential benefits that could be realized with advanced EMR use and adoption across Ontario.
3.2.1 Case Studies
At the core of the second phase were the case studies, designed to understand the benefits realized and key lessons
learned through the use of EMRs in selected clinics across Ontario that were identified as high performing users of
EMRs. While the emphasis of the study was to understand both quantitative and qualitative processes and
outcomes as per the indicators defined in the BR framework, due consideration was given to identifying relevant
insights, experiences and lessons learned that supported benefits realization and demonstrate the role of EMRs
(direct and indirect) in realizing these benefits.
Case Study Site Selection Process
In order to select case study sites, a preliminary list of ten primary care clinics was compiled by the PeHP and PwC
with input from stakeholders, including OntarioMD. Each potential site was assessed based on the following
preferred criteria.
EMR Maturity/Experience
Location / Geography / Site Type
Vendor System and Integration
Available and Accessible Documentation
Innovation
Willingness to Participate
1. EMR Maturity/Experience: The case study site will ideally have an EMR in use by clinicians for at least
three years and demonstrate mature use of the EMR. The OntarioMD EMR Maturity Model (EMM) illustrated
in Figure 1 was used as a guide to provide information on EMR effectiveness and to identify sites that were
considered “high performing”, with the ability to measure the impacts on the various indicators. See Appendix
E for a more comprehensive description of the EMM.
Benefits Realization Study for EMRs in Ontario
PwC 15
Figure 1: OntarioMD EMR Maturity Model
2. Location / Geography / Site Type: To ensure representation from across Ontario and different clinic types,
geographic factors were considered e.g. urban and rural, in addition to different primary care clinic types, e.g.
small practice groups, large groups.
3. Vendor System and Integration: Representation from clinics using a variety of Ontario-certified vendor
systems was considered. Another site consideration was to select those with various degrees of connectivity
and interoperability (connectivity with OLIS, and access to electronic receipt of hospital reports).
4. Available and Accessible Documentation: The ability of the sites to provide access to documentation that
would facilitate measurement of the indicators in the framework was an important criterion.
5. Innovation: Practices that were seen as being innovative and leading edge in the way they use their EMR
were rated more highly.
6. Willingness to Participate: The willingness of sites to participate in all aspects of the study was an
important criterion.
Candidate sites were contacted and asked to participate. Six practices agreed and were the subjects of the case
studies.
Case Study Tools and Assessment
Case studies were conducted at the selected six practices. The case studies focused on addressing the following high
level study questions as a means of gathering information related to the subset of indicators, advanced application
of EMRs within the clinical setting and lessons learned:
How are EMRs used in high performing clinical practices?
What are the best practices and lessons learned with regards to adoption and use of EMRs?
Benefits Realization Study for EMRs in Ontario
PwC 16
Evidence for the case studies was collected from the following four sources. Descriptions of the tools and methods
are described below:
Surveys (Organizational and Provider)
Interviews
Direct observation
Data extraction
Surveys
Prior to the case study site visits, two surveys were administered by PwC:
1. Organizational Survey: This survey focussed on gathering general background information related to
context, organizational resources and EMR users. It was completed by the Clinical Lead or designate at the
site.
2. Provider Survey: Physicians, physician assistants and nurse practitioners were invited to complete an
online provider survey prior to the site visit. This survey focussed on gathering information on EMR
adoption and current use. All providers were given the option of completing the tool; however, those who
had been selected for an interview were required to complete it.
a. Provider Survey Part 1: EMR Benefits- Part 1 of the survey focused on providers’
perception of benefits realized from EMR use.
b. Provider Survey Part 2: OntarioMD Progress Survey- Part 2 of the survey focused
on the measurement of maturity across key functional areas, and was representative of
OntarioMD’s EMR progress survey.
The survey responses provided valuable information prior to the site visits about each practice, as well as: EMR use
and adoption; EMR maturity; and, general perceptions around EMR use. The information obtained was used to
tailor the interview guide for each individual on-site visit. The Organizational and Provider Surveys are presented
in Appendices F and G respectively.
Interviews
On-site interviews were conducted with clinicians and team members (including administrative staff) whose
responsibilities involved interaction with some aspect of the EMR. For the physicians, the Clinical Lead along with
other physicians (a representative number relative to the size of the practice) were interviewed. A total of thirty-
nine physicians and clinical staff were interviewed across the six sites.
The interviews contained open-ended and closed questions to expand the data gathering and to increase the
number of sources of information. A structured interview guide was developed and emailed to each interviewee
prior to the interview (see Appendix H). As well, the case study subset of EMR indicators was also provided to the
site prior to the site visit (Appendix D). Qualitative data from the interviews were used as a proxy for indicator
measurement where EMR date extraction was not feasible.
Direct Observation
Where possible, the study team observed clinician and staff use of the EMR while on-site. This supported the
information gathering related to workflow and specific use of EMRs in the practice setting. In addition, it served to
validate the information obtained through the surveys and interviews. Because of Personal Health Information
restrictions, the study team did not partake in any activities related to direct patient care or where unique patient
identifiers were visible to the study team.
Benefits Realization Study for EMRs in Ontario
PwC 17
Data Extraction
With assistance from Clinical Leads, IT leads, or data analysts, EMR reports were extracted from the EMR systems
in each site, where possible, in order to measure benefits related to specific indicators.
3.2.2 Final Revisions to Benefits Realization Framework
Following the completion of all case study site visits, the framework and indicators were reviewed and discussed
with members of the Steering Committee. Based on data obtained and feedback received, the framework was
revised for analysis and reporting purposes. The Steering Committee agreed to focus case study analysis on
indicators that would be representative of the BR framework and were most readily accessed and quantified and/or
qualified. The revised BR framework with five categories and eleven indicators is presented in Table 2:
Table 2: Final BR Framework for Case Study Reporting Purposes
Category Indicator
Reference
Number
Indicator
Laboratory Management LM2 Average time between laboratory time of service and test resultsavailable in EMR
Communication and
Coordination of Care
CC1 Average time to receive discharge summary following inpatientdischarge
CC3 Average time from referral decision to when the referral is sent
CC4 % of practices where the care team has access to and uses theEMR system
CC5 % of physicians who have remote access to EMR and use it forpatient care
Chronic Disease
Management
CDM1 % of PHC clients/patients, 18 years and over, with diabetesmellitus in whom the last HbA1c was 7.0% or less in the last 15months
CDM2 % of patient population, age 18 and older, with diabetes mellituswho received testing for diabetic complications
Health Promotion,
Screening and
Prevention
HPSP1 % of practice population, age 65 and older, who received aninfluenza immunization
HPSP2 % of practice population, age 50 to 74, who had a screening testordered for colon cancer
HPSP3 % of practice population, age 18 and older, who have had theirblood pressure measured by their primary health care providerwithin last 15 months
Efficiency E1 % change in time to complete clinical/admin documentation
Benefits Realization Study for EMRs in Ontario
PwC 18
The values collected from case study sites for the indicators in this framework are referenced and discussed in
Section 4, Case Study Findings.
3.2.3 Modeling and Forecasting of Benefits
An important activity undertaken in Phase 2 was the modeling and forecasting of the potential benefits associated
with broad, mature use of EMRs in Ontario in five to ten years. A fundamental assumption is that this timeframe
will allow widespread and more mature use of EMRs, similar to the advanced users of EMRs today, as represented
by the case study sites. Benefits were extrapolated from indicators collected through the EMRs during the site
visits and further validated with peer reviewed literature. It is to be noted that two studies (Hillestad et al, 2005
and Manitoba Health, 2012) have been referenced on multiple occasions for forecasting purposes, as the scope of
these studies are in alignment with this benefits realization study.
As outlined in Figure 2, development of each of the benefit estimates was an iterative approach. The EMR BR
Framework was used to guide the development of potential EMR benefits. Indicators from the framework were
selected for forecasting based on the availability of data for each indicator at the various sites as well as the
availability of supporting evidence for the extrapolation of these indicators to province-wide benefits.
Figure 2: Approach to Developing Benefit Estimates
Comparative Indicators
The first step consisted of comparing current values of indicators for leading practices and those for other practices
in Ontario. For each indicator selected, minimum, maximum and average values were calculated based on data
collected on site. In order to compare this with the current standard in Ontario, a search of the literature and data
collection agencies (e.g. Statistics Canada and Canadian Institute for Health Information) was conducted. Where
possible, Ontario estimates were used, although in certain instances Canadian or other provincial estimates had to
be used as a proxy. This comparative estimate is an average value of the indicator for the entire province.
The maximum value for each indicator collected on site was used as the projected indicator for Ontario practices in
five and ten years. This was done in order to assess the full potential if all Ontario practices realized benefits similar
to the high performing EMR-enabled practices, as represented by the case study sites.
Associated Relative Benefit
The second step involved assessing the relative benefit for the high-performing practices compared to other
practices in Ontario. Although indicators provide some sense of the expected benefit, they are not realized benefits
per se. For example, the share of the population that is immunized suggests improved health outcomes and avoided
health care costs, however the immunization rate must be extrapolated in order to estimate these actual benefits. A
search of the literature was conducted to attribute a benefit to each indicator. The search prioritized impacts on
resource use or costs.
Indicator estimatefrom site visits
Comparative indicator foraverage Ontario practice
Associated RelativeBenefit
Ontario Extrapolation Ontario Benefit Estimate
Benefit Indicator
Benefits Realization Study for EMRs in Ontario
PwC
Extrapolation to Ontario Population
In the third step, relative benefits were extrapolated to the entire p
collected for each benefit such as the number of benefit recipients in the province and the costs of various
resources. In order to estimate the benefit
projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of
Finance population projections and Cancer Care Ontario
estimated using various techniques to extrapolate to provincial levels.
Ontario Benefit Estimate
The obtained benefit estimates provide an indication of the potential avoided costs
providers in Ontario adopted an EMR and
are calculated using identified formulae and financial impacts are presented in 2012 dollars
Selected Indicators
The figure below outlines the indicators selected for modelling by benefit category (Effic
Screening & Prevention and Chronic Disease Management). As mentioned above, other indica
feasibly modeled due to the lack of data extracted
interviews conducted.
Due to study limitations and the inability to model and forecast benefit values for a
that the findings represent a subset of potential province
EMR use. In addition, the above indicators themselves
populations. For instance, benefits associated with influenza immunization were estimated in the over 65 age group
only based on available evidence.
3.3 Study Limitations
Limitations of the study approach include the following:
Generalization of Findings: The
the representativeness of the findings
considered as directional and a foundation for
through broad and mature use of
• Improved diabetes management
Chronic Disease Management
• Increased influenza immunization rates• Increased colon cancer screening rates
Health Promotion, Screening & Prevention
• Staff reduction in time spent on administrative tasks
Efficiency
Extrapolation to Ontario Population
third step, relative benefits were extrapolated to the entire province. In order to do this, additional data was
collected for each benefit such as the number of benefit recipients in the province and the costs of various
resources. In order to estimate the benefit five and ten years from now, the number of benefit re
projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of
Finance population projections and Cancer Care Ontario data. Where local data were not available, proxies were
echniques to extrapolate to provincial levels.
The obtained benefit estimates provide an indication of the potential avoided costs or quality of care outcomes
providers in Ontario adopted an EMR and achieved similar results to the high performing case study sites
and financial impacts are presented in 2012 dollars.
The figure below outlines the indicators selected for modelling by benefit category (Efficiency,
Prevention and Chronic Disease Management). As mentioned above, other indica
led due to the lack of data extracted from EMRs or the inability to derive estimates from the
Due to study limitations and the inability to model and forecast benefit values for all indicators, it is acknowledg
that the findings represent a subset of potential province-wide benefits that can be derived from broad and mature
ition, the above indicators themselves capture a subset of benefits and often for specific patient
. For instance, benefits associated with influenza immunization were estimated in the over 65 age group
imitations of the study approach include the following:
: The intentional focus on six practices with advanced use of
the representativeness of the findings for the province as a whole. Accordingly, the findings should be
a foundation for “the art of the possible” for benefits that can be realized
broad and mature use of EMRs in primary care settings across Ontario.
Improved diabetes management
Chronic Disease Management
Increased influenza immunization ratesIncreased colon cancer screening rates
Health Promotion, Screening & Prevention
Staff reduction in time spent on administrative tasks
19
rovince. In order to do this, additional data was
collected for each benefit such as the number of benefit recipients in the province and the costs of various
years from now, the number of benefit recipients was
projected to 2017 and 2022. The Ontario data was estimated using a number of sources including Ministry of
. Where local data were not available, proxies were
or quality of care outcomes, if all
case study sites. Values
iency, Health Promotion,
Prevention and Chronic Disease Management). As mentioned above, other indicators could not be
s or the inability to derive estimates from the
ll indicators, it is acknowledged
wide benefits that can be derived from broad and mature
and often for specific patient
. For instance, benefits associated with influenza immunization were estimated in the over 65 age group
practices with advanced use of EMRs limits
, the findings should be
benefits that can be realized
Benefits Realization Study for EMRs in Ontario
PwC 20
Sampling: Case study sites were selected based on identified criteria and in consultation with OntarioMD
and the PeHP at eHealth Ontario. Accordingly, the results reflect this sample bias. In addition, survey
participants were identified by the clinical leader rather than being randomly selected, which helped ensure
that interviewees were advanced EMR users but also introduced a sample bias.
Attribution of Benefits to EMRs: It is difficult to establish a strong correlation between EMR use and
measures among the different practice environments because there are many other variables that
contribute to outcomes measured by the indicators (e.g., interdisciplinary care, well established workflows,
payment models, etc.). Anecdotal evidence was acquired to support findings, and define the nature of the
attribution of EMR use to benefits realized. In many instances, the benefits are indirectly attributable to
EMR use, but it was widely acknowledged that several benefits cannot be realized as quickly nor as
effectively in the absence of EMR use, i.e. in a paper-based clinical setting.
EMR Maturity: While maturity of EMR use is improving, the use of EMRs is still evolving across
Ontario. Although the case study sites were identified as advanced users of EMR, there was variability in
EMR maturity and connectivity to other systems. Accordingly, interview data was not always comparable
across sites as interviewees were not able to provide a consistent perspective on EMR capability and, as a
result, provided varying levels of detail regarding EMR use and impact.
Lack of Data Quality, Standardization, and Extraction/Reporting Capabilities: To varying
degrees, each site had difficulty extracting high quality, standardized data from its EMR. Accordingly, it
was difficult for the sites to report EMR impact on specific indicators.
Lack of Baseline Measures: All case study sites estimated pre-implementation indicator values in the
absence of measured baseline information for the selected indicators. Indicator values prior to EMR
implementation were typically estimated from respondents’ memories and as a result, may not be accurate.
Ability to Forecast EMR Adoption and Use: The modeling approach assumes that, by 2017, all
community-based practices in Ontario will have adopted EMRs and will benefit from EMRs to the same
degree as the case study sites. The pace of EMR adoption in Ontario over the next five years is uncertain
and will likely not be on a “straight line” basis, i.e., rates of adoption and benefits realization will likely vary
on a year-to-year basis. Although there was a large increase in EMR adoption in Canada from 2009 to 2012
with the Commonwealth Fund Survey reporting an increase from 37% to 56%, a linear increase to 2017 and
in turn 2022 cannot be assumed.
Limited Evidence for Modeling: The model addresses a limited number of benefits based on data
collected by EMR practices to date and available evidence in the literature. For some indicators, practices
were not able to extract the data from their EMR. For other indicators, although values could be extracted,
there was insufficient evidence regarding the impact of the indicator on cost or quality of care outcomes
(e.g., average time to receive charts after emergency department). In addition, some benefits of EMRs are
not yet reflected by indicators so the benefits cannot be estimated (e.g., immunization in young children).
Benefits Realization Study for EMRs in Ontario
PwC 21
4. Case Study Results
Findings obtained from the six primary care case study sites are reported in the section below. The findings reflect
the quantitative and qualitative benefits that have been measured and realized by the six sites, with data compiled
from provider surveys, site visit data collection as per the selected indicators and in-person interviews. This section
presents results of the provider surveys, followed by indicator and interview results in alignment with indicator
categories.
Following the presentation of findings and modelling and forecasting results in Section 5 that follows, a discussion
is presented in Section 6 with a focus on the broader potential benefits that can be realized through mature EMR
adoption and use across Ontario in primary care settings.
4.1 Provider Survey Results
The survey of 28 physicians, physician assistants and nurse practitioners from the six case study sites focused on
providers’ assessments of the impact of EMRs on quality, communication and coordination of care, efficiency, and
the patient experience in the primary care setting. Results, demonstrating levels of agreement with specific impact
statements, were very positive and are presented in the four bar chart figures that follow.
Figure 3: EMR Impact on Quality
*Statements have been truncated (See Appendix G for complete survey questions).
70.4 70.455.6
40.7 48.1
85.2
33.355.6
29.6 25.937
48.1 37
14.8
51.9
40.7
3.7 7.4 7.4 14.8 7.43.73.7 7.4
Enh
ance
sth
eq
ual
ity
of
care
del
iver
edto
pat
ien
ts*
Imp
rove
sth
em
anag
emen
to
fch
ron
icd
isea
ses
Imp
rove
sp
atie
nt
safe
tyan
dp
roac
tive
mo
nit
ori
ng*
Enab
les
iden
itif
icat
ion
of
pat
ien
tsfo
rch
ange
sin
man
agem
ent*
Imp
rove
sth
ed
ecis
ion
-m
akin
ge.
g.d
ecis
ion
sup
po
rtto
ols
*
Sup
po
rts
rem
ote
acce
ssan
du
sefo
rp
atie
nt
care
Enab
les
the
pra
ctic
eto
do
nee
ds
pla
nn
ing*
Enab
les
the
pra
ctic
eto
aud
itan
dim
pro
veth
ep
ract
ice
Pe
rce
nt
Agr
ee
me
nt
or
Dis
agre
em
en
t
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 22
The eight survey questions posed reflect a very broad definition of “quality”, addressing aspects that include:
Quality of care delivered
Chronic disease management
Patient safety and proactive monitoring of conditions
Clinical decision-making
Remote access
Needs-based planning
Practice improvement
Very strong levels of agreement were received in response to all of the questions posed, reflecting the significant
range of impact and benefits that providers believe may be directly or indirectly attributed to the use of EMRs. On
average, 93.0% of respondents either strongly agreed or agreed with statements presented for EMR benefits related
to quality of care.
Figure 4: EMR Impact on Communication and Coordination of Care
Questions around the impact of EMRs on communication and coordination of care were more narrowly focused on
internal sharing of information, supporting interdisciplinary care coordination and overall practice’s ability to
coordinate patient care. 100% of survey respondents agreed or strongly agreed that the use of EMRs made a
positive impact on those activities related to communication and coordination of care, indicating the tremendous
impact that EMRs can have in the clinical practice setting.
92.6 92.674.1
7.4 7.425.9
Improves the sharing of patientinformation with providers internal
to our practice
Supports interdisciplinary carecoordination in our practice
Enhances our practice’s ability tocoordinate patient care
Pe
rce
nt
Agr
ee
me
nt
or
Dis
agre
em
en
t
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 23
Figure 5: EMR Impact on Efficiency
*Statements have been truncated (See Appendix G for complete survey questions).
Nine survey questions were posed around efficiency, again reflecting a broad definition of “efficiency”, addressing
aspects that include:
Efficiency of ordering lab tests and prescriptions
Practice productivity (increased output and reduction of effort)
Time spent responding to call backs or other pharmacist requests
Availability of test results (turnaround time)
Administrative efficiencies
Preventative care incentives
Claims management processes
Management of overhead costs
Overall, 81.5% of survey respondents strongly agreed or agreed with statements presented for EMR benefits related
to efficiency of practice. Findings suggest that while the majority of respondents perceive their EMR system to
positively impact the efficiency of their practice, there is some level of disagreement around the efficiency of certain
aspects of EMR use (vs. a paper-based environment). This is most pronounced around the ability of EMR use to
positively impact claims management processes and the management of overhead costs.
63 59.344.4
33.3
77.8 70.4
44.4 44.4
14.8
29.6 33.3
33.344.4
18.518.5
44.429.6
29.6
7.4 7.418.5 18.5
7.4 3.7
7.4
18.5
3.7 3.7 3.7
7.4
3.7 7.418.5
29.6
Enh
ance
sth
eef
fici
ency
of
ord
erin
gla
bte
sts,
pre
scri
pti
on
s,et
c.
Imp
rove
sth
ep
rod
uct
ivit
y(o
utp
ut)
of
my
pra
ctic
e
Imp
rove
sth
eef
fici
ency
(red
uct
ion
inef
fort
)o
fm
yp
ract
ice
Red
uce
sth
en
um
ber
of
call
bac
ksan
d/o
rti
me
spen
tre
spo
nd
ing
top
har
mac
ist
req
ues
ts*
Red
uce
sth
eti
me
fro
mw
hen
ala
bte
stre
sult
isav
aila
ble
tow
hen
the
resu
ltis
rece
ived
by
the
EMR
Imp
rove
sad
min
istr
ativ
eef
fici
enci
ese.
g.u
seo
fp
re-p
op
ula
ted
tem
pla
tes
Faci
litat
esp
reve
nta
tive
care
ince
nti
ves
e.g.
thro
ugh
coh
ort
man
agem
ent
Imp
rove
sth
ecl
aim
ssu
bm
issi
on
pro
cess
e.g.
dec
reas
ein
bill
ing
erro
rs
Has
faci
litat
eda
red
uct
ion
inn
eto
verh
ead
cost
se.
g.sp
ace,
off
ice
sup
plie
set
c.
Pe
rce
nt
Agr
ee
me
nt
or
Dis
agre
em
en
t
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 24
Figure 6: EMR Impact on the Patient Experience
The two questions posed around patient satisfaction and patient education indicated strong levels of agreement,
with 94.4% indicating that EMRs have a positive impact on both. However, the majority of respondents simply
agreed (rather than strongly agreed) with these statements. This suggests that the large majority of respondents
perceive their EMR improves the overall patient experience, but that there could be other, and perhaps stronger,
contributing factors to patient experience.
Survey results are further referenced in support of the findings presented on the indicators and interviews below.
4.2 Indicator and Interview Results
As described in Section 3, above, the priority indicators that were identified in Phase 1 were assessed for each of the
six case study sites. Data to support the priority indicators were obtained from interviews, provider surveys and
EMR data extractions. Benefits are assessed and presented as per the five categories and indicators identified in the
EMR BR Framework Case Study Subset and in Table 2. In addition, findings and anecdotes from interviews related
to each of the five categories are presented, with selected supporting quotes from providers. A sixth category,
Medication Management, emerged in discussion with several providers during the interviews. Although there were
no indicators studied for Medication Management, the interview findings are presented.
1. Laboratory Management
2. Communication and Coordination of Care
3. Chronic Disease Management
4. Health Promotion, Screening and Prevention
5. Efficiency
6. Medication Management
37 44.4
59.3 48.1
3.7 7.4
Improves patient satisfaction/experience with thecare they receive
Supports patient education (e.g. trending patternsfor BMI, blood sugar levels etc.)P
erc
en
tA
gre
em
en
to
rD
isag
ree
me
nt
Strongly Agree Moderately Agree Moderately Disagree Strongly Disagree Don’t Know or N/A
Benefits Realization Study for EMRs in Ontario
PwC 25
4.2.1 Laboratory Management
The management of laboratory information was determined to be a key area of interest and a feasible area to assess
in the current EMR environment. Primary care settings are connected (to an extent) to community (private)
laboratories through a direct feed enabling them to receive laboratory results electronically into their EMRs. This
simulates the benefits that will be obtained as the Ontario Laboratories Information System (OLIS) is rolled out in
primary care settings. Some physicians indicated very recent connection to the OLIS but results and/or trends were
not available at this time.
The impact of EMR use on Laboratory Management was assessed through a combination of provider survey results
and interviews for one case study indicator. Results are presented in Table 3 below.
Table 3: Laboratory Management Indicator Results
ID # Indicator Definition Source Indicator Values
LM2Average time from laboratory time ofservice and test results available inEMR
Interview Pre EMR: Total turnaround ≈12 hours to 5 days
Post EMR: Real time – 24 hours
Responses from the provider survey indicated that 100% of participants felt that the EMR reduces the time from
when a laboratory test result is available to when the result is received by the EMR. However, indicator values
varied among case study sites. Some case study sites reported having laboratories physically on site. Their pre-EMR
laboratory turn around time averaged 24 hours for routine bloodwork and was generally within 12-24hours with an
EMR. As expected, the time for transmission of results was not a significant factor prior to EMR implementation as
results were commonly “walked over”. However, sites that previously relied on paper delivery of results from
community laboratories and which now receive electronic reports directly into their EMR reported a noticable
difference in the turnaround time post-EMR implementation. Paper-based results were reported to take
approximately 3-5 days, but those sites now report receiving information in “real time” when the result is ready for
distribution. For most study sites, a minimum of 50% decrease in lab result turn around time was reported with
results received electronically in their EMR.
Physicians also perceived EMRs to improve the efficiency of ordering lab tests. Findings from the provider survey
indicated that 63% of respondents strongly agreed that the EMR enhances the efficiency of ordering lab tests,
prescriptions etc. contributing to the overall management of laboratory information.
Additional Interview Findings
Some physicians reported accessing their EMRs remotely in the morning before their clinic day or on weekends to
prepare for the day or week. Consequently, test results requiring immediate follow-up were able to be addressed
sooner by scheduling the patient for a follow-up visit, making a referral, or sending a prescription to the pharmacy
(depending on EMR capability). It was also reported that some physicians are able to access lab reports on their
smart phones, making remote access even easier.
Physicians are able to access lab results in “real time” throughout the day in
many locations. Some physicians indicated that they enjoyed the ability to,
throughout the day, go through their “inbox” (the results also were in the
patient chart) and sign off results as they come in, demonstrating how some
physicians have adopted a change in workflow to manage laboratory
information. Previously, results may have been signed off in batches at the
end of the day; now, physicians can access their EMR in the clinic room, in
their offices or in hallways, enabling them to review and sign laboratory
“I access labs and chartsfrom many locationsthroughout the day. As aresult I can make fasterdecisions related to nextsteps in patient care”.
Benefits Realization Study for EMRs in Ontario
PwC 26
results from many locations throughout the day.
A few case study sites reported the use of a patient portal in conjunction with their EMRs. Some sites reported that
their patients’ experiences were improved through the sharing of laboratory results via a secure patient portal.
Physicians reported a reduction in patients’ anxiety by enabling them to check their laboratory results themselves
from home, and others indicated that their patient portal allowed them to post comments along with the results
they selected for sharing with patients. This helped patients better understand the context of the result.
4.2.2 Communication and Coordination of Care
Communication among health care providers and coordination of care were both identified as key areas of interest
for the case studies. In particular, the need was identified to better understand how the use of EMRs has impacted
both intra-office and inter-office communications. Study sites were more responsive to discussing and identifying
benefits related to intra-office communication and coordination, as the use of EMRs in supporting inter-office
communication is perceived to be in its very early stages.
Case study findings related to five defined indicators regarding communication and coordination of care are
presented in Table 4.
Table 4: Communication and Coordination of Care Indicator Results
ID # Indicator Definition Source Indicator Values
CC1Average time to receive dischargesummary following inpatient discharge Interview
Pre EMR: 4 to 14 days
Post EMR: Real time to 48 hours
CC3Average time from referral decision towhen the referral is sent Interview
Pre EMR: 1 to 7 days
Post EMR: Real time to 1 day
CC4% of practices where the care team hasaccess to and uses the EMR system
Provider Survey
Interview100%
CC5% of physicians who have remote accessto EMR and use it for patient care
Provider Survey
Interview100%
For indicators CC1 and CC3, interviewees were able to estimate a pre-implementation value depending on their
personal experiences. While those estimates of pre-implementation values ranged significantly among providers,
the reduction in times quoted were, on average, significantly lower demonstrating reductions in time spent waiting
for discharge information and/or referrals in the range of 85% or more. These reductions in time spent waiting
for discharge summaries and referrals expedite and facilitate the coordination of care such that patient needs can
be addressed in a timely fashion. For indicator CC1, physicians reported that, prior to EMR implementation,
hospital discharge summaries were available within 1-2 weeks of discharge and are now available with tools such as
the Hospital Report Manager or direct connectivity, either in real-time or within two days.
With respect to referrals and indicator CC3 specifically, physicians were able to estimate the average time to
complete a referral letter and send it to the specialist. Physicians widely reported that the referral process1 is much
more efficient with EMRs because of the pre-populated data in the referral forms. Some physicians also reported
using macros or templates to further support the process. This finding was also supported in the provider survey
1 Herein refers to as “from referral decision being made to the reference request letter completed/faxed.”
Benefits Realization Study for EMRs in Ontario
PwC 27
with 70.4% of respondents reporting that EMRs improve administrative efficiencies through the use of features
such as pre-populated templates, forms and stamps. Following the development of the referral letter, there was a
mix of workflow processes reported. Some physicians reported they sent out the referral letter themselves through
the EMR with attachments while other physicians reported consolidating the letter and supporting documentation
to send to a referral clerk/ nurse for submission and tracking. Overall, it was observed that the EMR improves
physicians’ ability to make timely referrals, expediting the care process and facilitating inter-office
communications.
For indicator CC4, 100% of practices reported that their entire care team has access to the EMR. Survey responses
further emphasized the access and importance of EMRs in providing interdisciplinary care and intra-office
communication; 92.6% of survey respondents strongly agreed that the EMR system supports interdisciplinary care
coordination in their practice. In addition, 92% of respondents strongly agreed that the EMR system improves the
sharing of patient information among providers internal to the practice.
For indicator CC5, it was evident at all sites that providers are accessing the EMR and using it remotely for patient
care purposes. 85.2% of survey respondents strongly agreed that EMRs support remote access and use of
information by providers for patient care. The ability to access patient information at any time and in any place was
noted by all as a tremendous asset to providers and their patients. EMRs also improve the ability to communicate
within clinics and often with providers outside of clinics, with overall results improving the efficiency of patient
care.
Additional Interview Findings
Interviews with clinic staff revealed many additional benefits of care coordination
from EMRs. For example, EMRs facilitate the ability of physicians to set aside
dedicated time for same/next day appointments for patients with acute needs.
Physicians and team members reported that improvements in scheduling and
practice organization facilitate their ability to hold same/next day appointments
open. In addition, EMRs provide physicians and care team members with the ability
to view everyone’s schedule to better coordinate care across team members.
With respect to communication across the team, it
was overwhelmingly reported that the illegibility of
hand-written clinical notes was formerly a challenge
when sharing information among team members. Interviewees reported that the
sharing of legible information in the EMR has improved the quality of care delivered.
Team members are now able to communicate with one another in real time, ask
questions and assign tasks among the team. The patient chart is now accessible to
many multiple care team members at the same time, a feature that was not possible
in paper.
Through the use of EMRs, most physicians reported that the information they
provided in referral letters to specialists was more comprehensive, especially for complex cases. It was also
reported that specialists appreciate the extensive clinical information provided as part of the referral and facilitated
by EMRs. Some physicians reported that they were able to quickly select attachments e.g. diagnostic test results, to
send along with the referral letter.
“EMRs improvescheduling,resulting inincreased capacityand organizationof our practice, allof which results inincreased accessfor patients”.
“EMRs improvetransfer ofinformation tospecialists - it iseasier to provideconsultations withlots of relevantinformation”.
Benefits Realization Study for EMRs in Ontario
PwC 28
Lastly, there was consensus among users of the Hospital Report
Manager (HRM) (whereby dictated and transcribed hospital reports are
transformed into a standards-based data schema that can be directly
updated into the patient’s chart in an electronic format) that it has
decreased the average time to receive hospital reports. The
implementation of HRM has reduced transcription time and
significantly decreased hospital report turnaround time. Some practices
reported receiving hospital reports directly into their EMRs within
thirty minutes of transcription. As a result, proper clinical decisions can
be made more effectively to improve the quality of care for patients.
4.2.3 Chronic Disease Management
Chronic disease management is an emerging area of interest related to EMR use. Case study sites reported that
EMRs facilitate significant and improved changes to the way chronic conditions are managed. Both quantitative
and qualitative findings were significant, with physicians expressing a desire to expand their use of EMRs for this
purpose.
The impact of EMRs on the management of patients with chronic conditions was assessed through a combination
of provider survey results, interviews and EMR data extraction. Findings related to the three defined indicators are
presented in Table 5.
Table 5: Summary Chronic Disease Management Indicator Results
ID # Indicator Definition Source Indicator Values
CDM1
% of PHC clients/patients, 18 years andover, with diabetes mellitus in whomthe last HbA1c was 7.0% or less in thelast 15 months
EMR Data ExtractionRange: 41 - 70%
CDM2
% of patient population, age 18 andolder, with diabetes mellitus whoreceived testing for all of the following: Hemoglobin A1c Full fasting lipid profile screening Nephropathy screening Foot examination Blood pressure measurement Obesity/overweight screening
EMR Data Extraction 34%
Overall, provider survey respondents agreed that EMRs improve the management of chronic diseases. Specifically,
70.4% strongly agreed with the statement and 25.9% moderately agreed. 92.5% of survey respondents also reported
that the EMR system supports patient education. For example, the EMR provides access to handouts, references or
tools to trend patters for BMI, blood sugar, and other labs. Collectively, the educational materials and tools help
patients better understand and manage their chronic conditions.
CDM1 and CDM2 were measured across all sites, with values ranging from 41-70% and 34% respectively. These
results demonstrate the ability to identify a target diabetic population and deliver appropriate, evidence-based care.
Physicians and care teams are better able to monitor specific clinical indicators such as HbA1c, and develop care
plans tailored to populations. 85.2% of provider survey respondents agreed that EMRs enable practices to perform
“Through HRM and my EMR, Ireceive reports from the localhospital as soon as they areprepared. No more calling thehospital for reports. I have theinformation immediatelyavailable to review”.
Benefits Realization Study for EMRs in Ontario
PwC 29
needs-based planning e.g. plan for a specific patient population/community. Without EMRs, this is very time
consuming, costly and a nearly impossible task.
Additional Interview Findings
It was evident in discussions with physicians and care team members that they were all motivated to better manage
the health of various patient populations. There appears to be a movement away from “chair to chair care” with an
increasing emphasis on patient populations. Sites reported that because they are easily able to identify patient
populations, they are able to proactively follow up with patients, schedule recall appointments or send them for
further testing as needed.
Nurse practitioners reported that their practices had many patients with chronic diseases, and they found that the
use of EMR tools, e.g. diabetes template/ flow sheet, was very useful when seeing diabetic patients. Relevant
information was quickly accessible and consolidated, and the flow sheet ensures that best practice guidelines are
incorporated into care delivery. Care team members are aware of what needs to be attended to during the visit,
improving efficiency and what they should be planning for in the coming months in order to manage the condition.
Most physicians and care team members reported using trending orgraphing tools as patient education materials e.g. blood pressure,weight, HbA1C were commonly trended. It was reported that the abilityto review consult notes, labs etc with patients builds confidence of careand enhances communication”. In addition, some physicians noted thatthey were beginning to communicate with patients (especially thosewith chronic conditions) via secure email.
4.2.4 Health Promotion, Screening and Prevention
Keeping Ontarians healthy through health promotion, prevention and screening initiatives is an area of benefits to
be explored through the use of EMRs. This aspect of the study is complementary to that of chronic disease
management, with a focus on targeting at risk populations to improve quality of care.
The impact of EMRs on health promotion, prevention and screening was assessed through a combination of
provider survey results, interviews and EMR data extraction. Findings related to the three defined indicators are
presented in Table 6.
Table 6: Health Promotion, Screening and Prevention Indicator Results
ID # Indicator Definition Source Indicator Values
HPSP1% practice population, age 65 andolder, who received an influenzaimmunization*
EMR Data Extract Range: 52 - 80%
HPSP2% of practice population, age 50 to 74,who had a screening test ordered forcolon cancer
EMR Data Extract Range: 51 - 60%
HPSP3
% of practice population, age 18 andolder, who have had their bloodpressure measured by their primaryhealth care provider within last 15months
EMR Data Extract Range: 61 - 82%
“For diabetes care I can trendA1C for several years – youcan discuss and show thepatient how they are doing”.
Benefits Realization Study for EMRs in Ontario
PwC 30
Overall, EMR data extraction was quite feasible for the above selection of indicators, suggesting that case study
sites have been working towards standardizing their data input in order to use the data in a meaningful way to
maintain a healthy population. For each of the three indicators, identified patients who had received appropriate
preventative treatments (as per the indicators) were greater than 50% and as high as 82%. The role of the EMR in
facilitating the identification of patients, communication with those patients, and ordering and/or documentation
of the required testing is very important. Providers indicated that these tasks are greatly enabled by EMRs, and
much more difficult to accomplish in a paper-based environment.
Provider survey results further emphasize the role of the EMR system in promotion, prevention and screening
activities, with 92.6% of survey respondents agreeing that EMRs improve patient safety and the proactive
monitoring of overdue tests/ exams. Enabling case study sites to track their patient populations through the EMR
and compare their indicator values to provincial benchmarks gives them an opportunity to identify areas for
improvement in health promotion, screening and prevention activities.
Sites are in the early stages of generating reports to proactively screen their patient populations, and priority areas
of focus can be linked to financial incentives for preventative care. It should be noted that the provincial target for
colorectal cancer screening is 40% (Health Quality Ontario, 2012) and that the values reported by all the case study
sites exceeded the provincial average. In addition, 44.4% of provider survey respondents strongly agree, while
44.4% moderately agree that EMRs facilitate preventative care incentives e.g. through cohort management for
influenza immunizations.
Additional Interview Findings
All of the case study sites reported that they were motivated to use
EMRs for population health purposes. Physicians and care teams
performed simple searches with minimal parameters to identify target
populations for disease prevention. For example, as part of its family
health team quality improvement plan, one site reported pulling lists of
eligible/ not eligible patients for activities such as cervical cancer
screening, breast cancer screening and colon cancer screening, and
sharing the list among physicians to coordinate screening tests.
Another common example discussed with interviewees was the EMR’s
ability to support efforts to increase influenza immunization rates. Not only have practices adopted processes for
targeting populations and sending reminder letters (or emails where available) but they have also developed and
tested new ways of measuring their flu vaccination rates, including a dashboard that presents vaccination rates
among patients of physician peers.
Some physicians and care team members reported that they were better able to track preventative health tests and
recalls, with the EMR generating reminders for the proactive scheduling of wellness visits. Sites also reported that
they engage patients in community wellness talks through identification of those in target populations, and speak to
them about prevention and promotion topics relevant to their cohort. EMRs afford providers the ability to be very
proactive and interactive with patients in managing their own health.
“EMRs provide the ability tosurvey your practice to findpeople who are due forpreventative procedures orcare, including finding allthe people who have not hada pneumovax.”
Benefits Realization Study for EMRs in Ontario
PwC 31
4.2.5 Efficiency
Obtaining a better understanding of the overall practice efficiencies realized through the use of EMRs was a focus
for the case studies. As discussions evolved, it became evident that practice and provider efficiency was a recurring
theme that was applicable to several other discussion categories as well. Consequently, results are presented for
one indicator only in this section, but findings about efficiencies created are also referenced elsewhere in this
report.
Table 7: Summary Efficiency Indicator Findings
ID # Indicator Definition Source Indicator Values
E1
% change in time to completeclinical/admin documentation by usingpre-populated templates, forms andstamps
Interview ~ 50% decrease
Overall, 77.7% of survey respondents agreed with the statement that EMRs “improve the efficiency (reduction in
effort) of my practice” and of those 44.4% strongly agreed with the statement while 33.3% moderately agreed.
These survey responses mirror the comments collected during the interviews on overall practice efficiencies.
With respect to clinical and administrative documentation (indicator E1), physicians were able to report overall
that there have been efficiency changes related to certain activities. For example, 92.6% of survey respondents
agreed with the statement that EMRs “enhance the efficiency of ordering lab tests, prescriptions, etc” (63.0%
strongly agree and 29.6% moderately agree). During interviews, physicians reported that the auto-population of
templates (where appropriate, e.g. lab requisition form, s.o.a.p. format encounter notes and others) was an added
benefit and saved time for themselves and their administrative staff in completing fields such as demographics.
For indicator E1, forms (including templates with the ability to auto-populate existing EMR data) and stamps (e.g.
s.o.a.p note or diabetic assessment template used to capture information) were acknowledged as a benefit to clinical
and administrative documentation. However, physicians and care team members found it difficult to quantitatively
assess the impact, with most estimating the time savings at 50% when compared to a paper-based environment.
Survey results support the benefit of EMRs in administrative efficiencies with 70.4% of providers perceiving EMRs
to improve administrative efficiencies such as through the use of pre-populated templates, forms and stamps.
Additional Interview Findings
In addition to the discussions around the above indicator, there was significant
discussion on the efficiencies gained (or transferred to other activities) related
to EMR use. There was a general consensus from interviewees that EMR
features such as reminders allow for more efficient clinical encounters (e.g.
patient physical exams), and not surprisingly, physicians and care team
members reported that their time is not spent “flipping through papers” or
searching for misfiled results. Instead, clinicians reported being able to address
their list of planned activities such as reminding patients they are due for certain tests, procedures, medication
refills, etc.
“I have more efficientclinical encounters.The time is spent onmore important thingsrather than writingout prescriptionrenewals etc.”
Benefits Realization Study for EMRs in Ontario
PwC 32
Nurses and nurse practitioners frequently commented that they would call patients with test results or to discuss
care plans. They all reported that being able to instantaneously access the patient file while on the call allowed them
to have more detailed conversations with patients, providing greater confidence and support to patients.
Nurses also reported efficiencies in workflow related to patient arrival at the practice. For example, front desk staff
can check-in the patient which will appear in the schedule as “patient in waiting room”. Following the completion
of initial assessments, the nurses will then change the patient status in the schedule so that the physician knows the
patient is ready to be seen; this avoids duplication of work among providers and improves the efficiency of the
patient visit. In addition, physicians can view their clinical schedules in real time and be able to utilize their “break”
time more efficiently to dictate notes, clean up their inbox messages, review next patient’s files and other clinical
duties.
4.2.6 Medication Management
Although medication management was not formally identified as an area for study during case study site visits,
interviewees and survey data provided information on how EMRs have impacted the management of medication
for their patients. EMR benefits were consistently reported related areas such as medication recalls and medication
history. Other processes such as ePrescribing were “gray areas” of discussion and varied among practices.
Physicians and care team members reported an important patient
safety benefit of the EMR is the ability to quickly extract patient
lists for specific medications that have been recalled. Survey
results suggest that 88.8% of respondents agree that EMRs
enable practices to identify patients for changes in medication
based on new evidence such as drug recalls.
Physicians reported that they do not spend time “flipping through the chart” to look at historical information. The
medication history is in one place (i.e. CPP) in the EMR, and is up to date. With the EMR, physicians reported
being better able to track current and expired prescriptions and respond to requests with a comprehensive patient
history at their disposal.
Physicians reported that their overall workflow to prescribe medications and
refill prescriptions has evolved more efficiently. For some sites, EMR medication
alerts and reminders were used and found to be beneficial and efficient; however,
other sites reported that their medication alerts and reminders were not reliable.
In some instances, community pharmacies are provided with secure access to
EMRs. The sharing of medication information across the care team supports
improved patient safety reducing potential call backs to the physicians for
clarifications in dose or instructions.
Provider survey results report that 63% of respondents strongly agree that EMRs
enhance the efficiency of tasks such as the ordering of lab tests and renewing prescriptions. 77.7% of respondents
perceive the EMR to reduce the number of call backs and or time spent responding to pharmacist requests for
physician verification of prescriptions. In advance of the development and delivery of a provincial Medication
Management System, interviewees reported using a form of electronic prescribing or “ePrescribing” and electronic
prescription renewals facilitated by their EMRs. Some practices reported renewing prescriptions and sending them
directly through the EMR to pharmacies which improved the prescription renewal turnaround time. Other
practices reported using the EMR to document and print prescriptions. The legibility of electronic prescriptions
was reported to be an important benefit related to patient safety.
“the EMR can be used to find all thepatients on this medication inorder to alert them to the warningor change medications if needed”.
“Prescriptions arefacilitated with directfax to inbox andcomputer fax topharmacy, no papergenerated, less time,and fewer errors”.
Benefits Realization Study for EMRs in Ontario
PwC 33
5. Modeling and Forecasting
A model was developed to forecast the potential benefits of EMR use in Ontario based on similar levels of EMR use
as evidenced by the advanced users of EMRs assessed in the six case study practices. Benefits were extrapolated
from indicators collected through the case studies, and supported with peer reviewed literature. Where possible,
benefits are estimated on a population basis and forecasted to 2017 and 2022.
It is acknowledged that the approach to modeling and forecasting of benefits was limited by the sample size (six
case study sites), the ability to attribute benefits to EMR use and to generalize those findings across the province.
In addition, there is limited evidence in the literature for many of the selected indicators. However, the modeling
exercise is intended to demonstrate the “art of the possible” and to highlight the benefits that can accrue based on
mature adoption and use of EMRs across Ontario.
This section provides modeling and forecasting of benefits in relation to the following four indicators which relate
to three of the categories presented in the previous section. Forecasted benefits are further discussed in section 6,
Discussion.
Category 3: Chronic Disease Management
Diabetes management
Category 4: Health Promotion, Screening and Prevention
Increased influenza immunization rates Increased colon cancer screening rates
Category 5: Efficiency
Staff time reduction spent on administrative tasks
5.1 Diabetes Management
Costs for managing chronic diseases represent 58% of all annual health care spending in Canada, at a cost of $68
billion a year, and is growing. In addition, the indirect costs associated with income and productivity loss are
estimated at $122 billion, or nearly double the health care costs (Public Health Agency of Canada, 2011). The cost of
diabetes alone was estimated at approximately $12.2 billion in 2010 and is expected to rise by another $4.7 billion
by 2020 (Canadian Diabetes Association, 2009). Ontario is one of the provinces with the highest growth rates in
diabetes, and its population with diabetes is expected to quadruple over 20 years (Canadian Diabetes Association,
2009).
According to the Auditor General of Ontario, prevention is the most cost-effective strategy for coping with chronic
diseases. At present, evidence is largely related to the management of chronic disease, although some instances of
EMR benefits related to prevention have been noted, with findings suggesting that EMRs help identify patients at
risk for undiagnosed type 2 diabetes (Klein Woolthuis et al, 2007). Diabetes complications account for 69% of limb
amputations, 53% of kidney dialysis and transplants, 39% of heart attacks and 35% of strokes which are associated
with significant costs (Auditor General Report, 2012).
Benefit Model
This section estimates the potential avoided costs from reduced amputations, diabetic kidney disease and
macrovascular complications (e.g. heart attack and stroke) as a result of improved diabetes management by
Benefits Realization Study for EMRs in Ontario
PwC 34
leveraging EMR use. It should be noted that avoided costs are measured in isolation for survivors. Also, costs of
managing the disease are not included.
The case studies observed that the highest performing practices had a proportion of patients with diabetes mellitus
in whom the last HbA1c was 7.0% or less at sites visited was 70%. As a provincial comparison, a study completed in
2003 found that, among a random sample of non-academic family physicians, less than 30% of diabetic patients
had HbA1c levels at or under 7% (Harris et al, 2003). The value of 30% was used as the Ontario average for diabetic
control.
The adherence to HbA1c target levels was used as a proxy for mid- to long-term diabetic control. The probability of
complications was calculated for patients in diabetic control, then for patients otherwise. These probabilities were
then multiplied according to the mix of patients in both scenarios (i.e. 70% diabetic control for advanced EMR
users, and 30% diabetic control for Ontario average).
The probabilities of complications were based on a Manitoba Health study (Manitoba Health, 2012). In terms of
lower limb amputations, those in diabetic control have a 2.6% chance of developing peripheral neuropathy followed
by a 0.5% chance of requiring amputation given their neuropathy condition. Patients off control face a 4.6% chance
of developing neuropathy followed by a 2.8% probability of needing amputation.
In terms of diabetic kidney disease, patients in diabetic control have a 2% chance of developing microalbuminuria,
with an 8% probability that this will evolve into macroalbuminuria. There is then less than 1% chance that that will
lead to end stage renal disease (ESRD). Alternatively, patients who are not in diabetic control have an 8% chance of
developing microalbuminuria, a 17% chance that that will lead to macroalbuminuria and a 5% probability that this
will further evolve into ESRD.
The likelihood of macrovascular complications, myocardial infarction (MI) or stroke, increases with the number of
years the patient has had diabetes. On average, Canadians with diabetes have had the disease for nine years (ICES,
2005). Therefore, the risk of MI and stroke after 9 years from the Manitoba Health study was used. Patients in
diabetic control face a 1% risk of MI compared to 4% for patients not in control. Similarly, patients in diabetic
control face a 0.5% risk of MI compared to 1% for patients not in control.
The probability for each complication was multiplied by the number of diabetics in Ontario in order to obtain the
number of complications. The projected number of diabetics in Canada in 2020 was obtained from the Canadian
Diabetes Association and estimates for 2017 and 2022 were extrapolated. The number of diabetics in Ontario was
estimated as a percentage of Canadian diabetics, assuming an equal share to total population (i.e. 38%).
The cost of each complication was taken as the average cost as reported by Manitoba Health, representing a proxy
for the costs of similar complications in Ontario.
Assuming current state whereby 30% of the population is in diabetic control, on an annual basis it is estimated that
diabetes leads to:
1,170 lower limb amputations;
611 cases of ESRD;
43,000 MI; and
Average % diabeticpatients with
HbA1C<7%: 30%Max rate in sites
visited: 70%
Reduced risk of footamputation, kidneydisease, MI and stroke
Number of diabeticsprojected in Ontario in
2017: 1.3 MProjected in 2022: 1.5 M
Benefit = Increased number ofdiabetics in control * avoided
cost per complication perpatient in diabetic control
Diabetes management:
Benefits Realization Study for EMRs in Ontario
PwC 35
11,200 stroke cases.
Assuming a future state whereby 70% of the population is in diabetic control, annual occurrences would be reduced
to:
604 amputations, or 566 fewer amputations at $30,000 each;
270 cases of ESRD, or 341 fewer cases at $70,000 each;
25,600 MI, or 17,400 fewer cases at $2,600 each;
8,100 stroke cases, or 3,100 fewer cases at $12,700 each.
Based on the modeling exercise described above, it was estimated that proactive management of diabetes can
potentially save the health system $17 million in amputation costs, $26 million in ESRD treatment costs, $44
million in MI costs and $38 million in stroke costs (cost estimates are on an annual basis). Projecting another five
years forward results in further cost avoidance. Effective use of EMRs and other eHealth platforms could enable
providers to actively monitor their patients’ conditions to prevent their health from deteriorating. Without EMR
and related enablers, it would very challenging to achieve those goals. Results are shown in Figure 7 below.
Figure 7: Annual potentially avoided costs of select diabetes complications
5.2 Increased Influenza Immunization Rates
It is estimated that influenza costs the Canadian economy about $1 billion every year. This includes both health
care system costs as well as lost productivity, with approximately 1.5 million workdays lost annually (Canadian
Healthcare Influenza Immunization Network, 2009).
Effective use of EMRs has the ability to improve immunization rates by helping identify patients in need of specific
immunizations and providing reminders to physicians (and potentially patients) for updates. Some EMRs integrate
evidence-based recommendations for vaccines using patient demographic and clinical data such as gender, age and
family history to target patients requiring immunization. Others may also provide reminders to offer or review
0
20
40
60
80
100
120
140
160
180
2017 2022
$M
illio
ns MI
Stroke
Diabetic kidney disease
Foot amputations
Benefits Realization Study for EMRs in Ontario
PwC 36
immunizations during routine visits or provide reminders for patients to schedule care. Reminders to patients
generated by EMR systems have been shown to increase patient compliance with preventative care
recommendations (Hillestad et al., 2005).
Benefit Model
The estimated benefits of avoided costs to the health care system from increases in rate of influenza immunizationin the population over age of 65 are presented.
Among the high performing EMR case study sites visited, it was observed that a maximum value of 80% of patients
over the age of 65 had received influenza immunization. Comparatively, 64% of Canadians over the age of 65 are up
to date on their immunization vaccination as reported by Statistics Canada; this rate of immunization is used as the
baseline for comparative purposes
The model for estimating the benefits associated with increased rates of influenza immunization was based on a
model developed by the Canadian Health Services Research Foundation (CHSRF) (Dahrouge et al., 2012). It
models the difference in the number and costs of General Practitioner (GP), Emergency Department (ED) and
hospital visits for vaccinated patients compared to unvaccinated patients, resulting from prevented flu episodes as a
result of the vaccination. Based on the findings of Gross et al. (1995), CHSRF estimated that individuals are 50%
less likely to develop an influenza-like illness if vaccinated. This estimate was then used in combination with the
probability of episodes irrespective of vaccination status to obtain risks of GP visits, ED visits and hospital visits for
vaccinated and unvaccinated patients. The costs associated with each type of visit as well as vaccination were
obtained from various sources (as per the CHSRF study, Dahrouge et al., 2012).
The model was modified to incorporate the case study sites’ maximum rate of immunization (i.e. 80%), as well as
adapt the target population to individuals over age 65 in Ontario only. The projected target population was
obtained from the Ontario Ministry of Finance Projections for 2017 and 2022.
Compared to the baseline population (vaccinated at 65%), and as per the modeling exercise described above, it was
estimated that the effective use of EMRs could support a reduction in the number of family physician visits by
over 20,000, hospitals visits by over 7,000 and ED visits by over 9,000 in 2017. This would be associated with
avoided costs totalling $40.6 million in 2017. Projecting another five years forward, cost reductions of $48.5
million to the health care system could be realized in 2022. Results are presented in Figure 8 below.
Averageimmunization rate in
Canada: 64%Max rate in sites
visited: 80%
Immunization leadsto a 12% reduction
in GP visits, EDvisits, and hospital
visits
Cost of GP visit: $35,ED visit: $220,hospital visit: $6,417Ontario projectedtarget population in2017:2.4 M, in 2022:2.8 M
Benefit = (Vaccinated pop postEMR– Vaccinated pop base-Line)* (Cost of vaccinated
pop) + (Unvaccinated pop postEMR– Unvaccinated pop base-Line)* (Cost of unvaccinated
pop)
Influenza immunization:
Benefits Realization Study for EMRs in Ontario
PwC 37
Figure 8: Annual avoided costs from increased influenza immunization
5.3 Increased Colon Cancer Screening Rates
The costs for managing chronic disease represents 58% of all annual health care spending in Canada (Public Health
Agency of Canada, 2011), and it has been shown that there are opportunities to improve the care delivered by
enhancing provider decision support with the provision of more timely and comprehensive patient information.
Therefore, disease prevention is an area of potentially high leverage for EMRs to improve health care (Hillestad et
al., 2005).
EMRs can support disease prevention by assisting clinicians in (Hillestad et al., 2005):
Identifying people with a potential chronic disease (e.g. through predictive-modeling algorithms);
Improving screening and testing by tracking the frequency of preventative services and reminding
physicians to offer needed tests;
Distributing reminders to patients and modifying patient behaviour through web-based education; and
Adjusting preventative therapy (e.g. based on the use of easily accessible and regularly updated guidelines).
Benefit model
This section of the study focuses on estimating a subset of the benefits of EMRs associated with preventative care,
specifically the increase in colon cancer screening. The benefit is measured as the number of patients who are
diagnosed at earlier stages of cancer rather than later stages, leading to higher quality of life and avoided costs of
cancer treatment.
0
10
20
30
40
50
60
2017 2022
$M
illio
ns
Annual avoided costs fromincreased influenza immunization
Benefits Realization Study for EMRs in Ontario
PwC 38
Note: ∆% = percentage change
Based on Cancer Care Ontario statistics from 2010, it is estimated that 53% of Ontarians between the age of 50 and
74 are screened according to guidelines for colon cancer through fecal occult blood test (FOBT) within the past 24
months, sigmoidoscopy within the past five years, or colonoscopy within the past ten years. In comparison, the
maximum rate reported by sites visited was 60%. This represents an increase in screening rate of 7% that is
potentially facilitated through advanced EMR use.
Individuals who are not screened are more likely to be diagnosed once they show cancer symptoms, which are likely
to be associated with later stages of cancer. For patients who are screened, a positive FOBT will lead to a
colonoscopy which, if abnormal, is more likely to be associated with earlier stages of cancer. The probabilities of
being diagnosed with colon cancer, at various stages, were estimated for individuals who are not screened and
individuals who receive an FOBT every two years, and these probabilities are used for modelling purposes.
Organized colorectal cancer screening programs in Canada most commonly use fecal tests as the primary screening
tests for individuals aged 50 to 74 with average risk (Canadian Partnership Against Cancer, 2009) and therefore
this type of test was assumed for modelling purposes. Probabilities of various stages of cancer being diagnosed were
based on evidence from the Manitoba study (Manitoba Health, 2012) and have been applied to the Ontario
population for the purposes of this study. These probabilities were then multiplied according to the mix of patients
in both scenarios (i.e. 60% screening / 40% no screening for patients of advanced EMR users and 53% screening /
47% no screening for Ontario average) to obtain the change in the number of patients diagnosed at each stage of
cancer through advanced EMR use. These estimates were then were multiplied by the stage-specific cost of treating
colon cancer obtained in the literature (Telford, 2010).
The probabilities of various stages of cancer occurring in the absence of FOBT (Manitoba Health, 2012) are the
following:
0.5% chance of showing cancer symptoms
85% chance that, given symptoms, it is not stage 1 cancer
Given it is not stage 1, 42% chance it is stage 2
Given 58% chance it is not stage 2, 56% chance it is stage 3 and 44% chance it is stage 4
The probabilities associated with FOBT screening (Manitoba Health, 2012) are the following:
3% chance of positive FOBT
Given FOBT, 58% chance of abnormal colonoscopy
Given abnormal result, 83% chance it is stage 0
Given 17% chance it is not stage 0, 31% chance it is stage 1
Given 69% chance it is not stage 1, 46% chance it is stage 2
Given 54% chance it is not stage 2, 65% chance it is stage 3 and 35% chance it is stage 4
The probability for each cancer stage was multiplied by the number of individuals in Ontario eligible for screening
(the target population). The target population in Ontario in 2017 and 2022 was obtained from the Ontario Ministry
of Finance population projections.
Average adherence tocolon cancer screening
guidelines: 53%Max rate in sites
visited: 60%
Reduced chance ofdeveloping stage 2,stage 3 and stage 4cancer
Ontario projectedtarget population in2017: 4.3 MProjected in 2022:4.6M
Benefit = (∆% stage 1 + ∆ % stage 2 + ∆ % stage 3 + ∆ %
stage 4)*Ontario targetpopulation
Colon cancer screening:
Benefits Realization Study for EMRs in Ontario
PwC 39
Based on the modeling exercise described above, it was estimated that the optimal use of EMRs (with a maximum
screening rate of 60%) could potentially lead to:
220 fewer Ontarians diagnosed with stage 4 cancer;
217 fewer Ontarians diagnosed with stage 3 cancer;
267 fewer Ontarians diagnosed with stage 2 cancer;
703 additional Ontarians with no cancer or diagnosed with stage 0 or stage 1 cancer.
Costs of treatment are estimated to be the following: $15,537 for treating Stage 1, $30,505 for treating Stage 2,
$39,176 for treating Stage 3, and $99,115 for treating Stage 4 and terminal care (Telford et al, 2010). Multiplying
the number of cases with their respective costs leads to avoided treatment costs of $38 million in 2017 and $40
million in 2022. However, it should be noted that this estimate includes the cost of treatment only and ignores the
costs of testing and the costs associated with keeping patients alive. Therefore these estimates should not be
interpreted as overall cost savings. Results are presented in Figure 9 below.
Figure 9: Annual avoided costs from increased influenza immunization
5.4 Staff Time Reduction Spent on Administrative Tasks
EMRs have the ability to reduce staff time spent on administrative tasks by different members of the inter-
professional team. These tasks include:
Pulling charts. As EMRs reduce or eliminate the need to maintain paper patient files, office staff do not
need to retrieve or re-file paper charts for office visits or other transactions, and time wasted looking for
misplaced charts is eliminated (Girosi et al., 2005).
Managing laboratory results. Benefits of EMR use can also include faster, more accurate lab order entry,
accurate matching of lab results to charts, correct routing of results to the ordering provider, auto-
completion of EMR lab order status, and validation of complete insurance information (Wolfram et al.,
2009).
0
5
10
15
20
25
30
35
40
45
2017 2022
$M
illio
ns
Annual avoided colon cancertreatment costs
Benefits Realization Study for EMRs in Ontario
PwC 40
Scheduling. Although electronic scheduling does not necessarily require an EMR, EMRs enable scheduling
features to connect to patient information.
Billing. EMRs enable the capture of billable events as they happen clinically, reducing the chance of
missing billable activities and providing a thorough clinical documentation record. EMRs also generate
diagnostic and billing codes from plain language which can reduce the time spent looking up codes.
Writing in charts. Computerized data entry can reduce time spent recording patient and visit information
through shortcuts such as drop down lists and other automation such as voice dictation.
Order entry. EMRs reduce the time required to complete all fields to transmit, print, or otherwise complete
the process of ordering laboratory testing and radiology tests. EMRs can also produce downstream time
savings resulting from more complete order requests or improved legibility (Johnston et al., 2003).
Encounter management. EMRs provide increased accessibility to information required during a patient
encounter.
Benefits associated with a reduction in time spent on administrative tasks such as those described above are
defined in the benefits model and forecasting that follows.
Benefit model
Case study sites reported large time savings in performing administrative tasks, notably related to time spent with
documentation, pulling charts and scheduling. Maximum time savings were in the range of 22.5 hours/week. It
should be noted that time savings can be utilized in various ways, including redeploying staff to various non-
administrative tasks such as patient care and chronic disease management as well as reducing staff. The maximum
value of 22.5 hours was extrapolated to obtain an average number of hours saved annually per practice.
In order to estimate the number of administrative staff hours that could be redeployed per practice in a year, this
estimate was multiplied by the number of hours in a year. This estimate was further multiplied by the number of
primary care physicians in Ontario then divided by the average practice size (i.e. number of physicians per practice)
in order to obtain an estimate of redeployed hours for the province as a whole. The number of primary care
physicians (15,845) was obtained from Statistics Canada and the average practice size was assumed from data
collected on site. Therefore, the average practice size was estimated at 10 physicians. The number of physicians was
inflated by Ontario population growth to obtain estimates for 2017 and 2022.
Based on the modeling exercise described above, the annual redeployed hours were estimated to be nearly 1.8
million hours in 2017, representing 2.4 hours per physician.
The hours saved were converted to a dollar figure by multiplying hours saved by cost per hour of administrative
staff. It should be emphasized that this is not necessarily a cost saving but rather an estimate of the value of time
that could be redeployed to other clinical activities. The cost per hour of administrative work was obtained by
multiplying the number of medical secretaries (as per Statistics Canada) and the number of nurses working in
primary care offices (as per CIHI) by their respective salaries (as per Statistics Canada).
$73,000
Maximum time reductionon admin. tasks in
sites visited: 22.5hours per week
Number of PCP: 16,000Average medicalsecretary salary: $33,000
Average RN salary:$73,000
Benefit = Annual redeployed hours* cost per hour
Administrative time reduction
Benefits Realization Study for EMRs in Ontario
PwC 41
The annual benefit was estimated at $40 million in 2017 and $42 million in 2022, providing increased capacity for
administrative staff, who could potentially assume a variety of practice roles, including additional administrative
tasks currently assumed by clinical staff in the primary care setting, freeing those staff to perform patient care
activities needed elsewhere. Results are shown in Figure 10 below.
Figure 10: Annual benefits of administrative staff redeployment
5.4 Summary Findings
A model was developed to forecast potential benefits of EMR use in Ontario five and ten years from now if allproviders were adopting and using EMRs at the same level as the advanced EMR users in the case study sites.Benefits were extrapolated from indicators collected through the case studies, and supported with peer reviewedliterature.
Four indicators were selected from the BR framework for forecasting based on the availability of data for eachindicator at the various sites as well as the availability of supporting evidence to extrapolate these indicators tobenefits for Ontario. Each indicator along with its associated benefit and benefit estimate is provided in Table 8below. Benefits were measured in terms of financial impact. The financial impacts represent monetary benefitsarising from certain aspects of advanced EMR use and should not be interpreted as overall cost savings to thesystem (e.g. the cost of diabetes complications are only one aspect of the costs and benefits associated withdiabetes).
Although the selected benefits provide only a portion of benefits expected from EMRs, they indicate substantialpotential benefits to be realized if all providers in the province were to become high performing users of EMRs in 5– 10 years.
0
5
10
15
20
25
30
35
40
45
2017 2022
$M
illio
ns
Annual benefit of redeployedstaff
Benefits Realization Study for EMRs in Ontario
PwC 42
Table 8: Summary of Forecasted Benefits
Indicator Qualitative Annual Benefits Estimated Annual FinancialImpact in 2017
Percentage of patients, 18 years andover, with diabetes mellitus in whomthe last HbA1c was 7.0% or less (orequivalent test/reference rangedepending on local laboratory) in thelast 15 months
Patient and health system benefitsfrom management of complicationsand co-morbid conditions arisingfrom diabetes, including:
- 566 fewer foot amputations- 341 fewer cases of ESRD- 17,400 fewer MI cases- 3,100 fewer stroke cases
$125 million
($17 M from reduced footamputations, $26 M fromreduced kidney disease, $44Mfrom fewer MI, $38M fromfewer strokes)
Percentage of patient population, age65 and older, who received aninfluenza immunization
Patient and health system benefitsrelated to illness prevention,avoidance of clinic and/or hospitalvisits
$40.6 M in reduced health caresystem costs
Percentage of practice population,age 50 to 74, who had a screeningtest ordered for colon cancer
Identification and diagnosis ofcancers and malignancies at earlystages for improved prognosis andquality of life for patients
- 220 fewer Ontarians diagnosedwith stage 4 cancer;
- 217 fewer Ontarians diagnosedwith stage 3 cancer;
- 267 fewer Ontarians diagnosedwith stage 2 cancer;
- 703 additional Ontarians with nocancer or stage 0/stage 1 cancers
$38 M in reduced colon cancertreatment costs
Percentage reduction in time spenton administrative tasks
Increased staff capacity through areduction of 1.4 million hours thatcan be allocated to other importanttasks, including the provision ofpatient care
$40.0 M of increased staffcapacity
Benefits Realization Study for EMRs in Ontario
PwC 43
6. DiscussionIt is widely understood that the benefits of local, provincial and national investments in eHealth and EHR
initiatives can make a significant impact on patients, providers, and the health system itself take time to mature
and realize their full potential following their initial implementation. As a fundamental component of the planned
EHR to be implemented for all Ontarians and a major focus for investment, EMRs are no exception. With
penetration and adoption of EMRs occurring at differing paces by many different providers, different types and
levels of benefits are accrued over time to many stakeholders in the system.
Although the case study approach has some limitations, this study has demonstrated that Ontario’s investments in
EMRs to date have yielded noteworthy, tangible benefits. The benefits are diverse, and have accrued to patients,
providers and the broader health system. Case study participants provided a unique vantage point to convey
valuable insights into benefits from their first-hand experience in pioneering and optimizing the use of a variety of
EMR tools and functionalities across their patient care teams.
While the earliest benefits have been shown to be realized by providers and their patients, modeling and forecasting
of case study findings has demonstrated that continued investment in EMRs across the province can augment the
impact of EMR use on the broader health system, with the potential for EMRs to have very significant direct and
indirect impacts.
While it is acknowledged that the potential benefits of EMRs are broad and are not limited to those assessed, the
current study focused on six core areas that are discussed in alignment with the categories of findings presented in
Section 4, Case Study Results, as follows:
1. Laboratory Management
2. Communication and Coordination of Care
3. Chronic Disease Management
4. Health Promotion, Screening and Prevention
5. Efficiency
6. Medication Management
6.1 Laboratory Management
The ability to access, review and make decisions related to the timely availability of lab results in EMRs was
reported as one of the earliest and most demonstrable benefits associated with EMR use by case study sites.
EMRs provide clinicians with more timely access to laboratory information, aiding care decisions.
The ability to receive lab results through direct transmission to EMRs has reduced the time to receive those results
by 50% on average (compared to a paper-based environment). Consequently, clinicians are now well enabled to
respond to results quickly and effectively. This timely access to test results, while either in their clinics or from
remote locations, affords the opportunity to make timely clinical decisions, provide prompt and appropriate care,
expedite referrals, and improve patients’ access to care. In addition, by having results readily accessible within
EMRs, other clinicians involved in the patient’s care are less likely to order duplicate tests. Although not
specifically measured, the ability to address and attend to critical laboratory results requiring immediate attention
Benefits Realization Study for EMRs in Ontario
PwC 44
may contribute to downstream effects, including better management of patients with chronic conditions and
avoided unnecessary visits to emergency departments.
While the majority of results made available electronically into EMRs are from private community-based
laboratories (e.g., Gamma-Dynacare, CML, LifeLabs, Alpha Global), results from OLIS are being made increasingly
available as the province continues to invest in connecting OLIS to primary care providers through EMRs. In
addition to results from private laboratories, OLIS enables clinicians to search and download lab results from
connected hospitals and public health laboratories. The value of having comprehensive lab information in a timely
manner through all potential sources of lab information provides a comprehensive profile for patients and forms a
basis for clinical decision-making. The benefits of OLIS connectivity to EMRs will continue to be realized over time
as integration of systems continues. It is encougaging that eHealth Ontario has made significant progresses to
connect OLIS with EMRs, which will add significant value to EMR users in community care settings.
Ready access to timely laboratory results through EMRs helps to enhance the patient experience.
While providers benefit from the ability to quickly access lab test results, patients can also benefit from having this
information readily available to them, either directly through their providers or through online portal offerings that
can be linked to EMRs. Providers are better able to address any questions patients may have around their lab test
results, and share those results with patients. With the increasing needs to engage patients in the delivery of
quality care, patients and/or proxies will need to access their diagnostic results in “real-time”, allowing them to
review lab reports when and where it is convenient for them. Importantly, eHealth Ontario has funded some pilot
projects to enable “patient portals” tethered through EMRs. These initiatives can improve patients’ experiences
with their care, and provide them with some comfort, understanding and ownership in care processes.
6.2 Communication and Coordination of Care
EMRs play an instrumental role in improving the intra- and inter-office communication and coordination of care.
Associated benefits are realized fairly early in the adoption of EMRs, providing access to the patient record for all
members within a practice, and facilitating improved sharing of information with external providers as well. These
capabilities contribute to improved access to care, quality of care and efficiency as discussed below.
EMRs facilitate improved scheduling and coordination of patient visits, improving access to care.
EMRs provide physicians and care team members with the improved ability to schedule patient visits, improving
patients’ access to care and efficiency of the care team. With 100% of care team members having access to the EMR
and 96% of respondents reporting that EMRs support interdisciplinary coordination of care, care team members
are able to coordinate a patient’s care delivery much more effectively and efficiently. Physicians reported the ability
to plan their day more effectively, and patients benefit through avoiding repeat and/or unnecessary visits to
multiple providers in the care team.
Physicians and team members reported that improvements in scheduling and organization facilitate their ability to
hold same day appointments open, improving access to care. Ontario’s Health Action Plan (Ministry of Health and
Long-Term Care, 2012) identifies improved availability of same-day appointments as a key focus in improving
access to primary care. Access to same day care in the primary care setting may reduce visits to emergency rooms
for non-emergent purposes, improving access to care across the health system.
EMRs improve the availability and sharing of information among interdisciplinary team members
and enhances quality and efficiency of care.
Benefits Realization Study for EMRs in Ontario
PwC 45
The EMR allows each care team member to access the chart as required and for most, this access is also possible
from remote locations. An informed and coordinated care team supports enhanced care outcomes, promotes a
positive care experience for patients, and may produce innovative ways of addressing care planning. The ability to
access patient information at any time and in any place was noted by all as a tremendous asset to providers and
their patients.
EMRs also facilitate the ability to communicate within clinics and often with providers outside of clinics, overall
improving the efficiency of patient care. Quality of care, and the patient experience overall, is improved for patients
by ensuring all providers have ready access to the same patient information. For example, patients must no longer
share essential details (e.g., symptoms, medications) with multiple providers as the information is captured in a
legible format in one location that can be accessed by all.
EMRs facilitate the sharing of information with specialists, thereby improving the continuity and
efficiency of care.
A much touted benefit of EMRs by interviewees is the ability to make more efficient referrals and to provide
specialists with more comprehensive patient information. With the ease and improved efficiency of making
referrals (an approximate reduction in time of 85% to make a referral), the EMR allows primary care clinicians to
quickly provide the specialist with key pieces of information needed to understand the patient’s condition. In
addition, the need for specialists to spend time seeking additional administrative and/or clinical information is
greatly reduced. As a result, the referral process is expedited and specialist is better informed when making
decisions on treatment, and may avoid any duplication in diagnostics or other treatments at a later date.
In addition to the above, EMRs that interface with other locally developed software solutions (e.g., SharePoint) can
support the improved tracking of referrals, with the ability to better plan care and help patients to understand
expected wait times for specialist care. It is, however, noted that there is no synchronized eReferral platform across
the province. Referring processes still largely rely on the provider’s relationships without adequate transparency
into required information (e.g., wait times). It is recommended that eHealth Ontario further explore strategic
opportunities to improve referral processes and further enhance the value of EMRs. In the future, integration
between EMRs and planned eReferral solutions will better enable this on a province-wide basis.
Expedited delivery of hospital reports to EMRs facilitates timely and appropriate care.
Case study sites participating in the Hospital Report Manager pilot study, whereby reports are delivered directly
from hospitals to EMRs, reported a reduction of 85% in time spent waiting for discharge summaries as a result of
the direct transfer of this information to EMRs. The timely access to this information can further reduce patients’
wait times for required post-discharge care, and allow providers to put appropriate follow-up care into place in the
out-patient setting. Notably, by providing primary care clinicians with the necessary information to monitor and
maintain their patients’ care following discharge from hospitals, they are better able to reconcile medications and
prevent unnecessary hospital readmissions and ED visits.
6.3 Chronic Disease Management
Findings of this study suggest that EMRs have demonstrated significant benefits and continue to hold tremendous
potential to facilitate the management of chronic diseases in Ontario, a key priority identified by the province and
for eHealth Ontario. EMRs are being used increasingly by clinicians to manage the health of patient populations,
such as patients with diabetes, chronic obstructive pulmonary disease (COPD)/asthma, and others. Physicians and
their care teams are increasingly relying on their EMRs to effectively manage the care of patients with chronic
conditions that are costly to the system overall and consume significant resources within the health system.
Benefits Realization Study for EMRs in Ontario
PwC 46
EMRs are a necessary and effective tool to manage the health of defined patient populations.
Case study sites were overwhelmingly positive in their support for the use of EMRs in managing the health of their
patients with chronic diseases. 96% of survey respondents indicated that EMRs support the management of
chronic diseases, and 92% agreed that they support patient education for disease management purposes. Hillestad
et al. (2005) reported that EMRs can support chronic disease prevention and management by assisting clinicians in
identifying people with active or potential chronic disease, targeting specific services to patients based on risks,
monitoring specific indicators for conditions, and using evidence-based guidelines for treatment. It was evident
that case study sites were using their EMRs effectively for these purposes as well; for example, clinicians reported
the ability to identify patients with specific conditions and plan and monitor their care, and identified this as a very
significant benefit of EMRs.
Specific indicators measured in the case study sites (for HbA1c values and other testing for diabetes care)
demonstrate that providers are using their EMRs for the purposes of patient identification and communication,
with the goal of ensuring that patients receive timely and appropriate evidence-based care to manage their
conditions. Case study sites demonstrated that 41-70% of their patients had HbA1c levels at or under 7%, while a
2003 study indicated that fewer than 30% of Canadian diabetics had similar HbA1c levels of 7% (Harris, 2003). As
a Health Information Technology (HIT) tool, EMRs can enable proactive management of diabetic patients and
those with other chronic medical conditions.
Clinicians reported a high degree of willingness and enthusiasm to better manage the health of defined patient
populations in their practice, rather than simply providing episodic care. EMRs currently implemented in Ontario
provide a significant capability for clinicians to do this, an undertaking that is typically costly and time consuming
in a paper-based environment. In addition to saving costs and time for providers, the potential prevention of co-
morbid conditions and complications arising from chronic diseases can avoid costs to the health system and enable
patients to maintain an improved quality of life.
EMRs provide valuable tools to help both care providers and patients with care management and
education.
Engaging patients through education and care planning activities ensures that they are active participants in their
health. Patients that are better informed on what their target values for key indicators (e.g., HbA1c) should be are
better able to keep their conditions under control. Having a patient that is better informed of his/her condition, can
see trended information for targeted indicators, and is working with the physician to better manage their care can
result in avoided unnecessary office and/or hospital visits as well.
As data capture and reporting capabilities improve, EMRs will further allow practices to identify a baseline and
trend information to ensure their diabetic cohort of patients are monitored for those clinical needs which require
careful management and are prone to downstream co-morbid complications.
The broad and mature use of EMRs can reduce the costs and burden of illness associated with
caring for Ontario’s growing diabetic population.
Through the modelling and forecasting exercise, potential savings and a reduction in complications associated with
diabetes were identified. These complications and illnesses include foot amputations, end stage renal disease
(ESRD), stroke and myocardial infarction, all of which typically require hospitalization. While yet to be realized,
EMRs are a contributing factor to potential savings related to diabetic-related complications that are in the range of
$125 million annually by 2017.
Benefits Realization Study for EMRs in Ontario
PwC 47
The forecasted reductions in illness, avoidance of co-morbid conditions and costs are highly dependent on the
mature, proactive use of EMRs on a province-wide basis to manage diabetic care through prevention, education
and ongoing monitoring and management of specific conditions. Supporting the provincial government’s focus on
chronic disease management, EMRs can play a critical role in proactively managing diabetes in the community, by
equipping providers with necessary information to provide that care in the community. The above average
percentages of patients in case study clinics (41-70%) for whom HbA1c is less than 7% speaks to the impact that
EMR use can have on the health of those patients.
Diabetes must be actively monitored and managed in community settings and with active patient participation, in
order to prevent complications such as those forecasted above. Without EMRs and related enablers, it would be
very challenging to do so, given the support that EMRs provide to actively identify diabetic patients, keep their
conditions under control, and communicate on an ongoing basis.
6.4 Health Promotion, Screening and Prevention
Findings suggested that clinicians recognized that promotion, screening and prevention activities were greatly
enabled through the use of EMRs. For this category of benefits, there is great potential to realize benefits at a
health system level as EMR use becomes more mature across the province.
EMRs allow clinicians to survey patients and to proactively arrange screening and prevention
activities, while concurrently improving the efficiency of preventative care.
Case study sites were quite capable in tracking prevention and screening activities, and very knowledgeable about
the benefits they have realized to date in this regard. While several reporting capabilities related to screening and
prevention are in their early stages of development, advanced users of EMRs indicated that they were able to
generate information to identify and communicate with patients for preventative purposes. Without the EMR, this
type of prevention activity is much more complicated, requiring manual and time-consuming chart reviews. Now,
patients are contacted easily and in a timely fashion for preventative care.
The provincial target for colorectal cancer screening rates is 40% of targeted groups (Health Quality Ontario, 2012),
and all sites reported exceeding this target by a minimum of 11%. Given the incentives in place for providers and
the ability of EMRs to support the identification of targeted patients, the EMR is a valuable tool to support this
preventative activity, and the downstream benefits in quality of life for patients and potential cost savings to the
system as a whole.
The potential benefits of increased screening and prevention (including vaccination) activities over time can have a
tremendous impact on the health of the population as a whole, and on the sustainability of the health system.
Several physicians indicated an increased interest in caring for their patients through a population-based approach,
organizing educational sessions for targeted groups and standardizing their care. The benefits to these clinicians
are clear, and their EMRs support efficient ways of communicating and delivering care.
Information generated by EMRs for preventative care purposes is helpful in communicating with
patients and engaging them in self-care.
Effective prevention strategies engage patients in their care. Many sites reporting engaging patients early in their
care, interacting with them and proactively scheduling wellness visits through the use of EMRs. The ability to plan
ahead and schedule visits supports patient involvement in their care and continuous management of their health.
The widespread use of EMRs can increase the rates of influenza vaccination and yield potential
related health system savings.
Benefits Realization Study for EMRs in Ontario
PwC 48
Many sites indicated their ability to track and deliver “flu shots” to their patient populations through the use of
EMRs. With those 65 and older being particularly at risk for the flu with the strong potential to result in further
complications that often result in visits to EDs and/or hospitalization, the modelling and forecasting exercise was
undertaken to assess the potential savings to the health system. While yet to be realized, EMRs are a contributing
factor to potential savings related to influenza that are in the range of $40 million annually by 2017.
The forecasted reductions in illness and costs are highly dependent on the mature, proactive use of EMRs on a
province-wide basis to identify those at risk, to facilitate communication to encourage those patients to receive the
flu shot, and to track compliance. The estimates in avoided hospitalizations and cost savings are significant, but
only relate to those in the 65 years and older cohort. The potential for additional incremental benefits has not been
defined but is not to be underestimated, as influenza impacts many in Ontario each year. Influenza must be
actively managed in community settings, requiring active patient participation and willingness to receive
vaccination. EMRs enable this challenge, with many interviewees discussing their ability to vaccinate a broader
group of patients using their EMRs as a tool.
On a similar note, many providers and patients also identified the benefit of EMRs in tracking other immunizations
(e.g. DTaP shots, zostavax etc.). Prevention of illnesses other than influenzasave costs to the health system and
enhance the quality of life for many affected populations (e.g., senior citizens).
Use of EMRs can support the prevention of colon and other cancers through improved screening
rates and other preventative care.
Case studies revealed that EMRs can greatly facilitate the ease with which clinicians manage and deliver
preventative care for their patients. Although preventative care is a relatively new (and in some cases, advanced)
area of EMR use for many clinicians, the ability to identify patients requiring screening and in turn receive and act
upon results is possible with EMR use today. Modeling and forecasting activities suggested that the potential costs
that can be realized through the prevention of four stages of colon cancer through advanced EMR use is in the
range of $38 million annually by 2017.
Many provincial organizations, including Cancer Care Ontario, Canadian Institute for Health Information, Heart
and Stroke Foundation, and others are looking to EMRs to support disease management and reporting. Each of
these organizations recognizes the power and potential of EMRs in this regard. Specifically, there is an increasing
emphasis on behalf of Cancer Care Ontario to explore the use of EMRs to support screening initiatives in the near
future to support its widely implemented Colon Cancer Check program and related clinical guidelines. By ensuring
that screening protocols and alerts are incorporated in all EMR specifications, there will be an increased ability to
identify those patients requiring screening on a widespread basis. While the example modeled is limited in scope
with a focus exclusively on colon cancer, it does suggest that the overall potential to prevent cancers through the
use of EMRs as a key supporting tool is very promising.
6.5 Efficiency
EMRs have been demonstrated to increase efficiency among clinicians and administrative staff, particularly
following an initial period of implementation that lasts for approximately one year. During this time, clinicians and
staff grow accustomed to a new way of working, and roles change with some tasks being transferred among
providers. Once this period of transition has passed, case study sites reported that there are great gains to be
realized in efficiency through the use of EMRs.
EMRs promote the ability to readily locate patient information has improved the quality and
efficiency of patient interactions.
Benefits Realization Study for EMRs in Ontario
PwC 49
EMRs provide clinicians with access to comprehensive patient records that are organized and readily accessible in
one location that can be accessed by all. Without having to search for information, the quality and efficiency of
patient interactions improve. The patient experience and overall satisfaction is also improved, in part due to a lack
of having to repeat their concerns and/or health information to multiple caregivers.
EMRs facilitate clinical transformation in the primary care setting, improving the effectiveness and
efficiency of clinical and administrative activities.
The introduction of EMRs into the primary care setting has the potential to be transformative in nature. EMRs
enable changes in workflow and clinical decision-making, and can greatly improve the way that clinicians interact
with their patients on a regular basis. Findings suggested that 78% of respondents felt that EMRs improved the
efficiency of many administrative and patient care activities, including scheduling, communicating, ordering of
tests, referrals, and documentation. In particular, clinicians (92.6%) noted that order entry in EMRs increases
efficiency of requisitions and prescriptions remarkably.
Benefits associated with the effectiveness and efficiency of patient encounters and the general flow of patient
activity allow greater throughput and access to care, including same/next day visits. Specifically, many EMRs have
enabled “dashboards” with key patient information, and clinicians reported being more organized for each visit,
with readily available information and the ability to address patient concerns and care requirements quite easily.
These benefits have tremendous impact on patients, providers and the system as a whole. Gains in efficiency that
are obtained in the primary care setting can translate into reduced wait times for primary care, allowing providers
to see more patients daily, and reductions in return visits to clinics and visits to hospitals.
EMRs improve the productivity of administrative clinic staff.
Reductions in the time required for administrative tasks through the use of EMRs was estimated at 50% by
eliminating routine activities such as pulling charts, scheduling and other initiatives. The reduction in these
administrative activities provide the opportunity for administrative and/or clinical staff to redirect their clinic
activities, improving productivity of practices and in many instances, providing more time for direct patient
interaction.
The reductions in the time required for administrative tasks through the use of EMRs, estimated at 50%, was used
to model the potential benefits if EMRs were broadly and maturely used across Ontario. The exercise estimated
that approximately 1.8 million hours or a possible $40 million could be saved annually by 2017, providing
increased capacity for clinic staff.
It is also acknowledged that there is some productivity lost in the early stages of EMR implementation among many
clinicians and administrators, with many reporting that there is a period of approximately one year while all grow
accustomed to working with electronic records. However, informants and the forecasting model both suggest that
there are significant opportunities to improve the overall capacity among clinic staff, particularly among
administrative staff and/or nurses who were previously spending time doing administrative tasks. The greatest
value in this increased capacity lies in the opportunities and potential benefits associated with redirecting time
from non-value added tasks (such as pulling and filing charts, chasing paper forms, placing phone calls), to those
that improve efficiency, allow for direct patient interaction, and improve overall health outcomes.
6.6 Medication Management
The ability to improve medication management is an important provincial strategy, enabled through the use of
EMRs. Although not an explicit focus in the case studies, the findings related to the impact of EMRs on medication
management are significant and very supportive of eHealth Ontario’s strategies, demonstrating the important role
Benefits Realization Study for EMRs in Ontario
PwC 50
that EMRs play in achieving this goal. A most important benefit use of the EMR for medication management is the
improvement in overall patient safety, discussed below.
EMRs support the ability to rapidly identify impacted patients when drug warnings are issued,
improving patient safety.
A tremendous benefit of EMR use identified by many providers through the course of the study was the ability to
identify large numbers of patients to whom certain drugs have been prescribed. Approximately 89% of clinicians
surveyed identified this as a benefit, and many clinicians at case study sites identified this benefit during interviews.
When drug warnings or recalls are issued by Health Canada or other bodies, the health of patients is at risk. These
patients can be very quickly identified, and alternate means of treatment can quickly be administered, preventing
any downstream implications. As such, EMRs can have a tremendous impact on quality of care, and most
importantly, patient safety. The ability to quickly target these patients and act is almost impossible in a paper-
based clinic environment.
Access to complete medication profiles in the EMR increases efficiency and improves the accuracy
of medication management.
With accurate and complete medication profiles in EMRs, clinicians are able to make more informed care
decisions. Prescriptions are easily monitored, and patient concerns can be addressed. The risks associated with
over-prescribing, or prescribing contra-indicated medications are nearly eliminated, increasing patient safety and
reducing potential hospitalizations and in some instances, fatalities. There are also potential costs avoided by the
health system due to unnecessary hospitalization.
Where other providers outside of Family practices (e.g., community pharmacies) can communicate and collaborate
with prescribing physicians, there is the opportunity to further ensure patient safety and quality of care through
enhanced clinical decision-making among interprofessional providers.
Electronic prescribing and renewals via EMRs have improved medication management efficiencies
and patient safety.
The sharing of medication information across the care team through EMRs has resulted in greater efficiencies in
the patient care process and improved patient safety. Perhaps the most significant change noted by clinicians in
the case studies has been the generation of a printed prescription, eliminating error-prone handwritten
prescriptions and reducing the delays arising from “call backs” from pharmacists to physicians seeking clarification
on prescriptions. In paper-based clinics, handwritten prescriptions continue to pose risks to patients whereby
pharmacists may make inappropriate judgments about drug choices and/or dosage. When concerns are
experienced by pharmacists, the prescribing physician must be called to confirm the prescription, interrupting the
workflow of the pharmacist, physician, and potentially office staff as well. With EMRs, this risk to patient safety
and disruption in workflow is avoided.
Benefits Realization Study for EMRs in Ontario
PwC 51
7. Recommendations
Findings from this study provide compelling evidence to continue to advance EMR adoption and maturity across
Ontario. The benefits that have been demonstrated by EMR use in the selected case study settings and the
accompanying modeling of province-wide benefits demonstrate the “art of the possible” for Ontario. With the
implementation of a number of focused recommendations, the potential for wide ranging and transformative
benefits of EMR use can be further realized by providers, patients and the health care system as a whole.
The following recommendations are presented to policy-makers, funders, implementers and adopters of EMRs inOntario in support of continued benefits realization.
1. As an essential enabler and one of many important health information technology tools to
improve care delivery and related patient outcomes in primary care settings and beyond,
continued investments in EMRs should be made to ensure broad adoption and realization of
benefits across Ontario.
EMR adoption by primary care physicians in Ontario continues to increase, and over 70% of all community-
based clinicians are now enrolled in the EMR Adoption Program. Although the most recent commonwealth
survey indicates that U.S. and Canada are still behind other OECD countries in terms of primary care EMR use
(Schoen et al., 2012), significant progress has been made over the past few years, and Ontario has evolved as a
leading jurisdiction in Canada for EMR adoption and use. In addition, eHealth Ontario recognizes the
enhanced maturity level of EMR use and has placed a strategic focus on advancing maturity of use to further
realize the benefit of using EMRs and associated eHealth offerings.
There are numerous variables inherent in primary care settings that influence patient, provider and health
system outcomes, and it is at times challenging to identify the benefits that are directly attributable to EMR
adoption and use. For instance, variables such as the introduction of interdisciplinary care, enhanced
workflows, clinical care integration and payment models collectively impact improved patient care and
outcomes. However, through the course of the study it was widely acknowledged that without the use of EMRs,
the ability to realize these benefits is compromised. Indirectly and directly, EMRs are critical enablers of
enhanced patient care. Continued investment in EMRs and increased physician participation in the EMR
Adoption Program are essential.
Through this continued investment, the use of EMRs has the capability to provide a supportive underpinning
for the advancement of many current government priorities and to achieve desired system-wide benefits for
patients and the population as a whole. As an example, the recently introduced Health Links strategy for
Ontario (Ministry of Health and Long-Term Care, 2012) highlighted a new “sub-LHIN” care delivery model to
enhance the efficiency and outcome of care for patients in respective catchment areas, with access to primary
being the focus for ensuring that outcomes are achieved. Such a model would rely on enabling health
information management within primary care and between primary and acute care centres. Advanced EMR
use and clinical data management and sharing of information would provide significant value to implement
this government priority in the foreseeable future.
2. Continue to support increased maturity of use among current and future adopters of EMRs.
The effective realization of benefits is highly supported by EMR maturity (defined as the level of adoption and
use of the EMR in the practice setting). Previous studies (EMR Adoption Program Evaluation, Healthtech
Consultants, 2012) have demonstrated the value of the OntarioMD EMR Maturity Model (EMM), which
Benefits Realization Study for EMRs in Ontario
PwC 52
focuses on assessing meaningful use of EMRs along six levels. The level of maturity is characterized by varying
degrees of use of EMR functionality, which in turn can support the realization of increasingly levels of benefit.
The model supports an objective assessment of EMR use, and a pilot study conducted in 2011 revealed that the
majority of EMR users in the pilot study (74%) were using EMRs for basic and/or standardized use, with fewer
(13%) showing established use (with clinical decision support, capacity for chronic disease management), and
none in the study yet using EMRs for integrated care and/or regional collaboration to address population
health needs (OntarioMD, 2011). A continued focus in advancing EMR maturity among users will contribute to
a greater diversity of benefits with system-wide impact. Continued collaboration with other jurisdictions is also
recommended to advance maturity.
3. Continue to invest in effective change management strategies and user support that extends
beyond the initial period of EMR implementation.
Mature use of the EMR requires access to training when and where it is needed by all types of users, and should
be available well beyond the initial implementation phase as users transition through the “adoption curve”.
Literature findings also demonstrated the importance of effective change management strategies to support
EMR adoption including the designation of peer leaders, workflow and process redesign, and a clear
articulation of vision and expected benefits, communication, and system support (Nagle & Catford, 2008).
Effective change management will ensure an environment open and ready for transformation and also ensure
users are better prepared for changing roles and work flow, and to adapt and mature their EMR use for
targeted realization of benefits.
Some interviewees noted that the lack of incremental training provided post-implementation hindered their
ability to learn how to use the EMR to its full potential and explore the use of all functionality. For example,
following an initial period of use that allows clinicians to master essential EMR functions, training could be
further made available to address and support more sophisticated needs associated with reporting and
analytics for population-based planning and care.
As part of the ongoing support, there is great merit in continuing with the Peer Leader support program offered
by OntarioMD. Many clinicians turn to their peers for support and rely on their advice to enhance their
experience working with EMRs. These leaders are instrumental in gaining buy-in to workflow changes,
changing the practice culture, responding to educational needs and ensuring the practice continues to progress
as challenges in the EMR journey arise.
4. Improve the management of information within and across patient care settings through
focused efforts related to interoperability of systems, improved quality of data, and the flow of
data across care settings.
The inability of EMRs to connect and share information with external entities outside of primary care has been
cited as a common barrier to EMR implementation (Ramaiah et al., 2010). For many Ontario physicians, EMRs
remain stand-alone systems within their practice group. However, it was recognized through this study that
many primary care providers are interested in advancing clinical information management (IM). There are
current physician-lead initiatives to develop an IM strategy and methodology in the province to improve the
flow of data across care settings. The integration of systems external to the EMR setting, notably OntarioMD’s
Hospital Report Manager which delivers discharge summaries and other hospital reports directly to EMRs, is
important in enhancing the continuity of care for patients and the coordination of care among providers. The
integration of the Better Outcomes Registry and Network (BORN) and OLIS with EMRs are in earlier stages,
but early feedback from study sites is promising. Future in integration of EMRs with the Medication
Management System and eReferral applications will expand and extend these benefits.
The ability to achieve advanced use and benefits of EMRs will also be supported by improved quality of data.The ability to extract data elements within a practice and to share data or patient information across settings
Benefits Realization Study for EMRs in Ontario
PwC 53
relies heavily on the use of standardized data. In addition, quality data must exist in order to generate systemalerts and for trending analysis to be meaningful. Clinicians will require training and support to improve dataquality, which shall in turn support practices’ ability to track and report performance information within andacross practice settings. Quality improvement plans should include a focus on data quality and education sothat practices can report on priority quality of care measures that impact health care system performance andenable chronic disease management and prevention and promotion strategies, among others. Strategies shouldalso be developed to ensure that EMR systems are designed and implemented to enable ease of data extractionby practices for reporting, care management and decision-making purposes.
Interestingly, there are several initiatives in Ontario and across Canada with a focus on EMR
clinical/administrative data collection, analysis and clinical information management reporting; for example,
Canadian Institute for Health Information’s (CIHI) Primary Health Care Voluntary Reporting System, and
Canadian Primary Care Sentinel Surveillance Network (CPCSSN) and Heart and Stroke Foundation
Hypertension initiative. In addition, some practices are implementing dashboards with data captured from
EMRs, with data presented to physicians and team members to report how the physician/ practice is doing
related to managing patients with chronic conditions, prevention and promotion, screening, statistics on
referrals, etc.
5. Continue to invest in focused benefits realization studies.
Opportunities exist to collaborate with the research community and other organizations to conduct focused
EMR benefits realization studies with more quantitative and objective measurement. As EMR adoption and
maturity increase, the amount of quantitative data available will also increase. Focused studies will afford the
opportunity to measure more of the indicators that were defined in this study as part of the Benefits Realization
Framework (Appendix C) and have a greater understanding of the full scope of current and potential benefits
realized by EMRs in Ontario.
To ensure that future studies reflect current and potential strategic priorities, it is recommended that the
Benefits Realization Framework be updated on a regular (e.g., quarterly or semi-annual) basis. For example, in
its next iteration, the BR Framework should be updated to reflect a measurement focus on same/next day
scheduling of appointments and communication with patients via email in order to address the anticipated
requirements of quality plans in primary care.
Benefits Realization Study for EMRs in Ontario
PwC 54
8. Concluding Remarks
For the past decade, many health information technology projects across Canada and internationally have had a
heavy focus on the development of technical infrastructure. In the past four years, eHealth Ontario has invested
significantly in a number of transformative initiatives, including the implementation of EMRs in community
settings. Through the significant dedication and leadership of eHealth Ontario, its PeHP team members, along
with delivery partner OntarioMD, the Association of Ontario Health Centres (AOHC), and clinicians across the
province, the adoption of EMRs has passed its tipping point with over 70% of community-based physicians enrolled
in the EMR Adoption Program.
It is important to realize the benefits from Ontario’s strategic investments and to optimize the value for patients,
providers and the healthcare system’s stewards. Benefits of EMR use are diverse and have significant positive
impacts on patients, providers and the health system as a whole. Improved availability, integration and
communication of health care information will result in improved care for those with chronic diseases, greater
efficiency in interactions with patients, improved patient safety, improved patient participation in their own health
care, and many other positive quality outcomes (Ontario Hospital Association, 2008). Realizing EMR benefits is a
journey. The realization of benefits occurs over time as the adoption of EMRs and their use in the clinical setting
becomes more pervasive and mature.
This study has demonstrated that there has been significant realization of EMR benefits among a select group of
advanced EMR users in Ontario. However, the full value of EMRs has yet to be realized, as adoption continues and
process and maturity of use further evolves. The potential to extend and build upon the benefits realized to date are
significant, and the further investment in EMRs will support Ontario and its population well as the health system
continues to evolve along with the needs of all Ontarians.
Benefits Realization Study for EMRs in Ontario
PwC 55
9. References
1. Auditor-General. (2012). Annual Report. Office of the Auditor General of Ontario.http://www.auditor.on.ca/en/reports_en/en12/2012ar_en.pdf
2. Canadian Diabetes Association. (2009, December). An Economic Tsunami: The Cost of Diabetes inCanada.
3. Canadian Healthcare Influenza Immunization Network. (n.d.) Influenza Facts.
4. Canadian Medical Association. (2011). IM/IT Report.
5. Canadian Partnership Against Cancer. (2009, September). Quality Determinants for Colorectal CancerScreening in Canada.
6. Cancer Care Ontario. (2010). Colon Cancer Check 2010 Program Report, Appendix B, Table 4.
7. Commonwealth Fund, The. (2011). International Profiles of Health Care Systems 2011.
8. Dahrouge, S., Devlin, RA., Hogg, B., Russell, G., Coyle, D., and Fergusson, D. (2012). The Economic Impactof Improvements in Primary Healthcare Performance. Canadian Health Services Research Foundation.
9. eHealth Ontario (2013). What we do. http://www.ehealthontario.on.ca/en/about
10. eHealth Ontario (2009). Strategic Plan 2009 – 2012.
11. Girosi, F., Meili, R.C., & Scoville, R. (2005). Extrapolating Evidence of Health Information TechnologySavings and Costs. Santa Monica, CA: RAND Corporation.http://www.rand.org/pubs/monographs/MG410.
12. Harris S.B., Stewart M., Brown J.D., Wetmore S., Faulds C., Webster-Bogaert S., Porter S. (2003) Type 2diabetes in family practice: Room for improvement. Can Fam Physician, 49: 778-785.
13. Health Quality Ontario. (2012). Quality Monitor: 2012 Report on Ontario’s Health System.
14. Healthtech Consultants Inc. (2012, January 5). EMR Adoption Program Evaluation.
15. Hillestad, R., Bigelow, J., Bower, A., Girosi, F., Meili, R., et al. (2005). Can Electronic Medical RecordSystems Transform Health Care? Potential Health Benefits, Savings, And Costs. Health Affairs, 24 (5):1103-1117. http://content.healthaffairs.org/content/24/5/1103.full.pdf
16. Hodge, T. (2011). Canada Health Infoway – Infoway Connects (blog).
17. Institute of Clinical Evaluative Sciences. (ICES) (2005).
18. Johnston. D., Pan, E., Walker, J., Bates, D., and Middleton, B. (2003). CITL: The Value of ComputerizedProvider Order Entry in Ambulatory Settings. http://www.partners.org/cird/pdfs/CITL_ACPOE_Full.pdf
19. Klein Woolthuis E.P., de Grauw W.J., van Gerwen W.H., van den Hoogen H.J., van de Lisdonk E.H.,Metsemakers J.F., van Weel C. (2007). Identifying people at risk for undiagnosed type 2 diabetes using theGP's electronic medical record. Fam Pract. June; 24(3): 230-6.
Benefits Realization Study for EMRs in Ontario
PwC 56
20. Manitoba Health (2012, May). Manitoba’s Physician Integrated Network (PIN) Initiative, A BenefitsEvaluation Report May 2012.
21. Ministry of Health and Long-Term Care (2012). 2012 Physician Services Agreement Overview.
22. Ministry of Health and Long-Term Care (2012, December 6). Backgrounder: About Health Links.http://news.ontario.ca/mohltc/en/2012/12/about-health-links.html
23. Ministry of Health and Long-Term Care (2012). Ontario’s Action Plan For Health Care. ISBN 978-1-4435-8942-0. http://health.gov.on.ca/en/ms/ecfa/healthy_change/docs/rep_healthychange.pdf
24. Nagle. L & Catford.P. (2008). Toward a Model of Successful Electronic Health Record Adoption.Healthcare Quarterly , 11 (3), 84-91.
25. Ontario Hospital Association. (2008). Incentives for Transformation.
26. OntarioMD Inc. (2012). Benefits Evaluation: EMR Maturity Model Framework.
27. OntarioMD Inc. (2011). EMR Maturity Model, Pilot Data.
28. Ontario Ministry of Finance. (2012). Ontario Population Projections Update, 2011-2036.
29. Public Health Agency of Canada (2011). United Nations NCD Summit 2011. http://www.phac-aspc.gc.ca/media/nr-rp/2011/2011_0919-bg-di-eng.php
30. Ramaiah, M., Subrahmanian, E., Sriram, R. D., Lide, B. B. (2010, November). Workflow and ElectronicHealth Records in Small Medical Practices. National Institute of Standards and Technologyhttp://www.nist.gov/customcf/get_pdf.cfm?pub_id=903654
31. Schoen, C., Osborn, R., Squires, D., Doty, M., Rasmussen, P., Pierson, R. and Applebaum, S. (2012). ASurvey of Primary Care Doctors in Ten Countries shows Progress in use of Health Information Technology,Less in Other Areas. Health Affairs, 31(12): 1-12.
32. Service Ontario (2010). Excellent Care for All Act, 2010, S.O. 2010, CHAPTER 14. http://www.e-laws.gov.on.ca/html/statutes/english/elaws_statutes_10e14_e.htm
33. Statistics Canada. (n.d.) CANSIM Table 326-0020.
34. Statistics Canada. (2006). Census 2006.
35. Telford, J.J., Levy, A.R., Sambrook, J.C., Zou, D., and Enns, R.A. (2010). The cost-effectiveness ofscreening for colorectal cancer. CMAJ, 182 (12): 1307-1313.
36. Wolfram, P. (2009). EMR-Link Lab Orders: Measuring Return on Investment (ROI). Ignis Systems
Benefits Realization Study for EMRs in Ontario
PwC
Appendix A: Interviewee List
# Stakeholder Name Title Organization
1 Fariba Rawhani Senior VP- Development and
Delivery
eHealth Ontario
2 Christine Sham &
Mark Simmons
Manager, eHealth Liaison Branch Ministry of Health and Long-Term Care
3 Sarah Hutchison CIO Ontario Medical Association
4 Brian Forster CEO OntarioMD
5 Dr. Darren Larsen Senior Peer Leader OntarioMD
6 Frank Vassallo VP, Physician IT Adoption OntarioMD
7 Jeremy Smith &
Carol Rimmer
Program Director
Assistant Director
Physician Information Technology Office,
British Columbia
8 Dan Sheplawy Program Director Physician Office System Program , Alberta
9 Terry Moore &
Brent McGaw
Regional Executive Canada Health Infoway
10 Robert Lee CIO and eHealth Lead Toronto Central Local Health Integration
Network
11 Jan Kasperski Executive Director Ontario College of Family Physicians
12 Maureen Boon Senior Advisor, Executive office College of Physicians and Surgeons of Ontario
13 Dr. Liisa Jaakkimainen Physician, Research Scientist Institute for Clinical Evaluative Sciences,
University of Toronto
14 Ben Chan CEO Health Quality Ontario
15 Patricia Sullivan-
TaylorManager, Primary Health Care
Information
Canadian Institute for Health Information
16 Rod Burns CIO Association of Ontario’s Health Centres
17 Lana Palmer Executive Director Upper Grand Family Health Team
18 Cheryl Kennedy Interim Administrative Lead Haliburton Highlands Family Health Team
19 Dr. Stephen McLaren Lead Physician Markham Family Health Team
20 Glenn Lanteigne CIO and eHealth Lead South West Local Health Integration Network
21 Plumaletta Berry Program Evaluation Analyst OntarioMD
Benefits Realization Study for EMRs in Ontario
PwC
Appendix B: Consultation Guide
IntroductionOn behalf of eHealth Ontario, PwC has been engaged to conduct a study to assess the impact of EMR adoption and
use in Ontario. This study will focus on the impact of benefits realized to date and make projections for the future
potential benefits that can be realized. The first phase of this study is to validate the current draft benefits
evaluation framework for EMRs and identify a set of priority indicators for subsequent phases of data collection
and evaluation.
eHealth Ontario’s draft benefits evaluation framework for EMRs has identified several measurable benefits for
patients, providers and the health system, and grouped them into four categories (quality of care, access to care,
efficiency of care, health care sustainability). Several proposed indicators are aligned to each of the four categories
and are presented in the framework for your review. Your comments and feedback are greatly appreciated and
extremely valuable to this validation process. If you have any questions or comments related to this process, please
do not hesitate to contact Erika Norris at 416.941.8383 ext. 13667 or [email protected]
Interview Questions1. What is your overall opinion of the EMR BE framework?
a. Are these the most appropriate categories and indicators to facilitate an understanding of the impact of
EMRs?
b. Are there any categories or indicators missing? If so, please elaborate.
2. What are the strengths of the current BE framework? What are your greatest concerns about the framework
(e.g., risks)?
3. If you had to choose the top 5 – 8 indicators that would indicate whether or not EMRs have had an overall
impact on patients, providers and the health system, what would they be and why?
4. Can you comment on the feasibility and ease of measuring those priority indicators?
a. Can these priority indicators be measured today? If so, how?
b. What are (or could be) the best sources from which to collect this data for optimal measurement?
5. What lessons learned or observations will be important to pass on to the Physician eHealth Program at eHealth
Ontario about measuring and communicating the impact of EMRs?
6. Has your organization/team conducted a study for EMRs using a similar benefits evaluation framework? If so,
can you share any results with us?
7. Are there any reports (publicly available or otherwise) that you feel would be useful to our team? If so, how can
we access them?
8. Do you have any other comments, suggestions or advice?
Benefits Realization Study for EMRs in Ontario
PwC
Appendix C: Revised EMR BR
Framework
The following revised EMR BR Framework was finalized at the BR Framework Validation Workshop and presented
to Steering Committee members to conclude Phase 1. Of the sixty original indicators, twenty-one were identified as
priority indicators, laying the foundation for a longer term BR Framework and measurements that may be used for
future analysis.
Hypothesis/
Research
Question
Indicator Definition Ref # Category Domain
Benefit
Realizat
ion
(Time)
Type of
Analysis/Data
Source
Medication Management
Does EMR use
facilitate
medication
management?
% change in medication error rates
in terms of actual/potential
prescribing errors
MM1 Quality
Provider,
Health
System
Long-
Term
Quasi-experimental
before and after,
Administrative Data
% change in patients treated for
adverse drug reactions/eventsMM2 Quality
Health
System
Long-
Term
Quasi-experimental
before and after,
Administrative Data,
EMR data modeling
% change in call backs and/or time
responding to pharmacist requests
for physician verificationMM3
Quality,
Efficiency
Provider,
Health
System
Short-
Term
Quasi-experimental
before and after,
Provider Survey,
Interview
Laboratory Management
Does EMR use
reduce unnecessary
duplication of lab
tests?
% change in annual lab tests
duplication or unnecessary lab tests
LM1
Health
care
Sustainab
ility
Health
System
Long-
Term
Quasi-experimental
before and after,
Administrative Data,
Provider survey
Does EMR use
reduce lab
turnaround time?
Average time between laboratory
time of service and test results
available in EMR
LM2Efficiency,
Quality
Patient,
Provider
Medium-
Term
Administrative Data,
Provider Survey
Communication and Coordination
To what extent
does EMR use
improve access to
information
between settings?
(e.g. completeness
of referrals or other
documentation
passing between
care settings)
Average time to receive discharge
summary following inpatient
discharge
CC1Quality,
Efficiency
Patient,
Provider
Medium-
Term
Quasi-experimental
before and after,
Administrative Data,
HRM Data,
Provider Survey
Does EMR use
facilitate referral to
Average time from referral decision
to when the referral is sentCC3 Access
Patient,
Provider
Short-
Term
Quasi-experimental
before and after,
Benefits Realization Study for EMRs in Ontario
PwC
specialists? Provider Survey,
Interview
Does EMR use
facilitate
interdisciplinary/te
am care?
% of practices where the care team
has access to and uses the EMR
system
CC4
Access ,
Quality,
Efficiency
Patient,
Provider
Short-
Term
Organizational Survey,
Provider Survey
Do providers
access the EMR
remotely to provide
patient care?
% of physicians who have remote
access to EMR and use it for patient
care
CC5 AccessPatient,
Provider
Short-
Term
Provider Survey,
Interview
Chronic Disease Management
Does EMR use
improve chronic
disease
management?
% of PHC clients/patients, 18 years
and over, with diabetes mellitus in
whom the last HbA1c was 7.0% or
less (or equivalent test/reference
range depending on local
laboratory) in the last 15 months
CDM1 Quality
Patient,
Health
System
Medium-
Term
Administrative/ EMR
data, CIHI PHC-VRS
(CIHI #39), Trending
over time
% of patient population, age 18 and
older, with diabetes mellitus who
received testing for all of the
following:
Hemoglobin A1c (HbA1c); Full
fasting lipid profile screening;
Nephropathy screening; Foot
examination; Blood pressure
measurement; and
Obesity/overweight screening
CDM2 Quality
Patient,
Health
System
Medium-
Term
Administrative/ EMR
data, CIHI PHC-VRS
(CIHI #57-4.1 spec),
Trending over time
% of population, age 20 and older,
with diabetes mellitus who have
had an eye exam
CDM3 Quality
Patient,
Health
System
Medium-
Term
Administrative/ EMR
data, CIHI PHC-VRS
(CIHI#58-4.1 spec),
Trending over time
% of practices that track and
identify patients with chronic
disease
CDM4 Quality
Patient,
Health
System
Medium-
Term
Provider survey,
Interviews
Health Promotion, Screening and Prevention
Has EMR use
improved the
preventative
services provided?
% Percentage of patient population,
age 65 and older, who received an
influenza immunization
HPSP1Quality Health
System
Medium-
Term
Administrative/ EMR
data, CIHI PHC-VRS
(CIHI #41-4.1 spec),
Trending over time
% of practice population, age 50 to
74, who had a screening test
ordered for colon cancer.
HPSP2 QualityHealth
System
Medium-
Term
Administrative/ EMR
data, CIHI PHC-VRS
(CIHI # 48), Trending
over time
% of practice population, age 18
and older, who have had their
blood pressure measured by their
primary health care (PHC)
provider.
HPSP3 QualityHealth
System
Medium-
Term
Administrative/ EMR
data, CIHI PHC-VRS
(CIHI #54-4.1 spec),
Trending over time
Population Health
Does
implementation of
EMRs help
population health
% of physicians/practices that use
EMR to do needs planning (e.g.,
plan for patient
population/community)
PH1
Health
care
Sustainab
ility
Patient,
Health
System
Long-
Term
Provider survey,
Administrative Data,
Interviews
Benefits Realization Study for EMRs in Ontario
PwC
planning?
Patient Experience
To what extent has
EMR use improved
the patient
experience?
Patient perceived improvement in
encounter experience. [Focus will
be on specific experiences e.g.,
access, outcomes, communication,
interaction, patient tools such as
scheduling appt.]
PE1 Quality PatientMedium-
Term
Patient survey
Efficiency
To what extent
does EMR use
improve physician
efficiency?
% change in time to complete
clinical/admin documentationE1 Efficiency Providers
Medium-
Term
Provider survey,
Interviews
% of physicians who use EMR as a
referral toolE2 Efficiency
Patients,
Providers
Medium-
Term
Administrative Data,
Provider survey
EMR Maturity
Extent to which the
EMR has been
adopted and used
to its full potential?
% of providers at each level of the
EMR Maturity ModelEM1
OntarioMD EMR
Maturity Model Tool
Benefits Realization Study for EMRs in Ontario
PwC
Appendix D: EMR BR Framework-
Case Study Indicator Subset
Priority indicators were identified, specifically for assessment in Phase 2 of the Benefits Realization study based on
known available EMR functionality, maturity of use and feasibility of measurement at case study sites. A total of
fourteen indicators were initially selected for the EMR BR Framework - Case Study Indicator Subset.
Hypothesis/Research
QuestionIndicator Definition Ref # Data Source
Laboratory Management
Does EMR use reduce lab
turnaround time?
Average time between laboratory time
of service and test results available in
EMR
LM2Administrative Data, Provider
Survey
Communication and Coordination
To what extent does EMR use
improve access to information
between settings?
(e.g. completeness of referrals or
other documentation passing
between care settings)
Average time to receive discharge
summary following inpatient dischargeCC1
Administrative Data,
HRM Data, Provider Survey
Does EMR use facilitate referral to
specialists?
Average time from referral decision to
when the referral is sentCC3 Provider Survey, Interview
Does EMR use facilitate
interdisciplinary/team care?
% of practices where the care team has
access to and uses the EMR systemCC4
Organizational Survey,
Provider Survey
Do providers access the EMR
remotely to provide patient care?
% of physicians who have remote
access to EMR and use it for patient
care
CC5 Provider Survey, Interview
Chronic Disease Management
Does EMR use improve chronic
disease management?
% of PHC clients/patients, 18 years
and over, with diabetes mellitus in
whom the last HbA1c was 7.0% or less
(or equivalent test/reference range
depending on local laboratory) in the
last 15 months
CDM1
Administrative/ EMR data, CIHI
PHC-VRS (CIHI #39), Trending
over time
% of patient population, age 18 and
older, with diabetes mellitus who
received testing for all of the following:
Hemoglobin A1c (HbA1c); Full fasting
lipid profile screening; Nephropathy
screening; Foot examination; Blood
pressure measurement; and
Obesity/overweight screening
CDM2
Administrative/ EMR data, CIHI
PHC-VRS (CIHI #57-4.1 spec),
Trending over time
% of population, age 20 and older, with
diabetes mellitus who have had an eye
exam
CDM3
Administrative/ EMR data, CIHI
PHC-VRS (CIHI#58-4.1 spec),
Trending over time
Health Promotion, Screening and Prevention
Benefits Realization Study for EMRs in Ontario
PwC
Has EMR use improved the
preventative services provided?
% of practice population, age 65 and
older, who received an influenza
immunization*
HPSP1
Administrative/ EMR data, CIHI
PHC-VRS (CIHI #41-4.1 spec),
Trending over time
% of practice population, age 50 to 74,
who had a screening test ordered for
colon cancer
HPSP2
Administrative/ EMR data, CIHI
PHC-VRS (CIHI # 48), Trending
over time
% of practice population, age 18 and
older, who have had their blood
pressure measured by their primary
health care provider within last 15
months
HPSP3
Administrative/ EMR data, CIHI
PHC-VRS (CIHI #54-4.1 spec),
Trending over time
Efficiency
To what extent does EMR use
improve physician efficiency?
% change in time to complete
clinical/admin documentationE1 Provider survey, Interviews
% of physicians who use EMR as a
referral toolE2
Administrative Data,
Provider survey
EMR Maturity
Extent to which the EMR has been
adopted and used to its full
potential?
% of providers at each level of the EMR
Maturity ModelEM1 OntarioMD EMR MM Tool
Benefits Realization Study for EMRs in Ontario
PwC
Appendix E: EMR Maturity Model
Criteria Description
The Ontario MD EMR Maturity Model (EMM) is defined on a scale of 1-6 and demonstrates progressive levels of
maturity. The description of each maturity level outlines the anticipated benefits that may be derived as the user
builds on existing processes. Under the requirements for Ontario MD’s EMR Adoption Program, funding
requirements for EMR use begin at Level 2 capability.
Below is a description of the EMM Criteria (OntarioMD Inc., 2012).
Benefits Realization Study for EMRs in Ontario
PwC
Appendix F: Organizational
Survey
EMR Case Studies: Organizational Survey
PricewaterhouseCoopers LLP (PwC), on behalf of eHealth Ontario, is conducting this survey as part of the EMR
Benefit Realization (BR) project that your clinical site has agreed to participate in. The intent of this survey is to
gather general background information on your practice related to context setting, organizational resources and
EMR users. The information provided will be used to tailor the site visit interview guide to better identify areas of
discussion and interviewees.
Who should complete the survey?
Only one organizational survey is required for each clinical site. Ideally the questionnaire should be completed by
the person most familiar with the clinic operations and workflow; typically this is the Lead Clinician or
Administrative Manager. The survey will take approximately 10 minutes to complete. Please check only ONE
answer per question, unless otherwise indicated. Information gathered from the survey will be kept strictly
confidential and only used for the purposes of the EMR BR Study. If applicable, responses from the survey will be
presented in aggregate within the final report.
Prior to beginning the survey, please read the following instructions:
• "Back" and Next" buttons are provided at the bottom of each page of the survey. Please use these buttons to
navigate through the survey until it is complete. Please do not use the navigation arrows in your browser to do this.
• You may stop the survey at any point and resume at a later time as long as the survey is being completed from the
same location/computer.
• When you resume the survey, you will be asked whether you want to return to the survey where you left off, or
start at the beginning.
In order to begin the survey, please click on the "Next" button below.
If you experience any technical difficulties or have any questions about this survey, please contact Erika Norris at
416 941 8383 ext. 13667 or via e-mail at [email protected]
Thank you for your participation.
Benefits Realization Study for EMRs in Ontario
PwC
A) Identification of the Organization and Context
1. Enter the following practice information:
Your Practice or Group name ___________________
Number of office locations: ____________________
Address: _________________________________
City: ____________________________________
Telephone : _______________________________
Ext: ____________________________________
2. Position of the respondent:
Lead Clinician
Member of the general practitioners team
Administrative Manager
Other ____________________
3. How long has your practice been in operation?
1 to 4 years
5 to 9 years
Over 10 years
4. How would you characterize the place where you are currently practicing?
City
Suburb
Small town
Rural
5. Are you currently or has your practice participated in any of the following quality reporting initiatives via your
EMR?
CIHI PHC-VRS
The Canadian Primary Care Sentinel Surveillance Network (CPCSSN)
Other ____________________
N/A
B) Organizational Resources
6. Please complete the following regarding the number of general practitioners/family physicians in your practice
and their FTEs?
Number of GP/FP: ________
FTEs: _________________
Benefits Realization Study for EMRs in Ontario
PwC
7. Please complete the following regarding the number of staff in your practice:
Nurse Practitioner: ___________________
Registered Nurse: ____________________
Social Worker ______________________
Occupational Therapist: ________________
Pharmacist: ________________________
Physiotherapist: _____________________
Dietitian __________________________
Other ____________________________
8. How many administration staff (managerial, clerical, reception) currently work at your clinic?
____________________
C) EMR Practice Profile
9. Select the Ontario Certified EMR system the clinic is currently using?
ABELMed Inc.
Optimed Software Corporation
Bell Canada/ xwave
P&P Data Systems Inc.
Canadian Health Systems Inc
Alpha Global iT Inc.
Jonoke Software Development Inc.
Med Access Inc
Nightingale Informatix Corporation
OSCAR EMR
MD Physician Services Software Inc.
YES Medical System
YMS
10. Please specify if the EMR system is:
Local
ASP
11. How many years has the clinic been using an EMR system? ____________________
12. Do all the healthcare providers have access to the EMR system (main patient chart)?
____________________
Benefits Realization Study for EMRs in Ontario
PwC
13. What type of health providers currently use the EMR system (main patient chart) when they see
patients? (check all applicable)
Yes No Not
Applicable
Physician(s)
Nurse
Practitioner(s)
Nurses(s)
Other
healthcare
provider(s)
14. Where is the EMR system used? (check all applicable)
Reception
Exam rooms
Other treatment areas (e.g. other providers’ offices, treatment room for emergency cases)
Staff offices
Remotely (e.g. physician homes via VPN)
Other ____________________
D) Additional Information
15. If you have any additional comments about your use of the EMR system that you would like to share with us,
please write them down in the space provided below.
______________________________________________________________
______________________________________________________________
______________________________________________________________
THANK YOU FOR COMPLETING THIS IMPORTANT SURVEY.
YOUR TIME AND SUPPORT ARE HIGHLY APPRECIATED.
Benefits Realization Study for EMRs in Ontario
PwC
Appendix G: Provider Survey
EMR Case Studies: Provider Survey
PricewaterhouseCoopers LLP (PwC) is conducting this survey, on behalf of eHealth Ontario, as part of the EMR
Benefit Realization project that your clinical site has kindly offered and agreed to participate. The purpose of the
survey is to gather information on EMR adoption and current use. The survey is being distributed to selected
providers working in the clinical practice.
There are two parts to the survey:
Part I- EMR Experience Survey- consists of 6 questions focused on provider perception of benefits of EMR.
Part II- EMR Progress Survey- consists of questions on EMR User/Practice Profile and 25 questions across
each of seven key functional areas which allows for the measurement of maturity on a continuum of one through six
(indicating progressively advanced use or practices). Each level of maturity builds upon the functionality or
maturity state of the preceding level. EMR capability starts at Level 2 based on the funding requirements under the
OntarioMD’s EMR Adoption Program. The survey tool has been developed with direct input from OntarioMD
clinician and clinic manager peer leaders. A separate link for this survey is found at the end of the
EMR Experience Survey.
Prior to beginning the survey, please read the following instructions:
• Completing this surveys will take approximately 30-40 minutes of your time
• Information that is collected during this survey will be kept strictly confidential and only used for the purposes of
this study. The responses to the survey will be presented in the report on aggregate.
• "Back" and Next" buttons are provided at the bottom of each page of the survey. Please use these buttons to
navigate through the survey until it is complete. Please do not use the navigation arrows in your browser to do
this.
• You may stop the survey at any point and resume at a later time as long as the survey is being completed from the
same location/computer.
• When you resume the survey, you will be asked whether you want to return to the survey where you left off, or
start at the beginning.
In order to begin the survey, please click on the "Next" button below.
If you experience any technical difficulties or have any questions about this survey, please contact Erika Norris at
416 941 8383 ext. 13667 or via e-mail at [email protected]
Benefits Realization Study for EMRs in Ontario
PwC
Part I- EMR Experience Survey
A) Provider Profile
1a. Your Practice or Group name
(Complete name) ____________________
1b. Contact information
First Name ____________________
Last Name ____________________
2. Since the adoption of the EMR system, how would you rate yourself as an EMR user?
Expert
Above Average
Average
Below Average
Novice
3. How many years have you been using an EMR system: (Number) ____________________
B) Impact of EMR
4. Please indicate your level of agreement or disagreement with each of the following statements related to Quality:
The EMR System…
Strongly Agree Moderately
Agree
Moderately
Disagree
Strongly
Disagree
Don’t Know or
N/A
Enhances the quality of care
delivered to patients in our
practice
Improves the management of
chronic diseases
Improves patient safety and
proactive monitoring of overdue
tests/exams
Enables the practice to identify
patients for changes in
management based on new
evidence (e.g., drug recalls)
Improves the decision-making
via the clinical decision support
tools (e.g. alerts, reminders, etc.)
Supports remote access and use
by providers for patient care
Enables the practice to do needs
planning (e.g., plan for patient
population/community)
Enables the practice to audit and
improve the practice
Benefits Realization Study for EMRs in Ontario
PwC
5. Please indicate your level of agreement or disagreement with each of the following statements related to
Communication and Coordination:
The EMR System…
Strongly Agree Moderately
Agree
Moderately
Disagree
Strongly
Disagree
Don’t Know
or N/A
Improves the sharing of
patient information with
providers internal to our
practice
Supports interdisciplinary
care coordination in our
practice
Enhances our practice’s
ability to coordinate
patient care
6. Please indicate your level of agreement or disagreement with each of the following statements related to
Efficiency:
The EMR System…
Strongly Agree Moderately
Agree
Moderately
Disagree
Strongly
Disagree
Don’t Know or
N/A
Enhances the efficiency of
ordering lab tests,
prescriptions, etc.
Improves the productivity
(output) of my practice
Improves the efficiency
(reduction in effort) of my
practice
Reduces the number of call
backs and/or time spent
responding to pharmacist
requests for physician
verification of prescriptions
Reduces the time from
when a laboratory test
result is available to when
the result is received by the
EMR
Improves administrative
efficiencies e.g. use of pre-
populated templates, forms
and stamps, report chart
pulling
Facilitates preventative care
incentives e.g. through
cohort management for
influenza immunization
Benefits Realization Study for EMRs in Ontario
PwC
Improves the claims
submission process e.g.
decrease in billing errors
Has facilitated a reduction
in net overhead costs e.g.
space, office supplies etc.
7. Please indicate your level of agreement or disagreement with each of the following statements related to Patient
Experience:
The EMR System…
Strongly
Agree
Moderately
Agree
Moderately
Disagree
Strongly
Disagree
Don’t Know or
N/A
Improves patient
satisfaction/experience with
the care they receive
Supports patient education
(e.g. electronic educational
material/references or
trending patterns such as
BMI, blood sugar levels and
labs etc?
8. Please provide examples of how the use of an EMR system has impacted the delivery of care (e.g., the impact on
quality of care, access to care, efficiency of care, etc)
______________________________________________________________
______________________________________________________________
______________________________________________________________
9. Do you have any suggestions/recommendations on how to improve EMR use? (e.g. how do we ensure mature use
of EMRs?)
______________________________________________________________
______________________________________________________________
______________________________________________________________
C) Additional Information
10. If you have any additional comments about your use of the EMR that you would like to share with us, please
write them down in the space provided below.
______________________________________________________________
______________________________________________________________
______________________________________________________________
THANK YOU FOR COMPLETING PART I OF THE PROVIDER SURVEY.
YOUR TIME AND SUPPORT ARE HIGHLY APPRECIATED.
Benefits Realization Study for EMRs in Ontario
PwC
Part II- EMR Progress Survey
Upon completion of Part I- EMR Experience Survey, participants were required to select the hyperlink provided to
take them to Part II, which was administered through OntarioMD. Survey questions can be found in the attached
document (Page 13).
OntarioMD_EMR_Maturity_Model_Framework_v1_3.pdf
Benefits Realization Study for EMRs in Ontario
PwC
Appendix H: Site Visit Interview
GuidePhase 2 – EMR Benefits Realization Case Studies
General Interview Guide
Questions
EMR Benefits1. Related to your EMR use, please describe a typical visit and how often you document/use tools within the
EMR
2. What do you think the benefits are? What do you attribute the benefits to?
3. How do you assess the clinical effects of the EMR? How do you notice if the EMR has made things
better/worse? Are there any objective data to show the effects?
Quality of Care and Access to Care4. In your opinion, what are the benefits of the EMR on patient care? Please provide examples of how you use
your EMR system to improve your quality of care and access to care.
5. Do you use the EMR system for population health services (e.g., tests/screening; CDM)
6. Has the EMR impacted the patient encounter? If so, please describe/provide examples.
7. What is missing that would help you in caring for your patients?
Patient Experience8. Do you use any specific features/capabilities of your EMR with your patients?
9. What are some of the innovative things you can do regarding patient care?
Efficiency10. Has the EMR improved efficiency of the practice, providers, etc? Please provide examples.
EMR Best Practices and Lessons Learned11. What are some of the best practices and lessons learned regarding EMR adoption and use?
12. Looking into the future, what are the possibilities for the EMR? What are your future plans?
Other Comments13. Any other comments? Can you think of anything else that might help us to learn from implementing the
EMR in your practice?