running head: too much data, not enough data...

171
Running Head: TOO MUCH DATA, NOT ENOUGH DATA 1 Too Much Data, Not Enough Data: Providing Relevance to Care Connectivity Consortium Providers and Their Patients Kathleen Merkley, MS, RN, FNP, ANP Electronic Clinical Information Management Implementation Director Intermountain Health Care University of Utah College of Nursing “In partial fulfillment of the requirements for the Doctor of Nursing Practice”

Upload: vuongkhanh

Post on 21-Aug-2018

239 views

Category:

Documents


0 download

TRANSCRIPT

Running Head: TOO MUCH DATA, NOT ENOUGH DATA 1

Too Much Data, Not Enough Data: Providing Relevance to Care Connectivity

Consortium Providers and Their Patients

Kathleen Merkley, MS, RN, FNP, ANP

Electronic Clinical Information Management Implementation Director

Intermountain Health Care

University of Utah College of Nursing

“In partial fulfillment of the requirements for the Doctor of Nursing Practice”

TOO MUCH DATA, NOT ENOUGH DATA 2

Table of Contents

Executive Summary ………………………………………………………………………………6

Introduction………………………………………………………………………………………..7

Significance of the Project………………………………………………………………………...9

Project Objectives………………………………………………………………………………..11

Search Strategy ...………………………………………………………………………………..12

Literature Review ………………………………………………………………………………..12

Meaningful Use Regulations for Electronic Health Records…………………………….13

The Importance of Interoperability………………………………………………………15

The Care Connectivity Consortium……………………………………………………...17

Data Overload – A Risk for Success……………………………………………………..18

Big Data – an Additional Consideration…………………………………………………20

Privacy and Security Burdens Associated with Increased Data ……………………....21

Taxonomy of Value Accrual…………………………………………………………….23

Clinical Document Architecture and the Continuity of Care Document………………..24

Data Sharing Model and Transferring of Information…………………………………..25

Ethical and Legal Considerations of Data Selection ……………………………………26

Shared Accountability in Data Exchange ……………………………………………….27

The Provider-Patient Relationship and Data Exchange …………………………………29

Trusting of Exchanged Data……………………………………………………………..30

Patients as Gatekeepers – Personally Controlled Health Records……………………….31

What Constitutes Appropriate Data ……………………………………………………..33

Moderators/Types of HIE Access ….……………………………………………………36

TOO MUCH DATA, NOT ENOUGH DATA 3

Accessing Appropriate Data …….………………………………………………………38

Presentation of Received Data…………………………………………………………...40

Reconciliation of Received and Existing Data…………………………………………..41

Are Data Requirements Different Between Caregivers and Patient Types ……………..42

Theoretical Framework ………………………………………………………………………….44

Implementation…………………………………………………………………………………..46

Evaluation Plan…………………………………………………………………………………..51

Project Results…………………………………………………………………………………...54

Relevant Data by Specialty……………………………………………………….….......54

What Constitutes Relevant Data for Critical Care Patients……………………….……..56

Top Five Types of Priority Data Related to Specialty………………………….………..57

Presentation of Exchanged Data - Separate or Integrated……………………………….58

Reconciliation of All Data……………………………………………………………….58

What Makes Data Trustworthy…………………………………………………..………59

Time Limits of Different Types of Exchanged Data…………………………………….60

Discussion………………………………………………………………………………………..63

Project Recommendations……………………………………………………………………….66

Summary…………………………………………………………………………………………67

References………………………………………………………………………………………..68

Tables

Table 1- Data Value Classification Taxonomy …..……………………………………………...23

Table 2 - ED Physician Access of Data from a HIE ….…………………………………………43

Table 3 - ED Physician Preference of Data Display ………………………………….…………43

TOO MUCH DATA, NOT ENOUGH DATA 4

Table 4 - Relevant Data Points to be Surveyed …………………………………………………47

Table 5 - Time Line for Survey Completion ……………………………………………………52

Table 6 – Survey Participant Breakdown………………………………………………………..54

Table 7 – Relevant Data by Specialty……………………………………………………………55

Table 8 – Relevant Data for Critical Patients……………………………………………………56

Table 9 – What Makes Data Trustworthy…………………………………………………….....59

Table 10 – Relevant Data for LIPs in 2003……………………………………………………...64

Figures

Figure 1 - Data Value Classifications…………………………………………………………... 24

Figure 2 – Identification of Most Valuable Data………………………………………………..36

Figure 3 – Presentation of Received Data………………………………………………………40

Figure 4 – Reconciliation of Received and Existing Data………………………………………41

Figure 5 - DIKW Framework……………………………………….…………………………...45

Figure 6 – Top Five Data Types by Specialty…………………………………………………...58

Figure 7 – Useful Lifecycle of Data Types………………………………………………………62

Appendices

Appendix A- IRB Approval………………………………………………………………….....75

Appendix B – List of CCC Committees……………………………………………………......79

Appendix C – Pre-Survey Letter ……………………………………………………………….81

Appendix D – Survey Invitation Letter CCC Members………………………………….……..84

Appendix E – Survey Invitation Letter Intermountain Healthcare Participants………………..86

Appendix F – Thank You Letter ……………………………………………………………….88

Appendix G – CCC Recommendations………………………………………………………...90

TOO MUCH DATA, NOT ENOUGH DATA 5

Appendix H – Electronic Data Relevance Survey – Emergency……………………….……….96

Appendix I –Electronic Data Relevance Survey – Inpatient Nursing …………………….…..104

Appendix J – Electronic Data Relevance Survey – Inpatient……………………………….…112

Appendix K– Electronic Data Relevance Survey – Primary Care……………………………..120

Appendix L – Electronic Data Relevance Survey – Electronic Copy …………………………128

Appendix M – Electronic Data Relevance Survey – Pediatrics………………………………..136

Appendix N – Electronic Data Relevance Survey – Primary Care Nursing…………………...144

Appendix O – Relevant Data by Specialty……………………………………………………..152

Appendix P – Relevant Emergent Data………………………………………………………...159

Appendix Q - Presentation of Exchanged Data – Separate or Integrated……………………...164

Appendix R – Reconciliation of Exchanged Data……………………………………………..167

TOO MUCH DATA, NOT ENOUGH DATA 6

Executive Summary

The Care Connectivity Consortium (CCC), a consortium of five leading U.S. healthcare

organizations was recently formed to help promote electronic transfer of health information

across the country. Electronic health information transfer has also become a mandate for

“Meaningful Use” in a recent provision of the American Recovery and Reinvestment Act.

Working from the hypotheses that more data are not always helpful when transferred

electronically, this DNP scholarly project identified what electronic data are relevant to specialty,

patient type, patient acuity and chronicity of illness by surveying CCC clinicians. It also

determined what clinicians consider to be trustable data, what data should be exchanged during

emergent situations and what time limitations may be placed on certain data categories. Survey

questions on these topics were determined after an extensive review of the literature.

The following Essentials of Doctoral Education for Advanced Nursing Practice criteria

helped guide this project development. These include: Essential I: Scientific Underpinnings for

Practice, Essential II: Organizational and Systems Leadership for Quality Improvement and

Systems Thinking, Essential III: Clinical Scholarship and analytical Methods for Evidence Based

Practice, Essential IV: Utilizing Information Systems/Technology for the Improvement and

Transformation of Health Care, Essential VI: Interprofessional Collaboration for Improving

Patient and Population Health Outcomes, Essential VII: Clinical Prevention and Population

Health for Improving the Nation’s Health, and Essential VIII: Advanced Nursing Practice

(2006).

The philosophical foundation of the project is based on the Data-Information-

Knowledge-Wisdom (DIKW) theoretical framework taken from the Graves and Corcoran (1989)

article “The Study of Nursing Informatics”. Methodology for the project was modeled in part

using a modified Delphi technique. The survey was administered electronically or by paper

survey to 159 CCC clinicians across the country. Evaluation of results was accomplished with

the help of content experts from the CCC. The final evaluation relating to the success of the

project will be realized if survey recommendations are incorporated into the CCC data exchange

structure, which is outside the DNP project scope

Relevant, pertinent and timely electronic data passed through a health information

exchange must become a recognized and critical component in providing better patient care.

Knowing and understanding the exact health information that specific providers need at certain

point in the treatment process will only enhance the quality of care given to patients. This survey

has begun the exploratory process of identifying what health information should be exchanged.

TOO MUCH DATA, NOT ENOUGH DATA 7

A 65-year-old male from the Midwest is vacationing on the West Coast. He arrives at

6:00 a.m. to a local emergency department complaining of chest pain for the last 60 minutes. He

has a history of congestive heart failure and Type II Diabetes Mellitus. He is immediately given

oxygen, an aspirin, IV access is obtained, cardiac enzymes are drawn and an ECG is ordered.

The ECG reveals equivocal changes. The patient care nurse is asked to locate an old ECG

obtained two weeks ago from an emergency department in his home town. She spends the next

45 minutes contacting his hospital’s health information management (HIM) department, having

the patient sign a release of information, faxing it, re-contacting the HIM department who had

sent the ECG to the wrong fax number and finally, watching it spit out of the fax machine.

Precious time has been lost if the patient is indeed infarcting his heart muscle.

The scenario described above demonstrates that the ability to rapidly transfer patient

information between health care systems is currently inadequate. Because of the rapid pace of

emergency care, delays in accessing health information can impact outcomes. Yet there is hope

on the horizon. Data transfer is becoming a recognized and critical component in providing

better patient care.

The Care Connectivity Consortium (CCC), a consortium of five leading healthcare

organizations including Intermountain Healthcare, Mayo Clinic, Kaiser-Permanente, Group

Health and Geisinger Health Systems was formed in late 2011 to help promote the electronic

transfer of health information across the country. The CCC considers the development of

electronic medical information transfer as a crucial care support tool needed in healthcare

reform. First, support tools for patient identification, patient directed consents and authorization

will need to emerge in order to achieve the envisioned shared accountability models of

healthcare reform.

TOO MUCH DATA, NOT ENOUGH DATA 8

Next an inquiry surfaces related to the actual types of data that should be exchanged to

provide better coordination and improved care. Currently the CCC provides only the patient’s

problems, medication and allergy lists between its members. Very little information is currently

available on what types of data are appropriate to exchange. No precedence has yet been set on

this subject. It is hoped this project will help define these standards.

Working from the hypothesis that more data are not always helpful, this project attempted

to determine what additional data elements should be exchanged related to patient type, age,

severity and chronicity of the illness and requesting caregiver. The goal was to identify which

data exchanged is clinically pertinent to care providers receiving and caring for patients from

another care setting.

The long term clinical implications of this project will attempt to deliver health

information that will provide care givers with the right data at the right time and in the right

place across health care systems, regardless of organizational borders. This will ultimately lead

to better patient care.

Significance of the Project

It was the assumption of this project that knowing and understanding the specific health

information providers require at a certain point in the treatment process will not only enhance the

quality of care and services, but lead to better informed decisions, promote patient safety,

advance health outcomes, prompt stronger patient-provider relationships and decrease health

care expenditures. Several groups have facilitated, supported or reviewed this project. They

include the Care Connectivity Consortium Network Information Technology (IT) Board, the

Health Information Technology (IT) Standards Committee, Health Level 7 (HL7), and the Utah

Health Information Network (UHIN).

TOO MUCH DATA, NOT ENOUGH DATA 9

The first group is the CCC Network IT Board. This is the governing body of the CCC.

Membership includes representatives from all five consortium members. They approved group

participation in the survey process and helped identify and administer surveys to various

provider types within the consortium. This group will review the survey results; determine if the

recommendations are applicable and then, if so, will implement them into the CCC data

exchange process.

The Health IT Standards Committee falls under the Office of the National Coordinator

(ONC) for Health Information Technology (Office of Health Information Technology, 2012).

They have developed the Standard and Operability Framework (S&I Framework) with the

objective to create a robust, repeatable process that will enable the ONC to execute initiatives

that will help improve interoperability and adoption of standards and health information

technology. This committee is interested in the survey results because of their charge to develop

interoperability specifications to support health outcomes and healthcare priorities.

HL7 is an international IT standards development body that provides a framework for the

exchange, integration, sharing and retrieval of electronic health information. These standards

help define how information is packaged and communicated from one party to another (HL7,

2012). Because this group is studying care plan initiatives and what data should be included they

are also interested in survey results as they determine a care plan initiative data set.

The Utah Health Information Network (UHIN) has been working with the national health

care community since 1993 to develop a secure and efficient electronic data exchange network

for hospitals in Utah (Utah Health Information Network, 2012). They have working

relationships with 90% of hospitals and care providers in the state. They are also attentive to the

TOO MUCH DATA, NOT ENOUGH DATA 10

survey results as they are the group that defines what data should be exchanged between Utah

providers, hospitals and clinics.

Shareholders in this project have included CCC providers who have participated in the

project and then will be held accountable to use the shared data that has been defined as

pertinent. Patients seeking care at consortium facilities who must provide access to their data are

also vital in this process.

Project Objectives

To accomplish the goal of identifying which data exchanged is clinically pertinent to care

providers receiving and caring for patients from another care setting the following objectives

have been met.

1. A literature search explored: a) the importance of data sharing, b) associated federal

regulation, c) potential implications of data overload, d) development of a taxonomy

of value accrual, e) data sharing models, f) ethical and legal considerations of data

selection, g) shared accountability, h) the provider patient relationship, i) patients as

gatekeepers of their health information, j) what constitutes appropriate data, k) what

makes data trustworthy, l) presentation of received data, m) reconciliation of

disparate data that is exchanged and n) identification of data elements and related

patient populations and provider needs to be examined.

2. Based on the literature review, a survey was developed with potential data elements

to be exchanged and was distributed to consortium members for their input as to

importance and necessity.

3. The survey was administered to providers in various specialty areas who care for

different patient types. IRB approval was received for this.

TOO MUCH DATA, NOT ENOUGH DATA 11

4. Lastly, the survey results were analyzed and recommendations were made regarding

additional data elements that should be included in electronic data sharing related to

patient type and provider specialty. These recommendations were sent to the CCC

Network IT Board to be used as input on what future data should be shared with

consortium members.

Search Strategy

An extensive literature search was performed using Pub Med and CINAHL databases, the

Department of Health and Human Services, National Health Information Exchange, Office of

Health Information Technology (2012), HL7, Integrating the Healthcare Enterprise (IHE), and

Utah Health Information Network websites, as well as reviewing numerous issues of the Journal

of the American Medical Informatics Association (JAMIA). The terms health information

exchange, provider, care coordination, meaningful use regulations, continuity of care documents,

alert fatigue, national health information network, shared accountability, person health records,

data relevance and theoretical model were used in the search. In this document “provider” or

“clinician” is defined as a medical doctor, doctor of osteopathic medicine, nurse practitioner,

physician assistant or registered nurse. Licensed independent practitioner (LIP) is defined as a

medical doctor, doctor of osteopathic medicine, nurse practitioner or physician assistant.

Literature Review

Health information exchanges involves sharing of clinical, financial and demographic

data among health care stakeholders in support of care delivery, financing, public health

surveillance, research, and other health system activities. Balfour et al. (2009) reports, “Use of

these systems results in improvements in quality of care such as avoidance of redundant tests”

TOO MUCH DATA, NOT ENOUGH DATA 12

(p.11). Information exchange may also prevent hospital admissions related to medication

allergies, errors, or interactions and lowers costs of caring for chronically ill patients.

Johnson et al. (2011) recognizes that Americans increasingly seek healthcare from

multiple organizations because of insurance restrictions, availability of specialists and ease of

travel. The process of accessing information across disparate systems is difficult, particularly

when patients don’t remember where care was provided. This leads to incomplete patient data.

“This knowledge ‘blind spot’ increases healthcare costs when previously performed tests and

procedures must be duplicated to provide decisions makers with data” (p. 690).

Various studies on coordination of care indicate that people with multiple chronic

conditions are more likely to be hospitalized, see a variety of physicians, take several

prescription drugs and be visited at home by health workers. Furthermore, the poor coordination

of care has been associated with poor clinical outcomes such as unnecessary hospitalization,

duplicate tests, conflicting clinical advice and adverse drug reactions. All this suggests a need

for better care coordination and information sharing among providers (Burton, Anderson, Kues,

2004). Sharing of patient information is also known to provide higher quality at lower cost.

Meaningful Use Regulation for Electronic Health Records

Electronic health records (EHR) are becoming more widespread throughout the United

States. However the transition has been slow and cumbersome. Blumenthal and Tavenner

(2010) believe that the Health Information Technology for Economic and Clinical Health Act

(HITECH) sponsored by the Obama administration and passed by the 2009 Congress will

provide the health care community with a transformational opportunity to break through these

barriers.

TOO MUCH DATA, NOT ENOUGH DATA 13

The HITECH provision of the American Recovery and Reinvestment Act of 2009

provides billions of dollars in incentives for adoption and use of health information technology

by Medicare and Medicaid providers over the next ten years. To receive the financial incentives,

licensed independent practitioners and hospitals must achieve “Meaningful Use” (MU) of an

electronic health record. They are required to comply with a set of core objectives which

initially focus on entering in basic information (demographics, vital signs, active medications,

allergies, up-to date problem lists and active diagnoses) but will soon include incorporating lab

results, issuing reminders for care, providing specific patient education, clinical documentation,

decision support, disease and medication management, as well as quality and safety measures

(Office of Health Information Technology, 2012).

Another explicit goal of the act is a provider-to-provider data exchange which is essential

for the long-term success of the Affordable Care Act of 2010 (Rudin, Volk, Simons & Bates,

2011). Maslove, Rizk & Lowe (2012) agree that these required computer-based technologies

used to produce, manage and share health-related information, grouped under the umbrella term

“Health Information Technology” (HIT) are a means to improve the quality, safety, and

efficiency of health care delivery in a growing number of countries.

The recent HITECH economic stimulus package contains considerable funding for the

development of health information technology architecture that will support the nationwide

electronic exchange. This exchange will also be a means of providing research and surveillance

data. Most important however, Francis believes that, “What the provider records about the

patient – and what the provider recommends with respect to the patient will no longer reside in

the microcosm of a single office” (2010, p. 36). When expressed in these terms, one can see

how the HITECH provision will be “meaningful” to both patients and providers.

TOO MUCH DATA, NOT ENOUGH DATA 14

The Importance of Interoperability

Interoperability is the term used to define the ability of information technology (IT)

systems to exchange information and is the key element of the CCC. A vision of interoperability

and its benefits was defined in 2001. Since that time, important advances toward the goal have

been made related to a number of government initiates (Kuperman, 2011). The Office of the

National Coordinator for Health Information Technology (ONC) within the Health and Human

Services Department chartered a National Health Information Network Trial Implementations

project in 2010. The purpose of the project was to demonstrate data exchange among operational

health-information exchanges. The project sought to demonstrate that eight interoperability

scenarios could be technically accomplished. These scenarios included:

1. EHR-laboratory results – incorporate new lab results into the ordering provider’s

HER,

2. Emergency responder - provide the provider with access to the patient’s data in an

emergency scenario,

3. Medication management – support access to the patient’s medication and allergy data in a

medication reconciliation scenario,

4. Quality – communicate quality-related information from a provider organization to

another organization,

5. Social security administration (SSA) – allow the SSA to retrieve the patient’s data to

make a disability-benefits determination,

6. Bio-surveillance – data collection to support situational awareness, event detection and

outbreak management,

TOO MUCH DATA, NOT ENOUGH DATA 15

7. Consumer access to clinical information – allow consumers to access their data via a

personal health record, and

8. Consumer empowerment – allow the consumer to authorize the provider to have a view

of his or her data (National Health Information Network, p. 678).

When viewing this list, the potential and importance of sharing health care data across systems

become very apparent.

One specific model of interoperability, called the health information exchange (HIE) has

emerged to address patient-centered information access. The HIE “attempts to make available on

an incremental and local basis, comprehensive patient-centered information access where care is

needed” (Frisse, p 51). Early evaluations of HIEs reveal they make a difference in patient care

but have not yet begun to reach their potential.

One HIE now in operation, The Memphis Health Information Exchange, claims that more

information available to providers will uniformly impact provider-patient communication in

positive ways. Frisse (2010) says, “Our limited experience suggests that such data can be of

great anecdotal aid during acute situations by allowing patients and their physicians to begin with

a greater common knowledge of past medical history” (p.56). He reports that because of the

HIE, providers and patients have seen a new vision of care. At the institutional level, it has also

demonstrated that sharing patient data across traditional health organization boundaries is a low

cost solution for better patient care.

Of the more than one hundred HIEs in the United States, few are fully operational. Even

fewer have advanced data exchange capabilities such as providing access to comprehensive data

that may originate from many different medical practices in the form of aggregate patient-centric

records (Rudin et al., 2011). Multiplicity of systems with multiple HIEs requirements is a major

TOO MUCH DATA, NOT ENOUGH DATA 16

barrier. Interoperability remains elusive, yet there is hope on the horizon as healthcare

organizations begin to recognize its value to the patient, the provider, the organization and

national healthcare as a whole.

A side benefit of robust health information exchanges must also be examined in the

public health domain. Public health officials could survey disease trends and recognize

variations. This rapid surveillance could lead to timely interventions as well as lives saved.

Health information exchanges could also improve patient safety by tracking preventable deaths

and medication errors. Improved quality of care could be more easily tracked and healthcare

costs might be more easily assessed (Yasnoff, 2010). Currently some data is already exchanged

in public health departments across the country which is impacting health care for the better.

The Care Connectivity Consortium

In 2011, Kaiser-Permanente, Mayo Clinic, Geisinger Health Systems, Group Health

Cooperative and Intermountain Healthcare joined forces in an interoperability and data sharing

collaboration- the CCC. Each of the systems has individually demonstrated the value of health

IT by improving the quality of care for patients across the continuum of care. The consortium is

working together to expand the understanding of what type of data transfer is possible in

connected healthcare systems and to demonstrate better and safer care through better data

availability. The focus will be on accelerating the process to form the National Health

Information Network or NHIN (MTBC, 2011).

The CCC is creating a future where timely access to health information results in patient-

centered, community-wide, evidence-based care (2012. Securely connecting care givers to

patient data regardless of organizational boundaries, ensures better-informed decisions and

stronger patient-provider relationships.

TOO MUCH DATA, NOT ENOUGH DATA 17

George Halvorson, CEO of Kaiser-Permanente believes that the importance of such and

initiative cannot be overstated:

This collaborative effort exists because we all have reached the same conclusion about

linking and sharing patient-specific data. Our five organizations share the common

mission of improving healthcare in the United States and our belief is that when doctors

have real-time data about patients, care is better and effective (Intermountain

Stories, 2011, p.3).

It is the goal of the collaboration to accelerate the implementation of national health IT

standards. One concern of this type of data exchange is violation of patient privacy. Marc

Probst, chief information officer of Intermountain Healthcare asserts that “Patient privacy and

security are the overarching priorities” (Intermountain Stories, 2011, p. 2).

As stated above, currently the consortium only exchanges patient medication, problem

and allergy lists. Intermountain has been tasked with identifying what constitutes additional

appropriate data elements. This charge comes with the mission of identifying what information

will be meaningful to providers without jeopardizing patient-provider relationship, identifying

associated ethical and legal issues and determining what constitutes too much data.

Data Overload – A Risk for Success?

In As You Like It, William Shakespeare claims, “Why then, can one desire too much of a

good thing?” Indeed, can there be too much of a good thing when it relates to patient data?

Providers perform many tasks in their daily work which require summarization of data to

identify pertinent clinical information. As technology makes more data available, the challenges

of data overload become increasingly important. “Much attention has been given to the notion of

evidence-based medicine and how to address the information needs of providers to answer

TOO MUCH DATA, NOT ENOUGH DATA 18

clinical questions and support decision-making” (Tielman, Van Vleck, Stein, Stetson, &

Johnson, 2007, p. 761). Less attention, however, has been focused on how to help providers

navigate the substantial amount of clinical data that is accruing for each individual patient,

A practicing physician, Westby Fisher (2012) stresses providers are accumulating too

much data and not enough appropriate information. Another provider, Jennifer Dennard (2012)

says providers are swimming in too much electronic data. When observing the extensive patient

information in a hospital electronic health record (EHR) and then connecting two or more

institutions together the pool of information becomes even larger. It may then be difficult to

differentiate what information is actually pertinent.

Pho (2012) alleges that too much data- whether it is written or on a screen – can

overwhelm physicians and potentially place patients at harm. Review of test results can

specifically be overwhelming. He believes curating test results by ordering abnormal ones, will

really be the true power of electronic test reporting. The Wolters Kluwer Health 2011 Point-of-

Care Survey found that the second largest barrier to technology adoption by primary care

physicians is “Too much data, not enough actionable information” (32%) preceded only by “Too

expensive” (40%).

The downside of too much data, according to Daigh (2002) is that “A data dump will

waste a physician’s time, destroying the original intent of medical records with meaningless

repetition and templates, satisfying demands of third-party payers but not of physicians’

thoughtful review and analysis” (p. 1). Fisher (2012) avows that providers need better

information, not more of it. “We don’t need to know what ‘type’ of order we entered, for

instance, we need more time with our patients and less time with data entry” (p.3). Deciding

TOO MUCH DATA, NOT ENOUGH DATA 19

what information goes and what stays in the medical record must be considered the priority as

we move forward.

A phenomenon known as “Alert Fatigue” should also be taken into consideration when

considering the issue of too much data. Alert fatigue results because of repeated exposures to

alerts from various decision support mechanisms such as medication allergies , drug-drug

interactions, vital sign abnormalities, unsaved data etc. leading to a decline in user

responsiveness (Cash, 2009).

A limited literature review notes that it is a well-recognized fact that when providers are

exposed to frequent clinical decision support alerts they may eventually stop responding to them.

This is thought to be related to issues such as alert irrelevance and cognitive overload (Embi &

Leonard, 2012). Can the same assumptions be made about irrelevant and repetitive data?

Most of the criteria related to types of data sharing have not yet been specified for the

National Health Information Exchange, but official statements suggest that they will require

advanced HIE functionality in the form of “access to comprehensive patient data” (Department

of Health and Human Service, 2010, p.1). What is comprehensive patient data? This is a

question this project has helped to resolve.

Big Data – an Additional Consideration

If “comprehensive data” isn’t enough to create concern, a new concept called “big data”

must be added to the equation. Richmond (2012) says there has been a recent data explosion.

He calls hospitals “factories of data” (p. 2).

Big data, as described by the McKinsey Global Institute (2011) is, “Datasets whose size

is beyond the ability of typical database software tools to capture, store, manage and analyze”

(p.3). Big data can be considered both blight and an opportunity. Having such a paucity of data

TOO MUCH DATA, NOT ENOUGH DATA 20

makes is possible to do things never before possible. “Greater volumes of data to sift through to

find critical insights (the proverbial needle in the digital haystack) is a growing problem for

companies, organizations and governments the world over,” (Richmond, 2012, p.1).

The McKinsey Global Institute (2011) believes that correct management of this

information can generate significant financial value across certain sectors of the world-wide

economy, health care being one of those sectors. There is a huge opportunity for healthcare as a

whole. Much of this data if used correctly has the potential to reduce healthcare costs. Again

Richmond (2012) emphasizes that healthcare providers have a long way to go before they can,

Come even close to realizing value creation, efficiency improvements and cost savings.

A highly fragmented data environment begets disconnected strategies and uncoordinated

decision making. By connecting the dots and leveraging the power and promise of data

assets, hospitals can improve the practice, delivery, and economics of healthcare. To

accomplish these ambitious goals, hospitals need to first make some significant changes

in how they handle big data (p. 3).

Privacy and Security Burdens Associated with Increased Data

A patient is referred to a cardiologist’s office to see the nurse practitioner for new onset

congestive heart failure. The patient’s medical history contains information about a recent visit

with a dermatologist for a mole removal. If data were to be exchanged, would it need to include

information regarding the dermatology visit or just pertinent data about the ED visit three days

ago for chest pain? When does data stop being helpful and become a data dumping ground?

Many clinicians will avow that they want access to all of a patient’s data or worse yet

none at all. They want to start a patient encounter with a “clean slate”. Would it make more

sense to provide the clinician with a subset of relevant data potentially increasing its usage and

TOO MUCH DATA, NOT ENOUGH DATA 21

then providing the ability to obtain additional and logical segments as needed? Can a clinician

agree to the concept of limiting the data that should be exchanged? These are questions that are

yet to be answered.

Today, many systems have defined clinical summaries as the vehicle for data exchange.

A summary may contain up to 300 pages of information. This is a sizeable amount of data for

any clinician to digest. If these 300 pages have been exchanged from one site to another, is the

receiving clinician accountable for all data contained in this clinical summary? Jutta Williams,

Compliance Officer at Intermountain Healthcare believes he is. She posits that because of the

large volume of information exchanged, everyone’s liability is increased and she recommends

that a HIE should bring in only information that is relevant and has been parsed appropriately

(personal conversation, July 17, 2012).

Ms. Williams advocates using the “minimum necessary” standard identified in the Health

Insurance Portability and Accounting Act (HIPAA) as a go forward strategy with data exchange.

The minimum necessary standard asserts that

Protected health information should not be used or disclosed when it is not necessary to

satisfy a particular purpose or carry out a function. It requires entities to evaluate

practices and enhance safeguards as needed to limit unnecessary or inappropriate access

to and disclosure of protected health information (Department of Health and Human

Services, 2012, p.1).

This guideline was developed to protect patients, but perhaps it might also protect clinicians if

used in a similar context with data exchange (HIPPA Regulations, 2012). One thing becomes

apparent, information should be shared responsibly but only that information which is necessary

should be distributed.

TOO MUCH DATA, NOT ENOUGH DATA 22

Taxonomy of Value Accrual

Taxonomy is the science or technique of classification. Classifications are then ordered

into categories. From one type of taxonomy many classifications may be produced. The best

known type of taxonomy is used for the categorization of life forms (domain, kingdom, phylum,

class, order, etc.). It might also be helpful to identify a taxonomy of value accrual when looking

at what information is shared in a HIE. Table 1 below defines the following terms: relevance,

pertinence, appropriate, adequate, comprehensive, consistent, timeliness, precision, missing and

accuracy. These descriptors are believed to be valuable and necessary as an attempt is made to

provide clinicians with the consumable data. Following an elaboration of the definitions, a

diagram was developed to visually explain this classification as shown in Figure 1.

Table 1 Data Value Classifications Taxonomy

Relevance The condition of being relevant or connected with the matter at hand, bearing

upon or connected with the matter in hand, pertinent

Pertinence Pertinent or relating directly and significantly to the matter at hand; relevant

Appropriate Suitable or fitting for a particular purpose, person, occasion

Comprehensive A large scope, covering or involving much; inclusive

Adequate As much or as good as necessary for some requirement for purpose, fully

sufficient, suitable or fit

Competent Agreeing or accordant, compatible, not self-contradictory, constantly adhering

to the same principles, course, form, holding firmly together

Timeliness Occurring at a suitable time; seasonable; well-timed

Opportune Favorable or suitable, meets exactly the demands of the time or occasion

Accuracy The condition of being true, correct or exact; freedom from error, exactness,

correctness

Precision Accuracy, exactness, strictness

Available Having a beneficial effect, valid, usable

Missing Absent or lost

Merriam-Webster’s Collegiate Dictionary, 12th

Edition. Springfield, MA: G. & C. Merriam

Company, Publishers, 2012.

TOO MUCH DATA, NOT ENOUGH DATA 23

Figure 1 Data Value Classifications

Developed by K. Merkley, July 2012.

Clinical Document Architecture and the Continuity of Care Document

The Clinical Document Architecture (CDA) is a strategy developed to stipulate the

structure, semantics and encoding of clinical documents for electronic exchange (Wikipedia,

2011). It is the basis for the Continuity of Care Document (CCD) discussed below.

In July 2010, the Department of Health and Human Services approved the Continuity of

Care Document (CCD) as a way of meeting the goals of clinical data exchange for “Meaningful

Use”. The CCD is a patient summary that contains a fundamental data set of the most pertinent

clinical, administrative and demographic information in a patient’s healthcare encounters. “It

provides the means for one healthcare practitioner , system, or setting to aggregate all of the

pertinent data about a patient and forwards it to another practitioner, system, or setting to support

continuity of care,” (Wikipedia, 2012, p.1). The development of this document represents the

Comprehensive

•Accurate

•Percise

•Competent

•Available

•Missing

Suitable for a Particular Purpose

•Relevant

•Pertinent

•Adequate

•Appropriate

Delivery

•Timeliness

•Opportune

•Consistent

TOO MUCH DATA, NOT ENOUGH DATA 24

work of various national and international committees whose goal was to standardize a

continuity of care data set.

The CCD then is a minimum data set that includes provider information, insurance

information, patient’s health status (allergies, medications, vital signs, diagnoses, problem list,

recent procedures), recent care provided, as well as recommendations for a care plan and the

reason for the referral or transfer (Burton et al., 2004). In addition family history, genome

information, psychosocial information, and public health data are considered as important and

should be included.

In a research study looking at the CCD as it relates to interoperability and HIE, D’Amore

(2010) highlighted “the promise of CCDs for population health and recommended changes for

future interoperability standards” (p. 3). In addition, the CCD recommends a compliant structure

for the transfer of free-text as well as codified data. A great deal of effort has previously gone

into this body of standardized data. This information was helpful as the project survey was built.

Data Sharing Model and Transferring of Information

The most important feature required to institute a national HIE is the ability to ensure

that all electronic health systems exchange data is in a universal language. The National Health

Information Network’s President’s Council of Advisors on Science and Technology, (2010)

recommends this universal language be able to accommodate current hospital EMRs as well as

new recordkeeping systems and formats. The only requirement would be the ability to send and

receive data in a language structured as individual data elements (i.e. a mammography result)

together with metadata that delivers an annotation for each piece of data.

An example of this would be a 74-year-old female who has lived in several different

states in the past 30 years. She has had mammograms performed at various hospitals and clinics.

TOO MUCH DATA, NOT ENOUGH DATA 25

Her provider needs to retrieve numerous images of her breast tissue to determine whether a

current lump is a new finding. If a universal data exchange language was available, the data

elements the provider could retrieve would include all of her previous mammograms regardless

of the state in which they were performed. He would be able to review these images in a similar

manner as someone doing a Google search.

A national infrastructure for finding and controlling access to health data requires a

foundation titled “data-element access services” or DEAS. Of course it makes sense the fewer

DEAS the more feasible the project. The National Health Information Network, President’s

Council of Advisors on Science and Technology, (2010) recommend DEAS be interoperable

with the ability to communicate in accordance to a single Federal standard. In response to the

HITECH directive, the ONC is currently attempting to identify features on how to best to

operationalize these services.

The CCC has developed a DEA service amongst its members to transfer health care data.

One of its missions is to demonstrate the ease with which this can be exchanged. How should

states or healthcare organizations establish and operate DEAS? This is one of many questions

which must be answered as soon as possible if HIE is to become feasible.

Ethical and Legal Considerations of Data Selection

Is it a legal liability when providers access other providers’ data and then don’t review it?

There is little information on this topic. As case law offers little guidance on the liability of a

provider acting on clinical information made available but not requested, this topic needs

additional study (Burton et al., 2004).

The legal position on clinical decision support systems is also unclear, but reviews

suggests that” parsimonious or tailored warnings” do not raise the liability risk of system

TOO MUCH DATA, NOT ENOUGH DATA 26

manufacturers and providers as long as systems are designed well and providers continue to use

their best medical judgment (Kesselheim, Creswell, Phansalka, Bates, & Sheikh, 2011, p. 2312).

Can this recommendation be inferred when sharing either limited or extreme amounts of data in

HIE?

Data exchange also presents numerous ethical challenges. Little is known about patients’

attitudes toward sharing their clinical data with different providers. Privacy issues and

unwillingness to share certain data such as a history of mental illness or sexually transmitted

diseases may prevent any patient information from being exchanged (Burton et al., 2004).

Francis (2010) suggests that the information the provider, “Records about the patient –

and what the provider recommends with respect to the patient – will no longer reside in the

microcosm of a single office” (p.40). Both the patient’s confidences and the provider’s actions

will be theoretically accessible to others. An additional risk which cannot be ignored is that

providers might characterize or stigmatize patients as they react to data that previously might not

have been available to them.

Another ethical issue potentially surfaces when patients refuse to share all of their patient

information. A HIE can be undermined by the liberty of a patient. Does a patient have a

fundamental right to remain ambiguous? A HIE can only send and receive the data the patient is

willing to share. A patient may exert the right of owning their data. The difficulty comes when

the receiver obtains only parts of the data, which may be potentially dangerous for the patient.

How can providers emancipate patients to share all of their data? They should educate them on

reasons why it is important to be transparent about their data.

Shared Accountability in Data Exchange

TOO MUCH DATA, NOT ENOUGH DATA 27

Current Meaningful Use requirements advocate for patients to access their own medical

data. Presently many health systems share their data with patients and the number will grow.

Intermountain Healthcare has a portal called “My Health” where patients can view lab and

procedure results. No provider information (hospital discharge summaries, progress notes, etc.)

is disclosed at this time (personal communication, Select Health representative, July 20, 2012).

Several studies have documented the experience and benefits of incorporating patient-

generated data into their electronic medical records. An electronic blood pressure trial

demonstrated that home blood pressure monitoring with electronic communication through

secure email with a clinical pharmacist nearly doubled the number of people whose blood

pressure was controlled (Greene et al., 2008). A randomized trail done in 2009, showed

improved glycemic control in patients with Type 2 diabetes who had home glucose monitoring

with electronic communication with a care manager (Evert, Trence, Catton & Huynh, 2009).

Secure email access between provider and patient has also demonstrated improved care time and

time again.

Future MU requirements may require that patients will control what clinicians can access

from their data. This is called a shared data accountability model. This model prompts the

question that if patients are directing the contents of their medical records should they shoulder

more of the burden of responsibility. Currently if a patient omits sharing information, clinicians

are not responsible for the burden of that liability (personal conversation, Jutta Williams, July 17,

2012). There is also a case to be made that withholding information may increase health care

costs (unnecessarily repeating tests, etc.). How will privacy and secrecy, when secrecy may be

considered harmful, play out as patients manage their own health data? Will there truly be a

shared accountability? This is yet to be seen.

TOO MUCH DATA, NOT ENOUGH DATA 28

The Provider–Patient Relationship and Data Exchange

Provider patient trust is a two way relationship that has the real potential of being placed

in jeopardy because of information exchange. Balancing patients’ autonomy and best interests

may be difficult at times, but it should always be presumed that the provider’s role is patient-

focused. The provider must trust that patients have given them accurate information. This

relationship was previously encapsulated, but “with the advent of interoperable ERHs this

encapsulated trust relationship is exposed. It exposes both information about the patient and

what the provider did – or did not – do” (Francis, 2010, p. 37).

Public support of electronic health information exchange is crucial moving forward. In a

2012 Cornell University study by Ancker, Edwards, Miller, & Kaushal, 2012). New York

residents supported HIE among healthcare providers believing it would improve their medical

care. They also supported emergency data access without consent. Survey respondents

expressed some concerns about privacy and security but were supportive whether the

architecture involved a provider sending data to another provider, a provider sending data to a

patient who would then pass it on to another provider, or a provider accessing data from other

institutions.

In light of potential patient concerns about privacy, the National Committee on Vital

Health Statistics has argued that patients should be able to mask defined categories of sensitive

health information in interoperable EHRs. “If patients only have the option to opt into or out of

a system, they have no guarantee that information in their records is seen on a limited, need-to-

know basis” (Watson, 2006, p.2).

TOO MUCH DATA, NOT ENOUGH DATA 29

Patients are increasingly entering their own data into personal health records related to

their insurance companies or internet health website. In many institutions this information

becomes part of the system’s EHR (Francis, 2010).

There is no argument that interoperable electronic medical records create new, more

powerful, and more accurate means for oversight that allows trust to be validated by structures

external to the physician-patient relationship. But, they also “create new, more powerful means

for access to individual information that require in turn transparency, patient consent and

oversight of how the data are used if patient trust is to be maintained” (Francis, 2010, p. 46).

Trusting of Exchanged Data

A critically ill neonate is transferred to a tertiary NICU after delivery. The infant’s

mother had tested negative for Group B strep prior to delivery. This testing was done in another

health network. Can this data be trusted or should the test be repeated? Clinicians often trust the

familiar and can be very context dependent. A radiology reading performed by a colleague holds

much more credence than does a reading by a radiologist in another system. Electronic data

from another facility can certainly compound the uncertainty of an already precarious situation.

What makes exchanged data trustworthy? This has become a new question in medical

informatics and very little information has been published on this subject. Jay Jacobsen, a

medical ethicist and infectious disease physician at the University of Utah defines three issues

with trusting electronic data: 1) The originator of the data is unknown, 2) The effort of obtaining

the data is often based on convenience, and 3) Providers have been taught to be suspicious and to

think independently (personal communication, June 22, 2012).

Statistics show that for every five out of 100 electronic transfers of data there is difficulty

associating the correct data to the right patient. One figure implies that 20% of data exchanged

TOO MUCH DATA, NOT ENOUGH DATA 30

nationally may be wrong. Placed in a community data exchange perspective, only six out of 10

data elements may be accurate (Rand Survey, 2011). Obviously this rate is unacceptable, but

until precision is improved, do clinicians turn their backs on the data elements that are accurate?

Many clinicians would admit their clinical practice is driven by worries of risk. There

must be a change in this mentality. There is risk in everything and this is a world of uncertainty.

Clinicians need to be reminded the likelihood of all things being equal, the patient will be better

served if data is shared. The term “status quo ante data” needs to be changed to “status quo data

exchange”. Trust must begin to be used in appropriate ways. Only then will health care

providers begin to improve efficiency and health care costs.

Patients as the Gatekeepers – Personally Controlled Health Records

In the not too distant past, a patient’s access to his medical records was extremely

limited. The wrath of the nurse could be fierce when a patient or family member was caught

reading the nurses notes which were hung temptingly at the end of the bed. Gradually the

pendulum has swung the other way. A new paradigm of information sharing and patient access

has recently been seen in Brazil. The provider writes the order for the lab tests, but the patient is

responsible for selecting the lab. The results are returned only to the patient and he must notify

the provider of the results.

In the United States, Weitzman, Helemen, Kaci, & Mandl, (2009) recently promoted an

innovative approach for bringing improved data into the clinical arena called the personally

controlled health record (PCHR). “This is an individually controlled Web-based platform that

integrates personally reported, as well as clinically and administratively sourced data over sites

of care and time” (p.2). Currently PCHRs are not standards driven and few provide simple

methods for transporting records among different EHR products. Recently Microsoft and

TOO MUCH DATA, NOT ENOUGH DATA 31

Google, who both maintain web-based PCHRs, agreed to allow exchange of information

between their respective PCHR systems without charge to the patient (National Health

Information Network, 2010).

A survey of over 500 patients, obtained by Friction and Davies (2009) showed extreme

interest by both clinicians and patients in using a similar concept, called a Personal Health

Record (PHR). This would routinely be used for accessing and exchanging health information,

including medication reconciliation, patient history and education. A positive element of

personal health records is that they are “owned” by the patient which imparts the beginnings of

interoperability.

One of the tenets of future Meaningful Use requirements has patients controlling their

own health information. Eva Powell (2012), the director of health information technology

programs at the National Partnership for Women and Families believes this is an extremely

positive thing for patients. It allows them to make choices down to a single data item level.

They may choose whether to share the fact that they are on a psychotropic medication, have been

recently treated for a sexually transmitted infection (STI) or had an abortion at age sixteen.

Powell believes it is a concept whose time has come. Or has it?

From an informatics standpoint, it could be very difficult to offer the required MU

technology to allow these choices to be easily selected and blocked. These are high expectations

which may require extensive time and expensive development platforms. From a provider

standpoint, are we allowing the patient to practice medicine? If the patient will soon be

delivering a baby, is the psychotropic medication they are taking, the abortion at age sixteen and

previous STI important information that could impact the delivery and health of the mother and

newborn. This goes back to the concepts of relevance and pertinence. Regardless of the

TOO MUCH DATA, NOT ENOUGH DATA 32

concerns, a new nursing role will emerge that will require education to help patients understand

the implications about the data they choose not to disclose.

A study reviewing attitudes of pediatric patients and their families regarding data sharing,

reported that a majority of patients/families were willing to share personal health information

with other providers to support patient care or public health reviews which support health

supervision and research. They expressed concern about sharing family income and

transmittable disease information. Willingness to share health data by category was not

associated with patient’s age, race or health status, the number of children in the household or

income. “There was no association between reported income level and willingness to share with

public health in any information category” (Weitzman et al., 2009, p. 2).

The National Health Information Exchange (2012) believes the participation of patients

in their own healthcare could substantially improve care, especially in the management and

treatment of chronic conditions such as diabetes and obesity. Access to electronic personal

health information and interfaces make it easy for the public and private healthcare organizations

to enable providers and patients to collaborate in informed decision making.

There may also be significant cultural barriers related to sharing of PHI. These barriers

appear important but at this point in time have not yet been well defined.

What Constitutes Appropriate Data?

Work done by Van Vleck et al. (2007) suggests the feasibility and benefits of an

automated patient summary sheet in the primary care arena to recap key pieces of information, is

essential. It should also be deemed essential to have a deeper understanding of what

information is of most importance to providers when reviewing a patient’s medical record. This

understanding has begun to be explored by this project.

TOO MUCH DATA, NOT ENOUGH DATA 33

In an exploratory study that looked at factors which motivated and affected HIE usage by

Vest, Jasperson, Hongwei, Gamm & Ohsfeldtl. (2011), several important inferences were made

related to relevant data requirements and patient type:

1. While HIE provides access to previously inaccessible externally generated

information, not every encounter requires that type of information such as patients with

select conditions or injuries.

2. The main advantage of HIE appears to be access to diagnostic tests, existing

treatments and previous diagnoses.

3. Patient complexity and usage are correlated.

4. Time constraints present a barrier to HIE usage indicating that information must be

valuable enough to motivate the provider to spend time accessing it.

5. Directly placing the information made available by HIE into the organization’s HIE

removes barriers to seeking information.

6. For complex patients, the minimum information provided by the HIE system is not

sufficient.

7. Usage was less likely in emergency departments for unfamiliar patients. Patient

familiarity was deemed undesirable because it is indicative of patients with inappropriate

sources of care.

8. There appears to be a relationship between facility repeat patients, and the association

between payer type and usage (p.147).

In a series of structured interviews with residents at New York-Presbyterian Hospital,

Van Vleck et al. (2007) attempted to identify phrases in the medical record that each physician

perceived to be relevant when describing a patient’s history. Primary data sources for provider

TOO MUCH DATA, NOT ENOUGH DATA 34

review included all types of clinical data. Discharge summaries were excluded to persuade

clinicians to use other primary data sources.

During the structured interview process, physicians underlined 824 phrases that they

considered relevant to explaining the patient’s history. Subjects developed a list of

categories considered relevant to the process of patient familiarization and applied one to

each phrase of interest. These categories included: labs and tests, problem and

treatment, history, findings, allergies, meds, plan and identifying information (p. 763).

In every instance, the resident first located the admission note and read it thoroughly.

They next skimmed through the progress notes until they reached the last one. All study

participants said they would have referenced the discharge summary if it had been made

available. Simple rules based on location in the medical record may provide a useful start to

identifying data to include in a summary. The resident reviews mimicked the traditional

categories of the medical record. “Providers follow this methodology and hence future work

summarizing patient history should consider this structure, as it is core to the clinical thought

process” (Van Vleck, et al., 2007, p. 763).

Van Vleck et al. (2007) also attempted to study how providers visualize lab result trends

by mirroring everyday working conditions. Study results identified the importance of logical

tools that could aid in the rapid understanding of large volumes of information (such as graphs or

charts representing laboratory data).

Are the data requirements different between specialties and the patients they care for?

Vest et al. (2011) asserts that previous research argues for the examination of information

technology in children separately from adult populations due to the particular vulnerabilities and

unique needs of this population. Zeng, Climino and Zou. (2002) maintain that dependent on the

TOO MUCH DATA, NOT ENOUGH DATA 35

clinical tasks at hand, only certain subsets of data (referred to as views) are of interest to

providers. Providing appropriate views may be one way to address the problem of too much

data.

As is evident, a methodology must be proposed prior to beginning to load “everything”

into a HIE. There must be innovative and critical thinking which recognizes the needs of

clinicians and their patients and signification discussion to understand: 1) what data is desired

and wanted, 2) what is designated as accurate and correct data, and 3) what analysis must take

place to determine the logic of this process. A framework must be identified and a set of rules

developed to know what constitutes “relevant” data. Figure 2 demonstrates the importance of

understanding out of all existing data, what is most relevant when providing clinical care.

Figure 2. Identification of Most Valuable Data

Developed by K Merkley, September, 2012

Moderators/Types of HIE Access

What affects providers’ usage of HIE? Rudin et al. (2011) studied reasons why providers

access electronic information. They found it most helpful when the available information would

TOO MUCH DATA, NOT ENOUGH DATA 36

save time, help them avoid phone calls, when patients had trouble communicating, and to review

a pattern of patient visits (numerous clinics), and if there was ease of data access. Providers did

not find information helpful when it had gaps or was difficult to access or when notes were

locked (Shapiro, Kannry, Kushniruk & Kuppman, 2007).

Particular medical specialties showed different patterns of utilization. There was

intensive use amongst hospitalists and specialty services when caring for inpatients. ED

practitioners and pediatricians were more likely to access information when the patient history

was incomplete. Pathologists did not find the electronic information helpful, and no primary

care providers were studied. Regardless, the study stressed that for adoption the system must be

easy to use, meaningful data must be passed and it must fit provider workflows. (Shapiro,

Kannry, Kushniruk & Kuppman, 2007)

Gadd, Ho, Cala, Blakemore, Chen & Frisse, (2010) from Vanderbilt University looked at

a combination of factors related to user views on HIE usability in a product they were

implementing. They used a “technology acceptance model” called TAM. They determined that

the perception of a HIE was positive and that the product needed to be easy to use and provide

applicable information.

Hayrinen, Saranto, & Nykanen, (2007) in a review of the research literature looked at 55

studies related to the exchange of health information in Finland. Positive factors for adoption

identified included the quality of the data and the criteria of completeness and accuracy.

Structured data entry seemed to include more detailed data.

O’Malley, Grossman, Cohen, Kemper, & Pham (2009) suggest that a patient summary

may be the most appropriate way to establish electronic health information interoperability. A

patient summary includes patient history, allergies, active problems, test/procedure results, and

TOO MUCH DATA, NOT ENOUGH DATA 37

medications. However, further information can be included, depending on the intended purpose

of the summary and anticipated context of use. At the present time the CCC is sharing only part

of the patient summary (allergies, active problems and medications).

Accessing Appropriate Data

An understanding of how providers access data has been described in a classic study by

Krikelas (1983). The study identifies two types of activities, the first identified as “information

gathering” such as journal reading to keep current and “information seeking” which is done to

meet a perceived need for additional information. Before beginning to seek information, the

provider must decide whether to pursue new information at all.

Gorman and Helfund (1995) disclose results of a study of office-based physicians which

revealed that although as many as two questions arose for every three patients they cared for,

only 30% of these questions were actively pursued. They studied factors which motivated

primary care providers to answer clinical questions. It was determined that these providers were

more likely to pursue an answer to a question if they knew a definitive answer existed or if they

concluded that patient’s condition was urgent.

They report, “primary care providers are concerned and curious, but busy and practical.

While caring for patients, they have many questions about optimal management, but they invest

their time and effort pursuing questions only when they expect a direct, immediate benefit,”

(Gorman & Helfund, 1995, p. 118). Implications for accessing data can be extrapolated from

this study in that the provider must be shown straightforward and immediate benefits to help

solve the problems of patient care. No amount or type of data given to a provider can improve

patient care unless it is accessed. A goal should be established which requires delivery of data

providing timely answers to all clinical questions.

TOO MUCH DATA, NOT ENOUGH DATA 38

Zeng, et al. (2002) reasons all types of medical data can be categorized into three groups

– source-oriented views (which organize data on the basis of where they were collected); time-

oriented views (which primarily use time to organize data); and concept-oriented views (which

center on clinical concepts, such as diseases or organ systems (p. 294). They hypothesize that

instead of presenting caregivers with patient data in chronological order, organized by the source

of the information (labs, radiology), a knowledge based system organized around clinical

concepts such as disease or organ systems is a better way to present information, improve

retrieval precision and reduce the information burden.

When using concept oriented views, a user would enter a clinical term and select from a

list of matching concepts. For example if a patient has congestive heart failure, after selecting

“Radiology Reports”, the system returns a list of radiology reports related to congestive heart

failure. This structure shows exciting potential for reducing information overload. “On average,

each concept-oriented view contained only a fraction of all information about patients,” making

the process more time efficient (Zeng et al., 2002, p 300).

An ethnographic qualitative study done by Unertl, Johnson, & Lorenzi (2012) studied six

Memphis emergency departments (EDs) and eight ambulatory clinics in an attempt to understand

the relationship between HIE and clinical workflows at multiple sites. Two key workflow

processes surfaced across these sites. Nurse access to the system was motivated to identify

recent hospital visits with a goal to retrieve specific information for licensed independent

practitioners (LIP) use. LIPs use of the HIE was more global. The authors believe their research

“addressed a significant gap in the knowledge about the front-line impact of HIE on patient care

delivery” (p. 400). They also believe that the impact of a HIE system on patient care rarely

delivers clear-cut financial benefits.

TOO MUCH DATA, NOT ENOUGH DATA 39

It is also important to consider the possibility of bringing forward historical compliance

information? Did the patient fill all his scripts, cancel an appointment, or go to an alternative

health care provider? How many times did the patient access data online, delete data or initiate

corrections. This information would provide important compliance details which could be

factored into the total equation.

Presentation of Received Data

Once data has been received from another healthcare organization it should be presented

to clinicians using data visualization tools that have meaning to clinicians. These visualization

tools must display connections between different data presentations. Should data received be

presented to clinicians in a single document form or incorporated into the patient’s current

electronic medical record? Little evidence is available this subject. Figure 3 demonstrates how

data could be received as separate electronic documents or as integrated data within the patient’s

current EMR. This study attempted to identify a preferred method for data presentation.

Figure 3. Presentation of Received Data

Developed by K Merkley, September, 2012

TOO MUCH DATA, NOT ENOUGH DATA 40

Reconciliation of Received and Existing Data

Every clinician understands the importance of reconciling medications for a patient

admitted to the hospital. This reconciliation helps avoid medication errors and inconsistencies as

that patient traverses the different transitions in care. If data is sent from another healthcare

organization and is integrated within the patient’s current EMR, there may be a need to reconcile

duplications, discrepancies and contradictions, not only for medications, but allergies, the past

medical history and current problems. It is hoped that much of this reconciliation will be

completed by the computer, but human interaction/decision making will likely be needed as well.

(Figure 4).

Figure 4. Reconciliation of Received and Existing Data

Developed by K Merkley, September, 2012

TOO MUCH DATA, NOT ENOUGH DATA 41

Questions related to reconciliation of data were also addressed in this scholarly project.

They included asking if a clinician should be required to reconcile data between two institutions,

and if reconciliation was mandated who should perform it (the LIP caring for the patient in the

receiving institution, the nurse caring for the patient in the receiving institution, the first licensed

care giver encountering the patient in the receiving institution or should reconciliation be

dependent on the type of data shared; such as a pharmacist reconciling medications, a physician

reconciling labs and a nurse reconciling patient goals)?

Are Data Requirements Different Between Caregivers and Patient Types?

It is important to remember the amount and quality of information available to health care

professionals in patient care has an impact both on the outcomes and continuity of that care,

Hayrinen et al. (2007). With that mandate in mind it must be understood that all providers

access data at various times and in various ways.

Shapiro, Kannry, Kushniruk, & Kuppman, et al. (2007) describe emergency departments

as information intensive environments – yet they function at a baseline information deficit. They

recommend that preliminary work needs to be done to determine the data needs of the providers

and the proper way in which to implement IHE systems to be well integrated into ED workflow.

Their study results, done to determine emergency department provider perspectives on data, is

seen in Tables 2 and 3.

Table 2. ED Physician Access of Data from a HIE

Data

Percentage

Electrocardiograms 80%

Discharge summaries 66%

Medication list 65%

Laboratory results 59%

Radiology reports 59%

Problem lists 59%

Provider information 44%

TOO MUCH DATA, NOT ENOUGH DATA 42

Cardiology reports 38%

Allergy information 21%

Endoscopy reports 3.5%

Patient demographics 2.4%

Shapiro, J.S., Kannry, J., Kushniruk, A.W. & Kuppman, G. (2007). Emergency physician’s

perceptions of health information exchange. J Am Med Inform Assoc, 14:700-705.

Table 3. ED Physician Preference of Data Display

Data Image or Written Report Percentage

Cardiac catheterization Written Report 100%

Echocardiograms Written Report 100%

Nuclear Medicine scans Written Report 98%

Ultrasound Written Report 98%

Endoscopy Written Report 100%

CT Written Report 63%

Plain radiographs Image 74%

Electrocardiograms Image 98%

Shapiro, J.S., Kannry, J., Kushniruk, A.W. & Kuppman, G. (2007). Emergency physician’s

perceptions of health information exchange. J Am Med Inform Assoc, 14: 700-705.

The study also reported that it is unlikely that radiologists for example, when given

access to an HIE network, would be satisfied with only a written report for advanced studies.

They will likely want to see the actual images to draw comparisons and formulate their

own interpretation. Likewise, cardiologists will probably want to see the actual video or

images from advanced cardiac studies, to draw conclusion and make decisions for their

patients (Shapiro et al., p.704).

Unertl et al. (2012) also studied emergency department providers qualitatively and noted

that an unanticipated but frequently encountered reason for accessing a HIE involved trust issues

with a patient. Providers routinely searched the exchange for chief complaints of back pain and

headache and “red flag” behaviors related to concerns about narcotic abuse. The same study

noted ED providers reviewed labs, radiology and procedure reports as significant but indicated

that discharge summaries were the most helpful type of data.

TOO MUCH DATA, NOT ENOUGH DATA 43

In a 2011 study of a regional HIE of emergency departments and clinics, Johnson et al.

(2011) acknowledged users accessed the HIE just under seven percent of the time for all

encounters, with “higher rates of access for repeat visits, for patients with co-morbidities, for

patients known to have data in the exchange and at sites providing HIE access to both nurses and

licensed independent practitioners” (p. 609). Discharge summaries and lab results were most

frequently retrieved. Providers discerned that by having additional information, repeat testing

was prevented and hospitalizations were avoided.

Very little information is available to determine what the clinical information needs are

required for other specialties at this time. Similar studies with new study populations will need

to be conducted to gain this information. This project addressed some of the gaps in this area. It

is important that the process of building trust by providing relevant data begin now.

Theoretical Framework

T.S. Eliot asks, “Where is the wisdom we have lost in knowledge? Where is the

knowledge we have lost in information” (1934). The philosophical foundations of the Data-

Information-Knowledge-Wisdom (DIKW) framework will be utilized to guide this project. The

Graves and Corcoran (1989) article, “The Study of Nursing Informatics” identifies data,

information and knowledge as foundational concepts for this area of nursing.

In 2008, the American Nurses Association revised the Scope and Standards for Nursing

Informatics to include the additional concept of wisdom. Although this framework is based on

nursing informatics theory it can be useful to the health care community in general (Matney,

Brewster, Sward, Cloyes, & Staggers, 2011) and specifically to this project because the data

elements identified will be used to gain information and knowledge.

TOO MUCH DATA, NOT ENOUGH DATA 44

The framework of this project allows the structuring and processing of particular clinical

information to arrive at clinical decisions that will support and improve patient care. Figure 5

below shows the current model and illustrates how the concepts build and overlap on each other.

Figure 5. DIKW Framework

Matney, S., Brewster, P.J., Sward, K.A., Cloyes, K.G. & Staggers, N. (2011). Philosophical

approaches to the nursing informatics data-information-knowledge-wisdom framework.

Advances in Nursing Science, 16:53, 1-13.

Data are the smallest unit of the DIKW framework. In this case they represent elements

such as allergies, laboratory values, discharge summaries, behavioral health histories. A single

piece of data (datum) has only isolated meaning when seen in relation to patient information as a

whole.

Information in this framework may be considered as data associated with meaning and a

specific context. The substance of this project lies within the type of information that will be

provided to specific clinical roles.

A subset of relevant information can next be transformed into knowledge. Matney et al.

(2012) describes two types of knowledge. The first is tactic knowledge. It is personal and

TOO MUCH DATA, NOT ENOUGH DATA 45

context specific and has been described as background knowledge. The second type of

knowledge is explicit knowledge. It can be described as knowledge that can be captured, stored

and shared. This is the knowledge which will be relayed through specific data elements. It will

be shared with consortium members to aid them in decision making processes.

Wisdom is the appropriate use of knowledge to manage and solve human problems as

defined by the American Nursing Association (2008). Wisdom involves recognizing what is

most important by making distinctions among alternatives (Matney et al., 2011). These

distinctions can be measured by looking at choices made for data retrieval correlated with

specialty knowledge.

Data, information, knowledge and wisdom are crucial concepts in nursing. The

intertwining of these concepts provides a robust structure for this project in addition to allowing

theory and practice to merge in a very meaningful way.

Implementation

The factors associated with gathering clinician input related to the relevance of certain

data elements were modeled in part using a modified Delphi technique. Centered in the rational

“two heads are better than one,” this framework was designed to achieve a merging of opinion

regarding real-world knowledge petitioned from topic experts (Hsu & Sandford, 2007). Using

this research methodology, an initial survey, tailored to specific clinician workflows (emergency,

adult inpatient, pediatric inpatient, primary care, nursing inpatient and nursing primary care) was

designed and sent to experts. Additional surveys to other subsets of providers are beyond the

scope of this project, but the author will continue to be involved in further iterations to determine

the most relevant data sets to be exchanged.

TOO MUCH DATA, NOT ENOUGH DATA 46

Objective #1 - Identify data elements and related patient populations as well as provider

roles and specialties to be surveyed. The following data elements, related patient populations,

provider roles and specialties were identified for the survey focus of this project (Table 4). A

specific focus related to data elements was based on the S&I Framework (Health IT Standards

Committee, 2012) and the Integrating the Healthcare Enterprise (IHE) E-nursing summary, (IHE,

2012) recommendations.

Table 4. Relevant Data Points to be Surveyed

Data Elements Advanced directives

Allergy and intolerance information

Behavioral health history

Cardiology reports (written reports or images)

Cardiac catheterization

Echocardiograms

Nuclear Medicine scans

Cognitive abilities

Diet history

Discharge summary

Electrocardiograms (written reports or images)

Family history

Genome information

Goals

Health maintenance

Health insurance

Immunization history

Laboratory results

Language

Medical devices

Medication list

Mobility/falls risk

Operative summaries

Patient demographics

Patient instructions

Pending tests and procedures

Physical exams

Problem list

Procedures – invasive/noninvasive

Provider information (primary and designated)

Provider address and telephone number

Physical activity

TOO MUCH DATA, NOT ENOUGH DATA 47

Race

Radiology reports (written reports or images) (what period)

Ultrasound

CT

Plain radiographs

Endoscopy

Nuclear medicine scans

MRI

Review of systems

Special needs

Social history

Support contacts

Vital signs (what period)

Zip code

Patient Population Adult (age 18 and above)

Pediatric (age 0 to 18)

Provider Roles RN, NP, PA, MD, DO

Specialties Emergency Medicine LIPs

Inpatient LIPs

Pediatric LIPs

Primary Care LIPs

Inpatient Nursing

Primary Care Nursing

Objective #2 - Develop a survey of potential data elements to be exchanged. The survey was

divided into six different specialty areas (emergency, inpatient, pediatrics, primary care, inpatient

nursing, primary care nursing). Six different paper surveys were administered to Intermountain

clinicians and two electronic survey links (one for LIPs and one for nurses) were provided to

CCC members. The electronic surveys had embedded branching logic which led clinicians to

the appropriate questions related to their specialty area. All surveys were developed with a

descriptive cross-sectional design and were administered using convenience samples. Selection

bias was observed, but was not believed to be detrimental to the results.

Reliability of the survey was tested on a sample of seven physicians who participate on

the Intermountain Enterprise Clinical Information System (ECIS) Physician Advisory Board,

two nurse practitioners who work on the Intermountain ECIS Implementation Team and three

TOO MUCH DATA, NOT ENOUGH DATA 48

nurses who attend the Intermountain System Informatics Nursing Council. An attempt was made

to find two clinicians from each specialty area and this was accomplished with the exception of

having only one primary care nursing representative test the survey. Variations in the inpatient

and primary care nursing surveys were minimal so reliability was considered adequate as results

correlated with the inpatient nursing and LIP surveys.

The survey took less than ten minutes to complete. The ten minute timeframe was

determined by observing the ECIS Physician Advisory Board and recording how long it took

them to complete the information. Institutional Review Board (IRB) approval was obtained from

the University of Utah and Intermountain Healthcare before data collection was begun

(Appendix A). Privacy of respondents was strictly protected. No participant information was

collected. Study materials were electronically stored in a secure file within the Intermountain

Study Quizmo tool. Only study personnel had access to the data. When it became apparent that

CCC responses would be limited, the author asked to include University of Utah physicians at

Primary Children’s Medical Center and the 2012 DNP cohort. An amendment application was

made to the IRB and approved (Appendix A).

Objective #3- Distribute survey to consortium members. The surveys were taken to the

CCC Network IT Board where it was presented for approval on October 22, 2012 (Appendix B).

Each consortium member was given the option to participate. Those members who agreed to

participate were given the electronic survey links and were responsible to disseminate the

information to each organization’s team leaders. These organization team leaders distributed the

survey link to clinical operation leaders (ex: a cardiology department chair at Mayo Rochester,

ED department chair at Kaiser-Permanente) who distributed them to appropriate clinicians in

each facility. The actual method of recruitment was defined by these clinical operation leaders.

TOO MUCH DATA, NOT ENOUGH DATA 49

The University of Utah 2012 DNP Cohort was also sent an electronic survey link and asked to

respond. CCC members and the University of Utah 2012 DNP Cohort answered 65 electronic

surveys.

Paper surveys were administered to Intermountain Healthcare clinicians from August 22

to November 9, 2012. LIP groups surveyed included the corporate Emergency Department

Development Team, hospitalist teams at Intermountain Medical Center and Primary Children’s

Medical Center, the Medical Informatics Primary Care User Group, and an ED nurse practitioner

group at PCMC. Two Intermountain primary care groups were also surveyed. Nursing groups

surveyed included the corporate Emergency Department Development Team, the Nursing

Informatics User Group (NUG) and the Integrated Care Management team.

Inclusion criteria for provider roles consisted of any registered nurse or licensed

independent practitioner (defined as MD, DO, NP, or PA). Response rates were expected to be

at least 25-30 per consortium member, but turned out to be much less. Intermountain

respondents answered 95 surveys. A pre-survey letter from Intermountain Healthcare’s CMO

was sent out prior to the survey link (Appendix C). An initial letter of invitation, which

contained the survey link for consortium members, was sent out two days later (Appendices D).

The pre-survey and invitation letters were attached to the paper survey for all Intermountain

clinicians (Appendices C & E). A thank-you email was sent to each person who responded

electronically (Appendix F). Verbal thanks were given to Intermountain paper survey

participants along with cookies/brownies.

Objective #4 – Analyze survey results and make recommendations regarding what

additional data elements should be included in CCC electronic data sharing. Data results were

collected by the researcher and analyzed by a statistician hired to review the results. Results

TOO MUCH DATA, NOT ENOUGH DATA 50

were calculated using Microsoft Excel. A report was compiled and shared with the CCC

Network IT board (Appendix G).

Evaluation Plan

Objective #1 - Identify data elements and related patient populations as well as provider

roles and specialties to be surveyed. The data elements used in this cross-sectional self-

administered survey were obtained using results from an extensive literature review. Inclusion

of patient populations, clinician roles and specialty recommendations came from CCC experts.

Validity of selected data elements were surmised because of the broad national and international

oversight provided by S&I Framework and the IHE E-nursing Summary.

Objective #2 - Develop a survey of potential data elements to be exchanged. Survey

reliability was verified using a small group of local Intermountain clinicians. Internal

consistency reliability was measured using Cronbach’s alpha, with a result of 0.81. Construct

validity was also measured using expert clinician assessments of the thoroughness of data

element options. This was achieved through the project’s content expert’s review. Internal

consistency was further measured by evaluating responses separated by consortium members,

after the results had been returned.

Objective #3- Distribute survey to consortium members. Success of this objective was

related to the number of responses received. No previous surveys had been administered to the

CCC. It was unclear what the response would be. The author was hopeful that between 125 and

150 surveys would be returned based on the large number of clinicians employed by these

healthcare corporations. Actual survey responses were approximately 159. No power was

assigned to the study because of its pilot nature.

The survey time line is outlined in Table 5 below:

TOO MUCH DATA, NOT ENOUGH DATA 51

Table 5. Time Line of Survey Completion

September 24 Pre-survey letters signed by Intermountain Healthcare CMO/CNO

(Appendix C)

September 25 Survey administered Intermountain ED Development Team physicians

and nurse leads. (Appendices C, E and H)

September 27

Survey administered System Informatics Nursing Counsel -in patient

nursing representatives throughout the system. (Appendices E and I)

October 2 Survey administered Intermountain Urban Central Region hospitalists.

(Appendices E and J)

October 8 Survey administered Medical Informatics Group –Intermountain

primary care physicians (Appendices E and K)

October 12 Complete IRB addendum (Appendix A)

October 15 Surveys ready in Survey Gizmo tool (Appendix L)

October 20 Pre-survey letters sent to CCC leadership (Appendix C)

October 22 Invitation (Appendices D and L) and survey sent to CCC leadership

(electronic links)

October 29 Begin analysis of data

October 31 Survey Intermountain Nursing User Group (Appendices E and I).

November 1 Survey pediatric hospitalist group at PCMC. (Appendices E and M)

November 1 Survey Intermountain Integrated Care Management Team- Primary

Care nursing (Appendices E and N).

November 1 Survey DNP cohort via electronic link (Appendices D and L).

November 9 Survey McKay Dee Family Practice Residency Group (Appendices E

and K).

November 10 Send thank you letter to CCC participants. (Appendix F).

November 14

Analysis complete and recommendations compiled

(Appendix G.)

TOO MUCH DATA, NOT ENOUGH DATA 52

Objective #4 – Analyze survey results and make recommendations regarding what

additional data elements should be included in CCC electronic data sharing. Data analytics

included evaluation of relevant data by specialty, understanding of what constitutes trustable

data, emergent data that should be exchanged in the event of a life-threatening situation, time

limits of different types of data, presentation of received data and reconciliation of received and

existing data. This field assessment analysis was considered an exploratory investigation and in

many ways a cultural readiness assessment. An attempt was made to understand the actual needs

for data exchange. Programming requirements were in essences gathered and then refined using

descriptive statistical analysis.

Data relevance study findings will be submitted to the Care Connectivity Consortium

(CCC) Governing Board, IHE and HL7 (US standard governance boards). The final evaluation

relating to the success of this objective will be known if any or all of these recommendations are

incorporated into the actual CCC data exchange structure. The incorporation of the

recommendations is outside the scope of the DNP scholarly project. In addition, the author will

submit an article for publication to The Journal of the American Medical Informatics Association

and will apply to present at the American Medical Informatics Association (AMIA) and the

Health Information Management Systems Society (HIMSS) national conferences in 2013.

Project Results

Survey respondents equaled 159 (Table 6 for a breakdown of participants). Key

respondents for this project included Intermountain clinicians who willingly completed the data

relevance paper survey, the CCC clinicians and the 2012 University of Utah DNP cohort who

completed the electronic survey. A key barrier encountered was the length of time required for

IRB approval and the input of the surveys into the Intermountain survey tool. This led to a

TOO MUCH DATA, NOT ENOUGH DATA 53

diminished period of time for electronic responses. No unintended consequences were identified

from these delays. Project limitations were the timeframe for data collection as well as the

number of clinicians who did not respond.

Table 6 – Survey Participant Breakdown

Providers Electronic Survey Paper Survey

Adult Inpatient LIP 5 28

Emergency Department LIP 14 13

Pediatric Inpatient LIP 3 10

Outpatient LIP 15 23

Inpatient Nursing 20 21

Outpatient Nursing 7 0

Respondents/Category 64 95

Total Respondents 159

Relevant Data by Specialty

Binomial data (single data elements) was tallied against the number of respondents in

each category to determine an agreement or disagreement for each specific item through the use

of means, modes and slope. Means provided information about actual interest in data, mode

measurements defined what data sets were agreed upon as important by clinicians and slope data

specified what significance clinicians placed on certain data categories. Minimum and

maximum data ranges showed the importance of specific data elements for each specialty. The

main goal of this portion of the data analysis was to understand per cohort which data and when

data elements were valued. Individual data elements were not compared against the same data

elements for all specialties in this analysis (i.e. comparing the number of clinicians who wanted

to see advanced directives for each study group).

Relevant data per specialty was ascertained by asking clinicians to identify data they

would require to care for critical patients versus all patients. Results are listed in Table 7 below.

TOO MUCH DATA, NOT ENOUGH DATA 54

Table 7 – Relevant Data by Specialty

n Data

Elements

(total of 30)

Min/Max Average Mode Slope

Inpatient LIP 33 30 12 – 88% 49% 45% -2.74

Pediatrics LIP 13 30 15 – 92% 42% 42% -2.70

ED LIP 27 30 5 – 96% 60% 91% -3.10

Outpatient LIP 38 30 23 - 95% 70% 75% -2.20

Inpatient Nrsg. 41 30 22 – 92% 66% 78% -1.90

Outpatient Nrsg. 7 29 0 – 100% 63% 50% -2.82

Clinicians in five of the specialty groups wanted access to all 30 data elements.

Outpatient nursing requested 29 out of the 30 elements presented. Only four additional requests

were made in the “other” category for history and physicals (H &Ps). This indicates that the data

elements under consideration are appropriate. All data elements were given equal ranking.

Minimum and maximum data ranges showed that for example in the LIP inpatient realm,

88% of the sample viewed laboratory results or radiology test reports as important. Inversely

only 12% of the sample felt the ability to view patient instructions was valuable (Figure 6).

Mode measurements showed the value of data elements surveyed. Again using the LIP inpatient

figures as an example, 45% of the data elements were viewed as important by this cohort versus

outpatient LIPs who agreed 75% of the data elements were pertinent.

When looking at the slopes calculated between different groups of clinicians from a gross

perspective the overall importance and interest in data is definitely distinct. For shallow slopes

there is agreement that most data elements are pertinent whereas a steep slope shows that only a

small amount of data elements is helpful. LIPs caring for adult (slope -2.74) and pediatric

TOO MUCH DATA, NOT ENOUGH DATA 55

inpatient (slope -2.70) populations and nurses caring for outpatients (slope -2.82) were noted to

use a definite subset of information (slopes ranging from -2.70 to -2.82). LIPs caring for patients

in an emergency department setting required an even more limited set of data (with a steep slope

of -3.10). Outpatient LIPs (slope -2.20) and inpatient nurses (slope -1.90) requested a much

more comprehensive set of data when caring for their patients.

Pediatric LIP responders identified two unique data types versus other specialties by

requesting immunization history (90%) and growth records (85%) of the time (Appendix O,

Relevant Data per Specialty - Pediatrics). Provider information and telephone number was also

selected as highly relevant by inpatient adult and pediatric LIPs (Appendix O, Relevant Data Per

Specialty – Adult LIP).

What Constitutes Relevant Data for Critical Patients

Again data needs were different related to care of the critical patient and specialty. Table

8 reflects this data.

Table 8 – Relevant Data for Critical Patients

Data elements desired by ED clinicians were much more limited at 19, pediatric

clinicians at 24 items, inpatient clinicians at 26 and inpatient nursing at 30. Minimum and

maximum ranges showed much less need to access specific types of data for all patients. Means

and modes were also lower indicating less interest in a broad group of data. Only a very few

data elements were viewed as critical. As would be expected, it is apparent that clinicians are

n Data

Elements

Min/Max Mean Mode Slope

Inpatient LIP 33 26 0-27% 8% 22% -0.70

Pediatrics LIP 13 24 0-92% 34% 42% -3.17

ED LIP 27 19 0-42% 8% 26% -0.90

Inpatient Nrsg. 41 30 5-32% 17% 34% -0.75

TOO MUCH DATA, NOT ENOUGH DATA 56

only interested in data that is helpful in resuscitating the patient and are not concerned with other

data types.

Adult inpatient LIPs were interested in receiving information related to advanced

directives, ECG images, operative summaries, progress notes and radiology test reports and

images. Pediatric inpatient LIPs requested more data elements than other providers, and were

the only group to regard immunization history as an emergent element to be reviewed. ED

clinicians wanted confirmation of advanced directives, discharge summaries, and radiology tests

as well as information about a patient’s behavioral health history and cognitive abilities. See

Appendix P for a breakdown of emergent data by specialty.

Top Five Types of Priority Data Related to Specialty

Each group was asked to identify what five types of data would be most relevant when

caring for patients in their specialty. Discharge summaries were identified as one of the top

priorities in all six groups surveyed. Advanced directives status was identified as imperative for

all four inpatient clinician groups. Laboratory results were also flagged as essential in all groups.

See Figure 6 below for the top five types of data noted as necessary by each group.

Presentation of Exchanged Data – Separate or Integrated

A question was included to help identify how data visualization tools could provide

connections between different healthcare systems. There was an absence of conclusive

information obtained from this query (Appendix Q) as no one way of displaying data was

identified as superior. Respondents were given the option to receive data as a separate electronic

document, receive as integrated data /information within the patient’s current EMR or both.

Several questions arose as different data elements (i.e. labs vs. discharge summaries)

were contemplated. It became apparent that future surveys should review the use of multiple

TOO MUCH DATA, NOT ENOUGH DATA 57

formats. Labs results from different systems could clearly be combined, yet how should various

discharge summaries be incorporated? This discussion will require additional study.

Figure 6 – Top Five Data Types by Specialty

Top Five Data Types by Specialty

Reconciliation of All Data

The questions regarding reconciliation of all data addressed the hypothetical condition of

data sent from another healthcare system being integrated within the patient’s current EMR. It

was suggested there may be a need to reconcile duplications, inconsistencies and contradictions.

Part of this reconciliation would be completed by the computer but human interaction/decision

making would likely be needed as well. Inquiries about the necessity of data reconciliation and

who should do it again returned no conclusive information (Appendix R). Further investigation

is warranted and it may be important to consider that we may not be culturally ready to

understand data reconciliation from multiple sources.

Another question was asked about who should reconcile received and existing data.

Clinicians indicated it should be done, “but not by me” (Appendix R). Again an argument could

TOO MUCH DATA, NOT ENOUGH DATA 58

be made that clinicians don’t have enough intellectual experience around this concept. This

question will also require additional study.

What Makes Data Trustworthy

Clinicians indicated overwhelming positive responses (very important or important) to

six of eight choices. The exceptions were questions related to provider type and reimbursement,

which are mixed. The take away message from this section indicates clinicians understand the

basic issues of data provenance and integrity and would use integrated data if it were cleansed

and readily available (Table 9).

Table 9 – What Makes Data Trustworthy?

Question Scale (1-5)

1. Strong working relationship with out of network provider 3.1

2. The reputation of the organization sending the data 3.6

3. Integrity of the health data exchange 4.1*

4. Knowing the type of provider who ordered the procedure 2.3

5. The complexity of the patient 3.3

6. Tests can be repeated and reimbursed 3.5

7. Data organized in a reliable, intuitive format 3.9*

8. Data integrity 4.2*

Time Limits of Different Types of Exchanged Data

Clinicians responded to eight questions asking about useful data ranges (from one 1 week

to five years) for vital signs, laboratory results, procedures, previously administered medications,

TOO MUCH DATA, NOT ENOUGH DATA 59

radiographs, electrocardiograms, problem lists and discharge summaries. Results showed bi-

modal distributions requesting data either immediately or long-term for labs, procedures,

radiographs, electrocardiograms, problem lists and discharge summaries Time ranges for

medications showed a varied distribution indicating clinicians wished to review a medication

history throughout the continuum of care. Vital signs data was identified by clinicians as to be

helpful only in the recent past (Figure 7).

DNP Essentials

The following Essentials of Doctoral Education for Advanced Nursing Practice criteria

from the American Association of Colleges of Nursing (2006) were addressed in this project:

Essential I: Scientific Underpinnings for Practice. An extensive literature review was

completed on this topic. A survey was then designed and the data collected was

statistically analyzed.

Essential II: Organizational and Systems Leadership for Quality Improvement and

Systems Thinking. Working with Intermountain and other CCC members, a survey was

designed representing all healthcare systems. It is thought that results will improve the

quality of patient care with better access to pertinent data.

Essential III: Clinical Scholarship and Analytical Methods for Evidence-Based Practice.

Use of evidence based practice guidelines helped form the basis of survey development.

Essential IV: Utilizing Information Systems/Technology for the Improvement and

Transformation of Health Care. Through the use of HIT and interoperability, content

obtained from this project and instituted through the CCC begins the process of data

transfer and subsequent healthcare transformation.

TOO MUCH DATA, NOT ENOUGH DATA 60

Essential VI: Interprofessional Collaboration for Improving Patient and Population

Health Outcomes. This project enables collaboration across numerous healthcare

organizations and specialties.

Essential VII – Clinical Prevention and Population Health for Improving the Nation’s

Health: Recommendations to the CCC and national standard governance bodies will

allow for pertinent transfer of data throughout the country promoting preventative and

improved health for those patients who participate in electronic health information

exchanges.

Essential VIII– Advanced Nursing Practice. Survey creation, administration and

subsequent recommendations will assist in improving the care of patients as they are

cared for in various healthcare organizations.

The stated criteria helped guide this project in an attempt to provide a deeper

understanding of the roles of organizational and system leadership as well as playing an

increased part in health care policy.

Discussion

Discussion of Main Findings

The data set gathered and surveyed appears to be sufficient for recommendations made to

the Care Connectivity Consortium and the standard organizations, IHE and H7. The survey

indicates that the more specific the clinician realm of care, the less data needed in caring for

patients. ED LIPs require a relatively limited subset of information, inpatient LIPs both in the

adult and pediatric realm require slightly more but primary care providers’ and inpatient nurses

TOO MUCH DATA, NOT ENOUGH DATA 61

Figure 7 – Useful Lifecycle of Data Types

Labs, procedures, radiographs, and electrocardiograms all show a bimodal distribution indicating

that clinicians want immediate or more long-term results (i.e. lab results were designated as

important within one week at 23% and again at one (26%) and five (22%) years respectively).

They have relatively little interest in these four data categories between one month and one year.

TOO MUCH DATA, NOT ENOUGH DATA 62

Clinicians indicated the need to see medications throughout the lifecycle, indicating they were

interested in seeing all medications a patient had been taking within the last five years. Vital

signs were shown to be most helpful within the one week.

desire access to a larger percentage of data types. This makes sense as they care for the patient

as a whole versus a specialist who is focused on one specific aspect of the patient. Outpatient

nurses did not demonstrate a similar trend. They were more interested in a specific data set,

similar to inpatient and pediatric LIPs.

Relevant data for outpatient nursing was inconclusive and could be directly related to the

small sample size. This sample identified family history as the second most important data

element needed when caring for a patient, which doesn’t appear to make logical sense. There is

a question around the nursing definition of family history and why it would be identified as so

important. It is unclear why inpatient nurses identified health insurance as a high priority item

for review. It could be surmised that they might be focused on health care costs and use less

supplies if the patient is not insured. There also is a need to target outpatient nurses to

understand what they do with data and how they apply it to their nursing practice.

TOO MUCH DATA, NOT ENOUGH DATA 63

Advanced directives emerged as a relevant request for all survey groups. Provider

information and telephone number were also selected as highly relevant by patient adult and

pediatric LIPs. Zeng et al. (2003) studied a small group of clinicians in 2003 that ranked the

importance of data listed below in Table 10. Note that advanced directives is not listed and

provider information is ranked only at 44%. Increased significance of these data types would

indicate the newly emphasized regulatory importance of patient rights and safe handoff between

care givers.

Table 10 – Relevant Data for LIPs in 2003

Data

Percentage

Electrocardiograms 80%

Discharge summaries 66%

Medication List 65%

Laboratory results 59%

Radiology reports 59%

Problem lists 59%

Provider information 44%

Cardiology reports 38%

Allergy information 21%

Endoscopy reports 3.5%

Patient demographics 2.4%

Zeng, Q., Cimino, J.J. & Zou, K.H. (2002). Providing concept-oriented views for clinical data

using a knowledge-based system. J Am Med Inform Assoc, 9, 294-305.

A surprising finding showed emergency department LIPs requesting the ability to view a

patient’s behavioral health and cognitive abilities as pertinent when caring for a critically ill

patient. It is supposed that these providers would want to understand a patient’s baseline mental

status when treating a CVA, decreased level of consciousness, substance abuse or cerebral

hemorrhage. Predictably all specialty requests for advanced directives, laboratory results,

discharge summaries, etc. were expected.

One survey participant commented that integration of data was a new and interesting

concept, one that he had never considered before. Yet the question asked in the survey was not

TOO MUCH DATA, NOT ENOUGH DATA 64

as specific as it should have been. As stated above, integration of numeric data such as labs and

vital signs makes a great deal of sense. Not as clear is the process to integrate text documents

such as discharge summaries and progress notes. Information gleaned from this question

illustrates this is not an all or none proposal. Each piece of data should be evaluated individually

to determine the value of integration with other like data or to keep the information independent.

These survey questions did not take into account that different data types might require

different manners of reconciliation. Text documents such as discharge summaries would not

require reconciliation whereas lab results and medication lists would. As with the question of

exchanged data, each piece of data should be separately assessed.

Findings about data trustworthiness revealed varied results on two questions related to

provider and reimbursement. It could be argued these questions were used as the reverse balance

questions asked in the survey. For survey questions developed in a Likert scales format it is

recommended that for balance, both positive and negative questions should be asked. Future

work should include re-evaluation of these two topics. Overall there was an indication that each

of the clinical specialties appears to be ready to exchange information between healthcare

systems.

A better understanding of the useful lifecycle of data types denoted the importance of

providing current (one week to three months) and long-term (five years to a complete history)

access to different types of data. A full medication history was regarded as necessary and vital

signs information was viewed as valuable only from one week to three months.

Policy and Practice Implications

This study may be considered exploratory research and there is much work yet to be

accomplished. There will be numerous iterations before concrete recommendations can be

TOO MUCH DATA, NOT ENOUGH DATA 65

made. However this is the beginning of HIT advancements that will affect interoperability. This

research will not have an immediate effect on policy and practice, but it is informing and shaping

the quality of health information exchange based on the correct data to be exchanged, the

presentation, reconciliation, trustworthiness and the useful lifecycle.

Strengths and Limitations of the Study

The strengths of the study include the following: 1) The study answered real questions

relating to standards development today based on the work done by HL7, S&I Framework and

the CCC; 2) The study is grounded in reality, data was an actual working set of information that

clinicians use; 3) Ground breaking research questions covered topics not currently well

researched; 4) A broad group of clinicians was surveyed from across the country; and 5) It is

important that these questions have been answered.

Study limitations included: 1) Ambiguity in some of the questions. An example being

what is the definition of family history as indicated as important by the responses of outpatient

nursing cohort; 2) Outpatient nursing sample size; 3) Location information was not collected (it

was only known that electronic surveys were completed by Consortium members other than

Intermountain Healthcare and the University of Utah 2012 DNP cohort). Although this

information is not pertinent when the data is collated, it will matter as further analysis is done,

because it is impossible to define if there is a representative sample; 4) The survey groups may

have been too broad, the sample population should have been limited to less than six subgroups);

and 5) Development and implementation of the electronic survey was more challenging than

expected. This limited responses from CCC clinicians.

Project Recommendations

TOO MUCH DATA, NOT ENOUGH DATA 66

Future work related to this topic will be continuing iterations of this survey, focusing on

specific cohorts, additional work to understanding differences in presentation, attempting to

understand what efforts needs to be accomplished related to reconciliation as it links to data

types and identifying priority of data integration in health information exchange

This project is considered exploratory research and is only in the beginning of discovery.

Because there is little current information on this subject, any additional information obtained

will continue to bring high value to the process of electronic health information exchange.

Although not within the scope of this DNP project, the author and content experts will continue

to be involved in expanding this project. Using a modified Delphi approach, additional surveys

will be given to CCC clinicians to refine recommendations for appropriate data exchange.

Specific focus will be on display of shared data and how information should be reconciled.

Summary

Relevant, pertinent and timely electronic data passed through a HIE must become a

recognized and critical component in providing better patient care. It is evident that more data

are not always helpful and in fact may adversely affect both the patient and the provider.

Knowing and understanding the exact health information that specific providers need at certain

point in the treatment process will only enhance the quality of care given to patients. This survey

has begun the exploratory process of identifying what that exact health information includes and

the analysis that must occur to establish the logic of this process

TOO MUCH DATA, NOT ENOUGH DATA 67

References

American Association of Colleges of Nursing. (2006). The essentials of doctoral education for

advanced nursing practice. Washington, D.C., 3-27.

American Nurses Association. Nursing informatics scope and standards of practice. Silver

Spring, MD nursebooks.org:2008.

Ancker, J.S., Edwards, A.M., Miller, M.C. & Kaushal, R. (2012). Consumer perceptions of

electronic health information exchange. American Journal of Preventive Medicine 43(1),

76-80.

Balfour, D.C., Evan, S., Januska, J., Lee, H.Y., Lewis, S.J., Nolan, S.R., Noga, M., Stemple, C.

& Thapar, L. (2009). Health information technology – results from a roundtable

discussion. Journal of Managed Care Pharmacy, 15, 10-17.

Blumenthal, D. &Tavenner, M. (2010). The “Meaningful Use” regulation for electronic health

records. New England Journal, 363(6), 501-504.

Burton, L.C., Anderson, G.F & Kues, I.W. (2004). Using electronic health records to help

coordinate care. The Milbank Quarterly, 82(3), 457-481.

Cash, J.J. (2009). Alert fatigue. American Journal of Health, 66(1), 2098-2101.

Care Connectivity Consortium. (2012). Available online @

http://centerfortotalhealth.org/tag/care-connectivity-consortium/

Daigh, R. (2012). Too much data, not enough information. Available online @

http://www.md- it.com/too-much-data-not-enough-information. Accessed May 19,

2012.

D’Amore, J.D. (2011). The promise of the CCD: Challenges and opportunity for quality

improvement and population health. J Am Med Inform Assoc., 210, 1-10.

TOO MUCH DATA, NOT ENOUGH DATA 68

Denard, J. (2012). Swimming in too much EMR data, Available online @

http://www.emrandehr.com/2012/05/31/swimming-in-too-much-emr-data. Accessed 5-

23-2012.

Department of Health and Human Services. Final Rule. 2010. Available online@

http://www.ofr.gov/OFRUpload/OFR Data/2010-17207_PI.pdf , Accessed 5-23-2012.

Eliot, T.S. (1934). The Rock. Farber and Faber, London, England.

Embi, P.J. & Leonard, A.C. (2012). Evaluating alert fatigue over time to EHR-based clinical

trial alerts: Findings from a randomized controlled study. Journal American Medical

Informatics Association, doi: 10.1136/amiajnl-2011-00743.

Evert. A., Trence, D., Catton. & Huynh, P., (2009). Continuous glucose monitoring technology

for personal use: An educational program that educates and supports the patient.

Diabetes Education, 35(4), 565-580.

Fricton, J.R. & Davies, D. (2010). Personal health records to improve health information

exchange and patient safety. J Am Med Inform Assoc., 12, 1-10.

Fisher, W.G. Doctors are downing in too much data, not enough appropriate information.

Available online @ http://medcitynews.com/2012/03/doctors-are-drowning-in-too-much-

data-not-enough-appropriate-information/. Accessed 5-23-12.

Francis, L. P. (2010). The physician-patient relationship and a national health information

network. Journal of Law, Medicine and Ethics, Spring, 36-49.

Frisse, M.E. (2010). Health information exchange in Memphis: Impact on the physician-patient

relationship. (2010). Journal of Law, Medicine and Ethics, Spring, 50-57.

Gadd, C.S., Ho, Y.W., Cala, M.C., Blakemore, D., Chen, Q., Frisse, M.E. & Johnson, K.B.,

(2011). User perspectives on the usability of a regional health information exchange.

TOO MUCH DATA, NOT ENOUGH DATA 69

J AmMed Inform Assoc., 18, 711-716.

Graves, J.R. & Corcoran, S. (1989). An overview of nursing informatics. Journal of Nursing

Scholarship, 29, 227-231.

Green, B.B., Cook, A.J., Ralston, J.D., Fikshman, P.A., Catx, S.L., Carlson, D., Tyll. L., Larson,

E.B., & Thompson, R.S. (2008). Effectiveness of home blood pressure monitoring. Web

communication and pharmacist care on hypertension control: A randomized controlled

trial. JAMA, 299(24), 2857-67.

Gorman, P.N. & Helfand, M. (1995). Information seeking in primary care: How physicians

choose which clinical questions to pursue and which to leave unanswered. Medical

Decision Making, 15:2, 113-119.

HIPPA regulations. (2012). Available online @

http://www.hhs.gov/ocr/privacy/hipaa/understanding/coveredentities/minimumnecessary.

html. Accessed 6-14-12.

HL7. (2012). Available online @ http://www.hl7.org/implement/standards/. Accessed 7-4-12.

Hayrinen, K., Saranto, K., Nykanen, P. (2007). Definition, structure, content, use and impacts of

electronic health records: A review of the research literature. International Journal of

Medical Informatics, 22, 291-304.

Hsu, C.C., Sandford, B.A. (2007). The Delphi technique: Making sense of consensus. Practical

Assessment, Research and Evaluation, 12(10), 1-8.

IHE. (2012). Accessed 7-5-12. IHE patient care coordination, technical framework

supplement. Available online @

http://www.ihe.net/Technical_Framework/upload/IHE_PCC_TF_Supplement_eNursingS

ummary_PC_2010-06-01.pdf. 1-17. Accessed 6-2-12.

TOO MUCH DATA, NOT ENOUGH DATA 70

Intermountain Stories, (2011). Available online @

https://my.intermountain.net/intermountainstories /Documents/CO-2011-04-11.pdf.

Accessed 7-14-12

Johnson, K.B., Unertl, K.M., Chen, Q, Lorenzi, N.M., Nikan, H., Bailey, J. & Frisse, M. (2011).

Health information exchange usage in emergency departments and clinics: The who,

what and why. J Am Med Inform Assoc., 18, 690-697.

Kesselheim, A.S., Creswell, K., Phansalkar, S., Bates, D.W. & Sheikh, A. (2011). Clinical

decision support systems could be modified to reduce ‘alert fatigue’ while still

minimizing the risk of litigation. Health Affairs 30(12), 2310-2317.

Krikelas, J. (1983). Information seeking behavior: Patterns and concepts. Drexel Library

Quarterly, 19, 5-20.

Kuperman, G.J. (2011). Health-information exchange: Why are we doing it and what are we

doing? Journal American Medical Informatics Association, 18,678-682.

Matney, S., Brewster, P.J., Sward, K.A., Cloyes, K.G. & Staggers, N. (2011). Philosophical

approaches to the nursing informatics data-information-knowledge-wisdom framework.

Advances in Nursing Science, 16(53), 1-13.

Maslove, D.M., Rizk, N. & Lowe, H.J. (2012). Computerized physician order entry in the

critical care environment: A review of current literature. Journal of Intensive Care

Medicine, 26(3), 165-171.

McKinsey and Company. (2011). Available online @ www.mckinsey.com.mgi, 1-3.

Accessed 6-10-12.

MCTB. (2011). Available online @ http://222.mtbc.com/learnaingcenter/index,php/care-

connectivity-consortium-to-demonstrate-feasibility-of-cross-country-data-exchange .

TOO MUCH DATA, NOT ENOUGH DATA 71

(1-3). Accessed 5-21-2012.

Merriam-Webster’s Collegiate Dictionary, 12th

Edition. Springfield, MA: G. &C. Merriam

Company, Publishers, 2012.

National Health Information Exchange. Available online @

http://www.healthit.hhs.gov/portal/server.pt/community/healhit-hhs-gov-nibn-

exchange/1407. Accessed 12 June 2012.

National Health Information Network, President’s Council of Advisors on Science and

Technology, (2010). Report to the president realizing the full potential of health

information technology to improve healthcare for Americans: the path forward.

Available online @ http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-

health-it-report.pdf. Accessed 7-6-12.

O’Malley, A.S., Grossman, J.M., Cohen, G.R., Kemper, N.M. & Pham, H.H. (2009). Are

electronic medical records helpful for care coordination? Experiences of physician

practices. J Am Med Inform Assoc., 251, 177-185.

Office of Health Information Technology. 2012. Available online @

http://healthit.hhs.gov/blog/faca/index.php/2010/12/10/onc-seeks-comment-on-standards-

and-interoperability-framework-initiatives-by-december-23-2010/. Accessed 7-4-12.

Pho, K. Too much data can overwhelm physicians and harm patients. 2012. Available online

@ http://www.kevinmd.com/blog/2010/10/data-overwhelm-physicians-harm-

patients.html. Accessed 6-12-12.

Rand Survey Meta Data Repository (2011). Available online @

https://mmicdata.rand.org/meta/?section=survey&surveyid=60&display=. Accessed

7- 21-12.

TOO MUCH DATA, NOT ENOUGH DATA 72

Richmond, R. (2012). Why hospitals continue to fail in “connect the dots” with their data, and

what they can do to change. (2012). Available online @

http://thehealthcareblog.com/blog/2012/02/01/why-hospitals-continue-to-fail-in-

%E2%80%98connecting-the-dots%E2%80%99-with-their-data-and-what-they-can-do-

to-ch/. Accessed 7-21-12.

Rudin, R., Volk, L., Simon, S. & Bates, D. (2011). What affects clinicians’ usage of health

information exchange. Applied Clinical Informatics, 1(3), 250-262.

Shakespeare, W. (1961). As you like it. Fawcett Press, New York, NY.

Shapiro, J.S., Kannry, J., Kushniruk, A.W. & Kuppman, G. (2007). Emergency physicians’

perceptions of health information exchange. J Am Med Inform Assoc, 14:

700-705.

Tielman, T., Van Velck, M.A., Stein, D.M., Stetson, P.D. & Johnson, S.B. (2007). Assessing

data relevance for automated generation of a clinical summary. AMIA 2007

Symposium Proceedings, 761-765.

Unertl, K.M., Johnson, K.B. & Lorenzi, N.M. (2012). Health information exchange technology

on the front lines of healthcare: Workflow factors and patterns of use. J Am Med Inform,

19, 392-400.

Utah Health Information Network. (2012). Available online @ http://www.uhin.org/.

Accessed 7-4-12.

Van Vleck, T.T., Stein, D.M., Stetson, P.D., & Johnson, S.B. (2007). Assessing data relevance

for automated generation of a clinical summary. AMIA2007 Symposium Proceedings,

761-765.

Vest, J.R., Hongwei, Z., Jasperson, J., Gamm, L.D. & Ohsfeldt, R.L. (2011). Factors motivating

TOO MUCH DATA, NOT ENOUGH DATA 73

and affecting health information exchange usage. Journal American Medical Informatics

Association, 18, 143-149.

Vest, J.R., Jasperson, J., Hongwei, H., Gamm, L.D. & Ohsfeldt, R.L. (2011). Use of health

information exchange system in the emergency care of children. BMC Medical

Informatics and Decision Making, 11(78), 1-10.

Watson, N. (2006). Patients should have to opt in or out of national electronic records.

Available online @ http://www.bmj.com/content/suppl/2006/06/23/332.7556.DC1, 1-4.

Accessed 7-22-12.

Weitzman, E.R., Kelemen, S., Kaci, L. & Mandl, K.D. (2009). Willingness to share personal

health record data for care improvement and public health: A survey of experienced

personal health record users. BMC Medical Informatics and Decision Making. Available

online @ http://www.biomedcentral.com/1472-6947/12/39. Accessed 6-24-12.

Wikipedia. (2012). Clinical Document Architecture. Available online @

http://en.wikipedia.org/wiki/Clinical_Document_Architecture. Accessed 6-21-12

Wolters-Kluwer Health 2011 Point-of-Care Survey. Available online @

http://www.wolterskluwerhealth.com/News/Documents/White%20Papers/Wolters%20Kl

uwer%20Health%20Survey%20Executive%20Summary-Media.pdf. Accessed 6-21-12.

Yasnoff, W.A. National Health Infrastructure. Available online @

http://www.ehcca.com/presentations/HIPAA5/yasnoff.pdf. Accessed 10-13-12.

Zeng, Q., Cimino, J.J. & Zou, K.H. (2002). Providing concept-oriented views for clinical data

using a knowledge-based system. J Am Med Inform Assoc, 9, 294-305.

TOO MUCH DATA, NOT ENOUGH DATA 74

Appendix A

IRB Applications and Addendums

TOO MUCH DATA, NOT ENOUGH DATA 75

From: [email protected] Sent: Friday, August 17, 2012 10:28 AM To: Kathleen Merkley Subject: ERICA IRB New Study Approval

IRB: IRB_00058678

PI: Kathleen Merkley

Title: Electronic Data Relevance

Study

Thank you for submitting your request for approval of this study. The IRB has administratively reviewed your application and a designated IRB member has determined that your study is exempt from further IRB review, under 45 CFR 46.101(b), Category 2, from

the Federal regulations governing human research. It is the policy of the University of Utah that all human subject research which is exempt under this section will be conducted in accordance with (1) the Belmont report (http://ohrp.osophs.dhhs.gov/humansubjects/guidance/belmont.htm ), (2) this institution's administrative procedures to ensure valid

claims of exemption, and (3) orderly accounting for such activities. All research involving human subjects must be approved or exempted by the IRB before the research is conducted (http://www.research.utah.edu/irb/guidelines/pdf/IGS/IGS-ExemptResearch.pdf).

Since this determination is not an approval, it does not expire or need renewal. This determination of exemption from continuing IRB review only applies to the research study as submitted to the IRB

and you are expected to follow the protocol as outlined. Before

implementing any changes in the study, you must submit an amendment application to the IRB and secure either approval or a determination of exemption.

Please remember to submit final IRB approval from Intermountain Healthcare to the University Utah IRB by way of Amendment. If you have questions about this, please contact our office at 581-3655 and we will be happy to assist you. Thank you again for submitting your proposal.

Click IRB_00058678 to view the application. Please take a moment to complete our customer service survey. We

appreciate your opinions and feedback.

TOO MUCH DATA, NOT ENOUGH DATA 76

TOO MUCH DATA, NOT ENOUGH DATA 77

TOO MUCH DATA, NOT ENOUGH DATA 78

Appendix B

List of CCC Committees

TOO MUCH DATA, NOT ENOUGH DATA 79

4© 2012 Kaiser Foundation Health Plan, Inc. Confidential, not for distribution or duplication

CCC Committees (continued)

LegalAs needed

Develops agreements including

data use and reciprocal support-

like agreement for health

information exchange

Develops proposal for long-term

Collaborative legal structure

Prepares agreements

implementing any approved

proposal

Ed Zych* Jesse Matt* Morris Linton* Allen Samelson (L)* Sherry Hubert *

Kimberly Otte*

CommunicationsEvery week

Develops communication content

and plans

Develops process for media

inquiries and other public

communications

Develops content and plans for “go

live” public announcement

Amanda O’Rourke *

Susan Alcorn

Aliyah Quraishi

Mike Foley

Anne Robertson*

Bill Barnes

Daron Cowley

Jason Burgess

Stan Clark

Ravi Poorsina (L)*

Vishakha Sant

Amanda Higgins

Roma McCaig

Cristina Holmes

Holly Potter

Adrian Sanchez

Mallard Jennifer*

Rebecca Eisenman

Reg Smith

SecurityEvery two weeks

Ensure CCC's have appropriate

security controls that include

conducting threat and vulnerability

assessments, analyzing risk and

implementation compensating

controls, conducting security

monitoring, defining cross-

organizational security incident

plan and escalation plans, and

Sharing best practices, security-

related work efforts and learning’s

with sub-team

John Kravitz

Kevin Kerestus

Greg Romania

Darcy Curtiss

Chris Thompson

Aliyah Quraishi

Kurt Hardesty

Chris Grant

Katherine Augustin

Karl West

Carl Allen

Jason Gagner

Jason Zellmer

Michael Makstman

Simon Nazarian

Ross Kwok

Zach Gillen

Seth Selkow

Kevin Isbell

Reg Smith

Jack Mogren

Jacki Pemrick

Chad Hirsch

Audit &

ComplianceMeeting Frequency TBD

Responsible for defining,

implementing and overseeing the

Compliance and Audit functions for

Care Connectivity Consortium.

Ensure that CCC complies to all

applicable laws, rules, regulations,

codes of conduct, CCC policies

and standards of good practice

Ensure audit controls are in place

to periodically monitor and

evaluate risk.

Kevin Kerestus

Joy Campbell

Andy Kling

Robert Thieling Cavell Alexander

Jutta Williams

Sid Thornton

Scott Morgan

Ross Kwok

Jacki Pemrick

Chad Hirsch

Responsibility

* Denotes primary representative

L Denotes committee leader

TOO MUCH DATA, NOT ENOUGH DATA 80

Appendix C

PreSurvey Letter

TOO MUCH DATA, NOT ENOUGH DATA 81

September 23, 2012

You are one of a small group of clinicians that has been invited to provide feedback on a study

looking at the relevance of different types of data that can be exchanged within the Care

Connectivity Consortium (CCC). As you recall, the CCC is a consortium of five leading U.S.

healthcare organizations, including Mayo (Grouphealth, Geisinger, and Kaiser Permanente)

which was recently formed to help promote electronic transfer of health information across the

country.

Working from the hypotheses that more data are not always helpful when transferred

electronically, this study will attempt to identify what electronic data are relevant to specialty,

patient type, patient acuity and chronicity of illness. It will also determine what clinicians

consider to be trustworthy data, how exchanged data should be reconciled if there are

discrepancies and what time limitations may be placed on certain data categories.

Taking part in this survey is completely voluntary; however, I fully support the premise of this

important project and strongly encourage you to participate. The more surveys that are

completed, the more likely the results will be reliable. This information may only be obtained

with your help. I sincerely appreciate you taking the time to provide us with your valuable input

in this important endeavor.

Sincerely,

Brent Wallace, MD

Chief Medical Officer

Intermountain Healthcare

TOO MUCH DATA, NOT ENOUGH DATA 82

September 23, 2012

You are one of a small group of nurses that has been invited to provide feedback on a study

looking at the relevance of different types of data that can be exchanged within the Care

Connectivity Consortium (CCC). The CCC is a consortium of five leading U.S. healthcare

organizations, including Intermountain Healthcare which was recently formed to help promote

electronic transfer of health information across the country. This information will be merged

with recommendations already obtained from the Utah Health Information Network (UHIN).

Working from the hypotheses that more data are not always helpful when transferred

electronically, this study will attempt to identify what electronic data are relevant to nurses

related to specialty, patient type, patient acuity and chronicity of illness. It will also determine

what nurses consider to be trustworthy data, how exchanged data should be reconciled if there

are discrepancies and what time limitations may be placed on certain data categories.

Taking part in this survey is completely voluntary; however, I fully support the premise of this

important project and strongly encourage you to participate. The more surveys that are

completed, the more likely the results will be reliable. This information may only be obtained

with your help. I sincerely appreciate you taking the time to provide us with your valuable input

in this important endeavor.

Sincerely,

Kim Henrichsen. RN, MS

Chief Nursing Officer

Intermountain Healthcare

TOO MUCH DATA, NOT ENOUGH DATA 83

Appendix D

Invitation Letter – CCC Members

TOO MUCH DATA, NOT ENOUGH DATA 84

From: Kathleen Merkley/Chris Wood

Sent: September 10, 2012

To: XXXXXX

Subject: Providing Data Relevance to Care Connectivity Consortium Members

October 19, 2012

We are writing to ask for your participation in a survey that we are conducting through the Care

Connectivity Consortium. We are asking clinicians like you to reflect on the types of electronic

data you would find helpful when caring for a patient who has come to your facility from another

healthcare system.

Your responses to this survey are very important and will help in advancing the transfer of

relevant data between healthcare systems. As part of the survey, we are also asking you to

consider what makes you trust the data you are receiving.

This is a short survey and should take you no more than ten minutes to complete. Please click on

the link below to go to the survey website (or copy and paste the survey link into your Internet

browser) to acccess the survey.

Survey Link: http://www.surveygizmo.com/s3/1048374/Electronic-Data-Relevance-Survey

Your participation in this survey is entirely voluntary and all of your responses will be kept

confidential. No personally identifiable information will be associated with your responses in

any reports of this data. Should you have an further questions or comments, please feel free to

contact [email protected] or 801.633.1579.

We appreciate your time and consideration in completing this survey. Thank you for

participating in this study. It is only through the help of feedback from clinical experts that we

can provide appropriate data to help guide the direction of the CCC data exchange.

Many thanks,

Kathleen Merkley,APRN, MS, FNP Chris Wood, MD

ECIM Implementation Medical Director, InformationSystems

Intermountain Healthcare Intermountain Healthcare

TOO MUCH DATA, NOT ENOUGH DATA 85

Appendix E

Invitation Letter – Intermountain Healthcare Participants

TOO MUCH DATA, NOT ENOUGH DATA 86

From: Kathleen Merkley/Chris Wood

Subject: Providing Data Relevance to Care Connectivity Consortium Members

August 31, 2012

We are asking Intermountain clinicians to participate in a survey about electronic data relevance.

This is being done in conjunction with a larger survey of the Care Connectivity Consortium

(CCC) a group of health care organizations across the United States who recently formed to help

promote electronic transfer of health information across the country. Would you please reflect

on the types of electronic data you would find helpful when caring for a patient who has come to

your facility from another healthcare system.

Your responses to this survey are very important and will help in advancing the transfer of

relevant data between healthcare systems. As part of the survey, we are also asking you to

consider what makes you trust the data you are receiving. This is a short survey and should take

you no more than ten minutes to complete.

Your participation in this survey is entirely voluntary and all of your responses will be kept

confidential. No personally identifiable information will be associated with your responses in

any reports of this data. Should you have an further questions or comments, please feel free to

contact [email protected] or 801.633.1579.

We appreciate your time and consideration in completing this survey. Thank you for

participating in this study. It is only through the help of feedback from clinical experts that we

can provide appropriate data to help guide the direction of the CCC data exchange.

Many thanks,

Kathleen Merkley,APRN, MS, FNP Chris Wood, MD

ECIM Implementation Medical Director, InformationSystems

Intermountain Healthcare Intermountain Healthcare

TOO MUCH DATA, NOT ENOUGH DATA 87

Appendix F

Thank You Letter

TOO MUCH DATA, NOT ENOUGH DATA 88

From: Kathleen Merkley/Chris Wood

Sent: October 7, 2012

To: XXXXX

Subject: Providing Data Relevance to Care Connectivity Consortium Members

October 7, 2012

Thank you for agreeing to participate in this survey. Your inputs will be extremely valuable in

our efforts to determine which data components should be included related to provider type,

patient acuity, severity and chronicity, as well as how data can be considered more trustable and

timely.

As a respondent, the results of the survey will be displayed on the CCC Wiki by the end of

November. Thank you for your participation and please know the time you spent in providing

feedback will be utilized to enhance data sharing between consortium members.

Sincerely,

Kathleen Merkley,APRN, MS, FNP Chris Wood, MD

ECIM Implementation Medical Director, InformationSystems

Intermountain Healthcare Intermountain Healthcare

TOO MUCH DATA, NOT ENOUGH DATA 89

Appendix G

Recommendations – Care Connectivity Consortium

TOO MUCH DATA, NOT ENOUGH DATA 90

CCC Recommendations

1. Data Set Recommendations

Data/Information Type

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and telephone

number

Physical activity

Plain radiographic images

Radiology Tests (report)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

TOO MUCH DATA, NOT ENOUGH DATA 91

2. Data per Specialty Recommendations (top 10)

LIP Inpatient

Laboratory Results

Radiology Test Reports

Discharge Summaries

Advanced Directives

Provider Information

Medical Devices

Vital Signs

Procedure Reports

Operative Tests

Radiology Test Images

ED LIP

Operative Summaries

Procedure Reports

Plain Radiographic Images

Radiology Test Results

Vital Signs

Electrocardiograms

Immunizations

Provider Information

Family History

Lab Results

Pediatric LIP

Discharge Summaries

Immunizations

Lab Results

Radiology Tests

Provider Information

Plain Radiographic Images

Growth Records

Procedure Results

Radiology Test Images

Operative Summaries

TOO MUCH DATA, NOT ENOUGH DATA 92

Outpatient LIP

Discharge Summaries

Laboratory Reports

Medical Devices

Advanced Directives

Procedure Reports

Operative Summaries

Immunization History

Health Maintenance Records

Electrocardiograms

Vital Signs

Inpatient Nursing

Laboratory Results

Advanced Directives

Procedure Reports

Immunizations

Special Needs

Medical Devices

Pending Tests

Progress Notes

Radiology Reports

Mobility/Falls Risk

Outpatient Nursing

Discharge Summaries

Family History

Laboratory Results

Operative Summaries

Progress Notes

Vital Signs

Procedure Results

Health Insurance

Radiology Test Reports

Special Needs

TOO MUCH DATA, NOT ENOUGH DATA 93

1. Critical Data Recommendations (top ten)

Inpatient LIP

Electrocardiograms

Advanced Directives

Operative Summaries

Progress Notes

Radiology Tests – Images

Behavioral Health History

Diet

Health Maintenance

Plain Radiographic Images

Radiology Tests - Reports

Pediatric LIP

Advanced Directives

Discharge Summaries

Laboratory Results

Radiology Tests – Reports

Vital Signs

Procedure Notes

Plain Radiographic Images

Operative Summaries

Progress Notes

Electrocardiograms

ED LIP

Advanced Directives

Radiographic Tests - Images

Behavioral Health History

Cognitive History

Discharge Summaries

Electrocardiograms

Provider Information

Diet

Family History

Growth Record

TOO MUCH DATA, NOT ENOUGH DATA 94

Inpatient Nursing

Electrocardiograms

Review of Systems

Operative Reports

Discharge Summaries

Radiology Tests – Reports

Radiology Tests – Images

Plain Radiographs

Pending Tests

Progress Notes

Vital Signs

2. Priority Data (top five)

Priority Data

Discharge Summaries

Laboratory Data

Advanced Directives

Radiographic Tests - Reports

Electrocardiograms

3. Data Lifecycle Recommendations

Data Type Distribution 1 Bi-Modal Distribution

Radiographs 1 week – 3 months Within 5 years

Electrocardiograms 1 week – 1 month 1 -5 years

Labs 1 week – 1 month 1-5 years

Procedures 1 week – 1 month Within 5 years

Discharge Summaries 5 years to complete

Problem lists 5 years to complete

Medications Within 5 years

Vital Signs Within 1 week

TOO MUCH DATA, NOT ENOUGH DATA 95

Appendix H

Electronic Data Relevance Emergency Survey

TOO MUCH DATA, NOT ENOUGH DATA 96

Emergency Survey

Electronic Data Relevance Survey

With the advent of electronic health information exchange across health care systems, there are a

number of questions about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These questions are as follows:

Very little research has been conducted on what types of data and information would be

most important to exchange and the format in which it should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

results. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered went sent from another healthcare system.

Responses to this survey will begin to address these issues. Please answer the following

questions.

Identification of Most Valuable Data When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for information exchanged by another health care system.

In addition there is a concern about overwhelming clinicians with too much information that may

not be relevant. Actual data or information to be exchanged (to provide care coordination and the

best care) are in question. It is important to understand out of all existing data and information,

what is most relevant when providing clinical care.

TOO MUCH DATA, NOT ENOUGH DATA 97

1. As a clinician treating patients from another healthcare system in your emergency

department you understand you could receive any or all data or information from

that system. You already have immediate electronic access to your patients’ allergy,

medication and problem list. Please indicate what additional data/information

would be important for you to have in caring for your ED patients.

First, check the boxes next to all data and information types that you think would be

important in treating critical ED patients and then all ED patients.

Second, rank up to 5 data types you think would be MOST RELEVANT for you to

have in treating your ED patients, with “1” being the single most important type of

data and “5” being the fifth most important type of data or information.

Data/Information Type Critical

ED

Patients

All ED

Patients

Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and telephone

number

Physical activity

Plain radiographic images

Radiology Tests (report)

CT, endoscopy, nuclear medicine

TOO MUCH DATA, NOT ENOUGH DATA 98

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

Presentation of Received Data

Data and information received from another health care system may be received electronically

by clinicians in the form of a single document or incorporated into the patient’s current

electronic medical record (EMR).

2. In your opinion, should data/information received from another healthcare system

be displayed as a separate electronic document and/or as integrated data within the

patient’s current electronic medical record?

a. Received as a separate electronic document?

b. Received as integrated data/information within the patient’s current EMR?

c. Received as both a separate electronic document and as integrated data/information

within the patient’s current EMR?

Reconciliation of Received and Existing Data

If data/information sent from another healthcare system are integrated within the patient’s

current EMR, there may be a need to reconcile duplications, inconsistencies and contradictions.

Part of this reconciliation will be completed by the computer but human interaction/decision

making will likely be needed as well.

TOO MUCH DATA, NOT ENOUGH DATA 99

3. Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for

a patient they are currently caring for?

a. Yes

b. No, reconciliation is not necessary.

c. Only as needed on a case by case basis.

4. If such reconciliation of data or information was required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution.

b. The nurse caring for the patient in the receiving institution.

c. The first licensed care giver encountering the patient in the receiving institution.

d. Should be dependent on the data shared (examples would be: pharmacist

reconciling medications, physicians reconciling labs, problems or allergies, and

nursing reconciling patient goals).

e. Other___________________________

Trustworthiness of Data

5. How IMPORTANT would each of the following conditions be in deciding if you

trust the data/information received from another healthcare system?

TOO MUCH DATA, NOT ENOUGH DATA 100

a. The strength of a working relationship with the provider from an outside

healthcare system who ordered or performed the procedure or otherwise prepared

the information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

b. The reputation of the healthcare organization sending the data/information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

c. Trusting the integrity of the health data exchange. Not Important Somewhat Important Neutral Important Very Important Don’t Know

d. Knowing the type of provider who ordered or performed the procedure or

otherwise prepared the information (specialist vs. generalist). Not Important Somewhat Important Neutral Important Very Important Don’t Know

e. The complexity of the patient. Not Important Somewhat Important Neutral Important Very Important Don’t Know

f. I can repeat the test/procedure and the repeated test/procedure will again be

reimbursed. Not Important Somewhat Important Neutral Important Very Important Don’t Know

g. How well the data/information is organized in a reliable (i.e. is the format

intuitive). Not Important Somewhat Important Neutral Important Very Important Don’t Know

h. I trust the completeness, timeliness and accuracy of the data? Not Important Somewhat Important Neutral Important Very Important Don’t Know

Useful Date Ranges of Data

TOO MUCH DATA, NOT ENOUGH DATA 101

6. When looking for electronic health information exchanged across health care

systems what is the general time limit, by category that data/information is useful.

a. Vital Signs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

b. Laboratory Results

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

c. Procedures

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

d. Previously administered medications

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

TOO MUCH DATA, NOT ENOUGH DATA 102

e. Radiographs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

f. Electrocardiograms

Within 1 week Within 1 month Within 3 months Within 6 months Within 1 year Within 5 years Other

g. Problem List

Within 1 year

Within 2 years

Within 5 years

Complete

Other

h. Discharge Summaries

Within 1 year

Within 2 years

Within 5 years

Complete

Other

8. What is your profession?

Other ____________________________________________

TOO MUCH DATA, NOT ENOUGH DATA 103

Appendix I

Electronic Data Relevance Survey – Inpatient Nursing

TOO MUCH DATA, NOT ENOUGH DATA 104

In-Patient Nursing Survey

Electronic Data Relevance Survey

With the advent of electronic health information exchange across health care systems, there are a

number of questions about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These questions are as follows:

Very little research has been conducted on what types of data/information would be most

important to exchange and the format in which it should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

data. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered went sent from another healthcare system.

Responses to this survey will begin to address these issues. Please answer the following

questions.

Identification of Most Valuable Data When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for data/information exchanged by another health care

system. In addition there is a concern about overwhelming clinicians with too much

data/information that may not be relevant. Actual data/information to be exchanged (to provide

care coordination and the best care) is in question. It is important to understand out of all

existing data, what is most relevant when providing clinical care.

TOO MUCH DATA, NOT ENOUGH DATA 105

1. As a clinician caring for patients from another healthcare system on your unit you

understand you could receive any or all data/information from that system. You

already have immediate electronic access to your patients’ allergy, medication and

problem list. Please indicate what additional data/information would be important

for you to have in caring for your hospitalized patients.

First, check the boxes next to all data and information types that you think would be

important in treating your patients.

Second, rank up to 5 data/information types you think would be MOST

RELEVANT for you to have in treating your patients, with “1” being the single

most important type of data /information and “5” being the fifth most important

type of data/information.

Data/Information Type Critical

Patients

All

Patients

Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and telephone

number

Physical activity

Plain radiographic images

Radiology Tests (report)

TOO MUCH DATA, NOT ENOUGH DATA 106

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

Presentation of Received Data

Data and information received from another health care system may be received electronically

by clinicians in the form of a single document or incorporated into the patient’s current

electronic medical record (EMR).

2. In your opinion, should data/information received from another healthcare system

be displayed as a separate electronic document and/or as integrated data within the

patient’s current electronic medical record?

d. Received as a separate electronic document?

e. Received as integrated data within the patient’s current EMR?

f. Received as both a separate electronic document and as integrated data within the

patient’s current EMR?

TOO MUCH DATA, NOT ENOUGH DATA 107

Reconciliation of Received and Existing Data

If data /information from another healthcare system are integrated within the patient’s current

EMR, there may be a need to reconcile duplications, inconsistencies and contradictions. Part of

this reconciliation will be completed by the computer but human interaction/decision making

will likely be needed as well.

3. Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for

a patient they are currently caring for?

a. Yes

b. No, reconciliation is not necessary.

c. Only as needed on a case by case basis.

4. If such reconciliation of data or information was required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution.

b. The nurse caring for the patient in the receiving institution.

c. The first licensed care giver encountering the patient in the receiving institution.

d. Should be dependent on the data shared (examples would be: pharmacist

reconciling medications, physicians reconciling labs, problems or allergies, and

nursing reconciling patient goals).

e. Other___________________________

TOO MUCH DATA, NOT ENOUGH DATA 108

Trustworthiness of Data

5. How IMPORTANT would each of the following conditions be in deciding if you

trust the data/information received from another healthcare system?

a. The strength of a working relationship with the provider from an outside

healthcare system who ordered or performed the procedure or otherwise prepared

the information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

b. The reputation of the healthcare organization sending the data. Not Important Somewhat Important Neutral Important Very Important Don’t Know

c. Trusting the integrity of the health data exchange. Not Important Somewhat Important Neutral Important Very Important Don’t Know

d. Knowing the type of provider who ordered or performed the procedure or

otherwise prepared the information (specialist vs. generalist). Not Important Somewhat Important Neutral Important Very Important Don’t Know

e. The complexity of the patient. Not Important Somewhat Important Neutral Important Very Important Don’t Know

f. I can repeat the test/procedure and the repeated test/procedure will again be

reimbursed. Not Important Somewhat Important Neutral Important Very Important Don’t Know

g. How well the data is organized in a reliable (i.e. is the format intuitive). Not Important Somewhat Important Neutral Important Very Important Don’t Know

h. I trust the completeness, timeliness and accuracy of the data? Not Important Somewhat Important Neutral Important Very Important Don’t Know

TOO MUCH DATA, NOT ENOUGH DATA 109

Useful Date Ranges of Data

6. When looking for electronic health information exchanged across health care

systems what is the general time limit, by category that data/information is useful.

a. Vital Signs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

b. Laboratory Results

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

c. Procedures

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

d. Previously administered medications

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

TOO MUCH DATA, NOT ENOUGH DATA 110

e. Radiographs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

f. Electrocardiograms

Within 1 week Within 1 month Within 3 months Within 6 months Within 1 year Within 5 years Other

g. Problem List

Within 1 year

Within 2 years

Within 5 years

Complete

Other

h. Discharge Summaries

Within 1 year

Within 2 years

Within 5 years

Complete

Other

8. What is your level of education?

AD □

ADN□

BS□

BSN □

Masters Prepared □

Clinical Area of Expertise ____________________________________

TOO MUCH DATA, NOT ENOUGH DATA 111

Appendix J

Electronic Data Relevance Survey – Inpatient

TOO MUCH DATA, NOT ENOUGH DATA 112

Inpatient Survey

Electronic Data Relevance Survey

With the advent of electronic health information exchange across health care systems, there are a

number of questions about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These questions are as follows:

Very little research has been conducted on what types of data/information would be most

important to exchange and the format in which it should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

data. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered went sent from another healthcare system.

Responses to this survey will begin to address these issues. Please answer the following

questions.

Identification of Most Valuable Data When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for data/information exchanged by another health care

system. In addition there is a concern about overwhelming clinicians with too much

data/information that may not be relevant. Actual data/information to be exchanged (to provide

care coordination and the best care) is in question. It is important to understand out of all

existing data, what is most relevant when providing clinical care.

TOO MUCH DATA, NOT ENOUGH DATA 113

1. As a clinician treating admitted patients from another healthcare system in your

hospital you understand you could receive any or all /information from that system.

You already have immediate electronic access to your patients’ allergy, medication

and problem list. Please indicate what additional data/information would be

important for you to have in caring for your admitted patients.

First, check the boxes next to all data and information types that you think would be

important in treating critically ill patients and then all admitted patients.

Second, rank up to 5 data types you think would be MOST REVELANT for you to

have in treating your admitted patients, with “1” being the single most important

type of data/information and “5” being the fifth most important type of

data/information.

Data/Information Type Critically

Ill

Patients

All

Admitted

Patients

Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and

telephone number

Physical activity

Plain radiographic images

Radiology Tests (report)

TOO MUCH DATA, NOT ENOUGH DATA 114

CT, endoscopy, nuclear

medicine

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear

medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

Presentation of Received Data

Data and information received from another health care system may be received electronically

by clinicians in the form of a single document or incorporated into the patient’s current

electronic medical record (EMR).

2. In your opinion, should /information received from another healthcare system be

displayed as a separate electronic document and/or as integrated data within the

patient’s current electronic medical record?

g. Received as a separate electronic document?

h. Received as integrated data within the patient’s current EMR?

i. Received as both a separate electronic document and as integrated data within the

patient’s current EMR?

TOO MUCH DATA, NOT ENOUGH DATA 115

Reconciliation of Received and Existing Data

If data/information sent from another healthcare system are integrated within the patient’s

current EMR, there may be a need to reconcile duplications, inconsistencies and contradictions.

Part of this reconciliation will be completed by the computer but human interaction/decision

making will likely be needed as well.

3. Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for

a patient they are currently caring for?

a. Yes

b. No, reconciliation is not necessary.

c. Only as needed on a case by case basis.

4. If such reconciliation of data or information was required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution.

b. The nurse caring for the patient in the receiving institution.

c. The first licensed care giver encountering the patient in the receiving institution.

d. Should be dependent on the data shared (examples would be: pharmacist

reconciling medications, physicians reconciling labs, problems or allergies, and

nursing reconciling patient goals).

e. Other___________________________

TOO MUCH DATA, NOT ENOUGH DATA 116

Trustworthiness of Data

5. How IMPORTANT would each of the following conditions be in deciding if you

trust the data/information received from another healthcare system?

a. The strength of a working relationship with the provider from an outside

healthcare system who ordered or performed the procedure or otherwise prepared

the information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

b. The reputation of the healthcare organization sending the data. Not Important Somewhat Important Neutral Important Very Important Don’t Know

c. Trusting the integrity of the health data exchange. Not Important Somewhat Important Neutral Important Very Important Don’t Know

d. Knowing the type of provider who ordered or performed the procedure or

otherwise prepared the information (specialist vs. generalist). Not Important Somewhat Important Neutral Important Very Important Don’t Know

e. The complexity of the patient. Not Important Somewhat Important Neutral Important Very Important Don’t Know

f. I can repeat the test/procedure and the repeated test/procedure will again be

reimbursed. Not Important Somewhat Important Neutral Important Very Important Don’t Know

g. How well the data is organized in a reliable (i.e. is the format intuitive). Not Important Somewhat Important Neutral Important Very Important Don’t Know

h. I trust the completeness, timeliness and accuracy of the data? Not Important Somewhat Important Neutral Important Very Important Don’t Know

TOO MUCH DATA, NOT ENOUGH DATA 117

Useful Date Ranges of Data

6. When looking for electronic health information exchanged across health care

systems what is the general time limit, by category that data/information is useful.

a. Vital Signs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

b. Laboratory Results

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

c. Procedures

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

d. Previously administered medications

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

TOO MUCH DATA, NOT ENOUGH DATA 118

e. Radiographs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

f. Electrocardiograms

Within 1 week Within 1 month Within 3 months Within 6 months Within 1 year Within 5 years Other

g. Problem List

Within 1 year

Within 2 years

Within 5 years

Complete

Other

h. Discharge Summaries

Within 1 year

Within 2 years

Within 5 years

Complete

Other

8. What is your profession?

Other

TOO MUCH DATA, NOT ENOUGH DATA 119

Appendix K

Electronic Data Relevance Survey – Primary Care

TOO MUCH DATA, NOT ENOUGH DATA 120

Primary Care Survey

Electronic Data Relevance Survey

With the advent of electronic health information exchange across health care systems, there are a

number of questions about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These questions are as follows:

Very little research has been conducted on what types of data/information would be most

important to exchange and the format in which it should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

data. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered went sent from another healthcare system.

Responses to this survey will begin to address these issues. Please answer the following

questions.

Identification of Most Valuable Data When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for data/information exchanged by another health care

system. In addition there is a concern about overwhelming clinicians with too much

data/information that may not be relevant. Actual data /information to be exchanged (to provide

care coordination and the best care) are in question. It is important to understand out of all

existing data and information, what is most relevant when providing clinical care.

TOO MUCH DATA, NOT ENOUGH DATA 121

1. As a clinician treating ambulatory patients from another healthcare system in an

primary care setting you understand you could receive any or all data/information

from that system. You already have immediate electronic access to your patients’

allergy, medication and problem list. Please indicate what additional

data/information would be important for you to have in caring for your clinic

patients.

First, check the boxes next to all data and information types that you think would be

important in treating clinic patients.

Second, rank up to 5 data types you think would be MOST RELEVANT for you to

have in treating your clinic patients with “1” being the single most important type of

data and “5” being the fifth most important type of data/information.

Data/Information Type All

Clinic

Patients

Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and telephone

number

Physical activity

Plain radiographic images

Radiology Tests (report)

TOO MUCH DATA, NOT ENOUGH DATA 122

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

Presentation of Received Data

Data and information received from another health care system may be received electronically

by clinicians in the form of a single document or incorporated into the patient’s current

electronic medical record (EMR).

2. In your opinion, should data/information received from another healthcare system

be displayed as a separate electronic document and/or as integrated data within the

patient’s current electronic medical record?

j. Received as a separate electronic document?

k. Received as integrated data within the patient’s current EMR?

l. Received as both a separate electronic document and as integrated data within the

patient’s current EMR?

Reconciliation of Received and Existing Data

TOO MUCH DATA, NOT ENOUGH DATA 123

If data/information sent from another healthcare system are integrated within the patient’s

current EMR, there may be a need to reconcile duplications, inconsistencies and contradictions.

Part of this reconciliation will be completed by the computer but human interaction/decision

making will likely be needed as well.

3. Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for

a patient they are currently caring for?

a. Yes

b. No, reconciliation is not necessary.

c. Only as needed on a case by case basis.

4. If such reconciliation of data or information was required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution.

b. The nurse caring for the patient in the receiving institution.

c. The first licensed care giver encountering the patient in the receiving institution.

d. Should be dependent on the data shared (examples would be: pharmacist

reconciling medications, physicians reconciling labs, problems or allergies, and

nursing reconciling patient goals).

e. Other___________________________

Trustworthiness of Data

TOO MUCH DATA, NOT ENOUGH DATA 124

5. How IMPORTANT would each of the following conditions be in deciding if you

trust the /information received from another healthcare system?

a. The strength of a working relationship with the provider from an outside

healthcare system who ordered or performed the procedure or otherwise prepared

the information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

b. The reputation of the healthcare organization sending the data. Not Important Somewhat Important Neutral Important Very Important Don’t Know

c. Trusting the integrity of the health data exchange. Not Important Somewhat Important Neutral Important Very Important Don’t Know

d. Knowing the type of provider who ordered or performed the procedure or

otherwise prepared the information (specialist vs. generalist). Not Important Somewhat Important Neutral Important Very Important Don’t Know

e. The complexity of the patient. Not Important Somewhat Important Neutral Important Very Important Don’t Know

f. I can repeat the test/procedure and the repeated test/procedure will again be

reimbursed. Not Important Somewhat Important Neutral Important Very Important Don’t Know

g. How well the data is organized in a reliable (i.e. is the format intuitive). Not Important Somewhat Important Neutral Important Very Important Don’t Know

h. I trust the completeness, timeliness and accuracy of the data? Not Important Somewhat Important Neutral Important Very Important Don’t Know

Useful Date Ranges of Data

TOO MUCH DATA, NOT ENOUGH DATA 125

6. When looking for electronic health information exchanged across health care

systems what is the general time limit, by category that data/information is useful.

a. Vital Signs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

b. Laboratory Results

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

c. Procedures

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

d. Previously administered medications

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

TOO MUCH DATA, NOT ENOUGH DATA 126

e. Radiographs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

f. Electrocardiograms

Within 1 week Within 1 month Within 3 months Within 6 months Within 1 year Within 5 years Other

g. Problem List

Within 1 year

Within 2 years

Within 5 years

Complete

Other

h. Discharge Summaries

Within 1 year

Within 2 years

Within 5 years

Complete

Other

8. What is your profession?

Other ____________________________________________

TOO MUCH DATA, NOT ENOUGH DATA 127

Appendix L

Electronic Data Relevance Survey – Electronic Copy

TOO MUCH DATA, NOT ENOUGH DATA 128

Electronic Survey

Electronic Data Relevance Survey

Welcome to the Electronic Data Relevance Survey. Thank you for taking some time to provide

your feedback.

With the advent of electronic health information exchange across healthcare systems, there are a

number of issues about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These issues include the following:

Very little research has been conducted on what types of data and information would be

most relevant to exchange and the format in which they should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

results. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered to be when received from another healthcare system.

Responses to this survey will help begin to address these issues. Please answer the questions that

follow.

Identification of Most Relevant Data

When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for information exchanged by another healthcare system.

In addition there is a concern about overwhelming clinicians with too much information that may

not be relevant. Actual data or information to be exchanged (to provide care coordination and the

best care) are in question. It is important to understand out of all existing data and information,

what is most relevant when providing clinical care.

The figure below is intended to illustrate the idea of identifying the most relevant information to

share between healthcare systems.

TOO MUCH DATA, NOT ENOUGH DATA 129

1. If as a clinician treating patients in an emergency department, you could receive

data and information electronically from another healthcare system on those

patients who have received care there.

If you knew that you could have immediate electronic access to those patients’

allergy, medication and problem lists from the other healthcare system, please

indicate what additional data/information you think would be relevant for you to

have from that system in caring for those ED patients.

Data/Information Type All ED

Patients

Critical ED

Patients

only

No ED

Patients

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

TOO MUCH DATA, NOT ENOUGH DATA 130

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and telephone

number

Physical activity

Plain radiographic images

Radiology Tests - Reports

(CT, endoscopy, nuclear medicine

scan, MRI, ultrasound)

Radiology Tests - Images

(CT, endoscopy, nuclear medicine

scan, MRI, ultrasound)

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

2. Again, if as a clinician in an emergency department you could receive data and

information electronically from another healthcare system on those patients who

have received care there.

Please rank up to 5 data/information types that you think would be MOST

RELEVANT for you to receive from that system in treating those ED patients, with

“1” being the single most important and “5” being the fifth most important type.

Data/Information Type Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

TOO MUCH DATA, NOT ENOUGH DATA 131

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results – invasive/noninvasive

Progress notes

Provider information and telephone number

Physical activity

Plain radiographic images

Radiology Tests - Reports

(CT, endoscopy, nuclear medicine

scan, MRI, ultrasound)

Radiology Tests - Images

(CT, endoscopy, nuclear medicine

scan, MRI, ultrasound)

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

a. Comments?

Presentation of Received Data/Information

Data and information received from another healthcare system may be received electronically by

clinicians in the form of a single document or integrated into the patient’s current electronic

medical record (EMR).

TOO MUCH DATA, NOT ENOUGH DATA 132

3. In your opinion, should data/information received from another healthcare system be

displayed as a separate electronic document and/or as integrated data within the

patient’s current electronic medical record?

m. Received as a separate electronic document

n. Received as integrated data/information within the patient’s current EMR

o. Received as both a separate electronic document and as integrated data/information

within the patient’s current EMR

a. Comments?

Reconciliation of Received and Existing Data

If data/information sent from another healthcare system are integrated within the patient’s

current EMR, there may be a need to reconcile duplications, inconsistencies and contradictions

(See figure below). Part of this reconciliation will be completed by the computer but human

interaction/decision making will likely be needed as well.

4. Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for a

patient they are currently caring for?

TOO MUCH DATA, NOT ENOUGH DATA 133

a. Yes

b. No, reconciliation is not necessary

c. Only as needed on a case by case basis

5. If such reconciliation of data or information were required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution

b. The nurse caring for the patient in the receiving institution

c. The first licensed care giver encountering the patient in the receiving institution

d. Should be dependent on the data shared (examples would be: pharmacist reconciling

medications, physicians reconciling labs, problems or allergies, and nursing reconciling

patient goals)

e. Other___________________________

a. Comments?

Trusting & Relying on Exchanged Health Data/Information

6. How IMPORTANT would each of the following be in determining how much you

would trust and rely on the data/information received from another healthcare system

when treating one of your ED patients?

No

t at

all

Sli

gh

tly

So

mew

hat

Ver

y

Ex

trem

ely

Do

n’t

Kn

ow

The complexity of the patient being treated

The reputation of the healthcare system sending the

data/information.

The type of provider who ordered or performed the

procedure or otherwise prepared the information (specialist

vs. generalist)

The strength of a working relationship with the provider

from the outside healthcare system who ordered or

performed the procedure or otherwise prepared the

information

How complete the data/information appear to be

How accurate the data/information appear to be

The reliability of the process and tools for exchanging health

data/information with the outside healthcare system

How well the data/information received are organized or

presented (i.e., the extent to which the format is intuitive to

understand)

The timeliness in receiving the data/information

TOO MUCH DATA, NOT ENOUGH DATA 134

If you can repeat tests/procedures performed at the other

healthcare system and the repeated tests/procedures would

again be reimbursed

a. Comments?

Time Limits for Received Data

NOTE: The timeframes below are intended to reflect the amount of time from when the data/information

were originally created or generated within the other healthcare system (e.g., when the vital signs were

taken, when the discharge summary was written), not the time from when the patient arrives at your

hospital and when the data/information are sent and received there.

7. When receiving health data/information electronically from other healthcare systems,

what would be the general time limits within which you would find the following types

of data/information useful when treating your ED patients?

Within 1

week

Within

1 month

Within 3

months

Within 6

months

Within 1

year

Within

5 years

Within

patient’s

lifetime

Other

(specify

below)

Vital Signs

Laboratory Results

Procedures

Previously administered medications

Radiographs

Electrocardiograms

Problem List

Discharge Summaries

a. Comments?

Your Profession

Your answer to this question will help in categorizing your responses.

8. What is your profession?

Physician

Nurse Practitioner

Physician Assistant

Other ____________________________________________

TOO MUCH DATA, NOT ENOUGH DATA 135

Appendix M

Electronic Data Relevance Survey – Pediatrics

TOO MUCH DATA, NOT ENOUGH DATA 136

Pediatric Survey

Electronic Data Relevance Survey

With the advent of electronic health information exchange across health care systems, there are a

number of questions about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These questions are as follows:

Very little research has been conducted on what types of data/information would be most

important to exchange and the format in which it should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

data. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered went sent from another healthcare system.

Responses to this survey will begin to address these issues. Please answer the following

questions.

Identification of Most Valuable Data When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for data/information exchanged by another health care

system. In addition there is a concern about overwhelming clinicians with too much information

that may not be relevant. Actual data/information to be exchanged (to provide care coordination

and the best care) is in question. It is important to understand out of all existing

data/information, what is most relevant when providing clinical care.

TOO MUCH DATA, NOT ENOUGH DATA 137

1. As a clinician treating pediatric patients from another healthcare system in an

inpatient setting, you understand you could receive any or all data/information from

that system. You already have immediate electronic access to your patients’ allergy,

medication and problem list. Please indicate what additional data/information

would be important for you to have in caring for your hospitalized pediatric

patients.

First, check the boxes next to all data and information types that you think would be

important in treating critically ill pediatric patients and then all hospitalized

pediatric patients.

Second, rank up to 5 data/information types you think would be MOST

REVELANT for you to have in treating your hospitalized pediatric patients, with

“1” being the single most important type of data/information and “5” being the fifth

most important type of data/information.

Data/Information Type Critical

Pediatric

Patients

All

Pediatric

Patients

Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and

telephone number

Physical activity

Plain radiographic images

TOO MUCH DATA, NOT ENOUGH DATA 138

Radiology Tests (report)

CT, endoscopy, nuclear

medicine

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear

medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

Presentation of Received Data

Data and information received from another health care system may be received electronically

by clinicians in the form of a single document or incorporated into the patient’s current

electronic medical record (EMR).

2. In your opinion, should data/information received from another healthcare system

be displayed as a separate electronic document and/or as integrated

data/information within the patient’s current electronic medical record?

p. Received as a separate electronic document?

q. Received as integrated data within the patient’s current EMR?

r. Received as both a separate electronic document and as integrated data within the

patient’s current EMR?

Reconciliation of Received and Existing Data

TOO MUCH DATA, NOT ENOUGH DATA 139

If data/information sent from another healthcare system are integrated within the patient’s

current EMR, there may be a need to reconcile duplications, inconsistencies and contradictions.

Part of this reconciliation will be completed by the computer but human interaction/decision

making will likely be needed as well.

3. Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for

a patient they are currently caring for?

a. Yes

b. No, reconciliation is not necessary.

c. Only as needed on a case by case basis.

4. If such reconciliation of data or information was required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution.

b. The nurse caring for the patient in the receiving institution.

c. The first licensed care giver encountering the patient in the receiving institution.

d. Should be dependent on the data shared (examples would be: pharmacist

reconciling medications, physicians reconciling labs, problems or allergies, and

nursing reconciling patient goals).

e. Other___________________________

TOO MUCH DATA, NOT ENOUGH DATA 140

Trustworthiness of Data

5. How IMPORTANT would each of the following conditions be in deciding if you

trust the data/information received from another healthcare system?

a. The strength of a working relationship with the provider from an outside

healthcare system who ordered or performed the procedure or otherwise prepared

the information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

b. The reputation of the healthcare organization sending the data. Not Important Somewhat Important Neutral Important Very Important Don’t Know

c. Trusting the integrity of the health data exchange. Not Important Somewhat Important Neutral Important Very Important Don’t Know

d. Knowing the type of provider who ordered or performed the procedure or

otherwise prepared the information (specialist vs. generalist). Not Important Somewhat Important Neutral Important Very Important Don’t Know

e. The complexity of the patient. Not Important Somewhat Important Neutral Important Very Important Don’t Know

f. I can repeat the test/procedure and the repeated test/procedure will again be

reimbursed. Not Important Somewhat Important Neutral Important Very Important Don’t Know

g. How well the data is organized in a reliable (i.e. is the format intuitive). Not Important Somewhat Important Neutral Important Very Important Don’t Know

h. I trust the completeness, timeliness and accuracy of the data? Not Important Somewhat Important Neutral Important Very Important Don’t Know

TOO MUCH DATA, NOT ENOUGH DATA 141

Useful Date Ranges of Data

6. When looking for electronic health information exchanged across health care

systems what is the general time limit, by category that data/information is useful.

7. Vital Signs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

8. Laboratory Results

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

9. Procedures

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

10. Previously administered medications

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

TOO MUCH DATA, NOT ENOUGH DATA 142

11. Radiographs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

12. Electrocardiograms

Within 1 week Within 1 month Within 3 months Within 6 months Within 1 year Within 5 years Other

13. Problem List

Within 1 year

Within 2 years

Within 5 years

Complete

Other

14. Discharge Summaries

Within 1 year

Within 2 years

Within 5 years

Complete

Other

8. What is your profession?

Other ____________________________________________

TOO MUCH DATA, NOT ENOUGH DATA 143

Appendix N

Electronic Data Relevance Survey – Primary Care Nursing

TOO MUCH DATA, NOT ENOUGH DATA 144

Primary Care Nursing Survey

Electronic Data Relevance Survey

With the advent of electronic health information exchange across health care systems, there are a

number of questions about the actual data and information that should be exchanged to provide

better coordination and improved patient care. These questions are as follows:

Very little research has been conducted on what types of data/information would be most

important to exchange and the format in which it should be reviewed.

Exchanging data or information across healthcare systems may introduce inconsistent

data. This results in the need to understand who should be expected to resolve such

inconsistencies.

There is limited understanding of the factors that might influence how trustworthy data or

information may be considered went sent from another healthcare system.

Responses to this survey will begin to address these issues. Please answer the following

questions.

Identification of Most Valuable Data When choosing which data or information to send from one institution to another, the question of

relevance becomes a priority. Although no regulations have been defined, legal precedence

indicates that a clinician is responsible for information exchanged by another health care system.

In addition there is a concern about overwhelming clinicians with too much data/information that

may not be relevant. Actual data/information to be exchanged (to provide care coordination and

the best care) is in question. It is important to understand out of all existing data/information,

what is most relevant when providing clinical care.

TOO MUCH DATA, NOT ENOUGH DATA 145

4. As a clinician caring for patients from another healthcare system in an ambulatory

setting you understand you could receive any or all data/information from that

system. You already have immediate electronic access to your patients’ allergy,

medication and problem list. Please indicate what additional data/information

would be important for you to have in caring for your ambulatory patients.

First, check the boxes next to all data and information types that you think would be

important in treating patients in an ambulatory setting.

Second, rank up to 5 data/information types you think would be MOST

REVELANT for you to have in treating your ambulatory patients with “1” being

the single most important type of data/information and “5” being the fifth most

important type of data/information.

Data/Information Type All

Clinic

Patients

Priority

Advanced directives

Behavioral health history

Cognitive abilities

Diet history

Discharge summaries

Electrocardiogram images

Family history

Genome information

Growth records

Health maintenance information

Health insurance

Immunization history

Laboratory results

Medical devices

Mobility/falls risk

Operative summaries

Patient goals

Patient instructions

Pending tests and procedures

Procedure results –

invasive/noninvasive

Progress notes

Provider information and telephone

number

Physical activity

Plain radiographic images

TOO MUCH DATA, NOT ENOUGH DATA 146

Radiology Tests (report)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Radiology Tests (images)

CT, endoscopy, nuclear medicine

scan, MRI, ultrasound

Review of systems

Special needs

Social history

Vital signs

Other (please specify)

Presentation of Received Data

Data and information received from another health care system may be received electronically

by clinicians in the form of a single document or incorporated into the patient’s current

electronic medical record (EMR).

1 In your opinion, should data/information received from another healthcare system

be displayed as a separate electronic document and/or as integrated data within the

patient’s current electronic medical record?

s. Received as a separate electronic document?

t. Received as integrated data within the patient’s current EMR?

u. Received as both a separate electronic document and as integrated data within the

patient’s current EMR?

Reconciliation of Received and Existing Data

If data/information sent from another healthcare system is integrated within the patient’s current

EMR, there may be a need to reconcile duplications, inconsistencies and contradictions. Part of

TOO MUCH DATA, NOT ENOUGH DATA 147

this reconciliation will be completed by the computer but human interaction/decision making

will likely be needed as well.

2 Do you think a clinician should be required to reconcile inconsistencies between

data/information received from another healthcare system and the current EMR for

a patient they are currently caring for?

a. Yes

b. No, reconciliation is not necessary.

c. Only as needed on a case by case basis.

3 If such reconciliation of data or information was required, who do you think should

complete the reconciliation?

a. The physician caring for the patient in the receiving institution.

b. The nurse caring for the patient in the receiving institution.

c. The first licensed care giver encountering the patient in the receiving institution.

d. Should be dependent on the data shared (examples would be: pharmacist

reconciling medications, physicians reconciling labs, problems or allergies, and

nursing reconciling patient goals).

e. Other___________________________

Trustworthiness of Data

TOO MUCH DATA, NOT ENOUGH DATA 148

4 How IMPORTANT would each of the following conditions be in deciding if you

trust the data/information received from another healthcare system?

a. The strength of a working relationship with the provider from an outside

healthcare system who ordered or performed the procedure or otherwise prepared

the information. Not Important Somewhat Important Neutral Important Very Important Don’t Know

b. The reputation of the healthcare organization sending the data. Not Important Somewhat Important Neutral Important Very Important Don’t Know

c. Trusting the integrity of the health data exchange. Not Important Somewhat Important Neutral Important Very Important Don’t Know

d. Knowing the type of provider who ordered or performed the procedure or

otherwise prepared the information (specialist vs. generalist). Not Important Somewhat Important Neutral Important Very Important Don’t Know

e. The complexity of the patient. Not Important Somewhat Important Neutral Important Very Important Don’t Know

f. I can repeat the test/procedure and the repeated test/procedure will again be

reimbursed. Not Important Somewhat Important Neutral Important Very Important Don’t Know

g. How well the data is organized in a reliable (i.e. is the format intuitive). Not Important Somewhat Important Neutral Important Very Important Don’t Know

h. I trust the completeness, timeliness and accuracy of the data? Not Important Somewhat Important Neutral Important Very Important Don’t Know

Useful Date Ranges of Data

TOO MUCH DATA, NOT ENOUGH DATA 149

5 When looking for electronic health information exchanged across health care

systems what is the general time limit, by category that data/information is useful.

15. Vital Signs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

16. Laboratory Results

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

17. Procedures

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

18. Previously administered medications

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

TOO MUCH DATA, NOT ENOUGH DATA 150

19. Radiographs

Within 1 week

Within 1 month

Within 3 months

Within 6 months

Within 1 year

Within 5 years

Other

20. Electrocardiograms

Within 1 week Within 1 month Within 3 months Within 6 months Within 1 year Within 5 years Other

21. Problem List

Within 1 year

Within 2 years

Within 5 years

Complete

Other

22. Discharge Summaries

Within 1 year

Within 2 years

Within 5 years

Complete

Other

8. What is your level of education?

AD □

ADN□

BS□

BSN □

Masters Prepared □

Clinical Area of Expertise ____________________________________

TOO MUCH DATA, NOT ENOUGH DATA 151

Appendix O

Relevant Data by Specialty

TOO MUCH DATA, NOT ENOUGH DATA 152

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%D

isch

arge

sum

mar

ies

Imm

un

izat

ion

his

tory

Lab

ora

tory

res

ult

s

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Pro

vid

er i

nfo

rmat

ion a

nd t

elep

hone…

Pla

in r

adio

gra

phic

im

ages

Gro

wth

rec

ord

s

Pro

cedure

res

ult

s –

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Oper

ativ

e su

mm

arie

s

Die

t his

tory

Pen

din

g t

ests

and p

roce

dure

s

Pro

gre

ss n

ote

s

Beh

avio

ral

hea

lth h

isto

ry

Fam

ily h

isto

ry

Soci

al h

isto

ry

Advan

ced d

irec

tives

Ele

ctro

card

iogra

m i

mag

es

Med

ical

dev

ices

Cognit

ive

abil

itie

s

Physi

cal

acti

vit

y

Rev

iew

of

syst

ems

Spec

ial

nee

ds

Vit

al s

igns

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Mobil

ity/f

alls

ris

k

Pat

ient

goal

s

Hea

lth i

nsu

rance

Pat

ient

inst

ruct

ions

Gen

om

e in

form

atio

n

Relevant Data

Pediatric

Percent

TOO MUCH DATA, NOT ENOUGH DATA 153

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Lab

ora

tory

res

ult

s

Rad

iolo

gy T

ests

- R

eport

s

Dis

char

ge

sum

mar

ies

Advan

ced d

irec

tives

Pro

vid

er i

nfo

rmat

ion

and…

Med

ical

dev

ices

Vit

al s

igns

Pro

cedure

res

ult

s – …

Oper

ativ

e su

mm

arie

s

Rad

iolo

gy T

ests

- I

mag

es

Ele

ctro

card

iogra

m i

mag

es

Pla

in r

adio

gra

phic

im

ages

Pro

gre

ss n

ote

s

Beh

avio

ral

hea

lth h

isto

ry

Hea

lth i

nsu

rance

Imm

uniz

atio

n h

isto

ry

Pen

din

g t

ests

and p

roce

dure

s

Soci

al h

isto

ry

Fam

ily h

isto

ry

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Cognit

ive

abil

itie

s

Pat

ient

goal

s

Spec

ial

nee

ds

Mobil

ity/f

alls

ris

k

Rev

iew

of

syst

ems

Gen

om

e in

form

atio

n

Physi

cal

acti

vit

y

Die

t his

tory

Gro

wth

rec

ord

s

Pat

ient

inst

ruct

ion

s

Relevant Data

Inpatient LIP

TOO MUCH DATA, NOT ENOUGH DATA 154

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Oper

ativ

e su

mm

arie

s

Pro

cedure

res

ult

s –

Pla

in r

adio

gra

phic

im

ages

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Vit

al s

igns

Ele

ctro

card

iogra

m i

mag

es

Imm

uniz

atio

n h

isto

ry

Pro

vid

er i

nfo

rmat

ion a

nd…

Fam

ily h

isto

ry

Lab

ora

tory

res

ult

s

Med

ical

dev

ices

Pro

gre

ss n

ote

s

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Dis

char

ge

sum

mar

ies

Soci

al h

isto

ry

Beh

avio

ral

hea

lth h

isto

ry

Spec

ial

nee

ds

Advan

ced d

irec

tives

Pen

din

g t

ests

and p

roce

dure

s

Mobil

ity/f

alls

ris

k

Rev

iew

of

syst

ems

Gro

wth

rec

ord

s

Hea

lth i

nsu

rance

Cognit

ive

abil

itie

s

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Pat

ient

inst

ruct

ions

Gen

om

e in

form

atio

n

Pat

ien

t goal

s

Physi

cal

acti

vit

y

Die

t his

tory

Relevant Data

ED LIP

TOO MUCH DATA, NOT ENOUGH DATA 155

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Dis

char

ge

sum

mar

ies

Lab

ora

tory

res

ult

s

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Med

ical

dev

ices

Advan

ced d

irec

tives

Pro

cedure

res

ult

s – …

Oper

ativ

e su

mm

arie

s

Imm

uniz

atio

n h

isto

ry

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Ele

ctro

card

iogra

m i

mag

es

Vit

al s

igns

Beh

avio

ral

hea

lth h

isto

ry

Cognit

ive

abil

itie

s

Gro

wth

rec

ord

s

Pen

din

g t

ests

and p

roce

dure

s

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Pro

gre

ss n

ote

s

Pro

vid

er i

nfo

rmat

ion a

nd…

Fam

ily h

isto

ry

Soci

al h

isto

ry

Pla

in r

adio

gra

phic

im

ages

Spec

ial

nee

ds

Rev

iew

of

syst

ems

Gen

om

e in

form

atio

n

Hea

lth i

nsu

rance

Pat

ient

goal

s

Pat

ient

inst

ruct

ion

s

Mobil

ity/f

alls

ris

k

Physi

cal

acti

vit

y

Die

t his

tory

Relevant Data

Outpatient LIP

TOO MUCH DATA, NOT ENOUGH DATA 156

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Lab

ora

tory

res

ult

s

Advan

ced d

irec

tives

Pro

cedure

res

ult

s – …

Imm

uniz

atio

n h

isto

ry

Spec

ial

nee

ds

Med

ical

dev

ices

Pen

din

g t

ests

and p

roce

dure

s

Pro

gre

ss n

ote

s

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Mobil

ity/f

alls

ris

k

Pro

vid

er i

nfo

rmat

ion a

nd…

Hea

lth i

nsu

rance

Dis

char

ge

sum

mar

ies

Oper

ativ

e su

mm

arie

s

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Soci

al h

isto

ry

Beh

avio

ral

hea

lth h

isto

ry

Vit

al s

igns

Cognit

ive

abil

itie

s

Ele

ctro

card

iogra

m i

mag

es

Fam

ily h

isto

ry

Pla

in r

adio

gra

phic

im

ages

Rev

iew

of

syst

ems

Die

t his

tory

Physi

cal

acti

vit

y

Pat

ient

inst

ruct

ions

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Pat

ient

goal

s

Gen

om

e in

form

atio

n

Gro

wth

rec

ord

s

Relevant Data

Inpatient Nursing

TOO MUCH DATA, NOT ENOUGH DATA 157

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Dis

char

ge

sum

mar

ies

Fam

ily h

isto

ry

Lab

ora

tory

res

ult

s

Oper

ativ

e su

mm

arie

s

Pro

gre

ss n

ote

s

Vit

al s

igns

Pro

cedure

res

ult

s – …

Hea

lth i

nsu

rance

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Spec

ial

nee

ds

Pro

vid

er i

nfo

rmat

ion a

nd t

elep

hone…

Rev

iew

of

syst

ems

Advan

ced d

irec

tives

Beh

avio

ral

hea

lth h

isto

ry

Imm

uniz

atio

n h

isto

ry

Cognit

ive

abil

itie

s

Med

ical

dev

ices

Mob

ilit

y/f

alls

ris

k

Pat

ient

inst

ruct

ions

Pen

din

g t

ests

and p

roce

dure

s

Physi

cal

acti

vit

y

Pla

in r

adio

gra

phic

im

ages

Soci

al h

isto

ry

Die

t his

tory

Ele

ctro

card

iogra

m i

mag

es

Gro

wth

rec

ord

s

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Pat

ient

goal

s

Gen

om

e in

form

atio

n

Relevant Data

Outpatient Nursing

TOO MUCH DATA, NOT ENOUGH DATA 158

Appendix P

Relevant Emergent Data

TOO MUCH DATA, NOT ENOUGH DATA 159

0%

5%

10%

15%

20%

25%

30%

Ele

ctro

card

iogra

m i

mag

es

Advan

ced d

irec

tives

Oper

ativ

e su

mm

arie

s

Pro

gre

ss n

ote

s

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Beh

avio

ral

hea

lth h

isto

ry

Die

t his

tory

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Pla

in r

adio

gra

phic

im

ages

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Cognit

ive

abil

itie

s

Pat

ient

goal

s

Pen

din

g t

ests

and p

roce

dure

s

Dis

char

ge

sum

mar

ies

Imm

uniz

atio

n h

isto

ry

Med

ical

dev

ices

Mobil

ity/f

alls

ris

k

Vit

al s

igns

Gen

om

e in

form

atio

n

Hea

lth i

nsu

rance

Lab

ora

tory

res

ult

s

Pat

ient

inst

ruct

ions

Pro

cedure

res

ult

s – …

Pro

vid

er i

nfo

rmat

ion a

nd…

Physi

cal

acti

vit

y

Spec

ial

nee

ds

Fam

ily h

isto

ry

Gro

wth

rec

ord

s

Rev

iew

of

syst

ems

Soci

al h

isto

ry

Emergent Data

Inpatient LIP

TOO MUCH DATA, NOT ENOUGH DATA 160

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Advan

ced d

irec

tives

Dis

char

ge

sum

mar

ies

Lab

ora

tory

res

ult

s

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Vit

al s

igns

Pro

cedure

res

ult

s – …

Pla

in r

adio

gra

phic

im

ages

Oper

ativ

e su

mm

arie

s

Pro

gre

ss n

ote

s

Ele

ctro

card

iogra

m i

mag

es

Imm

uniz

atio

n h

isto

ry

Pro

vid

er i

nfo

rmat

ion a

nd…

Med

ical

dev

ices

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Pen

din

g t

ests

and p

roce

dure

s

Gro

wth

rec

ord

s

Fam

ily h

isto

ry

Die

t his

tory

Cognit

ive

abil

itie

s

Mobil

ity/f

alls

ris

k

Spec

ial

nee

ds

Soci

al h

isto

ry

Beh

avio

ral

hea

lth

his

tory

Gen

om

e in

form

atio

n

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Hea

lth i

nsu

rance

Pat

ient

goal

s

Pat

ient

inst

ruct

ions

Physi

cal

acti

vit

y

Rev

iew

of

syst

ems

Emergent Data

Pediatric LIP

TOO MUCH DATA, NOT ENOUGH DATA 161

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Advan

ced d

irec

tives

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Beh

avio

ral

hea

lth h

isto

ry

Cognit

ive

abil

itie

s

Dis

char

ge

sum

mar

ies

Ele

ctro

card

iogra

m i

mag

es

Pro

vid

er i

nfo

rmat

ion a

nd…

Die

t his

tory

Fam

ily h

isto

ry

Gro

wth

rec

ord

s

Lab

ora

tory

res

ult

s

Mobil

ity/f

alls

ris

k

Oper

ativ

e su

mm

arie

s

Pro

cedure

res

ult

s – …

Pro

gre

ss n

ote

s

Pla

in r

adio

gra

phic

im

ages

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Rev

iew

of

syst

ems

Spec

ial

nee

ds

Gen

om

e in

form

atio

n

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Hea

lth i

nsu

rance

Imm

un

izat

ion

his

tory

Med

ical

dev

ices

Pat

ient

go

als

Pat

ient

inst

ruct

ions

Pen

din

g t

ests

and p

roce

dure

s

Physi

cal

acti

vit

y

Soci

al h

isto

ry

Vit

al s

igns

Emergent Data

ED LIP

TOO MUCH DATA, NOT ENOUGH DATA 162

0%

5%

10%

15%

20%

25%

30%

35%

Ele

ctro

card

iogra

m i

mag

es

Rev

iew

of

syst

ems

Oper

ativ

e su

mm

arie

s

Dis

char

ge

sum

mar

ies

Rad

iolo

gy T

ests

- R

eport

s (C

T,…

Rad

iolo

gy T

ests

- I

mag

es (

CT

,…

Pla

in r

adio

gra

phic

im

ages

Pen

din

g t

ests

and p

roce

dure

s

Pro

gre

ss n

ote

s

Vit

al s

igns

Advan

ced d

irec

tives

Cognit

ive

abil

itie

s

Die

t his

tory

Fam

ily h

isto

ry

Pro

vid

er i

nfo

rmat

ion a

nd…

Spec

ial

nee

ds

Hea

lth m

ainte

nan

ce i

nfo

rmat

ion

Imm

uniz

atio

n h

isto

ry

Med

ical

dev

ices

Pro

cedure

res

ult

s –

Phy

sica

l ac

tivit

y

Lab

ora

tory

res

ult

s

Gro

wth

rec

ord

s

Beh

avio

ral

hea

lth h

isto

ry

Pat

ient

goal

s

Soci

al h

isto

ry

Gen

om

e in

form

atio

n

Pat

ient

inst

ruct

ions

Mobil

ity/f

alls

ris

k

Hea

lth i

nsu

ran

ce

Emergent Data

Inpatient Nursing

TOO MUCH DATA, NOT ENOUGH DATA 163

Appendix Q

Presentation of Exchanged Data – Separate or Integrated

TOO MUCH DATA, NOT ENOUGH DATA 164

0%

5%

10%

15%

20%

25%

30%

35%

40%

Received as a separate electronic

document

Received as integrated

data/information within the

patient's current EMR

Received as both a separate

electronic document and as

integrated data/information

within the patient's current EMR

Presentation of Exchanged Data

for All LIP

TOO MUCH DATA, NOT ENOUGH DATA 165

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Received as a separate electronic

document

Received as integrated

data/information within the patient's

current EMR

Received as both a separate

electronic document and as integrated

data/information within the patient's

current EMR

Presentation of Exchanged Data

for All Nursing

TOO MUCH DATA, NOT ENOUGH DATA 166

Appendix R

Reconciliation of Exchanged Data

TOO MUCH DATA, NOT ENOUGH DATA 167

0%

10%

20%

30%

40%

50%

60%

70%

Yes No, reconciliation is not

necessary

Only as needed on a case by case

basis

Reconciliation of Exchanged Data

LIP

TOO MUCH DATA, NOT ENOUGH DATA 168

0%

10%

20%

30%

40%

50%

60%

Yes No, reconciliation is not

necessary

Only as needed on a case by case

basis

Reconciliation of Exchanged Data

RN

TOO MUCH DATA, NOT ENOUGH DATA 169

0%

10%

20%

30%

40%

50%

60%

The physician caring

for the patient in the

receiving institution

The nurse caring for

the patient in the

receiving institution

The first licensed care

giver encountering the

patient in the receiving

institution

Should be dependent

on the data shared

Other (please specify)

Who should complete reconciliation?

LIP

TOO MUCH DATA, NOT ENOUGH DATA 170

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

The physician caring

for the patient in the

receiving institution

The nurse caring for

the patient in the

receiving institution

The first licensed

care giver

encountering the

patient in the

receiving institution

Should be dependent

on the data shared

Other (please

specify)

Who should complete reconciliation?

RN

TOO MUCH DATA, NOT ENOUGH DATA 171