icsa labs onc hit certification program cy 2016 ... · onc hit certification program cy 2016...

ICSA Labs ONC HIT Certification Program

CY 2016 Surveillance Plan

Document Version1.0 January 11, 2016

www.icsalabs.com

http://www.icsalabs.com/

ONC HIT Certification Program CY 2016 Surveillance Plan

of 23 Document Version 1.0 Copyright 2016 ICSA Labs. All Rights Reserved. Effective date: January 11, 2016 Printed copies are not controlled and may not be official.

ICSA Labs - ONC-Authorized Certification Body (ACB) CY 2016 Surveillance Plan

I. Introduction and Overview

The ICSA Labs ONC HIT Certification Program Surveillance Plan was developed to meet requirements per

ISO/IEC 17065:2012, the Permanent Certification Program Final Rule, 2014 Edition Release 2 Final Rule1,

2015 Edition Final Rule2, and the latest and most relevant ONC Program Policy Guidance documents. In

developing this plan ICSA Labs also considered industry best practices, ongoing feedback offered by

customers, the ONC Approved Accreditor (ANSI), the ONC and suggestions developed collaboratively by

industry associations such as the HIMSS Electronic Health Records Association (EHRA).

II. Surveillance Approach

ICSA Labs conducts regular surveillance on all certified products to ensure continued conformance to

the standards and requirements under which the product was certified – not only in a controlled testing

environment, but also when implemented and used in a production environment, as mandated by ONC.

Surveillance activities are tracked and documented as part of the ISO/IEC 17065:2012 requirements for

an accredited certification body and generally consist of in the field surveillance, proactive and random

surveillance, and reactive surveillance.

In order to gauge ongoing compliance with certification requirements surveillance approaches will

combine both administrative reviews and technical assessments based on random selection, selection

triggered by complaints and/or feedback from users, in-the-field surveillance for a subset of certified

products, retesting, and customer and end-user surveys.

A. In-the-Field

ICSA Labs performs surveillance of certified health IT in-the-field to determine whether the

technology continues to conform to the requirements of its certification once implemented and

in use in a production environment. Certification status in the ONC HIT Certification Program is

subject to ongoing surveillance, including the evaluation of certified capabilities in the field. In-

1 2014 Edition Release 2 Electronic Health Record (EHR) Certification Criteria and the ONC Health IT Certification

Program; Regulatory Flexibilities, Improvements, and Enhanced Health Information Exchange; Final Rule (79 FR 54430) (2014 Edition Release 2 Final Rule). 2 2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base Electronic Health

Record (EHR) Definition, and ONC Health IT Certification Program Modifications final rule (80 FR 62601).



the-field-surveillance is a component of proactive and reactive surveillance (see sections below

on Proactive and Reactive Surveillance for more information). In the case of proactive

surveillance, applicable ONC-prioritized capabilities of a certified product will be assessed. In

the case of reactive surveillance, the testing would be focused on the criteria and associated

criteria where conformity was in question.

1. Methodology

In-the-field testing involves testing a certified product that has been implemented at a

vendor or health IT developer’s customer site (hospital or ambulatory clinic) as opposed

to the testing an ATL typically conducts with a vendor or developer in a controlled

testing environment. The purpose of in-the-field surveillance is to identify deficiencies

that may be difficult to anticipate or that may not become apparent until after certified

health IT is implemented and used in a production environment.

ICSA Labs anticipates unique circumstances and considerations when performing

surveillance in-the-field and will attempt to objectively focus on the certification

capability requirements and conduct an efficient yet thorough assessment of the

capabilities in question. When possible, ICSA Labs will attempt to adhere as closely as

possible to the testing methodologies that an ATL would use to test a product in a

controlled environment with the health IT vendor or developer. This would include the

use of ATL test proctors, ATL-generated test scripts, results reports, worksheets, and

evaluation procedures. In the interest of time and resources for all parties, the

preference will be to conduct the inspection remotely, though on-site testing is possible

when needed. Note ONC requires the use of live data, not test data by default, unless an

exception is granted by ONC.

In addition to methodologies developed by ONC ATLs, in order to determine compliance

ICSA Labs may supplement its in-the-field observations with information including, but

not limited to, that collected from:

User feedback/surveys

Review of past attestation materials/previous testing artifacts

Complaint logs/complaint process/record of complaint resolution

User manuals/training materials

Other vendor or developer provided information

Evaluation of product in controlled testing environment

Evaluation of product at other sites

ICSA Labs will engage and work with both the vendor/developer and end-users of the

product when conducting in-the-field testing as conformance to the certification

requirements is in the best interest of all parties. ICSA Labs will communicate the scope

of the engagement, which criteria will be tested and how, and offer the site opportunity



to allocate resources and carry out any necessary preparations or review of test scripts,

etc. It is assumed the vendor/developer will work with their customer to help

demonstrate the product capabilities, but it is not required.

2. Non-Conformities

If any non-conformities are uncovered during in-the-field surveillance, ICSA Labs will

work with both the vendor/developer and the implementation site to determine the

root cause and corrective action being that it is in the interest of the site to have a

working product and the vendor to maintain their certification status. See Corrective

Action Procedures for more information.

In the event that potential non-conformities are discovered that are not related to

technical capabilities, but are rather issues related to business practices or

implementation practices of the health IT developer that adversely affect the

performance of certified HIT in the field, those issues will be considered against

certification requirements and may impact certification status. Any non-conformities

resulting from discrepancies in the non-disclosure of material information about

limitations or attestation provided with regard to additional types of costs associated

with certified health IT may also impact certification status adversely. See the ICSA Labs

Certification Program Manual section on “Certification Suspension and Withdrawal” for

more information.

3. Findings, Analysis and conclusions

ICSA Labs will document any findings, analysis, and conclusions in a test results

summary that complies with ONC requirements. Additional analysis or commentary that

would not typically be found in the test results summary will be provided in the test

results summary appendix and/or as additional, appended documentation.

B. Randomized and Other Proactive Surveillance

Randomized and other proactive surveillance focuses on ensuring certified Health IT maintains conformity to the ONC prioritized certification criteria, and adherence to guidelines around public facing information about a certified product. Administrative surveillance is conducted regularly on all certified products to ensure vendors and product developers:

Clearly and correctly communicate to prospective consumers and implementers of said technology the mandatory disclosure requirements at 45 CFR § 170.523(k)(1)pertaining to Certified EHR Technology;

Appropriately use the ONC and ICSA Labs certification marks;

Provide and follow internally documented procedures such as a product developer’s complaints resolution process.

Surveillance is carried out by monitoring customer websites, reviewing and approving press releases for ONC HIT Certified products, and periodic review of other publicly available



materials. Additionally, surveys are distributed to certified product developers to collect specific feedback with regards to certified products. ICSA Labs also conducts randomized surveillance of certified products in the field. See the section above on “In-the-field” for more information.

1. Sampling Selection

As mandated per ONC, ICSA Labs will randomly select and perform in-the-field

surveillance of 2% of all ICSA Labs certified products at one or more location where it is

implemented. For each product randomly selected, ICSA Labs will work with the

appropriate personnel, including an ONC-ATL, as appropriate, to perform in the field

surveillance of each capability (where applicable) prioritized by the National

Coordinator.

ICSA Labs will randomly select the products by assigning all ICSA Labs certified vendors a

unique ID and randomly generating candidates for surveillance to meet the required

threshold, using a random number generator tool (such as https://www.random.org/).

ICSA Labs will attempt to include different vendors and product types and domains as

part of the random selection.

As encouraged by ONC, ICSA Labs will assign greater weight to products (or product

vendors) with certifications that are more widely adopted and used so as to increase the

likelihood that the products selected will include at least some products with a large

number of installations and users, thereby increasing the likelihood of discovering and

addressing non-conformities that affect a large number of providers and users. ICSA

Labs will use the most recent ONC and HHS provided data such as the information found

at http://dashboard.healthit.gov/quickstats/quickstats.php to assign weights to more

widely adopted products using the guidelines below.

Number of EPs reporting vendor Weight Assigned

>9,999 1

10,000-29,999 2

30,000-50,000 3

>50,000 4 http://dashboard.healthit.gov/quickstats/pages/FIG-Vendors-of-EHRs-to-Participating-Professionals.php Number of Hospitals reporting vendor Weight Assigned

0-20 1

21-49 2

50-99 3

100-200 4

200+ 5 http://dashboard.healthit.gov/quickstats/pages/FIG-Vendors-of-EHRs-to-Participating-Hospitals.php

http://dashboard.healthit.gov/quickstats/quickstats.php



As an example, in the random selection list, an EP product vendor with less than 10,000

reporting EPs would be listed once, whereas a vendor with >50,000 products would be

listed 4 times, increasing the chance of selection.

Once the product has been selected the vendor will be contacted to provide all of the

sites that the product is implemented at. ICSA Labs will randomly select from the sites

provided using a similar methodology (assigning a random number to each entry

corresponding to a site) for in-the-field testing.

2. Prioritized Capabilities

When a product is selected for randomized surveillance in the field, the assessment will

include the review of any capabilities that are within the scope of the certification

criteria to which the technology is certified, with priority given to criterion prioritized by

the National Coordinator (including the equivalent criteria in the 2015 Edition).

Surveillance of prioritized capabilities may include further assessment by ICSA Labs or

additional evaluation by an ONC ATL. Potential next steps to determine conformance

may include requests to the production site for:

Sample files and generated output to verify conformance to standards;

Corroborating documentation to ensure previously certified functionality has not been compromised;

Verification via live demonstration that the product is conformant in the field, as appropriate.

3. Exclusion and Exhaustion

Once ICSA Labs has selected a certified product and site for randomized surveillance, staff will make efforts to contact the site and schedule the surveillance activities. If ICSA Labs cannot complete in-the-field surveillance of the product at a particular randomly-selected location for reasons beyond its control, ICSA Labs may exclude the location and substitute another location that meets the random selection requirements. Similarly, in the event that ICSA Labs exhausts all potential locations for a particular certified product’s in-the-field testing, ICSA Labs may exclude that product and substitute another randomly selected certified product per ONC Policy Guidance 15-01A.

4. Developer Customer Lists

ICSA Labs is required to obtain and integrate health IT developers’ customer and user lists for randomized sampling and other surveillance activities per ONC. ICSA Labs will collect that information as part of the certification process for new customers, and as part of the product update process and quarterly requests for complaint information. Additionally, ICSA Labs will request updated information from the health IT developer prior to initiating any in-the-field surveillance activities. As an ongoing condition of certification, per ONC, health IT developers must furnish to ICSA Labs upon request, accurate and complete customer lists, user lists, and other information the ONC-ACB determines necessary to carry out its surveillance



responsibilities. Access to accurate customer and user lists is essential to the certification body’s ability to contact users for in-the-field surveillance and to conduct surveys and other activities necessary to obtain and synthesize information about the performance of certified health IT. If a health IT developer refuses to provide this information to ICSA Labs, the refusal may be regarded as a refusal to participate in surveillance under the ONC Health IT Certification Program and institute appropriate procedures, consistent with the ONC-ACB's accreditation to ISO 17065, to suspend or terminate the health IT Module/Complete EHR certification.

C. Reactive Surveillance

Reactive surveillance involves the certification body acting on information concerning ongoing compliance with certification requirements. In order to determine ongoing compliance and what if any corrective actions are necessary to ensure compliance, ICSA Labs may request, obtain, and analyze information including but not limited to the following:

Complaints and other information about certified health IT submitted directly to ICSA Labs by customers or users of ICSA Labs Certified health IT, by the National Coordinator, or by other persons.

Results of collected feedback from surveys or by notification of:

Changes significantly affecting the product’s design or specification, or

Changes in the standards to which compliance of the product is certified, or

Changes in the ownership, structure or management of the customer, if relevant, or in the case of any other information indicating that the product may no longer comply with the requirements of the certification system.

Repeated number of inherited certified status requests (pursuant to 45 CFR 170.545(d) and 45 CFR 170.550(f) – (Products requesting 3 or more inherited certified status requests)

ONC or ONC-ACB identified priority criteria

Reviews of complaint logs and service tickets submitted by Health IT developers,

and other documentation concerning the analysis and resolution of complaints or

issues as reported to the developer. ( “Review of Developer Complaint Processes”

for more information)

Developers’ public and private disclosures regarding certified health IT capabilities,

including any discrepancies or failures to disclose known material information about

certified capabilities, as required by § 170.523(k)(1). (See section IV A, “Surveillance

of Developers’ Disclosures” for more information)

Information from publicly available sources (e.g., a developer’s website or user forums).

Other facts and circumstances of which ICSA Labs is aware.

In the event ICSA Labs is contacted either by ONC or by a customer in possession of HIT certified by ICSA Labs with complaints about a product’s ability to comply with the certification criteria,



ICSA Labs will notify the vendor/developer and investigate the complaint to take appropriate action. A record of all complaints received, the action taken and its effectiveness will be maintained. All nonconformities identified during surveillance activities will be communicated to the customer (See Section V Corrective Action Procedures for more information). In order to determine whether the technology remains in conformance, ICSA Labs will take into account all information collected including the volume, substance, and credibility of any complaints about the certified product, as well as the response from the vendor/developer (including past submissions and the results of previous surveys and surveillance artifacts.

Further assessment by ICSA Labs or additional evaluation by an ONC ATL may be potential next steps to determine conformance by requesting:

Sample files and generated output to verify conformance to standards

Corroborating documentation to ensure previously certified functionality has not been compromised

Verification via live demonstration that the product is conformant in the field, as appropriate;

The customer is provided an opportunity to correct the nonconformities before the issue is escalated. See the ICSA Labs ONC HIT Certification Program Manual’s section on “Certification Suspension and Withdrawal” and Section V of this document, “Corrective Action Procedures” for more information. Note: products that have been rebranded may be candidates for surveillance testing to ensure that certified functionality remains intact and in accordance to the original product certified.

III. Prioritized Elements

Four categories of capabilities for CY16 have been identified in the ONC Program Policy Guidance #15-01A: Interoperability and Information Exchange, Safety-related capabilities, Security capabilities, and Population Management capabilities. The below are representative of the specific 2014 Edition criteria and are further enumerated within these four categories as elements that are to be prioritized throughout the calendar year as surveillance activities are conducted. Related 2015 Edition criteria will also be prioritized for surveillance of 2015 Edition certified products.

Interoperability and Information Exchange a. 170.314(b)(1) Transitions of care – receive, display and incorporate transition of

care/referral summaries b. 170.314(b)(2) Transitions of care–create and transmit transition of care/referral

summaries c. 170.314(b)(7) – Data portability d. 170.314(b)(8) – Optional – transitions of care e. 170.314(e)(1) (View, download, and transmit to 3rd party



f. *170.314(h)(1) Optional - Transmit - Applicability Statement for Secure Health g. *170.314(h)(2) Optional - Transmit - Applicability Statement for Secure Health Transport

and XDR/XDM for Direct Messaging

Safety-related capabilities a. 170.314(a)(2) Drug-drug, drug-allergy interaction checks b. 170.314(a)(8) Clinical decision support c. 170.314(a)(16) Inpatient setting only – electronic medication administration record d. 170.314(b)(4) Clinical information reconciliation (For this certification criterion ONC

recommends focusing on medication reconciliation) e. *170.314(b)(9) – Optional Clinical information reconciliation and incorporation

Security capabilities a. 170.314(d)(2) Auditable Events and Tamper-Resistance b. 170.314(d)(7) End-User Device Encryption

Population management capabilities a. 170.314(c)(2) Clinical quality measures – import and calculate (For this specific criterion,

ONC recommends focusing on the calculation of one or more specific clinical quality measures)

In addition to these prioritized capabilities, ONC also considers the following elements a priority for

surveillance in CY16:

The assessment of developers’ disclosures, as required by 45 CFR 170.523(k) and the evaluation

of potential non-conformities resulting from the failure to disclose material information about

limitations or additional types of costs associated with certified health IT

The assessment of potential non-conformities resulting from implementation or business practices of a health IT developer that could affect the performance of certified capabilities in the field.

The adequacy of developers’ user complaint processes, including customer complaint logs,

consistent with ISO/IEC 17065 § 4.1.2.2 (j)

Appropriate use of the ONC Certification Mark

Additional categories prioritized by ICSA Labs for surveillance activities may include:

Interoperability and Information Exchange - 170.314(h)(3) Transmit - SOAP Transport and Security Specification and XDR/XDM for Direct Messaging

Safety-related capabilities – 170.314(a)(1) Computerized provider order entry, *170.314(a)(18) Computerized provider order entry – medications, *170.314(a)(19) Computerized provider order entry – laboratory and *170.314(a)(20) Computerized provider order entry – diagnostic imaging and exchange capabilities

Security capabilities- 170.314(d)(1) Authentication, access control, and authorization and 170.314(d)(3) Audit Reports

See Appendix A - Customer Surveys for details on how ICSA Labs will address the elements of surveillance prioritized by the National Coordinator. * 2014 Edition Release 2 criteria



IV. Transparency and Disclosure Requirements

The transparency and disclosure requirements adopted in the 2015 Edition Final Rule, and prioritized in

this surveillance plan for CY 2016, are documented in the ICSA Labs ONC HIT Certification Program

Manual and will be reinforced in messaging to HIT product developers via email, program webinars, the

ICSA Labs website, and other various forms of communication to ensure proper understanding and

conformance.

Product developers will be required to adhere to the transparency and disclosure requirements adopted

in the 2015 Edition Final Rule 45 CFR § 170.523(k) which states:

A Health IT developer must conspicuously include the following on its Web site and in all marketing

materials, communications statements, and other assertions related to the Complete EHR or Health IT

Module's certification:

“This [Complete EHR or Health IT Module] is [specify Edition of EHR certification criteria] compliant and has been certified by an ONC-ACB in accordance with the applicable certification criteria adopted by the Secretary of Health and Human Services. This certification does not represent an endorsement by the U.S. Department of Health and Human Services.”

And

a. The vendor name b. The date certified c. The product name and version d. The unique certification number or other specific product identification e. Where applicable, the certification criterion or criteria to which each EHR module has been

tested and certified f. The clinical quality measures to which a complete EHR or EHR module has been tested and

certified g. And where applicable, any additional software a complete EHR or EHR module relied upon

to demonstrate its compliance with a certification criterion or criteria adopted by the Secretary

h. And where applicable, any additional types of costs that a user may be required to pay to implement or use the Complete EHR or Health IT Module's capabilities, whether to meet meaningful use objectives and measures or to achieve any other use within the scope of the health IT's certification. (Examples given include: fixed, recurring, transaction-based, or otherwise that are imposed by a health IT developer (or any third-party from whom the developer purchases, licenses, or obtains any technology, products, or services in connection with its certified health IT) to purchase, license, implement, maintain, upgrade, use, or otherwise enable and support the use of capabilities to which health IT is certified; or in



connection with any data generated in the course of using any capability to which health IT is certified.)

i. And where applicable, any limitations (whether by contract or otherwise) that a user may encounter in the course of implementing and using the Complete EHR or Health IT Module's capabilities, whether to meet meaningful use objectives and measures or to achieve any other use within the scope of the health IT's certification. (Examples given include, but not limited to technical or practical limitations of technology or its capabilities, that could prevent or impair the successful implementation, configuration, customization, maintenance, support, or use of any capabilities to which technology is certified; or that could prevent or limit the use, exchange, or portability of any data generated in the course of using any capability to which technology is certified.)

A developer may satisfy the requirement to disclose the information required by § 170.523(k)(1) in its

marketing materials, communications statements, and other assertions related to a Complete EHR or

Health IT Module's certification by providing an abbreviated disclaimer, appropriate to the material and

medium, provided the disclaimer is accompanied by a hyperlink to the complete disclosure on the

developer's website.

Where a hyperlink is not feasible (for example, in non-visual media), the developer may use another

appropriate method to direct the recipient of the marketing material, communication, or assertion to

the complete disclosure on its website.

A. Surveillance of Developers’ Disclosures

As noted in section B (Randomized and Other Proactive Surveillance), ICSA Labs will proactively select health IT developers and products for surveillance to ensure a developers’ compliance with the mandatory disclosure requirements found in 45 CFR § 170.523(k)(1). Surveillance is carried out by monitoring customer websites, reviewing and approving, press releases for ONC HIT Certified products, and periodic reviews of other publicly available materials. During surveillance activities, ICSA Labs will review a health IT product developers public materials (i.e. websites, press releases, marketing materials, etc.) and assess whether the information displayed matches the information attested to on the product developer’s registration form. As noted in Policy Guidance #15-01A, developers are not required to disclose information of which they are not and could not reasonably be aware, nor to account for every conceivable type of cost or implementation hurdle that a customer may encounter. “Developers are required, however, to describe with particularity the nature, magnitude, and extent of the limitations or types of costs.” A developer’s disclosure possesses the requisite particularity if it contains sufficient information and detail from which a reasonable person under the circumstances would, without special effort, be able to reasonably identify the specific limitations he may encounter and reasonably understand the potential costs he may incur in the course of implementing and using capabilities for any purpose within the scope of the health IT's certification. Any discrepancies or obvious issues with the information disclosed will be communicated to the product developer with an opportunity for remediation (See Corrective Action Procedures). The



customer is provided an opportunity to correct the nonconformities before the issue is escalated. See the ICSA Labs ONC HIT Certification Program Manual’s section on Certification Suspension and Withdrawal for more information.

B. Attestation Requirement

As a condition of certification, health IT developers must make one of the following attestations:

In the affirmative: In support of enhanced marketplace transparency and visibility into the costs and performance of certified health IT products and services, and the business practices of health IT developers, [Developer Name] hereby attests that it will provide in a timely manner, in plain writing, and in a manner calculated to inform, any part (including all) of the information required to be disclosed under 45 CFR §170.523(k)(1)under the following circumstances: In the affirmative: To all persons who request such information. To all persons who request or receive a quotation, estimate, description of services, or other assertion or information from [Developer Name] in connection with any certified health IT or any capabilities thereof. To all customers prior to providing or entering into any agreement to provide any certified health IT or related product or service (including subsequent updates, add-ons, or additional products or services during the course of an on-going agreement). -OR –

In the negative:

[Developer Name] hereby attests that it has been asked to make the voluntary attestation described by 45 CFR § 170.523(k)(2)(i)in support of enhanced marketplace transparency and visibility into the costs and performances of certified health IT products and services, and the business practices of health IT developers.

[Developer Name] hereby declines to make such attestation at this time.

A developers’ adherence to their attestations is voluntary, however ICSA Labs is required to include the developers’ attestations in the hyperlink submitted to the National Coordinator for inclusion in the CHPL so that the public can determine which developers have attested to taking the additional actions to promote transparency of their technologies and business practices. ONC notes that a developer’s attestation under 45 CFR § 170.523(k)(2) does not broaden or change the scope of the information a developer is required to disclose under 45 CFR § 170.523(k)(1).



ICSA Labs administers the transparency attestation requirement by including it as part of new 2015 Edition certification registrations, the update process for previously certified products, and as part of the follow up that is required for quarterly reporting information.

V. Corrective Action Procedures

If a certified product is found to be non-conformant to the requirements of its certification, ICSA Labs

will notify the vendor/developer of any findings. The vendor/developer is required to submit to ICSA

Labs a proposed corrective action plan (CAP) for the applicable certification criterion, certification

criteria, or certification requirement. Related information will also be publicly reported to the ONC’s

open data CHPL as required by ONC.

A. Corrective Action Plan Elements

To further clarify, per ONC a CAP is required under §556 any time an ACB finds that a product or a developer is non-compliant with any certification criterion or any other requirement of certification, including the transparency and disclosure requirements.

Corrective action plans submitted by a developer to an ONC-ACB must include the following elements:

i. A description of the identified non-conformities or deficiencies;

ii. An assessment of how widespread or isolated the identified non-conformities or

deficiencies may be across all of the developer’s customers and users of the certified

technology;

iii. How the developer will address the identified non-conformities or deficiencies, both at

the locations under which surveillance occurred and for all other potentially affected

customers and users;

iv. How the developer will ensure that all affected and potentially affected customers and

users are alerted to the identified non-conformities or deficiencies, including a detailed

description of how the developer will assess the scope and impact of the problem,

including identifying all potentially affected customers; how the developer will promptly

ensure that all potentially affected customers are notified of the problem and plan for

resolution; how and when the developer will resolve issues for individual affected

customers; and how the developer will ensure that all issues are in fact resolved.

v. The timeframe under which corrective action will be completed

vi. An attestation by the developer that it has completed all elements of the approved

corrective action plan, or the target dates of completion.

B. Corrective Action Plan Submission and Review

The CAP must be provided to ICSA Labs within 30 days of notification. A non-response may be grounds for further punitive action. Extensions may be granted on a case by case basis. ICSA



Labs will review the CAP within 10 business days and make a determination as to whether the plan will be approved, needs any revisions, or is altogether rejected. The determination will be based on a review of the thoroughness and completeness of the submitted CAP based on the CAP requirements outlined above, and whether the timelines and proposed corrective actions provide confidence to the certification body that the product is in conformance or will be by a certain target date. Depending on the degree and scope of the non-conformities, it is still possible that the certification may be suspended or withdrawn. See the ICSA Labs Certification Program Manual for more information.

C. Corrective Action Plan Submission and Review

Once the CAP is approved, ICSA Labs will follow up within 30 days to ensure adherence to the approved corrective action plan and in order to verify that requirements of the corrective action plan have been completed. ICSA Labs will request an attestation that the vendor/developer has completed all required elements of the approved correction plan. ICSA Labs may conduct additional follow up with the vendor/developer, as well as end-users, to verify the attestation and ensure that the corrective actions have been implemented for all affected and potentially affected customers and users. Similarly, the product may be a candidate for future surveillance.

VI. Submission of Corrective Action and Surveillance Information

A. Submission of Corrective Action Information

Any non-conformity/non-compliance would be reported to the CHPL immediately (under the weekly reporting requirement) and then updated to include the CAP and the activities surrounding execution of the CAP. At any point during surveillance ICSA Labs may notify ONC of its activities, especially if there are concerns about safety, information blocking, etc. As part of ICSA Labs’ weekly reporting requirement to ONC, the following corrective action information would be submitted to ONC for inclusion in the CHPL:

The CHPL Product number of each Complete EHR or Health IT Module that failed to conform to its certification and for which corrective action was instituted under 45 CFR § 170.556.

The specific certification requirements to which the technology failed to conform.

A summary of the deficiency or deficiencies identified by the ICSA Labs as the basis for its determination of non-conformity.

When available, the health IT developer's explanation of the deficiency or deficiencies.

The dates surveillance was initiated and completed.

The results of randomized surveillance, including pass rate for each criterion in instances where the Complete EHR or EHR Module is evaluated at more than one location.

The number of sites that were used in randomized surveillance.



The date of the ONC-ACB's determination of non-conformity.

The date on which the ONC-ACB approved a corrective action plan.

The date corrective action began (effective date of approved corrective action plan).

The date by which corrective action must be completed (as specified by the approved corrective action plan).

The date corrective action was completed.

A description of the resolution of the non-conformity or non-conformities.

B. Submission of Surveillance Information

1. Surveillance Narratives and Corroborating Documentation

ICSA Labs reports surveillance results to the National Coordinator on a rolling basis (i.e., no less frequently than quarterly) throughout CY16. When submitting annual surveillance results, ICSA Labs will identify each instance of surveillance performed during CY16 and the results of that surveillance, including a detailed narrative and corroborating documentation and evidence to support any determinations or findings, including:

Each certified Complete EHR or Health IT Module (identified by its CHPL product

ID), each certification criterion, and each certification program requirement that

was subject to surveillance.

The type of surveillance (proactive, reactive) initiated in each case.

The grounds for initiating surveillance and for deciding whether or not to

evaluate the certified health IT in the field.

Whether or not the surveillance activities confirmed a non-conformity.

The substantial factors that, in the certification body’s assessment, caused or

contributed to the apparent non conformity (e.g., implementation problem,

user error, limitations on the use of capabilities in the field, a failure to disclose

known material information, etc.).

The steps the certification body took to obtain and analyze evidence and to arrive at its conclusions.

When documenting the surveillance activities, ICSA Labs will include the following information in the report:

Methodologies and techniques employed to determine whether to initiate

surveillance, what type of surveillance to perform (e.g., in-the-field surveillance

or other forms of surveillance), and how to evaluate suspected non-

conformities.

How the certification body engaged and worked with developers and end-users

to analyze and determine the causes of any suspected non-conformities and

related deficiencies.



How the certification body evaluated any non-conformities resulting from

implementation or business practices of the health IT developer which then

potentially affected the performance of certified capabilities in the field.

How the certification body evaluated any potential non-conformities resulting

from the non-disclosure of material information about limitations or additional

types of costs associated with certified health IT.

2. Review of Developer Complaint Processes

Vendors and product developers are required to provide details of their complaint handling process for complaints relating to the scope of functionality certified in the ONC HIT Certification Program on an annual basis. The Complaint handling process will include details as to how customers can report defects or make complaints about the product including:

Methods customers can use to the report issue,

The process used to track the issue,

The process used to analyze the issue

How issues are resolved

How customers are subsequently notified

All product developers must also:

Provide ICSA Labs with documentation outlining internal complaint handling processes;

Maintain a record of all customer complaints related to a product's compliance with the ONC HIT Certification criteria against which it was tested;

Retain a log of actions taken in response to such complaints; and Provide customer complaint records and associated actions to ICSA Labs on a

quarterly basis.

The complaint handling processes of any developer whose technology was subject to surveillance during the applicable calendar year will be reviewed by ICSA Labs to determine whether the appropriate actions were taken as reported in their complaint handling processes. If the issues were not properly addressed, ICSA Labs will follow up, as necessary with the vendor/developer and end user as a next step and reported to ONC. ICSA Labs will also evaluate the frequency of complaints made to the developer that were associated with the prioritized surveillance elements noted in Section III – Prioritized Elements.



C. Due Process and Exclusion of Certain Sensitive Information

1. Meaningful Opportunity for Input and Comment on ONC-ACB Findings

Prior to making a non-conformity or other determination and prior to submitting

surveillance results (and, where applicable, corrective action information) to the

National Coordinator, ICSA Labs will attempt to conduct a thorough and complete

review of all relevant facts and circumstances including a review of all findings and an

opportunity to the developer to explain any deficiencies identified by the certification

body or complaint.

2. Exclusion of Certain Information from Submission of Corrective Action

Information and Surveillance Results

In order to safeguard confidentiality, prior to submitting corrective action information

and surveillance results to the National Coordinator, ICSA Labs will conduct a review to

ensure the exclusion of information that would identify any health IT developer

customer or user, any health care provider, location, or practice site that participated in

or was subject to surveillance, or any person who submitted a complaint or other

information to a health IT developer or ONC-ACB. This review would include de-

identifying any names or locations in reports or narratives, as well as any testing

artifacts.

3. Exclusion of Certain Information from Submission of Corrective Action

Information

With respect to the submission of corrective action information to the National

Coordinator for inclusion in the CHPL, ICSA Labs will not submit any information that is

in fact legally privileged or protected from disclosure and that therefore should not be

listed on a publicly available website. ICSA Labs may also implement other appropriate

safeguards, as necessary; to protect information that, while not legally protected from

disclosure, ICSA Labs believes should not be reported to a publicly available website. As

intended by ONC, any such safeguards will be narrowly tailored and consistent with the

goal of promoting the greatest possible degree of transparency with respect to certified

health IT and the business practices of certified health IT developers, especially the

disclosure of material information about limitations and types of costs associated with

certified health IT.

VII. Public Accountability

Please note that the ONC recommends that all ONC-ACBs make their annual surveillance plans and

surveillance results publicly available after submission to ONC in an effort to strengthen the value

stakeholders receive from the ONC HIT Certification Program. It is ICSA Labs intent to publish

surveillance plans and results publicly.



Appendix A – Customer Surveys

ONC’s Program Policy Guidance for Surveillance requires ICSA Labs to systematically obtain and synthesize feedback from users of certified EHR technology to determine if certain capabilities should be evaluated with the EHR technology developer or with the user in the field, or both. As outlined in the surveillance approach section, certified vendors will be surveyed periodically so that ICSA Labs may collect information with regards to the certified product and any potential changes, as well as information with regards to the user base, as appropriate, to sample the performance of products in the field. A subset of randomly selected certified vendors and product developers that are surveyed will be asked to provide a list of users of their 2014 or 2015 Edition certified version of the software. (Ideally, these users should have already successfully attested to meaningful use in a previous year to assure that they are using the functionality to be surveyed.) ICSA Labs will then randomly sample from the collective group of users a minimum of three users. Each user will be surveyed for the use of at least one item from the four domains of exchange capabilities, patient safety capabilities, security capabilities, and population management capabilities. For these four categories, ICSA Labs will cumulatively address the nine capabilities specified within these four categories, i.e., the ACB should assess each of these nine capabilities at least once in CY 2016. The survey should be spread out over various technologies. See Section III, Surveillance Elements – Priorities, for details on the domains and specific capabilities. Below is a subset of questions asked of users of 2014 Edition Certified Health IT for the purposes of the survey. These are representative of the questions that would be asked for 2015 Edition certified software. See Corrective Action Procedures, for a summary of how any issues or findings that may be uncovered during surveillance would be handled and reported.

1. Questions for Exchange capabilities a. 170.314(b)(1) Transitions of care – receive, display and incorporate transition of

care/referral summaries

Does your CEHRT allow you to receive a summary of care document?

If no, is this because your CEHRT does not support that functionality?

If yes, can you display a summary of care document and incorporate summary care data?

b. 170.314(b)(2) Transitions of care–create and transmit transition of care/referral summaries

Are you creating a summary of care document (CCDA) when referring a patient to another provider and transferring them to another care provider or care setting?

If no, is this because your CEHRT does not support that functionality? (Collect details sufficient to verify reasons they are not doing this.)

If yes, are you able to transmit all CCDAs that you would like electronically?



c. 170.314(b)(7) –Data Portability

Does your CEHRT allow you to provide the ability for batch export of patient data in CCDA format?

If no, what are the problems? (Collect details sufficient to identify if the problem is caused by a defect in the CEHRT.)

d. 170.314(b)(8) – Optional – transitions of care

Does your CEHRT allow you to access your patient’s record?

If yes, are you able to create a summary of care record? If no, is this because your CEHRT does not support that functionality?

e. 170.314(e)(1) View, download, and transmit to 3rd party

Does your CEHRT allow you to provide your patients with the ability to view their medical information electronically?

Are your patients able to reliably download information?


Are they able to send it to a 3rd party? f. *170.314(h)(1-3) Optional Secure Health Transport criteria (as applicable)

Does your CEHRT allow you to securely transmit health information to patients or a 3rd party?


2. Questions for Safety-related capabilities

a. 170.314(a)(2) Drug-drug, drug-allergy interaction checks

Are your providers able to consistently see drug-allergy interaction alerts when appropriate?


Are your providers able to see drug-drug interaction alerts?

If no, what are the problems? (Collect details sufficient to identify if the problem is caused by a defect in the CEHRT. If a problem is reported with this measure, investigate whether interactions have been turned off or set to a lower level of interaction than the respondent expects.)

b. 170.314(a)(8) Clinical decision support

Are your providers able to select one or more electronic clinical decision support interventions (in addition to drug-drug and drug-allergy contraindication checking) based on each one and at least one combination of the following data: Problem list, medication list, medication allergy list, demographics, laboratory test and values/results and vital signs?


c. 170.314(a)(16) Inpatient setting only – electronic medication administration record

Are your providers asked to verify the following before administering medication(s): Right patient, Right medication, Right dose, Right route, Right time and Right documentation?




d. 170.314(b)(4) Clinical information reconciliation/*170.314(b)(9) Clinical information reconciliation incorporation (For this certification criterion ONC recommends focusing on medication reconciliation) Verify if they are using the CEHRT for reconciliation of medications received electronically from outside sources. If they are only doing medication reconciliation with paper charts, skip this question.

Are you using the CEHRT for reconciliation of medications received electronically from outside sources?

Identify/describe what sources for external information are being used (e.g., specific eRx network, medication history, CCDA, HIE)?

Are there problems with the accuracy or completeness of the CEHRT consolidated medication list?

If yes, what are the problems? (Collect details sufficient to identify if the problem is caused by a defect in the CEHRT.)

e. *170.314(b)(9) – Optional Clinical information reconciliation and incorporation

Does the CEHRT allow you to match a patient transition of care (TOC)/referral summary to an existing patient?


Does the CEHRT allow you to electronically create a single reconciled list of data elements in a patient’s active medication list?


Does the CEHRT allow you to electronically create a single reconciled list of data elements in a patient’s active problem list?


Does the CEHRT allow you to electronically create a single reconciled list of data elements in a patient’s active medication list?


Does the CEHRT allow you to incorporate all of this information into the EHR?


3. Questions for Security capabilities a. 170.314(d)(2) Auditable Events and Tamper-Resistance. The HIPAA security risk analysis

measure requires that CEHRT audit log files be reviewed periodically to identify attempted access or other inappropriate activities in the system.

Are you able to answer technical questions about the auditing functionality of your CEHRT?

If not, is there someone we can contact to discuss this with?

Do you have access to the audit logs?

Have you identified any problems with the auditing capabilities of your CEHRT?

Are the log files accurate?



Does the system prevent you from accidentally or deliberately changing the audit logs?

b. 170.314(d)(7) End-User Device Encryption. This question should only be asked of users who have technical knowledge and expertise. The certification requirement states “EHR technology that is designed to locally store electronic health information on end-user devices must encrypt the electronic health information stored on such devices after use of EHR technology on those devices stops.“ Note: This question only applies to files with patient sensitive information generated by the EHR product automatically and not ones where a user deliberately saves a file to the local machine.

Are all CEHRT created files that are automatically generated by the CEHRT on local computers encrypted?

4. Questions for Population management capabilities a. 170.314(c) (2) Clinical quality measures – import and calculate (For this specific criterion,

ONC recommends focusing on the calculation of one or more specific clinical quality measures)

Is your product certified for clinical quality measure import? Are you using your CEHRT module to import clinical quality measures or quality measure data?

If yes, are there any problems with this CEHRT functionality? (Collect details sufficient to identify if the problem is caused by a defect in the CEHRT.)

Are you calculating any of these measures: Controlling High Blood Pressure: Preventive Care and Screening; Tobacco Use: Screening and Cessation Intervention; Documentation of Current Medications in the Medical Record; Childhood Immunization Status?

Is your CEHRT able to accurately calculate these clinical quality measures?

If they do not seem accurate, have you attempted to verify them?

What are the exact problems? (Collect details sufficient to identify if the problem is caused by a defect in the CEHRT.)

Are there some measures that your CEHRT was certified for that cannot be calculated and others that can?

5. Questions related to EHR Technology Developer’s Complaint Processes Confirm that respondent has authority to submit a support request. If they do not, the question should be forwarded to someone who does. Note complaints must be specific in scope to functionality certified under the ONC HIT Certification Program and the relevant ONC Test Methods.

Are you authorized to submit a support request? “If no, please provide the name and email address for an individual who is authorized to submit a support request. ICSA Labs will send a link via email to ensure this section of the surveillance survey is completed.

Have you ever reported a serious defect or made complaints to your vendor about the software’s capability to perform functionality it was certified for?

Does your vendor have a process to track and manage complaints? If so, detail it.

If applicable, how did you report and track support requests or complaints with your vendor? Telephone, fax, web portal, other?



If you submitted a complaint, were you notified of resolution of your complaint?

Does your ONC Certified HIT vendor notify you of any defects found which could affect patient safety?

If yes, how do they notify you of these patient safety issues?

Have you ever reported a defect to your vendor that was related to the certified technology that caused actual patient harm?

If yes, please describe sufficiently to follow-up with the vendor, leaving out any PHI.

May ICSA Labs contact you for additional information if necessary? * 2014 Edition Release 2 criteria

icsa labs onc hit certification program cy 2016 ... · onc hit certification program cy 2016...

Documents