king county behavioral health organization · bhos administer services by contracting with...
Post on 07-Aug-2020
1 Views
Preview:
TRANSCRIPT
King County Behavioral Health Organization External Quality Review Report
Department of Social and Health Services
Division of Behavioral Health and Recovery
December 2017
Qualis Health prepared this report under contract with the Washington State Department of Social and
Health Services Division of Behavioral Health and Recovery (Contract No. 1534-28375).
As Washington’s Medicaid external quality review organization (EQRO), Qualis Health provides external
quality review and supports quality improvement for enrollees of Washington Apple Health managed care
programs and managed behavioral healthcare services. Our work supports the Washington State Health
Care Authority (HCA) and Department of Social and Health Services (DSHS) Division of Behavioral
Health and Recovery (DBHR).
This report has been produced in support of the DSHS DBHR, documenting the results of external review
of the state’s Behavioral Health Organizations (BHOs). Our review was conducted by Ricci Rimpau, RN,
BS, CPHQ, CHC, Operations Manager; Crystal Didier, M.Ed, Clinical Quality Specialist; Wesley Jordan,
MS, Clinical Quality Specialist; Lisa Warren, Clinical Quality Specialist; and Joe Galvan, Project
Coordinator.
Qualis Health is one of the nation’s leading population health management organizations, and a leader in
improving care delivery and patient outcomes, working with clients throughout the public and private
sectors to advance the quality, efficiency, and value of healthcare for millions of Americans every day. We
deliver solutions to ensure that our partners transform the care they provide, with a focus on process
improvement, care management, and effective use of health information technology.
For more information, visit us online at www.QualisHealth.org/WAEQRO.
PO Box 33400
Seattle, Washington 98133-0400
Toll-Free: (800) 949-7536
Office: (206) 364-9700
Table of Contents
Table of Contents .......................................................................................................................................... 3
Executive Summary ...................................................................................................................................... 5
Introduction ...................................................................................................................................... 5
KCBHO Transition Implementation Status....................................................................................... 6
Description of EQR Activities ........................................................................................................... 7
Summary of Results ......................................................................................................................... 7
Compliance with Regulatory and Contractual Standards ........................................................................... 13
Compliance Scoring ....................................................................................................................... 13
Summary of Compliance Review Results ...................................................................................... 14
Section 1: Availability of Services .................................................................................................. 19
Section 2: Coordination and Continuity of Care ............................................................................. 25
Section 3: Coverage and Authorization of Services ...................................................................... 32
Section 4: Provider Selection ......................................................................................................... 35
Section 5: Subcontractual Relationships and Delegation .............................................................. 38
Section 6: Practice Guidelines ....................................................................................................... 40
Section 7: Health Information Systems .......................................................................................... 42
Section 8: Quality Assessment and Performance Improvement Program .................................... 44
Performance Improvement Project (PIP) Validation ................................................................................... 47
PIP Scoring .................................................................................................................................... 47
PIP Validity and Reliability ............................................................................................................. 48
PIP Validation Results: Clinical PIP ............................................................................................... 49
PIP Validation Results: Non-Clinical Children’s PIP ...................................................................... 61
PIP Validation Results: Substance Use Disorder (SUD) PIP ........................................................ 72
Information Systems Capabilities Assessment (ISCA) ............................................................................... 81
ISCA Methodology ......................................................................................................................... 81
Summary of Results ....................................................................................................................... 82
ISCA Section Details ...................................................................................................................... 84
Encounter Data Validation (EDV) ............................................................................................................... 99
Validating BHO EDV Procedures ................................................................................................... 99
Qualis Health Encounter Data Validation..................................................................................... 100
Electronic Data Checks ................................................................................................................ 100
On-site Clinical Record Review ................................................................................................... 100
Sampling Methodology................................................................................................................. 101
Scoring Criteria ............................................................................................................................ 102
KCBHO EDV Procedures............................................................................................................. 102
Qualis Health Encounter Data Validation..................................................................................... 104
Electronic Data Checks ................................................................................................................ 105
Golden Thread Focus Study ..................................................................................................................... 111
Summary of Results ..................................................................................................................... 111
Appendix A: All Recommendations Requiring Corrective Action Plans (CAPs) ....................................... 115
Appendix B: Review of Previous-Year Recommendations Requiring Corrective Action Plans (CAPs) ... 120
Appendix C: Readiness Assessment Follow-up ....................................................................................... 129
Appendix D: Coordination of Care Clinical Chart Review ......................................................................... 132
Appendix E: Regulatory and Contractual Standards ................................................................................ 135
Appendix F: Acronyms .............................................................................................................................. 163
5 Executive Summary
Executive Summary
Introduction
The Washington State Department of Social and Health Services (DSHS) Division of Behavioral Health
and Recovery (DBHR) contracts with nine Behavioral Health Organizations (BHOs) throughout the state
of Washington to provide comprehensive and culturally appropriate mental health and substance use
disorder (SUD) treatment services for adults, children, and their families. BHOs administer services by
contracting with behavioral health agencies (BHAs)—community mental health agencies, SUD treatment
providers, and private nonprofit agencies—to provide mental health and SUD services and treatment. The
BHOs are accountable for ensuring that services are delivered in an integrated manner that complies with
legal, contractual, and regulatory standards for effective care.
As the State’s external quality review organization (EQRO), Qualis Health is contracted to conduct a
yearly assessment of the accessibility, timeliness, and quality of managed mental health and SUD
treatment services provided by BHOs to Medicaid enrollees.
This report presents the results of the 2017 external quality review for King County BHO (KCBHO). In
addition to managing a network of mental health and SUD treatment providers to help Medicaid enrollees
receive inpatient treatment, outpatient treatment, and recovery support services, KCBHO provides crisis
services to the entire King County population. During 2016, KCBHO served over 47,000 distinct Medicaid
consumers residing in King County. The BHO has contracts and subcontracts with over 50 behavioral
health agencies, which include mental health agencies and SUD treatment agencies.
Because of the transition of RSNs to BHOs in April 2016 and the concurrent integration of the mental
health and SUD treatment services, DBHR directed Qualis Health to perform a readiness review for the
2016 external quality review (EQR) on the integration. The readiness review included an assessment and
evaluation of each BHO’s transition plan submitted to the State, as well as each BHO’s status in
converting from an RSN to a BHO. For the 2017 review, Qualis Health is performing a follow-up to the
2016 readiness review to ascertain the current status of each BHO’s transition.
Qualis Health’s additional EQR activities for each BHO consisted of assessing the BHO’s overall
performance and identifying strengths and opportunities for improvement regarding the BHO’s
compliance with State and federal requirements for access, timeliness, and quality measures. This
included assessing compliance with standards related, but not limited to, availability of services and
quality assessment and performance improvement; validating encounter data submitted to the State;
validating the BHO’s performance improvement projects (PIPs); and performing an Information Systems
Capabilities Assessment (ISCA). Additionally, for each BHO, Qualis Health interviewed two mental health
agencies and two SUD treatment providers. The 2017 review also includes a “Golden Thread” review of
SUD treatment services to assess for clear, consistent care linkages between an individual’s needs,
diagnosis, and treatment. Finally, Qualis Health reviewed the BHO’s previous-year recommendations.
The results of this review and the readiness review follow-up appear in Appendices B and C.
This report, in fulfillment of federal requirements under 42 CFR §438.350, describes the results of this
2017 external quality review.
6 Executive Summary
KCBHO Transition Implementation Status
At the time of the 2017 EQR, KCBHO was one year into its transition from an RSN to a BHO and
its integration of mental health and SUD treatment services. Although the transition has been
successful overall, it has also presented a number of challenges.
One of these challenges includes the BHO’s difficulty in establishing timely transmission of
encounter data by the SUD treatment agencies. KCBHO created a web portal to assist the BHAs
in submitting information in a timely manner, but at the time of the review, several providers were
still failing to submit accurate and timely data. Several SUD treatment providers, additionally,
were not performing edit checks on the data prior to submission to the BHO. To help alleviate
these problems, KCBHO hired additional staff to support the SUD treatment providers with direct
technical assistance.
Additionally, the increase in enrollees resulting from the Affordable Care Act has created the need
for additional workforce. Hiring qualified clinicians for both mental health and SUD treatment
services has posed many obstacles for the BHO. KCBHO has been actively working to develop
solutions for recruiting staff, including participating on a Washington State governor’s task force
dealing with these issues.
To account for the increase in requests for services, KCBHO has hired several staff into its
authorization and case management department who are dually licensed mental health providers
and chemical dependency professionals. The authorization and case management staff
developed new SUD treatment utilization review tools for both authorizing and monitoring SUD
treatment services. Requests for service authorizations are typically approved within 24 hours,
unless additional information is required. The BHO has also modified the payment structure for
SUD treatment services, replacing the fee-for-service model with a case rate, which has allowed
the BHO to submit payment to the agencies within a shorter timeframe.
KCBHO has undergone significant work in designing a co-occurring disorder treatment benefit and
practice guideline, collaborating with both its contracted providers and DBHR in the process. The BHO
has also been working on modifying its internal and external policies and procedures to include both
mental health and SUD treatment services. During the last year, the BHO has concentrated heavily on
educating all providers on the importance and significance of the grievance system.
Residential treatment for SUD treatment services is a new business for the BHO. The BHO stated that
over 50 percent of its enrollees are referred out of the county for these services, because King County
lacks adequate service capacity. At the time of the review, the BHO had contracts with 39 residential
treatment facilities across the state. To assist in locating available residential beds, the BHO uses a
residential bed tracker database system. Residential treatment facilities use the database to indicate
when the facility has an available bed. The tool has proven helpful in identifying available beds throughout
the state, and several other BHOs are now using the bed tracker as well.
KCBHO works together with the BHAs in helping to improve the care and services provided to its
enrollees. The BHO has made substantial progress in its transition during the prior year, and the
interviews with the contracted providers indicated there is a collaborative relationship between the BHAs
and KCBHO.
7 Executive Summary
Description of EQR Activities
EQR Federal regulations under 42 CFR §438.358 specify the mandatory and optional activities that the
EQR must address in a manner consistent with protocols of the Centers for Medicare & Medicaid
Services (CMS). This report is based on information collected from the BHO in connection with the CMS
EQR protocols and includes results from the following activities:
compliance monitoring through document review, on-site interviews at the BHO, on-site
reviews of SUD treatment provider agencies, and telephone interviews with mental health and
SUD treatment provider agencies. The purpose of the 2017 compliance review is to determine
the status of the BHO’s integration of SUD treatment and mental health agencies and the BHO’s
capability in meeting regulatory and contractual standards governing managed care.
encounter data validation (EDV) conducted through data analysis and clinical record review
validation of three performance improvement projects (PIPs) to determine whether the BHO
met standards for conducting these required studies
an Information Systems Capabilities Assessment using information collected in the ISCA
data collection tool, responses to interview questions, and results of the claims/encounter
analysis walkthroughs and security walkthroughs
follow-up on previous-year recommendations
Together, these activities answer the following questions:
1. What is the status of the integration of mental health and SUD treatment services within managed
care under the auspices of the BHO?
2. What is the status of the BHO in meeting the CMS regulatory requirements?
3. What is the status of the BHO in meeting the requirements of its contract with the State and the
Washington Administrative Code (WAC)?
4. What processes and procedures does the BHO have in place to monitor and oversee contracted
providers in their performance of any delegated activities to ensure regulatory and contractual
compliance?
5. What progress has the BHO made in conducting the three required PIPs?
6. Is the encounter data the BHAs submitted to the State accurate, complete, and valid?
7. Does the BHO’s information technology infrastructure support the production and reporting of
valid and reliable performance measures?
Summary of Results
Scoring Icon Key
● Fully Met (pass) ● Partially Met (pass) ● Not Met (fail) ● N/A (not applicable)
Compliance Review Results
This review assesses the BHO’s overall performance, identifies strengths, notes opportunities for
improvement, and recommends corrective action plans (CAPs) in areas where the BHO did not clearly or
comprehensively meet federal and/or State requirements. In addition, in cases in which the BHO has not
addressed a previous-year recommendation, Qualis Health may have issued a recommendation requiring
8 Executive Summary
a corrective action plan. The following opportunities and recommendations offer guidance on how the
BHO may achieve full compliance with State contractual and federal CFR guidelines. The results are
summarized below in Table A-1. Please refer to the compliance review section of this report for complete
results.
Table A-1: Summary Results of Compliance Monitoring Review
CMS EQR Protocol Results
Section 1.
Availability of Services
● Partially Met (pass)
Section 2.
Coordination and Continuity of Care
●Partially Met (pass)
Section 3.
Coverage and Authorization of
Services
● Fully Met (pass)
Section 4.
Provider Selection
● Partially Met (pass)
Section 5. Subcontractual
Relationships and Delegation
● Partially Met (pass)
Section 6. Practice Guidelines ● Partially Met (pass)
Section 7. Health Information Systems ● Partially Met (pass)
Section 8. Quality Assessment and
Performance Improvement Program
● Partially Met (pass)
Performance Improvement Project (PIP) Validation Results
As a mandatory EQR activity, Qualis Health evaluated the BHO’s performance improvement projects to
determine whether the projects have been designed, conducted, and reported in a methodologically
sound manner. The projects must be designed to achieve, through ongoing measurements and
intervention, significant improvement sustained over time that is expected to have a favorable effect on
health outcomes and enrollee satisfaction. The results for the BHO’s clinical, non-clinical, and substance
use disorder-focused PIPs are found in Table A-2. Further discussion can be found in the performance
improvement project section of this report.
Table A-2: Performance Improvement Project Validation Results
Results Validity and Reliability
Clinical PIP: Effectiveness of the Transitional Support Program ● Partially Met (pass)
High confidence in reported results
Non-Clinical Children’s PIP: Improved Coordination with Primary Care for Children and Youth
● Partially Met (pass) Not enough time has elapsed to assess meaningful change
Substance Use Disorder (SUD) PIP: SUD Residential Treatment Length of Stay ● Partially Met (pass)
Not enough time has elapsed to assess meaningful change
9 Executive Summary
Information Systems Capability Assessment (ISCA) Results
The BHO’s information systems and data processing and reporting procedures were examined to
determine the extent to which they support the production of valid and reliable State performance
measures and the capacity to manage care of BHO enrollees.
The ISCA procedures were based on the CMS protocol for this activity, as adapted for the Washington
BHOs with DBHR's approval. For each of seven ISCA review areas, the following methods were used to
rate the BHO’s performance:
information collected in the ISCA data collection tool
responses to interview questions
results of the claims/encounter analysis walkthroughs and security walkthroughs
The organization was then ranked as fully meeting, partially meeting, or not meeting standards. Although
not rated, the BHO’s meaningful use of EHR systems for informational purposes was also evaluated.
The results are summarized below in Table A-3. Please refer to the ISCA section of this report for
complete results.
Table A-3: Results of the BHO’s Information Systems Capabilities Assessment (ISCA)
ISCA Section
Description ISCA Result
Overall ISCA Score This is the overall score for the BHO's ISCA. ● Partially Met (pass)
A. Information Systems
This section assesses the BHO’s management of the information systems.
● Partially Met (pass)
B. IT Infrastructure This section assesses the BHO’s network infrastructure.
● Partially Met (pass)
C. Information Security
This section assesses the security of the BHO’s information systems.
● Partially Met (pass)
D. Encounter Data Management
This section assesses the BHO’s ability to capture and report accurate encounter data.
● Partially Met (pass)
E. Eligibility Data Management
This section assesses the BHO’s ability to capture and report accurate Medicaid eligibility data.
● Fully Met (pass)
F. Provider Data Management
This section assesses the BHO’s ability to maintain accurate provider information.
● Fully Met (pass)
G. Performance Measures and Reporting
This section assesses the BHO's performance measure and reporting processes.
● Fully Met (pass)
H. Meaningful Use of Electronic Health Records (EHRs)
This section is used to assess how the BHO and its providers use EHRs. This section is not scored.
● N/A
10 Executive Summary
Encounter Data Validation (EDV) Results
EDV is a process used to validate encounter data submitted by BHOs to the State. Encounter data are
the electronic records of services provided to BHO enrollees by both institutional and practitioner
providers (regardless of how the providers were paid), when the services would traditionally be a billable
service under fee-for-service (FFS) reimbursement systems. Encounter data are used by the BHOs and
the State to assess and improve the quality of care and to monitor program integrity. Additionally, the
State uses encounter data to determine capitation rates paid to the BHOs.
Qualis Health performed independent validation of the procedures used by the BHO to perform its own
encounter data validation. The EDV requirements included in the BHO’s contract with DBHR were used
as the standard for validation. Qualis Health obtained and reviewed the BHO’s encounter data validation
report submitted to DBHR as a contract deliverable for calendar year 2016. The BHO’s encounter data
validation methodology, encounter and enrollee sample size(s), selected encounter dates, and fields
selected for validation were reviewed for conformance with DBHR contract requirements. The BHO’s
encounter and/or enrollee sampling procedures were reviewed to ensure conformance with accepted
statistical methods for random selection. Table A-3 shows the results of the review of the BHO’s
encounter data validation processes.
Table A-3: Results of External Review of the BHO’s Encounter Data Validation Procedures
EDV Standard Description Result—Mental Health Result—SUD
Sampling Procedure
Sampling was conducted using an appropriate random selection process and was of adequate size.
●Partially Met (pass) ●Partially Met (pass)
Review Tools Review and analysis tools are appropriate for the task and were used correctly.
●Fully Met (pass) ●Fully Met (pass) Methodology and Analytic Procedures
The analytical and scoring methodologies are sound, and all encounter data elements requiring review were examined.
●Not Met (fail) ●Not Met (fail)
Qualis Health conducted its own validation to assess the BHO’s capacity to produce accurate and
complete encounter data. The encounter data submitted by the BHO to the State was analyzed to
determine the general magnitude of missing encounter data, types of potentially missing encounter data,
overall data quality issues, and any issues with the processes the BHO has in compiling encounter data
and submitting the data files to the State. Clinical record review of encounter data was performed to
validate data sent to the State and confirm the findings of the analysis of the State-level data.
Table A-4 summarizes results of Qualis Health’s EDV. Please refer to the EDV section of this report for
complete results.
Table A-4: Results of Qualis Health Encounter Data Validation
EDV Standard Description Result—Mental Health Result—SUD
Electronic
Data Checks
Full review of encounter data
submitted to the State indicates
no (or minimal) logic problems
or out-of-range values.
●Fully Met (pass) ●Fully Met (pass)
11 Executive Summary
On-site
Clinical
Record
Review
State encounter data are
substantiated in audit of patient
charts at individual provider
locations. Audited fields include
demographics (name, date of
birth, ethnicity, and language)
and encounters (procedure
codes, provider type, duration
of service, service date, and
service location). A passing
score is that 95 percent of the
encounter data fields in the
clinical records match.
●Not Met (fail) ●Not Met (fail)
Golden Thread Review of SUD Treatment Services
For the 2017 EQR focus study, Qualis Health examined the degree to which SUD treatment providers’
clinical records demonstrated adherence to the “golden thread,” a series of clear, consistent care linkages
between an individual’s needs, diagnosis, and treatment. In evaluating provider records, reviewers asked
the following questions:
1. Does the individual’s assessment contain sufficient documentation to support the diagnosis, and
does it include all of the individual’s needs?
2. Are the documented needs reflected in specific goals in the treatment plan/individual service plan
(ISP)? Does the ISP address the individual’s diagnosis and all of the identified needs in the
assessment?
3. Do the progress notes address the individual’s treatment plan goals and the needs identified in
the assessment?
To answer these questions, Qualis Health reviewed adult patient charts randomly selected from the
BHO’s EDV sample. Reviewers specifically evaluated documentation in three areas: assessments and re-
assessments, individual service planning, and progress notes. Results for these review areas are
discussed in the Golden Thread section of this report.
12 Executive Summary
This page intentionally left blank.
13 Compliance
Compliance with Regulatory and Contractual Standards
The 2017 compliance review addresses the BHO’s compliance with federal Medicaid managed care
regulations and applicable elements of the contract between the BHO and the State. The relevant CFR
sections and results of the 2017 compliance review are included in Table B-1, below.
The CMS protocols and scoring criteria used for conducting the compliance review are included in
Appendix E. The protocols can also be found here: http://www.medicaid.gov/Medicaid-CHIP-Program-
Information/By-Topics/Quality-of-Care/Quality-of-Care-External-Quality-Review.html.
Each section of the compliance review contains elements corresponding to relevant sections of the CFR
(primarily 42 CFR Part 438), DBHR’s contract with the BHO, the Washington Administrative Code (WAC),
and other State regulations where applicable. Qualis Health evaluated King County BHO’s performance
on each element of the protocol by
performing desk audits on documentation submitted by the BHO
conducting telephone interviews with two of the BHO's contracted mental health agencies and
two of its substance use disorder (SUD) treatment providers
conducting on-site interviews with BHO staff on standards related to quality assessment and
performance improvement (QAPI) and performance improvement projects (PIPs)
performing encounter data validation (EDV)
following up on the prior year’s corrective action plans (CAPs)
following up on the prior year’s SUD treatment provider readiness review
following up on the status of the BHO’s transition plan
auditing credentialing files at selected mental health and SUD treatment provider agencies
reviewing for care coordination at selected mental health agencies
auditing for “golden thread” practices at SUD treatment provider agencies
This review assesses KCBHO’s overall performance, identifies strengths, notes opportunities for
improvement, and presents recommendations for corrective action plans (CAPs) in areas where the BHO
did not clearly or comprehensively meet federal and/or State requirements at the time of the review. The
accompanying recommendations and opportunities for improvement offer guidance on how the BHO may
achieve full compliance with State contractual, WAC, and CFR standards.
Compliance Scoring
Qualis Health uses CMS’s three-point scoring system in evaluating compliance. The three-point scale
allows for credit when a requirement is partially met and the level of performance is determined to be
acceptable. The three-point scoring system includes the following levels:
● Fully Met means all documentation listed under a regulatory provision, or component thereof, is
present and BHO staff provided responses to reviewers that were consistent with each other’s
responses and with the documentation.
● Partially Met means all documentation listed under a regulatory provision, or component
thereof, is present, but BHO staff were unable to consistently articulate evidence of compliance,
or BHO staff could describe and verify the existence of compliant practices during the
interview(s), but required documentation is incomplete or inconsistent with practice.
14 Compliance
● Not Met means no documentation is present and BHO staff had little to no knowledge of
processes or issues that comply with regulatory provisions, or no documentation is present and
BHO staff had little to no knowledge of processes or issues that comply with key components of
a multi-component provision, regardless of compliance determinations for remaining, non-key
components of the provision.
Scoring Icon Key
● Fully Met (pass) ●Partially Met (pass) ● Not Met (fail) ● N/A (not applicable)
Summary of Compliance Review Results
Table B-1: Summary Results of Compliance Monitoring Review, by Section
CMS EQR Protocol Results
Section 1.
Availability of Services
● Partially Met (pass)
Section 2.
Coordination and Continuity of Care
●Partially Met (pass)
Section 3.
Coverage and Authorization of
Services
● Fully Met (pass)
Section 4.
Provider Selection
● Partially Met (pass)
Section 5. Subcontractual
Relationships and Delegation
● Partially Met (pass)
Section 6. Practice Guidelines ● Partially Met (pass)
Section 7. Health Information Systems ● Partially Met (pass)
Section 8. Quality Assessment and
Performance Improvement Program
● Partially Met (pass)
Summary of Opportunities for Improvement and Recommendations
Requiring Corrective Action Plans (CAPs), by Section
Section 1: Availability of Services
Opportunity for Improvement
The BHO’s last geographical mapping was performed in 2015, and the BHO stated it is committed to
completing another geo mapping by the end of 2017.
15 Compliance
With the addition of SUD treatment services, KCBHO should follow through with completing a geo
mapping by the end of 2017 to assess its network adequacy.
Recommendations Requiring CAP
Enrollee participation varies greatly among the agencies, from 9,000 enrollees for one agency to 30
enrollees for another agency. Regardless of the participation rate, workforce development remains a
major concern for the BHO.
KCBHO needs to continue to work with the BHAs in developing successful talent management
and recruitment strategies, as demand for services has increased as a result of the Affordable
Care Act and the opioid epidemic.
The BHO indicated that during its most recent administrative audit of the BHAs, the SUD treatment
agencies were not aware of how to arrange for second opinions and had not implemented policies on
second opinions. The BHO stated it provided consultation with its SUD treatment BHAs on how to
arrange for second opinions and encouraged the BHAs to adopt the BHO’s policy on second opinions.
KCBHO needs to follow up with the SUD treatment BHAs to ensure they have adopted and
implemented policies on second opinions.
KCBHO noted it does not have a system for tracking requests for out-of-network services.
KCBHO needs to implement a procedure to track requests for out-of-network services and use
this information for network planning.
KCBHO indicated it follows its credentialing policies and procedures to ensure all out-of-network
providers are appropriately credentialed. The BHO requires all out-of-network providers to complete and
sign a single-case service agreement, which requires the provider to submit license(s)/credentials and
attest to a background check, and provides assurance that the provider is not on the excluded provider
list. However, during the on-site interview, KCBHO indicated it needed to update its credentialing policies
and procedures to ensure all providers are appropriately credentialed.
KCBHO needs to update its credentialing policies and procedures to ensure all providers are
appropriately credentialed, including out-of-network providers and those with a single-case
agreement.
Section 2: Coordination and Continuity of Care
Opportunity for Improvement
KCBHO’s 2017 on-site clinical record review tool included monitoring to ensure that documentation of
coordination of activities is evident in the enrollee’s clinical record and that communication occurs within
the scope of the consent and release(s) given by the enrollee.
KCBHO should continue its efforts, through its 2017 clinical site review monitoring, to ensure that
documentation of coordination of activities is evident in the enrollee’s clinical record and that
communication occurs within the scope of the consent and release(s) given by the enrollee.
Recommendations Requiring CAP
Although KCBHO has a policy and procedure for coordinating enrollee care, the policy lacks a definition
or standard of what constitutes care coordination and how it is measured.
In its policy on enrollee care coordination, KCBHO needs to include a definition or standard of
what constitutes care coordination and how it is measured. The definition should include:
o the standard for coordinating care between settings of care
16 Compliance
o appropriate discharge planning for short-term and long-term hospital or institutional stays
o services the enrollee receives from any other BHO
o services the enrollee receives in fee-for-service (FFS) Medicaid
o services the enrollee receives from community and social support providers
KCBHO requires the BHAs to initiate a referral to a primary healthcare provider appropriate for the
enrollee’s needs when the enrollee does not have a primary healthcare provider. KCBHO stated that
through its clinical record review and monitoring of the intake process it ensures each enrollee has
access to a primary healthcare provider. The BHO has also included monitoring in its 2017 on-site clinical
record review tool to ensure that documentation of coordination of activities is evident in the enrollee’s
clinical record. However, during the BHA on-site record review conducted by the EQRO, the majority of
charts lacked evidence of care coordination with the primary healthcare provider. In addition, the 2016
summary report results from KCBHO’s administrative review indicated that the majority of agencies had
difficulty articulating coordination of care in their policies and procedures.
KCBHO needs to continue its monitoring efforts to ensure its BHAs have clearly defined care
coordination in their policies and procedures. The BHO needs to continue training, educating, and
monitoring its BHAs to ensure enrollees are referred to a primary healthcare provider when
appropriate and that care coordination is documented in the clinical record.
KCBHO’s clinical record and administrative reviews include criteria for reviewing treatment plans to
ensure the treatment plans are developed with the enrollee’s participation, and in consultation with any
specialists caring for the enrollee. However, the results of the on-site EQR of the care coordination
records indicated that some BHAs are not using or completing treatment plans and that many treatment
plans that were in place did not include enrollee voice and participation.
KCBHO needs to ensure that all BHAs have treatment plans in place and that the treatment plans
include documentation that the plans were developed with the enrollee’s participation and in
consultation with any specialists caring for the enrollee.
KCBHO does not have a policy and procedure on direct access to specialists for enrollees with special
healthcare needs.
The BHO needs to develop a policy and procedure regarding direct access to specialists for
enrollees with special healthcare needs.
KCBHO does not monitor the availability of direct access to specialists.
KCBHO needs to add criteria to its monitoring tool to assess availability of direct access to
specialists.
Section 3: Coverage and Authorization of Services
N/A
Section 4: Provider Selection
Recommendations Requiring CAP
During the on-site interview, the BHO indicated it is not monitoring its BHAs to ensure they have a policy
and procedure on credentialing and re-credentialing.
KCBHO needs to add criteria to its monitoring tool in order to review each BHA’s credentialing
and re-credentialing policy and procedure.
17 Compliance
At the time of the 2017 EQR, the BHO indicated it had created an internal process to ensure monthly
exclusion checks were performed on all staff, including its BHAs. However, it was determined during the
on-site visit that the BHO is not performing monthly Office of the Inspector General (OIG) List of Excluded
Individuals and Entities (LEIE) checks on its entire staff at the BHO, including the county executives, BHO
leadership, interns, volunteers, and staff who make authorization decisions.
KCBHO needs to ensure the BHO as well as the BHAs are performing exclusion checks on a
monthly basis and at the time of hire on all staff, including county executives, BHO leadership,
board members, custodial staff, and volunteers.
Section 5: Subcontractual Relationships and Delegation
Recommendations Requiring CAP
KCBHO’s BHA contracts do not contain language holding the BHAs’ subcontractors accountable for
delegated services.
KCBHO needs to add language to its BHA contracts that clarifies that BHA subcontractors are
accountable for any delegated services.
KCBHO does not include delegation monitoring in its annual QA evaluation.
KCBHO needs to add delegation monitoring to its annual QA evaluation.
Section 6: Practice Guidelines
Opportunity for Improvement
Although KCBHO posts its practice guidelines on its website, they are not easy to locate. The titles are
not self-explanatory, and the guidelines are filed in the Quality Management and Extraordinary
Occurrences section of the website as appendices, which could make it very difficult for enrollees and
providers to access them.
KCBHO should consider including “practice guidelines” in the title of the practice guidelines
located on the BHO’s website in order to make them easily identifiable for both enrollees and
providers.
Recommendation Requiring CAP
KCBHO’s policy states that practice guidelines are disseminated to all affected providers and, upon
request, to enrollees, but it does not specify the mechanism or frequency of distribution.
KCBHO needs to include in its policy the mechanism and frequency with which it distributes the
practice guidelines to providers and enrollees.
Section 7: Health Information Systems
Recommendations Requiring CAP
The BHO does not have an external policy and procedure to ensure its BHAs are checking their data for
quality and integrity before submitting them to the BHO.
The BHO needs to create and implement a policy and procedure to ensure its BHAs are checking
their data for quality and integrity before submitting them to the BHO. The policy should include:
o the requirement for providers to submit written attestations of data accuracy
o a form letter for providers to complete attesting to data accuracy
o a system for the form letters to be transmitted electronically to the BHO
18 Compliance
o monitoring by BHO contract monitors to ensure timely submission of the attestation
letters
KCBHO reported that the SUD treatment BHAs were experiencing challenges submitting data in the file
formats required by the Service Encounter Reporting Instructions (SERI) as this was a new requirement
for the providers. The SUD treatment BHAs were encouraged to use a web portal the BHO designed for
data submission until the challenges were resolved. However, the EQRO was unable to perform EDV at
the time of the review because the SUD treatment BHAs had not submitted all of the required data.
KCBHO needs to continue to train and assist its SUD treatment BHAs to ensure the BHAs can
submit timely, accurate, and complete data.
Section 8: Quality Assessment and Performance Improvement Program
Opportunity for Improvement
KCBHO does not comply with the State Quality Strategy as the State does not have a current quality
strategy.
KCBHO will need to comply with the State Quality Strategy once the State has developed and
implemented its plan.
Recommendation Requiring CAP
In the past year, KCBHO has not been able to report all of its SUD treatment performance data to the
State.
KCBHO needs to continue to monitor and support its SUD treatment BHAs in order to ensure it
receives all performance data.
19 Compliance
Section 1: Availability of Services
Table B-2: Summary of Compliance Review for Availability of Services
Protocol Section CFR Result
Availability of Services
1. Delivery Network/Network Adequacy
Standards
438.206 (b)(1), 438.68 ● Partially Met (pass)
2. Second Opinion 438.206 (b)(3) ● Partially Met (pass)
3. Out-of-network Services 438.206 (b)(4) ● Partially Met (pass)
4. Payment of Out-of-network Services 438.206 (b)(5)
● Fully Met (pass) 5. Provider Credentials 438.206 (b)(6), 438.214
● Partially Met (pass)
6. Timely Access 438.206 (c)(1)
● Fully Met (pass)
7. Access and Cultural Considerations 438.206 (c)(2–3) ● Fully Met (pass) Overall Result for Section 1. ● Partially Met (pass)
Delivery Network/Network Adequacy Standards: 438.206 (b)(1), 438.68
Scoring Criteria
The BHO maintains and monitors an appropriate network of BHAs that is supported by
written agreements.
The BHO’s BHA network is sufficient to provide adequate access to all services covered
under the contract.
The BHO annually conducts geo mapping that includes the cultural, ethnic, racial, and
linguistic needs of its members; distance; travel time; the means of transportation
ordinarily used by Medicaid enrollees; and whether the location provides physical access
for Medicaid enrollees with disabilities.
In establishing and maintaining the network, the BHO considers:
o the anticipated Medicaid enrollment
o the expected utilization of services, taking into consideration the characteristics and
healthcare needs of specific Medicaid populations represented in the BHO
o the numbers and types (training, experience, and specialization) of BHAs required to
furnish the contracted Medicaid services
o the number of network BHAs that are not accepting new Medicaid patients
The BHO has formal procedures in place to monitor its BHA network to ensure adequacy.
Reviewer Determination
● Partially Met (pass)
20 Compliance
Strengths
KCBHO created its internal policy and procedure Managing Network Sufficiency as a mechanism
to monitor change in network sufficiency and to provide reports to DBHR in a timely fashion.
KCBHO regularly monitors its network sufficiency by measuring and reviewing:
o the penetration rate of enrollees in both SUD treatment and mental health services
o service utilization patterns, including provider caseloads
o enrollees’ timely access to care and services
o the geographic location of providers and Medicaid enrollees, considering distance,
travel time, and the means of transportation ordinarily used by enrollees
o contractor and subcontractor compliance with Americans with Disabilities Act (ADA)
requirements for physical access
o availability of specialists
o use of clinical services
o provision of outreach services
KCBHO stated it has a mix of culturally diverse agencies that primarily provide services to
enrollees of Hispanic, Asian, and African-American descent.
Several of the BHO’s contracted BHAs have systems in place, including mobile medical vans
and visiting medical staff, to provide care to enrollees who have both mental health and medical
needs.
KCBHO described how the BHO maintains and monitors an appropriate network of BHAs that is
supported by written agreements. At the time of the EQR, the BHO had 40 direct contract
providers and 22 subcontracted providers and continues to:
o monitor its BHAs to measure and anticipate growth
o determine through dialogue with new agencies which populations they are bringing to the
network
o interview potential providers to determine if they can help meet the needs of the county’s
various populations
The BHO has been proactive by meeting with enrollees and asking for their feedback on needed
services. The BHO has expanded this program to include enrollees with multiple uses of the
crisis system to help identify service needs and decrease the use of the crisis system.
Opportunity for Improvement
The BHO’s last geographical mapping was performed in 2015, and the BHO stated it is committed to
completing another geo mapping by the end of 2017.
With the addition of SUD treatment services, KCBHO should follow through with completing a geo
mapping by the end of 2017 to assess its network adequacy.
Recommendation Requiring CAP
Enrollee participation varies greatly among the agencies, from 9,000 enrollees for one agency to 30
enrollees for another agency. Regardless of the participation rate, workforce development remains a
major concern for the BHO.
KCBHO needs to continue to work with the BHAs in developing successful talent management
and recruitment strategies, as demand for services has increased as a result of the Affordable
Care Act and the opioid epidemic.
21 Compliance
Second Opinion: 438.206 (b)(3)
Scoring Criteria
The BHO maintains policies and procedures related to second opinions that meet the
contract and WAC standards.
BHO staff are knowledgeable about State and federal requirements, as well as internal
policies and procedures.
The BHO provides literature or other materials available to enrollees to provide
information about the enrollee’s right to a second opinion.
The BHO provides for a second opinion from a qualified healthcare professional within
the network, or arranges for the enrollee to obtain one outside the network, at no cost to
the enrollee.
The BHO has an effective process in place to monitor compliance with standards.
Reviewer Determination
● Partially Met (pass)
Strengths
KCBHO’s policy and procedure 4.17 Client Second Opinion is well written and includes the
requirement that providers develop policies and procedures for providing requested second
opinions. The policy also states that providers may request that second opinions be arranged with
another provider agency within the KCBHO network. Such a request must be made in writing to
KCBHO’s Client Assistance Services and be accompanied by an explanation describing why the
opinion cannot be provided at the enrollee’s current agency. Second opinions are provided at no
cost to the enrollee.
KCBHO staff are very knowledgeable about the BHO’s second opinion policy. The contract
monitoring staff audit the agencies’ policies on second opinions during the yearly administration
review.
KCBHO requires that the second opinion occur within 30 days of the enrollee’s request,
unless the enrollee requests a delay.
Recommendation Requiring CAP
The BHO indicated that during its most recent administrative audit of the BHAs, the SUD treatment
agencies were not aware of how to arrange for second opinions and had not implemented policies on
second opinions. The BHO stated it provided consultation with its SUD treatment BHAs on how to
arrange for second opinions and encouraged the BHAs to adopt the BHO’s policy on second opinions.
KCBHO needs to follow up with the SUD treatment BHAs to ensure they have adopted and
implemented policies on second opinions.
Out-of-network Services: 438.206 (b)(4)
Scoring Criteria
The BHO provides documentation of adequate and timely covered services for out-of-
network enrollees when the network is unable to provide necessary services covered
under the contract.
The BHO provides up-to-date existing agreements and/or contracts with out-of-network
providers.
22 Compliance
The BHO has a process to track out-of-network encounters and reviews this information for
network capacity planning.
Reviewer Determination
● Partially Met (pass)
Strength
KCBHO stated its network of providers is able to provide all services to enrollees except for
eating disorder treatment services. If the BHO receives a request for a referral for eating disorder
treatment, the BHO works very closely with the provider agency to establish the treatment plan
and arrange the services.
Recommendation Requiring CAP
KCBHO noted it does not have a system for tracking requests for out-of-network services.
KCBHO needs to implement a procedure to track requests for out-of-network services and use
this information for network planning.
Payment of Out-of-network Services: 438.206 (b)(5)
Scoring Criteria
The BHO has a documented process and policy that require out-of-network providers to
coordinate with the BHO with respect to payment.
The BHO has a documented process for how out-of-network providers are paid.
The BHO ensures and has a documented policy and process that cost to the enrollee is
not greater than it would be if the out-of- network services were furnished within the
network.
The BHO has a process for the action taken if an enrollee receives a bill for out-of-network
services.
Reviewer Determination
● Fully Met (pass)
Strength
KCBHO has an appropriate policy covering how services not currently provided within the
network are paid for and documented, and clarifying that the cost of these services to the enrollee
is no greater than it would be for in-network services.
Provider Credentials: 438.206 (b)(6), 438.214
Scoring Criteria
The BHO has a documented process and policy to ensure that its network providers are
appropriately credentialed.
Reviewer Determination
● Partially Met (pass)
Recommendation Requiring CAP
KCBHO indicated it follows its credentialing policies and procedures to ensure all out-of-network
providers are appropriately credentialed. The BHO requires all out-of-network providers to complete and
23 Compliance
sign a single-case service agreement, which requires the provider to submit license(s)/credentials and
attest to a background check, and provides assurance that the provider is not on the excluded provider
list. However, during the on-site interview, KCBHO indicated it needed to update its credentialing policies
and procedures to ensure all providers are appropriately credentialed.
KCBHO needs to update its credentialing policies and procedures to ensure all providers are
appropriately credentialed, including out-of-network providers and those with a single-case
agreement.
Timely Access: 438.206 (c)(1)
Scoring Criteria
The BHO has and implements a policy for timely access.
The BHO requires its providers to meet State Medicaid standards for timely access to
care and services.
The BHO ensures that the network BHAs offer hours of operation that are no less than the
hours of operation offered to commercial enrollees or comparable to Medicaid fee-for-
service, if the BHA serves only Medicaid enrollees.
The BHO has established mechanisms to ensure services included in the contract are
available 24 hours a day, 7 days a week, when medically necessary.
The BHO takes corrective action and has documentation of such corrective action if BHAs fail to
comply with access standards.
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO is monitoring and tracking its BHAs in several areas to assess timely access to care.
During the last four years, the BHO has witnessed a steady incline in the percentage of enrollees
who are receiving services within 14 days of a request for service. In 2016, BHAs were meeting
the stated goals for 84 percent of clients, compared to 73 percent in 2015.
In July 2015, the BHO issued a CAP to one of its BHAs for non-compliance with access
standards. KCBHO provided evidence that the BHO monitored the BHA’s implementation of the
corrective action plan.
Access and Cultural Considerations: 438.206 (c)(2–3)
Scoring Criteria
The BHO has a documented policy and procedure related to the delivery of services in a
culturally competent manner to all enrollees, including those with limited English
proficiency and diverse cultural and ethnic backgrounds.
The BHO monitors and has documented tracking of the delivery of services to those with
limited English proficiency and diverse cultural and ethnic backgrounds.
The BHO has documentation of any cultural competency training(s).
The BHO has a documented policy and procedure related to accessibility considerations
to ensure that BHAs provide physical access, reasonable accommodations, and
accessible equipment for Medicaid enrollees with physical or mental disabilities.
The BHO monitors its BHAs to ensure they provide adequate physical access, reasonable
accommodations, and accessible equipment for Medicaid enrollees with physical or mental
24 Compliance
disabilities.
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO’s policies pertaining to cultural competency are comprehensive and well written. They
specify that services be “age appropriate, culturally relevant and linguistically competent.”
KCBHO’s policy Purpose of the King County Behavioral Health Plan articulates that through the
choice of providers in the network, the BHO provides access to care, appropriate levels of care,
and services that are both age appropriate and culturally relevant.
KCBHO maintains several programs and contracts to provide services for enrollees from diverse
cultural and ethnic backgrounds.
The 2017 administrative contract review includes assessing how the BHAs provide services that
are:
o culturally responsive, including awareness of racial and ethnic factors impacting
treatment, and informed by the cultural context of the individual
o linguistically appropriate
o responsive to the needs of persons with disabilities
o appropriate to the chronological age and developmental needs of the individual
o trauma-informed
o attuned to promoting recovery and resiliency
25 Compliance
Section 2: Coordination and Continuity of Care
Table B-3: Summary of Compliance Review for Coordination and Continuity of Care
Protocol Section CFR Result
Coordination and Continuity of Care
Primary Care and Coordination of
Healthcare Services
438.208 (b)(1–5) ● Not Met (fail)
Enrollee Privacy and HIPAA Compliance 438.208 (b)(6)
45 CFR 160.310,
160.316
● Fully Met (pass)
Confidentiality of Records 42 U.S.C. 290dd–2 ● Fully Met (pass)
Confidentiality of Alcohol and Drug Abuse
Patient Records—Access and
Restrictions
2.12 (a–c), 2.16 (a),
2.19, 2.22 (a–c), 2.23 ● Fully Met (pass)
Confidentiality of Alcohol and Drug Abuse
Patient Records—Audit and Evaluation
2.53 ● Fully Met (pass)
Distribution of Enrollee Information 431.300 (a–c),
431.301, 431.302,
431.307 (a)(1)
● Fully Met (pass)
Additional Services for Enrollees with
Special Healthcare Needs
438.208 (c)(2) ● Fully Met (pass)
Treatment/Service Plans 438.208 (c)(3) ● Not Met (fail)
Direct Access to Specialists 438.208 (c)(4)
● Not Met (fail)
Overall Result for Section 2. ●Partially Met (pass)
Primary Care and Coordination of Healthcare Services: 438.208 (b)(1–5)
Scoring Criteria
The BHO must have and implement a policy and procedure for delivering care and coordinating
healthcare services for all enrollees.
The BHO must ensure enrollees have an ongoing source of care appropriate for their needs and
access to providers responsible for coordinating enrollees’ care and services.
The BHO ensures that each enrollee has access to a primary healthcare provider appropriate to
the enrollee’s needs.
The BHO has a process in place to communicate enrollees’ healthcare needs with other service
providers to prevent duplication of activities.
The BHO has a process in place to monitor care coordination and other healthcare services
furnished to enrollees with its BHAs.
Reviewer Determination
● Not Met (fail)
26 Compliance
Opportunity for Improvement
KCBHO’s 2017 on-site clinical record review tool included monitoring to ensure that documentation of
coordination of activities is evident in the enrollee’s clinical record and that communication occurs within
the scope of the consent and release(s) given by the enrollee.
KCBHO should continue its efforts, through its 2017 clinical site review monitoring, to ensure that
documentation of coordination of activities is evident in the enrollee’s clinical record and that
communication occurs within the scope of the consent and release(s) given by the enrollee.
Recommendations Requiring CAP
Although KCBHO has a policy and procedure for coordinating enrollee care, the policy lacks a definition
or standard of what constitutes care coordination and how it is measured.
In its policy on enrollee care coordination, KCBHO needs to include a definition or standard of
what constitutes care coordination and how it is measured. The definition should include:
o the standard for coordinating care between settings of care
o appropriate discharge planning for short-term and long-term hospital or institutional stays
o services the enrollee receives from any other BHO
o services the enrollee receives in fee-for-service (FFS) Medicaid
o services the enrollee receives from community and social support providers
KCBHO requires the BHAs to initiate a referral to a primary healthcare provider appropriate for the
enrollee’s needs when the enrollee does not have a primary healthcare provider. KCBHO stated that
through its clinical record review and monitoring of the intake process it ensures each enrollee has
access to a primary healthcare provider. The BHO has also included monitoring in its 2017 on-site clinical
record review tool to ensure that documentation of coordination of activities is evident in the enrollee’s
clinical record. However, during the BHA on-site record review conducted by the EQRO, the majority of
charts lacked evidence of care coordination with the primary healthcare provider. In addition, the 2016
summary report results from KCBHO’s administrative review indicated that the majority of agencies had
difficulty articulating coordination of care in their policies and procedures.
KCBHO needs to continue its monitoring efforts to ensure its BHAs have clearly defined care
coordination in their policies and procedures. The BHO needs to continue training, educating, and
monitoring its BHAs to ensure enrollees are referred to a primary healthcare provider when
appropriate and that care coordination is documented in the clinical record.
Enrollee Privacy and HIPAA Compliance: 438.208 (b)(6), 45 CFR 160.310, 160.316
Scoring Criteria
The BHO ensures that in the process of coordinating care, each enrollee’s privacy is protected in
accordance with the privacy requirements of HIPAA (45 CFR 160).
The BHO monitors for compliance to HIPAA regulations and takes action to correct any
deficiencies.
The BHO has a policy and procedure to provide its own records and compliance reports, as well
as those of its covered entities or business associates, indicating all parties will comply with
applicable HIPAA administrative simplification provisions.
The BHO has a policy and procedure to ensure a covered entity or business associate may not
threaten, intimidate, coerce, harass, discriminate against, or take any other retaliatory action
against any individual or other person for filing a complaint with the State regarding HIPAA
compliance.
The BHO has a policy and procedure in place ensuring the BHO and its contracted entities
cooperate with HIPAA complaint investigations and compliance reviews.
27 Compliance
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO’s policies Privacy and Security of Information Systems Data and Client Records and
Confidentiality of Client Records are both comprehensive and well written. Each policy thoroughly
addresses how the BHO and BHAs shall comply with HIPAA requirements.
Additionally, KCBHO requires its BHAs to develop comprehensive information security and
privacy policies and procedures to ensure data security and the protection and confidentiality of
client records.
KCBHO has a privacy officer who is responsible for the oversight and monitoring of all client
information and protected health information (PHI).
The privacy officer reviews each routine and recurring disclosure of PHI to ensure it meets the
minimum necessary restriction, when that restriction applies.
All BHO and BHA staff are required to annually participate in confidentiality training.
KCBHO has worked closely with the BHAs to define the information that can be shared and with
whom it can be shared.
To support an individual’s recovery, KCBHO ensures that each enrollee’s privacy is protected by
obtaining releases of information at intake, and throughout treatment, to allow for cooperative
service planning, information sharing, crisis planning, and safe prescribing of medications.
Confidentiality of Records: 42 U.S.C. 290dd–2
Scoring Criteria
The BHO has a policy and procedure to deliver care to and coordinate healthcare services for all
enrollees ensuring confidentiality and appropriate disclosure only with the enrollee’s consent.
The BHO has a policy and procedure to deliver care to and coordinate healthcare services for all
enrollees identifying the purposes and circumstances of disclosure affecting the enrollee
regardless of consent.
Reviewer Determination
● Fully Met (pass)
Strength
The BHO has several memorandums of understanding (MOUs) and data sharing agreements
with various entities and health plans and actively meets with several MCOs to share information
on enrollees with comorbidities.
Confidentiality of Alcohol and Drug Abuse Patient Records—Access and Restrictions:
2.12 (a–c), 2.16 (a), 2.19, 2.22 (a–c), 2.23
Scoring Criteria
28 Compliance
The BHO ensures that in the process of coordinating care, each enrollee’s privacy is protected in
accordance with the privacy requirements detailed in 42 CFR Part 2.
The BHO has a policy and procedure to ensure that when coordinating care for enrollees, the
BHO and its business associates, covered entities, and contracted providers follow restrictions on
disclosure of enrollee information, security for written records, and disposition of records for
discontinued programs.
The BHO has a policy and procedure to ensure the confidentiality of enrollee drug and alcohol
records, including when disclosure will be granted, patient access, and restrictions on use of
information.
The BHO has a process in place to monitor its BHAs for care coordination and all other healthcare
services furnished to enrollees to make certain policies regarding confidentiality of enrollee drug
and alcohol records, as well as disclosure, enrollee access, and restrictions on use of information
are followed.
The BHO notifies enrollees of the circumstances in which disclosure is permitted without the
enrollee’s consent.
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO’s Privacy and Security Committee is responsible for developing and implementing
privacy and security policies and procedures; reviewing and modifying policies and procedures
related to electronic technology as technology changes; monitoring adherence by the BHO and
BHAs to the privacy and security policies and procedures; and reviewing and approving training
curricula for new staff at point of hire as well as curricula for ongoing training for current staff. The
committee is chaired by the privacy officer, and meetings are attended by both the compliance
and security officers.
KCBHO’s enrollees receive a Notice of Privacy Practices, which informs them of privacy rights
and how to exercise them; how the BHO accesses, uses, and discloses PHI without client
consent or authorization; and how the BHO protects client privacy.
Confidentiality of Alcohol and Drug Abuse Patient Records—Audit and Evaluation: 2.53
Scoring Criteria
The BHO has a procedure and policy in place that describe the auditing and evaluation of SUD
treatment records in accordance with §2.53.
The BHO has a process in place to ensure that if enrollee records are copied or removed from a
BHA during an audit, the confidentiality of PHI is maintained in accordance with the security
requirements discussed in 42 CFR.
The BHO has a process in place to make sure all PHI removed from the BHAs during an audit is
destroyed upon completion of the audit.
Reviewer Determination
● Fully Met (pass)
Meets Criteria
29 Compliance
Distribution of Enrollee Information: 431.300 (a–c), 431.301, 431.302, 431.307 (a)(1) Scoring Criteria
The BHO ensures that in the process of coordinating care, each enrollee’s privacy is protected in
accordance with the privacy requirements in §§431.300, 431.301, and 431.302.
The BHO has a policy and procedure in place to ensure information exchanged by the BHO and
its BHAs is made available only to the extent necessary to assist in the valid administrative needs
of the program receiving the information and is adequately stored and protected against
unauthorized disclosure.
The BHO has a mechanism in place to monitor that distribution of information directly relates to
the administration of the State plan.
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO’s privacy officer and the King County Information Technology (KCIT) Security Team are
responsible for ensuring that privacy and security policies and procedures are adequate and are
implemented within KCBHO.
KCBHO’s policy and procedure on confidentiality includes:
o how the BHO ensures that information exchanged by the BHO and its BHAs is made
available to other entities only to the extent necessary to assist in the valid administrative
needs of the program receiving the information
o how the BHO ensures that in the process of coordinating care, each enrollee’s privacy is
protected in accordance with the privacy requirements specified in the CFR
o how the BHO monitors that distribution of information directly relates to the administration
of the State plan
Additional Services for Enrollees with Special Healthcare Needs: 438.208 (c)(2)
Scoring Criteria
The BHO has a policy and procedure for assessing each enrollee in order to identify any ongoing
special conditions that may require a special course of treatment or regular care monitoring.
The BHO has a method for identifying enrollees with special healthcare needs. Special healthcare
needs include any physical, developmental, mental, sensory, behavioral, cognitive, or emotional
impairment or limiting condition that requires medical management, healthcare intervention, and/or
use of specialized services or programs. The condition may be congenital or developmental, or
acquired through disease, trauma, or environmental cause and may impose limitations in
performing daily self-maintenance activities or substantial limitations in a major life activity.
The BHO has a process in place to monitor compliance with this requirement.
The BHO has in effect mechanisms to detect both underutilization and overutilization of services,
and to assess the quality and appropriateness of care furnished to enrollees with special
healthcare needs.
Reviewer Determination
● Fully Met (pass)
30 Compliance
Strengths
The BHO has a method for identifying enrollees with special healthcare needs. Per policy and
procedure, KCBHO requires its BHAs to obtain and document the following information for
individuals receiving outpatient services:
o the name of any current primary medical care provider
o any current physical health concerns
o current medications and any related concerns
o history of any substance use/abuse and treatment, including tobacco use
o any disabilities or special needs
KCBHO has a mechanism in place to monitor and manage service utilization by ensuring
providers:
o have a comprehensive utilization management process that identifies patterns of service
utilization by all clients, and strategies to ensure that the right services are provided at the
right time in the right place
o review the agency-specific outpatient service utilization reports provided by KCBHO to
identify service utilization patterns for all mental health outpatient benefits
o develop and implement protocols for the utilization management of their clients who are
frequently served by other costly systems, such as residential or inpatient psychiatric
care
KCBHO tracks enrollees who have had three or more authorized psychiatric hospitalizations or
multiple residential SUD treatment admissions in the preceding 12 months and works with
providers to provide case management for these enrollees.
Treatment/Service Plans: 438.208 (c)(3)
Scoring Criteria
The BHO ensures that treatment plans for enrollees with special healthcare needs are developed
with the enrollee’s participation and in consultation with any specialists caring for the enrollee.
The enrollee’s treatment plan incorporates the enrollee’s special healthcare needs.
Reviewer Determination
● Not Met (fail)
Recommendation Requiring CAP
KCBHO’s clinical record and administrative reviews include criteria for reviewing treatment plans to
ensure the treatment plans are developed with the enrollee’s participation, and in consultation with any
specialists caring for the enrollee. However, the results of the on-site EQR of the care coordination
records indicated that some BHAs are not using or completing treatment plans and that many treatment
plans that were in place did not include enrollee voice and participation.
KCBHO needs to ensure that all BHAs have treatment plans in place and that the treatment plans
include documentation that the plans were developed with the enrollee’s participation and in
consultation with any specialists caring for the enrollee.
Direct Access to Specialists: 438.208 (c)(4)
Scoring Criteria
The BHO has policies and procedures regarding direct access to specialists for enrollees with
special healthcare needs.
31 Compliance
The BHO monitors the availability of direct access to specialists.
Reviewer Determination
● Not Met (fail)
Recommendations Requiring CAP
KCBHO does not have a policy and procedure on direct access to specialists for enrollees with special
healthcare needs.
The BHO needs to develop a policy and procedure regarding direct access to specialists for
enrollees with special healthcare needs.
KCBHO does not monitor the availability of direct access to specialists.
KCBHO needs to add criteria to its monitoring tool to assess availability of direct access to
specialists.
32 Compliance
Section 3: Coverage and Authorization of Services
Table B-4: Summary of Compliance Review for Coverage and Authorization of Services
Protocol Section CFR Result
Coverage and Authorization of Services
Coverage 438.210 (a) ● Fully Met (pass)
Authorization of Services 438.210 (b) ● Fully Met (pass)
Notice and Timeliness of Adverse Benefit
Determination
438.210 (c), 438.404 ● Fully Met (pass)
Timeframe for Decisions, Standard and
Expedited
438.210 (d)(1–2) ● Fully Met (pass)
Compensation for Utilization Management
Activities
438.210 (e) ● Fully Met (pass)
Emergency and Post-Stabilization Services 438.114 ● Fully Met (pass)
Overall Result for Section 3. ● Fully Met (pass)
Coverage: 438.210 (a)
Scoring Criteria
The BHO monitors to ensure that services are provided in an amount, duration, and scope
sufficient to achieve the purpose for which they are provided.
The BHO does not arbitrarily deny or reduce the amount, duration, or scope of a required service
solely because of diagnosis, type of illness, or condition of the beneficiary.
The BHO applies the State’s standard for medical necessity when monitoring provided services
and making authorization decisions.
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO’s policy on outpatient level of care services includes the State’s standard for medical
necessity for both mental health and SUD treatment services.
KCBHO’s care managers are licensed mental health professionals. The reviewers for SUD
treatment services are chemical dependency professionals (CDPs).
All requests for services are sent electronically to the BHO, which uses an algorithm based on the
level of care to approve services for the amount, duration, and scope sufficient to achieve the
purpose for which they are requested.
Authorization of Services: 438.210 (b)
Scoring Criteria
33 Compliance
The BHO has documented policies and procedures for the consistent application of review criteria
pertaining to the initial and continuing authorization of services.
The BHO consults with the requesting BHA when appropriate.
The BHO has a process to ensure that any decision to deny a service authorization request or to
authorize a service in an amount, duration, or scope that is less than requested is made by a
mental health or chemical dependency professional who has appropriate clinical expertise in
treating the enrollee's condition or disease.
In the event of an inpatient stay, only a psychiatrist or clinical psychologist can issue a denial for
inpatient psychiatric services.
Reviewer Determination
● Fully Met (pass)
Strengths
When problematic patterns of service use are identified, such as multiple uses of crisis services,
KCBHO’s care managers step in to manage and implement service plans. When needed,
KCBHO’s medical director assists in managing complex and sensitive enrollee cases.
KCBHO’s policy Client Services 3B Review Inter-Rater Reliability describes the procedure for
validating inter-rater reliability.
Notice and Timeliness of Adverse Benefit Determination: 438.210 (c), 438.404
Scoring Criteria
The BHO has a documented policy and procedure to notify the requesting BHA and the enrollee in
writing of any decision by the BHO to deny an initial or continuing service authorization request, or
to authorize a service in an amount, duration, or scope that is less than requested.
The BHO ensures the notice meets the requirements of §438.404.
Reviewer Determination
● Fully Met (pass)
Meets Criteria
Timeframe for Decisions, Standard and Expedited: 438.210 (d)(1–2)
Scoring Criteria
The BHO has a documented policy and procedure for standard and expedited authorization
decisions. The BHO has a process for tracking standard and expedited authorization decisions.
The BHO has mechanisms in place to ensure compliance with standard and expedited
authorization timeframes.
Reviewer Determination
● Fully Met (pass)
Strength
Interviews with KCBHO’s BHAs indicated that the turnaround time for standard authorizations is
well within three days and that decisions are usually made within 24 hours.
34 Compliance
Compensation for Utilization Management Activities: 438.210 (e)
Scoring Criteria
The BHO has a documented policy and procedure specifying that compensation to individuals or
entities that conduct utilization management activities is not structured so as to provide incentives
for the individual or entity to deny, limit, or discontinue medically necessary services to any
enrollee.
The BHO has mechanisms in place to ensure BHAs and/or utilization management contractors do
not provide staff with incentives to deny, limit, or discontinue medically necessary services.
Reviewer Determination
● Fully Met (pass)
Strength
KCBHO’s contracts and policies and procedures confirm that provider compensation is not
structured to provide incentives for providers to deny, limit, or discontinue medically necessary
services to enrollees.
Emergency and Post-stabilization Services: 438.114
Scoring Criteria
The BHO has written policies and procedures pertaining to crisis, stabilization, and post-hospital
follow-up services.
The BHO pays for treatment of conditions defined in its policies as urgent or emergent conditions.
The BHO tracks and monitors to ensure that there is no payment denial for crisis services.
The BHO tracks and monitors use of crisis services for inappropriate or avoidable use related to
access to routine care.
Reviewer Determination
● Fully Met (pass)
Strength
KCBHO has several robust policies and procedures for crisis response, evaluation and treatment,
and stabilization services. The BHO does not require authorization for these services, and the
policies state that these services are available at no cost to the enrollee.
35 Compliance
Section 4: Provider Selection
Table B-5: Summary of Compliance Review for Provider Selection
Protocol Section CFR Result
Provider Selection
Credentialing and Re-credentialing 438.214 (a),(b),(e) ● Partially Met (pass)
Nondiscrimination of Providers 438.214 (c), 438.12 ● Fully Met (pass)
Excluded Providers 438.214 (d) ● Partially Met (pass)
Overall Result for Section 4. ● Partially Met (pass)
Credentialing and Re-credentialing: 438.214 (a),(b),(e)
Scoring Criteria
The BHO has a credentialing and re-credentialing policy and procedure for providers who have
signed contracts or participation agreements.
The BHO has a documented process for credentialing.
The BHO has a documented process for re-credentialing.
The BHO annually monitors the credentialing and re-credentialing process.
The BHO ensures the BHAs have credentialing and re-credentialing polices and processes in
place.
Reviewer Determination
● Partially Met (pass)
Strengths
KCBHO’s Credentialing and Contract Monitoring policy was adopted on March 1, 2016, and
became effective on April 1, 2016. The policy indicates that provider and subcontractor
credentialing/re-credentialing is conducted prior to contracting for the ensuing year.
BHAs seeking participation in the KCBHO network must fill out a credentialing application, which
is then reviewed by the BHO’s contracting review team. Additional review steps include an on-
site visit with the provider by the contract monitors. The on-site review includes:
o a facility walkthrough
o review of encounter data
o review of clinical files
o review of policies and procedures
o review of utilization management/quality improvement plans
During an interview with a contracted BHA, the BHA indicated that it applied for a contract with
the BHO in 2016. The BHA described the application process as extremely comprehensive; the
agency was requested to provide evidence of licensure, accreditation, relevant policies and
36 Compliance
procedures, insurance, clinician qualifications, and certification. The BHA also underwent a re-
credentialing process at contract renewal.
Recommendation Requiring CAP
During the on-site interview, the BHO indicated it is not monitoring its BHAs to ensure they have a policy
and procedure on credentialing and re-credentialing.
KCBHO needs to add criteria to its monitoring tool in order to review each BHA’s credentialing
and re-credentialing policy and procedure.
Nondiscrimination of Providers: 438.214 (c), 438.12
Scoring Criteria
The BHO has provider selection policies and procedures, consistent with §438.12, that ensure the
BHO does not discriminate against particular providers that serve high-risk populations or
specialize in conditions that require costly treatment.
The BHO has policies and procedures in place that ensure the BHO does not discriminate in the
participation, reimbursement or indemnification of any provider who is acting within the scope of
their license or certification, solely on the basis of that license or certification.
The BHO has a process for notifying individuals or groups of providers when they are not chosen
for participation in the network.
Reviewer Determination
● Fully Met (pass)
Strength
KCBHO’s credentialing and contract monitoring policy and procedure indicates the BHO reviews
applications and that providers who:
o serve high-risk populations or specialize in conditions that require costly treatment will not be
discriminated against by KCBHO in its application review process
o are acting within the scope of their license or certification under applicable State law shall not
be discriminated against by KCBHO in its application review process with regard to the
provider’s participation, reimbursement, or indemnification based solely on that license or
certification.
Excluded Providers: 438.214 (d)
Scoring Criteria
The BHO has a policy and procedure to ensure the BHO does not employ or contract with
providers excluded from participation in federal healthcare programs.
The BHO can demonstrate the process and the documentation to determine whether individuals
or organizations are excluded providers.
The BHO ensures it is not employing or appointing on the governing board a person with
beneficial ownership of more than five percent of the BHO’s equity.
The BHO's provider contracts include the provision that providers do not knowingly have a
director, officer, partner, or other person excluded from participation in federal healthcare
programs with a beneficial ownership of more than five percent of the agency's equity.
Reviewer Determination
● Partially Met (pass)
37 Compliance
Recommendation Requiring CAP
At the time of the 2017 EQR, the BHO indicated it had created an internal process to ensure monthly
exclusion checks were performed on all staff, including its BHAs. However, it was determined during the
on-site visit that the BHO is not performing monthly Office of the Inspector General (OIG) List of Excluded
Individuals and Entities (LEIE) checks on its entire staff at the BHO, including the county executives, BHO
leadership, interns, volunteers, and staff who make authorization decisions.
KCBHO needs to ensure the BHO as well as the BHAs are performing exclusion checks on a
monthly basis and at the time of hire on all staff, including county executives, BHO leadership,
board members, custodial staff, and volunteers.
38 Compliance
Section 5: Subcontractual Relationships and Delegation
Table B-6: Summary of Compliance Review for Subcontractual Relationships and Delegation
Protocol Section CFR Result
Subcontractual Relationships and Delegation
Subcontractual Relationships and
Delegation
438.230 (a–c) ● Partially Met (pass)
Subcontractual Relationships and Delegation: 438.230 (a–c)
Scoring Criteria
The BHO has policies and procedures for oversight and accountability for any functions and
responsibilities it delegates to its subcontractors/BHAs.
The BHO performs pre-delegation assessments of contracted BHAs before delegation is granted
on the subcontractor's ability to perform the activities to be delegated.
The BHO has written contracts/agreements that address the specifics of the activities that have
been delegated to the subcontractor/BHA.
The BHO monitors the subcontractor's performance on an ongoing basis and subjects it to formal
review according to a periodic schedule established by the State, consistent with industry
standards or State laws and regulations.
The BHO includes in the delegation contract/agreement that the BHO is responsible for monitoring
and reviewing the subcontractor's/BHA's performance on an ongoing basis and provides criteria
for revoking delegation or imposing other sanctions if the subcontractor's performance is
inadequate.
The BHO initiates a corrective action if subcontractor/BHA performance does not meet industry
standards and/or requirements in the delegation or contract agreement.
The BHO follows up on any corrective action plan given to its subcontractor to ensure it has met
the criteria and, if the criteria is not met, imposes further corrective action, including revoking the
delegated activity.
The BHO requires the BHAs to follow the same CFR criteria for any services the BHAs delegate to
other entities.
The BHO monitors the BHAs’ delegated agreements with other entities.
Reviewer Determination
● Partially Met (pass)
Strengths
The BHO monitors delegated services of its BHAs and their delegates and assigns CAPs when
appropriate. KCBHO provided evidence that the BHO follows through on the CAPs it assigns to
its BHAs.
KCBHO maintains a matrix of all delegated functions and has effective mechanisms to monitor
the performance of those functions.
Recommendations Requiring CAP
KCBHO’s BHA contracts do not contain language holding the BHAs’ subcontractors accountable for
delegated services.
39 Compliance
KCBHO needs to add language to its BHA contracts that clarifies that BHA subcontractors are
accountable for any delegated services.
KCBHO does not include delegation monitoring in its annual QA evaluation.
KCBHO needs to add delegation monitoring to its annual QA evaluation.
40 Compliance
Section 6: Practice Guidelines
Table B-7: Summary of Compliance Review for Practice Guidelines
Protocol Section CFR Result
Practice Guidelines
Adoption of Practice Guidelines 438.236 (a–b) ● Fully Met (pass)
Dissemination of Guidelines 438.236 (c) ● Not Met (fail)
Application of Guidelines 438.236 (d) ● Fully Met (pass)
Overall Result for Section 6. ● Partially Met (pass)
Adoption of Practice Guidelines: 438.236 (a–b)
Scoring Criteria
The BHO has documented policies and procedures related to adoption of practice guidelines,
including consultation with contracting healthcare professionals.
The BHO’s guidelines are based on valid and reliable clinical evidence or a consensus of
healthcare professionals in the particular field.
The BHO has documentation of its enrollees’ needs and how the guidelines fit those needs.
The BHO has documentation that the guidelines are reviewed and updated yearly.
The BHO has a documented policy and procedure for how affiliated BHAs are consulted as
guidelines are adopted and re-evaluated.
Reviewer Determination
● Fully Met (pass)
Strengths
KCBHO has imbedded its practice guidelines selection criteria in its policy Quality Management
of the King County Behavioral Health Organization Plan, Including Extraordinary Occurrences.
KCBHO has several practice guidelines, including:
o Diagnosis-Specific Practice Guidelines
o Children’s Wraparound Practice Guidelines
o Developmental Practice Guidelines for Children ages 0–21
o Practice Protocols for Recovery and Resiliency
KCBHO has been meeting with its BHAs to discuss the development of a co-occurring disorder
(COD) treatment guideline to meet the needs of its enrollees who have a COD diagnosis. The
guideline has been reviewed at KCBHO’s Clinical Directors Group and is still in its early stages.
41 Compliance
Dissemination of Guidelines: 438.236 (c)
Scoring Criteria
The BHO has a policy and procedure on how to disseminate practice guidelines to all providers
and, upon request, to enrollees and potential enrollees.
The BHO can demonstrate it has disseminated the practice guidelines to all BHAs and to
enrollees upon request.
Reviewer Determination
● Not Met (fail)
Opportunity for Improvement
Although KCBHO posts its practice guidelines on its website, they are not easy to locate. The titles are
not self-explanatory, and the guidelines are filed in the Quality Management and Extraordinary
Occurrences section of the website as appendices, which could make it very difficult for enrollees and
providers to access them.
KCBHO should consider including “practice guidelines” in the title of the practice guidelines
located on the BHO’s website in order to make them easily identifiable for both enrollees and
providers.
Recommendation Requiring CAP
KCBHO’s policy states that practice guidelines are disseminated to all affected providers and, upon
request, to enrollees, but it does not specify the mechanism or frequency of distribution.
KCBHO needs to include in its policy the mechanism and frequency with which it distributes the
practice guidelines to providers and enrollees.
Application of Guidelines: 438.236 (d)
Scoring Criteria
The BHO has documented policies and procedures as well as documented meeting minutes
demonstrating that decisions for utilization management, enrollee education, coverage of services,
and other areas to which the guidelines apply are consistent with the guidelines.
The BHO annually monitors the effective use of practice guidelines by the BHAs.
The BHO has documentation of the interface process between the QAPI program and the practice
guidelines adoption process.
Reviewer Determination
● Fully Met (pass)
Strength
KCBHO audits the effective use of practice guidelines by the BHAs by monitoring extraordinary
occurrences, appeals, client concerns and grievances, authorization requests, case
consultations, and results of contract compliance site visits and record reviews.
42 Compliance
Section 7: Health Information Systems
Table B-9: Summary of Compliance Review for Health Information Systems
Protocol Section CFR Result
Heath Information Systems
General Rule, Utilization, Claims, Grievances
and Appeals, and Disenrollments
438.242 (a) ● Fully Met (pass)
Basic Elements and Enrollee Encounter Data 438.242 (b),(c) ● Not Met (fail)
Overall Result for Section 8. ● Partially Met (pass)
General Rule, Utilization, Claims, Grievances and Appeals, and Disenrollments:
438.242 (a)
Scoring Criteria
The BHO has a health information system that collects, analyzes, integrates, and reports data on
utilization, dis-enrollments, requests to change providers, grievances, and appeals.
The BHO utilizes reports from health information data to make informed management decisions.
The BHO uses the information it has collected and analyzed to identify trends in areas including
but not limited to utilization, grievances and appeals, dis-enrollments, and requests to change
providers.
The BHO analyzes the health information data to identify services needed for enrollees.
Reviewer Determination
● Fully Met (pass)
Strength
KCBHO monitors the reports produced by the IT department to review access to care standards,
analyze services by diagnosis, review inpatient and outpatient length of stay, study frequency of
opioid treatment services and the service utilization of high-intensity programs, and conduct
studies on over- and underutilization of services.
Basic Elements and Enrollee Encounter Data: 438.242 (b),(c)
Scoring Criteria
The BHO collects data on service encounters and on all provider and enrollee characteristics
included in the Consumer Information System (CIS) Data Dictionary.
The BHO ensures that data received from providers are accurate and complete by collecting data
in standardized formats and reviewing the data for accuracy, timeliness, completeness, logic, and
consistency.
The BHO certifies the data received from the BHAs.
The BHO makes all collected data available to the State and, upon request, to CMS.
Reviewer Determination
● Not Met (fail)
43 Compliance
Recommendations Requiring CAP
The BHO does not have an external policy and procedure to ensure its BHAs are checking their data for
quality and integrity before submitting them to the BHO.
The BHO needs to create and implement a policy and procedure to ensure its BHAs are checking
their data for quality and integrity before submitting them to the BHO. The policy should include:
o the requirement for providers to submit written attestations of data accuracy
o a form letter for providers to complete attesting to data accuracy
o a system for the form letters to be transmitted electronically to the BHO
o monitoring by BHO contract monitors to ensure timely submission of the attestation
letters
KCBHO reported that the SUD treatment BHAs were experiencing challenges submitting data in the file
formats required by the Service Encounter Reporting Instructions (SERI) as this was a new requirement
for the providers. The SUD treatment BHAs were encouraged to use a web portal the BHO designed for
data submission until the challenges were resolved. However, the EQRO was unable to perform EDV at
the time of the review because the SUD treatment BHAs had not submitted all of the required data.
KCBHO needs to continue to train and assist its SUD treatment BHAs to ensure the BHAs can
submit timely, accurate, and complete data.
44 Compliance
Section 8: Quality Assessment and Performance Improvement
Program
Table B-8: Summary of Compliance Review for Quality Assessment and Performance
Improvement Program
General Rules: 438.330 (a)
Scoring Criteria
The BHO has an ongoing quality assessment and performance improvement program (QAPI) for
the services it furnishes to its enrollees.
The BHO has a written QAPI program.
The BHO has a QA and PI process to evaluate the QAPI program and updates its annual plan.
The BHO collects, analyzes, and uses performance data to support its quality assessment and
performance improvement program.
The BHO has a Quality Management Committee that meets regularly, reviews results of
performance data, and reports to the governing board.
The BHO has effective mechanisms to assess the quality and appropriateness of care furnished to
enrollees.
The BHO conducts one clinical performance improvement project, one substance use disorder
performance improvement project, and one non-clinical performance improvement project each
year, one of which is a children-focused performance improvement project.
The BHO ensures its compliance with the State’s quality strategy plan.
Reviewer Determination
● Partially Met (pass)
Strengths
KCBHO developed and implemented its quality management plan for the integration of mental
health and SUD treatment services as a BHO effective April 1, 2016. The plan was reviewed by
the King County Behavioral Health Advisory Board, providers, and other stakeholders and is
monitored and revised by the KCBHO Quality Improvement Committee as needed.
Protocol Section CFR Result
Quality Assessment and Performance Improvement Program
General Rules 438.330 (a) ● Partially Met (pass)
Basic Elements 438.330 (b)(1–4) ● Partially Met (pass)
Performance Measurement 438.330 (c) ● Fully Met (pass)
Performance Improvement Projects 438.330 (d)(1–3) ● Fully Met (pass)
Program Review by the State 438.330 (e) ● Fully Met (pass)
Overall Result for Section 7. ● Partially Met (pass)
45 Compliance
KCBHO monitors its contracted BHAs’ performance on the delivery of services through various
mechanisms to ensure quality care is provided to enrollees, delivery of appropriate services is
continued, and adherence to contract requirements is maintained. Trends that develop for
individual providers and on a system-wide level are analyzed and reported. Areas monitored at
least annually include:
o access
o service availability
o PIHP penetration
o service utilization
o provider caseloads and specialties
KCBHO’s contract monitoring staff provide technical assistance and work with individual
providers to address specific performance issues. Continued inability to meet performance
expectations and/ or comply with corrective actions may lead to further sanctions and ultimately
contribute to a decision by KCBHO not to re-credential the provider.
KCBHO maintains individual provider files, including correspondence, credentialing, and
grievance/complaint information.
Opportunity for Improvement
KCBHO does not comply with the State Quality Strategy as the State does not have a current quality
strategy.
KCBHO will need to comply with the State Quality Strategy once the State has developed and
implemented its plan.
Basic Elements: 438.330 (b)(1–4)
Scoring Criteria
The BHO collects, analyzes, and uses performance data to support its quality assessment and
performance improvement program.
The BHO reports performance data to the State every year.
The BHO has a documented policy and procedure regarding the detection of both underutilization
and overutilization of services in all its programs.
The BHO has consistent criteria for identifying underutilization and overutilization.
The BHO has processes for routine monitoring of underutilization and overutilization.
The BHO has processes for taking corrective action to address underutilization and overutilization.
Reviewer Determination
● Partially Met (pass)
Strength
As a key component of its quality management program, King County BHO routinely monitors a
comprehensive set of process and outcome indicators.
Recommendation Requiring CAP
In the past year, KCBHO has not been able to report all of its SUD treatment performance data to the
State.
46 Compliance
KCBHO needs to continue to monitor and support its SUD treatment BHAs in order to ensure it
receives all performance data.
Performance Measurement: 438.330 (c)
Scoring Criteria
The BHO has a process in place to assess the standard performance measures.
The BHO measures and reports to the State on its performance, using the standard measures
required by the State.
The BHO has processes to submit data to DBHR, which enables the State to calculate the BHO’s
performance using standard measures.
Reviewer Determination
● Fully Met (pass)
Meets Criteria
Performance Improvement Projects: 438.330 (d)(1–3)
Scoring Criteria
The BHO has a process in place to assess the quality and appropriateness of care furnished to
enrollees.
The BHO monitors and tracks the quality and appropriateness of care furnished to enrollees.
The BHO has processes for taking action when quality and appropriateness of care issues are
identified.
Reviewer Determination
● Fully Met (pass)
Meets Criteria
Program Review by the State: 438.330 (e)
Scoring Criteria
The BHO has a process in place to submit its QAPI program evaluation to the State at least
annually.
The BHO monitors and tracks its required performance measures.
The BHO has processes to report outcomes and trended results of its performance improvement
projects.
The BHO has a process in place to evaluate the impact and effectiveness of its own QAPI
program.
Reviewer Determination
● Fully Met (pass)
Strength
The BHO is in the process of finalizing its evaluation of its program for 2016.
47 Performance Improvement Project Validation
Performance Improvement Project (PIP) Validation
Performance improvement projects (PIPs) are designed to assess and improve the processes and
outcomes of the healthcare system. They represent a focused effort to address a particular problem
identified by an organization. As prepaid inpatient health plans (PIHPs), Behavioral Health Organizations
(BHOs) are required to have an ongoing program of PIPs. BHOs must maintain one PIP each in clinical,
non-clinical, and substance use disorder (SUD)-focused areas (one of which must also include a child
focus). The PIPs must involve:
measurement of performance using objective quality indicators
implementation of systems interventions to achieve improvement in quality
evaluation of the effectiveness of the interventions
planning and initiation of activities for increasing or sustaining improvement
As a mandatory EQR activity, Qualis Health evaluates the BHOs’ PIPs to determine whether they are
designed, conducted, and reported in a methodologically sound manner. The PIPs must be designed to
achieve, through ongoing measurements and intervention in clinical and non-clinical areas, significant
improvement sustained over time that is expected to have a favorable effect on health outcomes and
enrollee satisfaction. In evaluating PIPs, Qualis Health determines whether
the study topic was appropriately selected
the study question is clear, simple, and answerable
the study population is appropriate and clearly defined
the study indicator is clearly defined and is adequate to answer the study question
the PIP’s sampling methods are appropriate and valid
the procedures the BHO used to collect the data to be analyzed for the PIP measurement(s) are
valid
the BHO’s plan for analyzing and interpreting PIP results is accurate
the BHO’s strategy for achieving real, sustained improvement(s) is appropriate
it is likely that the results of the PIP are accurate and that improvement is “real”
improvement is sustained over time
Following PIP evaluations, BHOs are offered technical assistance to improve their PIP study methodology
and outcomes. BHOs may resubmit their PIPs up to two weeks following the initial evaluation. PIPs are
assigned a final score following the final submission.
PIP Scoring
Qualis Health assessed the BHOs’ PIPs using the current CMS EQR protocol, available here:
http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Quality-of-Care-
External-Quality-Review.html.
Qualis Health assigns a score of “Met,” “Partially Met,” or “Not Met” to each of the 10 evaluation
components that are applicable to the performance improvement project being evaluated. Components
may be “Not Applicable” if the performance improvement project is at an early stage of implementation.
Components determined to be “Not Applicable” are not reviewed and are not included in the final scoring.
Scoring is based on the answers BHOs provide in the completion of a response form, which address
48 Performance Improvement Project Validation
questions listed under each evaluation component, following a review of written documentation and in-
person interviews. Opportunities for improvement, technical assistance, and recommendations requiring a
corrective action plan (CAP) are provided for each standard.
The table below presents the scoring key for the PIP standards.
Scoring Icon Key
●Fully Met (pass) ●Partially Met (pass) ●Not Met (fail) ●N/A (not applicable)
PIP Validity and Reliability
Qualis Health assesses the overall validity and reliability of the reported results for all PIPs. Because
determining potential issues with the validity and reliability of the PIP is sometimes a judgment call, Qualis
Health reports a level of confidence in the study findings based on a global assessment of study design,
development, and implementation. Levels of confidence and their definitions are included in Table C-2.
Table C-1: Performance Improvement Project Validity and Reliability Confidence Levels
Confidence Level Definition
High confidence in reported results The study results are based on high-quality study design
and data collection and analysis procedures. The study
results are clearly valid and reliable.
Moderate confidence in reported
Results
The study design and data collection and analysis
procedures are not sufficient to warrant a higher level of
confidence. Study weaknesses (e.g., threats to internal or
external validity, barriers to implementation, questionable
study methodology) are identified that may impact the
validity and reliability or reported results.
Low confidence in reported results The study design and/or data collection and analysis
procedures are unlikely to result in valid and reliable study
results.
Not enough time has elapsed to
assess meaningful change
The PIP has not advanced to at least the first re-
measurement of the study indicator.
49 Performance Improvement Project Validation
PIP Validation Results: Clinical PIP
Effectiveness of the Transitional Support Program
KCBHO is continuing its clinical PIP focused on the effectiveness of its Transitional Support Program
(TSP). The purpose of this PIP is to reduce the rate of psychiatric hospital readmissions and length of
stay to ultimately improve utilization and promote good clinical care by increasing focus on efficient and
effective discharge planning and strong connections and engagement in community-based outpatient
services. In 2016, the BHO demonstrated statistically significant improvement with both study indicators
(the number of psychiatric hospitalization admissions and the length of stay for those hospitalizations)
from the baseline measurement to the re-measurement period, one year prior to and one year post
enrollment in the Transitional Support Program. KCBHO reported that TSP staff were able to foster a
relationship between hospital discharge staff and outpatient mental health providers, which led to positive
outcomes as enrollees were able to stabilize in the community because of collaborative discharge
planning and transitional care.
Although the BHO was able to demonstrate sustained improvement through repeated measurements
over time, TSP staff have encountered difficulties implementing the TSP at two particular hospitals.
Throughout the study, TSP staff acknowledged various barriers and challenges working with several
hospitals, as some hospitals proved to be more collaborative than others. Some of these challenges
included difficulty for TSP staff in accessing enrollees and long wait times before access to the enrollee
was granted, struggles connecting with hospital social workers, hassles in obtaining discharge plans, and
lack of initiated contact from the hospital when enrollees were being discharged. In September 2016, the
BHO started regular transition planning meetings with all of the hospitals in the network to standardize
discharge planning practices. Because the two specified hospitals have the highest re-hospitalization
rates within the BHO network, the BHO is seeking to address the barriers and challenges of implementing
the TSP within those facilities. KCBHO is continuing this PIP with a specific focus on quality improvement
for the two hospitals. The BHO will continue to emphasize assisting hospital staff with assessment,
discharge planning, and care coordination of involuntarily detained enrollees. Hence, KCBHO plans to
assess length of stay and readmissions (30-day and 90-day) for a second 12-month re-measurement
period and monitor the proportion of admissions/detentions by hospital.
Study question: “whether the PIP intervention (TSP program) reduces the following 2 PIP indicators to a
statistically significant (p<.05) degree for the eligible PIP (TSP) population:
(1) the average length-of-stay of the hospitalizations in the year following the first TSP service compared
to the average length-of-stay of hospitalizations during the prior year
(2) the average number of hospitalizations in the year following the first TSP service compared to the year
prior to first TSP service.”
Dates of study period:
The baseline period will cover the target population's hospitalizations during the year prior to TSP
enrollment in the date range of July 1, 2013, through March 30, 2015. For example, if a person enrolled in
a TSP on July 1, 2014, their baseline measurement year would be July 1, 2013, through June 30, 2014,
whereas a person enrolled March 31, 2015, would have a baseline year of March 31, 2014, through
March 30, 2015.
The re-measurement period will cover the target population's hospitalization during the year following the
detention in which TSP enrollment occurs. For example, if a person was enrolled in a TSP on July 1,
50 Performance Improvement Project Validation
2014, their re-measurement year would begin as soon as they were discharged from that episode of
detention. If the person was discharged on July 2, 2014, the re-measurement period would be July 2,
2014, through July 1, 2015.
An additional re-measurement period will include an analysis extending the re-measurement period to a
second year following the detention in which TSP enrollment occurred to determine if improvements were
sustained over time through March 31, 2017. KCBHO is seeking to review the variation between hospitals
and determine whether the two hospitals with high barriers to access and high re-hospitalization rates
have improved from year one post- to year two post-first TSP service. Furthermore, the BHO will also
assess the TSP enrollment rate.
(KCBHO did not provide a specific study end date; Qualis Health recommended during the on-site review
that KCBHO implement the new intervention with the two hospitals identified and then retire the PIP in
2017.)
Table C-2: Clinical PIP Validation Results
Study Design
Activity
SCORE
Design 1 Appropriate study topic ● Partially Met (pass)
2 Clearly defined, answerable study question ● Partially Met (pass)
3 Correctly identified study population ● Fully Met (pass)
4 Correctly identified study indicator ● Fully Met (pass)
Implementation 5 Valid sampling technique ●N/A
6 Accurate/complete data collection ● Fully Met (pass)
7 Appropriate data analysis/
interpretation of study results ● Fully Met (pass)
Outcomes 8 Appropriate improvement strategies ● Fully Met (pass)
9 Real improvement achieved ●N/A
10 Sustained improvement achieved ●N/A
Overall Score
Confidence Level:
High confidence in reported results
● Partially Met (pass)
Reviewer Comments:
KCBHO previously completed a rendition of this PIP but identified barriers and challenges with
implementing the Transitional Support Program in two specific hospitals. As a result, the BHO
51 Performance Improvement Project Validation
recognized the need to continue its evaluation activities and quality improvement efforts to increase TSP
penetration rates and lower psychiatric readmissions at these facilities. KCBHO identified these two
hospitals as also having the highest re-hospitalization rates within the provider network. High numbers
of potentially preventable events can indicate deficiencies in quality of care. The BHO should implement
its new strategy at these facilities and then retire the PIP in its current format. As KCBHO monitors these
interventions, it is important to pay close attention to the organizational change process and the degree
to which it is affecting the specific intervention. During implementation, the BHO and key stakeholders
should make several checks to ensure that the hospitals are successfully integrating the changes.
Further, once the re-measurement is complete, if rates remain suboptimal, the BHO should continue its
efforts internally.
Standard 1: Selected Study Topic Is Relevant and Prioritized
Table C-3: Validation of PIP Selected Study Topic
Criterion Description Result
1.1 The study topic was selected through a comprehensive process
that involved data collection and analysis of enrollee needs,
care, and services.
● Fully Met (pass)
Reviewer Comments:
The study topic was selected after a review of KCBHO data related to involuntary hospitalizations and
readmission rates. There was no review of national or state data or of historical trends related to this issue.
KCBHO’s initial reasoning for pursuing this PIP was based on a 10-year decline in King County and
statewide in inpatient psychiatric hospital beds. The BHO focused its efforts on addressing the need to
reduce psychiatric hospital readmissions and lengths of stay with the use of the Transitional Support
Program. TSP goals are to improve enrollee transitions from the inpatient hospital setting to other care
settings, overall improving the quality of care. KCBHO later identified a need to continue this PIP as a
result of the distinct barriers and challenges the TSP team faced when working with two particular
hospitals.
1.2 The PIP is consistent with the demographics and epidemiology
of the enrollees. ●Not Met (fail)
Reviewer Comments:
KCBHO has not updated its data to demonstrate that this PIP is still consistent with the demographics and
epidemiology of the enrollees. The BHO presented the same enrollee data as during the 2016 EQR, which
consisted of demographics from July 2013 to June 2014. During this timeframe, King County crisis and
commitment staff administered 3,194 detentions; of these 1,391 were Medicaid-enrolled adults over the
age of 18 at the time of their hospital admission or within 90 days of discharge.
1.3 Input from enrollees, family members, peers, and/or advocates
was considered during the selection of the PIP. ● Fully Met (pass)
Reviewer Comments:
At the onset of this PIP, KCBHO solicited input from numerous stakeholders, including community hospital
staff, inclusive of social workers and utilization review coordinators, the King County Clinical Director’s
group, provider agency CEOs, and the internal RSN Quality Improvement Group. The BHO also sought
peer involvement to gain an understanding of how the use of peers can be beneficial to hospital discharge
assistance and outpatient engagement.
For the continuation of the PIP, King County BHO and the Transitional Support Program team held
meetings with hospital administrators and conducted a Prevention Task Force meeting, and the BHO
52 Performance Improvement Project Validation
medical director chaired meetings with the clinical directors as well as hospital leadership.
1.4 The PIP addresses a broad spectrum of key aspects of enrollee
care and services. ● Fully Met (pass)
Reviewer Comments:
This PIP addresses a specified high-need subpopulation as identified by KCBHO. The BHO believes
enrollees who become involuntarily detained due to danger to themselves or others who are gravely
disabled and are not already engaged in outpatient services have the greatest need. The strategies are
expected to impact a broad range of enrollee care and services by bringing forth more effective and
efficient care management approaches to stabilize the hospitalized enrollee quickly, hence reducing the
lengths of stay and rates of psychiatric hospital readmissions.
Opportunity for Improvement:
KCBHO did not provide updated data on enrollee demographics and epidemiology. Per CMS, the topic
should be “consistent with demographic and epidemiologic information of the current enrollees.”
KCBHO needs to provide current and relevant data on enrollee demographics and epidemiology.
Standard 2: Study Question Is Clearly Defined
Table C-4: Validation of PIP Study Question
Criterion Description Result
2.1 The study question(s) is clear, concise, and answerable. ●Not Met (fail)
Reviewer Comments:
KCBHO states it is assessing “whether the PIP intervention (TSP program) reduces the following 2 PIP
indicators to a statistically significant (p<.05) degree for the eligible PIP (TSP) population:
(1) the average length-of-stay of the hospitalizations in the year following the first TSP service compared
to the average length-of-stay of hospitalizations during the prior year
(2) the average number of hospitalizations in the year following the first TSP service compared to the year
prior to first TSP service”
The BHO did not construct its study question in the form of a question, but instead created statements
regarding potential outcomes of the intervention.
2.2 The study question sets the framework for goals, data
collection, analysis, and interpretation. ● Fully Met (pass)
Reviewer Comments:
The statements identify the focus of the PIP and set the framework for study aims, as well as for collecting
and analyzing the data.
2.3 The study question includes the intervention, the study
population (denominator), what is being measured (numerator),
a metric (percent or average), and a desired outcome.
● Fully Met (pass)
Reviewer Comments:
KCBHO describes the intervention as the Transitional Support Program. The study population is defined
as Medicaid enrollees, who are at least 18 years old, who are involuntarily detained at one of the two
identified hospitals and not successfully engaged in outpatient mental health services. One numerator is
the average difference in individual lengths of stay one year before and one year after initiation of TSP
enrollment. The second numerator is the average of the differences for each enrollee between the number
53 Performance Improvement Project Validation
of hospitalizations before and after the start of enrollment in the TSP. The metric and the desired outcome
for both indicators reduce the indicators by a statistically significant degree, p<.05.
Recommendation Requiring CAP:
Per the EQR Protocol 3: Validating Performance Improvement Projects, version 2.0, September 2012, the
criteria for determining the adequacy of the study question is The study question(s) should be clear,
simple, and answerable. In addition, they should be stated in a way that supports the ability to determine
whether the intervention has a measurable impact for a clearly defined population.
KCBHO did not state the focus of its study in the form of a question. While the BHO does provide
statements describing what it plans to achieve through the intervention, it is not posed in a clear, simple,
and answerable format.
KCBHO needs to formulate a study question that is truly in the structure of a question.
Standard 3: Study Population Is Clearly Defined, and, if a Sample Is Used, Appropriate
Methodology Is Used
Table C-5: Validation of PIP Study Population
Criterion Description Result
3.1 The enrollee population to whom the study question and indicator
are relevant is clearly defined. ● Fully Met (pass)
Reviewer Comments:
The study population is defined as Medicaid enrollees served by a TSP at the two hospitals from July 1,
2014, to March 31, 2017. In order to be served by a TSP, an enrollee must be at least 18 years old,
involuntarily committed to a community psychiatric hospital, and not successfully engaged in outpatient
publicly funded mental health services. Not engaged is defined as having fewer than six hours of
outpatient service in the previous 60 days.
3.2 The inclusion or exclusion criterion, if applicable, is clearly
defined. ●N/A
Reviewer Comments:
KCBHO notes that there is no formal inclusion or exclusion criterion.
3.3 The study population is reflective of the entire Medicaid enrollee
population to which the study indicator applies, or a sample is
used.
● Fully Met (pass)
Reviewer Comments:
KCBHO’s defined study population is the same as the Medicaid enrollee population to which the study
indicator applies.
3.4 Data collection approaches ensured all required information was
captured for all enrollees to whom the study question applied. ● Fully Met (pass)
Reviewer Comments:
All study population data are recorded in KCBHO’s electronic management information system (MIS). All
Medicaid enrollee TSP authorizations, lengths of stay, admissions, and discharges to psychiatric hospitals
are documented in the MIS. Additionally, KCBHO reports that DBHR data regarding admissions and
discharges from the state hospital are routinely updated. Medicaid eligibility files are received monthly from
the State.
Meets Criteria
54 Performance Improvement Project Validation
Standard 4: Study Indicator Is Objective and Measureable
Table C-6: Validation of PIP Study Indicator
Criterion Description Result
4.1 The study includes a clear description of the study indicator(s) and
clearly defined numerator and denominator. ● Fully Met (pass)
Reviewer Comments:
KCBHO will continue to study the original two PIP indicators—the annual average length of stay of
hospitalizations and annual average number of hospitalizations for an individual enrolled in a TSP at the
two hospitals. Additionally, the BHO will examine the rate of TSP enrollment by hospital over time from
July 1, 2014, to March 31, 2017, in six-month intervals.
The numerator is the number of TSP authorizations in a given six-month time period.
The denominator is the number of detentions for that time period.
4.2 The study includes an explanation of how the indicators are
appropriate and adequate to answer the study question, and
describes how the indicator objectively measures change to
impact the enrollee.
● Fully Met (pass)
Reviewer Comments:
The original PIP study indicators, average length of stay for psychiatric hospitalizations and number of
psychiatric hospitalizations pre and post intervention, as well as the new indicator, TSP enrollment by
hospital over time, measure change in a clear manner. All indicators are unbiased quantitative measures
that can assess the impact of the Transitional Support Program on enrollees.
4.3 There is a clear and realistic plan that includes where and how the
data on the indicator are collected, all of the elements of the data
collection plan are in place and viable, and mitigation strategies
are in place in case sufficient data are not able to be collected.
● Fully Met (pass)
Reviewer Comments:
KCBHO is capturing all data relevant to this PIP in its MIS. This includes all Medicaid enrollee TSP
authorizations, hospital admissions, and discharges. Data obtained from DBHR regarding enrollees who
have been hospitalized in the state hospital are routinely integrated into the MIS.
4.4 The baseline and first and second re-measurement periods are
unambiguously stated and appropriate in length. ● Fully Met (pass)
Reviewer Comments:
The measurement periods are articulated in a manner that is clear and unambiguous:
The baseline period covers the study population's hospitalizations during the year prior to TSP enrollment,
sometime between July 1, 2013, and March 30, 2015. For example, an individual enrolled in a TSP on July
1, 2014, would have a baseline year of July 1, 2013, through June 30, 2014; a person enrolled on March
31, 2015, would have a baseline year of March 31, 2014, through March 30, 2015.
Re-measurement periods covered the target population's hospitalizations during the year following the
detention in which TSP enrollment occurred, i.e., July 2, 2014, through March 3, 2016. For example, an
individual enrolled in a TSP on July 1, 2014, would have a re-measurement period upon discharge from
the hospital, which could be July 2, 2014, through July 1, 2015; a person enrolled in TSP on March 31,
2015, would have a re-measurement year of April 1, 2015, through March 31, 2016.
55 Performance Improvement Project Validation
An additional re-measurement period will include an analysis extending the re-measurement period to a
second year following the detention in which TSP enrollment occurred to determine whether improvements
were sustained over time through March 31, 2017. KCBHO is seeking to review the variation between
hospitals and find whether the two hospitals with high barriers to access and high re-hospitalization rates
have improved from year one post- to year two post-first TSP service.
Meets Criteria
Standard 5: Sampling Method
Table C-7: Validation of PIP Sampling Methods
Criterion Description Result
5.1 The method for defining and calculating the sample
size, the true and estimated frequency of the event, the
confidence level, and the acceptable margin error are
specified and clearly stated.
●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
5.2 The sampling technique is described, and whether the
sample is a probability or non-probability sample is
specified.
●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
5.3 Valid sampling techniques are employed to protect
against bias. ●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
5.4 The sample contains a sufficient number of enrollees. ●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
N/A
Standard 6: Data Collection Procedure
Table C-8: Validation of PIP Data Collection Procedures
Criterion Description Result
6.1 The study design clearly specifies the data to be collected. ● Fully Met (pass)
Reviewer Comments:
KCBHO plans to conduct an assessment of performance improvement efforts at the two identified
hospitals, which were initiated as a result of the barriers to hospitalized enrollee access and coordinated
discharge planning. To complete this assessment, KCBHO will examine the study indicators (the number
56 Performance Improvement Project Validation
of psychiatric hospitalization admissions and the length of stay for those hospitalizations) as well as the
TSP enrollment rate (number of TSP enrolled/total detentions) by hospital and look for improvement over
time in six-month intervals, particularly for the two hospitals with the highest re-hospitalization rates in the
first 12 months of TSP service. This PIP focuses on Medicaid enrollees who are Medicaid eligible at the
time of admission into the hospital or within 90 days of discharge. TSP enrollment is defined as an
individual who is at least 18 years old and has a TSP program authorization in KCBHO’s MIS.
6.2 The study design clearly specifies the sources of data. ● Fully Met (pass)
Reviewer Comments:
Medicaid enrollment is determined from Medicaid eligibility files received from the State on a monthly basis
and incorporated into the BHO’s MIS. TSP authorizations, psychiatric hospitalization admissions by type,
voluntary or involuntary, and hospital discharges are captured in the MIS. Additional information regarding
hospitalizations in the state hospital is routinely gathered from DBHR and integrated into the MIS.
6.3 The study design includes a description of the data collection
methods used, including the types of data collected, an
explanation of how the methods elicit valid and reliable data, the
intervals at which the data will be collected and, if HEDIS or other
formal methodology is used, a description of the process.
● Fully Met (pass)
Reviewer Comments:
KCBHO’s data collection is automated electronically. Medicaid eligibility is downloaded from the State to
the BHO’s MIS on a monthly basis using an automatic process. TSP enrollment data are uploaded to the
BHO’s MIS by the provider agency on a nightly basis. On a monthly basis, validation of TSP enrollment
occurs by comparing monthly caseload reports sent to the BHO by the provider with the information in the
MIS. Study indicator data are uploaded on a nightly basis to the MIS from the hospital and the crisis clinic.
State hospital data are downloaded into the BHO’s MIS each night. BHO staff reconcile hospital data with
State payment data monthly in order to validate the accuracy the data.
6.4 The study design includes a description of the instruments used
for data collection, including a narrative regarding how the
instrument provides for consistent and accurate data collection
over the time periods studied. If any additional documentation was
requested, it was provided and appropriate.
● Fully Met (pass)
Reviewer Comments:
All data necessary for this PIP are maintained in KCBHO’s MIS. A query based on MIS program
authorization pulls data on the TSP population, including Medicaid status. Baseline and re-measurement
hospitalization information for the specified population is also sorted. A Fourth Generation Programming
Language (4GL) code is used to extract the data. KCBHO provided a copy of the query used to extract the
data.
6.5 The study states who will be collecting the data, and includes their
qualifications to collect the data. ● Fully Met (pass)
Reviewer Comments:
KCBHO staff involved in collecting the data elements include a King County senior application developer
with King County IT, who has over a decade of programming experience. This individual extracts the data
from KCBHO’s MIS.
The initial data analysis steps were completed by a past KCBHO program evaluator who held a PhD in
psychology and had over 20 years of clinical research, program evaluation, and quality improvement
experience.
57 Performance Improvement Project Validation
The repeat analysis will be completed by KCBHO’s current performance measurement evaluator, who
holds a PhD in health services research and has over 15 years of research, program evaluation, and
quality improvement experience.
6.6 The study includes a description of how inter-rater reliability is
ensured. ●N/A
Reviewer Comments:
Not applicable; this PIP uses only administrative data.
Meets Criteria
Standard 7: Data Analysis and Interpretation of Study Results
Table C-9: Validation of PIP Data Analysis and Interpretation
Criterion Description Result
7.1 There is a clear description of the data analysis plan that includes
the type of statistical analysis used and the confidence level, and
the analysis was performed according to the plan.
● Fully Met (pass)
Reviewer Comments:
The BHO plans to use paired two-tailed t-tests to analyze the data for both indicators. For each participant,
the number of psychiatric hospitalizations and lengths of stay pre and post intervention will be compared.
A p<.05 probability/confidence level will be used to assess statistical significance. The PIP has not
progressed to the point of data analysis and interpretation.
7.2 Numerical PIP results and findings are accurately and clearly
presented. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.3 The data analysis methodology is appropriate to the study question
and data types. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.4 The analysis identified statistical significance of differences
between initial and repeat measurements, and was performed
correctly.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.5 If threats to internal or external validity were identified, the potential
impact and resolution was explained. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.6 The analysis of study data includes an interpretation of the extent to
which the PIP was successful, statistically significant or otherwise,
as well as a description of follow-up activities.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
Technical Assistance:
Valid data interpretation occurs when the analysis is carried out as planned. Results should be displayed
58 Performance Improvement Project Validation
in an easily understood format such as a table or graph with clear measurement periods, outcomes, and
other points of comparison clearly shown. The analysis should include a comparison of the initial and
repeat measurements, and any threats to validity should be noted. KCBHO should include in the
discussion whether progress toward the PIP’s goal was made and details regarding any follow-up actions.
Standard 8: Appropriate Improvement Strategies
Table C-10: Validation of PIP Improvement Strategies
Criterion Description Result
8.1 Steps were taken to identify improvement opportunities during
the PIP process. ● Fully Met (pass)
Reviewer Comments:
Since the initiation and re-measurement periods of the PIP, KCBHO performed an analysis on the
challenges and barriers in working with hospitals identified by TSP staff. It was discovered that two
hospitals in particular posed the majority of challenges and perpetual barriers. These included difficulty
with physical access to hospitalized enrollees and long wait times (typically 30–45 minutes) before being
granted access to the enrollees; communication challenges with hospital social workers regarding the
coordination of discharge plans and referrals; difficulty obtaining discharge plans (especially those with
sufficient detail), medical records, or prescription information from these facilities; and lack of notification
by the hospital when an enrollee was being discharged. The BHO recognizes that deficient discharge
planning and care coordination can impact ongoing enrollee treatment and quality of care because these
enrollees are not connected to outpatient services and are further likely to have challenges obtaining
medications in an outpatient setting.
The Transitional Support Program team determined that there was a correlation between excessive
hospital administrative staff turnover and the shortage of information and awareness regarding the TSP.
As a result, TSP staff access to hospitalized enrollees was also hindered. The lack of discharge resources,
such as access to ProviderOne, which enables providers to look up enrollee Medicaid eligibility, was also
identified as a challenge.
8.2 Reasonable interventions were undertaken to address
causes/barriers identified through data analysis and QI
processes.
● Fully Met (pass)
Reviewer Comments:
Through a thorough analysis of the identified improvement opportunities, KCBHO undertook many
interventions, specifically with the two hospitals that presented the most challenges with TSP access.
These facilities had the highest 90-day re-hospitalization rates as well. In an effort to enhance
collaboration, KCBHO met with the leadership of one of the hospitals and developed solutions that allowed
TSP staff to obtain quicker access to hospitalized enrollees; however, the staff were first required to
receive a flu vaccine and complete a TB test. Only a small number of the TSP staff agreed to these
stipulations. Additionally, the BHO staff began working together with King County hospitals on discharge
planning and relationship building. Although the TSP targets Medicaid enrollees and those eligible for
Medicaid, hospital staff at a few hospitals were unable to look up enrollees using ProviderOne due to lack
of access. KCBHO has remedied this by searching for individuals in ProviderOne to determine eligibility for
the Transitional Support Program.
TSP staff have also implemented some changes to enhance the PIP initiative. To increase knowledge and
59 Performance Improvement Project Validation
awareness of the program, they launched a marketing campaign, which included brochures and providing
education to new hospital administrative staff. They have been able to form connections by strengthening
rapport with hospital administrative staff, which has aided in their ability to gain access to hospitalized
enrollees. Last, in the absence of discharge plans, TSP staff have been able to create enrollee transition
plans.
8.3 The interventions are/were sufficient to be expected to improve
processes or outcomes. ● Fully Met (pass)
Reviewer Comments:
The implementation of the identified solutions is expected to improve processes and outcomes for the
Transitional Support Program. KCBHO reports that it is still in the process of assessing the connection
between the TSP team and hospital discharge planners from these two hospitals.
8.4 The interventions are/were culturally and linguistically
appropriate. ● Fully Met (pass)
Reviewer Comments:
KCBHO’s Transitional Support Program is based on the Association for Professionals in Infection Control
and Epidemiology (APIC) and Coleman models of care. Both models were developed and used in
culturally diverse settings. Additionally, the BHO states that the TSP team works with its clients in a
linguistically and culturally appropriate manner by utilizing translators and cultural specialty workers as
needed. Members of the TSP team work with enrolled clients who have been discharged from the hospital
to link them to outpatient care and other kinds of 24-hour crisis services available within the BHO in order
to avoid re-hospitalization.
Meets Criteria
Standard 9: Assess Whether Improvement Is “Real” Improvement
Table C-11: Validation of PIP Improvement Assessment
Criterion Description Result
9.1 The same methodology used for the baseline measurement was
used when measurement was repeated. ●N/A
Reviewer Comments:
Although the PIP has not progressed to the point at which improvement can be assessed, KCBHO has
indicated it will be using the same methodology for the second re-measurement period. Additionally, it will
conduct a data analysis by re-evaluating the baseline and the first re-measurement period data to ensure
that the methodology is consistent.
9.2 There is a description of the data analysis regarding improvements
in process or outcomes of care. ●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
9.3 There is an evaluation demonstrating that reported improvement in
performance appears to be the result of the planned quality
improvement intervention, or an analysis related to why there was
not improvement.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
60 Performance Improvement Project Validation
9.4 There is statistical evidence that any observed performance
improvement is true improvement, and statistical analysis was
performed thoroughly and accurately.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
Technical Assistance:
Whether a change does or does not occur, KCBHO must assess components of the PIP to determine
whether the change or lack of change is attributable to an event unrelated to the intervention, random
chance, or to the intervention. Evaluation of results is the focus of this step.
Standard 10: The BHO Has Sustained the Documented Improvement
Table C-12: Validation of PIP Sustained Improvement
Criterion Description Result
10.1 Sustained improvement was demonstrated through repeated
measurements over comparable time periods. If improvement
was not sustained, there is an explanation and an indicated plan
for next steps.
●N/A
Reviewer Comments:
Once KCBHO has completed the implementation of the quality improvement efforts at the two hospitals, it
should be able to address this step. The ultimate goal with this step is to demonstrate that sustained
improvement is achieved. However, if improvement is not continual, an evaluation of the PIP needs to be
conducted, and the BHO will need to consider a plan for retirement.
N/A
61 Performance Improvement Project Validation
PIP Validation Results: Non-Clinical Children’s PIP
Improved Coordination with Primary Care for Children and Youth
KCBHO is continuing its non-clinical PIP focused on improving coordination with primary care providers
for Medicaid children and adolescents.
Washington State’s Apple Health managed care organization (MCO) contracts set forth the requirement
that MCOs coordinate care for enrollees, including care coordination and data sharing between the MCOs
and the BHOs. With the implementation of this PIP, KCBHO seeks to better identify and coordinate care
for Medicaid youth dually enrolled with the BHO and Molina Healthcare Apple Health who have a high
utilization of acute care, whether medically or psychiatrically related. Previously, KCBHO sought to
conduct this PIP utilizing data from the five MCOs in the network; however, the BHO experienced difficulty
obtaining data from all of the MCOs. Thus, KCBHO is focusing its efforts on working with one MCO
(Molina) to ensure that care for their dually enrolled enrollees is coordinated, with an aim to make sure
these enrollees receive the right care at the right time, while avoiding unnecessary duplication of services.
The BHO has identified a care coordination process to improve enrollee care. This process includes 1)
identifying those with highest emergency department (ED) utilization and shared cases with Molina; 2)
identifying a treatment team; 3) selecting a lead entity (MCO or BHO) that develops initial care guidelines
for ED providers if the youth presents at an ED again; 4) if the youth presents at an ED, notifying the lead
(through the Emergency Department Information Exchange [EDIE]/PreManage system), who provides an
immediate response or intervention for the identified youth; 5) the lead coordinating with the team working
with the child/family to discuss the shared intervention plan; 6) the lead convening care coordination
meetings to agree on treatment goals, review gaps in care, collaborate around identified issues, and
develop a joint family-driven care plan. The intervention also includes an information systems component
of sharing the care plan and patient records on EDIE/PreManage for the BHO, the MCO, and providers to
access. BHO, MCO, emergency department staff and other treatment providers will be trained to check
the treatment plan on EDIE/PreManage when interacting with the youth and their family. KCBHO has
indicated it will implement this process with the other MCOs once it has deemed the new process
successful.
Study question:
“Does implementation of a BHO-MCO care coordination intervention* significantly reduce psychiatrically
related ED visits for Medicaid-enrolled children/youth (age <=18) in both ongoing outpatient BHO mental
health services and Molina Apple Healthcare and who had at least one psychiatrically related ED visit
during this period when comparing baseline to the re-measurement period?”
* The intervention is defined as being a shared case review and care coordination intervention with
Molina Healthcare Apple Health (including shared treatment plans on an information system accessible
by both the BHO/Molina and ER provider staff). The interventions may include Wraparound or CCORS
[Children’s Crisis Outreach Response System] (depending on the identified need).
Dates of study period:
Baseline measurement: December 1, 2015, through November 30, 2016
Re-measurement periods: September 2017, December 2017, and June 2018 (3, 6, and 12 months post
intervention)
62 Performance Improvement Project Validation
Table C-13: Non-Clinical PIP Validation Results
Study Design
Activity
SCORE
Design 1 Appropriate study topic ● Partially Met (pass)
2 Clearly defined, answerable study question ● Fully Met (pass)
3 Correctly identified study population ● Fully Met (pass)
4 Correctly identified study indicator ● Fully Met (pass)
Implementation 5 Valid sampling technique ●N/A
6 Accurate/complete data collection ● Fully Met (pass)
7 Appropriate data analysis/
interpretation of study results ●N/A
Outcomes 8 Appropriate improvement strategies ●N/A
9 Real improvement achieved ●N/A
10 Sustained improvement achieved ●N/A
Overall Score
Confidence Level:
Not enough time has elapsed to assess meaningful
change
● Partially Met (pass)
Reviewer Comments:
Because of KCBHO’s difficulty in obtaining data from all five of the Apple Health MCOs, this PIP was in
a very early stage for several years. KCBHO has followed through on last year’s recommendation to
fully formulate and begin the PIP. With baseline measurement and re-measurement periods redefined,
KCBHO should be able to progress further with collecting, reporting, and analyzing the data for youth
dually enrolled in the BHO and Molina Healthcare Apple Health. KCBHO indicated it has now
succeeded in procuring the essential data from Medicaid eligibility files and EDIE/PreManage, allowing
the BHO to collect and analyze the information necessary to interpret whether the identified
interventions will achieve the desired outcomes.
63 Performance Improvement Project Validation
Standard 1: Selected Study Topic Is Relevant and Prioritized
Table C-14: Validation of PIP Selected Study Topic
Criterion Description Result
1.1 The study topic was selected through a comprehensive process
that involved data collection and analysis of enrollee needs,
care, and services.
● Fully Met (pass)
Reviewer Comments:
This study topic focuses on improving processes and outcomes of healthcare provided by the BHO.
KCBHO selected this PIP through a comprehensive process, which involved discussions with numerous
stakeholders as well as review of acute care utilization for children and youth continuously enrolled in
outpatient, crisis, or hospital BHO services. The BHO also conducted an analysis of Molina Healthcare’s
Apple Health data. In assessing emergency department (ED) utilization for a period of six months, from
December 2014 to May 2015, the BHO found that of the 2,202 youth shared by the BHO and Molina
Healthcare Apple Health, 321 were involved in 443 ED events. Of the 443 ED events, 109 encounters
from 87 youth indicated a primary or secondary psychiatric diagnosis, translating to 25 percent of the
subpopulation having at least one psychiatrically related ED visit. KCBHO’s analysis of this data showed
that youth who were utilizing the emergency department were not having their needs met through routine
outpatient services. KCBHO determined this topic would be beneficial to pursue because it identifies a
group at particularly high risk of continued utilization and negative outcomes, and it dovetails with
healthcare integration efforts that afford new opportunities for identifying the population, coordinating care,
and tracking outcomes.
1.2 The PIP is consistent with the demographics and epidemiology
of the enrollees. ● Partially Met (pass)
Reviewer Comments:
KCBHO provided an initial data analysis for youth dually enrolled with the BHO and Molina. The data from
the date range of August 2015 through January 2016 showed that 113 MCO/BHO shared youth with a
psychiatric diagnosis or asthma had 160 ED visits. When assessing the needs of these youth, the BHO
found that 22 were already enrolled in BHO services.
The BHO solely reviewed data specific to individuals dually enrolled with KCBHO and Molina. It did not
include any overall local, state, or national data related to the study population. Based on the data that
were submitted, it is not clear whether the PIP is truly consistent with the demographics and healthcare
needs of King County’s enrollees.
1.3 Input from enrollees, family members, peers, and/or advocates
was considered during the selection of the PIP. ● Fully Met (pass)
Reviewer Comments:
KCBHO held various discussions with relevant stakeholders before choosing this topic. The BHO’s Youth
Clinical Directors group and Parent Partner Network group provided input and noted that coordination
between BHAs and primary care providers was a challenge. The Parent Partner Network participants
indicated that parents and youth find it challenging to navigate the physical healthcare and mental health
systems, especially when a child has a psychiatric crisis or a medical issue that appears to be
psychiatrically related.
1.4 The PIP addresses a broad spectrum of key aspects of enrollee
care and services. ● Fully Met (pass)
Reviewer Comments:
This study topic is meant to address high utilizers of acute care mental health services. The intervention
64 Performance Improvement Project Validation
will focus on preventative care and care coordination, conducted with the hospital and outpatient systems,
to address this high-risk, high-need population.
Opportunity Improvement:
KCBHO did not provide updated data on enrollee demographics and epidemiology. Per CMS, the topic
should be “consistent with demographic and epidemiologic information of the current enrollees.”
KCBHO needs to provide current and relevant data on enrollee demographics and epidemiology.
This should include overall standardized local, state, or national data linked to the study
population.
Standard 2: Study Question Is Clearly Defined
Table C-15: Validation of PIP Study Question
Criterion Description Result
2.1 The study question(s) is clear, concise, and answerable. ● Fully Met (pass)
Reviewer Comments:
KCBHO’s PIP study question is “Does implementation of a BHO-MCO care coordination intervention*
significantly reduce psychiatrically-related ED visits for Medicaid-enrolled children/youth (age <=18) in both
ongoing outpatient BHO mental health services and Molina Apple Healthcare and who had at least one
psychiatrically-related ED visit during this period when comparing baseline to the re-measurement
period?”
*The intervention is defined as being a shared case review and care coordination intervention with Molina
Healthcare Apple Health (including shared treatment plans on an information system accessible by both
the BHO/Molina and ER provider staff). The interventions may include Wraparound or CCORs (depending
on the identified need).
The study question is stated in a manner that is straightforward and answerable.
2.2 The study question sets the framework for goals, data
collection, analysis, and interpretation. ● Fully Met (pass)
Reviewer Comments:
The study question establishes a framework for the goal, data collection, and analysis of reducing ED
visits for Medicaid youth who are dually enrolled with KCBHO and Molina.
2.3 The study question includes the intervention, the study
population (denominator), what is being measured (numerator),
a metric (percent or average), and a desired outcome.
● Fully Met (pass)
Reviewer Comments:
KCBHO has stated its intervention as a shared case review and care coordination with Molina (including
shared treatment plans on an information system accessible by both BHO/Molina and ER provider staff).
Additionally, the BHO may utilize Wraparound or CCORs (depending on the identified need of the
Medicaid enrollee).
The study population is defined as Medicaid-enrolled youth 18 years of age and younger who are
continuously enrolled in both outpatient, crisis, or hospital BHO services and Molina Healthcare’s Apple
Health plan who have had the highest amount of hospital ED use—typically five or more ED visits per
year.
65 Performance Improvement Project Validation
The numerator is the average difference in the matched scores from baseline to re-measurement. This is
calculated by measuring the sum of the difference in the number of ED visits for the measurement periods
before and after the intervention divided by the total number of enrollees with pre- and post-intervention
measurement scores.
The denominator is the total number of individuals with pre- and post-measurement scores.
Meets Criteria
Standard 3: Study Population Is Clearly Defined, and, if a Sample is Used, Appropriate
Methodology Is Used
Table C-16: Validation of PIP Study Population
Criterion Description Result
3.1 The enrollee population to whom the study question and indicator
are relevant is clearly defined. ● Fully Met (pass)
Reviewer Comments:
KCBHO has defined the study population as Medicaid-enrolled youth 18 years of age and younger who
are continuously enrolled in both outpatient, crisis, or hospital BHO services and Molina Healthcare’s
Apple Health plan from November 2015 through November 2016 who have had the highest amount of ED
use during this time period (typically five or more ED visits per year).
3.2 The inclusion or exclusion criterion, if applicable, is clearly defined. ● Fully Met (pass)
Reviewer Comments:
KCBHO had previously defined the criterion as “psychiatrically related” ED visits, which meant a visit in
which an individual receives a primary or secondary psychiatric diagnosis, including DSM codes 290–390
(ICD-10 codes starting with F) or asthma code J450505. This criterion has been removed from the
description of the PIP study population.
No exclusions were reported.
3.3 The study population is reflective of the entire Medicaid enrollee
population to which the study indicator applies, or a sample is used. ● Fully Met (pass)
Reviewer Comments:
The study population and the Medicaid enrollee population are the same.
3.4 Data collection approaches ensured all required information was
captured for all enrollees to whom the study question applied. ● Fully Met (pass)
Reviewer Comments:
KCBHO reported that the data sources used to identify the PIP study population included two sets of
administrative data: Medicaid eligibility files and Emergency Department Information Exchange
(EDIE)/PreManage.
Medicaid eligibility files are obtained by the BHO from DBHR on a monthly basis. These files contain all
Medicaid-enrolled individuals for a given month, the BHO they are affiliated with, and the MCO in which
the individual enrolled. The BHO will look expressly at individuals enrolled with Molina.
66 Performance Improvement Project Validation
EDIE/PreManage contains hospital ED utilization data across the state of Washington. It identifies and
provides information on high-risk patients in real time. It is also a web-based communication tool to assist
with care coordination between different systems of healthcare.
A file of shared King County BHO-MCO clients is uploaded to EDIE/PreManage monthly, and reports can
be generated for the Molina-BHO clients.
Meets Criteria
Standard 4: Study Indicator Is Objective and Measureable
Table C-17: Validation of PIP Study Indicator
Criterion Description Result
4.1 The study includes a clear description of the study indicator(s) and
clearly defined numerator and denominator. ● Fully Met (pass)
Reviewer Comments:
The study indicator for this PIP is the change in the number of ED visits for the PIP study population.
The numerator is the average difference in the matched scores from baseline to re-measurement. This is
calculated by measuring the sum of the difference in the number of ED visits for the measurement periods
before and after the intervention divided by the total number of enrollees with pre- and post-intervention
measurement scores.
The denominator is the total number of individuals with pre- and post-measurement scores.
4.2 The study includes an explanation of how the indicators are
appropriate and adequate to answer the study question, and
describes how the indicator objectively measures change to
impact the enrollee.
● Fully Met (pass)
Reviewer Comments:
KCBHO indicated it is using one key indicator to answer the question of whether the intervention will
reduce the number of ED visits for its chosen study population. It is an impartial, quantifiable way to
assess the impact of the implemented interventions.
4.3 There is a clear and realistic plan that includes where and how the
data on the indicator are collected, all of the elements of the data
collection plan are in place and viable, and mitigation strategies
are in place in case sufficient data are not able to be collected.
● Fully Met (pass)
Reviewer Comments:
KCBHO reported challenges obtaining data from all five of the MCOs; thus, the BHO considered
alternative means of accessing enrollee healthcare data as well as hospital ED data. As a result, King
County BHO has contracted with PreManage to provide client lists to match with hospital ED utilization
data. Using Medicaid eligibility files and the information obtained using EDIE/PreManage, the BHO is able
to access and analyze the data necessary for the study indicator.
4.4 The baseline and first and second re-measurement periods are
unambiguously stated and appropriate in length. ● Fully Met (pass)
Reviewer Comments:
KCBHO has chosen December 1, 2015, through November 30, 2016, for its baseline measurement period
67 Performance Improvement Project Validation
and September 2017, December 2017, and June 2018 (3, 6, and 12 months post intervention), for its re-
measurement periods.
Meets Criteria
Standard 5: Sampling Method
Table C-18: Validation of PIP Sampling Methods
Criterion Description Result
5.1 The method for defining and calculating the sample
size, the true and estimated frequency of the event, the
confidence level, and the acceptable margin error are
specified and clearly stated.
●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
5.2 The sampling technique is described, and whether the
sample is a probability or non-probability sample is
specified.
●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
5.3 Valid sampling techniques are employed to protect
against bias. ●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
5.4 The sample contains a sufficient number of enrollees. ●N/A
Reviewer Comments:
Not applicable; no sampling was used for this PIP.
N/A
Standard 6: Data Collection Procedure
Table C-19: Validation of PIP Data Collection Procedures
Criterion Description Result
6.1 The study design clearly specifies the data to be collected. ● Fully Met (pass)
Reviewer Comments:
The data elements that will be collected for this PIP include:
enrollment in Molina Healthcare’s Apple Health Plan
continuous enrollment in Molina and the BHO during the measurement periods
BHO and MCO enrollees age 18 or under as of November 30, 2016
ED visits for the PIP population during the baseline period of December 1, 2015, through
November 30, 2016, and the re-measurement periods of September 2017, December 2017, and
June 2018 (3, 6, and 12 months post intervention)
6.2 The study design clearly specifies the sources of data. ● Fully Met (pass)
68 Performance Improvement Project Validation
Reviewer Comments:
The data sources for this PIP will include the administrative data from Medicaid eligibility files and
Emergency Department Information Exchange (EDIE)/PreManage.
6.3 The study design includes a description of the data collection
methods used, including the types of data collected, an
explanation of how the methods elicit valid and reliable data, the
intervals at which the data will be collected and, if HEDIS or other
formal methodology is used, a description of the process.
● Fully Met (pass)
Reviewer Comments:
Medicaid eligibility files are obtained by KCBHO from DBHR on a monthly basis. These files
display all Medicaid enrollees for the given month, BHO affiliation, and with which MCO the
individual is enrolled.
KCBHO is specifically looking at children and adolescents who are identified as being enrolled
with Molina.
Per KCBHO, the reliability/validity for this data is used by the State for Medicaid reconciliation and
penetration analyses and BHO payment, which is considered the “gold standard” for reliability.
EDIE/PreManage contains ED utilization data across the state of Washington. It identifies and
provides information on high-risk patients in real time. It is also a web-based communication tool
to assist with care coordination between different healthcare systems.
Per KCBHO, it is reliable and valid and used in several states.
Data collection processes:
1. From the monthly Medicaid eligibility files, the BHO extracts those individuals enrolled with Molina
during each of the 12 months of the baseline period.
2. From the BHO enrollment data, KCBHO then uploads a file containing this data to
EDIE/PreManage.
3. ED visits are identified and reported in EDIE/PreManage. A query is conducted to obtain ED
utilization reports.
4. KCBHO will utilize the report generated from EDIE/PreManage to analyze ED use at 3, 6, and 12
months post intervention.
6.4 The study design includes a description of the instruments used
for data collection, including a narrative regarding how the
instrument provides for consistent and accurate data collection
over the time periods studied. If any additional documentation was
requested, it was provided and appropriate.
● Fully Met (pass)
Reviewer Comments:
The monthly Medicaid eligibility files include all Medicaid enrollees and the MCO with which the individual
is enrolled. Using these files as a basis, KCBHO queries the baseline for 12 months of files to extract
enrollees who are continuously enrolled in the BHO and Molina within all months of the baseline
measurement period. This extraction produces a file of individuals continuously enrolled in both systems
during the 12-month baseline period. A list of continuously enrolled youth is created by subtracting their
date of birth from November 30, 2016, to establish those who are 18 years old or younger. KCBHO’s
Performance Measurement Evaluator will generate a report from EDIE/PreManage to analyze ED use at
3, 6, and 12 months post intervention. Through the uses of STATA (a data analysis and statistical software
program), the evaluator may conduct further analysis.
6.5 The study states who will be collecting the data, and includes their
qualifications to collect the data. ● Fully Met (pass)
Reviewer Comments:
69 Performance Improvement Project Validation
KCBHO staff involved in collecting the data elements include a King County senior application developer
with King County IT, who has over a decade of programming experience. This individual extracts the data
from the monthly Medicaid eligibility files.
The subsequent analysis will be completed by KCBHO’s current performance measurement evaluator,
who holds a PhD in health services research and has over 15 years of research, program evaluation, and
quality improvement experience.
6.6 The study includes a description of how inter-rater reliability is
ensured. ●N/A
Reviewer Comments:
Not applicable; this PIP uses only administrative data.
Meets Criteria
Standard 7: Data Analysis and Interpretation of Study Results
Table C-20: Validation of PIP Data Analysis and Interpretation
Criterion Description Result
7.1 There is a clear description of the data analysis plan that includes
the type of statistical analysis used and the confidence level, and
the analysis was performed according to the plan.
● Fully Met (pass)
Reviewer Comments:
KCBHO indicated it will use a paired two-tailed t-test to compare the average number of hospital ED visits
during the baseline period to the number during the re-measurement period. The total number of ED visits
for each individual during the baseline and re-measurement periods will be entered in STATA for
evaluation.
7.2 Numerical PIP results and findings are accurately and clearly
presented. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.3 The data analysis methodology is appropriate to the study question
and data types. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.4 The analysis identified statistical significance of differences
between initial and repeat measurements, and was performed
correctly.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.5 If threats to internal or external validity were identified, the potential
impact and resolution was explained. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.6 The analysis of study data includes an interpretation of the extent to
which the PIP was successful, statistically significant or otherwise,
as well as a description of follow-up activities.
●N/A
70 Performance Improvement Project Validation
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
Technical Assistance:
Valid data interpretation occurs when the analysis is carried out as planned. Results should be displayed
in an easily understood format. The analysis should include a comparison of the initial and repeat
measurements, and any threats to validity should be noted. KCBHO should include in the discussion
whether progress toward the PIP’s goal was made and details regarding any follow-up actions.
Standard 8: Appropriate Improvement Strategies
Table C-21: Validation of PIP Improvement Strategies
Criterion Description Result
8.1 Steps were taken to identify improvement opportunities during
the PIP process. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
8.2 Reasonable interventions were undertaken to address
causes/barriers identified through data analysis and QI
processes.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
8.3 The interventions are/were sufficient to be expected to improve
processes or outcomes. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
8.4 The interventions are/were culturally and linguistically
appropriate. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
Technical Assistance:
An improvement strategy is an intervention created to impact a cause/barrier in the PIP process. The
intent of the PIP is to implement real, sustained improvement through the ongoing use of a Plan, Do,
Study, Act (PDSA) problem-solving model. The interventions need to be sufficient to be expected to
improve processes or outcomes, and they must be culturally and linguistically appropriate.
Standard 9: Assess Whether Improvement Is “Real” Improvement
Table C-22: Validation of PIP Improvement Assessment
Criterion Description Result
9.1 The same methodology used for the baseline measurement was
used when measurement was repeated. ●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
71 Performance Improvement Project Validation
9.2 There is a description of the data analysis regarding improvements
in process or outcomes of care. ●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
9.3 There is an evaluation demonstrating that reported improvement in
performance appears to be the result of the planned quality
improvement intervention, or an analysis related to why there was
not improvement.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
9.4 There is statistical evidence that any observed performance
improvement is true improvement, and statistical analysis was
performed thoroughly and accurately.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
Technical Assistance:
Whether a change does or does not occur, KCBHO must assess components of the PIP to determine
whether the change or lack of change is attributable to an event unrelated to the intervention, random
chance, or to the intervention. Evaluation of results is the focus of this step.
Standard 10: The BHO Has Sustained the Documented Improvement
Table C-23: Validation of PIP Sustained Improvement
Criterion Description Result
10.1 Sustained improvement was demonstrated through repeated
measurements over comparable time periods. If improvement
was not sustained, there is an explanation and an indicated plan
for next steps.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which sustained improvement can be assessed.
Technical Assistance:
This step, specifically, should not be addressed until KCBHO has completed its PIP. The ultimate goal of
the PIP is to achieve sustained improvement; however, if improvement is not sustained, an evaluation of
the PIP will be conducted and a plan for further action or retirement will need to be assessed by the BHO.
72 Performance Improvement Project Validation
PIP Validation Results: Substance Use Disorder (SUD) PIP
SUD Residential Treatment Length of Stay
KCBHO identified a new SUD PIP topic in early 2017 and is still in the early stages of development and
implementation. The BHO intends to examine SUD residential lengths of stay from the initial authorization
through continued stay extension requests (re-authorizations). With the integration of mental health and
SUD treatment services in April 2016, KCBHO became responsible for the authorization and utilization
management of residential SUD treatment services. Because overseeing and authorizing SUD residential
treatment is a new business for the BHO and is a very costly service, KCBHO sought to assess lengths of
stay to determine if the BHO is authorizing SUD treatment for appropriate amounts of time. Enrollees who
are identified as needing residential treatment are among the most severely ill; thus, the BHO sees
importance in authoring appropriate lengths of stay at the onset to fully address the treatment needs of
these enrollees. KCBHO aims to provide the right treatment and the right amount of treatment at the right
time to those it serves.
Additionally, KCBHO seeks to minimize readmissions as well as reduce and/or prevent the number of
stay extensions. The BHO has not yet analyzed its SUD residential treatment data to determine where
the need exists.
Study question: “After implementation of an improved SUD residential treatment authorization process,
will we be able to reduce the number of stay extensions (reauthorizations) by X percent and rapid
readmissions (90-day readmission rate) by Y percent for adults (ages 18+) with Medicaid who are
seeking residential drug addiction treatment in King County?”
Dates of study period: January 2017 to October 2019
Table C-24: SUD PIP Validation Results
Study Design
Activity
SCORE
Design 1 Appropriate study topic ● Partially Met (pass)
2 Clearly defined, answerable study question ● Partially Met (pass)
3 Correctly identified study population ●N/A
4 Correctly identified study indicator ●N/A
Implementation 5 Valid sampling technique ●N/A
6 Accurate/complete data collection ●N/A
7 Appropriate data analysis/
interpretation of study results ●N/A
73 Performance Improvement Project Validation
Outcomes 8 Appropriate improvement strategies ●N/A
9 Real improvement achieved ●N/A
10 Sustained improvement achieved ●N/A
Overall Score
Confidence Level:
Not enough time has elapsed to assess meaningful
change
● Partially Met (pass)
Reviewer Comments:
KCBHO is still in the early stages of this PIP and does not have the relevant data to determine whether
a problem truly exists with the current authorization process as it relates to SUD residential treatment
lengths of stay. Once the data are obtained, the BHO should consider homing in on a particular focus
area, such as discharge planning and post-discharge linkages for continuity of care. This will also aid
the BHO in developing its intervention.
Standard 1: Selected Study Topic Is Relevant and Prioritized
Table C-25: Validation of PIP Selected Study Topic
Criterion Description Result
1.1 The study topic was selected through a comprehensive process
that involved data collection and analysis of enrollee needs,
care, and services.
● Partially Met (pass)
Reviewer Comments:
The BHO is conducting a data analysis project to better understand how length of stay influences
treatment outcomes, and what other factors may influence lengths of stay, so the PIP is still in the
development process. KCBHO authorization staff and quality managers have shared initial feedback from
SUD residential treatment providers, who indicated the current initial care authorizations for SUD inpatient
treatment lengths of stay (15 days) may be too short.
1.2 The PIP is consistent with the demographics and epidemiology
of the enrollees. ● Fully Met (pass)
Reviewer Comments:
KCBHO preliminarily looked at data on Medicaid enrollees who obtained a substance use disorder
residential treatment authorization for the timeframe between July 1, 2016, and December 31, 2016.
During this time the BHO provided authorizations for 667 enrollees for 728 admissions to residential
treatment for substance use disorders. There were about 60 readmissions during this six-month time
period. Of the total authorizations, 64 percent were granted stay extensions, and, on average, there were
1.4 stay extensions per client for those who needed one, with a range of stay extensions from zero to five.
Those in the residential SUD treatment population represent about 14 percent of the Medicaid clients who
engaged in SUD treatment services in King County between July and December 2016.
1.3 Input from enrollees, family members, peers, and/or advocates
was considered during the selection of the PIP. ●Not Met (fail)
74 Performance Improvement Project Validation
Reviewer Comments:
In the selection of this study topic, the BHO did not involve stakeholders, including enrollees. KCBHO
indicated it intends to seek enrollee input as part of the process once it has established the methodology
for soliciting feedback from enrollees.
1.4 The PIP addresses a broad spectrum of key aspects of enrollee
care and services. ● Partially Met (pass)
Reviewer Comments:
Enrollees who access SUD residential treatment have a high severity of drug use, and some are at high
risk for death due to overdose. KCBHO is seeking to understand how lengths of stay impact treatment
outcomes and what factors may affect optimal lengths of stay.
The BHO also identified the potential to “decrease administrative burden.” PIPs should not focus on
administrative fixes, but rather a process that will directly affect an enrollee’s experience with their SUD
treatment.
Opportunity for Improvement:
To determine the appropriateness of this study topic, the BHO should seek input from enrollees who have
received SUD residential treatment services. KCBHO has identified a plan to incorporate the input of
enrollees.
Standard 2: Study Question Is Clearly Defined
Table C-26: Validation of PIP Study Question
Criterion Description Result
2.1 The study question(s) is clear, concise, and answerable. ● Partially Met (pass)
Reviewer Comments:
The study question is “After implementation of an improved SUD residential treatment authorization
process, will we be able to reduce the number of stay extensions (reauthorizations) by X percent and rapid
readmissions (90 day readmission rate) by Y percent for adults (ages 18+) with Medicaid who are seeking
residential drug addiction treatment in King County?”
In its current format, the study question is not answerable, as the BHO has not defined specific metrics for
X and Y. The BHO should narrow its focus so the study question is concise and easily answerable.
2.2 The study question sets the framework for goals, data
collection, analysis, and interpretation. ●Not Met (fail)
Reviewer Comments:
The study question is not stated in a way that supports the ability to determine whether the intervention
has a measurable impact for the defined population.
2.3 The study question includes the intervention, the study
population (denominator), what is being measured (numerator),
a metric (percent or average), and a desired outcome.
● Partially Met (pass)
Reviewer Comments:
KCBHO needs to define the percentages for X and Y. The BHO should also define how extending the
original length of stay authorization impacts the readmission rate.
Opportunity for Improvement:
75 Performance Improvement Project Validation
In order to have a clear, concise, and answerable study question that will set the framework for the PIP,
KCBHO must first clarify the exact issue it wishes to focus on improving and then determine what
intervention will be used to achieve the desired outcome. Currently, the scope of the study question is too
large, and there is not a clear correlation between the initial authorization and rapid readmissions.
Furthermore, without any data it is difficult to discern if this is an appropriate study question.
Standard 3: Study Population Is Clearly Defined, and, if a Sample Is Used, Appropriate
Methodology Is Used
Table C-27: Validation of PIP Study Population
Criterion Description Result
3.1 The enrollee population to whom the study question and indicator
are relevant is clearly defined. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining its study population.
3.2 The inclusion or exclusion criterion, if applicable, is clearly
defined. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining its study population.
3.3 The study population is reflective of the entire Medicaid enrollee
population to which the study indicator applies, or a sample is
used.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining its study population.
3.4 Data collection approaches ensured all required information was
captured for all enrollees to whom the study question applied. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining its study population.
Technical Assistance:
KCBHO should clearly articulate the study population. The basis for any inclusions or exclusions should be
stated in a straightforward manner. The BHO needs to ensure that it has the capability to properly identify
individuals within the study population and that it can collect the required data.
Standard 4: Study Indicator Is Objective and Measureable
Table C-28: Validation of PIP Study Indicator
Criterion Description Result
4.1 The study includes a clear description of the study indicator(s) and
clearly defined numerator and denominator. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining a study indicator.
4.2 The study includes an explanation of how the indicators are
appropriate and adequate to answer the study question, and ●N/A
76 Performance Improvement Project Validation
describes how the indicator objectively measures change to
impact the enrollee.
Reviewer Comments:
The PIP has not progressed to the point of defining a study indicator.
4.3 There is a clear and realistic plan that includes where and how the
data on the indicator is collected, all of the elements of the data
collection plan are in place and viable, and mitigation strategies
are in place in case sufficient data is not able to be collected.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining a study indicator.
4.4 The baseline and first and second re-measurement periods are
unambiguously stated and appropriate in length. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of defining a study indicator.
Technical Assistance:
KCBHO needs to ensure that as it develops the study indicator, it includes a defined numerator and
denominator. The indicator should be objective, clear, and unambiguous, as well as actionable and
reliable. The BHO should include an explanation of how the indicators are appropriate and adequate to
answer the study question and describe how the indicator objectively measures change to impact the
enrollee.
Standard 5: Sampling Method
Table C-29: Validation of PIP Sampling Methods
Criterion Description Result
5.1 The method for defining and calculating the sample
size, the true and estimated frequency of the event, the
confidence level and the acceptable margin error are
specified and clearly stated.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of choosing a sampling method.
5.2 The sampling technique is described, and whether the
sample is a probability or non-probability sample is
specified.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of choosing a sampling method.
5.3 Valid sampling techniques are employed to protect
against bias. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of choosing a sampling method.
5.4 The sample contains a sufficient number of enrollees. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of choosing a sampling method.
Technical Assistance:
77 Performance Improvement Project Validation
If KCBHO uses a sampling technique, it must ensure that the sample is representative of the entire eligible
population. Sampling methods should be in line with generally accepted principles of research design and
statistical analysis.
Standard 6: Data Collection Procedure
Table C-30: Validation of PIP Data Collection Procedures
Criterion Description Result
6.1 The study design clearly specifies the data to be collected. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data collection.
6.2 The study design clearly specifies the sources of data. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data collection.
6.3 The study design includes a description of the data collection
methods used, including the types of data collected, an
explanation of how the methods elicit valid and reliable data, the
intervals at which the data will be collected and, if HEDIS or other
formal methodology is used, a description of the process.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data collection.
6.4 The study design includes a description of the instruments used
for data collection, including a narrative regarding how the
instrument provides for consistent and accurate data collection
over the time periods studied. If any additional documentation was
requested, it was provided and appropriate.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data collection.
6.5 The study states who will be collecting the data, and includes their
qualifications to collect the data. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data collection.
6.6 The study includes a description of how inter-rater reliability is
ensured. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data collection.
Technical Assistance:
In order to have accurate and valid study results, data must be properly collected. The data analysis plan
should specify what data are to be collected; what the data sources are; how and when the data will be
collected; who will collect the data, including verification that they are qualified to collect the data; and
identification of any tools used to collect data. KCBHO should consider all the information needed in order
to identify the right study population, calculate the study indicator, analyze the results meaningfully, and
establish the validity and reliability of the data. The process should be simple, easy, and clear.
78 Performance Improvement Project Validation
Standard 7: Data Analysis and Interpretation of Study Results
Table C-31: Validation of PIP Data Analysis and Interpretation
Criterion Description Result
7.1 There is a clear description of the data analysis plan that includes
the type of statistical analysis used and the confidence level, and
the analysis was performed according to the plan.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.2 Numerical PIP results and findings are accurately and clearly
presented. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.3 The data analysis methodology is appropriate to the study question
and data types. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.4 The analysis identified statistical significance of differences
between initial and repeat measurements, and was performed
correctly.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.5 If threats to internal or external validity were identified, the potential
impact and resolution was explained. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
7.6 The analysis of study data includes an interpretation of the extent to
which the PIP was successful, statistically significant or otherwise,
as well as a description of follow-up activities.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of data analysis and interpretation.
Technical Assistance:
An analysis plan should describe the schedule for data collection and analysis. KCBHO would benefit from
using a simple “task-by-timeline” format, as results should be displayed in an easily understood design.
The analysis should include a comparison of the initial and repeat measurements, and any threats to
validity should be noted. Interpretation involves looking at all the possible explanations for results and
factors that may have affected them, not just assuming that the intervention itself was the only operative
factor.
79 Performance Improvement Project Validation
Standard 8: Appropriate Improvement Strategies
Table C-32: Validation of PIP Improvement Strategies
Criterion Description Result
8.1 Steps were taken to identify improvement opportunities during
the PIP process. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
8.2 Reasonable interventions were undertaken to address
causes/barriers identified through data analysis and QI
processes.
●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
8.3 The interventions are/were sufficient to be expected to improve
processes or outcomes. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
8.4 The interventions are/were culturally and linguistically
appropriate. ●N/A
Reviewer Comments:
The PIP has not progressed to the point of identifying improvement strategies.
Technical Assistance:
An improvement strategy is an intervention created to impact a cause/barrier in the PIP process. A barrier
analysis should be conducted to derive the improvement strategies to be implemented. KCBHO should
consider the following sources for potential interventions: stakeholder input, research studies, promising
practices, and data obtained from high-performing SUD treatment providers throughout Washington.
Standard 9: Assess Whether Improvement Is “Real” Improvement
Table C-33: Validation of PIP Improvement Assessment
Criterion Description Result
9.1 The same methodology used for the baseline measurement was
used when measurement was repeated. ●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
9.2 There is a description of the data analysis regarding improvements
in process or outcomes of care. ●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
9.3 There is an evaluation demonstrating that reported improvement in
performance appears to be the result of the planned quality
improvement intervention, or an analysis related to why there was
not improvement.
●N/A
Reviewer Comments:
80 Performance Improvement Project Validation
The PIP has not progressed to the point at which improvement can be assessed.
9.4 There is statistical evidence that any observed performance
improvement is true improvement, and statistical analysis was
performed thoroughly and accurately.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which improvement can be assessed.
Technical Assistance:
When a change in performance occurs, it must be determined whether the change is real, attributable to
an event unrelated to the intervention, or random chance. Whether a change does or does not occur,
KCBHO must assess components of the PIP to determine whether the change or lack of change is
attributable to the actual intervention. Evaluation of results is the focus of this step to determine if the
results are statistically significant.
Standard 10: The BHO Has Sustained the Documented Improvement
Table C-34: Validation of PIP Sustained Improvement
Criterion Description Result
10.1 Sustained improvement was demonstrated through repeated
measurements over comparable time periods. Sustained
improvement was demonstrated through repeated measurements
over comparable time periods. If improvement was not sustained,
there is an explanation and an indicated plan for next steps.
●N/A
Reviewer Comments:
The PIP has not progressed to the point at which sustained improvement can be assessed.
Technical Assistance:
This step should not be addressed until KCBHO has completed its PIP. If real change has occurred, the
PIP should be able to achieve sustained improvement. Sustained improvement is demonstrated through
repeated measurements over time.
81 Information Systems Capabilities Assessment
Information Systems Capabilities Assessment (ISCA)
Qualis Health’s subsidiary, Outlook Associates, examined King County Behavioral Health Organization’s
information systems and data processing and reporting procedures to determine the extent to which they
support the production of valid and reliable State performance measures and the capacity to manage
care of BHO enrollees.
ISCA Methodology
The ISCA procedures were based on the CMS protocol for this activity, as adapted for the BHOs with
DBHR's approval. For each ISCA review area, Outlook Associates used the information collected in the
ISCA data collection tool, responses to interview questions, and results of the claims/encounter and
security walkthroughs to rate the BHO's performance for seven review areas. Scores are based on fully
meeting, partially meeting, or not meeting standards. Although not rated, the BHO's meaningful use of
EHR systems was also evaluated.
The ISCA review process consists of four phases:
Phase 1: Standard information about the BHO’s information systems is collected. The BHO
and two of its delegated provider agencies complete the ISCA data collection tool before the on-
site review.
Phase 2: The completed ISCA data collection tools and accompanying documents are
reviewed. Submitted ISCA tools are thoroughly reviewed. Wherever an answer seems
incomplete or indicates an inadequate process, it is marked for follow-up. If the desktop review
indicates that further accompanying documents are needed, those documents are requested.
Phase 3: On-site visits and walkthroughs with the BHO and two delegated provider
agencies are conducted. Claims/encounter walkthroughs and data center security walkthroughs
are conducted. In-depth interviews with knowledgeable BHO staff and delegated provider agency
staff are conducted. Additional documents are requested, if needed, based upon interviews and
walkthroughs completed at the BHO and at two delegated provider agencies.
Phase 4: Analysis of the findings from the BHO’s information system on-site review
commences. In this phase, the material and findings from the first three phases are reviewed in
cooperation with the BHO and selected delegate provider agencies to close out any open review
questions. The BHO-specific ISCA evaluation report is then finalized.
Each of the items in the review areas were evaluated against industry standards for health data
information systems, especially Medicaid Management Information Systems. Table D-1 describes those
standards.
82 Information Systems Capabilities Assessment
Table D-1: ISCA Scoring Standards
Citation Issuing Body Description of Standard
45 CFR 160 Health & Human Services (HHS) Federal regulations for general administrative requirements pertaining to security and privacy.
45 CFR Part 164 Health & Human Services (HHS) HHS Standards for Security and Privacy Part 164 covers security and privacy of individually identifiable health information.
ISO/IEEE 29119 International Standards Organization/Institute of Electrical and Electronics Engineers
Contains five standards that define an internationally agreed upon set of standards to support software testing.
NIST
National Institute of Standards and Technology
NIST Special Publications 800 series provides a catalog of security controls for all U.S. federal information systems except those related to national security. NIST standards have been incorporated into CMS security controls such as the Minimum Acceptable Risk Standards catalog of security and privacy controls.
ANSI ASC X 12 American National Standards Institute, the Accredited Standards Committee
Uniform standards for inter-industry electronic exchange of business transactions, namely electronic data interchange.
ISO/IEC 27000:2016, 27001:2013, and 27002:2013
International Organization for Standardization/International Electrotechnical Commission
Series of international standards providing best practice recommendations on information security management.
42 CFR Part 438 Health & Human Services (HHS), Centers for Medicare & Medicaid Services
Federal regulations for Medical Assistance Programs, Managed Care. Subpart F pertains to Grievance and Appeals Systems.
The table below presents the scoring key for the ISCA standards.
Scoring Icon Key
●Fully Met (pass) ●Partially Met (pass) ●Not Met (fail) ●N/A (not applicable)
Summary of Results
KCBHO passed the majority of the ISCA, which revealed only several minor issues. KCBHO’s systems
and associated processes are well established and managed. The King County Information Technology
(KCIT) department operates its own data center, with all servers located off site in the county facility
hosted by the Sabey Corporation Seattle data facility. The BHO is able to utilize resources from KCIT to
83 Information Systems Capabilities Assessment
assist with programming, infrastructure support, and security issues and is meeting all contractual
obligations for information systems management.
During the on-site interviews, the review team focused on data integrity and data security. KCBHO
described the process of obtaining data from the BHAs and the measures it takes to ensure that all data
are submitted to the BHO in a timely manner. A portion of the on-site interview included a walkthrough of
the BHO’s office to validate that best practice security measures were being followed.
Two minor issues of access control non-compliance were found during the review. The NIST security
standard is that user accounts are automatically disabled after 60 days of inactivity; however, KCBHO
procedure is removal after a year of inactivity. KCBHO also allows five failed login attempts before locking
the user out; the standard is three.
There were also two areas of concern related to disaster recovery and continuity of operations:
continuous monitoring or periodic review of plans and procedures
periodic testing of those plans
The KCIT Emergency Management Response Plan (EMRP) and Continuity of Operations Plan (COOP)
do not include any reference to review since the last update in 2013. The KCIT Continuity of Operations
Plan indicates that formatting changes were made to the document in January 2017 but does not specify
whether the document had been reviewed since approval in 2013.
The KCIT Business Continuity Plan describes the KCIT Enterprise Business Continuity Program
objectives, risks, benefits, key performance indicators, five-phase approach, and phase progress. The
program emphasizes the need for regular testing and updating of the Business Continuity Plan. But
whereas phases 1 and 2 appear complete as of July 2015, no dates of completion are indicated for
Phase 4—Design the Testing for Business Continuity Plan and Phase 5—Implement Testing.
The EMRP specifics for testing (technical and business) steps, from initial assessment through recovery
and return to facility, are well defined. The COOP also references a continuity testing, training, and
exercise (TT&E) program as a continuity objective; however, no details were present. CMS Security and
Privacy Controls (MARS-E CP-4) require annual testing of a contingency plan. Although industry
standards allow the organization to define the frequency of contingency testing (NIST SP 800-53r4),
these documents indicated nothing with regard to frequency.
The other issues highlighted in this review relate to weakness identified at the two provider agencies the
review team visited. The following sections identify opportunities for the BHO to increase monitoring,
technical assistance, and education of its providers in the areas of security and data quality.
In April 2016, KCBHO began accepting service data from its contracted substance use disorder (SUD)
treatment providers. The BHO acknowledged that while the initial process had been tedious, all providers
are now submitting data. The BHO provided training and systems support during the transition. KCBHO
also provides a web-based application developed in house for batch creation.
KCBHO monitors encounter data and provided analyses of completeness and timeliness of the data.
During site reviews, KCBHO contract monitors validate annual encounter data against chart reviews.
84 Information Systems Capabilities Assessment
Table D-2: ISCA Scores by Section
ISCA Section
Description ISCA Result
Overall ISCA Score This is the overall score for the BHO's ISCA. ● Partially Met (pass)
A. Information Systems
This section assesses the BHO’s management of the information systems.
● Partially Met (pass)
B. IT Infrastructure This section assesses the BHO’s network infrastructure.
● Partially Met (pass)
C. Information Security
This section assesses the security of the BHO’s information systems.
● Partially Met (pass)
D. Encounter Data Management
This section assesses the BHO’s ability to capture and report accurate encounter data.
● Partially Met (pass)
E. Eligibility Data Management
This section assesses the BHO’s ability to capture and report accurate Medicaid eligibility data.
● Fully Met (pass)
F. Provider Data Management
This section assesses the BHO’s ability to maintain accurate provider information.
● Fully Met (pass)
G. Performance Measures and Reporting
This section assesses the BHO's performance measure and reporting processes.
● Fully Met (pass)
H. Meaningful Use of Electronic Health Records (EHRs)
This section is used to assess how the BHO and its providers use EHRs. This section is not scored.
● N/A
ISCA Section Details
KCBHO’s detailed ISCA review findings are presented in the following sections. Each ISCA subsection
features a score corresponding to the “Met,” “Partially Met,” and “Not Met” scoring system. For each
subsection, if a recommendation requiring a corrective action plan (CAP) has been issued, the subsection
will receive a score of Not Met or Partially Met, depending on the severity, and the section will also
receive that corresponding score. If a subsection receives recommendations for an opportunity for
improvement, the subsection will receive a score of Partially Met or Fully Met, and the section will also
receive that corresponding score, unless another subsection received a score of Not Met. If no
recommendations are noted within a subsection, the subsection receives a score of Fully Met. If all
subsections receive a score of Fully Met, the section also receives a score of Fully Met.
Score Description
●Not Met (fail) Recommendation Requiring CAP(s) issued
●Partially Met (pass) Recommendation for Opportunity for Improvement(s) issued
●Fully Met (pass) No Recommendations issued
85 Information Systems Capabilities Assessment
ISCA Section A: Information Systems
This section assesses the BHO’s management of the information systems, specifically examining the
BHO’s capacity for collecting, storing, analyzing, and reporting client treatment data.
Characteristics of well-managed systems include:
data structure that supports complex queries that can be changed easily
secure access via authentication with permission levels
written policies and procedures that support industry standard and best practice IT management
reasonable system response times
complete and consistent testing procedures
clear version control procedures
ability to make changes to systems with minimal disruption to users
adequate training and user documentation
open communication with end users of information system changes and issues
KCBHO demonstrates the majority of these characteristics.
KCBHO has developed databases and applications in house, using the KCIT data center servers and
server applications as its primary management tools, for collecting data from its providers and for meeting
reporting requirements with the State. The KCIT data center is hosted at the Sabey Corporation Seattle
facility and provides secure network and remote access for KCBHO and its providers. KCIT utilizes the
IBM Informix Dynamic Server for its relational database, Hewlett-Packard DL380p Gen8 servers for its
database and application servers, Appeon PowerBuilder for client-server applications, SQL Server
Reporting Services for online reports, Informix 4GL for batch processing, Cisco AnyConnect VPN for
remote access, and Windows Server 2008R2 for its remote desktop server. KCBHO requires its
contractors to transmit their data files either by using secure FTP or by direct input, using the KCBHO-
supplied direct data entry application. However, many of the providers use other commercial software for
their internal electronic medical records (EMRs).
KCIT participates in DBHR’s Behavioral Health Data Group meetings to identify updates requiring
KCBHO system changes necessary to meet the State's needs. Changes are documented in the data
dictionary section of the KCBHO Policies and Procedures manual. Provider management and technical
staff are notified verbally and in writing of the modifications.
KCBHO uses industry standard procedures when upgrading or modifying its system. KCIT and KCBHO
restrict access to the data through a role-based mechanism.
Strengths
KCIT has sound IS management policies in place consistent with current technologies.
KCIT’s Business Continuity Program leverages the data center, dedicated infrastructure, hosted
services (AWS), and virtualization to minimize risk to information systems and business
operations.
All KCIT programming work is performed in house. Its programmers, averaging ten years of
experience and seven years of tenure, have experienced no turnover in the past three years.
KCIT budgets for programmers to receive approximately five days of training per year to enhance
their technical skills.
86 Information Systems Capabilities Assessment
Testing processes comply with industry standards. System changes and updates use formal best
practices protocols.
KCBHO holds regular meetings with its providers to discuss IT issues.
KCBHO performs reconciliations with the State for capitation payments, in-patient payments, and
data in ProviderOne.
Weaknesses
Written IT policies and procedures do not indicate the last date of review.
Providers require additional notice when changes to the data dictionary or file format occur.
Table D-3: Results for Section A: Information Systems
Sub-Area Issues Recommendation Score Standard
IS Management Policies
Annual review lacks indication of most recent review date
Include review section in documents or develop central document tracking policy for review dates
●
45 CFR §§160 and 164; BHO Contract Section 10.10.1
Reconciliation and Balancing
None None ● BHO Contract
Training None None ● 45 CFR §164.312
Testing Procedures
None None ● ISO/IEEE 29119
System Changes and Version Control
None
Allow a longer lead time on changes to the system so that providers have time to make changes to their systems
● NIST SP 800-28
EDI None None ●
45 CFR §164.312; ANSI ASC X12
Total Score ●
Opportunities for Improvement
Written IT policies and procedures do not indicate the last date of review.
KCBHO should make the currency of policies and procedural documentation available. One
option is to add a review section to include the creation or modification date and a brief
description of the change. Another option is to maintain a tracking sheet of all policies to include
the last review date and indication of whether the policy was modified.
Providers need at least a 90-day lead time to make changes to their EHR systems to allow for executing
the changes, communicating those changes to staff, and adequately testing those changes.
When changes to provider EHR systems are needed immediately, the BHO should work with the
providers to allow a grace period for filing edits and correcting errors.
87 Information Systems Capabilities Assessment
ISCA Section B: IT Infrastructure
This section assesses the BHO’s network infrastructure and the BHO’s ability to maintain its equipment
and telecommunications capacity to support end users’ needs.
Characteristics of a well-managed IT environment include:
adequate maintenance staff or maintenance contracts to ensure timely replacement of computer
equipment and/or software
adequate staff or contracts that ensure timely responses to emergent and critical system failures
redundancy within the data center hardware that minimizes the length of system outages, loss of
data, and disruption of end user service.
business continuity and disaster recovery (BC/DR) plans that are maintained and tested regularly.
KCBHO demonstrates all of these characteristics.
KCBHO’s infrastructure meets the needs of both the internal and external users. Its network servers are
configured to provide required redundancy. The system is available to the providers 24/7, and outages
are rare.
Strengths
KCIT adheres to industry standard backup procedures and data redundancy techniques.
System backups are performed every three hours to the cloud and tested monthly.
KCBHO uses secure role-based access for printing and faxing. Security administration
techniques conform to industry standards and best practices. All servers are located in a secure,
off-site facility monitored 24/7 with optimal physical security safeguards and redundancy for
HVAC, network connections, and power.
KCIT’s Sabey data facility provides excellent infrastructure security: power, cooling, network
connectivity, and 24/7 control and monitoring.
Systems are available 24/7. In addition, contracts exist with the vendor responsible for
equipment maintenance for four-hour response time on critical issues.
Backups are tested monthly by restoring the database and running business applications against
it.
Weaknesses
The business continuity and disaster recovery plans do not provide the frequency for periodic
testing. As mentioned in the Information Systems section, there is no record of when the plans
were last reviewed and updated.
Table D-4: Results for Section B: IT Infrastructure
Sub-Area Issues Recommendation Score Standard
Redundancy None None ●
45 CFR §164.308; NIST SP 800-34
Data Center/ Server Room
None None ● 45 CFR §164.308
88 Information Systems Capabilities Assessment
Sub-Area Issues Recommendation Score Standard
Back Up BC/DR plans lack testing information
Update the BC and DR plans to include testing parameters ●
45 CFR §164.308; NIST SP 800-34; BHO Contract
Network Availability
None None ●
ISO Network Management Model (FCAPS)
Total Score ●
Recommendation Requiring CAP
KCBHO’s business continuity and disaster recovery plans do not indicate frequency of periodic testing.
KCBHO needs to periodically test its BC/DR plans. The plans should indicate the frequency of
testing and the type of testing performed.
89 Information Systems Capabilities Assessment
ISCA Section C: Information Security
This section assesses the security of the BHO’s information systems and the safeguards in place to
proactively avoid malicious access to facilities and/or data systems, intrusions, and breaches of protected
health information (PHI) and personally identifiable information (PII).
Characteristics of good security management include:
physical security safeguards at all facilities
policies and procedures that adhere to national healthcare security standards, include specific
references and guidelines for mobile devices, and are routinely reviewed and updated
procedure to remove access to appropriate systems when an employee or contractor leaves,
which includes an expedited path in case of emergency
dedicated security administration staff, adequate to support the agency and its internal and
external users
policies and procedures that adhere to HIPAA Security and Privacy standards, including the
reporting and remediation of security and privacy breaches
KCBHO demonstrates these characteristics, with the exception noted below.
KCBHO takes security seriously and demonstrates understanding and adherence with national security
standards.
Strengths
KCBHO and its agencies have strong security policies in place. The policies cover current
technology, such as portable devices including smart phones and tablets.
The BHO has policies and procedures in place to quickly remove access to the system.
The KCIT Data Center provides a monthly report on penetration and vulnerabilities to all security
managers. Vulnerability tests are routinely performed, and risks must be mitigated or documented
for risk acceptance.
Random checks for privacy screen usage are conducted by the KCBHO compliance officer.
Weaknesses
The BHO policy for disabling inactive accounts is to disable after one year of inactivity, whereas
the NIST standard is for 60 days.
The BHO allows five invalid login attempts into its system before lock-out; the industry standard is
three.
During the on-site review of a KCBHO provider agency, the provider noted that it does not require
visitor login and badging. Best practice security measures are that any non-staff person should
sign a visitors log and be issued a visitor badge.
Table D-5: Results for Section C: Information Security
Sub-Area Issues Recommendation Score Standard
Physical Security
None None ●
45 CFR §164.310; NIST SP 800-66; BHO Contract Section 3.4.4
90 Information Systems Capabilities Assessment
Sub-Area Issues Recommendation Score Standard
Security Policies
None None ●
45 CFR §§164.308,164.312; NIST SP 800-39; BHO Contract Section 3.4.4
Security Testing None None ● NIST SP 800-53, 115
Access Removal Policies
Inactive accounts are not disabled according to industry standards
Implement automatic disabling of accounts inactive after 60 days
●
45 CFR §§164.308,164.312; ISO/IEC 27000:2016–27002; BHO Contract Section 3.4.4
Mobile Device Security and Policies
None None ● 45 CFR §§64.308,164.312; NIST SP 800-124
Total Score ●
Opportunities for Improvement
During the on-site review of a KCBHO provider agency, the provider noted that it does not require visitor
login and badging.
The BHO should notify its agencies that best practice security measures are that any non-staff
person sign a visitors log and be issued a visitor badge.
The BHO allows five invalid login attempts into its system before lock-out.
KCBHO should reduce the number of invalid login attempts into its system before lock-out from
five to three, per industry standard.
Recommendation Requiring CAP
The BHO policy for disabling inactive accounts is to disable an account after one year of inactivity; the
NIST standard is for 60 days.
KCBHO needs to modify its policy to disable inactive accounts after 60 days.
91 Information Systems Capabilities Assessment
ISCA Section D: Encounter Data Management
This section assesses the BHO’s ability to capture and report accurate encounter data.
Characteristics of good encounter data management include:
documented procedures on encounter data submission, which include timeframes and validation
check
automated edit and validity checks of key fields
production of error reports and procedures to correct those errors
periodic audits to validate the encounter data
regular meetings with agency staff to ensure all data are captured accurately and in a timely
manner
reconciliation procedures that compare BHO data to provider data
KCBHO demonstrates all of these characteristics.
KCBHO works with its providers to capture complete and accurate encounter data. The BHO has
established policies that include deadlines for data submission. KCBHO performs annual encounter data
validation of its providers’ data. The BHO provides a number of reports and tools to assist providers in
managing their service data. KCBHO’s IT environment is conducive to timely and complete data entry.
Service data are not accepted unless there is a valid authorization for the majority of the services
delivered by the providers.
Strengths
KCBHO has smart data management policies in place that require data to be edited at the
source. The BHO will return any incorrect data to the originator for modification.
KCBHO follows best practice standards by returning an acknowledgement file to the agencies
with the count of files received.
KCBHO conducts annual data validation of its provider agencies.
KCBHO has a procedure in place for reconciling data with the State to further ensure that
encounter data are complete.
KCBHO created an application for SUD residential treatment services that tracks movement out
of residential facilities and auto-generates the encounter for submission.
Weaknesses
KCBHO does not perform a random validation of services with members in order to detect fraud.
Provider agencies are struggling to correct their encounter data and resubmit in a timely manner.
Table D-6: Results for Section D: Encounter Data Management
Sub-Area Issues Recommendation Score Standard
Data Validation None None ●
42 CFR §438.242; 45 CFR §164.312
Error Handling BHAs are not consistently
Provide additional education to providers on error correction of ●
45 CFR §164.312
92 Information Systems Capabilities Assessment
Sub-Area Issues Recommendation Score Standard reconciling encounter data and correcting errors in a timely manner
encounters
Auditing Random validation is not performed
Perform random validation of services with members ●
42 CFR §438.608; 45 CFR §164.312
Total Score ●
Opportunities for Improvement
KCBHO does not perform a random validation of services with members in order to detect fraud.
KCBHO should perform a random validation of services with members by sending notices of
services to randomly selected members documented as having received those services and
asking members to review and acknowledge that they received those services.
Provider agencies are struggling to correct their encounter data and resubmit them in a timely manner.
The BHO should use its monthly IT meetings with providers to educate them about proper data
reconciliation and correction techniques.
93 Information Systems Capabilities Assessment
ISCA Section E: Eligibility Data Management
This section assesses the BHO’s ability to capture and report accurate Medicaid eligibility data.
Characteristics of good management of eligibility data include:
uploading of monthly eligibility data from the State with reconciliation processes in place
uploading and applying eligibility data changes from the State in between monthly files
managing internal eligibility files to eliminate duplicate member records
running reports to identify changes in eligibility that effect service data
KCBHO demonstrates most of these characteristics.
The BHO and all of its agencies use ProviderOne as the authority on eligibility data. KCBHO stated that it
is unable to apply the daily eligibility file changes from the State; however, it does send requests to
ProviderOne daily and receives the electronic transaction in return. That transaction is applied to the
KCBHO database. The monthly eligibility file is processed and checked for retroactive eligibility that may
impact encounters. The BHO deletes encounters that have been submitted during periods when eligibility
has been lost and submits encounters that were previously not eligible for submission upon learning of
retroactive eligibility. The BHO delegates eligibility management to its providers and requires them to do
regular and frequent checks of eligibility for their clients.
Strengths
The BHO has policies that require all providers to perform eligibility checks at the time of intake
and service delivery, as well as monthly, against ProviderOne. Some perform manual checks and
others submit eligibility requests electronically in batch mode.
BHAs do not immediately drop individuals when Medicaid coverage is lost. The agencies work
with the individuals to reinstate coverage or locate an alternate source to temporarily fund their
medically necessary services. Clinicians are notified of impending loss of eligibility and work with
the client to restore eligibility.
Reports are generated to identify duplicate clients who have been entered in error; these cases
are corrected manually. The providers work with the BHO to correct the records without losing
service history.
KCBHO manages retroactive eligibility and performs nightly checks against encounters to identify
those that can be sent to the State and those that need to be withdrawn.
Weaknesses
Maintaining up-to-date information in the eligibility data in ProviderOne can involve a considerable
amount of effort. Applying and managing the additions and removals that occur during the month
may require the BHO to contact the Health Care Authority (HCA) and DBHR to make manual
corrections.
Table D-7: Results for Section E: Eligibility Data Management
Sub-Area Issues Recommendation Score Standard Eligibility Updates and Verification
None None ●
42 CFR §§438.242, 438.608;
94 Information Systems Capabilities Assessment
Sub-Area Issues Recommendation Score Standard Process BHO
Contract
Duplicate Management
None None ●
42 CFR §§438.242, 438.608
Eligibility Loss Management
None None ●
42 CFR §§438.242, 438.608
Total Score ●
Meets Criteria
95 Information Systems Capabilities Assessment
ISCA Section F: Provider Data Management
This section assesses the BHO’s ability to maintain accurate and timely provider information.
Characteristics of good provider data management practices include:
establishing a communication process to update and maintain provider credentials, licenses, and
skill sets
supporting information systems that integrate provider information with member and service data
developing and maintaining policies and procedures that support timely exchange of provider
information
using provider data to edit encounter data to ensure that qualified providers are performing
services they are qualified to perform
KCBHO demonstrates these characteristics.
KCBHO integrates provider data into its database and uses it during encounter processing.
Strengths
The BHO has good processes for tracking credentialing and provider characteristics for the
provider directories and internal referrals.
Weaknesses
Occasionally, encounters are rejected because of provider data issues. Providers are required to
notify the BHO when license renewals or other changes to provider data occur, but they may not
always do so in a timely manner.
Table D-8: Results for Section F: Provider Data Management
Sub-Area Issues Recommendation Score Standard
Provider Directory Management
None None ●
42 CFR §§438.242, 438.608; BHO Contract
Payment Reconciliation
None None ●
42 CFR §§438.242, 438.608
Total Score ●
Meets Criteria
96 Information Systems Capabilities Assessment
ISCA Section G: Performance Measures and Reporting
This section assesses the BHO's performance measure and reporting processes.
Characteristics of good reporting practices include:
use of encounter data, member data, and service data from an integrated database as the
primary source for performance measurements
policies and procedures that describe how the organization maintains data quality and integrity
staff dedicated and trained in all tools to develop queries and tools for reporting
support for continuing education of staff responsible for reporting metrics
use of data for program and finance decision making
use of analytics software and other industry standard reporting tools
KCBHO demonstrates all of these characteristics.
KCBHO has worked diligently to use Avatar data in performance and management reporting. BHO staff
are gaining more experience with using data as a tool and thinking of creative ways to identify areas for
improvement. The BHO is using standard reporting and analytics tools in addition to utilizing reporting
mechanisms within Avatar.
Strengths
KCBHO has established regional and state performance measures, compiled from decision
support tables in its primary information system.
Table D-8: Results for Section G: Performance Measures and Reporting
Sub-Area Issues Recommendation Score Standard
Performance Measure Processes
None None ●
42 CFR §438.242; BHO Contract
Validation of Performance Metrics
None None ● BHO Contract
Documentation of Metrics
None None ● N/A
Appeals and Grievances None None ●
BHO Contract Section 7
Total Score ●
Meets Criteria
97 Information Systems Capabilities Assessment
ISCA Section H: Meaningful Use of Electronic Health Records (EHRs)
This section of the ISCA is not scored. All KCBHO providers are using EHRs.
98 Information Systems Capabilities Assessment
This page intentionally left blank.
99 Encounter Data Validation
Encounter Data Validation (EDV) Encounter data validation (EDV) is a process used to validate encounter data submitted by Behavioral
Health Organizations (BHOs) to the State. Encounter data are electronic records of the services provided
to Medicaid enrollees by behavioral health agencies (BHAs) under contract with a BHO. Encounter data
are used by BHOs and the State to assess and improve the quality of care and to monitor program
integrity. Additionally, the State uses encounter data to determine capitation rates paid to the BHOs.
Prior to performing the data validation for encounters, Qualis Health reviewed the State’s standards for
collecting, processing, and submitting encounter data to develop an understanding of State encounter
data processes and standards. Documentation reviewed included:
the Service Encounter Reporting Instructions (SERI) in effect for the date range of encounters
reviewed
the Behavioral Health Data System for BHOs
the Health Care Authority Encounter Data Reporting Guide for Managed Care Organizations,
Qualified Health Home Lead Entities, and Behavioral Health Organizations
the 837 Encounter Data Companion Guide ANSI ASC X12N (Version 5010) Professional and
Institutional, State of Washington
Qualis Health performed three activities supporting a complete encounter data validation for KCBHO: a
review of the procedures and results of the BHO’s internal EDV required under the BHO’s contract with
the State; State-level validation of all encounter data received by the State from the BHO during the
review period; and an independent validation of State encounter data matched against provider-level
clinical record documentation to confirm the findings of the BHO’s internal EDV.
Validating BHO EDV Procedures Qualis Health performed independent validation of the procedures used by the BHO to perform encounter
data validation. The EDV requirements included in the BHO’s contract with DBHR were the standards for
validation.
Qualis Health obtained and reviewed the BHO’s encounter data validation report submitted to DBHR as a
contract deliverable for calendar year 2016. The BHO’s encounter data validation methodology,
encounter and enrollee sample size(s), selected encounter dates, and fields selected for validation were
reviewed for conformance with DBHR contract requirements. The BHO’s encounter and/or enrollee
sampling procedures were reviewed for conformance with accepted statistical methods for random
selection.
The BHO submitted a copy of the tool (spreadsheet, database, or other application) used to conduct
encounter data validation, along with any supporting documentation, policies, procedures, or user guides,
to Qualis Health for review. Qualis Health’s analytics staff then evaluated the tool to determine whether its
functionality was adequate for the intended program.
100 Encounter Data Validation
Additionally, the BHO submitted documentation describing its data analysis methods, from which
summary statistics of the encounter data validation results were drawn. The data analysis methods were
then reviewed by Qualis Health analytics staff to determine validity.
Qualis Health Encounter Data Validation
Qualis Health’s encounter data validation process consists of electronic data checks—State-level
validation of all encounter data received by the State from the BHO during the review period—and a
clinical record review—independent validation of State encounter data matched against provider-level
clinical record documentation to confirm the findings of the BHO’s internal EDV.
Electronic Data Checks
Qualis Health analyzed encounter data submitted by the BHO to the State to determine the general
magnitude of missing encounter data, types of potentially missing encounter data, overall data quality
issues, and any issues with the BHO’s processes for compiling encounter data and submitting the data
files to the State. Specific tasks included:
a review of standard edit checks performed by the State on encounter data received by the BHO
and how Washington’s Medicaid Management Information System (MMIS) treats data that fail an
edit check
a basic integrity check on the encounter data files to determine whether expected data exist,
whether the encounter data element values fit within expectations, and whether the data are of
sufficient quality to proceed with more complex analysis
application of consistency checks, including verification that critical fields contain values in the
correct format and that the values are consistent across fields
inspection of data fields for general validity
analysis and interpretation of data for submitted fields, the volume and consistency of encounter
data and utilization rates, in aggregate and by time dimensions, including service date and
encounter processing data, provider type, service type, and diagnostic codes
On-site Clinical Record Review
Qualis Health performed clinical record reviews on site at provider agencies under contract with the BHO.
The process included the following:
selecting a statistically valid sample of encounters from the file provided by the State
loading data from the encounter sample into an auditing tool (MS Access database) to record the
scores for each encounter data field
providing the BHO with a list of the enrollees whose clinical charts were selected for review for
coordination with contracted provider agencies pursuant to the on-site review
Qualis Health staff reviewed encounter documentation included in the clinical record to validate data
submitted to the State and to confirm the findings of the analysis of State-level data.
101 Encounter Data Validation
Upon completion of the clinical record reviews, Qualis Health calculated error rates for each encounter
field. The error rates were then compared to error rates reported by the BHO to DBHR for encounters for
which dates of service fell within the same time period.
Sampling Methodology
With the integration of mental health and substance use disorder (SUD) treatment services, Qualis Health
revised the sampling methodology in order to ensure an appropriate representation of encounters in the
sampled data.
For each BHO, Qualis Health received two data files from the State: one with patient demographics and
another with encounter-level data. Qualis Health first processed the raw data, then validated that all data
contained sufficient information to be included in a sample (most encounters, for example, should contain
a first name, last name, and birthdate). Qualis Health then verified that the data were appropriate to the
agency size and type. For example:
The volume of encounter data should be in an amount proportional to agency size; i.e., large
agencies should have a larger number of encounters than smaller agencies.
Procedure codes and modifiers should be consistent with an agency’s primary business function;
i.e., methadone clinics should submit encounters for services expected to be provided at a
methadone clinic.
All expected services should be reflected in the data; i.e., if withdrawal services are typically
provided at a particular type of agency, procedure codes reflecting those services should appear
in the overall dataset.
Once Qualis Health verified the completeness of the data, the required number of agencies were
selected, including two mental health agencies and four SUD treatment agencies.
Using the SERI, the procedure codes and modifiers were mapped to seven service modalities:
1. Mental Health
2. Substance Use: Assessment Services
3. Substance Use: Outpatient Case Management
4. Substance Use: Opiate Substitution
5. Substance Use: Outpatient Treatment
6. Substance Use: Residential Services
7. Substance Use: Withdrawal Management
The mental health and substance use encounters were separated; then, using J.R. Chromy's method of
sequential random sampling, one sample was pulled for each group, with roughly 35 to 50 patients per
unique BHA/agency pair (RUID) combination. Once the sample patients were selected, the patients and
the service modalities were merged with the encounter data. Upon this merge, reviewers verified that the
samples contained the required number of encounters (at least 411 encounters for mental health and at
least 411 for SUD, for more than 822 encounters in total for the BHO) and that all service modalities for
SUD treatment services in a given BHO’s data were represented.
The substance use sample must contain encounters from each of the six substance use modalities listed
above in proportion to overall volume of these modalities in each of the selected BHO/agency
combinations. For example, if an agency only performs services in the withdrawal management modality,
the sample will only contain sample patients from that agency who have received services in this
102 Encounter Data Validation
modality. Likewise, for agencies providing services representing all six substance use modalities, the
sample will contain encounters for services the agency has provided from all six modalities.
Scoring Criteria
Scoring Icon Key
●Fully Met (pass) ●Partially Met (pass) ●Not Met (fail) ●N/A (not applicable)
KCBHO EDV Procedures
Qualis Health reviewed the sampling procedure and overall sample size to evaluate KCBHO’s adherence
to the contractually required sampling methodology.
Table E-1: Results for Review of BHO EDV Procedures
EDV Standard Description Result—Mental Health Result—SUD
Sampling Procedure
Sampling was conducted using an appropriate random selection process and was of adequate size.
●Partially Met (pass) ●Partially Met (pass)
Review Tools Review and analysis tools are appropriate for the task and were used correctly.
●Fully Met (pass) ●Fully Met (pass) Methodology and Analytic Procedures
The analytical and scoring methodologies are sound, and all encounter data elements requiring review were examined.
●Not Met (fail) ●Not Met (fail)
Sampling Procedure
KCBHO sampled from Medicaid-funded and non-Medicaid-funded encounters that occurred from April 1,
2016, through October 31, 2016, for 29 behavioral health agencies that provide both SUD treatment and
mental health services. An overall sample size of 1,693 (1,071 Medicaid-funded) encounters was
selected. KCBHO reviewed only Medicaid-funded encounters for 17 agencies.
The data source for the sample was the King County Behavioral Health and Recovery Division
Information System (BHRD-IS). This source contains demographics, authorizations, and service
encounters submitted by the BHAs for all clients enrolled in KCBHO services.
KCBHO used proportional random sampling based on agency size and client type (adult or child).
KCBHO selected the charts randomly, without regard to recipient mix. The same number of encounters
(up to five) was usually selected from each chart, unless a chart contained fewer encounters. The
resulting sample mix was representative of the age composition of the Medicaid population KCBHO
serves.
103 Encounter Data Validation
Strength
KCBHO stratified its samples by agency size and client type and pulled clients proportionally
based on the number of clients at each agency. This approach appears to have provided an
unbiased and representative sample.
Opportunities for Improvement
Ideally, a sequence of encounters would be selected for each client to ensure that all encounters in the
BHO (or State) database are present in the chart and that there are no duplicates.
To improve the efficiency of the EDV process, the BHO should select a smaller number of clients
and a larger set of encounters for each client, allowing reviewers to retrieve fewer client charts
during the on-site reviews.
KCBHO presented overall results of its EDV, rather than separating results for SUD and mental health.
In order to better identify trends, KCBHO should clearly state which agencies provide SUD
treatment or mental health services or both. Additionally, encounter samples should be broken
out by type of service for both SUD and mental health.
Recommendation Requiring CAP
KCBHO did not clearly document the number of client charts selected for its sample in its DBHR
deliverable, so it was not possible to determine whether the BHO met its contract-required minimum of
selected encounters.
KCBHO should clearly state the number of client charts selected for its sample within its
deliverable to DBHR.
Review Tools
KCBHO did not submit a copy of its tool to Qualis Health for review; however, the BHO provided a
description of the spreadsheet within its State deliverable with numerous screenshots. KCBHO developed
and utilized an Excel tool using macros and queries. A programmed Excel rating sheet for each agency
listed each client and the service encounters to be reviewed. The raters entered results from their review
directly onto the rater sheet. Findings were then rolled up into a master document. Data validation checks
within the Excel tool and data quality checking macros in the master sheet prevented errors.
In its review tool, KCBHO included the following data elements:
date of service
name of service provider
service location
procedure code
service unit/duration
provider type
KCBHO’s spreadsheet did not contain the field “service code agrees with treatment described,” a DBHR
contract-required element.
Recommendations Requiring CAP
KCBHO did not submit a copy of its review tool in its State deliverable, which is a contract requirement
and would facilitate a more effective review of the tool.
KCBHO needs to submit a copy of its review tool in its DBHR deliverable.
104 Encounter Data Validation
In a repeat finding from the 2016 EQR, KCBHO’s review tool did not include fields for all of the DBHR-
required elements; specifically, “service code agrees with treatment described” was not present in the
Excel spreadsheet tool.
KCBHO needs to include all contract-required elements in its review tool.
Methodology and Analytic Procedures
KCBHO notified each agency one month prior to its site visit of the date(s) of the upcoming record review,
and sent the list of clinical charts to be reviewed three business days prior to the day of the on-site visit.
The EDV review team consisted of six bachelors- and masters-level contract monitors who received
annual training prior to the on-site reviews. The review team participated in the development of review
tools. Reviews were conducted on-site at the agencies.
KCBHO reported an overall match rate of 65 percent, which is below the 95 percent contract requirement.
The “no match” rate was 35 percent, of which 30 percent was erroneous, 3 percent was missing, and 1
percent was unsubstantiated. Of the 17 (out of 29) mental health agencies reviewed, none met or
exceeded the 95 percent match rate. KCBHO reported the aggregate results for each agency, but did not
report the error rate for each data element. The BHO’s contract with DBHR requires analysis and
reporting of error rates for each data element, as well as aggregation of the results.
Recommendation Requiring CAP
KCBHO reported only aggregate results for each agency, whereas the BHO-DBHR contract requires
analysis and reporting of the error rate for each data element. This was a repeat finding from the 2016
EQR.
KCBHO needs to provide the error rates for all data elements in its State deliverable.
Qualis Health Encounter Data Validation
Results below reflect each of the EDV activities performed, including electronic data checks of
demographic and encounter data provided by DBHR, on-site reviews comparing electronic data to data
included in the clinical record, and a comparison of Qualis Health’s EDV findings to the internal findings
reported by the BHO to DBHR for the same encounter date range.
Table E-2: Qualis Health Encounter Data Validation Results
EDV Standard Description Result—Mental Health Result—SUD
Electronic
Data Checks
Full review of encounter data
submitted to the State indicates
no (or minimal) logic problems
or out-of-range values.
●Fully Met (pass) ●Fully Met (pass)
On-site
Clinical
Record
Review
State encounter data are
substantiated in audit of patient
charts at individual provider
locations. Audited fields include
demographics (name, date of
birth, ethnicity, and language)
and encounters (procedure
codes, provider type, duration
●Not Met (fail) ●Not Met (fail)
105 Encounter Data Validation
of service, service date, and
service location). A passing
score is that 95 percent of the
encounter data fields in the
clinical records match.
Electronic Data Checks
Qualis Health analysts reviewed all demographic details and encounters for KCBHO from ProviderOne for
the April 2016 through October 2016 reporting period, comprising 1,214,155 encounters and 70,401
patients. Fields for each encounter were checked for completeness and to determine if the values were
within expected ranges. Results of the electronic data checks are provided in Table E-3.
Table E-3: Results of Qualis Health’s Encounter Data Validation
Measure State Standard Mental Health Performance
SUD Performance
Demographic Data
BHO ID 100% complete, all values in range 100% 100%
Consumer ID 100% complete 100% 100%
First Name 100% complete 100% 100%
Last Name 100% complete 100% 100%
Date of Birth Optional 100% 100%
Gender Optional 100% 100%
Ethnicity/Race 100% complete, all values in range 100% 100%
Language Preference 100% complete, all values in range 100% 100%
Social Security Number
Optional 77% 92%
Sexual Orientation 100% complete 89% 94%
Encounter Data
BHO ID 100% complete, all values in range 100% 100%
Consumer ID 100% complete, all values in range 100% 100%
Agency ID 100% complete, all values in range 100% 100%
Primary Diagnosis 100% complete 100% 100%
Service Date 100% complete 100% 100%
Service Location 100% complete, all values in range 100% 100%
Provider Type 100% complete, all values in range 100% 100%
Procedure Code 100% complete 99% 100%
Claim Number 100% complete 100% 100%
Units of Service 100% complete 97% 100%
106 Encounter Data Validation
On-site Clinical Record Review Results
Qualis Health reviewed clinical charts for 134 individuals randomly selected from two mental health
agencies and 98 individuals randomly selected from four substance use disorder treatment agencies. For
each individual, Qualis Health reviewed up to six sequential encounters to determine the completeness
and accuracy of the data submitted to the State. The exact number of encounters and the encounter
initiating the reviewed sequence were selected using a random number generator in SAS 9.4. In all, 556
encounters were reviewed for mental health services, and 437 encounters were reviewed for substance
use disorder treatment services. Qualis Health reviewed encounter data fields required for review in the
BHO contract with DBHR, including:
date of service
name of service provider
procedure code
service units/duration
service location
provider type
verification that the service code agrees with the treatment described in the encounter
documentation
Additionally, Qualis Health reviewed all demographic fields delineated in the Behavioral Health Data
System native transaction, including:
first name
last name
gender
date of birth
ethnicity/race
Hispanic origin
preferred language
Social Security Number
sexual orientation
Results from these reviews are displayed in the following tables. Tables E-4 and E-5 show results of the
comparison of demographic data included in the clinical record to demographic data extracted from the
DBHR system.
107 Encounter Data Validation
Table E-4: Demographic Data Validation—Mental Health
Demographic Data (N = 134)
Field Match No Match— Erroneous
No Match— Missing
No Match— Unsubstantiated
Last Name 99% 1% 0% 0%
First Name 100% 0% 0% 0%
Gender 99% 1% 0% 0%
Date of Birth 100% 0% 0% 0%
Ethnicity/Race 87% 12% 0% 1%
Hispanic Origin 82% 14% 0% 4%
Preferred Language 4% 50% 37% 9%
Social Security Number 75% 0% 22% 3%
Sexual Orientation 49% 19% 8% 24%
Table E-5: Demographic Data Validation—SUD
Demographic Data (N = 98)
Field Match No Match— Erroneous
No Match— Missing
No Match— Unsubstantiated
Last Name 97% 2% 1% 0%
First Name 99% 0% 1% 0%
Gender 97% 1% 1% 1%
Date of Birth 96% 2% 1% 1%
Ethnicity/Race 79% 17% 2% 2%
Hispanic Origin 85% 11% 2% 2%
Preferred Language 4% 37% 57% 2%
Social Security Number 83% 1% 13% 3%
Sexual Orientation 69% 8% 10% 12%
Results of the comparison of encounter data included in the clinical record to encounter data extracted
from the ProviderOne database are shown in Tables E-6 and E-7.
Table E-6: Encounter Data Validation—Mental Health
Encounter Data (N = 556)
Field Match No Match— Erroneous
No Match— Missing
No Match— Unsubstantiated
Procedure Code 76% 16% 0% 8%
Date of Service 98% 0% 0% 2%
Service Location 92% 1% 0% 7%
Service Duration 76% 22% 0% 3%
Provider Type 76% 16% 0% 8%
Clinical Note Matches Procedure Code 64% 34% 0% 2%
108 Encounter Data Validation
Table E-7: Encounter Data Validation—SUD
Encounter Data (N = 437)
Field Match No Match— Erroneous
No Match— Missing
No Match— Unsubstantiated
Procedure Code 47% 14% 0% 39%
Date of Service 64% 0% 0% 36%
Service Location 51% 9% 0% 39%
Service Duration 40% 24% 0% 36%
Provider Type 47% 14% 0% 39%
Clinical Note Matches Procedure Code 35% 29% 0% 36%
The comparison of the total match rate from the Qualis Health review to the total match rate from the
KCBHO internal EDV for demographic data is shown in Table E-8. Note: KCBHO did not report these
data, so Qualis Health is unable to present a comparison of results.
Table E-8: Comparison of Qualis Health and BHO Demographic Data Validation—Overall
Field Qualis Health
Match BHO
Match Variance
Last Name 98% NR NA
First Name 100% NR NA
Gender 98% NR NA
Date of Birth 98% NR NA
Ethnicity/Race 84% NR NA
Hispanic Origin 83% NR NA
Preferred Language 4% NR NA
Social Security Number 78% NR NA
Sexual Orientation 59% NR NA
The comparison of the total match rate from the Qualis Health review to the total match rate from the
KCBHO internal EDV for encounter data is shown in Table E-9. Note: KCBHO did not report these data,
so Qualis Health is unable to present a comparison of results.
Table E-9: Comparison of Qualis Health and BHO Encounter Data Validation—Overall
Field Qualis Health Match BHO Match Variance
Procedure Code 64% NR NA
Date of Service 83% NR NA
Service Location 74% NR NA
Service Duration 60% NR NA
Provider Type 64% NR NA
Clinical Note Matches Procedure Code 51% NR NA
109 Encounter Data Validation
Qualis Health's review found that in numerous cases, while data elements may have matched the
encounter, other elements of the encounter either did not follow the State’s SERI or WAC requirements,
contained insufficient documentation, did not match the code that was submitted, or did not reflect a
service that should have been submitted. Examples included the following:
Provider (upon instruction from the BHO) encountered location 12—residence when a client was
provided with methadone to take home, instead of location 57—Non-residential substance abuse
facility (as required by the SERI).
Non-encounterable activities were submitted, including writing and/or mailing a letter, writing a
treatment summary after the client had been closed to services without the client present, leaving
voicemail, checking messages, scheduling and rescheduling appointments, client no-showing
an appointment, calling in a medication refill.
SUD group documentation did not contain enough information to support the encounter duration.
CDPs did not sign off on CDPT progress notes or did not sign off on them in a timely manner.
Groups were encountered as H2027, H0036, and H0038, all of which are individual codes.
Clinicians did not date signatures, preventing assessment of timeliness of the documentation.
Documentation was written months after the service occurred, without explanation of why the
note was not written in a timely manner.
Documentation was cloned.
Bachelors-level interns and staff created individual service plans without MHP sign-off.
Provider type listed in the clinical note did not match the type listed in the State database.
Progress notes lacked a procedure code.
Family therapy did not contain a clinical intervention to support the encounter.
Clinicians did not sign with credentials.
H0019—mental health services in a residential setting was encountered when then clinician was
not present a minimum of 8 hours per day 7 days per week.
E&M documentation for 99214 did not contain all the required elements.
H0038 staff name did not include certified peer credentials.
Documentation that was lacking location information defaulted to code 99.
Regular outpatient codes were submitted for an individual in jail instead of a rehabilitation case
management code.
Case management encounters were submitted for case review when the client was not present.
The BHO submitted encounters on the behalf of agencies without evidence that a given individual
received a service for the day.
Recommendation Requiring CAP
For all encounter fields, Qualis Health found a substantial level of disagreement between encounter data
extracted from ProviderOne and data included in the clinical record.
KCBHO needs to:
o use State data to capture duplicates and missing encounters, and ensure accuracy of data
submission
o work with its network on documentation standards to ensure that clinical interventions are
being well documented, which will enable KCBHO to match the correct code to the service
o train the BHAs on encounterable services and on providing documentation that indicates
treatment addressing behavioral health
o work with prescribing staff within the network on evaluation and management documentation
and rules
110 Encounter Data Validation
o train providers on documentation requirements, SERI requirements, and WAC requirements
for documentation and timeliness
111 Golden Thread
Golden Thread Focus Study
For the 2017 EQR focus study, Qualis Health examined the degree to which substance use disorder
(SUD) treatment providers’ clinical records demonstrated adherence to the “golden thread,” a series of
clear, consistent care linkages between an individual’s needs, diagnosis, and treatment. In evaluating
provider records, reviewers asked the following questions:
1. Does the individual’s assessment contain sufficient documentation to support the diagnosis, and
does it include all of the individual’s needs?
2. Are the documented needs reflected in specific goals in the treatment plan/individual service plan
(ISP)? Does the ISP address the individual’s diagnosis and all of the identified needs in the
assessment?
3. Do the progress notes address the individual’s treatment plan goals and the needs identified in
the assessment?
To answer these questions, Qualis Health reviewed 18 adult patient charts randomly selected from the
BHO’s EDV sample. Reviewers specifically evaluated documentation in three areas: assessments and re-
assessments, individual service planning, and progress notes. Each section (assessments/re-
assessments, ISPs, and progress notes) contained several subsections that were reviewed and scored.
These areas and the results of the review are discussed in the following section.
Summary of Results
Assessments
In reviewing clinical chart assessments, Qualis Health analyzed five different areas, evaluating how well
the chart documentation adhered to the following items presented in Table F-1.
Table F-1: Results for Review of Assessments in Clinical Charts
Criterion Percentage of charts meeting standard
1. Evidence of medical necessity based on the presence of a DSM-5 substance-related diagnosis
90.3%
2. Sufficient information within assessment to justify the diagnosis 67.7%
3. Recommendation(s) based on ASAM criteria 80.6%
4. Clear presentation of the individual’s concerns in the assessment 51.6%
5. Client voice present throughout the assessment 48.4%
Individual Service Plans
In reviewing individual service plans, Qualis Health analyzed eight different areas, evaluating how well the
chart documentation adhered to the following items presented in Table F-2.
112 Golden Thread
Table F-2: Results for Review of Individual Service Plans in Clinical Charts
Criterion Percentage of charts meeting standard
1. All concerns in assessment are addressed in the service plan 25.8%
2. Service plan is individualized 80.6%
3.1. Service plan includes all substance use needing treatment, including tobacco
32.3%
3.2. Service plan includes the individual’s bio-psychosocial problems 54.8%
3.3. Service plan includes treatment goals 80.6%
3.4. Service plan includes estimated dates or conditions for completion of each treatment goal
45.2%
3.5. Service plan includes potential approaches to resolve identified problems
25.8%
4. Goals and/or objectives are measureable 3.2%
5. Interventions are aligned with the problems identified in the assessment 35.5%
6. If the individual service plan includes assignment of work to an individual, the assignment has therapeutic value
35.5%
7. The plan was updated to address applicable changes in identified needs, or as requested by the individual, at least once a month for the first three months and at least quarterly thereafter
9.7%
8. The plan was updated to identify achievement of goals and/or objectives
9.7%
Progress Notes
For the final portion of the Golden Thread review, Qualis Health examined whether or not the progress
notes indicated that the services provided were connected to the interventions, objectives, and goals
identified in the individual service plan resulting from the needs and concerns identified in the assessment
or reassessment process. This review area included seven criteria, presented in Table F-3.
Table F-3: Results for Review of Progress Notes in Clinical Charts
Criterion Percentage of charts meeting standard
1. Progress notes were written in a timely manner in accordance with WAC 388-877B-0350
77.4%
2. Documentation clearly states the focus of each session 54.8%
3. Documentation clearly states the interventions described in the service plan
9.7%
4. Documentation describes the individual’s response to the intervention 25.8%
5. Interventions address the goals and objectives in the service plan 32.3%
6. Documentation indicates progress, or lack thereof, toward meeting the goals and objectives in the service plan
19.4%
7. Services provided align with the individual’s assigned level of care 51.6%
113 Golden Thread
Concluding Notes
During the clinical record review, reviewers noted a number of issues that may have contributed to lower
scores on the assessment. These include the following examples:
Assessment was not completed in a timely manner.
Clinical record did not contain evidence of an updated assessment or reassessment.
Assessments did not contain documentation that justified the diagnosis.
Assessments did not contain documentation of all of the individual's concerns.
Assessments did not contain treatment recommendations.
Assessments and ISPs were not present in the clinical record.
ISP did not indicate the services to be provided.
ISP did not include the frequency, duration, and scope/amount of services to be provided.
Goals did not contain timelines or conditions for completion.
Goals and/or objectives were not measureable.
Interventions included in the ISP resembled homework more so than interventions to be provided
by the clinician.
ISP was not updated to include new needs and goals.
ISP was not completed within the WAC timeframes.
ISP was not reviewed during the course of treatment.
ISP review occurred monthly but was never updated or changed.
Clinical records contained canned/cloned ISPs that were not specific to the individual.
Progress note content was not related to the needs identified in the assessment and the ISP.
Progress notes did not contain documentation of interventions.
Progress notes were written by the individual instead of the clinician.
Client was assessed as needing a higher level of care, but the facility lacked capacity to provide
the higher level of care immediately and did not provide services in the interim.
Group notes did not meet minimum documentation standards.
Documentation did not substantiate the duration of the service.
Case management reviews were submitted a year after the service occurred.
Clinic provided services to a client prior to completing the individual's ISP.
114 Golden Thread
This page intentionally left blank.
115 Appendix
Appendix A: All Recommendations Requiring Corrective
Action Plans (CAPs)
Compliance with Regulatory and Contractual Standards
Section 1: Availability of Services
Recommendation Requiring CAP
Enrollee participation varies greatly among the agencies, from 9,000 enrollees for one agency to 30
enrollees for another agency. Regardless of the participation rate, workforce development remains a
major concern for the BHO.
1. KCBHO needs to continue to work with the BHAs in developing successful talent management
and recruitment strategies, as demand for services has increased as a result of the Affordable
Care Act and the opioid epidemic.
Recommendation Requiring CAP
The BHO indicated that during its most recent administrative audit of the BHAs, the SUD treatment
agencies were not aware of how to arrange for second opinions and had not implemented policies on
second opinions. The BHO stated it provided consultation with its SUD treatment BHAs on how to
arrange for second opinions and encouraged the BHAs to adopt the BHO’s policy on second opinions.
2. KCBHO needs to follow up with the SUD treatment BHAs to ensure they have adopted and
implemented policies on second opinions.
Recommendation Requiring CAP
KCBHO noted it does not have a system for tracking requests for out-of-network services.
3. KCBHO needs to implement a procedure to track requests for out-of-network services and use
this information for network planning.
Recommendation Requiring CAP
KCBHO indicated it follows its credentialing policies and procedures to ensure all out-of-network
providers are appropriately credentialed. The BHO requires all out-of-network providers to complete and
sign a single-case service agreement, which requires the provider to submit license(s)/credentials and
attest to a background check, and provides assurance that the provider is not on the excluded provider
list. However, during the on-site interview, KCBHO indicated it needed to update its credentialing policies
and procedures to ensure all providers are appropriately credentialed.
4. KCBHO needs to update its credentialing policies and procedures to ensure all providers are
appropriately credentialed, including out-of-network providers and those with a single-case
agreement.
Section 2: Coordination and Continuity of Care
Recommendation Requiring CAP
Although KCBHO has a policy and procedure for coordinating enrollee care, the policy lacks a definition
or standard of what constitutes care coordination and how it is measured.
5. In its policy on enrollee care coordination, KCBHO needs to include a definition or standard of
what constitutes care coordination and how it is measured. The definition should include:
116 Appendix
o the standard for coordinating care between settings of care
o appropriate discharge planning for short-term and long-term hospital or institutional stays
o services the enrollee receives from any other BHO
o services the enrollee receives in fee-for-service (FFS) Medicaid
o services the enrollee receives from community and social support providers
Recommendation Requiring CAP
KCBHO requires the BHAs to initiate a referral to a primary healthcare provider appropriate for the
enrollee’s needs when the enrollee does not have a primary healthcare provider. KCBHO stated that
through its clinical record review and monitoring of the intake process it ensures each enrollee has
access to a primary healthcare provider. The BHO has also included monitoring in its 2017 on-site clinical
record review tool to ensure that documentation of coordination of activities is evident in the enrollee’s
clinical record. However, during the BHA on-site record review conducted by the EQRO, the majority of
charts lacked evidence of care coordination with the primary healthcare provider. In addition, the 2016
summary report results from KCBHO’s administrative review indicated that the majority of agencies had
difficulty articulating coordination of care in their policies and procedures.
6. KCBHO needs to continue its monitoring efforts to ensure its BHAs have clearly defined care
coordination in their policies and procedures. The BHO needs to continue training, educating, and
monitoring its BHAs to ensure enrollees are referred to a primary healthcare provider when
appropriate and that care coordination is documented in the clinical record.
Recommendation Requiring CAP
KCBHO’s clinical record and administrative reviews include criteria for reviewing treatment plans to
ensure the treatment plans are developed with the enrollee’s participation, and in consultation with any
specialists caring for the enrollee. However, the results of the on-site EQR of the care coordination
records indicated that some BHAs are not using or completing treatment plans and that many treatment
plans that were in place did not include enrollee voice and participation.
7. KCBHO needs to ensure that all BHAs have treatment plans in place and that the treatment plans
include documentation that the plans were developed with the enrollee’s participation and in
consultation with any specialists caring for the enrollee.
Recommendation Requiring CAP
KCBHO does not have a policy and procedure on direct access to specialists for enrollees with special
healthcare needs.
8. The BHO needs to develop a policy and procedure regarding direct access to specialists for
enrollees with special healthcare needs.
Recommendation Requiring CAP
KCBHO does not monitor the availability of direct access to specialists.
9. KCBHO needs to add criteria to its monitoring tool to assess availability of direct access to
specialists.
Section 3: Coverage and Authorization of Services
N/A
Section 4: Provider Selection
117 Appendix
Recommendation Requiring CAP
During the on-site interview, the BHO indicated it is not monitoring its BHAs to ensure they have a policy
and procedure on credentialing and re-credentialing.
10. KCBHO needs to add criteria to its monitoring tool in order to review each BHA’s credentialing
and re-credentialing policy and procedure.
Recommendation Requiring CAP
At the time of the 2017 EQR, the BHO indicated it had created an internal process to ensure monthly
exclusion checks were performed on all staff, including its BHAs. However, it was determined during the
on-site visit that the BHO is not performing monthly Office of the Inspector General (OIG) List of Excluded
Individuals and Entities (LEIE) checks on its entire staff at the BHO, including the county executives, BHO
leadership, interns, volunteers, and staff who make authorization decisions.
11. KCBHO needs to ensure the BHO as well as the BHAs are performing exclusion checks on a
monthly basis and at the time of hire on all staff, including county executives, BHO leadership,
board members, custodial staff, and volunteers.
Section 5: Subcontractual Relationships and Delegation
Recommendation Requiring CAP
KCBHO’s BHA contracts do not contain language holding the BHAs’ subcontractors accountable for
delegated services.
12. KCBHO needs to add language to its BHA contracts that clarifies that BHA subcontractors are
accountable for any delegated services.
Recommendation Requiring CAP
KCBHO does not include delegation monitoring in its annual QA evaluation.
13. KCBHO needs to add delegation monitoring to its annual QA evaluation.
Section 6: Practice Guidelines
Recommendation Requiring CAP
KCBHO’s policy states that practice guidelines are disseminated to all affected providers and, upon
request, to enrollees, but it does not specify the mechanism or frequency of distribution.
14. KCBHO needs to include in its policy the mechanism and frequency with which it distributes the
practice guidelines to providers and enrollees.
Section 7: Health Information Systems
Recommendation Requiring CAP
The BHO does not have an external policy and procedure to ensure its BHAs are checking their data for
quality and integrity before submitting them to the BHO.
15. The BHO needs to create and implement a policy and procedure to ensure its BHAs are checking
their data for quality and integrity before submitting them to the BHO. The policy should include:
o the requirement for providers to submit written attestations of data accuracy
o a form letter for providers to complete attesting to data accuracy
o a system for the form letters to be transmitted electronically to the BHO
o monitoring by BHO contract monitors to ensure timely submission of the attestation
letters
118 Appendix
Recommendation Requiring CAP
KCBHO reported that the SUD treatment BHAs were experiencing challenges submitting data in the file
formats required by the Service Encounter Reporting Instructions (SERI) as this was a new requirement
for the providers. The SUD treatment BHAs were encouraged to use a web portal the BHO designed for
data submission until the challenges were resolved. However, the EQRO was unable to perform EDV at
the time of the review because the SUD treatment BHAs had not submitted all of the required data.
16. KCBHO needs to continue to train and assist its SUD treatment BHAs to ensure the BHAs can
submit timely, accurate, and complete data.
Section 8: Quality Assessment and Performance Improvement Program
Recommendation Requiring CAP
In the past year, KCBHO has not been able to report all of its SUD treatment performance data to the
State.
17. KCBHO needs to continue to monitor and support its SUD treatment BHAs in order to ensure it
receives all performance data.
Performance Improvement Project (PIP) Validation
PIP Validation Results: Clinical PIP
Recommendation Requiring CAP
KCBHO did not state the focus of its study in the form of a question. While the BHO does provide
statements describing what it plans to achieve through the intervention, it is not posed in a clear, simple,
and answerable format.
18. KCBHO needs to formulate a study question that is truly in the structure of a question.
PIP Validation Results: Non-Clinical Children’s PIP
N/A
PIP Validation Results: Substance Use Disorder (SUD) PIP
N/A
Information Systems Capabilities Assessment (ISCA)
Recommendation Requiring CAP
KCBHO’s business continuity and disaster recovery plans do not indicate frequency of periodic testing.
19. KCBHO needs to periodically test its BC/DR plans. The plans should indicate the frequency of
testing and the type of testing performed.
Recommendation Requiring CAP
The BHO policy for disabling inactive accounts is to disable an account after one year of inactivity; the
NIST standard is for 60 days.
119 Appendix
20. KCBHO needs to modify its policy to disable inactive accounts after 60 days.
Encounter Data Validation (EDV)
Recommendation Requiring CAP
KCBHO did not clearly document the number of client charts selected for its sample in its DBHR
deliverable, so it was not possible to determine whether the BHO met its contract-required minimum of
selected encounters.
21. KCBHO should clearly state the number of client charts selected for its sample within its
deliverable to DBHR.
Recommendation Requiring CAP
KCBHO did not submit a copy of its review tool in its State deliverable, which is a contract requirement
and would facilitate a more effective review of the tool.
22. KCBHO needs to submit a copy of its review tool in its DBHR deliverable.
Recommendation Requiring CAP
In a repeat finding from the 2016 EQR, KCBHO’s review tool did not include fields for all of the DBHR-
required elements; specifically, “service code agrees with treatment described” was not present in the
Excel spreadsheet tool.
23. KCBHO needs to include all contract-required elements in its review tool.
Recommendation Requiring CAP
KCBHO reported only aggregate results for each agency, whereas the BHO-DBHR contract requires
analysis and reporting of the error rate for each data element. This was a repeat finding from the 2016
EQR.
24. KCBHO needs to provide the error rates for all data elements in its State deliverable.
Recommendation Requiring CAP
For all encounter fields, Qualis Health found a substantial level of disagreement between encounter data
extracted from ProviderOne and data included in the clinical record.
25. KCBHO needs to:
o use State data to capture duplicates and missing encounters, and ensure accuracy of data
submission
o work with its network on documentation standards to ensure that clinical interventions are
being well documented, which will enable KCBHO to match the correct code to the service
o train the BHAs on encounterable services and on providing documentation that indicates
treatment addressing behavioral health
o work with prescribing staff within the network on evaluation and management documentation
and rules
o train providers on documentation requirements, SERI requirements, and WAC requirements
for documentation and timeliness
120 Appendix
Appendix B: Review of Previous-Year Recommendations
Requiring Corrective Action Plans (CAPs)
As part of the 2017 EQR activities, Qualis Health reviewed the previous-year recommendations BHOs
received in areas of compliance with State and federal regulatory standards, performance improvement
project validation, and encounter data validation. The BHO’s recommendations and its progress to date
implementing the accompanying corrective action plans are presented below.
CFR/Review
Area
2016 QH Recommendation/
DBHR CAP
BHO Activity Since the Prior
Year
Current Status
438.228 (a),(b)
Grievance
Systems
KCBHO received only one
grievance in the first quarter of
2016. This seems highly unlikely
given the size of the population of
King County. The BHO did state
that the BHAs and the Ombuds
only ask for written grievances,
not for oral grievances, and also
stated that the BHO and its BHAs
are not recording all complaints of
dissatisfaction. Interviews with the
BHAs indicated there were
concerns and grievances voiced
by enrollees that had not been
logged and provided to the BHO.
KCBHO needs to re-educate the
BHAs, the Ombuds and its own
staff on what constitutes a
grievance: all expressions of
dissatisfactions. All grievances
need to be recorded and logged
and provided to the BHO
quarterly. Grievances should then
be tracked and trended for quality
improvement purposes.
Grievances should also be
reviewed and analyzed by the
BHO’s quality improvement
committee and compliance
committee to identify
opportunities for improvement as
well as potential areas of risk.
The BHO intends to continue to
communicate and provide
training to its staff and the
BHAs about logging grievances
and submitting quarterly
grievance reports.
Resolved.
438.228 (a),(b)
Grievance
Systems
The BHO stated that a
methodology for logging all
grievances has not been
established with all of the BHAs.
The BHO has provided and
stated it intends to continue
providing training to its staff and
the BHAs about logging
Resolved.
121 Appendix
KCBHO needs to work with the
BHAs to establish a methodology
for recording and logging all
grievances received by the BHAs.
The BHAs then need to submit all
logs quarterly to the BHO.
grievances and submitting
quarterly grievance reports.
438.416
Recordkeeping
and Reporting
Although the BHO requires the
BHAs to file grievances
separately from clinical records,
ensure the grievances are locked
in a secure file cabinet or
electronic file, ensure grievances
are kept for at least six years, and
ensure that only people who need
to know have access to the
grievances; there is no formal
written policy and procedure
detailing these requirements.
KCBHO needs to develop a
record retention policy and
procedure that includes the above
requirements.
The current KCBHO Policy and Procedure, Section XI. Grievance System: 3.2.8 states: “G. Full records of all
grievances and materials
received or compiled in the
course of processing and
attempting to resolve the
grievance are maintained and:
1. Kept for six years after
the completion of the
grievance process;
ADD KC Record
Retention Reference…
2. Made available to
Department of Social
and Health Services
(DSHS) upon request
as part of the State
Quality Strategy;
3. Kept in confidential files
separate from the
individual’s clinical
record; and
4. Not disclosed without
the individual’s written
permission, except to
DSHS or as necessary
to resolve the
grievance.”
Resolved.
438.600
Provider
Eligibility
KCBHO’s policy and procedure
ensures staff are not listed by the
Office of the Inspector General
(OIG) as debarred, excluded or
otherwise ineligible for Federal
program participation, as required
by Federal or State laws, or found
to have a conviction or sanction
related to healthcare as listed in
the Social Security Act, Title 11.
However, the policy and
Section 4.0 of KCBHO’s
program integrity policy
includes the language:
“Providers shall report to the
KCBHO and the KCBHO shall
report to Washington State
Division of Social and Health
Services (DSHS):
4.4.1. Any excluded individuals
and entities discovered in the
Resolved.
122 Appendix
procedure does not specifically
include the BHO’s intention to
report to DSHS within 10
business days any excluded
individuals or entities discovered
in the OIG screening process.
KCBHO needs to add to its policy
and procedure its intention to
report to DSHS within ten
business days any excluded
individuals and entities
discovered in the OIG excluded
checks.
screening within 10 business
days.”
438.602
Data
Certification
On April 1, 2016, KCBHO
developed an internal policy and
procedure that states: “The BHO
will establish policies and
procedures that address data
quality, assure certification occurs
concurrently with the submission
of data to the State, develop
protocols for data certification and
instructions of how to submit the
data, and maintain internal
records of certifications.” The
policy does not include ensuring
the BHAs are also attesting to the
quality and integrity of data being
submitted to the BHO.
The BHO needs to create an
external policy and procedure to
ensure its BHAs are checking
their data for quality and integrity
before the data are submitted to
the BHO.
The BHO intends to:
revise its policies to include the requirement for providers to submit a written attestation of data accuracy
develop a form letter for providers to complete, attesting to data accuracy
inform providers of change and seek additional input on the KCBHO process
design and implement a system for letters to be transmitted electronically to KCBHO
have contract monitors follow up on attestation letter submissions
In progress.
438.606
Source,
Content and
Timing of
Certification
The BHO does not maintain
internal logs of attestations in
accordance with its internal policy
and procedure.
KBHO needs to create a log of
attestations to formalize its
emailed attestation and data
transaction submissions, and
keep this on file per record
retention guidelines.
The BHO has updated its policy
to include a log of attestations
and intends to store the
attestations in a shared folder.
Resolved.
123 Appendix
438.608 (a)(b)
Program
Integrity
Requirements
KCBHO has not been performing
annual risk assessments.
The BHO needs to perform risk
assessments annually, and
discuss the results with its
executive team, board and
appropriate committees. The
discussions should include
developing action plans to
regularly monitor risks and
vulnerable areas, and seek
interventions where appropriate
to mitigate risks. Additionally, the
team needs to include the results
of the annual risk assessment in
its annual compliance self-
evaluation.
The BHO indicated that it will
include the annual performance
of risk assessments in its
compliance plan. However, the
BHO is not yet annually
performing risk assessments.
The BHO indicated that it will
initiate a risk assessment after
it hires a compliance officer.
While on-site, the EQRO
provided extensive technical
assistance for this
recommendation.
In progress.
438.608 (a)(b)
Program
Integrity
Requirements
Although the county has a code
of ethics summary, the BHO does
not have its own documented
code of ethics/standards of
conduct nor does it have a
mechanism in place to monitor
attestations of its own staff and
contracted entities.
KCBHO needs to adopt its own
code of ethics/conduct and
develop a policy and procedure
for ensuring its own
staff/contracted entities are
adhering to the code of
ethics/conduct. The BHO could
include training on the code of
ethics in its annual compliance
training program for its BHO staff
and board, and its BHAs.
During the first quarter of 2017,
KCBHO reviewed the current
King County Code of Ethics
and, as needed, revised it to
incorporate any additional
requirements as defined in CFR
438. Additionally, KCBHO
reviewed and, as needed,
developed a policy to describe
staff adherence and training for
the code of ethics. The policy
addresses the inclusion of the
code of ethics in the annual
compliance training program for
BHO and BHA staff.
The KCBHO Compliance Plan
includes references to the King
County Code of Ethics. All
county employees are expected
to adhere to this policy, and
KCBHO documents require
adherence to this county
standard.
Training will include
reaffirmation/ acknowledgment
of the King County Code of
Ethics. Additionally, identified
county staff must also complete
Resolved.
124 Appendix
the King County COI
attestation. The BHO indicated
it will begin monitoring its BHAs
to ensure they maintain a code
of ethics.
The EQR staff provided
technical assistance during the
on-site interview regarding this
recommendation.
438.608 (a)(b)
Program
Integrity
Requirements
KCBHO indicated it was uncertain
of whether its new employee
orientation covered training on
whistleblower protections.
KCBHO needs to ensure training
on its policy and procedure
related to whistleblower
protections, which includes no
retaliation, is provided in its new
employee orientation and in the
annual compliance training
program.
KCBHO has affirmed that its
compliance training includes
whistleblower protections.
Resolved.
438.608 (a)(b)
Program
Integrity
Requirements
KCBHO does not keep a formal
log of reports of suspected fraud,
waste and abuse.
The BHO needs to develop a
formal tracking process for all
reports of suspected fraud, waste
and abuse in order to ensure it
has effective lines of
communication between the
compliance officer and the
organization’s employees. This
formal log should be reviewed by
the compliance committee and
incorporated into the committee’s
meeting agenda as a standing
agenda item.
KCBHO has developed a
centralized process for tracking
all reports of suspected fraud,
waste, and abuse as part of the
overall KCBHO compliance
plan.
Resolved.
438.608 (a)(b)
Program
Integrity
Requirements
The BHO does not review
grievance and critical incident
data, including data from the
providers and the Ombuds, or
complaints from contractors and
community members, for specific
incidents or trends that may
indicate fraud and abuse.
KCBHO intends to incorporate
review of grievance and critical
incident data into its ongoing
Quality Improvement
Committee agenda. The review
process has been included in
the overall KCBHO Compliance
Plan.
Resolved.
125 Appendix
The KBHO needs to develop a
process and procedure to review
grievance and critical incident
data, including data from the
providers and the Ombuds and
complaints from contractors and
community members, for specific
incidents or trends that may
indicate fraud and abuse.
438.608 (b)
Compliance
Programs
KCBHO’s compliance officer
indicated the BHO utilizes the
Plan Management Group (PMG)
as its compliance committee. This
committee doesn’t have a formal
charter, routine agendas or
meeting minutes.
The BHO needs to develop a
formal charter and compliance
committee that convenes at least
quarterly, if not monthly. The
charter needs to indicate the
duties of the compliance
committee, its members, and the
frequency of meetings. The
compliance committee should
follow a formal agenda with
reportable meeting minutes in
order to reflect the oversight of its
compliance program.
KCBHO has produced a draft of
a written compliance plan,
which has undergone first-level
review by the BHO’s contracts
manger, quality manager,
managed care specialist, and
BHO administrator. The plan
will next be reviewed by
executive leadership.
In progress.
438.608 (b)
Compliance
Programs
KCBHO’s compliance officer
indicated that division-wide
compliance training has not been
conducted on an annual basis.
The BHO needs to ensure it
provides annual compliance
training for its BHO staff,
governing board and its
delegated entities.
KCBHO reported that initial
compliance training for all BHO
staff will be completed by June
30, 2017.
In progress.
431.107
Record
Retention
The BHO follows the record
retention policy and procedure
used by the county, and does not
have its own policy and
procedure.
KCBHO needs to create a BHO-
KCBHO has initiated discussion
with the King County archivist
and records and information
manager to review and
incorporate specific record
retention requirements into the
King County Record Retention
In progress.
126 Appendix
centric policy and procedure
related to record retention, and
ensure its BHAs have a record
retention policy that mirrors the
BHO policy.
Schedule. The proposed
timeline is 4–6 months.
This will include the new ten-
year record retention
requirements for records
specified in 42 CFR Part 438.3.
431.107
Record
Retention
KCBHO’s administrative audit tool
does not include annual
monitoring of its delegated
entities for record retention.
KCBHO needs to update its
administrative audit tool to include
annual monitoring of its delegated
entities for record retention,
including but not limited to
credentialing and re-credentialing,
incident reporting, requests for
services, authorizations, clinical
records, complaints, grievances,
appeals, referrals for fraud, waste
and abuse, and outcomes of
fraud, waste and abuse.
KCBHO has included
monitoring of its delegated
entities for record retention in
its policy Monitoring: Monitoring
Plans for BHAs.
Resolved.
455.100
Excluded
Entities
Section 4 of the BHO’s program
integrity policy references
excluded provider checks but
doesn’t include information
regarding the requirement to
perform checks on a monthly
basis not only for new staff but
also for existing staff, or to
maintain a formal monthly log for
monitoring and reporting
purposes.
KCBHO should update its
program integrity policy to include
language regarding the
requirement to perform monthly
exclusion checks on all staff and
add language that it will retain a
formal monthly log for monitoring
and reporting purposes.
KCBHO is currently revising its
process for excluded providers,
which will be completed by
June 30, 2017. An active list of
all staff, including department
management, will be included
in initial and ongoing monthly
screenings. The active list will
be available for review at future
EQR site visits. The program
integrity policy and procedure
will be reviewed and, as
needed, revised to incorporate
this process by June 30, 2017.
In progress.
Performance
Improvement
Project (PIP)
Validation:
Non-Clinical
The BHO has not fully followed
through on last year’s
recommendation to fully formulate
and begin this PIP.
KCBHO has fully formulated
this PIP and have moved
beyond the initial phases. The
BHO has revised its data
sharing approach and has
Resolved.
127 Appendix
PIP KCBHO needs to clearly state its
interventions as well as clear
criteria for how it will determine
which intervention will be
implemented for which youth.
KCBHO needs to begin collecting,
reporting and analyzing data for
youth dually enrolled in the BHO
and Molina Apple Healthcare plan
as stated by the BHO in its PIP
study question.
identified the intervention
(CCORS or Wraparound)
based on the needs of the
enrollee.
KCBHO has been successful in
collecting the data from Molina
and has identified a care
coordination process to be used
for the youth involved in this
PIP.
Due to the previous difficulties
in obtaining data, KCBHO no
longer uses the monthly BHO-
to-MCO data files and MCO-to-
BHO data files.
Performance
Improvement
Project (PIP)
Validation:
Non-Clinical
PIP
KCBHO has noted Wraparound
and CCORS as possible
interventions for youth involved in
this PIP. The BHO has not set a
clear plan how it will be
determined which youth will
receive which intervention.
KCBHO needs to clearly
articulate its criteria for
determining which intervention a
youth will receive if they are
involved in this PIP.
The BHO has revised its data
sharing approach and has
identified the intervention
(CCORS or Wraparound)
based on the needs of the
enrollee.
The revised intervention
includes an information
systems component of sharing
the care plan and patient
records on EDIE/PreManage
for the BHO and Molina and
providers to access.
Resolved.
Performance
Improvement
Project (PIP)
Validation:
Non-Clinical
PIP
While there is value in continually
striving to work to obtain data
from all of the MCOs, this does
not have to be completed in
conjunction with a PIP. KCBHO
has been in the beginning stages
of this PIP for multiple years,
without collecting and analyzing
data from a single MCO, including
Molina.
KCBHO needs to begin the work
of this PIP. KCBHO needs to
collect and analyze the data it
proposes to aggregate and
interpret for enrollees it shares
with Molina Apple Health and
KCBHO has revised its data
collection strategies and is
focusing on collaborating with
Molina at this time. The BHO
has made strides in obtaining
and sharing data with Molina on
dually enrolled enrollees.
Resolved.
128 Appendix
investigate if its proposed
intervention will achieve the
desired outcome.
129 Appendix
Appendix C: Readiness Assessment Follow-up
As a part of the 2016 EQR, Qualis Health assessed each BHO’s status in integrating SUD treatment
agencies into the BHO structure. Reviewers conducted SUD provider interviews, as well as SUD provider
agency walkthroughs. The following table presents the observations and opportunities for improvement
identified during those review activities, as well as Qualis Health’s 2017 follow-up of the BHO’s progress
in addressing those items.
Opportunity for Improvement BHO Progress Since 2016
Review
Status
Employee Conduct
At one SUD agency, visitors
were not required to sign in or
review the confidentiality notice
posted on the wall in the lobby,
nor were they given visitor
badges. At this SUD provider
agency, the employees were not
wearing identification badges.
While performing the on-site
walkthrough of the provider
facility, Qualis Health was unable
to distinguish outside guests and
enrollees from employees.
KCBHO did not follow up with the
agency.
Opportunity remains.
Workstation Use
At one agency, workstations and
computer monitors were not
positioned to prevent
unauthorized persons from
viewing ePHI.
KCBHO did not follow up with the
agency.
Opportunity remains.
At one agency, unattended
computers are not returned to the
logon screen (automatically
or by user) or have password-
enabled screensavers when not
in use.
KCBHO did not follow up with the
agency.
Opportunity remains.
Access Controls
N/A
Environmental Controls
At one agency, server equipment
was not stationed away from
sprinklers and other water
supplies.
KCBHO did not follow up with the
agency.
Opportunity remains.
Enrollee Rights and Grievances
Enrollee rights were posted but
were out of date, and SUD
providers were unaware of how
KCBHO did not follow up with the
agencies.
Opportunity remains.
130 Appendix
often and by what means the
BHO informs enrollees about
their rights, responsibilities and
benefits. ADA Requirements
One facility did not meet ADA
accessibility requirements for
individuals with physical
disabilities. Although the
sidewalk leading up to the front
entrance was accessible, the
door had to be opened manually.
The SUD director stated that
anyone in a wheelchair trying to
gain access to the building would
need to wait outside until
someone noticed the person
waiting. The lobby was on the
second floor, so the client would
either have to wait for another
client to help or wait for an SUD
public safety officer to open the
door.
KCBHO did not follow up with the
agency.
Opportunity remains.
Medication Monitoring
N/A
Seclusion and Restraint
One provider stated that with
regard to seclusion and restraint,
the agency engaged in a “hands-
off policy.” However, the agency
had no written policy in place
indicating the provider didn’t
engage in seclusion or restraint,
and the provider had not adopted
the BHO’s policy indicating the
provider doesn’t engage in
seclusion and restraint.
KCBHO did not follow up with the
agency.
Opportunity remains.
Miscellaneous Environment of Care
At one provider, poisonous
chemicals or caustic materials
were not safely stored and
locked up.
KCBHO did not follow up with the
agency.
Opportunity remains.
At one provider, no visible
evacuation plans and maps of
evacuation routes were posted in
the lobby.
KCBHO did not follow up with the
agency.
Opportunity remains.
131 Appendix
132 Appendix
Appendix D: Coordination of Care Clinical Chart Review
Results and Scoring Definitions
42 CFR §438.208—Coordination and continuity of care specifies that “each MCO, PIHP, and PAHP” (the
BHO, as applied to this review) must implement procedures to deliver care and coordinate services for all
enrollees. These procedures must meet State requirements and must do the following:
ensure the BHO has and implements a policy and procedure for delivering care and coordinating
healthcare services for all enrollees
ensure enrollees have an ongoing source of care appropriate for their needs and access to
providers responsible for coordinating their care and services
ensure that each enrollee has access to a primary healthcare provider appropriate to the
enrollee’s needs
ensure the BHO has a process in place to communicate enrollees’ healthcare needs with other
service providers to prevent duplication of activities
ensure the BHO has a process in place with its BHAs to monitor care coordination and other
healthcare services furnished to enrollees
To assess the degree of coordination of care at the BHO’s behavioral health agencies (BHAs), Qualis
Health completed 17 clinical chart reviews evaluating coordination of care at two agencies. Although this
activity was not completed for all of the BHO’s contracted providers, it does provide a snapshot of the
care provided by the two agencies where charts were reviewed. The results of this assessment for
KCBHO are presented in the table below.
Clinical Record Review: N=17 charts
Number of charts in which the intake assessment included:
the name of the individual’s treatment supporters
9/17
o substance use disorder treatment provider
0
o school staff/teacher 3
o Aging and Long-Term Support Administration
0
o tribes 0
o physician/psychiatrist 9
o criminal justice 0
o Offender Re-entry Community Safety Program
0
o Children's Administration 0
o Department of Corrections 0
133 Appendix
o Division of Vocational Rehabilitation
0
o other 5
a description of concerns 13
medications currently taken 5
Number of charts in which the individual service plan included:
a plan to coordinate care with relevant treatment supporters
4/17
Number of charts in which the clinical record contained:
documentation of coordination 9/17
o a description of care coordination in progress note
10
o correspondence with supporters 8
o documentation received from supporters
2
o release of information for supporters
10
medication records, if applicable 1
(5 N/A)
laboratory records, if applicable 0
(11 N/A)
Overall scoring
Number of charts in which the clinical record reflects:
very good care coordination 1
good care coordination 2
fair care coordination 3
poor care coordination 1
very poor care coordination 10
Scoring Definitions
“Very good care coordination” means the clinical record reflects consistent, active coordination of care
with treatment supporters. Concerns and/or conditions are clearly identified, and the individual service
plan addresses coordination of care with relevant supporters. There is evidence of effective
communication flow between the behavioral health provider and treatment supporters.
134 Appendix
“Good care coordination” means the clinical record reflects coordination of care with treatment
supporters. Concerns are clearly identified, and the individual service plan addresses coordination of care
with relevant supporters. There is evidence of communication between the behavioral health professional
and treatment supporters, but it is not of sufficient consistency or quality to substantiate a higher rating.
“Fair care coordination” means the clinical record reflects only a moderate level of care coordination with
treatment supporters. Concerns and/or conditions are not clearly identified, or identified concerns and/or
conditions are not adequately addressed in the individual service plan. The record contains some
evidence of communication between the behavioral health and treatment supporters, but it is either
inconsistent or the content is questionably meaningful.
“Poor care coordination” means the clinical record reflects minimal care coordination.
Concerns/conditions are not clearly identified, and/or the individual service plan does not address medical
issues. The record may contain some evidence of communication between behavioral health providers
and treatment supporters, but it is generally administrative in nature.
“Very poor coordination” means the clinical record contains no evidence of care coordination with
treatment supporters. Concerns and/or conditions are not identified. The individual service plan does not
address concerns. The record contains no evidence of communication between behavioral health
providers and treatment supporters.
135 Appendix
Appendix E: Regulatory and Contractual Standards
Following are the regulatory standards cited in the Code of Federal Regulations (CFR) that BHOs are
required to meet, as well as the applicable elements of the BHOs’ contract with DBHR. The standards are
followed by the corresponding scoring criteria Qualis Health’s review team used to assess the BHOs on
their compliance with these standards. The results of that assessment are reflected in the compliance
chapter of this report.
Availability of Services §438.206 Availability of services.
(a) Basic rule. Each State must ensure that all services covered under the State plan are available and
accessible to enrollees of MCOs, PIHPs, and PAHPs in a timely manner. The State must also ensure that
MCO, PIHP and PAHP provider networks for services covered under the contract meet the standards
developed by the State in accordance with §438.68.
(b) Delivery network. The State must ensure, through its contracts, that each MCO, PIHP and PAHP,
consistent with the scope of its contracted services, meets the following requirements:
(1) Maintains and monitors a network of appropriate providers that is supported by written agreements
and is sufficient to provide adequate access to all services covered under the contract for all enrollees,
including those with limited English proficiency or physical or mental disabilities.
§438.68 Network adequacy standards.
(a) General rule. A State that contracts with an MCO, PIHP or PAHP to deliver Medicaid services must
develop and enforce network adequacy standards consistent with this section.
(b) Provider-specific network adequacy standards. (1) At a minimum, a State must develop time and
distance standards for the following provider types, if covered under the contract:
(i) Primary care, adult and pediatric.
(iii) Behavioral health (mental health and substance use disorder), adult and pediatric.
(iv) Specialist, adult and pediatric.
(v) Hospital.
(vi) Pharmacy.
(viii) Additional provider types when it promotes the objectives of the Medicaid program, as determined by
CMS, for the provider type to be subject to time and distance access standards.
(3) Scope of network adequacy standards. Network standards established in accordance with paragraphs
(b)(1) and (2) of this section must include all geographic areas covered by the managed care program or,
if applicable, the contract between the State and the MCO, PIHP or PAHP. States are permitted to have
varying standards for the same provider type based on geographic areas.
(c) Development of network adequacy standards. (1) States developing network adequacy standards
consistent with paragraph (b)(1) of this section must consider, at a minimum, the following elements:
(i) The anticipated Medicaid enrollment.
(ii) The expected utilization of services.
(iii) The characteristics and health care needs of specific Medicaid populations covered in the MCO,
PIHP, and PAHP contract.
(iv) The numbers and types (in terms of training, experience, and specialization) of network providers
required to furnish the contracted Medicaid services.
(v) The numbers of network providers who are not accepting new Medicaid patients.
(vi) The geographic location of network providers and Medicaid enrollees, considering distance, travel
time, the means of transportation ordinarily used by Medicaid enrollees.
136 Appendix
(vii) The ability of network providers to communicate with limited English proficient enrollees in their
preferred language.
(viii) The ability of network providers to ensure physical access, reasonable accommodations, culturally
competent communications, and accessible equipment for Medicaid enrollees with physical or mental
disabilities.
(ix) The availability of triage lines or screening systems, as well as the use of telemedicine, e-visits, and/or
other evolving and innovative technological solutions.
(2) States developing standards consistent with paragraph (b)(2) of this section must consider the
following:
(i) All elements in paragraphs (c)(1)(i) through (ix) of this section.
(ii) Elements that would support an enrollee's choice of provider.
(iii) Strategies that would ensure the health and welfare of the enrollee and support community integration
of the enrollee.(d) Exceptions process. (1) To the extent the State permits an exception to any of the
provider-specific network standards developed under this section, the standard by which the exception
will be evaluated and approved must be:
(i) Specified in the MCO, PIHP or PAHP contract.
(ii) Based, at a minimum, on the number of providers in that specialty practicing in the MCO, PIHP, or
PAHP service area.
(2) States that grant an exception in accordance with paragraph (d)(1) of this section to a MCO, PIHP or
PAHP must monitor enrollee access to that provider type on an ongoing basis and include the findings to
CMS in the managed care program assessment report required under §438.66.
(e) Publication of network adequacy standards. States must publish the standards developed in
accordance with paragraphs (b)(1) and (2) of this section on the Web site required by §438.10. Upon
request, network adequacy standards must also be made available at no cost to enrollees with disabilities
in alternate formats or through the provision of auxiliary aids and services.
State Regulation/BHO Agreement Sources
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 10.9.3.3, 5.8.4, 5.8.5, 5.12, 10.4.13
EQR Scoring Criteria
The BHO maintains and monitors an appropriate network of BHAs that is supported by
written agreements.
The BHO’s BHA network is sufficient to provide adequate access to all services covered
under the contract.
The BHO annually conducts geo mapping that includes the cultural, ethnic, racial, and
linguistic needs of its members; distance; travel time; the means of transportation
ordinarily used by Medicaid enrollees; and whether the location provides physical access
for Medicaid enrollees with disabilities.
In establishing and maintaining the network, the BHO considers:
o the anticipated Medicaid enrollment
o the expected utilization of services, taking into consideration the characteristics and
healthcare needs of specific Medicaid populations represented in the BHO
o the numbers and types (training, experience, and specialization) of BHAs required to
furnish the contracted Medicaid services
o the number of network BHAs that are not accepting new Medicaid patients
The BHO has formal procedures in place to monitor its BHA network to ensure adequacy.
137 Appendix
§438.206 Availability of services.
(b) Delivery network. The State must ensure, through its contracts, that each MCO, PIHP and PAHP,
consistent with the scope of its contracted services, meets the following requirements:
(3) Provides for a second opinion from a network provider within the network, or arranges for the enrollee
to obtain one outside the network, at no cost to the enrollee.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 10.9.3.3, 11.11
EQR Scoring Criteria
The BHO maintains policies and procedures related to second opinions that meet the
contract and WAC standards.
BHO staff are knowledgeable about State and federal requirements, as well as internal
policies and procedures.
The BHO provides literature or other materials available to enrollees to provide
information about the enrollee’s right to a second opinion.
The BHO provides for a second opinion from a qualified healthcare professional within
the network, or arranges for the enrollee to obtain one outside the network, at no cost to
the enrollee.
The BHO has an effective process in place to monitor compliance with standards.
§438.206 Availability of services.
(b) Delivery network. The State must ensure, through its contracts, that each MCO, PIHP and PAHP,
consistent with the scope of its contracted services, meets the following requirements:
(4) If the provider network is unable to provide necessary services, covered under the contract, to a
particular enrollee, the MCO, PIHP, or PAHP must adequately and timely cover these services out of
network for the enrollee, for as long as the MCO, PIHP, or PAHP's provider network is unable to provide
them.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 10.9.3.3, 15.3, 15.4
EQR Scoring Criteria
The BHO provides documentation of adequate and timely covered services for out-of-
network enrollees when the network is unable to provide necessary services covered
under the contract.
The BHO provides up-to-date existing agreements and/or contracts with out-of-network
providers.
The BHO has a process to track out-of-network encounters and reviews this information
for network capacity planning.
§438.206 Availability of services.
(b) Delivery network. The State must ensure, through its contracts, that each MCO, PIHP and PAHP,
consistent with the scope of its contracted services, meets the following requirements:
138 Appendix
(5) Requires out-of-network providers to coordinate with the MCO, PIHP, or PAHP for payment and
ensures the cost to the enrollee is no greater than it would be if the services were furnished within the
network.
State Regulation/BHO Agreement Sources
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 15.3, 15.4
EQR Scoring Criteria
The BHO has a documented process and policy that require out-of-network providers to
coordinate with the BHO with respect to payment.
The BHO has a documented process for how out-of-network providers are paid.
The BHO ensures and has a documented policy and process that cost to the enrollee is
not greater than it would be if the out-of- network services were furnished within the
network.
The BHO has a process for the action taken if an enrollee receives a bill for out-of-
network services.
§438.206 Availability of services.
(b) Delivery network. The State must ensure, through its contracts, that each MCO, PIHP and PAHP,
consistent with the scope of its contracted services, meets the following requirements:
(6) Demonstrates that its network providers are credentialed as required by §438.214.
§438.214 Provider selection.
(a) General rules. The State must ensure, through its contracts, that each MCO, PIHP, or PAHP
implements written policies and procedures for selection and retention of network providers and that
those policies and procedures, at a minimum, meet the requirements of this section.
(b) Credentialing and recredentialing requirements. (1) Each State must establish a uniform
credentialing and recredentialing policy that addresses acute, primary, behavioral, substance use
disorders and requires each MCO, PIHP and PAHP to follow those policies.
(2) Each MCO, PIHP, and PAHP must follow a documented process for credentialing and
recredentialing of network providers.
(c) Nondiscrimination. MCO, PIHP, and PAHP network provider selection policies and procedures,
consistent with §438.12, must not discriminate against particular providers that serve high-risk
populations or specialize in conditions that require costly treatment.
(d) Excluded providers. (1) MCOs, PIHPs, and PAHPs may not employ or contract with providers
excluded from participation in federal health care programs under either section 1128 or section
1128A of the Act.
(c) [Reserved]
(e) State requirements. Each MCO, PIHP, and PAHP must comply with any additional requirements
established by the State.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 8, 8.5
EQR Scoring Criteria
The BHO has a documented process and policy to ensure that out-of-network providers are
appropriately credentialed.
139 Appendix
§438.206 Availability of services.
(c) Furnishing of services. The State must ensure that each MCO, PIHP and PAHP contract complies with
the requirements:
(1) Timely Access. Each MCO, PIHP and PAHP must do the following:
(i) Meet and require its providers to meet State standards for timely access to care and
services, taking into account the urgency of the need for services.
(ii) Ensure that the network providers offer hours of operation that are no less than the hours of
operation offered to commercial enrollees or comparable to Medicaid FFS, if the provider
serves only Medicaid enrollees.
(iii) Make services included in the contract available 24 hours a day, 7 days a week, when
medically necessary.
(iv) Establish mechanisms to ensure compliance by network providers.
(v) Monitor network providers regularly to determine compliance.
(vi) Take corrective action if there is a failure to comply by a network provider.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 10.9.3.2
EQR Scoring Criteria
The BHO has and implements a policy for timely access.
The BHO requires its providers to meet State Medicaid standards for timely access to
care and services.
The BHO ensures that the network BHAs offer hours of operation that are no less than
the hours of operation offered to commercial enrollees or comparable to Medicaid fee-for-
service, if the BHA serves only Medicaid enrollees.
The BHO has established mechanisms to ensure services included in the contract are
available 24 hours a day, 7 days a week, when medically necessary.
The BHO takes corrective action and has documentation of such corrective action if BHAs fail to
comply with access standards.
§438.206 Availability of services.
(c) Furnishing of services. The State must ensure that each MCO, PIHP and PAHP contract complies with
the requirements:
(2) Access and cultural considerations. Each MCO, PIHP, and PAHP participates in the State's efforts
to promote the delivery of services in a culturally competent manner to all enrollees, including those
with limited English proficiency and diverse cultural and ethnic backgrounds, disabilities, and
regardless of gender, sexual orientation or gender identity.
(3) Accessibility considerations. Each MCO, PIHP, and PAHP must ensure that network providers
provide physical access, reasonable accommodations, and accessible equipment for Medicaid
enrollees with physical or mental disabilities.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 5.8.5.4, 10.9.3.4, 11.4.2.2, 11.9, 12.6.1.4
EQR Scoring Criteria
140 Appendix
The BHO has a documented policy and procedure related to the delivery of services in a
culturally competent manner to all enrollees, including those with limited English
proficiency and diverse cultural and ethnic backgrounds.
The BHO monitors and has documented tracking of the delivery of services to those with
limited English proficiency and diverse cultural and ethnic backgrounds.
The BHO has documentation of any cultural competency training(s).
The BHO has a documented policy and procedure related to accessibility considerations
to ensure that BHAs provide physical access, reasonable accommodations, and
accessible equipment for Medicaid enrollees with physical or mental disabilities.
The BHO monitors its BHAs to ensure they provide adequate physical access,
reasonable accommodations, and accessible equipment for Medicaid enrollees with
physical or mental disabilities.
Coordination of Care §438.208 Coordination and continuity of care.
(b) Care and coordination of services for all MCO, PIHP, and PAHP enrollees. Each MCO, PIHP, and
PAHP must implement procedures to deliver care to and coordinate services for all MCO, PIHP, and
PAHP enrollees. These procedures must meet State requirements and must do the following:
(1) Ensure that each enrollee has an ongoing source of care appropriate to his or her needs and a person
or entity formally designated as primarily responsible for coordinating the services accessed by the
enrollee. The enrollee must be provided information on how to contact their designated person or entity;
(2) Coordinate the services the MCO, PIHP, or PAHP furnishes to the enrollee:
(i) Between settings of care, including appropriate discharge planning for short term and long-term
hospital and institutional stays;
(ii) With the services the enrollee receives from any other MCO, PIHP, or PAHP;
(iii) With the services the enrollee receives in FFS Medicaid; and
(iv) With the services the enrollee receives from community and social support providers.
(3) Provide that the MCO, PIHP or PAHP makes a best effort to conduct an initial screening of each
enrollee's needs, within 90 days of the effective date of enrollment for all new enrollees, including
subsequent attempts if the initial attempt to contact the enrollee is unsuccessful;
(4) Share with the State or other MCOs, PIHPs, and PAHPs serving the enrollee the results of any
identification and assessment of that enrollee's needs to prevent duplication of those activities;
(5) Ensure that each provider furnishing services to enrollees maintains and shares, as appropriate, an
enrollee health record in accordance with professional standards.
State Regulation/BHO Agreement Source(s)
BHO Agreement Section(s) 10.3.1
EQR Scoring Criteria
The BHO must have and implement a policy and procedure for delivering care and coordinating
healthcare services for all enrollees.
The BHO must ensure enrollees have an ongoing source of care appropriate for their needs and
access to providers responsible for coordinating enrollees’ care and services.
The BHO ensures that each enrollee has access to a primary healthcare provider appropriate to
the enrollee’s needs.
The BHO has a process in place to communicate enrollees’ healthcare needs with other service
providers to prevent duplication of activities.
The BHO has a process in place to monitor care coordination and other healthcare services
141 Appendix
furnished to enrollees with its BHAs.
§438.208 Coordination and continuity of care.
b) Care and coordination of services for all MCO, PIHP, and PAHP enrollees. Each MCO, PIHP, and
PAHP must implement procedures to deliver care to and coordinate services for all MCO, PIHP, and
PAHP enrollees. These procedures must meet State requirements and must do the following:
(6) Ensure that in the process of coordinating care, each enrollee's privacy is protected in accordance
with the privacy requirements in 45 CFR parts 160 and 164 subparts A and E, to the extent that they are
applicable.
45 CFR Part 160, Subpart C—Compliance and Investigations
This subpart applies to actions by the Secretary, covered entities, business associates, and others with
respect to ascertaining the compliance by covered entities and business associates with, and the
enforcement of, the applicable provisions of this part 160 and parts 162 and 164 of this subchapter.
§160.310 Responsibilities of covered entities and business associates.
(a) Provide records and compliance reports. A covered entity or business associate must keep such
records and submit such compliance reports, in such time and manner and containing such information,
as the Secretary may determine to be necessary to enable the Secretary to ascertain whether the
covered entity or business associate has complied or is complying with the applicable administrative
simplification provisions.
(b) Cooperate with complaint investigations and compliance reviews. A covered entity or business
associate must cooperate with the Secretary, if the Secretary undertakes an investigation or compliance
review of the policies, procedures, or practices of the covered entity or business associate to determine
whether it is complying with the applicable administrative simplification provisions.
(c) Permit access to information.
(1) A covered entity or business associate must permit access by the Secretary during normal business
hours to its facilities, books, records, accounts, and other sources of information, including protected
health information, that are pertinent to ascertaining compliance with the applicable administrative
simplification provisions. If the Secretary determines that exigent circumstances exist, such as when
documents may be hidden or destroyed, a covered entity or business associate must permit access by
the Secretary at any time and without notice.
(2) If any information required of a covered entity or business associate under this section is in the
exclusive possession of any other agency, institution, or person and the other agency, institution, or
person fails or refuses to furnish the information, the covered entity or business associate must so certify
and set forth what efforts it has made to obtain the information.
(3) Protected health information obtained by the Secretary in connection with an investigation or
compliance review under this subpart will not be disclosed by the Secretary, except if necessary for
ascertaining or enforcing compliance with the applicable administrative simplification provisions, if
otherwise required by law, or if permitted under 5 U.S.C. 552a(b)(7).
§160.316 Refraining from intimidation or retaliation.
A covered entity or business associate may not threaten, intimidate, coerce, harass, discriminate against,
or take any other retaliatory action against any individual or other person for—
(a) Filing of a complaint under §160.306;
(b) Testifying, assisting, or participating in an investigation, compliance review, proceeding, or hearing
under this part; or
(c) Opposing any act or practice made unlawful by this subchapter, provided the individual or person has
a good faith belief that the practice opposed is unlawful, and the manner of opposition is reasonable and
142 Appendix
does not involve a disclosure of protected health information in violation of subpart E of part 164 of this
subchapter.
State Regulation/BHO Agreement Source(s)
BHO Agreement Section(s) 10.9.3.5, 6.12.1.8
EQR Scoring Criteria
The BHO ensures that in the process of coordinating care, each enrollee’s privacy is protected in
accordance with the privacy requirements of HIPAA (45 CFR 160).
The BHO monitors for compliance to HIPAA regulations and takes action to correct any
deficiencies.
The BHO has a policy and procedure to provide its own records and compliance reports, as well
as those of its covered entities or business associates, indicating all parties will comply with
applicable HIPAA administrative simplification provisions.
The BHO has a policy and procedure to ensure a covered entity or business associate may not
threaten, intimidate, coerce, harass, discriminate against, or take any other retaliatory action
against any individual or other person for filing a complaint with the State regarding HIPAA
compliance.
The BHO has a policy and procedure in place ensuring the BHO and its contracted entities
cooperate with HIPAA complaint investigations and compliance reviews.
42 U.S.C. §290dd–2 Confidentiality of records
(a) Requirement
Records of the identity, diagnosis, prognosis, or treatment of any patient which are maintained in
connection with the performance of any program or activity relating to substance abuse education,
prevention, training, treatment, rehabilitation, or research, which is conducted, regulated, or directly or
indirectly assisted by any department or agency of the United States shall, except as provided in
subsection (e) of this section, be confidential and be disclosed only for the purposes and under the
circumstances expressly authorized under subsection (b) of this section.
(b) Permitted disclosure
(1) Consent
The content of any record referred to in subsection (a) of this section may be disclosed in accordance
with the prior written consent of the patient with respect to whom such record is maintained, but only to
such extent, under such circumstances, and for such purposes as may be allowed under regulations
prescribed pursuant to subsection (g) of this section.
(2) Method for disclosure
Whether or not the patient, with respect to whom any given record referred to in subsection (a) of this
section is maintained, gives his written consent, the content of such record may be disclosed as follows:
(A) To medical personnel to the extent necessary to meet a bona fide medical emergency.
(B) To qualified personnel for the purpose of conducting scientific research, management audits,
financial audits, or program evaluation, but such personnel may not identify, directly or indirectly, any
individual patient in any report of such research, audit, or evaluation, or otherwise disclose patient
identities in any manner.
(C) If authorized by an appropriate order of a court of competent jurisdiction granted after application
showing good cause therefor. In assessing good cause the court shall weigh the public interest and the
need for disclosure against the injury to the patient, to the physician-patient relationship, and to the
treatment services. Upon the granting of such order, the court, in determining the extent to which any
143 Appendix
disclosure of all or any part of any record is necessary, shall impose appropriate safeguards against
unauthorized disclosure.
State Regulation/BHO Agreement Source(s)
BHO Agreement Section(s) 3.4.1, 3.4.2
EQR Scoring Criteria
The BHO has a policy and procedure to deliver care to and coordinate healthcare services for all
enrollees ensuring confidentiality and appropriate disclosure only with the enrollee’s consent.
The BHO has a policy and procedure to deliver care to and coordinate healthcare services for all
enrollees identifying the purposes and circumstances of disclosure affecting the enrollee
regardless of consent.
CFR Part 2—Confidentiality of Alcohol and Drug Abuse Patient Records
§2.12 Applicability.
(a) General. (1) Restrictions on disclosure. The restrictions on disclosure in these regulations apply to any
information, whether or not recorded, which:
(i) Would identify a patient as having or having had a substance use disorder directly, by reference to
other publicly available information, or through verification of such an identification by another person;
and
(ii) Is drug abuse information obtained by a federally assisted drug abuse program after March 20, 1972
(part 2 program) or is alcohol abuse information obtained by a federally assisted alcohol abuse program
after May 13, 1974 (part 2 program);or if obtained before the pertinent date, is maintained by a part 2
program after that date as part of an ongoing treatment episode which extends past that date; for the
purpose of treating a substance use disorder, making a diagnosis for that treatment, or making a referral
for that treatment.
2) Restriction on use. The restriction on use of information to initiate or substantiate any criminal charges
against a patient or to conduct any criminal investigation of a patient (42 U.S.C. 290dd–2(c)), applies to
any information, whether or not recorded, which is drug abuse information obtained by a federally
assisted drug abuse program after March 20, 1972 (part 2 program), or is alcohol abuse information
obtained by a federally assisted alcohol abuse program after May 13, 1974 (part 2 program); or if
obtained before the pertinent date, is maintained by a part 2 program after that date as part of an ongoing
treatment episode which extends past that date; for the purpose of treating substance use disorder,
making a diagnosis for the treatment, or making a referral for the treatment.
(b) Federal assistance. A program is considered to be federally assisted if:
(1) It is conducted in whole or in part, whether directly or by contract or otherwise by any department or
agency of the United States (but see paragraphs (c)(1) and (c)(2) of this section relating to the Veterans'
Administration and the Armed Forces);
(2) It is being carried out under a license, certification, registration, or other authorization granted by any
department or agency of the United States including but not limited to:
(i) Participating provider in the Medicare program;
(ii) Authorization to conduct maintenance treatment or withdrawal management; or
(iii) Registration to dispense a substance under the Controlled Substances Act to the extent the controlled
substance is used in the treatment of substance use disorders;
(3) It is supported by funds provided by any department or agency of the United States by being:
(i) A recipient of federal financial assistance in any form, including financial assistance which does not
directly pay for the substance use disorder diagnosis, treatment, or referral activities; or
144 Appendix
(ii) Conducted by a State or local government unit which, through general or special revenue sharing or
other forms of assistance, receives federal funds which could be (but are not necessarily) spent for the
substance use disorder program.
(c) Exceptions.
(3) Communication within a part 2 program or between a part 2 program and an entity having direct
administrative control over that part 2 program. The restrictions on disclosure in the regulations in this
part do not apply to communications of information between or among personnel having a need for the
information in connection with their duties that arise out of the provision of diagnosis, treatment, or referral
for treatment of patients with substance use disorders if the communications are
(i) Within a part 2 program or
(ii) Between a part 2 program and an entity that has direct administrative control over the program.
(4) Qualified Service Organizations. The restrictions on disclosure in these regulations do not apply to
communications between a part 2 program and a qualified service organization of information needed by
the organization to provide services to the program.
(5) Crimes on part 2 program premises or against part 2 program personnel. The restrictions on
disclosure and use in these regulations do not apply to communications from part 2 program personnel to
law enforcement agencies or officials which—
(i) Are directly related to a patient's commission of a crime on the premises of the part 2 program or
against part 2 program personnel or to a threat to commit such a crime; and
(ii) Are limited to the circumstances of the incident, including the patient status of the individual
committing or threatening to commit the crime, that individual's name and address, and that individual's
last known whereabouts.
§2.16 Security for written records.
(a) The part 2 program or other lawful holder of patient identifying information must have in place formal
policies and procedures to reasonably
protect against unauthorized uses and disclosures of patient identifying information and to protect against
reasonably anticipated threats or
hazards to the security of patient identifying information. These formal policies and procedures must
address: (1) Paper records, including:
(i) Transferring and removing such records;
(ii) Destroying such records, including sanitizing the hard copy media associated with the paper printouts,
to render the patient identifying information non-retrievable;
(iii) Maintaining such records in a secure room, locked file cabinet, safe, or other similar container, or
storage facility when not in use;
(iv) Using and accessing workstations, secure rooms, locked file cabinets, safes, or other similar
containers, and storage facilities that use or store such information; and
(v) Rendering patient identifying information non-identifiable in a manner that creates a very low risk of re-
identification (e.g., removing direct identifiers).
(2) Electronic records, including:
(i) Creating, receiving, maintaining, and transmitting such records;
(ii) Destroying such records, including sanitizing the electronic media on which such records are stored, to
render the patient identifying information nonretrievable;
(iii) Using and accessing electronic records or other electronic media containing patient identifying
information; and
(iv) Rendering the patient identifying information non-identifiable in a manner that creates a very low risk
of re-identification (e.g., removing direct identifiers).
§2.19 Disposition of records by discontinued programs.
145 Appendix
(a) General. If a part 2 program discontinues operations or is taken over or acquired by another program,
it must remove patient identifying information from its records or destroy the records, including sanitizing
any associated hard copy or electronic media, to render the
patient identifying information nonretrievable in a manner consistent with the policies and procedures
established under § 2.16, unless:
(1) The patient who is the subject of the records gives written consent (meeting the requirements of
§2.31) to a transfer of the records to the acquiring program or to any other program designated in the
consent (the manner of obtaining this consent must minimize the likelihood of a disclosure of patient
identifying information to a third party); or
(2) There is a legal requirement that the records be kept for a period specified by law which does not
expire until after the discontinuation or acquisition of the part 2 program.
(b) Special procedure where retention period required by law. If paragraph (a)(2) of this section applies,:
(1) Records, which are paper, must be
(i) sealed in envelopes or other containers labeled as follows: “Records of [insert name of program]
required to be maintained under [insert citation to statute, regulation, court order or other legal authority
requiring that records be kept] until a date not later than [insert appropriate date]”;
(A) All hard copy media from which
the paper records were produced, such as printer and facsimile ribbons, drums, etc., must be sanitized to
render the data non-retrievable; and
(B) [Reserved]
(ii) Held under the restrictions of the regulations in this part by a responsible person who must, as soon as
practicable after the end of the required retention period specified on the label, destroy the records and
sanitize any associated hard copy media to render the patient identifying information non-retrievable in a
manner consistent with the discontinued program’s or acquiring program’s policies and procedures
established under § 2.16.
§2.22 Notice to patients of federal confidentiality requirements.
(a) Notice required. At the time of admission to a part 2 program or, in the case that a patient does not
have capacity upon admission to understand his or her medical status, as soon threreafter as the patient
attains such capacity, each part 2 program shall:
(1) Communicate to the patient that federal law and regulations protect the confidentiality of substance
use disorder patient records; and
(2) Give to the patient a summary in writing of the federal law and regulations.
(b) Required elements of written summary. The written summary of the federal law and regulations must
include:
(1) A general description of the limited circumstances under which a part 2 program may acknowledge
that an individual is present or disclose outside the part 2 program information identifying a patient as
having or having had a substance use disorder;
(2) A statement that violation of the federal law and regulations by a part 2 program is a crime and that
suspected violations may be reported to appropriate authorities consisted with §2.4, along with contact
information .
(3) A statement that information related to a patient's commission of a crime on the premises of the part 2
program or against personnel of the part 2 program is not protected.
(4) A statement that reports of suspected child abuse and neglect made under state law to appropriate
state or local authorities are not protected; and
(5) A citation to the federal law and regulations.
(c) Program options. The part 2 program must devise a notice to comply with the requirement to provide
the patient with a summary in writing of the
146 Appendix
federal law and regulations. In this written summary, the part 2 program also may include information
concerning state law and any of the part
2 program’s policies that are not inconsistent with state and federal law on the subject of confidentiality of
substance use disorder patient records.
§2.23 Patient access and restrictions on use.
(a) Patient access not prohibited. These regulations do not prohibit a part 2 program from giving a patient
access to their own records, including the opportunity to inspect and copy any records that the part 2
program maintains about the patient. The part 2 program is not required to obtain a patient's written
consent or other authorization under these regulations in this part in order to provide such access to the
patient.
(b) Restriction on use of information. Information obtained by patient access to his or her patient record is
subject to the restriction on use of his information to initiate or substantiate any criminal charges against
the patient or to conduct any criminal investigation of the patient as provided for under §2.12(d)(1).
State Regulation/BHO Agreement Source(s)
BHO Agreement Section(s) 3.4.1, 3.4.2
EQR Scoring Criteria
The BHO ensures that in the process of coordinating care, each enrollee’s privacy is protected in
accordance with the privacy requirements detailed in 42 CFR Part 2.
The BHO has a policy and procedure to ensure that when coordinating care for enrollees, the
BHO and its business associates, covered entities, and contracted providers follow restrictions on
disclosure of enrollee information, security for written records, and disposition of records for
discontinued programs.
The BHO has a policy and procedure to ensure the confidentiality of enrollee drug and alcohol
records, including when disclosure will be granted, patient access, and restrictions on use of
information.
The BHO has a process in place to monitor its BHAs for care coordination and all other
healthcare services furnished to enrollees to make certain policies regarding confidentiality of
enrollee drug and alcohol records, as well as disclosure, enrollee access, and restrictions on use
of information are followed.
The BHO notifies enrollees of the circumstances in which disclosure is permitted without the
enrollee’s consent.
42 CFR Part 2—Confidentiality of Alcohol and Drug Abuse Patient Records
§2.53 Audit and evaluation activities.
(a) Records not copied or removed. If patient records are not downloaded, copied or removed from the
part 2 program premises or forwarded electronically to another electronic system or device, patient
identifying information, as defined in § 2.11, may be disclosed in the course of a review of records on the
part 2 program premises to any individual or entity who agrees in writing to comply with the limitations on
re-disclosure and use in paragraph (d) of this section and who:
(1) Performs the audit or evaluation activity on behalf of:
(i) Any federal, state, or local governmental agency which provides financial assistance to the part 2
program or is authorized by law to regulate its activities; or
(ii) Any individual or entity who provides financial assistance to the part 2 program, which is a third party
payer covering patients in the part 2 program, or which is a quality improvement organization performing
a utilization or quality control review; or
147 Appendix
(2) Is determined by the part 2 program to be qualified to conduct the audit or evaluation of the part 2
program.
(b) Copying, removing, downloading, or forwarding patient records. Records containing patient identifying
information may be copied or removed from part 2 program premises by any individual or entity who:
(1) Agrees in writing to:
(i) Maintain and destroy the patient identifying information in a manner consistent with the policies and
procedres established under §2.16;
(ii) Retain records in compliance with applicable federal, state, and local record retention laws; and (iii)
Comply with the limitations on disclosure and use in paragraph (d) of this section; and
(2) Performs the audit or evaluation activity on behalf of:
(i) Any federal, state, or local governmental agency which provides financial assistance to the part 2
program or is authorized by law to regulate its activities; or
(ii) Any individual or entity who provides financial assistance to the part 2 program, which is a third part
payer covering patients in the part 2 program, or which is a quality improvement organization performing
a utilization or quality control review.
(c) Medicare, Medicaid, Children’s Health Insurance Program (CHIP), or related audit or evaluation. (2) A
Medicare, Medicaid, or CHIP audit or evaluation under this section includes a civil or administrative
investigation of a part 2 program by any federal, state, or local government agency with oversight
responsibilities for Medicare, Medicaid, or CHIP and includes administrative enforcement, against the part
2 program by the government agency, of any remedy authorized by law to be imposed as a result of the
findings of the investigation.
(d) Limitations on disclosure and use. Except as provided in paragraph (c) of this section, patient
identifying information disclosed under this section may be disclosed only back to the program from which
it was obtained and used only to carry out an audit or evaluation purpose or to investigate or prosecute
criminal or other activities, as authorized by a court order entered under §2.66 of these regulations.
State Regulation/BHO Agreement Source(s)
BHO Program Agreement Sections: 3.4.1
EQR Scoring Criteria
The BHO has a procedure and policy in place that describe the auditing and evaluation of SUD
treatment records in accordance with §2.53.
The BHO has a process in place to ensure that if enrollee records are copied or removed from a
BHA during an audit, the confidentiality of PHI is maintained in accordance with the security
requirements discussed in 42 CFR.
The BHO has a process in place to make sure all PHI removed from the BHAs during an audit is
destroyed upon completion of the audit.
§431.300 Basis and purpose.
(a) Section 1902(a)(7) of the Act requires that a State plan must provide safeguards that restrict the use
or disclosure of information concerning applicants and beneficiaries to purposes directly connected with
the administration of the plan. This subpart specifies State plan requirements, the types of information to
be safeguarded, the conditions for release of safeguarded information, and restrictions on the distribution
of other information.
(b) For purposes of this subpart, information concerning an applicant or beneficiary includes information
on a non-applicant, as defined in §435.4 of this subchapter.
148 Appendix
(c) Section 1137 of the Act, which requires agencies to exchange information to verify the income and
eligibility of applicants and beneficiaries (see §435.940 through §435.965 of this subchapter), requires
State agencies to have adequate safeguards to assure that—
(1) Information exchanged by the State agencies is made available only to the extent necessary to assist
in the valid administrative needs of the program receiving the information, and information received under
section 6103(l)(7) of the Internal Revenue Code is exchanged only with agencies authorized to receive
that information under that section of the Code; and
(2) The information is adequately stored and processed so that it is protected against unauthorized
disclosure for other purposes.
§431.301 State plan requirements.
A State plan must provide, under a State statute that imposes legal sanctions, safeguards meeting the
requirements of this subpart that restrict the use or disclosure of information concerning applicants and
beneficiaries to purposes directly connected with the administration of the plan.
§431.302 Purposes directly related to State plan administration.
Purposes directly related to plan administration include—
(a) Establishing eligibility;
(b) Determining the amount of medical assistance;
(c) Providing services for beneficiaries; and
(d) Conducting or assisting an investigation, prosecution, or civil or criminal proceeding related to the
administration of the plan.
§431.307 Distribution of information materials.
(a) All materials distributed to applicants, beneficiaries, or medical providers must—
(1) Directly relate to the administration of the Medicaid program.
State Regulation/BHO Agreement Source(s)
BHO Program Agreement Sections: 3.4.1
EQR Scoring Criteria
The BHO ensures that in the process of coordinating care, each enrollee’s privacy is protected in
accordance with the privacy requirements in §§431.300, 431.301, and 431.302.
The BHO has a policy and procedure in place to ensure information exchanged by the BHO and
its BHAs is made available only to the extent necessary to assist in the valid administrative needs
of the program receiving the information and is adequately stored and protected against
unauthorized disclosure.
The BHO has a mechanism in place to monitor that distribution of information directly relates to
the administration of the State plan.
§438.208 Coordination and continuity of care.
(c) Additional services for enrollees with special health care needs or who need LTSS—(1) Identification.
The State must implement mechanisms to identify persons with special health care needs to MCOs,
PIHPs and PAHPs, as those persons are defined by the State. These identification mechanisms—
(i) Must be specified in the State's quality strategy under §438.340.
(ii) May use State staff, the State's enrollment broker, or the State's MCOs, PIHPs and PAHPs.
(2) Assessment. Each MCO, PIHP, and PAHP must implement mechanisms to comprehensively assess
each Medicaid enrollee identified by the State (through the mechanism specified in paragraph (c)(1) of
this section) and identified to the MCO, PIHP, and PAHP by the State as having special health care
needs to identify any ongoing special conditions of the enrollee that require a course of treatment or
149 Appendix
regular care monitoring. The assessment mechanisms must use appropriate providers or individuals
meeting coordination requirements of the State or the MCO, PIHP, or PAHP as appropriate.
State Regulation/BHO Agreement Source(s)
WAC 388-865-0420
BHO Agreement Section(s)12.1.6, 12.2, 12.3
EQR Scoring Criteria
The BHO has a policy and procedure for assessing each enrollee in order to identify any ongoing
special conditions that may require a special course of treatment or regular care monitoring.
The BHO has a method for identifying enrollees with special healthcare needs. Special
healthcare needs include any physical, developmental, mental, sensory, behavioral, cognitive, or
emotional impairment or limiting condition that requires medical management, healthcare
intervention and/or use of specialized services or programs. The condition may be congenital or
developmental, or acquired through disease, trauma, or environmental cause and may impose
limitations in performing daily self-maintenance activities or substantial limitations in a major life
activity.
The BHO has a process in place to monitor compliance with this requirement.
The BHO has in effect mechanisms to detect both underutilization and overutilization of services,
and to assess the quality and appropriateness of care furnished to enrollees with special
healthcare needs.
§438.208 Coordination and continuity of care.
(3) Treatment/service plans. MCOs, PIHPs, or PAHPs must produce a treatment or service plan meeting
the criteria in paragraphs (c)(3)(i) through (v) of this section for enrollees who have special healthcare
needs and, if the State requires, must produce a treatment or service plan meeting the criteria in
paragraphs (c)(3)(iii) through (v) of this section for enrollees with special health care needs that are
determined through assessment to need a course of treatment or regular care monitoring. The treatment
or service plan must be:
(i) Developed by an individual meeting special healthcare service coordination requirements with enrollee
participation, and in consultation with any providers caring for the enrollee;
(iii) Approved by the MCO, PIHP, or PAHP in a timely manner, if this approval is required by the MCO,
PIHP, or PAHP;
(iv) In accordance with any applicable State quality assurance and utilization review standards; and
(v) Reviewed and revised upon reassessment of functional need, at least every 12 months, or when the
enrollee's circumstances or needs change significantly, or at the request of the enrollee per
§441.301(c)(3) of this chapter.
State Regulation/BHO Agreement Source(s)
BHO Agreement Section(s) 10.9.3.4, 10.9.3.5, 16.4, 3.6.19
EQR Scoring Criteria
The BHO ensures that treatment plans for enrollees with special healthcare needs are developed
with the enrollee’s participation, and in consultation with any specialists caring for the enrollee.
The enrollee’s treatment plan incorporates the enrollee’s special healthcare needs.
§438.208 Coordination and continuity of care.
150 Appendix
(4) Direct access to specialists. For enrollees with special health care needs determined through an
assessment by appropriate health care professionals (consistent with §438.208(c)(2)) to need a course of
treatment or regular care monitoring, each MCO, PIHP, and PAHP must have a mechanism in place to
allow enrollees to directly access a specialist (for example, through a standing referral or an approved
number of visits) as appropriate for the enrollee's condition and identified needs.
State Regulation/BHO Agreement Source(s)
WAC 388-865-0430
BHO Agreement Section(s) 10.9
EQR Scoring Criteria
The BHO has policies and procedures regarding direct access to specialists for enrollees with
special healthcare needs.
The BHO monitors the availability of direct access to specialists.
Coverage and Authorization of Services §438.210 Coverage and authorization of services.
(a) Coverage. Each contract between a State and an MCO, PIHP, or PAHP must do the following:
(1) Identify, define, and specify the amount, duration, and scope of each service that the MCO, PIHP, or
PAHP is required to offer.
(2) Require that the services identified in paragraph (a)(1) of this section be furnished in an amount,
duration, and scope that is no less than the amount, duration, and scope for the same services furnished
to beneficiaries under FFS Medicaid, as set forth in §440.230 of this chapter, and for enrollees under the
age of 21, as set forth in subpart B of part 440 of this chapter.
(3) Provide that the MCO, PIHP, or PAHP—
(i) Must ensure that the services are sufficient in amount, duration, or scope to reasonably achieve the
purpose for which the services are furnished.
(ii) May not arbitrarily deny or reduce the amount, duration, or scope of a required service solely because
of diagnosis, type of illness, or condition of the beneficiary.
(4) Permit an MCO, PIHP, or PAHP to place appropriate limits on a service—
(i) On the basis of criteria applied under the State plan, such as medical necessity; or
(ii) For the purpose of utilization control, provided that—
(A)The services furnished can reasonably achieve their purpose, as required in paragraph (a)(3)(i) of this
section;
(B) The services supporting individuals with ongoing or chronic conditions are authorized in a manner that
reflects the enrollee's ongoing need for such services and supports;
(5) Specify what constitutes “medically necessary services” in a manner that—
(i) Is no more restrictive than that used in the State Medicaid program, including quantitative and non-
quantitative treatment limits, as indicated in State statutes and regulations, the State Plan, and other
State policy and procedures; and
(ii) Addresses the extent to which the MCO, PIHP, or PAHP is responsible for covering services that
address:
(A) The prevention, diagnosis, and treatment of an enrollee's disease, condition, and/or disorder that
results in health impairments and/or disability.
(B) The ability for an enrollee to achieve age-appropriate growth and development.
(C) The ability for an enrollee to attain, maintain, or regain functional capacity.
State Regulation/BHO Agreement Source(s)
151 Appendix
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 6
EQR Scoring Criteria
The BHO monitors to ensure that services are provided in an amount, duration, and scope
sufficient to achieve the purpose for which they are provided.
The BHO does not arbitrarily deny or reduce the amount, duration, or scope of a required service
solely because of diagnosis, type of illness, or condition of the beneficiary.
The BHO applies the State’s standard for medical necessity when monitoring provided services
and making authorization decisions.
§438.210 Coverage and authorization of services.
(b) Authorization of services. For the processing of requests for initial and continuing authorizations of
services, each contract must require—
(1) That the MCO, PIHP, or PAHP and its subcontractors have in place, and follow, written policies and
procedures.
(2) That the MCO, PIHP, or PAHP—
(i) Have in effect mechanisms to ensure consistent application of review criteria for authorization
decisions.
(ii) Consult with the requesting provider when appropriate.
(3) That any decision to deny a service authorization request or to authorize a service in an amount,
duration, or scope that is less than requested, be made by an individual who has appropriate expertise in
addressing the enrollee's medical, behavioral health, or long-term services and supports needs.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-777A, 388-877B
BHO Agreement Section(s) 6
EQR Scoring Criteria
The BHO has documented policies and procedures for the consistent application of review criteria
pertaining to the initial and continuing authorization of services.
The BHO consults with the requesting BHA when appropriate.
The BHO has a process to ensure that any decision to deny a service authorization request or to
authorize a service in an amount, duration, or scope that is less than requested is made by a
mental health or chemical dependency professional who has appropriate clinical expertise in
treating the enrollee's condition or disease.
In the event of an inpatient stay, only a psychiatrist or clinical psychologist can issue a denial for
inpatient psychiatric services.
§438.210 Coverage and authorization of services.
(c) Notice of adverse benefit determination. Each contract must provide for the MCO, PIHP, or PAHP to
notify the requesting provider, and give the enrollee written notice of any decision by the MCO, PIHP, or
PAHP to deny a service authorization request, or to authorize a service in an amount, duration, or scope
that is less than requested. For MCOs, PIHPs, and PAHPs, the enrollee's notice must meet the
requirements of §438.404.
§438.404 Timely and adequate notice of adverse benefit determination.
(a) Notice. The MCO, PIHP, or PAHP must give enrollees timely and adequate notice of an adverse
benefit determination in writing consistent with the requirements below and in §438.10.
152 Appendix
(b) Content of notice. The notice must explain the following:
(1) The adverse benefit determination the MCO, PIHP, or PAHP has made or intends to make.
(2) The reasons for the adverse benefit determination, including the right of the enrollee to be provided
upon request and free of charge, reasonable access to and copies of all documents, records, and other
information relevant to the enrollee's adverse benefit determination. Such information includes medical
necessity criteria, and any processes, strategies, or evidentiary standards used in setting coverage limits.
(3) The enrollee's right to request an appeal of the MCO's, PIHP's, or PAHP's adverse benefit
determination, including information on exhausting the MCO's, PIHP's, or PAHP's one level of appeal
described at §438.402(b) and the right to request a State fair hearing consistent with §438.402(c).
(4) The procedures for exercising the rights specified in this paragraph (b).
(5) The circumstances under which an appeal process can be expedited and how to request it.
(6) The enrollee's right to have benefits continue pending resolution of the appeal, how to request that
benefits be continued, and the circumstances, consistent with state policy, under which the enrollee may
be required to pay the costs of these services.
(c) Timing of notice. The MCO, PIHP, or PAHP must mail the notice within the following timeframes:
(1) For termination, suspension, or reduction of previously authorized Medicaid-covered services, within
the timeframes specified in §§431.211, 431.213, and 431.214 of this chapter.
(2) For denial of payment, at the time of any action affecting the claim.
(3) For standard service authorization decisions that deny or limit services, within the timeframe specified
in §438.210(d)(1).
(4) If the MCO, PIHP, or PAHP meets the criteria set forth for extending the timeframe for standard
service authorization decisions consistent with §438.210(d)(1)(ii), it must—
(i) Give the enrollee written notice of the reason for the decision to extend the timeframe and inform the
enrollee of the right to file a grievance if he or she disagrees with that decision; and
(ii) Issue and carry out its determination as expeditiously as the enrollee's health condition requires and
no later than the date the extension expires.
(5) For service authorization decisions not reached within the timeframes specified in §438.210(d) (which
constitutes a denial and is thus an adverse benefit determination), on the date that the timeframes expire.
(6) For expedited service authorization decisions, within the timeframes specified in §438.210(d)(2).
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-777A, 388-877B
BHO Agreement Section(s) 6.9, 7.3
EQR Scoring Criteria
The BHO has a documented policy and procedure to notify the requesting BHA and the enrollee
in writing of any decision by the BHO to deny an initial or continuing service authorization request,
or to authorize a service in an amount, duration, or scope that is less than requested.
The BHO ensures the notice meets the requirements of §438.404.
§438.210 Coverage and authorization of services.
(d) Timeframe for decisions. Each MCO, PIHP, or PAHP contract must provide for the following decisions
and notices:
(1) Standard authorization decisions. For standard authorization decisions, provide notice as
expeditiously as the enrollee's health condition requires and within State-established timeframes that may
not exceed 14 calendar days following receipt of the request for service, with a possible extension of up
to 14 additional calendar days, if—
(i) The enrollee, or the provider, requests extension; or
153 Appendix
(ii) The MCO, PIHP, or PAHP justifies (to the State agency upon request) a need for additional
information and how the extension is in the enrollee's interest.
(2) Expedited authorization decisions. (i) For cases in which a provider indicates, or the MCO, PIHP, or
PAHP determines, that following the standard timeframe could seriously jeopardize the enrollee's life or
health or ability to attain, maintain, or regain maximum function, the MCO, PIHP, or PAHP must make an
expedited authorization decision and provide notice as expeditiously as the enrollee's health condition
requires and no later than 72 hours after receipt of the request for service.
(ii) The MCO, PIHP, or PAHP may extend the 72 hour time period by up to 14 calendar days if the
enrollee requests an extension, or if the MCO, PIHP, or PAHP justifies (to the State agency upon request)
a need for additional information and how the extension is in the enrollee's interest.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-777A, 388-877B
BHO Agreement Section(s) 6
EQR Scoring Criteria
The BHO has a documented policy and procedure for standard and expedited authorization
decisions. The BHO has a process for tracking standard and expedited authorization decisions.
The BHO has mechanisms in place to ensure compliance with standard and expedited
authorization timeframes.
§438.210 Coverage and authorization of services.
(e) Compensation for utilization management activities. Each contract between a State and MCO, PIHP,
or PAHP must provide that, consistent with §§438.3(i), and 422.208 of this chapter, compensation to
individuals or entities that conduct utilization management activities is not structured so as to provide
incentives for the individual or entity to deny, limit, or discontinue medically necessary services to any
enrollee.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-777A, 388-877B
BHO Agreement Section(s) 6.12
EQR Scoring Criteria
The BHO has a documented policy and procedure specifying that compensation to individuals or
entities that conduct utilization management activities is not structured so as to provide incentives
for the individual or entity to deny, limit, or discontinue medically necessary services to any
enrollee.
The BHO has mechanisms in place to ensure BHAs and/or utilization management contractors
do not provide staff with incentives to deny, limit, or discontinue medically necessary services.
§438.114 Emergency and post-stabilization services.
(a) Definitions. As used in this section—
Emergency medical condition means a medical condition manifesting itself by acute symptoms of
sufficient severity (including severe pain) that a prudent layperson, who possesses an average
knowledge of health and medicine, could reasonably expect the absence of immediate medical attention
to result in the following:
(1) Placing the health of the individual (or, for a pregnant woman, the health of the woman or her unborn
child) in serious jeopardy.
154 Appendix
(2) Serious impairment to bodily functions.
(3) Serious dysfunction of any bodily organ or part.
Emergency services means covered inpatient and outpatient services that are as follows:
(1) Furnished by a provider that is qualified to furnish these services under this Title.
(2) Needed to evaluate or stabilize an emergency medical condition.
Post-stabilization care services means covered services, related to an emergency medical condition that
are provided after an enrollee is stabilized in order to maintain the stabilized condition, or, under the
circumstances described in paragraph (e) of this section, to improve or resolve the enrollee's condition.
(b) Coverage and payment: General rule. The following entities are responsible for coverage and
payment of emergency services and post-stabilization care services.
(1) The MCO, PIHP, or PAHP.
(2) The State, for managed care programs that contract with PCCMs or PCCM entities.
(c) Coverage and payment: Emergency services. (1) The entities identified in paragraph (b) of this
section—
(i) Must cover and pay for emergency services regardless of whether the provider that furnishes the
services has a contract with the MCO, PIHP, PAHP, or PCCM entity; and
(ii) May not deny payment for treatment obtained under either of the following circumstances:
(A) An enrollee had an emergency medical condition, including cases in which the absence of immediate
medical attention would not have had the outcomes specified in paragraphs (1), (2), and (3) of the
definition of emergency medical condition in paragraph (a) of this section.
(B) A representative of the MCO, PIHP, PAHP, or PCCM instructs the enrollee to seek emergency
services.
(2) A PCCM or PCCM entity must allow enrollees to obtain emergency services outside the primary care
case management system regardless of whether the case manager referred the enrollee to the provider
that furnishes the services.
(d) Additional rules for emergency services. (1) The entities specified in paragraph (b) of this section may
not—
(i) Limit what constitutes an emergency medical condition with reference to paragraph (a) of this section,
on the basis of lists of diagnoses or symptoms; and
(ii) Refuse to cover emergency services based on the emergency room provider, hospital, or fiscal agent
not notifying the enrollee's primary care provider, MCO, PIHP, PAHP or applicable State entity of the
enrollee's screening and treatment within 10 calendar days of presentation for emergency services.
(2) An enrollee who has an emergency medical condition may not be held liable for payment of
subsequent screening and treatment needed to diagnose the specific condition or stabilize the patient.
(3) The attending emergency physician, or the provider actually treating the enrollee, is responsible for
determining when the enrollee is sufficiently stabilized for transfer or discharge, and that determination is
binding on the entities identified in paragraph (b) of this section as responsible for coverage and payment.
(e) Coverage and payment: Poststabilization care services. Poststabilization care services are covered
and paid for in accordance with provisions set forth at §422.113(c) of this chapter. In applying those
provisions, reference to “MA organization” and “financially responsible” must be read as reference to the
entities responsible for Medicaid payment, as specified in paragraph (b) of this section, and payment
rules governed by Title XIX of the Act and the States
(f) Applicability to PIHPs and PAHPs. To the extent that services required to treat an emergency medical
condition fall within the scope of the services for which the PIHP or PAHP is responsible, the rules under
this section apply.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-777A, 388-877B
BHO Agreement Section(s) 10.9, 15, 6.3
155 Appendix
EQR Scoring Criteria
The BHO has written policies and procedures pertaining to crisis, stabilization, and post-hospital
follow-up services.
The BHO pays for treatment of conditions defined in its policies as urgent or emergent conditions.
The BHO tracks and monitors to ensure that there is no payment denial for crisis services.
The BHO tracks and monitors use of crisis services for inappropriate or avoidable use related to
access to routine care.
Provider Selection §438.214 Provider selection.
(a) General rules. The State must ensure, through its contracts, that each MCO, PIHP, or PAHP
implements written policies and procedures for selection and retention of network providers and that
those policies and procedures, at a minimum, meet the requirements of this section.
(b) Credentialing and recredentialing requirements. (1) Each State must establish a uniform credentialing
and recredentialing policy that addresses acute, primary, behavioral, substance use disorders, as
appropriate, and requires each MCO, PIHP and PAHP to follow those policies.
(2) Each MCO, PIHP, and PAHP must follow a documented process for credentialing and recredentialing
of network providers.
(e) State requirements. Each MCO, PIHP, and PAHP must comply with any additional requirements
established by the State.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 5.8, 8, 10, 11
EQR Scoring Criteria
The BHO has a credentialing and re-credentialing policy and procedure for providers who have
signed contracts or participation agreements.
The BHO has a documented process for credentialing.
The BHO has a documented process for re-credentialing.
The BHO annually monitors the credentialing and re-credentialing process.
The BHO ensures the BHAs have credentialing and re-credentialing polices and processes in
place.
§438.214 Provider selection.
(c) Nondiscrimination. MCO, PIHP, and PAHP provider selection policies and procedures, consistent with
§438.12, must not discriminate against particular providers that serve high-risk populations or specialize
in conditions that require costly treatment.
§438.12 Provider discrimination prohibited.
(a) General rules. (1) An MCO, PIHP, or PAHP may not discriminate in the participation, reimbursement,
or indemnification of any provider who is acting within the scope of his or her license or certification under
applicable State law, solely on the basis of that license or certification. If an MCO, PIHP, or PAHP
declines to include individual or groups of providers in its provider network, it must give the affected
providers written notice of the reason for its decision.
(2) In all contracts with network providers, an MCO, PIHP, or PAHP must comply with the requirements
specified in §438.214.
156 Appendix
(b) Construction. Paragraph (a) of this section may not be construed to—
(1) Require the MCO, PIHP, or PAHP to contract with providers beyond the number necessary to meet
the needs of its enrollees;
(2) Preclude the MCO, PIHP, or PAHP from using different reimbursement amounts for different
specialties or for different practitioners in the same specialty; or
(3) Preclude the MCO, PIHP, or PAHP from establishing measures that are designed to maintain quality
of services and control costs and are consistent with its responsibilities to enrollees.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 3.8, 5
EQR Scoring Criteria
The BHO has provider selection policies and procedures, consistent with §438.12, that ensure
the BHO does not discriminate against particular providers that serve high-risk populations or
specialize in conditions that require costly treatment.
The BHO has policies and procedures in place that ensure the BHO does not discriminate in the
participation, reimbursement, or indemnification of any provider who is acting within the scope of
their license or certification, solely on the basis of that license or certification.
The BHO has a process for notifying individuals or groups of providers when they are not chosen
for participation in the network.
§438.214 Excluded providers.
(d) Excluded providers. MCOs, PIHP s, and PAHPs may not employ or contract with providers excluded
from participation in federal health care programs under either section 1128 or section 1128A of the Act.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 8.5, 8.8
EQR Scoring Criteria
The BHO has a policy and procedure to ensure the BHO does not employ or contract with
providers excluded from participation in federal healthcare programs.
The BHO can demonstrate the process and the documentation to determine whether individuals
or organizations are excluded providers.
The BHO ensures it is not employing or appointing on the governing board a person with
beneficial ownership of more than five percent of the BHO’s equity.
The BHO's provider contracts include the provision that providers do not knowingly have a
director, officer, partner, or other person excluded from participation in federal healthcare
programs with a beneficial ownership of more than five percent of the agency's equity.
Subcontractual Relationships and Delegation §438.230 Subcontractual relationships and delegation.
(a) Applicability. The requirements of this section apply to any contract or written arrangement that an
MCO, PIHP, PAHP, or PCCM entity has with any subcontractor.
(b) General rule. The State must ensure, through its contracts with MCOs, PIHPs, PAHPs, and PCCM
entities that—
157 Appendix
(1) Notwithstanding any relationship(s) that the MCO, PIHP, PAHP, or PCCM entity may have with any
subcontractor, the MCO, PIHP, PAHP, or PCCM entity maintains ultimate responsibility for adhering to
and otherwise fully complying with all terms and conditions of its contract with the State; and
(2) All contracts or written arrangements between the MCO, PIHP, PAHP, or PCCM entity and any
subcontractor must meet the requirements of paragraph (c) of this section.
(c) Each contract or written arrangement described in paragraph (b)(2) of this section must specify that:
(1) If any of the MCO's, PIHP's, PAHP's, or PCCM entity's activities or obligations under its contract with
the State are delegated to a subcontractor—
(i) The delegated activities or obligations, and related reporting responsibilities, are specified in the
contract or written agreement.
(ii) The subcontractor agrees to perform the delegated activities and reporting responsibilities specified in
compliance with the MCO's, PIHP's, PAHP's, or PCCM entity's contract obligations.
(iii) The contract or written arrangement must either provide for revocation of the delegation of activities or
obligations, or specify other remedies in instances where the State or the MCO, PIHP, PAHP, or PCCM
entity determine that the subcontractor has not performed satisfactorily.
(2) The subcontractor agrees to comply with all applicable Medicaid laws, regulations, including
applicable subregulatory guidance and contract provisions;
(3) The subcontractor agrees that—
(i) The State, CMS, the HHS Inspector General, the Comptroller General, or their designees have the
right to audit, evaluate, and inspect any books, records, contracts, computer or other electronic systems
of the subcontractor, or of the subcontractor's contractor, that pertain to any aspect of services and
activities performed, or determination of amounts payable under the MCO's, PIHP's, or PAHP's contract
with the State.
(ii) The subcontractor will make available, for purposes of an audit, evaluation, or inspection under
paragraph (c)(3)(i) of this section, its premises, physical facilities, equipment, books, records, contracts,
computer or other electronic systems relating to its Medicaid enrollees.
(iii) The right to audit under paragraph (c)(3)(i) of this section will exist through 10 years from the final date
of the contract period or from the date of completion of any audit, whichever is later.
(iv) If the State, CMS, or the HHS Inspector General determines that there is a reasonable possibility of
fraud or similar risk, the State, CMS, or the HHS Inspector General may inspect, evaluate, and audit the
subcontractor at any time.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-777A, 388-877B
BHO Agreement Section(s) 10
EQR Scoring Criteria
The BHO has policies and procedures for oversight and accountability for any functions and
responsibilities it delegates to its subcontractors/BHAs.
The BHO performs pre-delegation assessments of contracted BHAs before delegation is granted
on the subcontractor's ability to perform the activities to be delegated.
The BHO has written contracts/agreements that address the specifics of the activities that have
been delegated to the subcontractor/BHA.
The BHO monitors the subcontractor's performance on an ongoing basis and subjects it to formal
review according to a periodic schedule established by the State, consistent with industry
standards or State laws and regulations.
The BHO includes in the delegation contract/agreement that the BHO is responsible for
monitoring and reviewing the subcontractor's/BHA's performance on an ongoing basis and
158 Appendix
provides criteria for revoking delegation or imposing other sanctions if the subcontractor's
performance is inadequate.
The BHO initiates a corrective action if subcontractor/BHA performance does not meet industry
standards and/or requirements in the delegation or contract agreement.
The BHO follows up on any corrective action plan given to its subcontractor to ensure it has met
the criteria and, if the criteria is not met, imposes further corrective action, including revoking the
delegated activity.
The BHO requires the BHAs to follow the same CFR criteria for any services the BHAs delegate
to other entities.
The BHO monitors the BHAs’ delegated agreements with other entities.
Practice Guidelines §438.236 Practice guidelines.
(a) Basic rule. The State must ensure, through its contracts, that each MCO and, when applicable, each
PIHP and PAHP meets the requirements of this section.
(b) Adoption of practice guidelines. Each MCO and, when applicable, each PIHP and PAHP adopts
practice guidelines that meet the following requirements:
(1) Are based on valid and reliable clinical evidence or a consensus of providers in the particular field.
(2) Consider the needs of the MCO, PIHP, or PAHP's enrollees.
(3) Are adopted in consultation with contracting health care professionals.
(4) Are reviewed and updated periodically as appropriate.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 6.12, 9.5
EQR Scoring Criteria
The BHO has documented policies and procedures related to adoption of practice guidelines,
including consultation with contracting healthcare professionals.
The BHO’s guidelines are based on valid and reliable clinical evidence or a consensus of
healthcare professionals in the particular field.
The BHO has documentation of its enrollees’ needs and how the guidelines fit those needs.
The BHO has documentation that the guidelines are reviewed and updated yearly.
The BHO has a documented policy and procedure for how affiliated BHAs are consulted as
guidelines are adopted and re-evaluated.
§438.236 Practice guidelines.
(c) Dissemination of guidelines. Each MCO, PIHP and PAHP disseminates the guidelines to all affected
providers and, upon request, to enrollees and potential enrollees.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 9.5
EQR Scoring Criteria
The BHO has a policy and procedure on how to disseminate practice guidelines to all providers
and, upon request, to enrollees and potential enrollees.
The BHO can demonstrate it has disseminated the practice guidelines to all BHAs and to
159 Appendix
enrollees upon request.
§438.236 Practice guidelines.
(d) Application of guidelines. Decisions for utilization management, enrollee education, coverage of
services and other areas to which the guidelines apply are consistent with the guidelines.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 10.9, 9.5
EQR Scoring Criteria
The BHO has documented policies and procedures as well as documented meeting minutes
demonstrating that decisions for utilization management, enrollee education, coverage of
services, and other areas to which the guidelines apply are consistent with the guidelines.
The BHO annually monitors the effective use of practice guidelines by the BHAs.
The BHO has documentation of the interface process between the QAPI program and the
practice guidelines adoption process.
Health Information Systems §438.242 Health information systems.
(a) General rule. The State must ensure, through its contracts that each MCO, PIHP, and PAHP
maintains a health information system that collects, analyzes, integrates, and reports data and can
achieve the objectives of this part. The systems must provide information on areas including, but not
limited to, utilization, claims, grievances and appeals, and disenrollments for other than loss of Medicaid
eligibility.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 11, section 13 (Behavioral Health Data Management) and 9.7 (Encounter
Data Validation).
EQR Scoring Criteria
The BHO has a health information system that collects, analyzes, integrates, and reports data on
utilization, dis-enrollments, requests to change providers, grievances, and appeals.
The BHO utilizes reports from health information data to make informed management decisions.
The BHO uses the information it has collected and analyzed to identify trends in areas including
but not limited to utilization, grievances and appeals, dis-enrollments, and requests to change
providers.
The BHO analyzes the health information data to identify services needed for enrollees.
Quality Assessment and Performance Improvement Program §438.330 Quality assessment and performance improvement program.
(a) General rules. (1) The State must require, through its contracts, that each MCO, PIHP, and PAHP
establish and implement an ongoing comprehensive quality assessment and performance improvement
program for the services it furnishes to its enrollees that includes the elements identified in paragraph (b)
of this section.
(2) After consulting with States and other stakeholders and providing public notice and opportunity to
comment, CMS may specify performance measures and PIPs, which must be included in the standard
160 Appendix
measures identified and PIPs required by the State in accordance with paragraphs (c) and (d) of this
section. A State may request an exemption from including the performance measures or PIPs established
under paragraph (a)(2) of this section, by submitting a written request to CMS explaining the basis for
such request.
(3) The State must require, through its contracts, that each PCCM entity described in §438.310(c)(2)
establish and implement an ongoing comprehensive quality assessment and performance improvement
program for the services it furnishes to its enrollees which incorporates, at a minimum, paragraphs (b)(2)
and (3) of this section and the performance measures identified by the State per paragraph (c) of this
section.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 9.1, 9.8
EQR Scoring Criteria
The BHO has an ongoing quality assessment and performance improvement program (QAPI) for
the services it furnishes to its enrollees.
The BHO has a written QAPI program.
The BHO has a QA and PI process to evaluate the QAPI program and updates its annual plan.
The BHO collects, analyzes, and uses performance data to support its quality assessment and
performance improvement program.
The BHO has a Quality Management Committee that meets regularly, reviews results of
performance data and reports to the governing board.
The BHO has effective mechanisms to assess the quality and appropriateness of care furnished
to enrollees.
The BHO conducts one clinical performance improvement project, one substance use disorder
performance improvement project, and one non-clinical performance improvement project each
year, one of which is a children-focused performance improvement project.
The BHO ensures its compliance with the State’s quality strategy plan.
§438.330 Quality assessment and performance improvement program.
(b) Basic elements of quality assessment and performance improvement programs. The comprehensive
quality assessment and performance improvement program described in paragraph (a) of this section
must include at least the following elements:
(1) Performance improvement projects in accordance with paragraph (d) of this section.
(2) Collection and submission of performance measurement data in accordance with paragraph (c) of this
section.
(3) Mechanisms to detect both underutilization and overutilization of services.
(4) Mechanisms to assess the quality and appropriateness of care furnished to enrollees with special
health care needs, as defined by the State in the quality strategy under §438.340.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 9.3, 9.81, 9.82
EQR Scoring Criteria
The BHO collects, analyzes, and uses performance data to support its quality assessment and
performance improvement program.
161 Appendix
The BHO reports performance data to the State every year.
The BHO has a documented policy and procedure regarding the detection of both underutilization
and overutilization of services in all its programs.
The BHO has consistent criteria for identifying underutilization and overutilization.
The BHO has processes for routine monitoring of underutilization and overutilization.
The BHO has processes for taking corrective action to address underutilization and
overutilization.
§438.330 Quality assessment and performance improvement program.
(c) Performance measurement. The State must—
(1)(i) Identify standard performance measures, including those performance measures that may be
specified by CMS under paragraph (a)(2) of this section, relating to the performance of MCOs, PIHPs,
and PAHPs;
(2) Require that each MCO, PIHP, and PAHP annually—
(i) Measure and report to the State on its performance, using the standard measures required by the
State in paragraph (c)(1) of this section;
(ii) Submit to the State data, specified by the State, which enables the State to calculate the MCO's,
PIHP's, or PAHP's performance using the standard measures identified by the State under paragraph
(c)(1) of this section; or
(iii) Perform a combination of the activities described in paragraphs (c)(2)(i) and (ii) of this section.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 9.82
EQR Scoring Criteria
The BHO has a process in place to assess the standard performance measures.
The BHO measures and reports to the State on its performance, using the standard measures
required by the State.
The BHO has processes to submit data to DBHR, which enables the State to calculate the BHO’s
performance using standard measures.
§438.330 Quality assessment and performance improvement program.
(d) Performance improvement projects. (1) The State must require that MCOs, PIHPs, and PAHPs
conduct performance improvement projects, including any performance improvement projects required by
CMS in accordance with paragraph (a)(2) of this section, that focus on both clinical and nonclinical areas.
(2) Each performance improvement project must be designed to achieve significant improvement,
sustained over time, in health outcomes and enrollee satisfaction, and must include the following
elements:
(i) Measurement of performance using objective quality indicators.
(ii) Implementation of interventions to achieve improvement in the access to and quality of care.
(iii) Evaluation of the effectiveness of the interventions based on the performance measures in paragraph
(d)(2)(i) of this section.
(iv) Planning and initiation of activities for increasing or sustaining improvement.
(3) The State must require each MCO, PIHP, and PAHP to report the status and results of each project
conducted per paragraph (d)(1) of this section to the State as requested, but not less than once per year.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
162 Appendix
BHO Agreement Section(s) 9.81
EQR Scoring Criteria
The BHO has a process in place to assess the quality and appropriateness of care furnished to
enrollees.
The BHO monitors and tracks the quality and appropriateness of care furnished to enrollees.
The BHO has processes for taking action when quality and appropriateness of care issues are
identified.
§438.330 Quality assessment and performance improvement program.
(e) Program review by the State. (1) The State must review, at least annually, the impact and
effectiveness of the quality assessment and performance improvement program of each MCO, PIHP,
PAHP, and PCCM entity described in §438.310(c)(2). The review must include—
(i) The MCO's, PIHP's, PAHP's, and PCCM entity's performance on the measures on which it is required
to report
(ii) The outcomes and trended results of each MCO's, PIHP's, and PAHP's performance improvement
projects
(2) The State may require that an MCO, PIHP, PAHP, or PCCM entity described in §438.310(c)(2)
develop a process to evaluate the impact and effectiveness of its own quality assessment and
performance improvement program.
State Regulation/BHO Agreement Source(s)
WAC 388-865, 388-877, 388-877A, 388-877B
BHO Agreement Section(s) 9.8
EQR Scoring Criteria
The BHO has a process in place to submit its QAPI program evaluation to the State at least annually.
The BHO monitors and tracks its required performance measures.
The BHO has processes to report outcomes and trended results of its performance improvement projects.
The BHO has a process in place to evaluate the impact and effectiveness of its own QAPI
program.
163 Appendix
Appendix F: Acronyms
ACA Affordable Care Act
ADA Americans with Disabilities Act
ANSI ASC American National Standards Institute, the Accredited Standards Committee
ASAM American Society of Addiction Medicine
BC/DR Business Continuity and Disaster Recovery
BCP Business Continuity Plan
BHA Behavioral Health Agency
BHDS Behavioral Health Data System
BHO Behavioral Health Organization
CANS Child and Adolescent Needs and Strengths
CAP Corrective Action Plan
CDP Chemical Dependency Professional
CFR Code of Federal Regulations
CMS Centers for Medicare & Medicaid Services
CPT Current Procedural Terminology
DBHR Division of Behavioral Health and Recovery
DCFS Division of Children and Family Services
DSHS Department of Social and Health Services
DSM-5 Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5)
E&T Evaluation and Treatment
ED Emergency Department
EDI Electronic Data Interchange
EDV Encounter Data Validation
EHR Electronic Health Record
EQR External Quality Review
EQRO External Quality Review Organization
FFS Fee for Service
GIS Geographic Information Systems
HCA Health Care Authority
HHS Health & Human Services
HIPAA Health Insurance Portability and Accountability Act
ISCA Information System Capability Assessment
ISO/IEC International Organization for Standardization/International Electrotechnical Commission
ISO/IEEE International Standards Organization/Institute of Electrical and Electronics Engineers
ISP Individual Service Plan
IT Information Technology
LEIE List of Excluded Individuals and Entities
MCO Managed Care Organization
MMIS Medicaid Management Information System
NIST National Institute of Standards and Technology
PAHP Prepaid Ambulatory Health Plans
PCP Primary Care Provider
PDSA Plan-Do-Study-Act
PHI Protected Health Information
PII Personally Identifiable Information
PIHP Prepaid Inpatient Health Plan
164 Appendix
PIP Performance Improvement Project
QA/PI Quality Assurance and Performance Improvement
QAPI Quality Assessment and Performance Improvement
QRT Quality Review Team
RFT Residential Treatment Facility
RSN Regional Support Network
SERI Service Encounter Reporting Instructions
SUD Substance Use Disorder
WAC Washington Administrative Code
WISe Wraparound with Intensive Services
WSIPP Washington State Institute for Public Policy
top related