west county health centers data validation case study pptÂ

CSLC Case Study

“You Can’t Make Good Decisions With Bad Data”

Data Validation Initiatives at West County Health Centers

A Meeting of the Clinical Systems Learning Learning Community

Tuesday 8/21/2007

CSLC Case Study WCHC - 04/25/07 2

Introducing West County Health Centers

Clinical Sites: Russian River Health Center - Guerneville, CA Occidental Area Health Center – Occidental, CA Teen Clinic – Forestville, CA

Number of Staff (total), Number of Providers 75 employees in five locations; 58 FTEs 22 providers (including 3 dentists, 4 mental health counselors and 1 psychiatrist); 14 FTEs

2006 UDS data about visits/patients 8,150 patients 36,799 visits

Primary Care Services – Newborns to Elders: Pre-natal and Obstetrical Care Well Child Exam and Low Cost Immunizations School and Sports Physical Exams Routine, Annual and Employment Physicals Reproductive Health Care HIV/AIDS Primary Care Child Health and Disability Program (CHDP) Breast Canter Early Detection Program (BCEDP)

Special Services: Dental Services HIV Case Management Mental Health Counseling


Goals of This Presentation

To explore the issues currently present for CSLC members in terms

of Data Validation. To show WCHC’s approach to

“diving” into the data To stimulate discussion about the

best approach to the Data Validation Process


Technology-Enabled Quality Improvement

Technology-enabled quality improvement allows for more automated and timely data collection and reporting, larger scale initiatives and broader participation without adding resources.

Chart Audits, while highly accurate, are time consuming and resource intensive. Participating in a new disease collaborative often means adding staff.


A Critical Juncture for Proving HIT Value

As more data is available to providers, we see the “Light Bulb Moment” occurring

Registries have provided an insight into patient population characteristics that was not previously available

Creates a “thirst” for more data – driven internally by providers rather than by external reporting requirements

Sets the foundation for data-informed decision making, as well as EHR implementation and use

Data accuracy in these early stages is critical in building trust Consequences of billing data error vs. clinical data error P4P and Provider Incentives Bad data undermines providers’ trust in technology

As more data collection systems are introduced and interfaced, multiple points of failure are introduced

Reports contain data from multiple sources and systems Process definition, redesign and communication becomes as important as the

technology.


An Environment in Transition

Paper Charts

Fully Integrated and Interoperable

EHRs

You Are Here


Accuracy is Fraught with Perils

What Actually Happened

What Appears on Report What Appears on Report


Data Entry at the Point of Care

The closer the data entry is to the actual clinical event, the less chance of errors and omissions.

No need to navigate the “swamp” of connecting systems and processes.

However, the source document for 90% of the data in health centers is still the Encounter Form (Superbill, Billing Slip).


Multiple Points of Failure

“Our systems are interfaced; what could go wrong?”

Patient is registered or checks in. Billing

Slip printed.

Patient is registered or checks in. Billing

Slip printed.

Patient is seen and Provider

completes Billing Slip

Patient is seen and Provider

completes Billing Slip

Billing Slip sent to

Billing Dept. for entry to PM System

Billing Slip sent to

Billing Dept. for entry to PM System

Tracking system

receives data

through interface.

Tracking system

receives data

through interface.

Lab or other tracking

data entered into system.

Lab or other tracking

data entered into system.

Data entry errors

Data entered on wrong patient

Duplicate patients in system

Codes are not mapped correctly

Interface program incorrect

Duplicate patients in system

Data entry errors

No patient in tracking system to match result

Missing source data slips

Billing Slip is incomplete

Billing Slip is inaccurate

Billing Slip gets lost altogether

Data entry errors

Data entered on wrong patient

Extra time for errors & missing info


Principles for Maintaining Healthy Data: S T R I V E

Standards There are well-defined and frequently-measured standards for data integrity Start from the beginning of your “data entry process” AND involve ALL STAFF

Training Training to remediate errors is targeted and frequent, and addresses the standards the

organization has set. Responsibility

Someone is accountable for data stewardship Expectation of and standards for data accuracy are called out in job descriptions Fewer errors occur if the “owner” of the data understands the particular care process

Incentives (and Consequences) Benefits of eliminating waste and rework are shared with the team that meets their goals. There are consequences for data inaccuracy.

Verifiable Understanding what is the source data document or system Testing the systems in a controlled environment Checks and balances: reports from two systems, chart audit

Education End-to-end data flow processes are documented and understood by all Accuracy measurements are posted where all can see and staff can interpret the results Checks and balances: reports from two systems, chart audit


AGENCY LEADERSHIP

WCHC Executive Director and Management Team committed to the allocation of resources

Training through RCHC: The QCS Series Moving us toward “Culture of Quality”

Training of all Staff at Quarterly Meetings regarding Data Integrity at ALL LEVELS

I2i Tracks- Tool for Accurate Data Collection

System Improvement as Preparation for EMR


Steps Toward Data Validation

Where to begin? REGISTRATION: Accuracy of Entry PMS Clean up Project CODING: Superbill ACT Project BILLING: Data Entry, Feedback to Providers CLINICAL DATA: Data Stewards Accuracy of Entry and Tracking in i2i Tracks VERIFICATION OF DATA IN REPORTS DATA VALIDATION w Chart Review


The Source Patient Data Base – Centricity PMRegistration Data

Before Centricity, each clinic had their own separate PM system.

Patients could be seen at either site, and potentially registered twice.

Centricity implementation combined databases, duplicates and all. Over the years, duplicates have been merged but others

created There is now an established process for clinic staff to call

Billing Dept. if duplicates are discovered.

Maria G. Smith Mary Garcia Maria Garcia-Smith Marie Garcia


The Source Patient Data Base – Centricity PM

Special data quality project was done to systematically review the patient database and merge duplicate patients. 1 Staff Member spent 100 hours merging duplicate files,

inactivating patients and correcting demographic info (27,000 files) 4,000 files became inactive Project included review of front desk procedures and patient

look up to reduce duplicates. Process also included identifying duplicate patients in

i2iTracks and having the vendor merge those patients as well.

Maria Garcia-Smith


“An Ounce of Prevention”- Superbill ACTCoding Data

For data quality and accuracy, getting the data correct up front is critical

No other document is as all-encompassing and important as the Superbill

• Accurate• Complete• TimelyA C T =A C T =

Problem: Billing slips (Superbills) not being submitted before patient leaves the clinic. Some slips are missing charges, signatures, billing codes and diagnoses. Some slips are missing altogether.

Consequences: Missed fee collection, front office staff searching for slips, Billing Dept. looking for missing information, charges not billed, missing encounter/QI tracking data.


Superbill Life Cycle

1 2

34

5

6

789


What We Measured

Superbill on time to check out? (within 2 min. of patient arriving at checkout)

Does Front Office staff have to track down slip?

Completed insurance information on right side of slip? Was sliding scale payment

collected?

Missing charges on Superbill?

Was Pap code circled if Pap done?

Missed information found in billing dept.

Number of Superbills missing altogether at the end of day


The Estimated Cost of Waste

Missed Charges: Average of 7 Superbills with missed charges per day out of 69 Superbills. Average cost of missed charges = $20

Cost of Missed Charges: $140 x 240 days = $33,600/year

Missing Superbills or 12 in the first four weeks of measurementInaccurate Dx: (Jan.’07), each one $80

Unbilled EncountersPer Month: $960 x 12 = $11,520/year

Approximately 22,000 Superbills per month5.5% rate of missing Superbills

TOTAL: $45,120/year (not including staff time)

Staff Time: Twenty times in six days to track down Superbills and/or information…


Date Collection Results:Superbills on Time

WCHC SUPERBILL A.C.T.: PERCENT OF ON TIME SUPERBILLS

Avg=84

UCL=103

LCL=64

39

49

59

69

79

89

99

109

1/19am

1/19pm

1/22am

1/22pm

1/23am

1/23pm

1/24am

1/24pm

1/25am

1/25pm

1/26am

1/26pm

1/29am

1/29pm

3/5am

3/5pm

3/6am

3/6pm

3/7am

3/7pm

3/8am

3/8pm

3/9am

3/9pm

4/18am

4/18pm

4/19am

4/19pm

4/20am

4/20pm

4/23am

4/23pm

4/24am

4/24pm

4/25am

4/25pm

Date and Shift

Perc

ent o

f Sup

erbi

lls T

urne

d in

On

Tim

e

AFTERPDSA #1INTERVENTION

BASELINE DATA1/19-1/29

AFTERPDSA #2INTERVENTION


DATA ENTRY


CLINICAL DATA ENTRY

Diabetes Care Data Entry Data Stewardship Data Printed on Visit Summary (Double checked by Providers During Visit)


CSLC Measures Report – Verifying the Population

CSLC Clinical Measures Report

Item

Value %

1. Diabetic Patient Population - Patients who A.) Had 2 or more ambulatory care encounters during the reporting period or prior year, B.) Have received on two or more different dates of service a diagnosis of diabetes, C.) Were between 18 and 75 years old at the end of the reporting period.

A. Total Patient Count 317 100%

B. Patients with at least 1 HbA1C test in the last year 208 66%

1. Last test < 9% 174 84%

2. Last test < 7% 115 55%

C. Patients with at least 1 LDL-C Test in the last 2 years 238 75%

1. Last test >= 130 50 21%

2. Last test < 130 188 79%

3. Last test < 100 119 50%

Why would there be a discrepancy between the number of patients on the CSLC report vs. the i2iTracks roster of diabetic patients?



Diabetic Patients in Tracks – Not on CSLC Report

Out of Age Range

(20 / 51%)

Deceased, Still in Tracks (4 / 10%)

Pts. With No DM Coded Visit in PMS Visit (1 / 3%)

Pt. With 1 DM Coded Visit in PMS (14 / 36%)

296 Patients on Tracks Diabetes Roster

3939PatientsPatients



Diabetes Patients on the CSLC Report – Not in Tracks

Patients Transferred or Inactive (31 / 53%)

Has DM, Not in Tracks (Process Error) (6 / 10%)DM Managed by

Specialist (4 / 17%)

Had DM Dx, Died w/in Past 2 Yrs. (7 / 12%)

Pre-diabetes > 2 Yrs. Ago, No DM Now (7 / 12%)

Gestational Diabetes (2 / 3%)

Had DM Dx, Now Cleared (2 / 3%)

5959PatientsPatients

316 Patients on CSLC Report


Spreadsheet of Comparisons

COMPARISON OF i2iTRACKS DM PT (MANUALLY ENTERED) VS CSLC REPORT

PTS IN DM TRACKING NOT IN CSLC PTS ON CSLC NOT IN DM TRACKING

4/07=39 8/07=35 1 2 3 4 5 6 7 8 9 10 11

DATE

Out of Age Range Deceased

Only 1 DM Code

NO DM Code

Transferred/ Inactive

DM/ Died in last 2 yrs

Ges tational

DM/ Cleared . 2 yrs ago

DM/ Cleared

Specialist

Has DM/ Not in Tracking

4/12/2007 20 4 14 1 31 7 2 7 2 4 6

8/12/2007 22 0 12 1 39 3 1 0 5 0 2


Proposed Data Validation Procedures

For Diabetes Tracking1. Establish standardized process for the Billing Department to notify

i2iTracks regarding deceased patients.• Can a “deceased” or “inactive” dummy code be interfaced

from the PM system to Tracks?• Provide access to Tracks from Billing Department?

2. Have Diabetes Case Manager check charts for patients in Tracks with only 1 (or 0)DM code in PM system.

• If seen twice for DM, instruct providers on checking 250 code

• Add codes to PM system historical visits where appropriate• Compare the 2 reports quarterly, follow up on discrepancies

Given the non-duplicated discrepancy of 98 patients between the CSLC Report and i2iTracks, WCHC proposes the following interventions:


Proposed Data Validation Procedures

For CSLC Report1. Decide as a group if we are going to continue to include those who

have moved or transferred care. For WCHC, this equals 10%.

2. Discuss ways of removing those patients with gestational diabetes or diabetes that clears because of bariatric surgery (or other cause)

3. Discuss report parameters, should report reads “Pts. with any of the 250 diagnosis codes in the past two years and have had two or more ambulatory visits in the past two years”?

4. Clarify if patients being managed by a specialist should be added to the tracking database.

Given the non-duplicated discrepancy of 98 patients between the CSLC Report and i2iTracks, WCHC proposes the following interventions:


Women’s Health Data

More patients, fewer issues for data validation. Cervical Cancer Screening: 3674 Women between 21-64 w at least 1 visit in 2 yrs

Rate of Pap Smear in 3 yrs=50% 17% of the 3674 were seen just one time 2% obtain GYN care elsewhere Breast Cancer Screening: 2164 Women between 42-69 w at least 1 visit in 2 yrs 18% seen just 1 time

Reviewed 60 charts for each screening using CSLC report Found no discrepancy with Pap Smear rate Found 2 mammograms not reported in report


Lessons Learned

Get it right from the start! Do whatever it takes to make sure the data goes into the system correctly.

Data validation and maintaining data integrity is a constant process, not just a one-time project.

Make sure someone owns the responsibility for data stewardship, supported by the leaders of the organization.

It’s critical to understand the sources of information for each data element, and to conduct controlled testing.


Summary of STRIVE in Action

Standards have been set for Superbill processing, All

Staff involved in DATA COLLECTION/ USE

Clear responsibility for data stewardship and management supported by ED;

Incentives toward data integrity- Data graphs for feedback to staff and Providers

Validation-Using two sources of information (CSLC Report and i2iTracks Roster) plus chart review

Education- Ongoing

Training provided to improve accuracy of coding, all data entry, and data validation steps

west county health centers data validation case study pptÂ

Technology

data accuracy

data entry

data collection systems

data validation process

terms of data validation

timely data collection

ftes2006 uds data

clinical data error