Hospital Quality Indicators in Iowa Rural Hospitals

Pengxiang (Alex) Li, Marcia M. Ward, Paul James, John E. Schneider

2008 AHRQ Annual Meeting Bethesda, Maryland

Support grant: Agency for Healthcare Research and Quality Grant # HS015009

Background

Hospital quality indicators were used to provide a perspective on hospital quality of care AHRQ Inpatient Quality Indicators (IQIs) AHRQ Patient Safety Indicators (PSIs)

Our analyses focus on Acute Myocardial Infarction (AMI) in-hospital

mortality (IQI-15) Four PSIs (PSI-5, PSI-6, PSI-7, and PSI-15)

Outline

Comparison of Iowa urban and rural hospitals on AMI in-hospital mortality James PA, Li P, Ward MM. Myocardial infarction mortality in rural and urban hospitals:

Rethinking measures of quality of care. Annals of Family Medicine, 5:105-111, 2007

Association between Critical Access Hospital (CAH) conversion and patient safety indicator performance Li, P., Schneider, J. E. & Ward, M. M., (2007) Effect of Critical Access Hospital

Conversion on Patient Safety. Health Services Research, 42 (6): 2089-2108

Exploration of a potential reason of patient safety change associated with CAH conversion Li, P., Schneider, J. E. & Ward, M. M., Effects of Critical Access Hospital Conversion

on the Financial Performance of Rural Hospitals Inquiry (in press)

How do Iowa urban and rural hospitals compare on AMI in-

hospital mortality?

James PA, Li P, Ward MM. Myocardial infarction mortality in rural and urban hospitals: Rethinking

measures of quality of care. Annals of Family Medicine, 5:105-111, 2007

Introduction

Observational studies find that the quality of care for myocardial infarction (MI) patients admitted to rural hospitals is substandard (Sheikh 2001, Baldwin 2004) Lower volumes of MI patients in rural hospitals Lacking cardiologists Lacking support services

Introduction Validity of these observational studies has been questioned

Unbalanced comparison groups

Patients admitted to rural hospitals tend to be older, poorer,

in poorer health, and have greater number of comorbidities

(Baldwin 2004, Chen 2000, Frances 2000)

Referral patterns of rural provider

Empirical study showed that less severe patients were

referred to urban hospitals (Metha 1999)

Unmeasured confounding may account for differences in

patient outcomes

Objectives of the study

To compare characteristics of MI patients admitted to rural and urban hospitals

To examine in-hospital mortality between rural and urban hospitals among MI patients Using traditional risk adjustment techniques

(Logistic regression) Using instrumental variable methods (IV)

Methods: Data Discharge data from Iowa State Inpatient Dataset (2002 & 2003) Inclusion criteria

A principal diagnosis of MI (ICD-9-CM: 410.01-410.91) Eighteen years or older

Exclusion criteria The hospital identification number was missing (n=9) Patient’s whose home county was not in Iowa (n=1,248) Patients’ zip code was missing (n=14) Patients’ sex was missing (n=1) Our primary analyses also excluded patients discharged or

transferred to another short term general hospital for inpatient care (n=1,618)

Most of our analyses are based on 12,191 MI patients

Methods: Variables Dependent variable

In-hospital mortality Independent variables

Urban vs Rural hospitals that patients admitted to Urban: 27 hospitals Rural: 89 hospitals

Payer: e.g. Medicare, private insurance, self-pay Admission type: e.g. emergency Race Risk adjustment index

Charlson comorbidity index All Patient Refined DRGs (APR-DRGs) risk index

Methods: Traditional Analytic Approach (Logistic Regression)

Univariate analyses of group comparisons Chi-square tests for dichotomous data ANOVAs for continuous data

Logistic regressions for multiple regression analyses

Methods: Pitfalls with Logistic Regression

Using administrative inpatient data, one cannot control all patients’ risk

factors (e.g. severity of illness)

If unmeasured variables are related to selection of the hospital, the

estimates of the hospital-specific contribution to mortality will be

biased.

For example, elderly MI patients with severe comorbid conditions,

which are unmeasured in administrative data, might prefer to remain in

the rural hospitals.

As a result, a higher risk-adjusted mortality rate in rural hospitals

might simply be due to more severe patients in rural hospitals.

Approaches to Minimize Bias Collect all the relevant patient-level variables: very costly Randomized controlled trial

Not feasible to this study Instrumental variable (IV) estimation

An econometric technique which enables us to obtain unbiased estimates of treatment effects in observational studies

An example: Wehby (2006) found that using the logistic regression model, early initiation of prenatal care is associated with a higher probability of low birth weight (LBW) Unmeasured confounders: women at a higher risk

demand more (or early) prenatal care compared to those at lower risk.

IV estimations showed that early time to prenatal care initiation is associated with a lower probability of LBW.

The Instrumental Variable (IV) estimation

IVs are used to achieve a “pseudo-randomization” The instrumental variable technique can extract variation

in the focal variable (rural hospital selection) that is unrelated to unmeasured confounders, and employ this variation to estimate the causal effect on an outcome

Assumptions for IV(s)1. IV(s) should correlate with treatment variable (choice of

rural hospital)2. IV(s) should not be correlated with the unmeasured

confounders

1** 210 uPmaIVsaaurban

2*)(* 210 uPmburbanpredictedbbdead

Methods: Instrumental Variable Technique

Instrumental Variable = Patients’ distance to the nearest urban hospital The distances between each patient’s home and all

urban hospitals in Iowa were obtained by calculating the distances between the centroids of each patient’s resident zip code and all urban hospitals’ zip codes.

Similar to Brooks (2003) approach, instrumental variables in the study are dummy variables that group patients based on the their distance to the nearest urban hospital.

Methods: IV Technique: First assumption

Patients who live closer to an urban hospital are more likely to choose an urban hospital than those who live farther away. Partial F-statistics for the IVs in the first stage

regression Small values of first-stage F-statistics imply failure

of assumption 1 Rule of thumb: F>10 indicates good association

(Staiger 1997)

Methods: IV Technique Second Assumption:

Distance to the nearest urban hospital is not associated with the severity or pre-morbid risks of patients with MI Descriptive comparison between two groups of

patients classified by IV If the instrument is independent of the

unmeasured confounders, it should also be independent of observed risk factors (e.g. age, and comorbidity index).

Over-identifying restrictions tests The null hypothesis is that the IV is not correlated

with unmeasured confounders

Methods: IV Technique

To examine the robustness of our findings: We used a range of patients’ groups for the

instrumental variable (2, 4, 8, and 12 groups). We varied the independent variables.

The syslin two-stage least squares (2SLS) procedure in SAS 9.1 was used to do IV estimation.

Results: Table 1: Baseline characteristics of MI patients* admitted

to rural and urban hospitals

Variables Rural (N= 1,426)

Urban(N= 10,765)

p-value

Age 82.35 68.89 <.0001

Male (%) 45.02 59.76 <.0001

Black (%) 0.14 1.13 0.0004

Number of secondary diagnoses 5.66 5.61 0.43

Charlson comorbidity index 0.96 0.69 <.0001

APR-DRG risk index 0.09 0.06 <.0001

In-hospital Mortality 0.14 0.06 <.0001

* Excluding patients discharged or transferred to another short term general hospital for inpatient care.

Results:

Table 2: Baseline characteristics of MI patients transferred

out of rural hospitals or staying in rural hospitals

Variables Stay in rural hospitals (N=1,426)

Transfer out* of rural hospitals

(N=730)

p-value

Age 82.35 71.46 <.0001

Male (%) 45.02 56.71 <.0001

Black (%) 0.14 0.14 0.99

Number of secondary diagnoses 5.66 4.24 <.0001

Charlson comorbidity index 0.96 0.67 <.0001

APR-DRG risk index 0.09 0.04 <.0001

* Patients discharged or transferred to another acute care hospital for inpatient care

Results: Table 3: Odds ratios of in-hospital mortality* among MI patients admitted to urban hospitals or to rural hospitals, using logistic

regression models (n=12,191)

Model components Odds ratio (Urban vs

Rural)

95% CI

p-value c-statistic

Unadjusted 0.42 0.36-0.50 <.0001 0.56

Adjusted for demographic variables (age, sex, race, admission type and source of payment)

0.70 0.59-0.84 <.0001 0.71

Adjusted for demographic variables and Charlson comorbidity index

0.70 0.59-0.84 0.0001 0.71

Adjusted for demographic variables and APR-DRG risk index

0.68 0.56-0.82 <.0001 0.86

* Excluding patients discharged or transferred to another short term general hospital for inpatient care

Results: Table 4: Characteristics among MI patients grouped by

distance to the nearest urban hospital Variables Distance to nearest

urban hospital <=14.08 miles*(N= 6,097)

Distance to nearest urban hospital >14.08 miles

(N= 6,104)

p-value

Mean Distance to the nearest urban hospital (miles)

4.94 34.20 <0.0001

Percent of patients admitted to urban hospitals (%)

99.54 77.07 <0.0001

Age 68.89 72.02 <0.0001

Male (%) 58.65 57.45 0.18

Black (%) 1.95 0.08 <0.0001

Number of secondary diagnoses 5.72 5.53 <0.0001

Charlson comorbidity index 0.72 0.72 0.67

APR-DRG risk index 0.07 0.07 0.48

In-hospital mortality rate (%) 7.07 7.52 0.34

*14.08 miles is the median distance from patient’s home to the nearest urban hospital

Results: Table 5: Instrumental variable estimates of the difference of in-patient mortality between urban and rural hospitals

* If a F-statistic is less than 10, the instrumental variables are weak.** If p-value is less than 0.05, one of the instrumental variables correlated with unmeasured confounders

IV models (n=12,191)  

Number of groups for

instrumental variable

Tests for instrumental variablesIV estimates

of mortality difference

Instrument

P-value for overidentifying

restrictions tests** Coefficients P-valueF-statistic*

Unadjusted

2 1540.16 - -0.0199 0.34

4 642.65 0.65 -0.0269 0.16

12 184.31 0.13 -0.0288 0.13

Adjusted for demographic variables

2 1568.24 - 0.0127 0.58

4 652.86 0.80 0.0081 0.69

12 187.14 0.10 0.0065 0.75

Adjusted for demographic variables and Charlson comorbidity index

2 1539.9 - 0.0090 0.69

4 642.51 0.92 0.0053 0.80

12 184.29 0.12 0.0040 0.84

Adjusted for demographic variables and APR-DRG risk index

2 1694.27 - -0.0034 0.87

4 640.61 0.92 -0.0069 0.72

12 202.50 0.01 -0.0063 0.74

Results: Sensitivity analyses

Repeat analyses in different samples Excluding transferred in MI patients Three-year state inpatient datasets (2001 to 2003)

Different IV estimation method Two-stage residual inclusion method to account

for the endogeneity in nonlinear (logistic) model Bivariate Probit model (using Stata 9.0)

The results are consistent with IV estimation in Table 5

Discussion This study confirms earlier studies

MI patients admitted to rural hospitals were older and sicker than their urban counterparts

Traditional models all indicate significantly higher in-hospital mortality for those admitted to rural hospitals

Discussion Our findings suggest that the traditional logistic

regression models are biased Admissions to rural or urban hospitals are

likely to be confounded by unmeasured patient variables

Referral patterns in rural hospitals Younger and less sick patients are

transferred to urban hospitals The clinical judgment about transfer of

rural senior patients with MI may rely on different criteria

Discussion Patient preferences are likely to play a significant role

in transfer decisions for older MI patients May reflect personal choice or existing serious

comorbidities Serious cases may choose to remain close to

home The transfer patterns may reflect rural doctors

respecting their patients’ wishes Using in-hospital MI mortality to measure quality of

care in rural hospitals is problematic.

Limitations of the study

The results of the IV estimation can only be generalized to patients for whom distance affects their choice The conclusion cannot be applied to MI patients

bypassing rural hospitals and seeking care in urban hospitals

The findings for hospitals in one state may not generalize to other states .

Analyses of in-hospital mortality rates may not generalize to mortality rates after hospitalization.

Conclusions

Mortality from MI in rural Iowa hospitals is not higher when controlled for unmeasured confounders.

Current risk-adjustment models may not be sufficient when assessing hospitals that perform different functions within the healthcare system.

Unmeasured confounding is a significant concern when comparing heterogeneous and undifferentiated populations.

Did conversion to Critical Access Hospital (CAH) status affect patient

safety indicator performance?

Li, P., Schneider, J. E. & Ward, M. M., (2007) Effect of Critical Access Hospital Conversion on

Patient Safety. Health Services Research, 42 (6): 2089-2108

Background In order to protect small, financially vulnerable rural

hospitals, the Medicare Rural Hospital Flexibility Program of the 1997 Balanced Budget Act allowed hospitals meeting certain criteria to convert to critical access hospitals (CAH)

This changed their Medicare reimbursement mechanism from prospective (PPS) to cost-based

One objective of the policy was to increase the quality of care in these hospitals

Timeframe for Conversion to CAH

0%

20%

40%

60%

80%

100%

1997 1998 1999 2000 2001 2002 2003 2004 2005

Rural PPS hospitals CAHs

Patient Safety

4 PSIs and Composite AHRQ recommends suppressing the estimates if fewer than 30

cases are in the denominator

Only five patient safety indicators are able to provide PSI measures for all rural Iowa hospitals PSI-5: foreign body left during procedure PSI-6: iatrogenic pneumothorax PSI-7: selected infections due to medical care PSI-15: accidental puncture or laceration PSI-16: transfusion reaction

Too rare to provide variability to differentiate hospitals in Iowa

A composite patient safety variable was created by summing the four PSIs (PSI-5, PSI-6, PSI-7, and PSI-15).

Number of Hospitals Having Better or Worse Performance after CAH Conversion

0

5

1015

20

25

3035

40

45

PSI-5 PSI-6 PSI-7 PSI-15 Compositescore of4PSIs

Better performance worse performance

Cross-sectional Analyses Cross-sectional comparisons showed that CAHs had

better performance than rural PPS hospitals on 4 of the 5 PSI measures.

However, the difference in patient safety indicators might be due to differences in patient mix, hospital characteristics besides CAH conversion, and differences in markets and environment.

Multivariable Analyses

We used multivariable Generalized Estimating Equations (GEE) models and sensitivity analyses to control for the impact of patient case mix, market variables, and time trend.

GEE models showed that CAH conversion was associated with significant better performance in PSI-6, PSI-7, PSI-15 and composite PSI.

Findings were robust among sensitivity analyses using different samples and different methods

Conclusions CAH conversion in rural hospitals resulted in enhanced

performance in PSIs

We speculate that the likely mechanism involved an increase in financial resources following CAH conversion to cost-based reimbursement for Medicare patients

How did Critical Access Hospital conversion affect rural hospital

financial condition?

Li, P., Schneider, J. E. & Ward, M. M., Effects of Critical Access Hospital Conversion on the

Financial Performance of Rural Hospitals Inquiry (in press)

Objectives

To study the effects of CAH conversion on Iowa rural hospitals’ operating revenue, cost, and profit margin

Study Sample and Study design

Sample Eight year (1997-2004) panel data for 89 Iowa

rural hospitals (rural PPS hospitals and CAHs) Unit of analysis is hospital-year

Study design Quasi-experimental designs that use both control

groups and pretests Panel data regression with fixed hospital effects

Models

Ad hoc models: Revenueit=f(CAHit,Pjt,Yit,Xit) Costit=f(CAHit,Wjt,Yit,Xit) Marginit=f(CAHit,Wjt, Pjt,Yit, Xit)

Variables: CAHit: hospital status (CAH or rural PPS) for ith hospital in

year t Pit: output prices for ith hospital in year t Wit: input prices for ith hospital in year t Yit: output volume for ith hospital in year t Xit: other variables for ith hospital in year t that empirically

affect dependent variables

CAH variables

One dummy variable CAH=1, if the hospital is in CAH status

Three dummy variables CAH1it=1, if the hospital is in the first year of CAH

status, otherwise CAH1it=0 CAH2it=1, if the hospital is in the second year of

CAH status, otherwise CAH2it=0 CAH3it=1, if the hospital is in CAH status for more

than 2 years, otherwise CAH3it=0 Comparison group: Rural PPS

Other covariates Pit: output prices for ith hospital in year t

Medicare Part A (hospital) adjusted average per capita cost (AAPCC) as proxy of hospital output price (county level)

Wit: input prices for ith hospital in year t Hourly wages for registered nurses (county level)

Yit: output volume for ith hospital in year t Total number of acute discharges, total number of outpatient

visits, and average length of stay of acute discharges The squared and cubed output measures and interaction terms

will be included

Others Xit: other variables for ith hospital in year t that empirically affect dependent

variables Hospital size (number of beds) Hospital case-mix

Hospital mean DRG weight, percent of emergency visits, and percent of Medicare and Medicaid days among acute inpatient days

Variables reflecting the hospital market (we assumed the county to be the relevant geographic market of hospital services.) Herfindahl-Hirschman Index (HHI), per capita income, and

population density in the county in which the hospital is located Year dummy variables which will adjust the effects of unmeasured, time-

specific factors Revenue and expense functions were log transformed

Data Sources

Iowa Hospital Association Profiles Iowa State Inpatient datasets Area Resource File Centers for Medicare and Medicaid Services American Hospital Association Annual Survey

Database Bureau of Labor Statistics

Result:Table 1: Changes in rural hospital patient care revenue, expense, and operating margin associated with CAH conversion, 1998-2004

Log(operating revenue) Log(operating expense) Operating margin Covariate

Coefficient Standard

error Coefficient

Standard error

Coefficient Standard

error

Hospital status

Rural PPS: CAH =0 Reference Reference Reference

CAH 0.0288** 0.0110 0.0199** 0.0089 0.0020 0.0078

…

Observations 623 623 623

Groups 89 89 89

R-Squared (within) 0.8833 0.9081 0.2177

* P-value< 0.1 ** P-value< 0.05

Table 2: Changes in rural hospital patient care revenue, expense, and operating margin during the first, second and third plus years of CAH conversion,

1998-2004 Log(operating revenue) Log(operating expense) Operating margin

Covariate Coefficient

Standard error

Coefficient Standard

error Coefficient

Standard error

Hospital status

Rural PPS: CAH =0 Reference Reference Reference

CAH1 0.0034 0.0114 0.0175* 0.0096 -0.0206** 0.0079

CAH2 0.0712** 0.0137 0.0206* 0.0114 0.0386** 0.0097

CAH3 0.0934** 0.0159 0.0483** 0.0133 0.0543** 0.0110

Observations 623 623 623

Groups 89 89 89

R-Squared (within) 0.8929 0.9101 0.3101

* P-value< 0.1 ** P-value< 0.05

Results Operating revenue

No change in the first year of conversion (paid an interim rate) Significant increases since the second year of CAH conversion

Operating expenses CAH conversion is associated with significant increase in hospital

operating expenses Hospitals increase expenses in the first year of conversion

Operating Margin Significant drop in the first year of conversion Significant increase since the second year of conversion

Sensitivity analyses showed similar results

Conclusions CAH conversion in rural hospitals resulted in better

patient safety.

Rural hospital CAH conversion was associated with significant increases in hospital operating revenues, expenses and margins

Summary: Limitations of measures

In-hospital mortality Substantial unmeasured confounders

Patient Safety Indicators Only small number of indicators can be applied to

rural hospitals Changes of indicators might reflect changes in

coding or reporting in administrative data We need hospital quality indicators specifically for

rural hospitals

Thank you Questions?

Contact information Pengxiang (Alex) Li University of Pennsylvania penli@mail.med.upenn.edu