Predictors of HEDIS Performance and Improvement

Dennis P. Scanlon, Ph.D. The Pennsylvania State University

Shailender Swaminathan, Ph.D.

The University of Alabama at Birmingham

Michael Chernew, Ph.D. The University of Michigan

James Bost, Ph.D.

University of Arkansas for Medical Sciences National Committee for Quality Assurance

John Shevock, M.S.M.

The Pennsylvania State University

September 26, 2003

Acknowledgments: This research was supported by a grant from the Agency for Healthcare Research and Quality (AHRQ) grant # P01-HS10771.

Abstract Objective: To identify factors related to an HMO’s performance on the standardized HEDIS and

CAHPS measures and factors related to improvement in those measures over time.

Study Design: Longitudinal analysis of a four year panel of HEDIS and CAHPS data (calendar

year 1998-2001). All plans reporting to NCQA, regardless of their decision to allow the data to

be publicly available, were included. All plans that reported data in at least one year of the panel

were included.

Data Sources: Data were obtained from a variety of sources including the National Committee

for Quality Assurance (NCQA), Interstudy, the Area Resource File, the U.S. Office of Personnel

Management, and the U.S. Department of Labor.

Methods: Multivariate growth models were estimated on market demographic and competition

variables, as well as health plan characteristics and decisions regarding data collection and

reporting.

Principal Findings: Our estimates don’t support the hypothesis that greater competition is

associated with better performance on the six HEDIS measures examined. HMO penetration is

positively related to HEDIS performance for four of the six measures. Health plan

characteristics such as plan profit status, model type, reporting method and decision to allow the

data to be made publicly available are all significantly related to better HEDIS performance. The

growth parameters indicate that the relationship between the covariates and HEDIS performance

is stable over time.

Keywords: Managed Care, Health Maintenance Organizations (HMOs), Quality, Performance

Measurement, National Committee for Quality Assurance, Health Employer Data and

Information Set (HEDIS), Competition, Markets, Profit Status

2

I. Background and Conceptual Framework

The quality of medical care in America has been the focus of considerable attention in

recent years. Highly publicized reports have made the shortcomings of the American health care

delivery system more transparent. One branch of that literature has focused on patient safety and

medical errors, catalyzed by the Institute of Medicine’s To Err is Human report [IOM, (1999)].

Another branch of literature has focused on the quality of medical care generally, and finds that

Americans often fail to receive health treatments and services considered appropriate by medical

experts [McGlynn et al., (2003), IOM, (2001)]. These findings suggest the importance of

understanding the determinants of quality and errors so these shortcomings can be addressed.

Attention to quality is nothing new. Researchers have been investigating the quality of

medical care for decades [Donebedian, (1980)]. Much of the early quality literature focused on

health care providers, particularly hospital care. However, as managed care has grown, the focus

has expanded to include the quality of care provided to health plan enrollees. Health

Maintenance Organizations (HMOs) have attracted most of the attention because the link

between health care financing and health care delivery is stronger in HMOs than in other forms

of managed care. While there are many types of HMO models, most plans have restricted

provider networks and utilize the primary care physician (PCP) as the point-of-referral for

specialist care and for expensive diagnostic or therapeutic services. Many HMOs are also

engaged in care management activities, aggressively developing (or contracting for) programs

for the management of chronic illnesses, and to manage the health needs of members with

multiple comorbidities. Hence, sophisticated HMOs may be well equipped to ensure that

members receive appropriate preventive, chronic and acute care.

3

Concern about the potential adverse influence that HMOs might have on quality has

fueled efforts to measure and report quality at the health plan level. Most of these efforts have

utilized the Health Plan Employer Data and Information Set (HEDIS), and the Consumer

Assessment of Health Plan Survey (CAHPS). HEDIS includes approximately 60 clinical care

process compliance and outcomes measures evaluating preventive and chronic illness care,

access and utilization. CAHPS is a survey based set of quality indicators measuring consumers’

self-reported experience and satisfaction with their health care and health plan. HEDIS and

CAHPS form the basis of many health plan report cards and are often required as a condition of

contracting by both private employers and public purchasers.

Research investigating factors related to the quality of care provided by HMOs has

focused on specific health plan or enrollee traits (e.g., plan ownership status and financial

performance, age, gender, and income distribution of the population, and race/ethnicity of plan

enrollment) but has not incorporated market traits such as HMO competition [Born and Simon

(2001), Himmelstein et al., (1999), Landon et al., (2001), Zaslavsky et al., (2000c)]. Other

literature has examined the percentage of variation in health plan performance on HEDIS and

CAHPS measures attributable to the region, the market, or the health plan, without relating the

variance to specific plan or market attributes [Zaslavsky et al., (2000b)]. A recent review of the

literature concurred that no studies have examined the relationship between health plan quality

and HMO competition while controlling for other important covariates [Morrissey, (2001)].

Since this review, one unpublished study has used cross-sectional data and latent variable

modeling to examine whether market and plan characteristics are related to HEDIS and CAHPS

performance [Scanlon et al., (2003)].

4

Understanding the correlates of HMO quality and the factors affecting quality

improvement is important for understanding the most effective means of achieving improvement.

Moreover, understanding whether competition and other market characteristics will drive

improvement is crucial to understanding whether desired quality improvements are best achieved

by regulatory or market approaches. Just as market structure is a central component of price

theory, with the existing literature relating the financial outcomes of providers to market

characteristics [Feldman et al., (1996), Wholey et al., (1996), Feldman et al., (1990)], so too

should health plan quality be reflective of market traits, because like price, quality is an

endogenous outcome of the competitive process.

Economic theory is equivocal regarding the impact of competition on quality or the

relationship between quality and health plan costs [McLaughlin & Ginsburg, (1998)]. The extent

to which market forces influence health plan quality depends on the relative value consumers

place on quality vs. costs, the extent to which better quality is costly to achieve (some contend

better quality actually reduces costs), the business objectives of health plans, and the availability

of accurate information about quality.

We hypothesize that plan performance and improvement on the HEDIS and CAHPS

measures is a function of health plan competition and managed care penetration. HMO

competition is potentially important since HEDIS and CAHPS are now commonly required as

part of contracting by employers, and since some employers provide employees with

comparative HEDIS and CAHPS information and in some cases even adjust employee out-of-

pocket premiums based on how plans’ score on these measures [Scanlon et al., (2002)]. HMO

penetration is important since providers are usually contractually affiliated with several HMOs,

and since aggregate physician practice patterns may converge towards compliance with the

5

HEDIS indicators as HMOs become more prevalent in the marketplace [Chernew et al., (2003)].

Since large employers and the Federal Employees Health Benefits Program (FEHBP) have been

major proponents of HEDIS and CAHPS, we hypothesize that plans’ may pay more attention to

improving these measures in markets with significant large employer presence.

In this paper we relate health plan performance on six HEDIS measures to market

characteristics, controlling for health plan traits and market demographics. We seek to identify

specific market and plan characteristics that are associated with favorable performance and

longitudinal improvement. Unlike Scanlon et al. (2003), we use a four year panel to examine

whether HMO competition and HMO penetration are related to better plan performance and

quality improvement.

II. Methods

Our empirical model follows the random growth models estimated in the earnings

literature where variations in earnings over time are decomposed into level and growth

components (Lillard and Weiss, 1979; Hause, 1980). These models also allow the level and

growth components to be correlated. Thus in our model plans that start at high levels of quality

may grow less quickly.

Specifically, we model the performance of health plan ‘i’ at time ‘t’, (qit), as a function of

plan and market traits (Xit). Since certain plan and market traits may be associated with

improved performance over time, we permit the coefficients for each covariate to follow a linear

time trend (T). Formally the model can be expressed as:

0 1

0 1

(1)it t t i it

t

q X T uT

β α γα α α

= + + += +

6

where uit is a stochastic error term. We partition the error, uit into an iid component (νit), and a

random component ( iη ), where iη accounts for unmeasured variations among plans in quality.

We also allow the error to follow an AR(1) process, such that:

1 1it i it itu u vη ρ −= + + where is the autocorrelation coefficient and 1ρ 2~ (0,v vNσ σ )

There might be important unobserved plan characteristics that could influence a plan’s

ability to improve its performance on the HEDIS measures over time. We allow the coefficient

on T (which measures the extent to which plans improve over time) to incorporate a plan specific

random effect ( ). iδ

1 0i iγ γ δ= +

It is the correlation between this plan specific random growth effect and the plan specific

random level effect ( iη ) which allows for the possibility that plans with strong baseline

performance will improve less quickly. If this were the case we would expect the correlation

between and iδ iη ( δηρ ) to be negative.

2

2

0( , ) ~ ( , )

0i i

SymN η

δη η δ δ

ση δ

ρ σ σ σ

Plans providing a single year of data were also included in the sample. However, a

majority of the plans provide multiple years of data which help us identify both the variance and

correlation in the growth and level components. After making the appropriate substitutions, the

final model we estimate is:

0 0 1 0 1 1( ) ( ) (2)it t i i it itq T X T uβ α α γ δ η ρ −= + + + + + + + v

The model is estimated by maximum likelihood methods and we seek to estimate the following

parameters: , , ,0β 0α 1α 0γ , ,δσ ησ , 1ρ , δηρ and vσ

7

If we ignore the AR(1) process for a moment:

0 0 1 0

0

1

( | , ) ( ) ( ) (3) is average effect of unit change in X on mean quality score. is change in that effect over time.

it t tE q X t t X tβ α α γαα

= + + +

To get a sense of how well our model does in predicting variations in performance across plans,

we can decompose the total variation in quality into a component that is explained by the X’s

and a component that is unmeasured (equation 4). This allows us to compute a pseudo R-

squared.

2 2 2 2 2 2 2 20 1( | , , ) ( ) 2 ( , ) (4)it t i i x vV q X t t t Covδ ηη δ α α σ σ σ δ η σ= + + + + +

III. Data and Sample

The primary data sources used to derive the analytic sample were NCQA’s HEDIS data

and the Interstudy Corporation’s MSA Profiler and Competitive Edge (calendar year 1998-2001)

data. In addition, other data sources were used as supplements (Table 1). NCQA data included

observations from all plans reporting HEDIS or CAHPS data in Quality Compass 1999 through

Quality Compass 2002 (i.e., the publicly reporting plans), as well as plans that reported data to

NCQA but requested that the information not be included in Quality Compass (i.e., the non-

publicly reporting plans)1. The HEDIS data reflects member health care encounters and survey

responses occurring during the prior calendar year (e.g., 1999 for Quality Compass 2000). The

number of health plans reporting data ranged from 338 for calendar year 2001 to 462 for

calendar year 1998, with the reduction over time due primarily to mergers and acquisitions.

In each year, about 30-35% of HMO and POS plans operating in the United States do not

report data (either publicly or non-publicly) to NCQA. In calendar year 1999, 37% of reporting

plans were HMOs, while 56% of the plans were HMOs with POS options. The other 7% of 1 Because NCQA researchers were involved with this project, NCQA provided access to the non-public reporting data under strict confidentiality and anonymity agreements.

8

plans were listed as POS plans and were dropped because comparable data was not available

from Interstudy. For our analysis we included any plan reporting in one or more of the four

years of our panel. Table 3 lists the percentage of plans reporting multiple years of data for each

measure. Essentially, plans providing only one year of data would only influence the average

effect estimates (i.e., the α0’s) but not the growth estimates (i.e., the α1’s)

Since NCQA and Interstudy do not use common health plan identification codes, we

merged these two data sources manually, relying primarily on the health plan name, the state of

service, and the MSA(s) the plan served. Statistics from these merges indicate that we were able

to match about 92% of the NCQA observations with an Interstudy observation in any given year.

While the number of plans reporting data to NCQA represented only 56% of the universe of

health plan-MSA combinations contained in Interstudy in 1999, these NCQA plans accounted

for 72% of the total pure commercial enrollment in the United States. Even though Interstudy

and NCQA each use their own common plan identification codes to link plans over time,

facilitating the longitudinal merging of the data, we also performed manual checks of the data

since there was a high rate of plan mergers, acquisitions, and name changes during this four year

period of HMO consolidation.

We include the following plan traits in the model; profit status, HMO model type, plan

age, whether the plan allows NCQA to publicly report the HEDIS data, and the data collection

method used for each HEDIS measure. These were selected because they are commonly used in

the literature and are routinely collected. The MSA demographic variables (e.g., per capita

income, % of MSA population that is non-white population) were obtained from the 2001 Area

Resource File (ARF), which primarily uses the 2000 census as the source of information for

these variables. In addition, we obtained MSA level information on the number of employees

9

working for large firms (firms with >500 employees) from the Bureau of Labor Statistics

employment data.

Because health plans do not report performance at the MSA level, although many operate

in multiple markets (MSAs), the unit of analysis in our study is the health plan. On average plans

operate in 5.06 MSAs (minimum= 1, maximum= 25 in 1999) 2. While variables such as the

Herfindahl index and the HMO penetration rate are all generated at the MSA level, as are the

market socio-demographic characteristics, we aggregate these variables to the plan level by

taking the weighted average for each plan over the markets that it serves. The weights are the

estimated plan specific enrollments in each MSA (based on calculations from the Interstudy

data). For example, if HMO A operated in 3 MSAs, we would compute a weighted average of

the HMO penetration, Hirschman-Herfindahl index, and other MSA socio-demographic

variables, where each MSA’s data is weighted according to the proportion of the plan’s total

enrollment accounted for by each plan/MSA combination.3 This is consistent with a model in

which plan level performance reflects the enrollee weighted average performance in each of the

MSAs the plan serves.

While there are approximately 60 HEDIS clinical process, outcome, and utilization

measures, and 10 CAHPS composite scores and ratings, we chose six HEDIS measures for our

initial analysis. For purposes of this paper, we chose two immunization measures (the childhood

DTP immunization rate and the adolescent measles, mumps and rubella (MMR) immunization

rate), two measures capturing recommended preventative cancer screenings for eligible women

(cervical and breast cancer screening rates), one screening measure for chronically ill members

with diabetes (annual eye exam rate), and one measure of appropriate prescription of medication

2 For example, the largest plan in 1999 operates in 25 MSAs in the state of California, but reported only one set of HEDIS/CAHPS data to NCQA. 3 This proportion is estimated by using both the Interstudy plan level and MSA level datasets.

10

for members discharged from the hospital with acute myocardial infarction (the beta blocker

prescription rate). For each of the six measures, we examined the reported data for potential

erroneous values or outlier observations. After consulting with NCQA, we assumed that plans

with measures that changed by more than 10 percentage points for each year the measure was

reported (e.g., a change of greater than 40 percentage points in absolute value for plans reporting

data for all four years for a measure) was erroneous. Because we were concerned that these

observations would influence our estimates, they are dropped from the results that we report

here.

Tables 3 provides information about our sample and the number of plans reporting data

for the four measures for which we observed four years of data, and the two measures for which

we observed three years of data. The table reveals that there is variation in the number of years

of data reported by plans for each measure, allowing us to identify key parameters in the model.

Table 4 provides descriptive statistics for the six HEDIS measures included in our analysis. The

tables suggests that the mean value for each measure increased over time, ranging from

improvement of three percentage points for the breast cancer screening rate to eleven percentage

points for the adolescent immunization rate. However, the statistics in Table 4 mask the degree

of change over time for a given plan, so table 5 presents the results of the average plan change

for each measure over time for plans reporting all eligible years of data for a given measure.

Similarly, Table 6 provides descriptive statistics for the covariates that we include in the growth

and fixed effects models that we estimated, for all years of data. The table indicates that firms

faced slightly more competition over time (1 = monopoly) and slightly less HMO penetration.

About 75% of the plans were for-profit organizations, while the majority of plans were

IPA/Network or mixed model HMOs. The rate of public reporting to NCQA increased over time

11

from 55% to 83%, and the average per-capita income for the MSAs plans’ served was about

$28,000.

IV. Results

Table 7 presents results from the random coefficient HEDIS growth model. The

estimates suggest that greater HMO competition (1 = monopoly) results in lower levels of

HEDIS performance for four of the six measures, but none of these estimates are not

significantly different from zero. Although not significantly different from zero, the estimates

suggest that plans serving areas with a weighted Herfindahl index of 0.25 have a one percentage

point lower score on breast cancer screening than plans serving areas with a Herfindahl of 0.75.

A similar difference in the Herfindahl index is associated with a 1.3 point difference for cervical

cancer screening and a 1.45 point difference on the eye exam scores, but for Beta Blocker

utilization, plans operating in the more monopolistic markets achieve a 2.25 percentage point

lower score.

In contrast, greater HMO penetration, holding plan competition constant, is associated

with better performance for four of the six HEDIS measures (at the alpha = 0.05 level of

significance). A one standard deviation increase (from the mean value) in the HMO penetration

rate increases breast cancer screening scores, cervical cancer screening and adolescent MMR

immunization scores by slightly over 1 percentage points while the childhood immunization

scores increase by approximately 2 percentage points. Eye exam and Beta Blocker scores are not

affected by increased managed care penetration.

No consistent pattern is observed for the effect of large firms on quality, and the only

significant effect is observed for cervical cancer screening where a negative association is

observed. Socioeconomic advantage is generally associated with better plan performance. In

12

particular plans operating in areas where the proportion of non-whites is 0.25 have between 1.5

and 5.4 percentage points lower scores than plans operating in areas where the proportion of non-

whites is 0.15. Higher per-capita income is associated with slightly higher scores for 2 of the 6

measures.

Our results regarding plan level covariates and performance are generally consistent with

the previous literature. For profit plans have significantly lower mean scores on 5 of the 6

measures, consistent with cross-sectional findings [McCormick et al., (2002), Scanlon et al.,

(2003)]. The scores are 2.75 percentage points lower for cervical cancer screening while they are

about 5 percentage points lower for the diabetic eye exam rate, with the effects on the other

measures lying within that range. Relative to “mixed” model plans, staff-group model HMOs

have higher scores, but the effect is significant only for the childhood immunization DTP rate.

Plans that publicly report their data have consistently higher scores than those that do not,

suggesting a ‘selection’ phenomenon around reporting. Finally, plans that report data using the

hybrid method have a 5.6 point higher score for breast cancer screening and over 25 point higher

score on childhood immunization, 28 points higher on adolescent immunization and 11 points

higher on Beta Blocker scores. These large differences for the immunization and beta blocker

rates probably reflect the difficulty of capturing utilization of these services from administrative

claims databases relative to patient charts. Experienced (older) plans also have higher quality

scores for all six measures. Each additional year a plan has been in operation increases mean

quality scores from between 0.14 to 0.5 percentage points. In addition, for the childhood

immunization and adolescent MMR measures, plans improve their scores by about 13 percentage

points each year.

13

How well do our covariates predict variations in quality across plans? Equation 4

provides some indication where the total variation in quality at any one point in time can be

separated into a component that is explained by the covariates and a component that is not.

When the “pseudo-R2” is computed for 1999, it ranges from a low of 0.15 for breast cancer

screening to a high of 0.32 for adolescent immunization.

The relationship between plan and market traits and performance appears to be stable

over time. The coefficients that allow the relationship between performance and the covariates

to follow a linear time trend (the ’s) are generally not statistically different from 0 and no

interesting patterns emerge, except that for the child and adolescent immunization rates, the

effect of using the hybrid reporting method appears to wane over time. For example, in the case

of childhood immunization, the scores on hybrid reporting plans are only 12 points higher in

1999 than the scores on non-hybrid reporting plans while the difference was 25 points in 1998.

While the effect of plan and market characteristics don’t appear to change over time, this finding

might be driven by within group correlation in the X’s. To explore the extent to which this is

true, we divided our set of covariates into three groups: group 1 was the competition variables,

group 2 was the market socio-demographic variables, and group 3 was the plan level variables.

Likelihood ratio tests on the joint significance of the variables in each of those groups were

conducted. Table 8 presents results from Likelihood ratio tests for the stability of parameter

estimates over time (the ). The LR tests also support the hypothesis that the relationship

between the competition measures and performance is stable over time. At the 5 percent level of

significance, we cannot reject the null hypothesis that the competition coefficients (i.e., group

1)are jointly zero for all the measures. However, the null hypothesis that the coefficients for the

market socio-demographic variables (i.e., group 2) are jointly equal to zero can be rejected for

1α

1 'sα

14

both the cancer screening measures. Likewise, we can also reject the null hypothesis of zero

coefficients on the health plan variables (i.e., group 3) for the cervical cancer screening rate and

the two immunization rates.

Although the estimates of the AR(1) parameter ρ1 did not converge in all models, the

estimates of this parameter for the cancer screening rates suggests that the parameter was small

and the results reported above are not substantially changed when the model is estimated without

the AR(1) assumption. The estimate of δηρ suggests a strong inverse relationship between base

level performance and improvement in performance, such that plans which performed well in

1998 had slower improvements over the study period.

Finally, there are important unmeasured differences among plans in quality. The standard

deviation of the unmeasured level component ( ησ ) is large in magnitude and accounts for over

80 percent of unmeasured variation in quality across plans for each of the six measures. There is

also significant variation in the growth (learning) component ( ) across plans. However,

relative to unmeasured variations in the levels, variations in growth explain fairly small fractions

of the total variation in health plan quality.

δσ

V. Discussion

Estimates from our HEDIS growth model suggest that the most significant predictors of

health plan performance on the six HEDIS measures examined are plan tax status, plan age, the

data collection method used by the health plan, and whether the plan allowed NCQA to report its

data publicly. MSA socio-demographic variables such as the percentage of non-white

population and per capita income also achieve significance for some of the HEDIS measures.

However, one of our principle measures of interest, HMO competition, was not

significantly related to health plan performance for any of the HEDIS measures examined.

15

There has been a fair amount of discussion in the health economics literature regarding the

measurement of managed care competition [Baker, (2001)], and it may be that our measure is not

capturing competition appropriately, rather than the case of competition being unimportant. In

particular, it may be important to consider the role of alternative insurance products such as

preferred provider organizations (PPOs) and other more ‘loosely’ managed products when

thinking about how to measure competition in insurance markets.

In addition, one avenue we intend to pursue is the inclusion of measures that capture the

lagged relative deviation of the HEDIS scores among plans within markets, under the assumption

that plans are aware of how their competitors are performing, and strive to not look significantly

worse than the best performing plan within the market. Unlike competition, managed care

penetration is positively related to HEDIS performance for four of the six measures, which

suggests convergence in performance as HMO prevalence increases. However, this result could

also be driven by overlapping provider networks, so future work should consider the underlying

reasons for the effect of HMO penetration. While our model allowed the impact of the

covariates to change over time, our estimates suggest that the importance of the covariates for

plan quality is relatively stable over time. However, we do find evidence that plans with higher

levels of initial performance experience less growth over time.

Our findings should be considered preliminary and are subject to several limitations.

First, we only used a small number of HEDIS measures, and we have yet to include the CAHPS

measures, which are based on consumer reported satisfaction with their health plan and health

care. We plan to conduct additional estimation that will include some of the survey based

CAHPS measures, and we will test the sensitivity of our results by using additional HEDIS

measures as well. Second, our models estimate the effects of measures separately, which makes

16

it difficult to get an overall sense of the relationship between plan ‘quality’ and the covariates of

interest when there are many performance measures. We overcome this problem in cross-

sectional work by using the multiple measures to estimate latent variable models [Scanlon et al.,

(2003)], hence future work will explore the estimation of longitudinal latent growth models,

though estimation using a panel requires strict assumptions about the stability of the measures

comprising the latent variables, and the degree of influence that each measure has on the latent

construct.

Though preliminary, our results have potentially important policy implications for those

interested in increasing quality generally, and HMO performance specifically. Many policy

analysts have speculated that the combination of increased HMO competition and the availability

of standardized quality information should lead to better performance on measurable aspects of

quality. Our preliminary results do not support the hypothesis that performance improves more

rapidly in highly competitive markets, but we do find evidence that performance is better where

HMOs are more prevalent. While the effect of HMO penetration may be due to overlapping

provider networks, it may also be due to increased acceptance within the medical community of

population health management approaches. In any event, with the trend in commercial insurance

moving away from plans that tightly integrate health care financing and delivery towards

discounted fee-for-service arrangements, it is important to understand whether competition

among alternative insurance products (e.g., HMOs v. PPOs) verses competition among similar

insurance products (e.g., among HMOs) has a differential impact on quality.

17

References:

Baker, LC. 2001. “Measuring Competition in Health Care Markets.” Health Services Research, 36(1), Part II, 223-251. Born PH and CJ Simon. 2001. “Patients and Profits: The Relationship Between HMO Financial Performance and Quality of Care”. Health Affairs, 20(2): 167-174. Donabedian, A. 1980. The Definition of Quality and Approaches to Its Assessment. Ann Arbor, MI: Health Administration Press. Feldman, R, Wholey, D, and J Christianson. 1996. “Effects of Mergers on Health Maintenance Organization Premiums.” Health Care Financing Review, 17(3):171-189. Feldman, R, Hung-Ching, C, Kralewski, J, Dowd, B, and J Shapiro. 1990. “Effects of HMOs on the Creation of Competitive Markets for Hospital Services.” Journal of Health Economics, 9(2):202-222. Hause, John (1980). "The Fine Structure of Earnings and On-the-Job Training Hypothesis", Econometrica 48(4), pp. 1013-1029. Himmelstein, DU, Woolhandler, S, Hellander, I, and SM Wolfe. 1999. “Quality of Care in Investor-Owned vs. Not-for-Profit HMOs”. Journal of the American Medical Association, 282(2):159-163. Institute of Medicine. 1999. To Err is Human: Building a Safer Health Care System. National Academy Press, Washington, D.C. Institute of Medicine. 2001. Crossing the Quality Chasm. National Academy Press, Washington, D.C. Landon, BE, Zaslavsky, AM, Beaulieu, ND, Shaul, JA and PD Cleary. 2001. “Health Plan Characteristics and Consumers’ Assessments of Quality.” Health Affairs, 20(4):274-286. Lillard, Lee and Yoram Weiss (1979). "Components of Variation in Panel Earnings Data: American Scientists 1960-1970", Econometrica 47(2), pp. 437-454. McCormick, D, Himmelstein, DU, Woolhandler, S, Wolfe, SM, and DH Bor. 2002. “Relationship Between Low Quality-of-Care Scores and HMOs’ Subsequent Public Disclosure of Quality-of-Care Scores”. Journal of the American Medical Association, 288(12):1484-1490. McGlynn, EA, Asch, SM, Adams, J, Keesey, J, Hicks, J, DeCristofaro, A, and EA Kerr. 2003. “The Quality of Health Care Delivered to Adults in the United States.” New England Journal of Medicine, 348(26): 2635-2645.

18

McLaughlin, CG and PB Ginsburg. 1998. “Competition, Quality of Care, and the Role of the Consumer.” Milbank Quarterly, 76(4):737-743. Morrisey, MA. 2001. “Competition in Hospital and Health Insurance Markets: A Review and Research Agenda.” Health Services Research, 36(1), Part II, 191-222. National Committee for Quality Assurance. 2002. State of Health Care Quality Report. Washington, D.C. National Committee for Quality Assurance. 2001. State of Managed Care Report. Washington, D.C. National Committee for Quality Assurance. 2000. State of Managed Care Report. Washington, D.C. National Committee for Quality Assurance. 1999. State of Managed Care Report. Washington, D.C. Scanlon, DP, Chernew, ME, Doty, HE, and DG Smith. 2001. “Options for Assessing PPO Quality: Accreditation and Profiling as Accountability Strategies.” Medical Care Research and Review. 58(Supplement 1): 70-100. Scanlon, DP, Chernew, ME, McLaughlin, C, and G Solon. 2002. “The Impact of Health Plan Report Cards on Managed Care Enrollment.” The Journal of Health Economics, 21(1):119-42. Scanlon, DP, Swaminathan, S, Chernew, M, Bost, J, and J Shevock. 2003. “Health Plan Performance: Evidence from Managed Care Insurance Markets.” Under review. Chernew, ME, McLaughlin, C, Wodchis, W, and DP Scanlon. 2003. “Overlap in HMO Provider Networks.” Under review. Wholey, D, Feldman, R, Christianson, JB, and J Engberg. 1996. “Scale and Scope Economies among Health Maintenance Organizations.” Journal of Health Economics, 15, 657-684. Zaslavsky, AM, Landon, BE, Beaulieu, ND, and PD Cleary. 2000b. “How Consumer Assessments of Managed Care Vary Within and Among Markets”. Inquiry, 37(2):146-161. Zaslavsky et al. 2000c. “Impact of Sociodemographic Case Mix on the HEDIS Measures of Health Plan Quality.” Medical Care, 38(10):981-992.

19

Table 1: Data Sources

Source of Data File Name Relevant Variables NCQA HEDIS 1999-2002 Independently audited calendar

year 1998-2001 HEDIS data for public and non-public reporting health plans.

Interstudy MSA Profiler MSA level managed care data, including plan total MSA enrollment across all products offered in the MSA – 1998-2001.

Competitive Edge Health plan level managed care data, including total plan enrollment reported separately by product type (e.g., HMO, POS) and population served (i.e., Medicare, Medicaid, Commercial) – 1998-2001.

Market Competition Derived data set calculating the Hirschman-Herfindahl index for the commercial HMO and HMO-POS population – 1998-2001.

Area Resource File 2001 ARF Population demographic characteristics for the MSAs (or county) served by managed health plans.

Bureau of Labor Statistics BLS Employment Data Percentage of employed persons in an MSA working for large firms (>500 employees).

20

Table 2: Number of Plans Reporting HEDIS Data by Year

Number of Plans Reporting Year HEDIS / CAHPS 1998 462 1999 390 2000 386 2001 338

Table 3: Number of Plans Reporting Multiple Years of Data for each HEDIS Measure Number of Plans HEDIS Variable No report 1-year 2-year 3-year 4-year Total

16 183 78 89 130 Childhood Immunization - DTP 3.2% 36.9% 15.7% 17.9% 26.2%

496

22 180 82 83 129 Adolescent Immunization - MMR 4.4% 36.3% 16.5% 16.7% 26.0%

496

11 179 87 88 131 Breast Cancer Screening 2.2% 36.1% 17.5% 17.7% 26.4%

496

7 183 87 88 131 Cervical Cancer Screening 1.4% 36.9% 17.5% 17.7% 26.4%

496

228 73 68 127 NA Beta Blocker1 46.0% 14.7% 13.7% 25.6% NA

496

105 122 83 186 NA Comprehensive Diabetes Care - Eye Exams1 21.2% 24.6% 16.7% 37.5% NA

496

1 - Measures only have 3 years of data (1999-2001)

21

Tab

le 4

: Des

crip

tive

Stat

s for

HE

DIS

Mea

sure

s

1998

1999

2000

2001

HED

IS V

aria

ble

Mea

nSt

d D

evM

ean

Std

Dev

Mea

nSt

d D

evM

ean

Std

Dev

Ave

rage

Num

ber o

f Pla

ns

Rep

ortin

g C

hild

hood

Imm

uniz

atio

n - D

TP

77.7

313

.05

79.4

512

.91

80.9

612

.76

82.3

29.

5531

8 A

dole

scen

t Im

mun

izat

ion

- M

MR

56

.86

18

.80

61.6

217

.98

64.6

217

.37

67.9

217

.44

304

Bre

ast C

ance

r Scr

eeni

ng

72.9

77.

4273

.52

7.37

74.6

76.

4775

.62

6.03

324

Cer

vica

l Can

cer S

cree

ning

71

.23

9.54

72.2

29.

2178

.61

7.52

80.2

16.

6732

4 B

eta

Blo

cker

N

/A

N/A

84

.86

10.3

989

.10

8.87

91.9

56.

3522

3 C

ompr

ehen

sive

Dia

bete

s Car

e -

Eye

Exam

s N

/A

N/A

45

.86

14.8

848

.33

14.4

851

.93

13.4

832

1

Tab

le 5

: Cha

nge

in P

lan

Perf

orm

ance

for

HE

DIS

Mea

sure

s*

HE

DIS

Cha

nge

Var

iabl

e N

M

ean

Std

Dev

M

in

Max

C

hild

hood

Imm

uniz

atio

n - D

TP

148

4.15

3356

5 8.

6421

614

-12.

5757

58

57.5

0517

6 A

dole

scen

t Im

mun

izat

ion

- MM

R

147

11.7

7382

4 12

.947

418

-33.

5306

1 52

.817

57

Bre

ast C

ance

r Scr

eeni

ng

147

2.80

8361

7 4.

9678

209

-18.

3937

49

18.7

5275

C

ervi

cal C

ance

r Scr

eeni

ng

148

8.50

0182

4 7.

3875

566

-21.

5926

08

33.7

9629

6 B

eta

Blo

cker

14

9

6.

2859

561

8.41

3899

9-2

0.92

1986

35.7

7863

6C

ompr

ehen

sive

Dia

bete

s Car

e - E

ye E

xam

s 20

4 5.

5565

009

9.51

0795

9 -4

1.36

253

34.0

6326

* T

hese

stat

istic

s wer

e on

ly c

ompu

ted

for p

lans

pro

vidi

ng a

ll ye

ars o

f dat

a fo

r eac

h m

easu

re

Tab

le 6

: Des

crip

tive

Stat

istic

s for

Gro

wth

Mod

el C

ovar

iate

s

19

98

1999

20

00

2001

V

aria

ble

Mea

n St

d D

evM

ean

Std

Dev

Mea

n St

d D

evM

ean

Std

Dev

MSA

Her

finda

hl (e

xclu

ding

Med

icar

e an

d M

edic

aid)

1

0.

420.

210.

430.

240.

440.

240.

400.

19

MSA

HM

O P

enet

ratio

n1

0.

340.

120.

340.

120.

330.

130.

310.

13

MSA

Em

ploy

men

t in

Larg

e Fi

rms1

0.51

0.07

0.51

0.07

0.48

0.11

0.50

0.06

% M

SA F

eder

al E

mpl

oyee

s1

0.

030.

020.

030.

020.

020.

020.

020.

02

MSA

Per

-cap

ita In

com

e ($

1,00

0s) 1

27

.81

4.64

28.2

64.

6726

.92

6.62

27.4

34.

77

% M

SA P

opul

atio

n N

on-W

hite

1

0.

150.

090.

160.

090.

150.

090.

140.

09

MSA

Pop

ulat

ion1

1,79

3,55

91,

614,

774

1,90

0,78

21,

704,

552

1,79

1,01

11,

729,

768

1,70

2,85

11,

650,

899

For-

Prof

it H

ealth

Pla

n

0.

760.

430.

760.

430.

750.

450.

750.

65St

aff/G

roup

Mod

el P

lan

0.06

0.23

0.04

0.20

0.03

0.18

0.03

0.17

IPA

/Net

wor

k M

odel

Pla

n

0.

600.

490.

640.

480.

650.

480.

660.

47M

ixed

Mod

el P

lan

0.34

0.47

0.32

0.47

0.32

0.47

0.31

0.46

Plan

Age

(yea

rs)

13.9

28.

8714

.28

8.47

15.0

08.

8016

.03

9.03

Publ

icly

Rep

orts

HED

IS/C

AH

PS

0.55

0.50

0.65

0.48

0.72

0.45

0.83

0.38

Chi

ldho

od Im

mun

izat

ions

- D

TP H

ybrid

Dat

a C

olle

ctio

n 0.

95

0.21

0.

95

0.21

0.

92

0.27

0.

95

0.23

A

dole

scen

t Im

mun

izat

ions

- M

MR

Hyb

rid D

ata

Col

lect

ion

0.87

0.

34

0.91

0.

29

0.92

0.

28

0.91

0.

28

Bre

ast C

ance

r Scr

eeni

ng -

Hyb

rid D

ata

Col

lect

ion

0.70

0.

46

0.68

0.

47

0.64

0.

48

0.68

0.

47

Cer

vica

l Can

cer S

cree

ning

Hyb

rid D

ata

Col

lect

ion

0.85

0.

36

0.87

0.

34

0.75

0.

43

0.79

0.

41

Bet

a B

lock

er H

ybrid

Dat

a C

olle

ctio

n 0.

88

0.

330.

750.

430.

770.

420.

760.

43C

ompr

ehen

sive

Dia

bete

s Car

e - E

ye E

xam

s Hyb

rid D

ata

Col

lect

ion

0.25

0.44

0.94

0.23

0.97

0.18

0.97

0.18

1 - w

eigh

ted

varia

bles

for p

lans

serv

ing

mul

tiple

MSA

s

Tab

le 7

: HE

DIS

Gro

wth

Mod

el E

stim

ates

(sta

ndar

d er

rors

) α o

C

hild

hood

- D

TP

Ado

lesc

ent

MM

R

Bre

ast C

ance

r Sc

reen

ing

Cer

vica

l Can

cer

Scre

enin

g D

iabe

tes E

ye

Exam

B

eta

Blo

cker

47

.147

1 **

* 12

.023

4 *

69.4

485

***

64.8

352

***

30.1

820

***

57.2

798

***

β 0 (I

nter

cept

) (4

.945

)

(6.1

72)

(2.8

06)

(3.8

98)

(10.

165)

(10.

411)

-0.5

164

1.

5677

1.96

822.

5837

2.90

77-4

.528

7H

erfin

dahl

1 (2

.382

)

(2.8

26)

(1.5

48)

(1.9

88)

(3.7

45)

(4.2

19)

15.6

413

***

10.2

325

* 8.

2463

***

9.

3820

**

7.19

3 1.

1789

M

SA H

MO

Pen

etra

tion1

(5.2

93)

(5

.814

)(2

.983

)(3

.864

)(6

.635

)(6

.171

)-1

3.44

86

-17.

6136

8.41

85-1

5.87

42**

-0.8

872

4.32

41%

Em

ploy

ed b

y La

rge

Firm

s1 (9

.183

)

(12.

879)

(5.3

30)

(7.2

74)

(15.

666)

(16.

417)

0.22

96

0.94

76 *

**

-0.1

149

0.27

83 *

* 0.

2736

0.

0327

Pe

r Cap

ita In

com

e1 (0

.175

)

(0.2

16)

(0.0

77)

(0.1

19)

(0.2

21)

(0.1

94)

-15.

4059

*

-54.

4094

***

-1

9.85

48 *

**

-21.

8829

***

-3

4.28

82 *

**

-8.7

623

Perc

ent N

on-W

hite

1 (8

.695

)

(7.0

91)

(4.2

52)

(5.5

50)

(10.

464)

(10.

578)

< 0.

000

***

<

0.00

0 **

<

0.00

0 *

<0.

000

< 0.

000

<0.0

00

MSA

Pop

ulat

ion1

(0.

0000

)

(0.0

000)

(0

.000

0)

(0.0

000)

(0

.000

0)

(0.0

000)

-4

.690

5 **

-4

.631

2 **

-1

.413

-2

.740

9 **

-4

.957

3 **

-4

.255

1 *

Tax

Stat

us

(1.8

23)

(1

.925

)(0

.872

)(1

.255

)(2

.138

)(2

.363

)6.

4586

**

1.89

2.

7769

3.

9421

7.

4989

4.

232

Staf

f Gro

up M

odel

(3

.183

)

(3.4

86)

(1.8

87)

(2.6

23)

(4.9

74)

(3.4

07)

0.85

76

0.59

860.

3555

-1.2

024

0.28

212.

5702

*IP

A N

etw

ork

Mod

el

(1.1

39)

(1

.551

)(0

.578

)(0

.869

)(1

.675

)(1

.467

)0.

2911

***

0.

3524

***

0.

1610

***

0.

1385

**

0.49

92 *

**

0.31

32 *

**

Plan

Age

(0

.063

)

(0.0

97)

(0.0

38)

(0.0

62)

(0.1

19)

(0.1

04)

3.20

34 *

**

4.14

23 *

**

1.64

44 *

**

2.80

83 *

**

6.49

72 *

**

3.18

30 *

* Pu

blic

Rep

orte

r (0

.959

)

(1.2

96)

(0.5

88)

(0.7

33)

(1.5

03)

(1.6

13)

25.5

895

***

28.2

637

***

0.39

82

5.59

50 *

**

-0.4

932

11.0

325

***

Hyb

rid D

ata

Col

lect

ion

(1.6

24)

(2

.750

)(0

.530

)(1

.023

)(3

.037

)(4

.223

)

γ o

Chi

ldho

od -

DTP

A

dole

scen

t M

MR

B

reas

t Can

cer

Scre

enin

g C

ervi

cal C

ance

r Sc

reen

ing

Dia

bete

s Eye

Ex

am

Bet

a B

lock

er

13.1

056

***

13.2

151

***

-0.0

152

2.67

14

0.60

56

2.79

88

Yea

r (2

.044

)

(2.8

76)

(1.3

21)

(1.7

61)

(5.0

61)

(4.6

95)

α 1

Chi

ldho

od -

DTP

A

dole

scen

t M

MR

B

reas

t Can

cer

Scre

enin

g C

ervi

cal C

ance

r Sc

reen

ing

Dia

bete

s Eye

Ex

am

Bet

a B

lock

er

0.13

66

-0

.986

4-0

.216

-0.2

756

1.82

193.

2456

Her

finda

hl1

(1.3

25)

(1

.610

)(0

.859

)(1

.252

)(1

.991

)(1

.987

)-2

.529

5

1.56

29-1

.287

9-1

.597

10.

3861

0.84

17M

SA H

MO

Pen

etra

tion1

(2.1

21)

(2

.420

)(1

.244

)(1

.819

)(3

.250

)(2

.440

)-1

.670

1

-7

.149

4-5

.714

2**

0.88

41-9

.416

4-0

.925

% E

mpl

oyed

by

Larg

e Fi

rms1

(3.7

50)

(4

.926

)(2

.328

)(3

.321

)(7

.648

)(7

.172

)0.

0188

-0.1

105

*0.

0941

***

-0.0

482

0.07

12-0

.012

4Pe

r Cap

ita In

com

e1 (0

.065

)

(0.0

63)

(0.0

33)

(0.0

45)

(0.0

95)

(0.0

83)

1.50

71

2.41

151.

5851

5.45

84**

0.73

04-4

.798

3Pe

rcen

t Non

-Whi

te1

(3.2

16)

(3

.712

)(1

.610

)(2

.362

)(4

.861

)(4

.428

)0

0

00

00

MSA

Pop

ulat

ion1

0.00

0

0.00

0

0.00

0

0.00

0

0.00

0

0.00

0

1.38

95 *

* 0.

9594

0.

2243

0.

732

1.22

51

0.85

24

Tax

Stat

us

(0.6

31)

(0

.652

)(0

.370

)(0

.514

)(0

.866

)(0

.805

)-0

.513

3

0.00

3-0

.196

-0.2

175

0.89

670.

5623

Staf

f Gro

up M

odel

(1

.983

)

(1.5

71)

(0.7

16)

(1.4

64)

(1.8

35)

(1.3

05)

0.47

15

0.67

230.

1338

1.16

24**

*1.

0535

-0.3

114

IPA

Net

wor

k M

odel

(0

.457

)

(0.6

01)

(0.2

90)

(0.4

44)

(0.7

89)

(0.6

15)

-0.0

404

0.01

18-0

.016

1-0

.031

6-0

.021

2-0

.071

9*

Plan

Age

(0

.030

)

(0.0

39)

(0.0

17)

(0.0

27)

(0.0

52)

(0.0

38)

0.35

8 1.

0787

*

0.55

70 *

0.

333

0.59

84

1.03

47

Publ

ic R

epor

ter

(0.4

77)

(0

.626

)(0

.300

)(0

.363

)(0

.852

)(0

.751

)-1

2.92

36 *

**

-6.8

905

***

-0.0

29

-0.6

86

0.89

52

-1.8

018

Hyb

rid D

ata

Col

lect

ion

(0.8

27)

(1

.499

)(0

.303

)(0

.549

)(1

.863

)(2

.072

)

26

C

hild

hood

- D

TP

Ado

lesc

ent

MM

R

Bre

ast C

ance

r Sc

reen

ing

Cer

vica

l Can

cer

Scre

enin

g D

iabe

tes E

ye

Exam

B

eta

Blo

cker

11

.561

0 **

* 15

.126

7 **

* 5.

8651

***

8.

6906

***

10

.049

8 **

* 10

.949

1 **

* σ η

(0

.447

)

(0.7

62)

(0.3

94)

(0.2

96)

(0.8

72)

(0.5

71)

2.09

97 *

**

2.60

90 *

**

0.98

91 *

**

2.70

95 *

**

1.96

73 *

**

2.50

68 *

**

σ δ

(0.3

08)

(0

.485

)(0

.372

)(0

.190

)(0

.577

)(0

.405

)-0

.646

4 **

* -0

.572

7 **

* -0

.823

2 **

* -0

.759

3 **

* -0

.472

4 **

* -0

.894

4 **

* ρ δ

η (0

.059

)

(0.0

72)

(0.1

31)

(0.0

23)

(0.1

50)

(0.0

47)

ρ 1

NA

N

A

0

.322

9 **

0.

3205

***

N

A

NA

N

A

NA

(0.1

598)

(0

.119

9)

NA

N

A

4.59

41 *

**

6.23

23 *

**

3.00

88 *

**

2.64

05 *

**

6.38

90 *

**

4.82

61 *

**

σ υ

(0.1

18)

(0

.167

)(0

.267

)(0

.247

)(0

.204

)(0

.218

)

ln-L

-397

1.8

-415

7.65

-334

3.29

-367

2.51

-339

1.66

-3

153.

3

pseu

do-R

2 - 19

99

0.3

36

0.3

24

0.15

0

0.1

50

0.2

46

0.15

6

N

1139

1

117

1156

1

159

926

6

21

1 - w

eigh

ted

varia

bles

fo

r pla

ns se

rvin

g m

ultip

le M

SAs

*** p<

0.00

1 *

* p<

0.01

* p

<0.0

5

27

Tab

le 8

: Lik

elih

ood

Rat

io T

ests

(Pro

babi

lity

that

Chi

Squ

are

(k) >

) 0

12[

()

()]

Log

LLo

gL

−−

G

roup

1 ( H

HI,

HM

O

pene

tratio

n)

Gro

up 2

( Em

ploy

men

t in

larg

e fir

ms E

mpl

oyee

s, Pe

r ca

pita

inco

me,

non

-whi

te a

nd

MSA

Pop

ulat

ion)

Gro

up 3

(Tax

stat

us,

Staf

f/Gro

up m

odel

HM

O,

IPA

/Net

wor

k m

odel

HM

O,

plan

age

, pub

licly

repo

rts

HED

IS d

ata,

hyb

rid

repo

rting

met

hod)

C

hild

hood

- D

TP

0.42

1

0.

982

0.00

0A

dole

scen

t MM

R

0.93

1

0.

063

0.00

2B

reas

t Can

cer

Scre

enin

g 0.

410

0.00

80.

283

Cer

vica

l Can

cer

Scre

enin

g 0.

524

0.03

30.

000

Dia

bete

s Eye

Ex

am

0.51

8

0.

553

0.16

9

Bet

a B

lock

er

0.07

0

0.

534

0.11

1