Eva Stensland MD, PhDNational Network of Competence for

Medical Quality Registries

08.12.14

How to evaluate quality in medicine

Quality in medicine - definition

“The degree to which health services for individuals and populations increase the likelihood of desired health

outcomes and are consistent with current professional knowledge.”

Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the

21st Century. Washington D.C.: National Academy Press.

Six elements of quality• Safe – avoiding injuries to patients from the care that is supposed to

help them.• Effective – providing services based on scientific knowledge to all

who could benefit and refraining from providing services to those not likely to benefit (avoiding underuse and overuse).

• Patient-centered – providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions.

• Timely – reducing waits and sometimes harmful delays for both those who receive and those who give care.

• Efficient – avoiding waste, in particular waste of equipment, supplies, ideas, and energy.

• Equitable – providing care that does not vary in quality because of personal characteristics, such as gender, ethnicity, geographic location, and socioeconomic status.

Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington D.C.: National Academy Press.

Challenges when presenting and evaluating quality in medicine

• Different groups can have different reasons for measuring quality and hence different measurement criteria and emphasis.

• Clinicians or those who manage and provide clinical care might be interested in evaluating quality so that they can monitor and improve the services they are providing to individual patients.

• Regulators might be interested in ensuring that care provided by a hospital/health care organization meets a minimal standard and/or is making credible efforts to improve care quality.

• Patients/population might be most interested to know if treatment in their local hospital is safe or to get information they can use to choose hospital.

StructureProcess (activity) Result/outcome

Material resourcesHuman resources

DiagnosisTreatment

MortalityMorbidityQuality of life

Monitoring health care (Donabedian)

Outcome

QU

ALIT

Y

Structure

Process

If quality-of-care criteria based on structural, process, or intermediate outcomes are to be credible, it must be demonstrated that variations in the attribute they measure lead to differences in health status outcomes.

What indicators should we choose?•

Palmer RH: Int J Qual Health Care 1998;10:477-83

• Of the structural indicators, measures that predict variations in processes or outcomes of care have the greatest utility

• Process indicators are especially useful when: - quality improvement is the goal of the measurement process - short time frames are necessary - performance of low volume providers is of interest - and when tools to adjust or stratify for patient factors are lacking.

• Outcome data are useful if: - long time-frames are possible - performance of whole systems should be studied - or if a high volume of cases are available.

Comparisons of process data are easier to interpret and more sensitive to small differences than comparisons of outcomes data.

Outcome data are most useful for tracking care given by high-volume providers over long periods of time, and for detecting problems in implementation of processes of care.

Factors determining the outcome

Factors that are frequently included in risk adjustment models include: • patient demographic• psychosocial characteristics (such as age, sex, and functional

status)• lifestyle factors (smoking, alcohol use)• severity of the illness that is the focus for measurement• health status• co-morbid conditions.

Risk adjustment is important before comparing patient outcomes across hospitals or providers. Process indicators (eg. adherence to guidelines) might be more appropriate for comparing quality between hospitals

Quality indicators

An ideal indicator would have the following key characteristics: (i) indicator is based on agreed definitions, and described

exhaustively and exclusively(ii) indicator is highly or optimally specific and sensitive, i.e. it

detects few false positives and false negatives(iii) indicator is valid and reliable(iv) indicator discriminates well(v) indicator relates to clearly identifiable events for the user

(relevant to clinical practice)(vi) indicator permits useful comparisons(vii) indicator is evidence-based

Mainz J Int J Qual Health Care 2003;15:523-530

The use of quality indicators in a quality register

Quality indicator 0,5 point 1 point

Reperfusion 80 % 85 %

Reperfusion in recommended time 75 % 90 %

Coronary angiography 75 % 80 %

Heparin 90 % 95 %

ASA 90 % 95 %

Clopidogrel/Ticlopidin 85 % 90 %

Beta blocker 85 % 90 %

Lipid-lowering drugs 90 % 95 %

ACE-inhibitor 85 % 90 %

RIKS-HIA quality index

Swedeheart: Annual report 2013

OECD: Health Care Quality Indicators

Health at a glance 2013:Case-fatality in adults >45 within 30 days after admission for AMI

Health at a glance 2013

Patient-reported data

• PROM: patient reported outcome measures• PREM: patient reported experience measures• (PRI: patient reported incidents)

• PROMs includes patient reported symptoms, function, health-related quality of life, ratings of health care

• Patient-reported measures can correlate poorly with physiologic measures.

• Patients with the same clinical status or physiologic state may have different responses to the condition.

Norwegian Registry for Spine Surgery- Function

(Oswestry Disability Index - ODI) range 0-100 where 0= no disability

- Back and leg pain (numerical pain scale - NRS)

range 0-10 where 0= no pain

- Health-related quality of life(EQ-5D) 0=death and 1= perfect health

- Global score of outcome of surgery 7 point scale

Lumbar dics herniation:

Pain, loss of function and quality of life

Can we define success criteria for lumbar disc surgery?Estimates for substantial amount of change in core outcome measures.Solberg T. Acta Orthopaedica 2013; 84 (2):196-201

The cut-off values for success for the mean change scores were 20 (ODI), 2.5 (NRS back), 3.5 (NRS leg), and 0.30 (EQ-5D). The ODI and leg pain scale showed the best ability to discriminate success.

Aim: estimate cut-off values for success criteria for the Oswestry disability index, pain scale and HRQL

Summary

• Part of the complexity in evaluating quality of care is that different groups can have different reasons for measuring quality and hence different views on variables to choose and measurement criteria.

• Elements of quality:

• Clinical measures and patient-reported measures assess different aspects of quality

Thank you!

Tromsø