validation of healthcare databases - · pdf filevalidation of healthcare databases aldana...
TRANSCRIPT
![Page 1: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/1.jpg)
Validation of Healthcare
DatabasesALDANA ROSSO, PH.D
LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017
![Page 2: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/2.jpg)
Example: Patients with Hearth Failure
• We want to study all-cause mortality in patients that
suffered hearth failure.
• How do we do it?
![Page 3: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/3.jpg)
Definition of Population
All patients that have HF
![Page 4: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/4.jpg)
ICD10 code for Hearth Failure
![Page 5: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/5.jpg)
Definition of Population
Patients with ICD10 I50*
All patients that have HF
![Page 6: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/6.jpg)
Definition of Population
Patients with ICD10 I50*
All patients that have HF
Patients fulfilling the HF
register definition
![Page 7: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/7.jpg)
Definition of Population: Completeness
Patients with ICD10 I50*
All patients that have HF
Patients fulfilling the HF
register definitionRegistered
patients
![Page 8: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/8.jpg)
Definition of Population
• We aware that your study population is different from your
population of interest. Generalizability of results??
![Page 9: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/9.jpg)
Example: Swedish HF Register
![Page 10: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/10.jpg)
Example: Swedish HF Register
![Page 11: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/11.jpg)
Difference in All-Cause Mortality
![Page 12: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/12.jpg)
Problem: Selection Bias
![Page 13: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/13.jpg)
What you Need to Know about Your
Study Population
• How the population in the register is defined.
• How many patients are registered (completeness). This is
determined by comparison with other registers.
• Which clinics are reporting to the register and why?
• This is needed to ”estimate” selection bias in your study.
![Page 14: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/14.jpg)
Example: Spanish National Acute Coronary
Syndrome register
![Page 15: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/15.jpg)
Example: Spanish National Acute Coronary
Syndrome register
• Audit of the Spanish national acute coronary syndrome register [Ferreira-Gonzalez et al, Circulation: Cardiovascular Quality and Outcomes,2009; 2: 540-547].
• They compared enrolled patients with those that were not enrolled for some participating hospitals (17 of 50).
• Missed patients were of higher risk and received less recommended therapies than the included patients. In-hospital mortality was almost 3 times higher in the missed population.
![Page 16: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/16.jpg)
http://www.thelovesensei.com/perfect-for-you-doesnt-necessarily-translate-to-perfect-human-being/
![Page 17: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/17.jpg)
Research using Healthcare Databases
Implies…
1. Wrong data: misclassification
2. Missing data
• REMEMBER: you are using data for research that was
created for other purposes!
![Page 18: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/18.jpg)
Example: Misclassification
Intensive Care Register
http://portal.icuregswe.org/Rapport.aspx
![Page 19: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/19.jpg)
Example: Misclassification
Intensive Care Register
Diagnoser Antal
Z04.9Undersökning och observation av icke
specificerat skäl 3200
J96.9 Respiratorisk insufficiens, ospecificerad 2000
I46.9 Hjärtstillestånd, ospecificerad 1500
R57.2 Septisk chock 1500
R65.1Systemiskt inflammatoriskt svarssyndrom
[SIRS] av infektiöst ursprung med organsvikt 1500
T07.9 Icke specificerade multipla skador 1400
R56.8 Andra och icke specificerade kramper 1400
K92.2 Gastrointestinal blödning, ospecificerad 1200
J15.9 Bakteriell pneumoni, ospecificerad 1200
http://portal.icuregswe.org/Rapport.aspx
![Page 20: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/20.jpg)
Implications of Misclassification
• It affects the statistical analyses, just accept it!
• How depends on how bad the misclassification is and the
reason for misclassfication => Sensitivity analysis.
![Page 21: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/21.jpg)
Exempel: IVA-diagnoser
Diagnoser Antal
Z04.9 3200
J96.9 2000
I46.9 1500
R57.2 1500
R65.1 1500
T07.9 1400
R56.8 1400
K92.2 1200
J15.9 1200
http://portal.icuregswe.org/Rapport.aspx
Andel felaktiga
diagnoser som
blev ”Z04.9”
Andel korrekta
ranking
2 % 4 % ( 2 % - 10 %)
5 % 5 % ( 2 % - 12 %)
![Page 22: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/22.jpg)
Implication of Misigness
• It depends on why the data is missing:
– missing at random: problematic from a power perspective
but it doesn’t bias the results.
– missing not at random: bias + power problems. That’s
why you need to know which clinics are reporting and
why!
![Page 23: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/23.jpg)
Consequences of Statistical Analysis with
Missing Data
• Some calculations are more sensitive than others.
• Ranking is specially bad!
![Page 24: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/24.jpg)
Example: Calculations with Missing
Data
• Monte Carlo simulation: a register with 20 000 primary
operations and 3 hospitals.
• 5% of those patients have a reoperation.
• How does the percentage of missing values affect the
percentage of reoperation?
![Page 25: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/25.jpg)
Example: Calculations with Missing
data
• 20000 patients, 5 % have a reoperation.
• 5 % reoperations missing at random.
Hospital Proportion
Reoperation
Proportion
with missing
data
95 % CI
1 0.048 0.046 (0.045,0.047)
2 0.050 0.048 (0.047,0.049)
3 0.052 0.049 (0.048,0.050)
![Page 26: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/26.jpg)
Example: Calculations with Missing Data:
Which clinic is reoperating more patients?
• 20000 patients, 5 % have a reoperation.
Percentage
missing data
Proportion
correct
ranking
5 % 95 %
7 % 87 %
![Page 27: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/27.jpg)
Example: Swedish Knee Arthroplasty
register
![Page 28: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/28.jpg)
Example: Neovascular Age Related
Macular Degeneration (AMD)
Leading cause of vision loss among
people age 50 and older.
Dry AMD: gradual breakdown of the
light-sensitive cells in the macula.
Neovascular (wet)AMD: abnormal
blood vessels grow underneath the
retina, which can leak fluid and
blood, which may lead to swelling
and damage of the macula.
https://nei.nih.gov/health/maculardegen/armd_facts
![Page 29: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/29.jpg)
nAMDNormal fundus
Treatment: Anti VEGF Injections
![Page 30: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/30.jpg)
Treatment: Anti-VEGF Injections
VEGF: Vascular endothelial growth factor
Aflibercept: Eylea, Bayer
Ranibizumab: Lucentis, Genentech
Bevacizumab: Avastin, Genentech
![Page 31: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/31.jpg)
Visual Acuity
”Lowest acceptable” :
20/70: 60 ETDRS letters
![Page 32: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/32.jpg)
The Swedish Macula Register
• A national register for treatment of neovascular AMD.
• 80 % coverage.
• National results for AMD treatment concerning age, sex, type of lesion,
treatment frequencies, and follow-up visits
• Medical outcome: distance visual acuity, near visual acuity and adverse
events.
• Analyze and compare different treatments and their outcome.
• Validation: Limited study gave information about errors in the database.
![Page 33: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/33.jpg)
Treatment Compliance
• Known important factors to succeed with treatment:
– Age
– Good VA at baseline
• Patients are active in the register about 1.5 years.
• Why do patients not continue with treatment and/or
control visits?
![Page 34: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/34.jpg)
Definition Treatment Termination
During Year 1
• Several possible definitions:
A. Patients that did not have a control visit at year 1 and
don’t have registered visits since the last visit for at least
4 months.
B. Patients terminated the treatment due to known reasons
+ patients without control visit.
C. Patients don’t have a 1 year control visit and the latest
registered VA was good.
![Page 35: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/35.jpg)
Statistical Methods
• Objective: calculate risk of early treatment termination for all causes (outcome B).
• Model: Poisson regression to calculate risk. OBS: It is preferable to use Logistic Regression and then calculate the risk with a macro with Stata but not with multiple imputation.
• Model Validation:
– Comparison predicted/observed probabilities
– Deviance residuals
– Comparison with logistic regression
– Other ideas that work with clustered data?
![Page 36: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/36.jpg)
Missing Data
• Missing values for VA. I used MI to calculate VA at
baseline for the treated (4 eyes) and fellow eyes (88 of
989 eyes, 9 %).
• Difficult to known whether there are missing visits, around
20% of missing patients compared to the National
Patients register.
• Errors in the database: VA has approximately 5 % wrong
registered values.
• MC simulations with 20 % missing visits and 5 %
changed VA values.
![Page 37: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/37.jpg)
Take Home Message
• We aware that your study population is different from your
population of interest. Check:
– How the population in the register is defined.
– How many patients are registered (completeness).
This is determined by comparison with other registers.
– Which clinics are reporting to the register and why?
– Missing values and misclassification: How much? Is it
possible to compare with other registers?
![Page 38: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/38.jpg)
![Page 39: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/39.jpg)
Error Sources in National Quality Registries
1. Before registration
2. During registration
3. After registration
![Page 40: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/40.jpg)
Error Sources in Healthcare Databases
1. Before registration:
– wrong data is registered in the journal.
– the patient is not enrolled in the register.
– It is very difficult to estimate the error proportion of this
type of error.
![Page 41: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/41.jpg)
Error Sources in Healthcare Databases
2. During registration:
– misinterpretation and inaccurate typing: wrong value,
wrong calculation, wrong alternative.
– incomplete data.
• Misinterpretation and inaccurate typing: 5 %.*
• Missing data: 3 %.*
*Defining and improving data quality in medical registries: A literature review, case study and generic framwork.
Arts D.G.T. et al, J Am Med Inform Assoc, 2002 (9) 600.
![Page 42: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/42.jpg)
Healthcare Databases
3. After registration:
– programming errors.
– Communication problems between different databases.
• They are very difficult to find and have severe consequences.
*Defining and improving data quality in medical registries: A literature review, case study and generic framwork.
Arts D.G.T. et al, J Am Med Inform Assoc, 2002 (9) 600.
![Page 43: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/43.jpg)
Is automatic registration the solution?
![Page 44: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/44.jpg)
Dutch National Intensive Care Evaluation
Register (NICE)
*Defining and improving data quality in medical registries: A literature review, case study and generic framwork.
Arts D.G.T. et al, J Am Med Inform Assoc, 2002 (9) 600.
NICE contains data from
patients who have been
admitted to Dutch
intensive care units and
provides insight into the
effectiveness and
efficiency of Dutch
intensive care.
![Page 45: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/45.jpg)
Validation of Healthcare Databases
• Contact a statistician before startning the project!
![Page 46: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/46.jpg)
Validation Using Sampling Theory
Principle: we only take a sample of some patients and compare
their journals to the registered data. We extend this information
to all the patients in the register.
![Page 47: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/47.jpg)
Validation of Healthcare Databases
References:
http://www.scb.se/Upload/NSM2016/theme4/C_3_Aldana_R
osso.pdf
![Page 48: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/48.jpg)
Simple Random Sampling
1. Select randomly some
patients.
2. Estimate the proportion
of incorrect data.
3. Extrapolate to the
register.
Register
![Page 49: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/49.jpg)
Simple Random Sampling
• Usually without replacement.
• It is easy to program.
• It may give a sample that doesn’t represent the register
very well.
• “It may require more patients than other sampling
techniques”.
• Expensive (transportation cost).
![Page 50: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/50.jpg)
Stratified Random Sampling
1. Divide the register in strata (e.g. hospitals, regions, etc.).
2. Within each strata, select some patients.
3. Estimate the proportion of incorrect data.
4. Extrapolate to the register.
Register
![Page 51: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/51.jpg)
Stratified Random Sampling
• It is possible to estimate the proportion of error for each
stratum (hospitals).
• It requires the participation in the validation of all the strata.
• The patients within the stratum can be selected in several
ways:
– a fixed amount of patients are selected randomly within
each stratum.
– selection of the same percentage as the stratum in the
frame.
![Page 52: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/52.jpg)
Stratified Random Sampling
• At least as effective as SRS for the same sample size.
• If information about the error distribution is known, the
design can be improved.
• It gives a more representative sample.
• Expensive (transportation cost).
![Page 53: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/53.jpg)
Stratified Random Sampling
register
1
2
3
4register
1
2
3
4
Efficient Stratification Non-efficient
Stratification
![Page 54: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/54.jpg)
Cluster Sampling
1. Divide the register in
clusters (e.g. hospitals,
regions, ect.).
2. Select some clusters.
3. Within each cluster, select
some patients.
4. Estimate the proportion of
incorrect data.
5. Extrapolate to the register.
Register
![Page 55: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/55.jpg)
Cluster Sampling
• Multistage with different sampling weights. For example:
– Level 1: cluster region.
– Level 2: cluster hospital.
– Level 3: sampling units patients.
• It can be used in combination with stratification.
![Page 56: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/56.jpg)
Cluster Sampling
• Lower transportation cost.
• Only some hospitals are represented in the validation.
• It requires more patients than SRS to achieve the same
precision.
![Page 57: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/57.jpg)
Cluster Sampling
register
1
2
3
4register
1
2
3
4
Non-efficient cluster Efficient cluster
![Page 58: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/58.jpg)
Another Issue…
• The patient actually had the diagnostic, rigth???
![Page 59: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/59.jpg)
How estimate the Proportion of
Correctly Diagnoses Patients
• It is done in a similar manner but with an expert group. It
is called ”Adjudication”.
• Usually reported as ”positive predicted value”. For
example in the HF article it was around 95 %.
![Page 60: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/60.jpg)
What happens after Validation?
• Ethical and legal viewpoints: We should correct the data
(Sweden).
• Bias concerns?
• Information about the data quality and the missing data in
the publications?
![Page 61: Validation of Healthcare Databases - · PDF fileValidation of Healthcare Databases ALDANA ROSSO, PH.D LUND UNIVERSITY AND SKÅNE UNIVERSITY HOSPITAL. 28 APRIL 2017](https://reader033.vdocuments.mx/reader033/viewer/2022051718/5a70eac87f8b9a9d538c6e97/html5/thumbnails/61.jpg)