cap.org v. 1 gynecologic consensus conference working group 1: monitoring interpretive rates,...

cap.org v. 1

Gynecologic Consensus ConferenceWorking Group 1: Monitoring Interpretive Rates, Concordance of Interpretations, Turnaround Time June 4, 2011

• Joseph Tworek, MD (Senior Author)

• Karen M. Clary, MD (Chair)

• R. Marshall Austin, MD

• Diane Davis Davey, MD

• Sonya Naryshkin, MD

• Chiara Sugrue, SCT

• Beth Anne Chmara, CT(ASCP), (CAP Staff)

Working Group 1

© 2011 College of American Pathologists. All rights reserved. 2

Presented by Dr. Sonya Naryshkin

Monitoring Interpretive Rates


• Basis: Respondent data and professional opinion

• 93% of respondents monitor the total abnormal rate, including ASC and higher.

• 81% of respondents monitor rates for both the laboratory as a whole and also for individuals (Table 6).

• 64% of respondents believe monitoring of ASC-US is helpful.

• 27% of respondents believe some TBS categories may not be useful to monitor; however, less than 10% of respondents list any single TBS category as not at all useful to monitor (Table 8).

Consensus Statement #1: Monitoring of interpretive rates for all TBS categories is potentially useful, as each TBS category is clinically relevant.


Consensus Statement #1: Monitoring of interpretive rates for all TBS categories is potentially useful, as each TBS category is clinically relevant.

1. Do you agree with the consensus statement?

A. Yes 94.3%B. No 5.7%

2. Should standard categories of interpretive rates be monitored in all laboratories?

C. Yes 85.1%D. No 6.3%

3. Should each individual laboratory choose which interpretive rates to monitor?

E. Yes 20.7%F. No 79.3%

Voting


• Basis: Respondent Data and Personal Observation

• Data in Table 6 shows that strong majorities are currently monitoring all TBS categories for cytotechnologists and also for the laboratory as a whole.

Consensus Statement # 2: It is most useful to monitor interpretive rates for cytotechnologists individually and in comparison for the entire laboratory


Consensus Statement # 2: It is most useful to monitor interpretive rates for cytotechnologists individually AND in comparison for the entire laboratory.

4. Do you agree with the consensus statement?

A. Yes 100%B. No -

Voting


• Basis: Respondent Data and Personal Observation

• Only a third of respondent laboratories monitor TBS categories for pathologists, perhaps reflecting the more varied volume and more varied case mix of cases reviewed by individual pathologists.

• Refer to Table 6

Consensus Statement #3: It is currently unclear whether or not monitoring interpretive rates for individual pathologists beyond laboratory rates as a whole is useful.

© 2011 College of American Pathologists. All rights reserved.

8

Consensus Statement #3: It is currently unclear whether or not monitoring interpretive rates for individual pathologists beyond laboratory rates as a whole is useful.

5. Is monitoring interpretive rates of individual pathologists useful to you?

A. Yes 86%B. No 13%C. Other 1%

6. Is this an area that should be explored?

C. Yes 90%D. No 3%E. Other 6%

Voting


• Basis: Workgroup recommendation.

• Monitoring of “dangerous abnormals” as defined above may assist laboratories in assessing performance of rare but potentially more clinically significant abnormal results.

Consensus Statement #4: Consider monitoring combined interpretive rates of “dangerous abnormals,” defined as cancer, suspicious for cancer, HSIL, AGC, and ASC-H


Consensus Statement #4: Consider monitoring combined interpretive rates of “dangerous abnormals,” defined as cancer, suspicious for cancer, HSIL, AGC, and ASC-H.

7. Do you think that using the combined category “dangerous abnormals” could be useful?

A. Yes, in low volume/low prevalence laboratories only 16%

B. Yes, in any laboratory 35%C. No 43%D. Don’t Know 6%

Voting



• 65% of respondents say they are monitoring rates monthly.

• If the LIS does not readily support monthly monitoring, biannual monitoring could more conveniently mirror CLIA-required biannual performance review requirements.

• Monthly monitoring may be useful if there is a change in laboratory operations or personnel.

Consensus Statement #5: Monthly monitoring of interpretive rates may be useful


Consensus Statement #5: Monthly monitoring of interpretive rates may be useful, if possible.

8. Is monthly monitoring:

A. Too frequent 43%B. Not frequent enough 2%C. Just right 55%

Voting


• To share individual interpretive rate data laboratory-wide, de-identify individuals

• To privately provide personal feedback, compare overall laboratory data in comparison to individual statistics

• File/retain such data with other QA documents

• 58% of respondents believe it is helpful in a quality plan that cytotechnologists and pathologists have access to their personal interpretive rates in comparison with others in the laboratory

Consensus Statement #6: Providing Feedback of interpretive rates is important



• Helps to refine skills/cytologic criteria/identify areas for improvement/”self QA”

• Provides peer pressure to change when one is an outlier

• Provides data to make job of counseling easier when quality issues need to be addressed

Feedback on interpretive rates

Voting

Consensus Statement #6: Providing Feedback of interpretive rates is important.

9. Should individual interpretive statistics be provided to cytotechnologists and pathologists as feedback?

A. Yes, regularly88%B. No, not at all 1%C. Only as a part of scheduled employee reviews

11%


Presented by Dr. Diane Davis Davey

Concordance of Interpretations


Consensus Statement #7: Actively monitoring rates at which a pathologist UPGRADES a cytotechnologist interpretation prior to sign out may be a useful quality metric


Basis: Most laboratories monitor these metrics

• Changes from NILM to SIL+ are most critical

• Upgrades from ASC-US to HSIL also useful

• Upgrades from LSIL to HSIL and ASC-H to HSIL not as useful

Which upgrade rates are monitored for Cytotechnologists?

© 2011 College of American Pathologists. All rights reserved.

19

Consensus Statement #7: Actively monitor rates at which a pathologist UPGRADES cytotechnologist interpretations prior to sign out.

10. Do you:

A. Agree, NILM to SIL+ (negative to SIL or higher) 26%B. Agree, NILM to SIL+, also ASC-US to HSIL 26%C. Agree, A and B plus NILM to ASC-US 18%D. Agree, any upgrades to abnormal plus LSIL or ASC-H to

HSIL 29%E. Do not monitor upgrades 0%

Voting


• 62.5% of laboratories actively monitor this rate

• Downgrade of HSIL or LSIL to NILM is most important and utilized by >90% of laboratories

• It is expected that a portion of ASC-US cases will be downgraded to NILM by pathologist so this rate not as important

• Similarly downgrades of HSIL to ASC-H or LSIL has less clinical significance

Consensus Statement #8: Actively monitoring rates at which a pathologist DOWNGRADES cytotechnologist interpretations prior to sign out may be a useful quality metric.


If pathologist downgrades a cytotechnologist’s interpretation at the time of initial sign out, which rates are actively monitored per Cytotechnologist?


Consensus Statement #8: Actively monitor rates at which a pathologist DOWNGRADES cytotechnologist interpretations prior to sign out.

11. Do you:

A. Agree, HSIL+ or LSIL to NILM only 65%B. Agree, ASC-US+ (all abnormal) to NILM 10%C. Agree: all abnormal to NILM, and HSIL to ASC-H or

LSIL 15%D. Do not monitor downgrades 10%

Voting


• Consensus Statement #9: Show discrepancies of 2 degrees or more to a third person when possibleo Preferably third person should be blindedo NILM to HSIL, ASC-H, or Atypical glandular cellso This applies to both upgrades and downgradeso Survey: this is done by 61% of laboratories

• Small laboratories with only one technologist or pathologist may find this difficult

• Some LIS systems may not be able to track cytotechnologist interpretations prior to sign-out

Adjudicating discrepancies and Potential limitations


• Review case with original cytotechnologist

• Obtain HPV testing for some cases

• Obtain cytologic-histologic correlation later as applicable

• Consider limited upgrade to ASC-H instead of HSIL

• Consider limited downgrade to ASC-US in certain cases, recognizing uncertainty of lesions

Other suggestions for adjudication, especially in small laboratories


• 78% of pathologists in online survey show downgrades of HSIL to another person

• 54% show downgrades of atypical glandular cells and these lesions also problematic

• Consider impact on patient care, follow-up

• If a third person is NOT available:o Send a portion of cases involving upgrades to

HSIL+ or atypical glandular cells ( or similarly downgrades of HSIL/Atypical glandular) for consultation

Other suggestions for adjudication (continued)


Consensus Statement #9: Show discrepancies of 2 degrees or more to a third person when possible.

12.Do you:

A. Strongly agree 63%B. Agree with reservations 29%C. Disagree 8%

Voting


• Applies to both upgrades and downgrades

• Depends on nature of discrepancy, patient history or findings

• Any glandular discrepancy or one that impacts on patient receiving colposcopy is considered more significant

• Consider reviewing at least some cases with original cytotechnologist or another person

Resolving discrepancies of one degree


• Applies to both upgrades and downgrades

• Compare rates to historical laboratory trends

• Compare individual rates to laboratory average

• Document discrepancy rates as part of 6 month workload assessment. Establish baseline rates for each individual and laboratory so that trends can be recognized

Monitoring of discrepancies


• HSIL+, ASC-H, and atypical glandular processes are frequently reviewed by a 3rd person (online survey, most commonly at discretion of pathologist 62%)

• Results that represent potentially dangerous or high-grade processes

• Impact on patient management

Consensus Statement #10: Some cases benefit from review by a 3rd person even if not up/downgraded


Consensus Statement #10: Some cases benefit from review by a 3rd person even if not up/downgraded (squamous).

13. Which cases benefit from 3rd person review (squamous)?A. ASC-H and greater 20%

B. HSIL and greater 18%

C. Squamous cell carcinoma only 23%

D. Not necessary 39.44%

Voting


Consensus Statement #10: Some cases benefit from review by a 3rd person even if not up/downgraded (glandular).

14.Which cases benefit from 3rd person review (glandular)?

A. Both Atypical glandular and adenocarcinoma 55%

B. Adenocarcinoma only 15%

C. Not necessary 28%

D. Other 2%

Voting


• Endometrial cells in women ≥ 40years, glandular cells post-hysterectomy (36-44% in online survey)

• Herpes: 81%

• Unsatisfactory cases: 59%

• Rationale: Impact on management, promote interobserver reproducibility, diagnostically difficult areas

Consensus Statement #11: Some categories of cases benefit from routine review by 2nd person even if CLIA does not require confirmation by a pathologist


Consensus Statement #11: Some cases benefit from routine review by a 2nd person even if CLIA does not require confirmation by a pathologist.

15.Which cases benefit from routine review by 2nd person (cytotechnologist or pathologist) even if not required by CLIA?

A. Herpes 4%B. Endometrial cells in women > 40 - 0%C. Glandular cells in women post hysterectomy 3%D. B and C only (glandular processes) 16%E. All examples (A, B, C) 59%F. Not necessary routinely (only at the discretion of

the screener) 18%

Voting


Presented by Dr. Karen M. Clary

Turnaround Time (TAT)


Consensus Statement #12: Turnaround Time (TAT) should be monitored in Gynecologic Cytology

• Historically TAT has been associated with pressuring Cytotechnologists to increase productivity at expense of quality

• Customer Service issue: Result availability and timeliness are quality issues

• Questionable if TAT standards should apply to Pap tests (screening tests) as they do to other clinical laboratory tests

Monitor Turnaround Time (TAT) in Gynecologic Cytology

• TAT may serve as a quality metric in monitoring staffing needs and facilitating timely patient care

• An increase in TAT may signify a problem in laboratory workflow

• Use of TAT monitoring should never compromise the quality of the Pap test evaluation

• 59% of laboratories agree TAT is an effective quality metric (online survey)

Potential Limitations of TAT monitoring

• 32% report monitoring TAT negatively affects quality (online survey)

• Use of TAT:o May cause undue pressure on Cytotechnologists

to meet screening quotao May lead to increase in false negative rates and

screening errorso May lead to increase in ASCUS rates

Voting

Consensus Statement #12: Turnaround Time (TAT) should be monitored in Gynecologic Cytology.

16. Do you agree?

A. Yes 80%B. No 20%

Consensus Statement #13: We should NOT attempt to establish a universally acceptable TAT in Gynecologic Cytology

• Basis: o Not supported by Literatureo Not required as certification criteria for

laboratory inspections (CAP, ASC, HCFA)

• Survey results: Wide range of laboratory TAT: 1 to ≥ 7 days

• Median TAT: 3 business days

Refer to Table 9

Voting

Consensus statement #13: We should NOT attempt to establish a universally acceptable TAT in Gynecologic Cytology.

17.Do you agree?

A. Yes 90%B. No 10%

Consensus Statement #14: Individual laboratories should determine how to measure/define TAT

• Survey data:

• 57% defined starting point as date/time of accessioning in the prep area

• 89% defined ending point as date/time report electronically signed out

Refer to section of table 10

Voting

Consensus Statement #14: Individual laboratories should determine how to measure/define TAT.

18. Do you agree?

A. Yes 74%B. No 26%

Consensus Statement #15: Individual laboratories should determine the frequency of TAT monitoring

• Most common frequency of TAT monitoring is monthly


Voting

Consensus Statement #15: Individual laboratories should determine the frequency of monitoring TAT.

19. Do you agree?

A. Yes 99%B. No 1%

Consensus Statement #16: Individual laboratories should determine the metric used to measure TAT variance

• Most common metric used for TAT variance was percentile distribution within a certain TAT


Voting

Consensus Statement #16: Individual laboratories should determine the metric used to measure TAT variance.

20.Do you agree?

A. Yes 91%B. No 9%

As a result of the first round of presentation for Work Group 1, the following votes were added.

Additional Votes


72.Endometrial cells in a Pap test in a woman known to be postmenopausal should be designated differently than routine reporting of endometrial cells from a woman over 40 years old. o ASCCP Guidelines recommend endometrial

sampling or clinical/ultrasound assessment of these patients

o These women are at increased risk for atypical hyperplasia and endometrial cancer

Do you agree?A.Yes 58.18%B.No 40%C.Other 1.82%

Voting


73.Revised Consensus statement #5: Regular monitoring of interpretive rates may be useful and the individual laboratory should determine the frequency of monitoring.

Do you agree?

A. Yes 98.15%B. No 1.85%

Formerly voting question #8

Voting


74.Revised Consensus Statement #7: Actively monitor rates at which a pathologist UPGRADES cytotechnologist interpretations prior to sign out. Definition of upgrades should be determined by the laboratory.

Do you agree:

A. Yes 79.66%

B. No 15.25%

C. Other 3.39%

D. Other 1.69%

Formerly voting question #10.

Voting


75.Revised Consensus Statement #9: Laboratories should have policies about which categories of discrepancies should be reviewed by a third individual prior to sign out.

Do you agree:

A. Yes 73.68%

B. No 22.81%

C. Other 3.51%

Formerly question #s12, 13 and 14.

Voting


76.Revised Consensus Statement #11: Laboratories should have policies as to which cases benefit from review by a second person (cytotechnologist or pathologist), even if not required by CLIA.

These may include:– Unsatisfactory– Endometrial cells in women >40– Glandular cells in women post hysterectomy– Herpes

Do you agree:

A. Yes 90.74%

B. No 9.26%

Formerly voting question #15.

Voting


cap.org v. 1 gynecologic consensus conference working group 1: monitoring interpretive rates,...

Documents

individual pathologists

laboratory rates

consensus statement

tbs categories

individual laboratory

entire laboratory

monitoring of asc

single tbs category