clinical practice guideline: verification and validation
TRANSCRIPT
Clinical Practice Guideline:
Verification and Validation
Presenters:
Pat Connelly, PhD, CCC-A
&
Elizabeth A. Lynner, BC-HIS
The views expressed in presentations made
at International Hearing Society (IHS)
educational events are those of the speaker
and not necessarily of IHS. Presentations
at IHS events, or the presence of a speaker
at an IHS event, does not constitute an
endorsement of the speaker's views.
From the desk of
our Attorney
Incorporate CPG on V & V into daily practice
Verification
Use objective methods for fitting accuracy
Speech mapping
Insertion gain
Validation
Select the validation method based on the patient/clients needs
Measure the benefit, satisfaction and reduction of disability and/or handicap through use of hearing aids.
Discuss V & V research that support “best practices”
Use competency-based assessments to audit professional development
Learning Objectives
Clinical Practice Guidelines (CPG) -
Definition
"…statements that include recommendations,
intended to optimize patient care, that are informed
by a systematic review of evidence and an
assessment of the benefits and harms of
alternative care options“
Health and Medicine Division of the National Academies of Sciences,
Engineering, and Medicine (Institute of Medicine)
• Reduce inappropriate variation in practice
• Provide a more rational basis for referral
• Promote efficient use of resources
• Act as focus for quality control, including audit
• Provide a focus for continuing education
• Describe appropriate care based on the best available scientific evidence and broad consensus
• Highlight shortcomings of existing literature and suggest appropriate future research
Evidence-based
Practice
Provides accountability for healthcare delivery to
Oversight and regulatory agencies
Payers (insurance;
government programs; charities)
Consumers
Evidence-based
Practice
Evidence-based
Practice
Clinical Practice
Guidelines
Clinical Practice Guidelines
Summarize the evidence
Provide recommendations
Foster evidence-based practice and accountability
Improve outcomes by providing a better quality of care
Evidence-based
Practice
Clinical Practice
Guidelines
Clinical Practice
Guidelines
• their knowledge
• their experience
• patient preferences
• evidence-based practice
How should the
hearing
healthcare
professional
determine the
best intervention?
IHS’s Current
Clinical Practice Guidelines
Bridge-and-Brace
Technique for Patient
Safety
Verification and
Validation
These guiding principles do not exist in
isolation. Rather, they represent crucial
elements within the context of a
comprehensive treatment plan.
Clinical Practice Guidelines
Evidence-based Healthcare
Gaining ground quickly over the past few years
Motivated by clinicians, politicians and
management concerned about quality,
consistency and costs.
Evidence-based CPGs support improvements in
quality and consistency in healthcare.
Clinical Practice Guideline
Verification and Validation
Performance of verification and validation is essential to a
“best practices” approach to hearing instrument fittings.
Verification and validation improve fitting and rehabilitation
outcomes for patients/clients and these same patients/clients
have fewer post-fitting adjustment appointments.
It is the position of the International Hearing Society that
verification and validation must be performed to ensure that
the hearing instrument fitting has been individualized and
maximized to the patient’s/client’s needs.
Verification is an objective measurement of hearing
instrument performance in the ear.
• Verification should be accomplished using real ear
measures; specifically speech mapping or insertion gain
techniques.
• Verification ensures that speech is audible, optimal
speech intelligibility is achieved, and loud sounds are not
uncomfortably loud.
Clinical Practice Guideline Verification and Validation
Validation is an outcomes-based process which ensures that
the fitting optimizes the patient’s/client’s satisfaction and
perceived benefit.
• Validation is accomplished using tools such as:
o COSI, APHAB, and other validated questionnaires/tools
o Unaided and aided speech tests performed in soundfield
• Validation ensures the patient’s/client’s satisfaction is apparent
while using amplification, determines the benefit for
conversational speech, and/or establishes the degree to which the
hearing instrument wearer’s perceived handicap is reduced.
Clinical Practice Guideline Verification and Validation
Why are V & V important?
• Evidenced-based practice
• Standardize practice
• Encouraging higher standard of practice
• Improve professionalism
Verification &
Validation
Justification
is found in
research:
• Improved fitting/rehabilitation outcomes
• Fewer post-fitting adjustment appointments
• Fewer returns for credit
• Increased audibility of speech is correlated with hearing aid benefit and associated with increased use.
• When REMs not used, significantly greater decline in wearer satisfaction ratings one after fitting compared to fittings completed using REMs.
Verification
Objectives
• Ensure that speech is
audible
• Provide sufficient
acoustic access to
speech for optimal
intelligibility
• Maintain loud sounds as
comfortable
Verification
Based on
prescriptive
methods for
hearing aid
fitting
Prescriptive methods provide logical, reliable, consistent approaches to specifying
• Gain
• Frequency Response
• Output
for an individual’s fitting.
They provide evidence-based, average standards or targets that must match the fitting as measured in situ or in the wearer’s ear.
Verification
Best Practices
It is the position of the IHS
and the dictates of the best
practices doctrine that
every hearing instrument
fitting requires that pure
tone thresholds, including
inter-octave frequencies,
AND frequency-specific
UCLs be measured for
every patient/client.
Verification
Best Practices
Fitting verification must be
performed on the default
setting (Program 1) of each
hearing instrument. It is
best-practices that all user
memories be verified.
Verification Always practice appropriate sanitation
procedures for otoscopy and make sure that a new probe tube is used for every patient/client.
Always perform pre-testing calibration procedures as recommendation by your equipment manufacturer.
Always comply with the electroacoustical calibration standards required by your state, providence, or other jurisdiction.
Verification
Speech mapping Procedure
Verification
Rear Ear Insertion Gain Procedure
VERIFICATION
Real Ear Demo
Verification
When can
verification be
performed?
• At the initial fitting of new hearing instruments once programming is completed and any adjustments made based the wearer’s reports about listening preferences
• At the follow-up visit
• When any action or event (adjustment, repair, change in earpiece) can potentially impact frequency response and output
What is the Goal?
Treatment can be organized into three different areas:
1. Treatment Effectiveness
Do hearing aid improve audibility and speech understanding?
2. Treatment Efficiency Do certain hearing aids and fitting algorithms work better than others for improving audibility and speech understanding in different listening situations?
3. Treatment Effects
Does the use of well-fitted hearing aids improve the patient’s social and emotional well-being, and overall quality of life
Mueller & Taylor, 2011
VALIDATION
WHAT IS IT AND WHY SHOULD I USE IT?
Validation
• Is an outcomes-based process which ensures that the fitting optimizes the patient’s/client’s satisfaction and perceived benefit.
• Merriam-Webster
– To recognize, establish, or illustrate the worthiness or legitimacy of
Validate, 2013
Why Use Validation?
The MarkeTrak VIII study provided evidence that including V&V as a best practice results in:
• Reduced patient/client office visits
• Increased patient/client satisfaction with hearing instruments
“MarkeTrack VIII: Reducing,” 2011.
Patient/Client Office Visits
“MarkeTrack VIII: Reducing,” 2011.
Figure 1. The effects of V&V on office visits (2011).
The Need for Outcome Measures
• Healthcare is becoming consumer driven.
• The consumer decides what treatment is selected and when it is complete
• “Because today’s patients are, on average, more savvy and better informed than our grandparents, they want to know how much benefit they are receiving in everyday listening situations.” (Bentler et al. (2016), p. 438)
Benefit vs. Satisfaction
Benefit
The difference between unaided and aided measurements.
Satisfaction
Fulfillment of a need or want
Clinic vs. Real World
Clinic Measures
Tend to be objective measures
Unaided and aided speech tests performed in soundfield
Real World
Tend to be subjective
Validated questionnaires/tools such as COSI, APHAB, and others
CLINIC MEASURES
Unaided vs. Aided Measures
Word Recognition Testing (WRS)
Measures ability to hear speech in quiet
QuickSIN Testing
Measures ability to hear speech in noise
WRS
Set-up: • Patient/Client is placed in front of soundfield speaker at a 0⁰
azimuth to the speaker approximately 1 meter from the speaker.
• Audiometer set-up for SF WRS testing set to a normal conversational level (65dB SPL/45dB HL)
1 M
eter
WRS
Unaided:
• Choose the type of word list you would like to use.
• Run list at normal conversational level without hearing aids.
• Record patient’s/client’s responses
WRS Aided:
• Use the same type of word list.
• Run list at normal conversational level with hearing aids.
• Record patient’s/client’s responses
Benefit:
• The difference between the unaided and aided scores.
• Aided (92%) – Unaided (76%) = 16% improvement!
QuickSIN Why use QuickSIN?
“The primary complaint of hearing-impaired persons is difficulty hearing in background noise. The measurement of SNR loss (signal-to-noise ratio loss) is important because speech understanding in noise cannot be reliably predicted from the pure tone audiogram.” (Killion & Niquette, 2000).
QuickSIN Methodology:
“A list of six sentences with five key words per sentence is presented in four-talker babble noise. The sentences are presented at pre-recorded signal-to-noise ratios which decrease in 5dB steps from 25 (very easy) to 0 (extremely difficult). The SNR’s used are: 25, 20, 15, 10, 5, and 0, encompassing normal to severely impaired performance in noise.” (Etymotic Reasearch,2006, p. 4).
QuickSIN Set-up: When presenting the QuickSIN test via soundfield speaker, present it through one speaker only, with the subject seated facing the loudspeaker (0⁰ azimuth). Calibration: Using the 1-kHz calibration tone on Track 1, adjust the audiometer so that the VU meter reads “0.” Some audiometers have two VU meters, one for each channel. When presenting the test via loudspeaker, it is only necessary to set the VU meter for the channel being directed to the loudspeaker. When presenting the test via earphones, it may be necessary with some audiometers to adjust both VU meters. NOTE: Tracks 24-35 were recorded with speech on one channel and babble on the other. When using these tracks, calibrate both channels.
QuickSIN Presentation Level: For pure tone average (PTA) <45 dB HL, set the attenuator dial to 70 dB HL. For PTA of 50 dB HL or greater, set the attenuator dial to a level that is judged to be “loud, but OK.” The sound should be perceived as loud, but not uncomfortably loud.
Test Instructions: “Imagine that you are at a party. There will be a woman talking and several other talkers in the background. The woman’s voice is easy to hear at first, because her voice is louder than the others. Repeat each sentence the woman says. The background talkers will gradually become louder, making it difficult to understand the woman’s voice, but please guess and repeat as much of each sentence as possible.”
QuickSIN
Scoring: Five key words are scored in each sentence. The key words are underlined on the score sheets. One point is given for each key word repeated correctly. The number of correct words for each sentence should be written in the space provided at the end of the sentence and the total correct calculated for the list. SNR Loss is calculated for each list by using the formula: SNR Loss = 25.5 – Total Correct.
Note: for greater accuracy, two or more lists should be averaged.
QuickSIN Scoring:
25.5 - # Correct = SNR Loss 12 + 11 = 23 ÷ 2 = 11.5 25.5 – 11.5 = 14dB SNR Loss
QuickSIN
Unaided vs. Aided
• Run two or more lists without hearing aids.
• Run two or more lists with hearing aids.
• Difference between Aided and Unaided is Benefit.
Aided (6dB SNR Loss) – Unaided (14dB SNR Loss) = 8dB Improvement!
~OR~ we moved from a Moderate SNR Loss to a Mild SNR Loss
REAL WORLD MEASURES
Self Reports
Open Ended: Allow patient/client to choose their own outcome measures or desired improvement areas.
– Can be tailored to the true communication needs of the individual – Client Oriented Scale of Improvement (COSI)
Closed Ended: Patient/client completes questionnaire that uses predetermined areas of concern.
– Can be compared to normative data – Abbreviated Profile of Hearing Aid Benefit (APHAB) – Hearing Handicap Inventory for Adults (HHIA) / Hearing Handicap Inventory for the Elderly
(HHIE)
Muller & Taylor, 2011
COSI™
OPEN ENDED
COSI™
“The goal of the COSI is for the patient to target specific listening situations when the hearing aids are fitted, and to report the degree of benefit obtained after a few weeks of hearing aid use.” (Muller & Taylor, 2011)
The listening situations chosen should be ones that the patient/client is familiar with and not new ones.
Downside to open ended self reports is the inability to compare the results to normed data.
COSI™
Phase I: Identification of Specific Listening Situations
Patient/Client is asked to identify between 1 – 5 specific listening situations they would like to hear better in.
“If the COSI™ information is going to be quantified and analyzed according to listening situation, the listening category should also be recorded. Categorize each identified situation into one of the sixteen general categories listed on the COSI™ form.” (NAL Client, n.d.)
COSI™
Phase I: Identification of Specific Listening Situations
Step 1: Identify specific listening needs
Categorize if desired
COSI™
Phase I: Identification of Specific Listening Situations
Step 2: Have the
patient/client rank each situation in
order of importance
COSI™
Phase II: Assessment of Improvement and Final Listening Ability
“Your goal should be for the patient to rate the degree of change for all five situations “better” or “much better” compared to the unaided condition. If you don’t receive a “better” or “much better” you may need to spend more time counseling the patient or perhaps doing some tweaking, and giving the patient more time before re-measuring benefit on this scale. If the patient reports “better” or “much better” (2 or 3 categories of improvement relative to the unaided condition) you can pat yourself on the back and assume that you have just documented a “successful” fitting.” (Muller & Taylor, 2011, p. 351).
COSI™
Phase II: Assessment of Improvement and Final Listening Ability
At a subsequent visit you will have the patient/client rank their “Degree of Change” and/or “Final Ability (with hearing aid)”
You may wish to schedule a “exit appointment” and administer the “Final Ability” at that time.
According to Dillon, et al (1997), the second part of the assessment was administered at an exit appointment (approx. 5.7 weeks after fitting) and again at a 3 month follow-up.
COSI™
Phase II: Assessment of Improvement and Final Listening Ability
Access the Degree of
Change at a subsequent
appointment. (2-3 weeks after fitting)
Consider making
adjustments for any scores less than
“Better.”
COSI™
Phase II: Assessment of Improvement and Final Listening Ability
Access “Final Ability” at 3 Month Follow-
up.
Remember – “Most of the Time” and
“Above Average” in 2-3 categories is
considered a “Successful Fitting!”
APHAB
CLOSED ENDED
APHAB “The goal of the APHAB is to quantify the disability (percent of problems) caused by hearing loss, and the reduction of that disability that was then achieved with the use of hearing aids.” (Muller & Taylor, 2011, p.352-353).
• 24 Items
• 4 Subscales
• Ease of Communication (EC)
• Reverberation (RV)
• Background Noise (BN)
• Averseness to Sounds (AV)
Published Norms – Can compare to other patients/clients of similar demographics
APHAB
• Patient/client answers 24 questions from each of the 4 domains for how they hear without their hearing aids and how they hear with their hearing aids.
• A 7 category scale is used.
– There are two types of descriptors.
• Numerical
• Text
APHAB
• Can be administered at the time of the test appointment as a “needs assessment” with the patient/client answering the questions in the column for “Without my hearing aid.”
• Should be administered again at a post fitting follow-up appointment with the patient/client answering the questions in the “With my hearing aid.”
• The difference between the “without” and “with” questions will be the benefit of the hearing aid fitting.
APHAB
• Instructions
APHAB
Patient/Client then answers all 24 questions for
the desired condition.
Encourage them to think of
similar situations if the one they are being asked about does not
apply.
APHAB
Scoring Manually
APHAB Some programs have the ability to score:
APHAB
APHAB
APHAB
U = Unaided A = Aided
B = Benefit
APHAB
HHIE
CLOSED ENDED
HHIE
– Designed for those 65 and older
– Other forms are available for a screener
and for adults under 65
• Widely used tool for measuring self-reported
handicap
• Provides insights into the relationship between
hearing loss and psychological/emotional and
social/communication variables
HHIE
• 25 item questionnaire that quantifies the emotional (13 items) and social/situational (12 items) problems associated with hearing loss in older adults
• 3 response options – “No” 0 points, “Sometimes” 2 points, and “Yes” 4
points
• Scores range in percent, with higher values representing a greater perceived handicap
• Results are more reliable when the provider reads the questions to the member
HHIE
• Scoring: • Fill in the number of points for each questions,
“Yes” = 4, “Sometimes” = 2, “No” = 0
• Subtotal Emotional questions
• Subtotal Situational questions
• Add subtotals together
0-16: No Handicap
17-42: Mild to Moderate Handicap
> 43: Significant Handicap
Did We Accomplish the Goal?
• Verification and Validation helps us to determine if the goals have been met.
• Have we provided: • A Benefit? • A Reduction in Disability? • A Reduction in Handicap?
• Have we provided evidence based proof to the patient/client that their hearing aid device provides benefit and are they satisfied?
Remember what Lindsey E. Jorgensen said…
V&V is an not an
References ABBREVIATED PROFILE OF HEARING AID BENEFIT - Form B. (1994). Retrieved August 22, 2016, from
http://www.harlmemphis.org//index.php?cID=130 Bentler, R. A., Mueller, H. G., & Ricketts, T. (2016). Modern hearing aids: Verification, outcome measures, and follow-up.
San Diego, CA: Plural Publishing. Dillon, H., James, A., & Ginis, J. (1997, February 8). Client Oriented Scale of Improvement (COSI) and It's Relationship to
Several Other Measures of Benefit and Satisfaction Provided by Hearing Aids. Journal of the American Academy of Audiology, 8(1), 27-43. Retrieved August 22, 2016, from http://studentacademyofaudiology.com/sites/default/files/journal/JAAA_08_01_04.pdf
Etymotic Research. (2006). QuickSIN Speech-in-Noise Test (Version 1.3) [Pamphlet]. Elk Grove Village, IL: Etymotic Research.
Instructions for COSI Administration. (n.d.). Retrieved August 22, 2016, from http://nal.gov.au/pdf/COSI-administration-instructions.pdf
Jorgensen, L. E. (2016). Verification and validation of hearing aids: Opportunity not an obstacle. Journal of Otology, 11(2), 57-62. doi:10.1016/j.joto.2016.05.001
Killion, M, Niquette, P (2000). What Can the Pure-Tone Audiogram Tell Us About A Patient’s SNR Loss? The Hearing Journal, 53 (3): 46-53
MarkeTrack VIII: Reducing Patient Vistits Through Verification & Validation. (2011, June 1). Retrieved from http://www.hearingreview.com/2011/06/marketrak-viii-reducing-patient-visits-through-verification-amp-validation/
Mueller, H. G., & Taylor, B. (2011). Fitting and Dispensing Hearing Aids. San Diego, CA: Plural Publishing. NAL Client Oriented Scale of Improvement. (n.d.). Retrieved August 22, 2016, from http://nal.gov.au/pdf/COSI-
Questionnaire.pdf Validate. (2013). In Merriam-Webster Dictionary for Apple iOS (Version 3.5) [Mobile application software]. Retrieved
from http://itunes.apple.com.