105_vi_jc.ppt
DESCRIPTION
TRANSCRIPT
Go with the flow: optimizing voice recognition to streamline
workflow
Go with the flow: optimizing voice recognition to streamline
workflow
Jeffrey Chenoweth MDSaint Louis VAMC
Kim Wilson MDTucson VAMC
Jeffrey Chenoweth MDSaint Louis VAMC
Kim Wilson MDTucson VAMC
Voice recognition to streamline workflow
• Jeffrey Chenoweth MD– Saint Louis VAMC
• Kim Wilson MD– Tucson VAMC
• Jeffrey Chenoweth MD– Saint Louis VAMC
• Kim Wilson MD– Tucson VAMC
Voice recognition to streamline workflow
• Driving forces behind VR
• Pushback – VR controversy
• VR development• Case study – VR
implementation• PACS setup and
reporting with VR– Kim Wilson MD
• Practical points for improving Radiologist workflow
• Future development of VR
• Driving forces behind VR
• Pushback – VR controversy
• VR development• Case study – VR
implementation• PACS setup and
reporting with VR– Kim Wilson MD
• Practical points for improving Radiologist workflow
• Future development of VR
Radiology reporting
• Basics unchanged in last century
• Product not images but report
• Communication to improve patient care
• Penultimate step in Radiology process
• Final step -- clinician action
• Basics unchanged in last century
• Product not images but report
• Communication to improve patient care
• Penultimate step in Radiology process
• Final step -- clinician action
If goals of reporting are unchanged why do we need VR
now?
Why VR?
• Absence of skilled transcriptionists?
• Transcription cost?– Probably not
• Absence of skilled transcriptionists?
• Transcription cost?– Probably not
Why VR?
• Improved report turnaround time
• Fewer report errors
• Improved report turnaround time
• Fewer report errors
Why VR?
• Transcription turnaround time
– Cassette tapes: week – 10 days
– Digital dictation: hours – 3 days
– VR: minutes• Decreases calls for
preliminary read• Clinicians expect
immediate report availability
• Transcription turnaround time
– Cassette tapes: week – 10 days
– Digital dictation: hours – 3 days
– VR: minutes• Decreases calls for
preliminary read• Clinicians expect
immediate report availability
Why VR?
• Improved report turnaround time
• Improved patient care
• Makes the Radiology report relevant
• Improved report turnaround time
• Improved patient care
• Makes the Radiology report relevant
Why VR?
• Problems with the traditional report correction editing process– Outside normal Radiologist workflow– Disruptive– Time consuming
• Problems with the traditional report correction editing process– Outside normal Radiologist workflow– Disruptive– Time consuming
Why VR?
• Error prone traditional report correction – editing process
– Time lag forgetfulness• Grammar checking vs.
content errors– Right – left errors– Date errors– DHCP blue screen daze
• After 50 + reports, how closely are you reading the report?
• Error prone traditional report correction – editing process
– Time lag forgetfulness• Grammar checking vs.
content errors– Right – left errors– Date errors– DHCP blue screen daze
• After 50 + reports, how closely are you reading the report?
Why VR?
• Report completion while image is in front of Radiologist
• Immediate error correction
• Once you’re done, you’re done
• Immediate report availability
• Report completion while image is in front of Radiologist
• Immediate error correction
• Once you’re done, you’re done
• Immediate report availability
VR controversy
VR controversy – Radiologist’s view
• Increased dictation time
• Increased error rate vs. good transcriptionist
• Removes focus on images
• Increased dictation time
• Increased error rate vs. good transcriptionist
• Removes focus on images
VR controversy – accuracy rate
• Is 95% acceptable?
• 90% of all reports have errors prior to sign off
• 10 % of reports have errors with transcriptionists
– J Digit Imaging Jun 2007
• Is 95% acceptable?
• 90% of all reports have errors prior to sign off
• 10 % of reports have errors with transcriptionists
– J Digit Imaging Jun 2007
VR economically justified?
• Decreased Radiologist productivity
– 50% longer dictation time– 24% shorter reports
– J Digit Imaging Jun 2007
• Decreased Radiologist productivity
– 50% longer dictation time– 24% shorter reports
– J Digit Imaging Jun 2007
VR – economically justified?
• Replacing lower paid transcriptionists with highly paid physicians
– Greater Radiologist productivity transcriptionists more cost effective than VR
– “… which course of action makes the most economic sense… is not always obvious.”
– JACR 2007; 4: 890
• Replacing lower paid transcriptionists with highly paid physicians
– Greater Radiologist productivity transcriptionists more cost effective than VR
– “… which course of action makes the most economic sense… is not always obvious.”
– JACR 2007; 4: 890
VR – two decades of controversy
• “Has considerable potential in the future… at present has limited function and definitely needs more technical improvement.”
– Radiology Nov 1988; 169: 580
• “… voice recognition systems are currently not ready for prime time.”
– JACR 2007; 4: 667
• “Has considerable potential in the future… at present has limited function and definitely needs more technical improvement.”
– Radiology Nov 1988; 169: 580
• “… voice recognition systems are currently not ready for prime time.”
– JACR 2007; 4: 667
VR – two decades of controversy
• “Speech recognition systems are used today in more than 1,000 radiology departments and are experiencing a growth rate typical of modern enabling technology.”
– JACR 2007; 4:670
• “Speech recognition systems are used today in more than 1,000 radiology departments and are experiencing a growth rate typical of modern enabling technology.”
– JACR 2007; 4:670
History of VR – a quarter century + of progress despite persistent
controversy
Evolution of Radiology reports
• Paper reports Electronic reports• Paper reports Electronic reports
Transcriptionist model – 1
• Tapes– Batch transcription– Batch correction,
signature
• Tapes– Batch transcription– Batch correction,
signature
Transcriptionist model – 2
• Digital transcription pool– In-line transcription– Batch correction,
signature
• Digital transcription pool– In-line transcription– Batch correction,
signature
Computer data acquisition systems
• Mark-sense forms
• GE RAPORT– AJR 1977; 128: 825
• Mark-sense forms
• GE RAPORT– AJR 1977; 128: 825
Computer data acquisition systems
• Microcomputers – CLIP Harvard
– Numeric codes for reporting– Radiology 1979; 133: 349
– Recognition of spoken numeric codes– Radiology 1981; 138: 585
• Microcomputers – CLIP Harvard
– Numeric codes for reporting– Radiology 1979; 133: 349
– Recognition of spoken numeric codes– Radiology 1981; 138: 585
True VR – Kurzweil system 1987
• Reported by several New England hospitals (including Boston VAMC)
• 1,000 word lexicon
• 5 sections by anatomy or subspecialty
– Radiology 1987; 164: 569.
• Reported by several New England hospitals (including Boston VAMC)
• 1,000 word lexicon
• 5 sections by anatomy or subspecialty
– Radiology 1987; 164: 569.
True VR – Kurzweil system 1987
• Able to dictate a report 88% of the time– 12% beyond scope of lexicon
• Use of macros
• Dictation time 20% longer
• Able to dictate a report 88% of the time– 12% beyond scope of lexicon
• Use of macros
• Dictation time 20% longer
True VR – Kurzweil system 1987
• Drawbacks
– Time and attention diverted from film analysis towards interaction with a monitor
– Increased dictation time proportional to degree of abnormality on film
– Problems with background noise– Problems with repeated interruptions– “Has considerable potential in the future… at
present has limited function and definitely needs more technical improvement.”
» Radiology Nov 1988; 169: 580
• Drawbacks
– Time and attention diverted from film analysis towards interaction with a monitor
– Increased dictation time proportional to degree of abnormality on film
– Problems with background noise– Problems with repeated interruptions– “Has considerable potential in the future… at
present has limited function and definitely needs more technical improvement.”
» Radiology Nov 1988; 169: 580
VR – state of the art 1999
– Error rate 30%
– Misrecognition of words
– Increased dictation timeRadioGraphics 1999; 19: 2.
– Error rate 30%
– Misrecognition of words
– Increased dictation timeRadioGraphics 1999; 19: 2.
VR – today
• Web architecture• Integration with
PACS– Improved efficiency– Decreased errors
• Improved recognition rates
• Decreased turnaround time
• Web architecture• Integration with
PACS– Improved efficiency– Decreased errors
• Improved recognition rates
• Decreased turnaround time
VR case study: Saint Louis VAMC
VR drivers
• PACS implementation– Fewer lost films – More reports
required
• PACS implementation– Fewer lost films – More reports
required
VR drivers
• CPRS implementation
• Universal availability of patient chart
• Clinical demand for faster reports
• CPRS implementation
• Universal availability of patient chart
• Clinical demand for faster reports
VR drivers
• Problem of preliminary reports– Clinical demand– Error correction– Legal issues
• Problem of preliminary reports– Clinical demand– Error correction– Legal issues
VR drivers
• Transcription problems– New contractor (low
bidder)– Cut and paste
errors– Variable
transcriptionist quality
• Transcription problems– New contractor (low
bidder)– Cut and paste
errors– Variable
transcriptionist quality
VR drivers
• Turn around time mandate– 90% completion in 48 hours
successful• 95% completion in 48 hours excellent
– Actual far less
• Turn around time mandate– 90% completion in 48 hours
successful• 95% completion in 48 hours excellent
– Actual far less
Analysis of options
• Hire more Radiologists– Full-time– Part-time
• Retired Radiologists• Fellows
– Recruiting difficulties
• Pay• Vacation
• Hire more Radiologists– Full-time– Part-time
• Retired Radiologists• Fellows
– Recruiting difficulties
• Pay• Vacation
Analysis of options
• Improve efficiency of reporting cycle
VR
• Improve efficiency of reporting cycle
VR
Proposal for VR system
• Strong administration support
• Support for VISN-wide solution– Some centers opted out
• Strong administration support
• Support for VISN-wide solution– Some centers opted out
System evaluation
• Radiologist input• Administration
– ADPAC– PACS coordinator– IT
• Literature review
• Radiologist input• Administration
– ADPAC– PACS coordinator– IT
• Literature review
System evaluation
• Vendor demonstrations– Radiologist trials
• Evaluation of administrator functions
• Vendor demonstrations– Radiologist trials
• Evaluation of administrator functions
Survey existing users
• Most sites only have experience with one system
• Hard to get good comparisons
• Your mileage may vary– Differences in
technical, administrative support for system
• Most sites only have experience with one system
• Hard to get good comparisons
• Your mileage may vary– Differences in
technical, administrative support for system
License issues
• Per unique user• Per workstation• Simultaneous users
vs. individual user
• Per unique user• Per workstation• Simultaneous users
vs. individual user
Vendor recommendation and selection
Planning
• Documentation review
• Site planning
• Documentation review
• Site planning
Results
• Report turnaround 90 – 95 % within 48 hours
• Cost savings
• Report turnaround 90 – 95 % within 48 hours
• Cost savings
VR implementation: lessons learned
• Plan, plan, plan• Plan, plan, plan
Lessons learned – project team
• Identify members– PACS administrator– Transcription
administrator– Editors– IT– Radiologist
• Dedication essential• Time consuming• Work closely with vendor• Read documentation
closely
• Identify members– PACS administrator– Transcription
administrator– Editors– IT– Radiologist
• Dedication essential• Time consuming• Work closely with vendor• Read documentation
closely
Lessons learned – conference calls
• Weekly calls
• Need everyone involved– IT– Administrators– Editors– Radiologist– Vendor
• Weekly calls
• Need everyone involved– IT– Administrators– Editors– Radiologist– Vendor
Lessons learned – conference calls
• Write questions in advance
• Keep minutes– Serves as a
resource– Complex project,
can’t remember everything
– Document to prevent misunderstandings
• Write questions in advance
• Keep minutes– Serves as a
resource– Complex project,
can’t remember everything
– Document to prevent misunderstandings
Lessons learned – installation issues
• Administrator training critical
• Get administrator manuals before vendor rep shows up
• Write questions in advance
• Take notes
• Administrator training critical
• Get administrator manuals before vendor rep shows up
• Write questions in advance
• Take notes
Lessons learned – test, test, test
• Test everything – don’t even think of implementation until this is done
• Vendor supplied checklist
• Test everything – don’t even think of implementation until this is done
• Vendor supplied checklist
Lessons learned – test, test, test
• Test system and test accounts
• Register procedure names and CPT codes
• Enter orders into Vista– Check request entry
into VR system
• Test system and test accounts
• Register procedure names and CPT codes
• Enter orders into Vista– Check request entry
into VR system
Lessons learned – test, test, test
• Dictate test reports– Check for proper
upload– Test addendums
and corrections– Input every type of
diagnostic code– Check parent and
descendants
• Dictate test reports– Check for proper
upload– Test addendums
and corrections– Input every type of
diagnostic code– Check parent and
descendants
Lessons learned – test, test, test
• Change orders• Minimum of 100 test patients• Test every Radiologist
– Include residents• Test sending to editor• Test telephony
• Change orders• Minimum of 100 test patients• Test every Radiologist
– Include residents• Test sending to editor• Test telephony
Lessons learned – Radiologist champion
• Change resistance• Promote system,
convince others that this is way to go
• Upfront buy-in from Radiologists
• Must see as improving patient care– vs. mandate from
administration
• Change resistance• Promote system,
convince others that this is way to go
• Upfront buy-in from Radiologists
• Must see as improving patient care– vs. mandate from
administration
Lessons learned – Radiologist champion
• Overcome objections– “I’m a physician not a transcriptionist!”
• Help others as one professional to another– Keep people going thru rough spots
• Need close communication with remainder of implementation team– Get feedback
• Overcome objections– “I’m a physician not a transcriptionist!”
• Help others as one professional to another– Keep people going thru rough spots
• Need close communication with remainder of implementation team– Get feedback
Lessons learned – Radiologist training
• Radiologists that have problems generally did not get good training
• Radiologists that have problems generally did not get good training
Lessons learned – Radiologist training
• Must have training schedule for every Radiologist
• Everyone has dedicated blocks for training, including follow-up– Minimum 4 hours with
trainer• Some may need more
attention
– Follow-up session
• Must have training schedule for every Radiologist
• Everyone has dedicated blocks for training, including follow-up– Minimum 4 hours with
trainer• Some may need more
attention
– Follow-up session
Lessons learned – Radiologist training
• Once trained, go cold turkey• Continuing support• Dealing with non-native English
speakers• Dealing with poor dictation styles• Refresher training
• Once trained, go cold turkey• Continuing support• Dealing with non-native English
speakers• Dealing with poor dictation styles• Refresher training
Lessons learned – site trainer training
• Must learn to train new users
• Critical if residents involver
• Individual training• Sit in on user
training sessions
• Must learn to train new users
• Critical if residents involver
• Individual training• Sit in on user
training sessions
Lessons learned – continual QC
• Continual effort and vital for long-term success
• Test plan• Test telephony• Dummy orders uploading
• Continual effort and vital for long-term success
• Test plan• Test telephony• Dummy orders uploading
Lessons learned – continual QC
• Pull real reports and monitor for errors
• Intervention if needed
• Retraining of dictator
• Rebuild voice model
• Pull real reports and monitor for errors
• Intervention if needed
• Retraining of dictator
• Rebuild voice model
Lessons learned – continual QC
• Look in CPRS– Report text ok– E-signature ok– Diagnostic codes
• Look in CPRS– Report text ok– E-signature ok– Diagnostic codes
Lessons learned – continual QC
• Monitor continually– Uploads– Orphan dictations– Exams without
reports
• Monitor continually– Uploads– Orphan dictations– Exams without
reports
Lessons learned – continual QC
• Need support contract
• Keep contacts handy
• Know who to call
• Need support contract
• Keep contacts handy
• Know who to call
Lessons learned – trouble log
• Take notes for every trouble call to vendor
• Resource to fix problems on your own– Record
• Day• Ticket #• Who spoke to• Problem• How it was resolved• Note recurring problem
• Take notes for every trouble call to vendor
• Resource to fix problems on your own– Record
• Day• Ticket #• Who spoke to• Problem• How it was resolved• Note recurring problem
Lessons learned – backup plan
• Backup VR server?
• Utilize another transcription contract?
• Other medical center?
• Backup VR server?
• Utilize another transcription contract?
• Other medical center?
PACS setup and reporting with VR
• Kim Wilson MD– Tucson VAMC
• Kim Wilson MD– Tucson VAMC
Radiologist workflow: practical points
Goals
• Increase dictation efficiency
• Maximize eyes on image
• Increase dictation efficiency
• Maximize eyes on image
Transcription models – read, edit, done
• Highly recommended
• Minimize turnaround• Make corrections
while image is in front of you
• Once it’s gone you don’t have to deal with it again
• Highly recommended
• Minimize turnaround• Make corrections
while image is in front of you
• Once it’s gone you don’t have to deal with it again
Transcription models – batch correct, sign
• Most efficient work flow state?
• Longer turnaround
• Error correction more difficult– Right – left– Dates
• Most efficient work flow state?
• Longer turnaround
• Error correction more difficult– Right – left– Dates
Transcription models – editor• Not recommended
• Transforms transcriptionist model correctionist
• Inefficient, expensive• Maximum turnaround
time• Must remember to
correct and sign reports
• Editor errors• When is it helpful?
– Non-native English speakers?
– Poor dictation technique
• Not recommended
• Transforms transcriptionist model correctionist
• Inefficient, expensive• Maximum turnaround
time• Must remember to
correct and sign reports
• Editor errors• When is it helpful?
– Non-native English speakers?
– Poor dictation technique
Transcription styles
• Free dictation
• Templates and macros
• Free dictation
• Templates and macros
Free dictation
• Advantage– Keeps eyes on
image
• Disadvantage– More time with
editing and corrections
• Advantage– Keeps eyes on
image
• Disadvantage– More time with
editing and corrections
Templates – advantages
• Improved time savings
• Improved report accuracy
• Consistent report structure– Personally– Across department– Need agreement among radiologists
• Facilitates structured reporting– BIRADS
• Improved time savings
• Improved report accuracy
• Consistent report structure– Personally– Across department– Need agreement among radiologists
• Facilitates structured reporting– BIRADS
Templates – disadvantages
• Takes eyes of the image
• May forget to delete non-relevant text
• Takes eyes of the image
• May forget to delete non-relevant text
Templates
• Especially useful for repetitive boilerplate– Biopsy– Angiography
• Especially useful for repetitive boilerplate– Biopsy– Angiography
• The patient was placed on the CT table in [<supine> ] position.
• Initial scans were obtained to localize the [ ].
• An appropriate site at the [ ] was marked.
• The patient was prepped and draped in the usual sterile manner. Local anesthesia was achieved with infiltration of 1% Xylocaine.
• The patient was placed on the CT table in [<supine> ] position.
• Initial scans were obtained to localize the [ ].
• An appropriate site at the [ ] was marked.
• The patient was prepped and draped in the usual sterile manner. Local anesthesia was achieved with infiltration of 1% Xylocaine.
Template approaches
• Few general reports– Fill in the blanks– Default fill in the
blanks
• Many specific reports
• Few general reports– Fill in the blanks– Default fill in the
blanks
• Many specific reports
• Case [ ]. [ ]• There is no evidence
of fracture, dislocation, or bony destruction.
• [<The joint spaces are within the limits of normal.>]
• [ < >]• Impression:• [<Negative
examination.>]
• Case [ ]. [ ]• There is no evidence
of fracture, dislocation, or bony destruction.
• [<The joint spaces are within the limits of normal.>]
• [ < >]• Impression:• [<Negative
examination.>]
Templates– itemized reports
– Lungs: [<normal.>]– Pleura: [<normal.>]– Mediastinum [<normal.>]– Hila: [<normal.>]– Other: [< >]– Comparison: [<None.>]
– Impression: [<normal>]
– Lungs: [<normal.>]– Pleura: [<normal.>]– Mediastinum [<normal.>]– Hila: [<normal.>]– Other: [< >]– Comparison: [<None.>]
– Impression: [<normal>]
Template tricks
• Standardize template naming convention
• Modality body part side, technique
• Standardize template naming convention
• Modality body part side, technique
Template tricks
• Make template easy to change on the fly
• Liberal use of paragraphs
• Make template easy to change on the fly
• Liberal use of paragraphs
• Case [ ].• Ultrasound abdominal aorta.
• Real-time ultrasound examination of the abdominal aorta was obtained in transverse and longitudinal projections.
• The patient [<does not have an>] abdominal aortic aneurysm.
• The abdominal aorta measures [ ] cm in maximal diameter.
• [< >]
• Impression:
• [<The patient does not have an abdominal aortic aneurysm.>]
• Case [ ].• Ultrasound abdominal aorta.
• Real-time ultrasound examination of the abdominal aorta was obtained in transverse and longitudinal projections.
• The patient [<does not have an>] abdominal aortic aneurysm.
• The abdominal aorta measures [ ] cm in maximal diameter.
• [< >]
• Impression:
• [<The patient does not have an abdominal aortic aneurysm.>]
Dictation technique
• Fast ok• Must be clear and
distinct– Think before
speaking– Know what you
want to say– No filler sounds
• Fast ok• Must be clear and
distinct– Think before
speaking– Know what you
want to say– No filler sounds
Dictation technique
• Speak in phrases– Get a flow– Correct in phrases rather than individual
words
• Use complete sentences• Use paragraphs liberally
• Speak in phrases– Get a flow– Correct in phrases rather than individual
words
• Use complete sentences• Use paragraphs liberally
Dictation technique
• Consistent style
• Keep reports short
• Don’t number items in impression
• Consistent style
• Keep reports short
• Don’t number items in impression
Dictation technique
• Dictate – then correct– Keep eyes on image
• Read and correct reports carefully before signing
• Dictate – then correct– Keep eyes on image
• Read and correct reports carefully before signing
Microphones
• Proper location
• Headsets?
• Proper location
• Headsets?
Environment
• Noise control
• Bullpen disruption
• Noise control
• Bullpen disruption
Monitor layout
• Open window in admin monitor– Don’t continually
check transcription– Dictate then edit
• Pop-up in admin monitor
• Open window in admin monitor– Don’t continually
check transcription– Dictate then edit
• Pop-up in admin monitor
Monitor layout
• Separate monitor?– VR– CPRS– Internet window –
Google– Decision support
software?– Teaching file
software?
• Separate monitor?– VR– CPRS– Internet window –
Google– Decision support
software?– Teaching file
software?
Training for problem words
• Case number December
• Pulmonary bony
• Adrenal no renal
• Case number December
• Pulmonary bony
• Adrenal no renal
Gotchas
• Impression:
• Dictate case number in every report– Troubleshooting
• Impression:
• Dictate case number in every report– Troubleshooting
Gotchas
• How reports look in VR may not be how report looks in PACS, Vista, or CPRS– Line spacing– New lines vs. paragraphs
• How reports look in VR may not be how report looks in PACS, Vista, or CPRS– Line spacing– New lines vs. paragraphs
Residents
• Pre-dictation by resident
• Make corrections and finalize report at time of checking
• Easy sign-off by attending
• Drawbacks – templating– May not learn elements of
a good report
• Pre-dictation by resident
• Make corrections and finalize report at time of checking
• Easy sign-off by attending
• Drawbacks – templating– May not learn elements of
a good report
Success rules for VR
• You must want system to work
• Training the VR vs. VR training you
• Rule of thirds
• You must want system to work
• Training the VR vs. VR training you
• Rule of thirds
Future development of VR
Improved recognition engines
• Better accent recognition
• Better recognition of small words
• Better accent recognition
• Better recognition of small words
Improved integration of PACS, HIS-RIS
• Too easy to mark case as read when not• Too easy to mark case unread when is
read• Too easy to hang up report
– Impression:
• Too easy to forget to sign off on report
• Too easy to mark case as read when not• Too easy to mark case unread when is
read• Too easy to hang up report
– Impression:
• Too easy to forget to sign off on report
Improved grammar checking
• There• Their• They’re
• Two• Too• To
• Capitalization
• There• Their• They’re
• Two• Too• To
• Capitalization
Structured reporting
• Standard lexicons
• Universal framework for reports– Improve readability– Minimize style
variation between Radiologists
– Data mining
• BIRADS
• Standard lexicons
• Universal framework for reports– Improve readability– Minimize style
variation between Radiologists
– Data mining
• BIRADS
Seamless integration of communication
• Clinical alerts
• Paging for critical findings
• Feedback to technologist, QA supervisor
• Clinical alerts
• Paging for critical findings
• Feedback to technologist, QA supervisor
Ultimate VR goal: universal recognition
Outlook Mailgroup
• VHA Radiology Voice Recognition
• VHA Radiology Voice Recognition