intelligibility severity rating of frenchay...
TRANSCRIPT
INTELLIGIBILITY SEVERITY RATING OF FRENCHAY SINGLE WORDS AND SENTENCES BY NAÏVE LISTENERS
SUBJECTS
Session 1
The raters were 5 health service professionals (4 females : 1 male) with no history of hearing impairment.None of the raters were familiar with the individuals from whom the speech samples had been taken, nor with the test items.
Session 2
The raters were 5 health service professionals (5 females) with no history of hearing impairment.None of the raters were familiar with the individuals from whom the speech samples had been taken, nor with the test items.
EQUIPMENT
Session 1
Written introduction and instructions for the rating procedure (see Appendix 1).Toshiba Satellite Pro 4300 series laptop computer.Opus Technology multimedia speaker system SP-699.Wave Lab Lite playback facility.
Session 2
Written introduction and instructions for the rating procedure (see Appendix 1).Toshiba Satellite 4090 CDS series laptop computer.Opus Technology multimedia speaker system SP-699.Microsoft Media Player.
SPEECH SAMPLE
Session 1/2
10 single words randomly generated from the word list of the Frenchay Dysarthria Assessment.10 sentences randomly generated from the sentence list of the Frenchay Dysarthria Assessment.Picture description (Appendix 2).
METHOD
Session 1/2
Rating took place within a quiet room on health authority premises. Each subject was given a copy of the introduction, instructions and transcription forms. Subjects were given the opportunity to ask questions prior to the starting of the transcription exercise. It was stressed that the rating exercise was not a ‘test’ of their ability to decipher the speech sample and that there should be no conferring once the speech samples were being played.Subjects were first played an example speech sample of 2 single words, 2 simple sentences (carrier phrase plus changing verb), and a longer sentence to illustrate connected speech. This example was spoken by a speaker with no history of dysarthria. The example was transcribed to act as a lead-in exercise, to check on the quality of the play back facility and to check on the inter-rater reliability of transcribing normal speech.Following the example transcription, subjects were given the opportunity to ask for clarification of the requirements of the transcription. The dysarthric speech samples were then played in the order single words, sentences, picture description. A short pause was left between each item to allow for transcription. Each set of samples took approximately 10 minutes to complete per speaker.At the end of the transcription the completed forms were collected by the examiner and responses tabulated.
RESULTS
Tables have only been completed for single words and sentences. For tables of individual responses, see Appendix 3.
Summary
SUBJECTCC PH GR JP JT
Single word Range 0–20% 0-50% 0 0-10% 10-40%Mode 10% NA 0 0 20%Mean 10% 22% 0 4% 22%
Sentences Range 0-10% 20-60% 0 0-10% 0-20%Mode 10% 30% 0 0 10%Mean 10% 34% 0 2% 10%
COMMENTS
All subjects fall within the severe range on judgements of intelligibilty. PH scored better on the sentences but largely due to the higher recognition rate of one rater.
Picture descripton has not been analysed at this time. Raters were asked to note down any specific words or phrases that they could discern. Analysis was initially to be made on the number of recognisable words as a percentage of the total number of
words in the sample. However, the spectrum of recognition was too broad across the group to make a valid judgement. In the second session, raters were asked to score the picture description according to the Frenchay test intelligibility judgements for connected speech. None of the subjects were rated above the level of ‘Ocasional words decipherable’. The majority of ratings were at the ‘Totally unintelligible’ level.
25/04/02MP
Appendix 1
STARDUST Project -Intelligibility Severity Rating – Naïve Listener
Frenchay – Single words and sentences
Connected Speech (Picture Description)
INTRODUCTION
STARDUST is a Department of Health funded project being run between the University of Sheffield and the Medical Physics Department of Barnsley District General Hospital. The aim of the project is to develop a new voice recognition system that can be used by people with severe physical disability and severe speech problems (dysarthria) to access an environmental control system. This voice recogniser would supplement or even replace the time consuming and often effortful scanning/switching system that many people have to use to access environmental controls.
Voice recognition systems have been developed for people with mild speech production problems. This project is aimed at those people with severer problems than are frequently reported in the literature. As part of the project it is necessary to rate the severity of the speech problem in terms of ‘intelligibility’. Intelligibility is actually quite difficult to judge. People very quickly tune in to different speech, so getting a ‘real’ rating of how easy somebody is to understand is difficult if you rely on people who are familiar with that person’s way of speaking. We are therefore asking people who have never met the volunteers who have agreed to take part in this research to listen to a series of words, sentences and a short passage ofconnected speech, to see how much they can understand.
WHAT AM I BEING ASKED TO DO?
You will shortly be asked to listen to 10 words. Each word will be played only once, with a short pause to allow you to write down what you think you have heard the person say. If you could not understand the word at all, just write ‘Don’t know’, otherwise write down exactly what you thought you heard.
After the single words you will hear 10 short sentences. Each sentence will be played only once. The first part of the sentence is the same each time. Only the last word will change. As before, please write down exactly what you thought you heard. If you could not understand at all, please write ‘Don’t know’.
The final part is a short passage of connected speech, based on a description of a picture. You will not be shown the picture. Please just write down any words or phrases that you understood the person to say.
When you are listening to the items please do NOT confer with your neighbour about what you thought the word or phrase might have been. It is very important in terms of rating the ‘severity’ of the speech that the responses that we receive are your own interpretation of what you heard. It is just as important that we know what you thought the person said, as much as getting a correct answer.
The information that you provide is for group analysis only. You do not have to write your name on any of the sheets.
If you have any questions, please ask them before the items are played.
Thank you for your time.
STARDUST – INTELLIGIBILITY SEVERITY RATING
Date:
Location:
EXAMPLE
Single Words
1.
2.
Sentences
1.
2.
Connected Speech
SUBJECT IDENTIFIER : GR
Single Words
1. 6.
2. 7.
3. 8.
4. 9.
5. 10.
Sentences
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Connected Speech Sample
Intelligibility Rating(Please circle the statement letter that you feel applies to this speech example)
a No abnormalityb Speech abnormal but intelligible – occasionally has to repeatc Speech severely distorted, can be understood half the time. Very often has to
repeatd Occasional words decipherablee Totally unintelligible
Appendix 2
Line drawing of a flood scene.
Appendix 3
PRE RATING TEST EXAMPLES
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 5SINGLE WORD CAT Cat Cat Cat Pat CatSINGLE WORD TABLE Table Table Table Table Table
SENTENCE THE MAN IS WALKING
Walking Walking Walking Walking Walking
SENTENCE THE MAN IS WASHING
Washing Washing Washing Washing washing
CONNECTED SPEECH
THE CAT IS SITTING ON THE TABLE AND WASHING ITS PAWS.
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
Total Items 15 15 15 15 14 15Total % Identified 100 100 100 93 100Mean Identification 98%
Intelligibility Testing – Frenchay Single words
Subject – CC
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 59 JACKET DK DK DK DK DK13 GROW Girl Gloves Go Clothes DK15 BUBBLE Gurgle Bottle DK DK DK16 STUBBLE DK Sugar Sugar Table Sugar18 DOUBLE Dribble DK Football Table DK22 AIR Air Lair Air Air Air26 SPRINKLE Twinkle DK DK DK DK49 FAT Cut Cat Cat Tap Top5 PARK Hack DK Clock Black Park2 SWARM Fall DK Room Wand Warm
Total Items 10 1 0 1 1 2Total % Identified 10 0 10 10 20Mean Identification 10%
Severity rating – SEVERE
Intelligibility Testing – Frenchay Sentences
Subject – CC
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 543 DANCING DK Standing DK Dancing DK11 RECEDING DK DK DK DK DK22 TICKING Sitting Kicking Kicking Kicking DK25 BLOCKING DK Broken DK Looking Broken29 CATCHING Catching DK DK Watching DK30 PITCHING DK DK DK DK DK33 PINNING DK DK DK Punching DK39 STANDING Chairing DK Standing Dancing DK40 BENDING DK DK DK DK Selfish4 CREEPING DK DK Crying Catching Crying
Total Items 10 1 0 1 1 0Total % Identified 10 0 10 10 0Mean Identification 6%
Severity rating – SEVERE
Intelligibility Testing – Frenchay Single Words
Subject – PH
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 547 LAIR Rare Glare There Lair Air9 JACKET DK DK DK Turkey Jackie20 HERE Year Year Here Here Year21 ERROR DK Everyone Everyone DK Table25 CYCLE Cycle Side door Cycle Cycle Side draw26 SPRINKLE Sprinkle DK Twinkle Sprinkle DK29 PLAY Parade DK Away Play DK42 THOUGHT DK Haunt Font DK DK46 LOYAL Loyal DK Loyal Royal Lion5 PARK DK DK Book DK Pane
Total Items 10 3 0 3 5 0Total % Identified 30 0 30 50 0Mean Identification 22
Severity rating – SEVERE
Intelligibility Testing – Frenchay Sentences
Subject – PH
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 546 TUCKING Talking Talking Talking Talking Talking5 PATCHING DK Batting Watching Catching DK14 DAMNING Damning DK Thinking DK DK22 TICKING Ticking Ticking Thinking Ticking DK24 AIRING Airing DK Hearing Hearing DK29 CATCHING Catching Catching Catching Catching Catching37 DRINKING Drinking Ringing Drinking Ringing DK39 STANDING Standing Standing Standing Standing Standing45 TUGGING DK DK Talking Talking DK3 CREEPING Cleaning Screaming Screaming Screaming Screaming
Total Items 10 6 3 3 3 2Total % Identified 60 30 30 30 20Mean Identification 34%
Severity rating – SEVERE – (moderate)
PRE RATING TEST EXAMPLES
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 5SINGLE WORD CAT Cat Cat Cat Pat CatSINGLE WORD TABLE Table Table Table Table Table
SENTENCE THE MAN IS WALKING
Walking Walking Walking Walking Walking
SENTENCE THE MAN IS WASHING
Washing Washing Washing Washing washing
CONNECTED SPEECH
THE CAT IS SITTING ON THE TABLE AND WASHING ITS PAWS.
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
The cat is sitting on the table and washing its paws
Total Items 15 15 15 15 14 15Total % Identified 100 100 100 93 100Mean Identification 98%
Intelligibility Testing – Frenchay Single words
Subject – GR
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 510 JAGGED DK DK DINNER BURGER DK11 GLOW DK DK BALL GO BUSINESS25 CYCLE DK DK SUGAR DIGGER DK30 PAY DK DK DK BALL DK31 BRIAR DK DK DK DK DK37 THOUGH DK DK TEA OK DK39 URGENT DK DK AGAIN DK GONE40 SERGEANT DK DK BIGGER DIGGER BETTER44 THORN GO GO DONE GO DK49 FAT DK DK DK FARMER DK
Total Items 10 0 0 0 0 0Total % Identified 0 0 0 0 0Mean Identification 0
Severity rating – SEVERE
Intelligibility Testing – Frenchay Sentences
Subject – GR
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 503 CREEPING DK GOING DK GOING GOING06 COMING DK DK GOING DK DK08 SPINNING BIGGER RUNNING LEAVING BIGGER DK13 TINKERING DK DK DK DK DK19 PRANCING BIGGER DK DK DK DK25 BLOCKING BIGGER BIGGER BIGGER BIGGER BIGGER32 THUMBING DK DK DIGGING DIGGING DK36 NAMING NEARER DK SMALLER LENDING NERVOUS46 TUCKING DK DK KICKING GIDDY DK48 GAGGING BIGGER DK DK GINGER DK
Total Items 10Total % Identified 0 0 0 0 0Mean Identification 0
Severity rating – SEVERE
Intelligibility Testing – Frenchay Single words
Subject – JP
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 504 SPARK WORK LOOK LOOK LOOK DRUNK12 GO GO DK DK GO GONE13 GROW NO DK DK ROAD BLOW15 BUBBLE BOOK DK BOOK BOOK DK22 AIR HAIR HAIR HAIR HAIR HAIR24 JUNGLE DK DK DK DK DK25 CYCLE CAKE DK DK CUP TAKE32 PRIOR CAKE DK CRY CRY PLAY41 BROUGHT WORK DK DK WORK WORK43 BRAWN WORK NICE WORK BOOK DK
Total Items 10 1 1Total % Identified 10% 0 0 10% 0Mean Identification 4%
Severity rating – SEVERE
Intelligibility Testing – Frenchay Sentences
Subject – JP
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 513 TINKERING DK DK DK DK DK29 CATCHING DK DK DK CATCHING DK30 PITCHING SPEAKING DK SLEEPING DK DK33 PINNING BIG DK PLAYING PLAYING DK34 FINDING VERY BIG DK HIDING DK HIDING40 BENDING READING DK DK DK DK41 BRINGING DK DK DK DK DK42 BITING DK DK WASHING WATCHING DK43 DANCING MAKING A
SHOEDK DK LOOKING DK
47 PICKING SPEAKING PAINTING KICKING PICKING DKTotal Items 10Total % Identified 0 0 0 10% 0Mean Identification 2%
Severity rating – SEVERE
Intelligibility Testing – Frenchay Single words
Subject – JT
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 512 GO GO GO GO GO GO14 GOAT GOAT GOOD READ GOAT DK26 SPRINKLE PEACOCK DK SWEETCORN THANKYOU BECAUSE28 SLAY SAY DK PLAY SLAY DK30 PAY PLAY GET DAY PAY BIRD31 BRIAR BEAR BEAR PAY BEER DK32 PRIOR FIRE DK HAY DK DK33 AREA AIR HAIR DK A DK34 FLOOR FOUR FLOOR FOUR FOR FLOOR41 BROUGHT BOAT WOOD BOAT BOAT BOAT
Total Items 10 2 2 1 4 2Total % Identified 20% 20% 10% 40% 20%Mean Identification 22%
Severity rating – SEVERE
Intelligibility Testing – Frenchay Sentences
Subject – JT
Item No. Target word Rater 1 Rater 2 Rater 3 Rater 4 Rater 506 COMING COMING COMING COMING DK DK13 TINKERING SLEEPING DK SPEAKING THINKING SPEAKING14 DAMNING BUSY GOING GOING HEAVY GOING24 AIRING DK DK DK HEAVY DK25 BLOCKING DK DK BATTING MOPING DK28 REAPING READING DK READING DK READING31 NUMBING DK DK DK DK DK43 DANCING DK DK BENDING BENDING DK44 FENCING DK DK THINKING BENDING DK49 HEARING EATING DK HEARING HEARING DK
Total Items 10 1 1 2 1 0Total % Identified 10% 10% 20% 10% 0Mean Identification 10%
Severity rating – SEVERE