helpmespeak ! an email access system for dysarthric speakers

Computer/Human Interaction Spring 2013

Northeastern University 1

HelpMeSpeak!

An email access system for dysarthric speakers.

Ganesh Arumugam CCIS MS 2014Stephen Flaherty PHI-CCIS PhD candidateMansoor Pervaiz PHI-CCIS PhD candidateZhichun Ye CCIS-MS 2014



The problem• People with various Neurological Speech

Motor disorders have impairments that limit their ability to communicate.



The problem• These people suffer from Dysarthria - a

motor speech disorder resulting from neurological injury that makes if difficult to pronounce words. (wikipedia)



The problem• Coupled with their

inability to use physical input devices, dysarthric speakers are not able to use standard computer interfaces and can struggle with assistive devices (AACs)(Best &Butler 2012).



Our user population

• Adult residents of The Boston Home-a residence and care center for adults with progressive neurological diseases.

www.thebostonhome.org



Motivation• Residents at Boston Home find it difficult

to communicate with their– caregivers (in one to one communication)– family members (through email and skype)



Motivation• Currently available systems are

cumbersome and difficult for people with speech and cognitive impairment to use (Hux, et al. 2000).

• Enable dysarthric speakers to create, send and view emails using only voice commands.



Motivation• Provide a customizable system to

recognize their individual vocal characteristics and fit their needs.

• Greatly reduce the time and effort needed for our target users to communicate with their friends and family.



Surprises

• In our ethnographic studies, we discovered that our target user population was very interested in using computers to interact with friends and family or surf the web.



Surprises

• Our target users do not want devices that speak for them.

• They want to continue to use their own voice as long as they can.



The best today• AAC devices do not fit our users needs

well.– Require physical touch or eye tracking– Slow– Expensive

Dynavox Maestro, www.dynavoxtech.com



Standard Speech Recognition• Dragon Naturally Speaking (www.nuance.com)

• Windows Speech Recognition



Standard Speech Recognition• Windows Speech Recognition

(www.windows.microsoft.com)

– Difficult to set up-requires physical interaction– Tedious and tiring to use



Standard Speech Recognition• Grid Systems

– Slow and awkward to use.

www.windows.microsoft.com



Innovation• We propose a very simple email system

that: – Uses pattern recognition instead to regular

speech recognition. – Sends an attached voice message instead of

text in the email.



Innovation• It was taking our target population at

least 40 minutes to write a short email.• Our users were keen to use their own

voice and they are understandable to their family and friends.



Demo

• HelpMeSpeak http://www.ccs.neu.edu/home/zhichun/team/T8/Home.html



User Testing Video



A UI challenge • Original design was only a few screens

that provided access to many functions, the idea to reduce navigation and put all the commands in front of the user.



A UI challenge • However, during prototyping the

abundance of options proved confusion to our users.

• Users only wanted the most critical functions.



A UI challenge • We had to scale back on features and

increase the number of different pages in the application.

• The result was a much sparser layout with larger buttons.



Tricky programming problem • Provide a popup message when the user

gave a command that was not recognized by the system.

• This proved difficult to implement – Expect the system to be always on in the

subject’s room – Could trigger off of any conversations in the

room.



Tricky programming problem • Require ability to distinguish specific

commands from continuous speech and take the correct action.



Tricky programming problem • Our system is based on pattern

recognition of recorded commands for each user, not live speech recognition and interpretation.



Evaluation method• Paper prototype testing with 3 subjects

at Boston Home.• Medium fidelity prototype testing at

Boston Home• Methods

– Standard performance tasks– Observation– Interviews



Redesign and evaluation• Initial application scope scaled down

based on user feedback and prototyping.

• Limited visual cognitive ability of user a huge factor.

• User testing with 3 subjects at Boston Home



Redesign and evaluation• Minimalist design with clear

functionality.• Reduce cognitive load on user• Seek high visibility of features.• Audio cues• Feedback from Team 6 helped identify

some consistency issues and broken components.



The future• Design limitations

– Minimal design look and feel could be improved with more development time.

– Functionality and features can be refined with more user testing.

– Improve user voice control options for recording and navigation

– User access was our biggest challenge



Our future process• Spend more time field testing

prototypes with our user group.• Understand that the system is based

around what they want and not what we want.



Our future process• Implement a more functional prototype

over the summer• If successful prepare a CHI paper for fall

submission



Acknowledgements• Images:

– www.nuance.com– www.microsoft.com

• The Boston Home www.thebostonhome.org

• Thanks to Team 6 for design feedback!• Coding resources:

– www.w3schools.com



References• Best, K., & Butler, S. (2012). Disability and Communication: A

Consideration of Cross-disability Communication and Technology. Disability Studies Quarterly,32(4).

• Hux, K., Rankin-Erickson, J., Manasse, N., & Lauritzen, E. (2000). Accuracy of three speech recognition systems: Case study of dysarthric speech.Augmentative and Alternative Communication, 16(3), 186-196.



HelpMeSpeak!• Web-based email system for dysarthric

speakers• Simple design for physically and visually

impaired users• Customizable for the user’s voice and

other needs

helpmespeak ! an email access system for dysarthric speakers

Documents