Shared User-Computer Control of a Robotic Wheelchair System

Holly Yanco

MIT AI Lab

Thesis Supervisor: Rod BrooksCommittee Members: Eric Grimson, Rosalind Picard

Problem Statement

• Some people are unable to drive standard powered wheelchairs and must rely upon caregivers

• This population is estimated to be at least 15,000 people in the United States

• A “conservative estimate indicates that over 2 million people with severe special needs within the EC could benefit from an individually configurable intelligent wheelchair” [Borgolte 98]

Potential User Groups

• Initial onset of Guillian-Barré syndrome

• Multiple sclerosis

• Cerebral palsy

• Spinal cord injury

• Brain injury

Research Goals

• Assist user with navigation in indoor and outdoor environments

• Immediately navigate novel environments safely

• Ensure the usability of the system by including an interface that can be controlled by many different access devices

Wheelesley

Research Contributions

• First indoor/outdoor robotic wheelchair system• Indoor navigation: 71% less effort• Outdoor navigation: 74% less effort• Indoor/outdoor mode detector: 1.7% error rate• Customizable user interface demonstrated with

eye tracking and single switch scanning

Standard Powered Wheelchair

Wheelesley

Related Work: Travel Restrictions

• Magnetic lane [Wakaumi et al 92]

• Maps [Radhakrishnan and Nourbakhsh 99] [Wang 97] [Madarasz 91]

• Trained paths [Yoder et al 96] [Stanton et al 91]

Related Work: Outdoor Navigation

• TAO Project [Gomi and Griffith 98]: tested outdoors with 3 ft high snow walls on either side of sidewalk

• Intelligent Wheelchair Project [Gribble et al. 98]: plans to include outdoor navigation

• [Radhakrishnan and Nourbakhsh 99]: plan to develop outdoor navigation

Related Work: Interfaces

• OMNI [Buhler et al 97]: user interface for joystick, customized for row/column scanning with switch

• VAHM [Bourhis and Pino 96]: interface for single switch scanning

• Joystick only: [Yoder et al 96] [Tahboub and Asada 99] [Simpson et al 99]

• Joystick with additional buttons or switches [Connell and Viola 90] [Miller and Slack 95]

• Voice control [Stanton et al 91] [Amori 92] [Simpson and Levine 97]

• Ultrasonic head control [Jaffe 81] [Ford and Sheredos 95]

• Face tracking [Adachi et al 98] [Bergasa et al 99]

Navigation

Typical Planning-Reaction

Architecture

Wheelesley architecture

General Navigation

• User provides high level control– Straight/left/right at path choices

• Wheelchair provides low level control– Path following– Obstacle avoidance

Indoor Navigation

• Sonar and infrared sensors

• Sensor clustering

• Robotic assistance– Hallway following– Obstacle avoidance

Indoor User Tests• 14 able-bodied subjects

– 7 men, 7 women– age 18 to 43

• 2 robotic trials, 2 manual trials

• 71% improvement in user effort

• 25% improvement in time to traverse course

Indoor User Tests: Results

Trial Manual Robotic

NumClicks

1 90.2(16.3)

25.6(4.9)

2 77.1(9.8)

22.0(3.3)

TotalTime

1 405.1(42.1)

299.1(18.4)

(sec) 2 397.3(43.7)

302.3(32.5)

Outdoor Navigation

• Vision system– STH-V1 Stereo Head

from Videre Designs

• Robotic assistance– Sidewalk following– Obstacle avoidance

Local Path Detection

Image Median filtering Edges from intensity gradient

Neighbor elimination Remove far points

Lines calculatedusing absolute deviation

Sidewalk following

• Select left or right line based upon which has more edge points on the line

• Use edge to steer

• If neither edge is a good candidate, move forward slowly to regain lines present in an earlier frame

Obstacle Detection

Obstacle avoidance

• Takes precedence over sidewalk following

• Slows if obstacle detected in far center region (~5 to 10 feet away)

• Stops if obstacle detected in close center region (~2 to 5 feet away)

image

Close center

Far center

Outdoor User Tests• 7 able-bodied subjects

– 4 women, 3 men– age 24 to 31

• 2 robotic trials, 2 manual trials

• 74% improvement in user effort

• 20% improvement in time to traverse course

grass

road

Outdoor User Tests: Results

Trial Manual Robotic

NumClicks

1 44.6(6.5)

12.3(5.9)

2 35.7(10.5)

8.9(8.9)

TotalTime

1 234.5(25.5)

181.4(26.6)

(sec) 2 213.1(31.9)

175.9(28.4)

Indoor/Outdoor Detector

• Uses multiple sensors to determine if chair is indoors or outdoors– Temperature

– Sonar

– Light: uv filter

– Light: ir filter

– Light, no filter

Mode detection

• C4.5 used to learn a decision tree

• Data set consists of 547 indoor data vectors and 647 outdoor data vectors

• Decision tree learned has a 1.7% error rate

User Interface

Access Methods

• Joystick• Joystick with plate• Single switch• Multiple switch arrays• Sip and puff• Chin joystick• Mouth plate• Eye tracking

• Means for controlling a powered wheelchair

• Usually selected by the wheelchair provider to meet the user’s needs and abilities

Access Method: EagleEyes• Eye tracking system developed

by Jim Gips at Boston College• Measure EOG using electrodes• Use measurements to control

mouse

Access Method: Single Switch Scanning

• Interface scans through 4 arrows:– Forward

– Right

– Left

– Back

• User hits switch when desired command is highlighted

Physical Therapist Evaluation

• 12 physical therapists at Spaulding Rehabiliation in Boston

• System demo

• Seen as tool for training as well as for everyday use

• Offered patients as future test subjects

Physical Therapist Evaluation

• Suggested changes– Sensor-guided driving for reverse– Appearance of wheelchair– Powered wheelchair brand

Summary

• Research resulted in first indoor/outdoor wheelchair system

• Navigation assistance reduces user effort and travel time

• Easily customized user interface can be used with many different access methods