Accessibility for All: Adaptive Computer Access Tools

for the Neuro-motor Disabled in India

Presented By

Animesh MukherjeeResearch Scholar

Department of Computer Science and EngineeringIIT Kharagpur

The Reality

• Suppose you are asked to use a computer which has

The Mouse Unplugged

The Keyboard Unplugged

The Divide

• Nevertheless there is a big population in India (14.56 million approx) that experiences such a difficulty every day

• These are people suffering from neuro-motor disorders

• For them the presence or the absence of a mouse or a keyboard is always synonymous to its absence

Neuro-Motor Disorder – What is it ?

• These disorders are caused by -

Faulty development of motor areas in the brain, or,

Total damage of these motor areas.

Produces Nerve Cells that Causes Movements of the

Body PartsServes to Modify the Movements

Consequences …• Severe difficulty with fine motor tasks (like writing,

stitching, using computer peripherals, and various other such tasks.)

• Severe difficulty with any kind of communication.

• In a nutshell,

Access to the computers is almost a “dream come true”

The presence/absence of the peripherals are irrelevant for them.

Can Computers Help

• Certainly computers can help this population by being

An easy medium of communication (which they find very difficult)

An intelligent companion by understanding the needs and thereby reducing the communication efforts

The Impetus: Something Indian!!

• Mainly the Indian scenario

Present systems are tuned to foreign socio-cultural context

All of them are imported – no local support

Costly for an average Indian user ( E Z Keys - $1400, Gyro-HeadMouse - $1495, CameraMouseTM - $695 + costly video camera)

Lack of Adaptation in existing systems

Contributions

• Implementation of a virtual adaptive mouse – SweepSticks

This work was a joint effort of myself and one of my fellow researchers Mr. Koushik Chakraborty

• Design and Implementation of the prediction support for a virtual keyboard (both Hindi and Bengali) – SulekhA

• Field Testing and analysis of both SweepSticks and SulekhA

The Prelims: Special Access Mechanisms

• Hardware Component – Depending upon the degree of their motor control the disabled people can use either one or at most two switches (specially designed for them) in order to access the computer.

The Switch Emulating theShift Operation

The Switch Emulating theRegister Operation

The Interface with the Computer

Courtesy IICP, Kolkata

Special Access Mechanisms (contd…)

• Software Component

Scanning Mechanisms – Guided / periodic focusing and defocusing of screen elements.

Shift of focus – Shift operation (needs one switch)

Selection of a particular screen element – Register operation (needs another switch)

Methods of Scanning

Co-ordinate Scan Matrix Scan (3D, 2D, 1D)

Cartesian Polar

Direction of movementof the mouse pointer

Direction of rotation ofthe axes

Direction of movement of themouse pointer

Direction of movementof the x-coordinate

selector

Row Level Scan

Cell Level Scan

Block Level Scan

• SweepSticks: A DemoNumeric

KeysVowelKeys

ConsonantKeys

MatraKeys

ConjugateKeys

Text Area

CommandMenu

Prediction Panel This panel is the

contribution of thecurrent work. It doesboth character and

word levelpredictions.

The red rectangleis the highlighterindicating row-

level scan

This row doesprediction from adynamic corpuswhich tries tocapture userpreferences

These rows doprediction from a

static corpus

Probable words tobe typed next

SulekhA: A Demo

Learning User Preferences

ROOT of BST(-1000,-1000)

[0]

User clicks the point (25,40) on the screen (25,40)

[1]

(15,25)

[1]

User clicks the point (15,25) on the screen

(30,45)

[1]

User clicks the point (30,45) on the screen

(50,80)

[1]

User clicks the point (50,80) on the screen

(15,20)

[1]

User clicks the point (15,20) on the screen

Forming PathsNow if the user clicks the point (30,45) once again ???

(-1000,-1000)

[0]

(25,40)

[1]

(15,25)

[2]

(15,20)

[2]

(30,45)

[2]

(50,80)

[2]

fq = 1

User clicks the point (15,25) once again

fq = 1

User clicks the point (50,80) once again

fq = 1

User clicks the point (15,20) once again

fq = 1

Statistics … Statistics

• SulekhA uses

Bigram Prediction Strategy for Word LevelThe training corpus at present contains approximately 1

million words and 0.12 million distinct bi-grams.

The format of the corpus is shown below, <frequency bigram1 bigram2>

Unigram Prediction Strategy for Character LevelThe training corpus at present contains approximately

1.3 million words and 0.05 million distinct unigrams.

The format of the corpus is shown below, <frequency unigram>

The Strategies

H1 H2

B ¢j

j e

a¤¢j

S m

j ¡b¡

B j

c¤d

M¤h

a¡C B l

M¡l ¡f

B S ®L e

i ¡m

M¡h

S eÉ

®L

5590

75

45

100

70

55

69

120

8558

B

j

¡

l

Smarker = trueweight = 90

marker = trueweight = 65

marker = falseweight = 0

marker = falseweight = 0

marker = trueweight = 100

®marker = false

weight = 0

Lmarker = trueweight = 110

lmarker = trueweight = 75

.

.

.

………………. ………… …………...

………………… …………………

…………………

Word Level Character Level

Shradha Writes with SulekhA

Assessments…

• SweepSticks Presently tested for four subjects at IICP Kolkata

Testing is done by measuring success and failure

Success - Each user is asked to open a particular application using SweepSticks. If the user is able to do the same in one go, with the minimum number of shift and register operations actually required to open the application, then it is a success, else failure.

Success and failures are measured for all the users for a number of sessions both in absence and presence of the adaptive help

Number of Successes 100Number of Successes + Number of Failures

%Success =

Results

0

10

20

30

40

50

60

70

80

90

D1 D3 D5 D7 D9 D11 D13

Sessions

% S

ucce

ss o

f Bar

sha

Success Rate ofBarsha withoutAdaptive Help

Success Rate ofBarsha withAdaptive Help

0

10

20

30

40

50

60

70

80

90

D1 D3 D5 D7 D9 D11

Sessions

% S

ucce

ss o

f Sra

ddha

Success Rate ofSraddha withoutAdaptive Help

Success Rate ofSraddha withAdaptive Help

0

10

20

30

40

50

60

70

80

90

D1 D3 D5

Sessions

% S

ucce

ss o

f Sub

hajit

Success Rate ofSubhajit withoutAdaptive Help

Success Rate ofSubhajit withAdaptive Help

010

2030

40

50

60

70

8090

D1 D3 D5

Sessions

% S

ucce

ss o

f Cha

ndan

Success Rate ofChandan withoutAdaptive Help

• SulekhA Typing rate (number of characters typed per minute) was

measured Measurements were taken when the prediction was not in

use and also when in use

Assessments (contd…)

0

0.5

1

1.5

2

2.5

3

D1 D3 D5 D7

Sessions

Typi

ng R

ate

of B

arsh

a Rate of Increasein Typing Speedfor Barsha inAbsence ofPrediction

Rate of Increasein Typing Speedfor Barsha inPresence ofPrediction 0

0.5

1

1.5

2

2.5

3

D1 D3 D5 D7

Sessions

Typi

ng R

ate

of S

radd

ha

Rate of Increasein Typing Speedfor Sraddha inAbsence ofPrediction

Rate of Increasein Typing Speedfor Sraddha inPresence ofPrediction

0

0.5

1

1.5

2

2.5

3

D1 D3 D5

Sessions

Typi

ng R

ate

of S

ubha

jit

Rate of Increasein Typing Speedfor Subhajit inAbsence ofPrediction

Rate of Increasein Typing Speedfor Subhajit inPresence ofPrediction

0

0.5

1

1.5

2

2.5

3

D1 D3 D5

Sessions

Typi

ng R

ate

of C

hand

anRate of Increasein Typing Speedfor Chandan inAbsence ofPrediction

Usability

0

1

2

3

4

5

6

D1 D3 D5 D7 D9 D11 D13

Sessions

User

Gra

des

Usability Curvefor Barsha

Usability Curvefor Sraddha

Usability Curvefor Subhajit

Usability Curvefor Chandan

0

1

2

3

4

5

6

D1 D3 D5 D7

Sessions

User

Gra

des

Usability Curvefor Barsha

Usability Curvefor Sraddha

Usability Curvefor Subhajit

Usability Curvefor Chandan

5 – Excellent, 4 – Good, 3 – Average, 2 – Difficult, 1 – Very Difficult

References• [1] Hufschmidt-Schneider M., Kuhme Thomas and Malinowski U.,• Adaptive User Interfaces, Principles and Practice.• [2] Ahmed Seffah and Homa Javahery, Multiple User Interfaces,• Cross-Platform Applications and Context-Aware Interfaces.• [3] http://www-csli.stanford.edu/cll/aui.html• [4] http://www.words-plus.com• [5] http://www.advancedperipheral.com• [6] http://www.logitech.com• [7] http://cameramouse.com• [8] http://www.cirque.com• [9] http://orin.com/index.htm• [10] http://www.quadjoy.com• [11] http://www.censusindia.net/disability/disability_mapgallery.html • [12] http://www.webhealthcentre.com/general/cp_india.asp • [13] Kaul Sudha and Warrick A., Their Manner of Speaking, Indian• Institute of Cerebral Palsy, Kolkata, India, 1997.

Questions