a multiple-attribute decision-making approach to assess the disability of visually impaired workers

Journal of Multi-Criteria Decision Analysis, Vol. 4, 160-1 76 (1995)

A Multiple-attribute Decision-making Approach to Assess the Disability of Visually Impaired Workers

MIN-DER KO AND JEN-GWO CHEN Department of Industrial Engineering, Universiy of Houston, Houston, TX 77204-48 12, U.S.A.

ABSTRACT

The Americans with Disabilities Act (ADA) has redefined employability and disability. The measurement and evaluation of the disabled worker’s residual capabilities, in terms of worker strengths and weaknesses, job requirements and work environment, is the first step to comply with the ADA. This study presents an application of multiple-attribute decision-making procedures. The task is to develop a single index for use in assessing the disability of visually impaired workers through a consideration of factors defined by the U.S. Employment Service for successful job performance. The index can also be used to identify and prioritize the need for reasonable accommodation and to match visually impaired workers to appropriate employment.

KEY WORDS Analytic Hierarchy Process; disability assessment; multiple-attribute decision making; Disability Index

1. INTRODUCTION

The American Foundation for the Blind estimated that in 1986 there were about 11.4 million people in the United States with some kind of visual impairment (Grayson, 1986). In 1990 the National Health Interview Survey showed that there were approximately 600,000 adult Americans whose work activities were limited owing to visual disabilities and most of them were either unemployed or underemployed (Department of Education, 1990). The visually impaired represent as a whole a relatively large population within our society whose potential has neither been realized nor fully evaluated and used. In fact, two-thirds of disabled people would want to work if the appropriate job opportunities were available (Tompkins, 1993). Eighty percent of Americans recognize that people with disabilities have an underused potential to contribute by working and producing (Brown, 1993). The ADA, which went into effect in July 1992, was designed to assist and accelerate the return t o work of disabled people. The ADA has redefined employability and disability. Employers are asked to look at the abilities of disabled workers and make reasonable accommodations in their jobs and working environments. Traditional rehabilitation and training simply cannot fulfil the requirements of the ADA; thus

CCC 1057-9214/95/030160-17 0 1995 by John Wiley & Sons, Ltd.

Received 18 March 1993 Accepted 4 March 1994

M.-D. KO and J.-G. Chen 161

multidisciplinary approaches (e.g. engineering, rehabilitation, special education) should be employed to provide a proper assessment and reasonable accommodations to comply with the ADA (Milas, 1992).

Traditionally, the assessment of disability is accomplished through various types of standard vocational evaluation systems (e.g. Valpar, SAGE) with appropriate modifications. Each system may comprise several elementary tests examining certain characteristics of the client (e.g. aptitude, interest, general education development). These tests are effective in identifying a broad range of deficits. However, they do not show how an individual’s deficits might interact with task and environmental demands (Department of Education, 1992). Thus a functional assessment approach in response to specific tasks and task-related situations should be implemented throughout the vocational rehabilitation process. The first step of this functional assessment is to establish a measurement representing the physical and psychological capacities and weaknesses of handicapped persons which would be meaningful for industrial purposes. After the need for any modification is identified, reasonable accommodation can be provided and appropriate job opportunities can then be created for disabled workers.

Job-oriented research has been devoted to the development of measuring overall physical ability of the disabled. Birdsong (1972) established the Disability Index (DI) to evaluate the brain injury to a person by forming a basis with Motion and Time Measurement (MTM) variables. He also used the DI to rate a worker’s potential capacity by comparing the handicapped person’s MTM score with the MTM norm. However, that index was targeted to brain-damaged patients and only the client’s vocational potential was measured. Rahimi and Malzahn (1984) proposed a system called the Available Motions Inventory (AMI) for measuring human physical ability of individuals with neuromuscular impairments based on components of manual industrial tasks and evaluated the reliability of the system through multivariate canonical correlation analysis. Although AM1 was interacted with task, AM1 only assessed physical ability without taking other important factors (e.g. aptitudes, vocational preparations) into consideration. Chen (1992) developed an overall index for evaluating the disability of visually impaired workers and accommodating the workplace with ergonomic considerations based on the value of the overall index. This index was derived from a simple additive weighting method by considering various physical, behavioural and job factors. Although the aforementioned approaches are different, they all have a common feature: the index is scaled with respect to normal test performance.

Froberg and Kane (1989) reviewed the methodology for measuring health state preference. Strategies such as holistic, explicitly decomposed and statistically inferred decomposed and scaling methods such as standard gamble, time trade-off, rating scale, magnitude estimation, equivalence and willingness-to-pay methods were discussed and evaluated on the basis of their reliability, validity and feasibility. Since judges’ or subjects’ preferences change over time, it would be a tedious task if the approach required the assessment of all health states. Also, although the provision for scaling constants can express the preferred level of each attribute, it does not reflect the relative importance or worth among attributes. Therefore a more structured approach for measuring and interpreting disability, ill-health and quality of life is needed.

This study presents an application of multiple-attribute decision-making (MADM) procedures to develop a single index with the integration of a range of vocational tests to assess the disability of visually impaired workers. The prototype of the overall measurement, proposed by Chen (1992), is refined with the implementation of robust MADM procedures. Most MADM procedures are designed for ranking a set of alternatives through weighting and scaling, but in our study we modify the approach and focus of our efforts on the attainment of an index score for each subject that predicts hidher ability in performing industrial assembly jobs.

162 Assessment of Visually Impaired Disability

Overall Disability Index

Subindex for Worker, Job, Enwonmental Charactensncs

Various Evaluation Systems for the Specific Sub-index (Second Level)

Various Elementary Tests for Each System (First Level)

Figure 1. The hierarchical structure of the Disability Index

2. DESCRIPTION OF THE DISABILITY INDEX MODEL

The Disability Index (DI) was developed for the evaluation of visually impaired people’s residual capacities in performing industrial assembly tasks. In essence, the DI considers all aspects of a visually impaired person as defined by the U.S. Employment Service for a successful job performance. As shown in the DI hierarchical structure (Figure l), the DI comprises eight sub- indexes in the third level to reflect the person’s capabilities, interests and opinions, physical and mechanical skills, vocational preparations and environmental situations related to a specific job: (1) Achievements Index (e.g. general education development); (2) Aptitudes Index (e.g. spatial perception, colour co-ordination); (3) Interests and Opinions Index (e .g. industry, business, plants and animals); (4) Temperaments Index (e.g. interpretation of feeling, influencing people); ( 5 ) Work Samples Index that shows a worker’s specific vocational preparation (e.g. assembly, computer programming); (6) Job Training Times Index that shows the learning curve and performance; (7) Physical Demands Index (e.g. strength, talking and hearing, handling); and (8) Work Environments Index (e.g. noise, hazard). Each sub-index can be assessed by different vocational assessment and evaluation systems in the second level. For example, Apticom development by the Vocational Research Institute can be used to assess 10 of the 11 aptitudes defined by the Department of Labor, 12 interest areas and general education development (Green County School System, 1986). Each system includes a number of elementary tests in the first level to measure the specific worker, job and environment characteristics. For example, the following 11 specific tests of Apticom assess 10 aptitudes: object identification, abstract shape matching, clerical matching, eye-hand-foot co-ordination, pattern visualization, computation, finger dexterity, numerical reasoning, manual dexterity, work meaning and eye-hand co- ordination.


Since the DI is designed for comparing the test performance data of the visually impaired with that of normal people, the average normal person’s test performance data should be measured and scaled with a scale developed from normal people’s data set. The disabled individual’s performance data are also measured and scaled in a similar fashion and an overall weighted DI is then derived by comparing these two values.

The DI for visually impaired workers can be developed through the following steps.

Step 1. Analyse the client’s evaluation results and construct the hierarchical structure of the DI. Step 2. Have an expert evaluate the relative importance of two sub-indexes in the third level

pairwise to derive a set of subjective weights for all sub-indexes; have the expert assess two evaluation systems in the second level pairwise to derive a set of subjective weights for all evaluation systems.

Step 3. Measure the entropy of each elementary test in the first level from the normal people performance data to derive a set of objective weights for all elementary tests.

Step 4. If the derived subjective weight of the individual sub-index or evaluation system obtained from Step 2 is weighted twice more than the value of the equally weighted index, have the expert evaluate the relative importance of two elementary tests in the first level in the same evaluation system pairwise to derive a set of subjective weights for all elementary tests and integrate both subjective and objective weights to form the overall weights for each elementary test in the first level; otherwise, use objective weight as the overall weight for each elementary test.

Step 5 . Scale each set of test performance data in the same evaluation system of the average normal individual with associated weights obtained from Step 4 into a value between Oand 1.

Step 6. Scale each set of test performance data in the same evaluation system of the visually impaired individual with associated weights obtained from Step 4 into a value between 0 and 1.

Step 7. Calculate the weighted sub-indexes of each evaluation system in the second level with the values obtained from Steps 5 and 6 with the associated subjective weights derived from Step 2.

Step 8. Calculate the weighted DI of a visually impaired individual from all weighted sub-indexes obtained from Step 7 with the associated subjective weights derived from Step 2.

2.1. The subjective weighting For the case of a single attribute the decision problem is trivial, The performance of the single activity of all subjects can be measured by setting two extreme values and scaling the interval between these two extreme points. As the system becomes more complex, more than one activity is under consideration and the relative importance of each activity has to be identified.

Zeleny (1982) defined the weight of attribute importance with two components: a relatively stable concept of a priori attribute importance to reflect an individual’s cultural, genetic psychological, societal and environmental background; and a relatively unstable, context- dependent concept of informational importance. Both concepts to derive weights for all attributes (e.g. elementary tests or evaluation systems) are employed in our study.

The subjective weights fur sub-indexes, systems and elementary tests are established by the Analytic Hierarchy Process (AHP) (Saaty, 1977). This approach provides a vector of weights expressing the relative importance of several elements. For a complex system a hierarchical structure may be constructed by grouping several activities together. A vector of weights can be composed from different levels of this hierarchical structure. Each group of the hierarchical


Table I . The subjective judgment scale

Value* Definition

1 Equal importance 3 or + 5 or + I or + 9 or +

One is slightly better than the other One is essentially better than the other One has demonstrated itself over the other One is absolutely better than the other

~ ~ ~

*2,4, 6, 8: intermediate values between the two adjacent judgments.

structure represents a specific function of the system. Saaty (1990) asserted that pairwise comparisons are the most effective way to concentrate judgments, since they are conducted with standards established by experience or through training.

After the client's evaluation result has been analysed and constructed in a hierarchical structure, the pairwise comparisons are then conducted by a domain expert according to the judgment scale (Saaty, 1977). Table I depicts the scale's respective meanings. For example, if the evaluator considers Valpar # 1 test is slightly more important than Valpar # 2 test among various Valpar Sample Tests, he/she will enter a value of 3 in the decision matrix and a value of + in its reciprocal position.

The pairwise comparison matrix A, subjective weight vector B and eigenvalues X i can be defined as

A=(ai j ) V i , j = l , . . .,n

and

or

(A - n1)B = 0

where the matrix A has all positive entries and satisfies the reciprocal property

aj, = 1 /a,

and consistency condition

ajk =aik/a,, i , j , k= 1 , , . ., n For such a matrix of order n the sum of eigenvalues equals n. If every row is a constant multiple

of the first row, all but one of the eigenvalues X i , i = 1 , . . ., n, of A are zero. This leaves the non-zero eigenvalue to be equal to n. In general, human judgments may not always be consistent despite the best efforts of those making them. A simple example can explain the meaning of consistency. For example, if test A is more important than test B and if test B is also more


important than test C, then it would follow that test A is much more important than test C. A judgment that test C is more important than test A would jeopardize the relation among tests A, B and C. A decision maker should notice this conflict and is responsible for making a consistent judgment. Therefore a consistency index from 0 to 1 is used to measure and ensure the consistency. The consistency index is defined as C = (A,, - n) / (n - l), where A,, is the largest eigenvalue. In a consistent case the index equals 0. Wind and Saaty (1980) suggested that a consistency index of 10To or less should be considered very good and acceptable. The consistency may be improved by revising the estimates in the pairwise comparison matrix.

Saaty (1977) compared 26 commonly used scales and suggested the use of the scale of 1 to 9 as shown in Table I. The scale is not limited to integer values and the user can estimate any real numbers between 1 and 9. Since human judgment is not so precise that they could specify numbers to the digits, the use of integer values in the pairwise comparison would be sufficient for expressing the decision maker’s cognition or judgment. As we mentioned in Section 1, many traditional approaches do not reflect the relative importance or worth among attributes. The eigenvalue approach, as Saaty addressed (1977), is ‘excellent for bargaining purposes as it permits people to debate the reasons for their estimates, arrive at a consensus, and make compromises here and there’.

In this subjective weighting procedure all sub-indexes are compared pariwise. The comparisons of elementary tests of each sub-index are performed optionally, depending on the evaluator’s preference. By rule of thumb we may set up a rule as in Step 4 of developing DI. If the outcome (e.g. 0.5) of one of the sub-indexes is weighted twice more than the value of the equally weighted index (e.g. 0.2 with five attributes in the system), then we should go further to the lowest level of the entire system-the elementary test. This scenario is also depicted in the case study that follows. The subjective weights of the elementary test are combined with the objective weights derived from the next subsection to form a set of overall weights to be used in the scaling procedure. Only the weights of sub-indexes are used in calculating the weighted DI.

While conducting the subjective weighting, to avoid bias, only the meanings and functions of the elements (i.e. elementary test or sub-index) are presented to the evaluator. Other information such as the highest, average or lowest score of the element does not help the decision maker to compare two elements. We shall utilize this information as reference values in the scaling procedure.

Since the subjective weights may vary owing a lack of knowledge or a difference in viewpoints, an analysis of the effect of the weights on the DI and sub-indexes is performed. The purpose of this analysis is to provide a set of recommended weights of specific job category which are acceptable to the users. It is done by enumerating all possible combinations and varying the value of elements in the matrix by one-up and one-down. The set of recommended weights is obtained by taking the mean of upper bound weight and lower bound weight of each attribute (i.e. sub-index).

2.2. The objective weighting The data themselves may contain some decision-relevant information transmitted by the attribute (e.g. the mean or standard deviation of a specific vocational test) from a pairwise assessment of attributes. Although visually impaired people possess some degree of disability, the same approach should be taken to attain the individual’s overall performance score for disabled people and normal people. To represent the disabled person’s performance measurement in terms of that of the average normal person, a set of objective weights must be derived from the normal persons’ performance measures. Unlike the subjective weighting, the objective weighting is irrelevant to job variations. In our study the objective weights are obtained through the Entropy

166

Method (Shannon and Weaver, 1947; Nijkamp, 1977). Martin and England (1981) described the definition of entropy mathematically.

Every time an outcome of a random experiment results in the event E, we gain some information from the occurrence of event E. The information or uncertainty is a real-valued function of events which depends only on the probabilities of the events. The outcomes of the events are countable measurable partitions. The entropy of a countable partition is the average amount of uncertainty or average amount of information contained in the experiment represented by the partition. Entropy is a criterion of measuring this expected information content. This measure of uncertainty is given with negative value as

Assessment of Visually Impaired Disability

n

j = 1 S(p1, * * .,P,)= - K C Pjlnpj

where pi is the probability distribution of performance data of each attribute and K is a constant. If the data of an attribute contain more distinct and differentiated scores of different subjects (alternatives), more information can be transmitted by that attribute. On the other hand, if all the values of an attribute for different subjects are identical, no information can be perceived. The following steps show how to derive a set of weights of attribute importance from the contrast intensity of each attribute.

Step 1. Consider a decision matrix D of m alternatives (i.e. m subjects) and n attributes (i.e. n elementary tests),

D = (xu) V i = 1, . . . , m, j = 1, . . ., n Step 2. Define the probability pu of each attribute j as

rn p..=x.. C x u v i , j

i = I rJ JJ /

Step 3. Compute the entropy Ej of each attribute j as m ...

Ej= - K pulnp,j V j i = 1

where K = l/lnm is a constant.

attribute j, Step 4. Obtain the degree of diversification of information conveyed by the outcomes of

d j = l - E j V j

Step 5 . Evaluate the relative worthiness of each attribute (or each elementary test),

j = I

Step 6. If the decision maker has a set of subjective assessments on each attribute of the subtests, S j , then we have the overall weight for each attribute j as

w; = 6 . J w. J / 8, wj v j j= I


The performance measures of all elementary tests form the decision matrix and each evaluation system has its own decision matrix to derive a unique set of weights. In this study the data for deriving the objective weights are randomly generated as normal distributions with the mean and standard deviation of individual tests from a normal population.

2.3. Scaling approach-modified TOPSIS After we have determined subjective and/or objective weights for every attribute, the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) (Hwang and Yoon, 1981) is used to obtain a sub-index for each evaluation system. TOPSIS yields an acceptable preference order of solutions based on the concept that the most preferred alternative should have the shortest distance from the ideal point and the farthest from the negative ideal point. The distance under consideration here is the Euclidean distance, the shortest distance in a geometrical sense. Yoon (1987) discussed this distance measurement of TOPSIS in depth.

We can describe the ideal alternative A * and the negative ideal alternative A - in a two- dimensional space as shown in Figure 2. These ideal and negative points can be and are frequently displaced as Zeleny (1982) asserted: ‘It is responsive to changes in the available set of alternatives, objectives, evaluations, measurements, and even errors. It responds to new technological advances, inventions, and discoveries of oversights. It becomes a moving target, a point of reference which provides an anchor for human adaptivity, intransitivity, and dynamic adjustment of preferences.’

In our case study all achievable scores are the performance measurements of various characteristic tests represented by raw scores with respect to the normal scale. It is reasonable to place f 3-fold standard deviations from the means of performance measurements of each attribute obtained from normal people as both the ideal alternative and the negative ideal alternative. The rationale is that the range of six times the standard deviation covers more

L I A

Atnibute X , Figure 2. Euclidean distances to the ideal and negative ideal solutions in two-dimensional space

168

than 99% of the population. On the basis of the above assumption, the performance scores of normal people may be displaced through job training and redesigning of the work environment.

Originally TOPSIS was designed for ranking a set of alternatives with several attributes in each alternative and assumes that each attribute takes the monotonically increasing or decreasing utility. In our case study, instead of ranking all alternatives, the purpose is to attain an overall score for each subject. Therefore a slight modification is needed when we regard the entire working population as the number of subjects (or alternatives). From here the words ‘subject’ and ‘alternative’ are handled interchangeably. This modification is made by first determining the ideal point and the negative ideal point with three-times-standard-deviation control intervals and then normalizing the decision matrix with the inclusion of two artificial alternatives.


The algorithm of the modified TOPSIS method is stated briefly as follows.

Step 1. Determine the ideal and negative ideal alternatives. The decision matrix is augmented with the inclusion of two artificial alternatives that will give us the separation measure of any alternative to these two artificial alternatives.

Step 2. Construct the normalized decision matrix which allows interattribute comparisons; we obtain element rjj of the normalized decision matrix R as

The elements in the two artificial alternatives are

and

These equations imply that all columns have the same unit length of vector. Step 3. Construct the weighted normalized decision matrix considering the weights

w = (w1, w2, . . ., wj, . . ., w,) where

n c wj=1 j = I


and the W here may be either w or w* from both weighting methods. Therefore the weighted normalized decision matrix V is

V = , where vii= wjrU

Step 4. Calculate the separation measures; the n-dimensional Euclidean distance is the separation of each alternative from the ideal one and from the negative ideal one:

(vii-v7)2, i= 1,2, . . ., m j = I

and

S; = C (vii-v,:)2, i = 1 , 2 , . . . , m J n j = 1

Step 5 . Calculate the relative closeness to the ideal alternative; the relative closeness of each alternative Ai with respect to the ideal alternative of Ai is defined as

where

1 i f A i = A * 0 i fA i=A-

Ci* =

These scores C: are the overall performance measures of each subject. Ai gets closer to A* as CF approaches 1 .

2.4. Disability Index We calculate the overall scores from test results with their associated weights for each visually impaired individual and normal individual respectively. The final score, the Disability Index, of a visually impaired individual is defined as:

weighted overall score of visually impaired individual weighted overall score of average normal individual

Disability Index =

The DI is scaled between 0 and 1: the larger the index value, the less serious the disability and the closer to a normal worker’s performance.

3. CASE STUDY

The Vocational Evaluation programme at the Lighthouse of Houston is designed to provide an assessment of a visually impaired worker’s vocational potential. A variety of tests are employed

170

to measure the individual’s vocational potential on aptitudes (e.g. SRA Office Skills Test, Differential Aptitude Test, Crawford Small Parts Dexterity Test, Computer Operator Aptitude Battery, Computer Programmer Aptitude Battery), physical capacities (e.g. Valpar Work Samples, Medical Physical Capacity Report), social skills, work habits, occupational interests (e.g. Wide Range Interest Opinion Test, AAMD Becker, Career Occupational Preference Systems), personality, emotional stability, educational achievements (e.g. Wide Range Achievement Test, Basic Achievement of Common Knowledge and Skills Test, SRA Reading Index, SRA Arithmetic Index) and work-related capabilities (e.g. mobility, communication skills). A series of industrial bench assembly tasks (e.g. pen clipping, U-bolt assembly, plumbing coupling assembly/disassembly) are also designed to train the subject’s vocational skills. The appropriate job placement is then recommended based on the client’s evaluation and training results.

Reports from 92 visually impaired individuals from the Lighthouse of Houston are compiled and analysed in this study: 62 males and 30 females with ages ranging from 20 to 50. Different clients may take different tests owing to the time limitation and the availability of tests when clients were admitted to the Lighthouse of Houston for vocational evaluation. Most subjects are able and willing to perform sedentary and/or standing work (e.g. assembly work). Most of them are able but not willing to work under bad working conditions (e.g. around fumes or gases, poorly ventilated areas, noise). The normal performance data are generated based on the available means and standard deviations.

The following case study is selected from vocational reports to demonstrate the development of the DI. The client took the Wide Range Achievement Test (WRAT) (Jastak and Wilkinson, 1984), Wide Range Interest Opinion Test (WRIOT) (Jastak and Jastak, 1979), AAMD Becker Interest Inventory (Becker, 1982), COPS Interest Inventory Test (Knapp and Knapp, 1982), MDC Behavior Identification Test (Materials Development Center, 1974) and Valpar Component Work Sample-B Kit (Valpar Corporation, 1980). Therefore the following four sub-indexes were established based on the DI hierarchical structure: Achievements Index, Interests and Opinions Index, Temperaments Index and Work Samples Index (Figure 3).

Based on the subjective judgment scale, the evaluator forms pairwise comparison matrices for the four sub-indexes in the third level (Table 11) and for the evaluation systems for Interests


-1 Fourth Level (el (-) (-)(-)T.--

valpar #1 valpar #2

valpar #8 valpar #9

vdPw Firsr Level

Figure 3. The Disability Index structure for the case study


Table 11. Pairwise comparison matrix and subjective weights for four sub-indexes in third level

Achievements Work samples Temperaments Interests and Index Index Index Opinions Index

Achievements Index 1 .000 0.333 5.000 3.000 Work Samples Index 3.000 1 .000 8.000 6.000 Temperaments Index 0.200 0.125 1 .000 2.000 Interests and Options Index 0.333 0.167 0.500 1 .Ooo

Subjective weight 0.259 0.587 0.084 0.071 Amax 4.190 Consistency index 6.33%

Lower bound 0.197 0.505 0.057 0.052 Recommended 0.261 0.572 0.086 0.079

Upper bound 0.331 0.650 0.117 0.106

Table 111. Pairwise comparison matrix and subjective weights for three Interests and Opinions Index evaluation systems in second level

WRIOT AAMD COPS

WRIOT AAMD COPS

1 .000 0.200 0.143 5.OOO 1 ,000 0.333 7.000 3.000 1 .OOo

Subjective weight 0.072 0.279 0.649 Amax 3.065 Consistency index 3.25%

Upper bound 0.089 0.357 0.707 Lower bound 0.059 0.222 0.567 Recommended 0.074 0.289 0.636

and Opinions Index in the second level (Table 111). The derived subjective weights for each comparison matrix are also given in Tables I1 and 111. In the process of attaining subjective weights, the evaluations of pairwise comparison have been revised constantly to obtain an acceptable consistency of the pairwise comparison matrices. As shown in Tables I1 and 111, we have A,, equal to 4.190 and 3.065 from the pairwise comparison matrix of the sub-indexes and that of the Interests and Opinions Index evaluation systems respectively. These results yield a consistency index of 6.3% and 3.3% respectively, both of which are within the 10% acceptable level. We have also conducted a sensitivity analysis for subjective weighting and recommended a set of weights for the user's acceptance. The results of the sensitivity analysis are also presented in Tables I1 and 111, with a set of recommended weights (0.261, 0.572, 0.086, 0.079) for the four sub-indexes in the third level and (0.074,0.289,0.636) for the Interests and Opinions Index in the second level.

Following Step 3 in the DI model, the entropy of each elementary test in the first level is measured to derive objective weights (Table IV).

Since the Valpar Work Samples Index is weighted twice more than the equally weighted vaIue of 0.25 among the four sub-indexes in the third level (Table 11), based on our predetermined rule of thumb, a more detailed assessment for this sub-index should be conducted. Thus a pairwise comparison matrix is formed (Table V) to derive subjective weights for all Valpar Work Sample tests (e.g. Valpar # 1, # 2, # 4, # 8, and # 9) and the consistency index is 7.8% with A,, equal

172 Assessment a f Visually Impaired Disability

Table IV. Objective weights for all elementary tests in first level

Objective Objective System Elementary test weight System Elementary test weight

WRAT Spelling test Mathematics test Reading test

MDC Hygiene Irritating habits Odd behaviours Communication skills Attendance Punctuality Cope with work problems Complaints Work energy Stamina Steadiness Distractibility Shop rules conformity Work changes Unpleasant work Social skills Supervision needs Supervisor authority Tension Assistance request Criticism Work method

COPS Science profession Science skilled Technology profession Technology skilled Business profession Business skilled Clerical profession Clerical skilled Arts profession Arts skilled Service profession Service skilled Consumer economics Outdoor

0.447 0.292 0.261

0.051 0.051 0.044 0.050 0.036 0.045 0.041 0.041 0.044 0.037 0.045 0.057 0.050 0.055 0.040 0.051 0.038 0,047 0.049 0.045 0.037 0.046

0.107 0.072 0.043 0.035 0.108 0.055 0.085 0.078 0.093 0.071 0.087 0.057 0.057 0.052

WRIOT Art Literature Music Drama Sales Management Office work Personal service Protective service Social service Social science Biological science Physical science Number Mechanics Machine operation Outdoor Athletics Sedentariness Risk Ambition

Valpar Valpar # 1 Valpar #2 Valpar # 4-Dom Valpar # 4-0th Valpar # 4-Dis Valpar # 4-Tot Valpar # 8 Valpar # 9

AAMD Automotive Building trades Animal care Janitorial Materials handling Clerical Food service Patient care Horticulture Personal service Laundry service

0.028 0.042 0.076 0.061 0.070 0.024 0.046 0.077 0.062 0.039 0.045 0.021 0.016 0.035 0.054 0.077 0.069 0.033 0.046 0.053 0.027

0.140 0.101 0.146 0.145 0.141 0.095 0.111 0.122

0.067 0.061 0.109 0.087 0.042 0.106 0.100 0.116 0.106 0.113 0.093

to 8.545. Before scaling the overall index, the subjective weights and objective weights of the Valpar Work Samples Index should be combined to obtain an overall weight for each elementary test in the Valpar system (Table V).

The overall scales of average normal person and visually impaired person for each evaluation system and sub-index are presented in Table VI. The resultant DI is then obtained as

0'425 - 0.838 83.8% Disability Index = weighted overall score of visually impaired individual - weighted overall score of average normal individual 0.507


Table V. Pairwise comparison matrix and weights for Valpar Work Sample evaluation system ~~ ~

Valparl Valpar2 Vp4Dom Vp40th Vp4Dis Vp4Tot Valpar8 Valpar9

Valpar 1 Valpar2 Valpar4Dom Valpar40th Valpar4Dis Valpar4Tot Valpar8 Valpar9

1 .000 0.333 0.200 0.143 0.250 0.200 0.167 0.333

3.000 1 .000 0.333 0.200 0.500 0.333 0.250 2.000

5.000 3.000 1 .000 0.333 5.000 1 .000 0.500 3.000

7.000 5.000 3.000 1 .000 4.000 3.000 2.000 5.000

4.000 2.000 0.200 0.250 1 .000 0.250 0.200 0.500

5.000 3 .000 1 .000 0.333 4.000 1 .000 0.500 5.000

6.000 4.000 2.000 0.500 5.000 2.000 1 .000 4.000

3.000 0.500 0.333 0.200 2.000 0.200 0.250 1 .000

Subjective weight 0.342 0.152 0.055 0.023 0.169 0.054 0.037 0.162 Objective weight 0.140 0.101 0.146 0.145 0.141 0.095 0.111 0.122 Overall weight 0.374 0.120 0.063 0.031 0.186 0.040 0.032 0.155

~

&lax 8.545 Consistency index 7.78%

Upper bound 0.368 0.156 0.055 0.027 0.169 0.053 0.037 0.162 Lower bound 0.342 0.152 0.050 0.020 0.169 0.042 0.028 0.162 Recommended 0.355 0.154 0.053 0.023 0.169 0.048 0.032 0.162

Table VI. Disability Index and weighted sub-indexes for each sub-index and evaluation system

Sub-index and evaluation system Average normal Visually impaired person

Achievements Index WRAT system

Interests and Opinions Index WRIOT system AAMD system COPS system

Temperaments Index MDC behaviour system

Work Samples Index Valpar system

0.500

0.394 0.296 0.304

0.750

0.500

0.395

0.486 0.396 0.245

0.823

0.375

Weighted index 0.507 0.425 ~

Disability Index 1 0.838

Table VII. Industrial jobs and productivities

Industrial job Productivity

Pen clipping 74%

Plumbing coupling assembly and disassembly Pen packaging 102%

Long-screw pegboard assembly and disassembly 50% 88%

U-bolt assembly 115%

Five industry assembly jobs have been performed by the client and their productivities are measured and recorded (Table VII). The average performance of the client is 85.8% while the predicted performance from the DI is 83.8%. This value demonstrates the predictability of the developed DI.


4. CONCLUDING REMARKS

The DI provides an overall measurement for evaluating individual residual capacities of the visually impaired in performing industrial assembly tasks compared with those performed by normal individuals. Several ways to enhance the validity of the DI are available, namely the inclusion of other existing vocational assessment and evaluation systems (e.g. Career Evaluation System, Skills Assessment Module) or having several evaluators judge the relative importance of each elementary test and yield a consensus on the specific test category.

A visually impaired individual’s index from a single test system may be greater than that of normal people. This can be found especially in the Interests and Opinions Test category and Behavior Test category. Bauman (1973) points out that there is a marked tendency among visually handicapped persons to make many choices in the social service areas and in music. This occurrence may lead to a sub-index value of visually impaired of that specific test group that is greater than average normal individuals.

In this study we used random numbers with a normal distribution to serve as the normal people’s performance data. Though the validity of the data may be questionable and may cause some variations in obtaining objective weights at each run, the overall index for normal people is quite consistent, with a variation of less than 1%.

The subjective weights obtained through pairwise comparisons by evaluators may be different if the evaluators are from different fields or target different job categories. Evaluators with psychology backgrounds or who work at shelters may weigh one of the functional characteristics higher than the others. For instance, they may consider that the client’s work and work-related behaviours (e.g. work attitude, work relationship with co-workers) are more important than the worker’s motor abilities (e.g. dexterity, muscle power). On the other hand, evaluators with engineering backgrounds or who are located in workshops may think that motor abilities are more important than other characteristics. Thus the derived DI created by two different groups of evaluators will be different. This would raise the question as to which one will be more reliable. As we pointed out in Section 1, our definition of the DI is job-category-dependent. The relative importance of the characteristics given by evaluators will vary against different job categories, but it should be consistent under the same job category. Since the evaluators will be targeting different objectives, a discrepancy should exist. In this study we did not intend to develop an index representing an individual’s ability for all purposes, yet we preserved the flexibility for evaluators (i.e. decision makers with different backgrounds) to determine what the objectives of the assessment should be (i.e. what work is expected to be accomplished by these functionally impaired people).

In cases where multiple experts are consulted, group decision-making procedures can be implemented and yield a resolution for deriving the Disability Index. Hwang and Lin (1987) provided a thorough survey on this topic. Although it is not a necessary criterion, it is preferred that a consensus of judgment be reached when assessing the same job functions.

Beyond the stage of developing the Disability Index, some questions should be asked. What are the implications of the DI? How is the DI to be interpreted? How else can the DI help to identify potential restrictions of the visually disabled worker as well as the need for work environment modifications. The rehabilitation staff of the Texas Commission for the Blind adopted a measure to determine clients’ vocational levels during the initial year of referral (Dial et al., 1991). Chen (1992) proposed an ergonomic and expert system approach to answer the question of workplace modification. Currently we are working on the integration of the DI and VITAL (Chen, 1992). The derived DI is used to identify potential restrictions of the disabled worker and reasonable accommodations to meet the needs of an individual’s deficits are recommended by an ergonomic expert system module.


In conclusion, the integration of our proposed DI through MADM and ergonomic expert system approaches may match worker skills to job requisites effectively and ultimately provide real assistance to those agencies who seek to obtain productive employment of visually impaired workers.

ACKNOWLEDGEMENTS

This research was supported by Texas Higher Education Coordinating Board under Grant 003652001-ARP. The authors also gratefully acknowledge the Lighthouse of Houston for their support.

REFERENCES

Bauman, M. K., ‘An interest inventory for the visually handicapped’, Educ. Visually Handicapped, 5 ,

Becker, R. L., AAMD Becker Interest Znventory, American Association on Mental Deficiency, 1982. Birdsong, J. H., ‘MTM and rehabilitation: a combination for potential profits’, J. Methods-Time Meas.,

Brown, D. S., ‘Quality through equality: using TQM to hire and retain workers with disability’, Znd. Eng.,

Chen, J.-G., ‘Development of vision impaired task and assignment lexicon via an ergonomics and expert system approach’, Znt. J. Znd. Ergon., 10, 225-240 (1992).

Department of Education, ‘Proposed funding priorities for the National Institute on Disability and Rehabilitation Research for fiscal years 1991-1992; notice’, Fed. Regist. Part VZ, 55(129), 27,786-27,793 (1990).

Department of Education, ‘Special projects and demonstrations for providing vocational rehabilitation services to individuals with severe handicaps’, Fed. Regist., Part ZZ, 57(214), 52,682-52,686 (1992).

Dial, J. G., Chan, F., Mezger, C., Parker, H. J., Zangla, K., Wong, D. W. and Gray, S., ‘Comprehensive vocational evaluation system for visually impaired and blind persons’, J. Visual Impair. Blindness, 84,

Froberg, D. G. and Kane, R. L., ‘Methodology for measuring health-state preferences-I: measurement

Froberg, D. G. and Kane, R. L., ‘Methodology for measuring health-state preferences-11: scaling methods’,

Froberg, D. G. and Kane, R. L., ‘Methodology for measuring health-state preferences-111: population

Grayson, D., Facts about Blindness and Visual Impairment, New York: American Foundation for the

Green County School System, Apticom Assessment, Greenville City, TN: Green County School System, 1986. Hwang, C.-L. and Lin, M.-J., Group Decision Making under Multiple Criteria: Methods and Applications,

Hwang, C.-L. and Yoon, K. S., Multiple Attribute Decision Making: Methods and Applications, Berlin:

Jastak, S. and Jastak, J. F., Wide Range Interest Opinion Test Administration Manual, Wilmington, DE:

Jastak, S and Wilkinson, G. S., Wide Range Achievement Test Administration Manual, Wilmington, DE:

Knapp, R. R. and Knapp, L., COPS Interest Inventory, Educational and Industrial Testing Service, 1982. Martin, N. F. G. andEngland, J. W., Mathematical TheoryofEntropy, Reading, MA: Addison-Wesley, 1981. Materials Development Center, MDC Behavior Ident~ication Format, Menomonie, WI: Materials

Milas, G. H., ‘IE’s role in implementing the Americans with Disabilities Act’, Znd. Eng., 24(1), 36-39 (1992). Nijkamp, P., ‘Stochastic quantitative and qualitative multicriteria analysis for environmental design’, Papers

Rahimi, M. and Malzahn, D. E., ‘Task design and modification based on physical ability measurement’,

78-83 (1973).

17(4), 3-8 (1972).

25(1), 60-62 (1993).

153-157 (1991).

strategies’, J. Clinical Epidemiology, 42, 345-354 (1989a).

J. Clinical Epidemiology, 42, 459-471 (1989b).

and context effects’, J. Clinical Epidemiology, 42, 585-592 (1989~).

Blind, 1986.

Heidelberg: Springer, 1987.

Springer, 1981.

Jastak Associates, 1979.

Jastak Associates, 1984.

Development Center, University of Wisconsin-Stout, 1974.

Region. Sci. Assoc., 39, 175-199 (1977).

Human Factors, 26(6), 715-126 (1984).

176

Saaty, T. L., ‘A scaling method for priorities in hierarchical structures’, J. Math. Psychol., 15(3), 234-281

Saaty, T. L., ‘How to make a decision: the Analytic Hierarchy Process’, Eur. J. Oper. Res., 48(1), 9-26

Shannon, C. E. and Weaver, W., The Mathematical Theory of Communication, Urbana, IL: University

Tompkins, N. C., ‘ADA redefines doctrine on disability diagnosis’, Occup. Health Safety, 61(7), 34-36

Valpar Corporation, Valpar Component Work Sample Manual: For B-Kit, Tucson, AZ: Valpar Corporation,

Wind, Y. and Saaty, T. L., ‘Marketing application of the Analytic Hierarchy Process’, Manag. Sci., 26,

Yoon, K., ‘A reconciliation among discrete compromise solutions’, J. Oper. Res. SOC., 38, 277-286 (1987). Zeleny, M., Multiple Criteria Decision Making, New York: McGraw-Hill, 1982.


(1977).

(1990).

of Illinois Press, 1947.

(1993).

1980.

641-658 (1980).

a multiple-attribute decision-making approach to assess the disability of visually impaired workers

Documents