a 360-degree performance appraisal model dealing with...
TRANSCRIPT
A 360-Degree Performance Appraisal ModelDealing with Heterogeneous Information and
Dependent Criteria
M. Espinilla, R. de Andres ∗, F.J. Martınez, and L. MartınezDepartment of Computer Sciences,
University of [email protected],[email protected],[email protected]
∗Department of Economic and Economic History,University of Salamanca
December 29, 2011
Abstract
360-degree performance appraisal process is used as a tool that provides anevaluation about employees performance. It is based on the opinion of differentgroups of reviewers who socialize with evaluated employees. Many criteria mustbe considered in performance appraisal that may have different nature and usuallypresent uncertainty. Therefore, it seems necessary and appropriate to provide a het-erogeneous framework adapted to the nature of the criteria. In this context, criteriacan be asses by reviewers, according to their background and degree of knowledgeabout evaluated employees. Furthermore, the interaction among criteria, as wellas reviewers weights should be taken into account to ensure an effective aggrega-tion process of the information, providing interpretable and understandable evalu-ations. In this paper, we present an integrated model for 360-degree performanceappraisal that can manage heterogenous information and computes a final linguis-tic evaluation for each employee, applying an effective aggregation that considersthe interaction among criteria and reviewers weights. We also present a real casestudy to show the usefulness and effectiveness of the proposed model.
1 IntroductionPerformance appraisal is a key tool in companies that provides information about em-ployees performance in order to make important decisions, such as salary adjustments,promotions, identification of training and development needs, documentation of per-formance levels or behaviors that may merit firing or sanctions. Furthermore, there isan evident link among performance appraisal and attitudes, efforts and behaviors of
1
the employees, that implies improvements in the financial results obtained by compa-nies [12].
Currently, performance appraisal process is based on the opinion of different groupsof reviewers who socialize with evaluated employees, since they can truly respond tohow an employee develops his/her job. Moreover, the process includes the opinion ofemployee about her/himself (see Figure 1). This kind of performance appraisal processis so-called 360-degree appraisal or integral evaluation [10, 12, 24] and it overcomessome disadvantages from traditional evaluation such as lack of objectivity, prejudice orhalo errors, etc. [1, 12].
Figure 1: Points of view in360-degree performance ap-praisal
The decision analysis [7] has been successfully ap-plied to 360-degree performance appraisal process andothers areas, such as sensory evaluation [25, 26] or,sustainable energy [19]. In this way, we can find inthe literature, performance appraisal models proposedin [2, 12], where the set of criteria are evaluated in aquantitative way, providing numerical results. This factimplies a lack of accuracy in the performance appraisalprocess, due to the difficulty of expressing in a pre-cise way, qualitative criteria and subjective judgments.Moreover, a numerical domain is not a very suitable ex-pression domain for evaluation results, since these re-sults should be interpreted in a easy way by differentmembers of the company. Performance appraisal mod-els presented in [8, 9] provide a flexible linguistic eval-uation framework in which criteria are assessed in multiple linguistic scales, accordingto the degree of knowledge of the reviewers, providing results easy to interpret in alinguistic domain. However, the main issue of these models is that they do not considerquantitative criteria, which are very usual in performance evaluation.
All the previously mentioned models fall into two classes that present either a strictevaluation framework or uninterpretable results. Additionally, these models only takeinto account a set of independent criteria, disregarding the possible interaction amongthem, which is very common in these processes. In order to obtain a successful 360-degree performance appraisal process to make right decisions in the company, threemain aspects should be considered:
1. The evaluation framework should be flexible, such that the reviewers can pro-vide their judgments within different domains (numerical, interval-valued andlinguistic), according to the uncertainty and nature of criteria and the backgroundof each reviewer, i.e., a heterogeneous evaluation framework.
2. The application of an adequate set of aggregation operators that take into accountthe interaction among criteria and reviewers weights.
3. The evaluation of the results should be close to human natural language in orderto be understandable and interpretable by multiple members of the company.
In this paper, we present an integrated model for 360-degree performance appraisalbased on the decision analysis scheme [7] that can manage heterogenous information
2
and computes a final linguistic evaluation for each employee, considering the inter-action among criteria and reviewers weights. To do so, we propose the use of thelinguistic 2-tuple representation model [15, 16] and its extended approach to deal withheterogeneous information [17], which is based on the unification of the heterogeneousinformation into linguistic information, thus facilitating understandable results. To ob-tain results close to human beings and taking into account the linguistic modeling of theunified information, we shall implement processes of computing with words (CWW)[28, 29], by using a multi-step aggregation process, considering the ordered weightedaverage operator, the weighted arithmetic average operator and the Choquet integralas aggregation operators, because they take into account reviewers weights and the in-teraction among criteria. Finally, the paper shows a real case study to illustrate theusefulness and effectiveness of the proposed model.
The paper is organized as follows. Section 2 describes the relationship betweendecision analysis and evaluation problems. Section 3 reviews a CWW approach basedon linguistic 2-tuple model to deal with heterogeneous information. Section 4 presentsa novel heterogeneous model for 360-degree performance appraisal. Section 5 showsa real case study of the proposed model. Finally, in Section 6 some conclusions arepointed out.
2 Decision Analysis and Evaluation ProcessDecision analysis [7] is a discipline, belonging to decision theory, that has been suc-cessfully applied to solve evaluation problems in some different fields such as sustain-able energy [18, 30, 31], sensory evaluation [25, 26], quality of service [5, 13] or digitallibraries [3], engineering systems [27] and new product development [21, 22].
The goals of an evaluation process is to compute a set of overall assessments thatsummarizes the information gathered and to provide useful information about the setof evaluated elements. To do so, it is necessary to establish a set of evaluated elementsin the evaluation framework, gather the information and finally, compute a final assess-ments for each element. It is therefore obvious that decision analysis is an excellenttool for evaluation, since it includes a wide variety of methods for the evaluation of aset of alternatives, considering all relevant criteria in a decision problem and involvingexperts [23].
Make a decision
Decision Making
Identify decision
Identify alternatives
FrameworkRating
alternativesGathering information
Decision Analysis
Sometimes: Emotional wayRational way
Identify objetives
Figure 2: Decision making scheme
3
Thereby, decision analysis methods provide a rational analysis in a simple andquick way, since the highest final assessment would normally correspond to the bestevaluated element [20]. A classical decision analysis scheme consists of the followingphases (see Figure 2 [7]):
• Identify decision, objectives and alternatives of the problem.
• Framework: It defines the structure of the problem and the expression domainsin which the preferences can be assessed.
• Gathering information: Decision makers provide their information.
• Rating alternatives: This phase obtains a collective assessment for each alterna-tive.
• Make a decision.
The application of the decision analysis scheme to an evaluation process does notnecessarily involve all phases. The essential phases regarding an evaluation problemthat will be used in our proposal are: establishing framework, gathering informationand rating the alternatives.
3 Computing with words in a Heterogenous FrameworkCWW is a methodology in which objects of computation are words or sentences from anatural language. It emulates human cognitive processes to make reasoning processesand decisions in environments of uncertainty and imprecision [37]. Zadeh introducedthe concept of linguistic variable [36] in order to deal with linguistic variables is neces-sary to choose the appropriate linguistic descriptors for the term set and their semantics.There are different possibilities to carry out these selections (see [34, 36]).
The CWW methodology considers that inputs and output results should be ex-pressed in a linguistic domain in order to be close to human natural language, provid-ing interpretable and understandable results [28, 29]. This fact is key in performanceappraisal where evaluation results are used to make hard decisions by the human re-sources department. The linguistic 2-tuple model [15, 16] and its extended approaches[11, 14, 17], providing methods to deal with complex decision making problems, fol-lowing the CWW methodology.
In our proposal, we consider a heterogenous framework, in which the reviewerscould use numerical, linguistic and interval-valued information. In this section, wereview the extended approach to deal with heterogeneous information presented in [17]based on the linguistic 2-tuple representation model [15, 16]. Figure 3 shows a basicscheme of this approach, whose steps are further described below.
3.1 Unification ProcessThis step is based on the unification of heterogeneous information into linguistic infor-mation, thus facilitating comprehensive results for decision makers and it consists ofthe following states:
4
Heterogeneous
Framework
Linguistic Values
Numerical Values in [0,1]
Interval Values in [0,1]
Fuzzy Sets
in the S
UNIFICATION
PROCESS
[0,g] (si, i)
AGGREGATION
PROCESS
Linguistic 2-tuple
Computational
Model
(sk, k),sk S
s0sg
Transformation
into Fuzzy Sets
Transformation
into Linguistic 2-tuples
APPROACH BASED ON FUSION OF THE INFORMATION
Methodology CWW
TNS
TIS
TSS
S
Figure 3: Approach based on Information Fusion into Linguistic Values
1. Transformation of the information into Fuzzy Sets in S.
The heterogeneous information will be unified into a specific linguistic domain,called Basic Linguistic Terms Set and noted as S = {s0, s1, . . . , sg}, that isselected with the aim of keeping as much knowledge as possible (see [17]). Eachreal number, interval value and linguistic value is then transformed into a fuzzyset on S, F(S), using an adequate transformation function for its expressiondomain [17].
(a) Numerical domain. When v ∈ [0, 1], a numerical transformation function,φNS : [0, 1] → F (S), is defined by:
TNS(v) =
g∑i=0
(si/γi) (1)
where γi = µsi(v) ∈ [0, 1] is the membership degree of v to si ∈ S.
(b) Interval domain. When v ∈ P ([0, 1]), an interval transformation function,φIS : P ([0, 1]) → F (S), is defined by:
TIS(v) =
g∑i=0
(si/γi) (2)
where γi = maxy
min{µI(y), µsi(y)}, i = 0, . . . , g, and
µI(y) =
0 if y < d,1 if d ≤ y ≤ e,0 if y > e.
(3)
being y ∈ [0, 1].
5
(c) Linguistic domain. Being v ∈ S, such that S = {sj , j = 1 . . . , h} andh ≤ g, a linguistic transformation function, φSS : S → F (S), is definedby:
TSS(v) =
g∑i=0
(si/γi) (4)
where γi = maxy
min{µsj (y), µsi(y)}, i = 0, . . . , g.
2. Transformation of fuzzy sets, F (S), into Linguistic 2-tuples in S.
The previous fuzzy sets are conducted into linguistic 2-tuples, which facilitatethe processes of CWW and produce interpretable results [15, 16]. The linguistic2-tuple representation model is based on the concept of symbolic translation[15] and represents the linguistic information through a 2-tuple (s, α), where sis a linguistic term and α is a numerical value representation of the symbolictranslation [15].
Thereby, being β ∈ [0, g] the value generated by a symbolic aggregation opera-tion, we can assign a 2-tuple (s, α) that expresses the equivalent information ofthat given by β.
Definition 1 [15]. Let S = {s0, . . . , sg} be a set of linguistic terms. The 2-tuple set associated with S is defined as ⟨S⟩ = S × [−0.5, 0.5). We define thefunction ∆S : [0, g] −→ ⟨S⟩ given by,
∆S(β) = (si, α), with{
i = round (β),
α = β − i,(5)
where round assigns to β the integer number i ∈ {0, 1, . . . , g} closest to β.
We note that ∆S is bijective [15, 16] and ∆−1S : ⟨S⟩ −→ [0, g] is defined by
∆−1S (si, α) = i + α. In this way, the 2-tuples of ⟨S⟩ will be identified with the
numerical values in the interval [0, g].
As aforementioned, fuzzy sets are transform into linguistic 2-tuples in the S atthis stage by using the function χ defined as:
Definition 2 [17]. Given the linguistic term set S = {s0, s1, . . . , sg}, the func-tion χ : F(S) −→ ⟨S⟩ is defined by
χ ({γ0, γ1, . . . , γg}) = ∆S
g∑
j=0
j γj
g∑j=0
γj
= (si, α) ∈ S. (6)
6
3.2 Aggregation ProcessThe linguistic 2-tuple representation model has a linguistic computing model associ-ated [16], that accomplish processes of CWW in a precise way. Different aggregationoperators have been proposed for linguistic 2-tuple [16, 35], in our proposal we willuse the ordered weighted average operator, the weighted arithmetic average operatorand the Choquet integral for linguistic 2-tuple that will be defined when they will beused.
4 360-Degree Performance Appraisal ModelOur aim in this paper is to present a novel integrated model for 360-degree performanceappraisal based on a decision analysis scheme with a flexible framework in which re-viewers can express their judgments in different domains, providing linguistic results,considering the interaction of the criteria and reviewers weights.
To do so, the 360-degree performance appraisal model consists of the phases shownin Figure 4 that cover the essential phases of decision analysis [7]. The followingsubsections present in further detail these phases of the performance appraisal model.
HeterogeneousEvaluation Framework
Numerical
Interval
Linguistic
RatingProcess
Multi-step aggregation
Gathering Information
Supervisors
CustomersCollaborators
Employee
Unification of the
information
Figure 4: 360-degree performance appraisal model
4.1 Heterogeneous Evaluation FrameworkIn this phase, the evaluation framework is determined to fix the structure of the 360-degree performance appraisal process. It is necessary to define the main features andterminology of this process in which personnel are evaluated from different points ofview, using multiple expression domains.
7
Let us suppose there is a set of employees X = {x1, . . . , xn} to be evaluated bythree collectives: a set of supervisors (executive staff) A = {a1, . . . , ar}, a set ofcollaborators (fellows) B = {b1, . . . , bs} and a set of customers C = {c1, . . . , ct}.Employees will be evaluated attending to a set of criteria Y = {Y1, . . . , Yp}.
The assessments provided by the members of the collectives ai ∈ A, bi ∈ B andci ∈ C about the employee xj , according to the criterion Yk are denoted by aikj ,bikj and cikj , respectively. Moreover, xjk
j is the assessment of xj about himself withrespect to Yk. Therefore, there are (r + s+ t+ 1) p assessments, maximum, for eachemployee provided by different collectives and the employee himself.
Thereby, we consider an heterogeneous evaluation framework in which each re-viewer of the different collectives can use different domains to assess each criterionY k, k = 1, . . . , p, depending on either the nature of the criteria or the reviewer’s back-ground, their degrees of knowledge, their perceptions and feelings about the evaluatedemployees. Our model includes in the heterogeneous evaluation framework: numericalvalues, v ∈ [0, 1], interval-values, v ∈ P ([0, 1]), and linguistic values with differentgranularity v ∈ S = {s0, . . . , sh}.
4.2 Gathering InformationOnce the framework has been defined, the reviewers of the different collectives providetheir judgment in a numerical, interval or linguistic domain, according to the evaluatedemployees, xj , j = {1, ..., n} and the evaluated criteria Yk, k = {1, ..., p}. The opin-ions of each collective are provided by means of assessment vectors: {ai1j , ..., aipj }with i ∈ {1, ..., r} for supervisors, {bi1j , ..., bipj } with i ∈ {1, ..., s} for collaborators,{ci1j , ..., cipj } with i ∈ {1, ..., t} for customers, and finally, {xj1
j , ..., xjpj } for the evalu-
ated employee, where ˆ represents an assessment in an expression domain (numerical,interval-valued or linguistic), according to the uncertainty and nature of criteria and thebackground of each reviewer.
4.3 Rating ProcessThe aim of this process is to obtain a linguistic global assessment for each employee,easy to understand and interpretable to make right decisions by the company.
To do so, we propose the application of the approach reviewed in Section 3. Firstthe heterogenous information is unified into linguistic 2-tuple on the S, second a globalassessment is computed for each employee in a multi-step aggregation process, andfinally, a classification and a raking of employees are provided. The following sub-sections present in detail the rating process phases that were graphically described inFigure 4.
4.3.1 Unification of Heterogeneous Information
According to the approach reviewed in Section 3, the heterogeneous information isconducted in a linguistic domain to carry out the CWW processes, providing linguisticresults.
8
To do so, we fix the unification domain, i.e., the basic linguistic term set, S. Theheterogeneous information is then conducted by fuzzy sets on the S according to theexpression domain, by using the respective transformation functions (see equations (1),(2) and (4)). In order to facilitate the understandability of the results, the fuzzy sets aretransformed in S into linguistic 2-tuples, ⟨S⟩, by using the equations (5) and (6) (seeFigure 3).
In this way, the heterogeneous information provided by the different collectives hasbeen already unified into linguistic 2-tuples in S:
• {ai1j , ..., aipj }, i ∈ {1, ..., r} with ai1j = (si, αi)i1j ∈ ⟨S⟩.
• {bi1j , ..., bipj }, i ∈ {1, ..., s} with bi1j = (si, αi)i1j ∈ ⟨S⟩.
• {ci1j , ..., cipj }, i ∈ {1, ..., t} with ci1j = (si, αi)i1j ∈ ⟨S⟩.
• {xj1j , ..., xjp
j } with xj1j = (si, αi)
j1j ⟨S⟩.
4.3.2 Multi-step Aggregation Process
The aim of the performance appraisal process is to obtain a global assessment for eachevaluated employee that summarizes her/his weaknesses and strengths, considering theinteraction among criteria and reviewers weights. These assessments will be expressedin a linguistic domain that facilitates their interpretation and understandability.
Collective criteria values
Multi-Step Aggregation Process
Supervisors Collaborators
Customers Employees
Global criteria values
Global values
Step 1
Step 2
Step 3
OWA operator
Weighted averageoperator
Choquet Integral
Lin
gu
istic 2
-tu
ple
Figure 5: Steps of multi-step aggregation process
To do so, our integrated model proposes the aggregation of the unified informationinto linguistic 2-tuples by means of a multi-step aggregation process [4], in which anadequate set of aggregation operators is used, according to the needs of the aggregationphase (see Figure 5).
A point worthy of mention is that usually the opinions of employees about them-selves are not include in the aggregation process because can bias the result of theprocess. These opinions are used by companies in the rating phase with the purposeof improving their performance and synchronizing companies goals with employeesgoals (see Figure 4). This part of the performance appraisal procedure is so-called“feedback” and coaching process [12]. Following, we present in further detail eachstage of the multi-step aggregation process.
9
4.3.2.1 Computing collective criteria values, vk(xj). For each reviewers’ collec-tive, their assessments about a given criterion Yk are aggregated by means of the 2-tupleOWA operator, Gw. We suggest this operator because it does not distinguish the originof the assessments, since it is anonymous. Therefore, we use it in the first step of themulti-step aggregation process.
Definition 3 [15]. Let ((s1, α1), . . . , (sm, αm)) ∈ ⟨S⟩m be a vector of linguistic 2-tuples, and w = (w1, . . . , wm) ∈ [0, 1]m be a weighting vector such that
∑mi=1 wi =
1. The 2-tuple OWA operator associated with w is the function Gw : ⟨S⟩m −→ ⟨S⟩defined by
Gw((s1, α1), . . . , (sm, αm))= ∆S
(m∑i=1
wi β∗i
)= (sl, αl) ∈ ⟨S⟩,
where β∗i is the i-th largest element of
{∆−1
S(s1, α1), . . . ,∆
−1
S(sm, αm)
}.
Under these assumptions, the supervisors’ assessment about criterion k, regardingthe employee xj , vkA(xj) is computed by the function Gw
A,k : ⟨S⟩r → ⟨S⟩ defined by
vkA(xj) = GwA,k(a
1kj , ..., arkj ) ∈ ⟨S⟩
where aikj ∈ ⟨S⟩ for i = 1 . . . r are linguistic 2-tuples belong to S.In the same way, it is possible to obtain the collective criteria assessments about
criterion Yk for the others reviewers’ collectives: vkB(xj) (collaborators), vkC(xj) (cus-tomers), and vkX(xj) (evaluated employees themselves).
4.3.2.2 Computing global criteria values, vk(xj). In this step of the multi-stepaggregation process, the previous collective assessments for each collective and eachcriterion are aggregated by means of a 2-tuple weighted average operator to obtain aglobal value for each criterion and for each employee. We propose this operator toaggregate the information because it allows companies to establish different weightsfor each reviewers’ collective, taking into account their knowledge about the evaluatedcriteria and their significance in performance appraisal process.
Definition 4 [15]. Let ((s1, α1), . . . , (sm, αm)) ∈ ⟨S⟩m be a vector of linguistic 2-tuples, and w = (w1, . . . , wm) ∈ [0, 1]m be a weighting vector such that
∑mi=1 wi =
1. The 2-tuple weighted average operator associated with w is the functionFw : ⟨S⟩m −→ ⟨S⟩ defined by
Fw((s1, α1), . . . , (sm, αm))= ∆S
(m∑i=1
wi ∆−1S (si, αi)
)= (sl, αl) ∈ ⟨S⟩.
The collective assessments then obtained in the previous step, vkA(xj), vkB(xj) andvkC(xj) for every k ∈ {1, . . . , p}, are aggregated by Fw
k : ⟨S⟩3 −→ ⟨S⟩, wherew = (wA, wB ,wC) and
∑w− = 1, obtaining a global value for each criterion Yk
modeled by a linguistic 2-tuple:
vk(xj) = Fwk
(vkA(xj), v
kB(xj), v
kC(xj)
)∈ ⟨S⟩.
10
4.3.2.3 Computing a global value, v(xj). Eventually, companies work out a globalvalue v(xj), for each employee regarding all criteria, taking into account the interac-tion among them. This value is obtained by aggregating the global criteria valuesrelated to each employee xj ∈ X , by means of a Choquet integral [6] that considersthe mutual interaction among criteria.
The use of the Choquet integral requires the computation of the coefficients thatrepresent criteria relations, i.e, fuzzy measures [32]. In our model the relations amongcriteria are fixed by company’s Human Resource Department.
Definition 5 [32]. Let N = {1, . . . ,m} be a set of m criteria. A fuzzy measure is aset function ϕ : N −→ [0, 1] that satisfies the following conditions:
• ϕ(∅) = 0, ϕ(N) = 1
• ϕ(S) ≤ ϕ(T ) whenever S ⊆ T (ϕ is monotonic).
Definition 6 [35]. Let ((s1, α1), . . . , (sm, αm)) ∈ ⟨S⟩m be a vector of linguistic 2-tuples, N be the set of attributes and ϕ be a fuzzy measure on [0, 1], the linguistic2-tuple Choquet integral is given by Mϕ : ⟨S⟩m −→ ⟨S⟩
Mϕ ((s1, α1), . . . , (sm, αm)) = ∆S
(n∑
i=1
[(ϕ(H)σ(i))− (ϕ(H)σ(i−1))
]∆−1
S (si, αi)
),
where (σ(1), σ(2), . . . , σ(n)) is a permutation of (1,2,. . . ,n) such that ((sσ(1), ασ(1)) ≥(sσ(2), ασ(2)) ≥ . . . ≥ (sσ(m), ασ(m))), being xσ(i) the criterion corresponding to the(sσ(1), ασ(1)) and (H)σ(i) = {xσ(k)|k ≤ i} for i ≥ 1 with (H)σ(0) = ∅.
Considering the previous definitions, the multi-step aggregation process is carriedout as follows:
v(xj) = Mϕ
(v1(xj), v
2(xj), . . . , vp(xj)
)∈ ⟨S⟩.
The global results obtained in each step of the multi-step aggregation process,vkA(xj), vkB(xj), vkC(xj), vk(xj), for k = 1, . . . , p, and v(xj), are used either forsorting and ranking the employees or to establish the companies’ policy in the rankingphase.
In our integrated model, linguistic results are provided at all the stages. Theseresults are easy to understand and interpret as required by the human resources depart-ment.
4.3.3 Ranking Phase
Usually, the global assessments are used by the company for sort and rank the em-ployees for applying some specific policy. For this fact, we propose this phase in ourintegrated model in which we can sort employees in each step of the multi-step aggre-gation process. Furthermore in this phase, the opinions of employees about themselvesare used in order to improve their performance and synchronizing companies goalswith employees goals (feedback and coaching process) [12].
11
Thus, the employees’ ranking is obtained by the comparison of the correspondinglinguistic 2-tuple in S, according to the ordinary lexicographic order presented in [15].It provides a complete ranking of the set of evaluated employees from the best to theworst according to their computed assessments in each step.
5 Case StudyIn this section, we show a real case study to show the usefulness and effectivenessof the proposed integrated model that can manage heterogenous evaluation frameworkand provides linguistic assessments in order to be close to human natural language,while considering the interaction among criteria and reviewers weights.
5.1 Heterogeneous Evaluation FrameworkWe have developed our study on a well-known multinational clothing company, whicheach month carries out an integral evaluation process over their employees in eachstore. Our case study involves opinions from supervisors, collaborators/co-workers,customers and employees themselves. Specifically, the reviewers’ collectives are thefollowing:
• A set of supervisors that, in the case of this company, is made up of three subset:
1. The area manager subset: AA = {aA1 }.
2. The direct supervisors subset: AD = {aD1 , aD2 , aD3 }.
3. The non-direct supervisors subset: AND = {aND1 , aND
2 , aND3 }.
Therefore, the collective of supervisors reviewers is defined as A = AA∪AD
∪AND
where AA ∩AD ∩AND = ∅.
• A set of collaborators or/and co-workers consists of:
1. A subset of selling employees: BS = {bS1 , bS2 , bS3 , bS4 , bS5 , bS6 , bS7 }.
2. A subset of non-selling employees: BNS = {bNS1 , bNS
2 , bNS3 , bNS
4 }.
Consequently, the set of collaborators/co-workers reviewers is defined asB = BS
∪BNS , where BS ∩BNS = ∅.
• And finally, a set of customers: C = {c1, . . . , c20}. Obviously, such amount isnot the total of customers that visited the shop for a month. We have selectedtwenty-representative of them with the purpose of simplifying the case study.
In this company, the employee’s performance is characterized by eleven criteriaY = {Y1, . . . , Y11}, which are shown below:
• Y1: Productivity. This criterion is the amount of employee’s sales divides amongthe employee’s hour worked.
• Y2: Level of sales. This criterion is the employee’s sales amount.
12
• Y3: Average closing time. After the closing time, each employee has to arrangea specific shop zone. This criterion measures how it takes employee to arrangeher/his zone.
• Y4: Identification with the company. Identification with the product that com-pany sales. Think company’s values. Trust and defend the company. Like fash-ion.
• Y5: Training interest. Positive attitude to learn and improve.
• Y6: Work tasks. Open truck and alarmed. Organization of stock. Replacementof products. Coordinate and shape. Neat and clean work. Serve and sell inchanging room.
• Y7: Customers service. Available for posting. Always say hello. Thank thepurchase.
• Y8: Responsibility. Achieve daily work correctly. Not leave the shop beforefinishing work. Do tasks that are not her/his responsibility. Assume the conse-quences of her/his acts. Be always on time.
• Y9: Initiative. Get in before work needs. Propose ideas and solutions. Makedecisions within their responsibilities. Accept supervision. Help collaboratorsand co-workers.
• Y10: To serve as an example. Fulfil all the shop rules and methods. Serve as amodel for co-workers and collaborators. Be positive. Look forward solutions.Constructive attitude.
• Y11: Personal image. Dresses stylishly. Be polite. Have pleasant manners. Dresswith taste.
We can note that each group of reviewers express their opinions on selling employeesabout each criterion according to the uncertainty and nature of such criteria and thebackground of each reviewer, through a specific domain: Numerical (N), Interval (I) orLinguistic (L). Specifically in this case study, reviewers can express their linguistic as-sessments in four linguistic domains with different number of terms {S3, S5, S7, S9}.Each linguistic term set is symmetrically and uniformly distributed and its syntax is asfollows:
S3 ={Null (N), Medium (M), Perfect (P)}S5 ={Null (N), Low (L), Medium (M), High (H), Perfect (P)}S7 ={Null (N), VeryLow (VL), Low (L), Medium (M), High (H), VeryHigh (VH), Perfect (P)}S9 ={Null (N), AlmostNull (AN), VeryLow (VL), Low (L), Medium (M) High (H)
SlightlyHigh (VH), VeryHigh (VH), Perfect (P)}
Obviously, all reviewers do not evaluate all the criteria, as there are collectives that donot have sufficient knowledge to evaluate certain criteria. The collective of reviewersfor each criterion and the expression domain used for evaluating are shown in Table 1.
13
AA AD AND BS BNS C
Cost Y1 N+ - - - - -Y2 N+ - - - - -Y3 - - N+ - - -
Benefit Y4 S7 S7 S7 S5 S5 -Y5 S5 S5 S5 - - -Y6 S9 S9 S9 - - -Y7 S3 S3 S3 S3 S3 S5
Y8 S7 S7 S7 - - -Y9 S7 S7 S7 S3 S3 -Y10 S7 S7 S7 S3 S3 -Y11 S5 S5 S5 S5 S5 S3
Table 1: Reviewers’ collective for each criterion
5.2 Gathering InformationOnce the evaluation framework has been fixed, the reviewers express their opinionsabout evaluated employees. In this case study, we manage a great amount of informa-tion: seven evaluated employees, eleven criteria and 38 reviewers, being seven supervi-sors, eleven collaborators and 20 customers. Due to this fact, we omit the assessmentsprovided by the reviewers about selling employees for the set of criteria in this paper,but it can be consulted in the additional material1.
5.3 Rating ProcessAccording to the integrated proposed model in Section 4, we carry out the three stepsof the rating process.
5.3.1 Unification the Heterogeneous Information
The heterogenous information is unified into a basic linguistic terms set, ⟨S⟩. In ourcase study the ⟨S⟩ corresponds to the linguistic terms set with nine labels, S9, in orderto keep as much knowledge as possible. This domain is key in our case study becausethe global assessments will be expressed in it in order to offer results easy to interpret,following the CWW methodology.
An important aspect to consider in our case study is that criteria Y1, Y2 and Y3 arecost criteria. Therefore, preferences about these criteria must be normalized accordingto their expression domains. Once this is done, we unify the gathered information,according to the revised approach in the Section 32.
1Currently, the additional material is a document to the reviewers of the paper, but if the paper is accepted,this material will be available to interested readers in order that they can find additional information andinclude their own results.
2In the same way that the gathered information, the unified information into linguistic 2-tuple in the ⟨S⟩can be consulted in the additional material for the reviewers of the paper.
14
5.3.2 Multi-step Aggregation Process
In this stage, we compute a global assessment for each selling employee in a multi-stepaggregation process that considers an adequate set of aggregation operators in order tomeet the needs of performance assessment, according to the interaction among criteriaand reviewers weights.
5.3.2.1 Computing collective criteria values. In the first step of this process, weapply the 2-tuple OWA operator, which requires a OWA weighting vector. This vectorcan be determined by means of different ways, in our case study, the Human ResourceDepartment uses a non-decreasing linguistic quantifiers [33], specifically, the linguisticquantifier “Most”.
Table 2: Collectives’ assessments for each selling employee about each criterion
x1 x2 x3 x4 x5 x6 x7
Y1 A (V L,−.3) (L,−.5) (N, 0) (AN,−.2) (M, .4) (L, 0) (P, 0)Y2 A (V L,−.5) (L,−.5) (N, 0) (AN, 0) (H, 0) (M,−.5) (P, 0)Y3 A (P, 0) (L,−.5) (N, 0) (AN, 0) (H, 0) (M,−.5) (P, 0)Y4 A (SH, .3) (M, 0) (H, .2) (V L, .3) (M, 0) (SH,−.5) (V H,−.1)Y4 B (SH, 0) (H,−.3) (H,−.3) (AN, .4) (L, .4) (H, 0) (V H, .4)Y5 A (H,−.4) (SH,−.5) (M,−.2) (L,−.1) (V L,−.2) (L,−.1) (V H,−.1)Y6 A (SH, .3) (V H, 0) (L, 0) (L, .3) (SH, 0) (M, 0) (V H, .3)Y7 A (V H, 0) (M, 0) (M, 0) (L, 0) (M, 0) (M, 0) (P, 0Y7 B (SH, 0) (M, 0) (L,−.2) (AN, .3) (M, 0) (H, .3) (P, 0)Y7 C (M,−.4) (V L, 0) (N, 0) (V L, 0) (V L, 0) (H,−.2) (SH,−.2)Y8 A (SH,−.2) (SH,−.5) (L, .3) (L, 0) (L, .3) (V L,−.5) (P, 0)Y9 A (L, 0) (L, 0) (AN, .3) (L, .1) (SH, .1) (H, .2) (V H,−.4)Y9 B (M, 0) (H, .3) (V L, 0) (N, 0) (L,−.2) (N, 0) (V H,−.2)Y10 A (M, .2) (L,−.3) (L,−.2) (AN, 0) (AN, .3) (AN, .3) (L,−.3)Y10 B (N, .4) (M, 0) (N, 0) (N, 0) (V L, .4) (V L, .4) (SH, .4)Y11 A (SH, 0) (M,−.5) (M, 0) (V H,−.1) (M, 0) (SH,−.5) (P, 0)Y11 B (H, 0) (L,−.3) (M,−.3) (H,−.3) (L, .4) (M,−.3) (V H,−.3)Y11 C (SH,−.4) (SH,−.4) (M,−.4) (M, .4) (M, 0) (H,−.2) (P,−.4)
In this way, collectives’ assessments for each selling employee about each criterionare computed and these are shown in the Table 2.
5.3.2.2 Computing global criteria values. In the second step of this process, weapply the linguistic 2-tuple weighted average operator, which also requires a weight-ing vector. This vector is defined by the Human Resource Department, attending tothe evaluated criterion and the reviewer collective. Therefore, global criteria values ob-tained with the weights fixed by Human Resource Department are shown in Table 3.
Y4 : wA = 0.5 wB = 0.5 Y7 : wA = 0.25 wB = 0.25 wC = 0.5Y9 : wA = 0.75 wB = 0.25 Y10 : wA = 0.6 wB = 0.4Y11 : wA = 0.33 wB = 0.33 wC = 0.33
15
Table 3: Global criteria and collectives weights fixed by human resource department
x1 x2 x3 x4 x5 x6 x7
Y1 (V L,−.3) (L,−.5) (N, 0) (AN,−.2) (M, .4) (L, 0) (P, 0)Y2 (V L,−.5) (L,−.5) (N, 0) (AN, 0) (H, 0) (M,−.5) (P, 0)Y3 (P, 0) (L,−.5) (N, 0) (AN, 0) (H, 0) (M,−.5) (P, 0)Y4 (SH,−.2) (M, .3) (H,−.1) (V L,−.1) (M,−.3) (H, .3) (V H, .2)Y5 (H,−.1) (SH,−.5) (M,−.2) (L,−.1) (V L,−.2) (L,−.1) (V H,−.1)Y6 (SH, .3) (V H, 0) (L, 0) (L, .3) (SH, 0) (M, 0) (V H, .3)Y7 (H, .3) (L, 0) (V L,−.3) (V L, .1) (L, 0) (H,−.3) (V H,−.1)Y8 (SH,−.2) (SH,−.5) (L, .3) (L, 0) (L, .3) (V L,−.5) (P, 0)Y9 (L, .3) (M,−.5) (V L,−.5) (V L, .3) (H, .3) (M,−.1) (V H,−.4)Y10 (L,−.3) (L, .2) (V L,−.4) (AN,−.4) (V L,−.3) (V L,−.3) (M, .2)Y11 (SH,−.5) (M,−.1) (M,−.2) (H, .3) (M,−.2) (H,−.3) (V H, .4)
5.3.2.3 Computing a global value. This last step integrates a Choquet integral forlinguistic 2-tuple, which is able to cope with interaction among criteria. In our casestudy, fuzzy measures have been established after several meetings by company’s Hu-man Resource Department as follows:
ϕ(Y1) = 0.1 ϕ(Y2) = 0.1 ϕ(Y1, Y 2) = 0.43 ϕ(Y3) = 0.02ϕ(Y4) = 0.02 ϕ(Y5) = 0.02 ϕ(Y6) = 0.1 ϕ(Y7) = 0.1ϕ(Y8) = 0.02 ϕ(Y6, Y7, Y8) = 0.45 ϕ(Y9) = 0.02 ϕ(Y10) = 0.02ϕ(Y11) = 0.02 In other case ϕ(S
∪T )= ϕ(S) + ϕ(T ), S, T ∈ Y , S
∩T=∅
We can see that criteria Y1 and Y2 are positively correlated, since ϕ(Y1
∪Y2) >
ϕ(Y1) + ϕ(Y2). Due to the fact that these criteria present some degree of complemen-tarity. In the same way to the criteria Y6, Y7 and Y8.
Finally, global values for each selling employees are computed, using the Choquetintegral for linguistic 2-tuple, which are shown in Table 4.
Table 4: Assessments for each selling employee
x1 x2 x3 x4 x5 x6 x7
(M,−.2) (M,−.5) (AN, .3) (V L,−.1) (M, 0) (L, 0) (V H, .2)
In this phase of our case study, we have obtained linguistic results for each em-ployee in each step of the multi-step aggregation process, following the CWW method-ology. These results are close to human natural language and providing interpretabilityand understandability.
5.3.3 Ranking Phase
Finally, we put in order all computed values and we establish a ranking among evalu-ated employees with the purpose of identifying the best ones.
In the case study, (V H, .2) = (V eryHigh, .2) ∈ ⟨S⟩ is the highest global valueand it corresponds to the employee x7. Therefore, x7 is the best selling employee forthis evaluation and the total ranking is:
16
x7 ≻ x5 ≻ x1 ≻ x2 ≻ x6 ≻ x4 ≻ x3
6 Concluding remarksPerformance appraisal is a relevant process used for companies in order to make impor-tant decisions, such as promotions, salaries, needs, etc. In this paper, we have proposedan integrated 360-degree performance appraisal model that provides a flexible eval-uation framework in which reviewers can provide their assessments within differentdomains, according to the uncertainty and nature of such criteria and the backgroundof each reviewer. Moreover, with the aim of ensure an effective aggregation of theinformation, the proposed model applies an adequate set of aggregation operators tocope with the interaction among criteria and reviewers weights, providing results closeto human natural language by using the computing with words methodology in order tobe understandable and interpretable by different members of the company. Finally, wehave present a real case study to show the usefulness and effectiveness of the integratedproposed model.
AcknowledgementsThis paper has been partially supported by the “Consejerıa de Innovacion, Cienciay Empresa (Junta de Andalucıa)” with the Research Project AGR-6487 and by the“Ministerio de Ciencia e Innovacion” with the Research Project TIN2009-08286 andthe Research Project ECO2009-07332.
References[1] C.G. Banks and L. Roberson. Performance appraisers as test developers.
Academy of Management Review, 10:128–142, 1985.
[2] J. N. Baron and D. M. Kreps. Strategic Human Resources, Frameworks for Gen-eral Managers. Wiley & Sons, Nueva York, 1999.
[3] F.J. Cabrerizo, J. Lopez-Gijon, A.A. Ruız, and E. Herrera-Viedma. A modelbased on fuzzy linguistic information to evaluate the quality of digital libraries.International Journal of Information Technology and Decision Making, 9(3):455–472, 2010.
[4] T. Calvo, R. Mesiar, and R.R. Yager. Quantative weight and aggregation. IEEETransactions on Fuzzy Systems, 12:62–69, 2004.
[5] C.T. Chen. Applying linguistic decision-making method to deal with servicequality evaluation problems. International Journal of Uncertainty, Fuzziness andKnowlege-Based Systems, 9(SUPPL.):103–114, 2001.
17
[6] G. Choquet. Theory of capacities, volume 5, pages 131–295. Annales de l’institutFourier, 1953.
[7] R.T. Clemen. Making Hard Decisions. An Introduction to Decision Analisys.Duxbury Press, 1995.
[8] R. de Andres, M. Espinilla, and L. Martınez. An extended hierarchical linguisticmodel for managing integral evaluation. International Journal of ComputationalIntelligence Systems, 3(4):486–500, 2010.
[9] R. de Andres, J.L. Garcıa-Lapresta, and L. Martınez. A multi-granular linguisticmodel for management decision-making in performance appraisal. Soft Comput-ing, 14(1):21–34, 2010.
[10] M. Edwards and E. Ewen. Automating 360 degree feedback. HR focus, 70:3,1996.
[11] M. Espinilla, J. Liu, and L. Martınez. An extended hierarchical linguisticmodel for decision-making problems. Computational Intelligence, 27(3):489–512, 2011.
[12] C. Fisher, L. F. Schoenfeldt, and J. B. Shaw. Human Resources Management.Houghton Mifflin Company, Boston, 2006.
[13] S. Gramajo and L. Martınez. A linguistic decision support model forqos priorities in networking. Knowledge-Based Systems. Article in Press.doi=10.1016/j.knosys.2011.08.016.
[14] F. Herrera and Martınez L. Herrera-Viedma, E. and. A fuzzy linguistic method-ology to deal with unbalanced linguistic term sets. IEEE Transactions on FuzzySystems, 16(2):354–370, 2008.
[15] F. Herrera and L. Martınez. A 2-tuple fuzzy linguistic representation model forcomputing with words. IEEE Transactions on Fuzzy Systems, 8(6):746–752,2000.
[16] F. Herrera and L. Martınez. The 2-tuple linguistic computational model. Advan-tages of its linguistic description, accuracy and consistency. International Journalof Uncertainty, Fuzziness and Knowledge-Based Systems, 9(Suppl.):33–49, 2001.
[17] F. Herrera, L. Martınez, and P.J. Sanchez. Managing non-homogeneous infor-mation in group decision making. European Journal of Operational Research,166(1):115–132, 2005.
[18] I.B. Huang, J. Keisler, and I. Linkov. Multi-criteria decision analysis in environ-mental sciences: Ten years of applications and trends. Sci Total Environ, 2011.
[19] C. Kahraman, I. Kaya, and S. Cebi. Renewable energy system selection basedon computing with words. International Journal of Computational IntelligenceSystems, 3(4):461–473, 2010.
18
[20] R.L. Keeney and H. Raiffa. Decisions with multiple objectives: preferences andvalue tradeoffs. John Wiley & Sons, New York, 1976.
[21] J. Lu, J. Ma, G. Zhang, Y. Zhu, X. Zeng, and L. Koehl. Theme-based compre-hensive evaluation in new product development using fuzzy hierarchical criteriagroup decision-making method. IEEE Transactions on Industrial Electronics,58(6):2236–2246, 2011.
[22] J. Lu, Y. Zhu, X. Zeng, L. Koehl, J. Ma, and G. Zhang. A linguistic multi-criteriagroup decision support system for fabric hand evaluation. Fuzzy Optimizationand Decision Making, 8:395–413, December 2009.
[23] A. Marcomini, G.W. Suter, and A. Critto. Decision support systems for risk basedmanagement of contaminated sites. Springer Verlag, New York, 2009.
[24] S. Marshall. Complete turnaround 360-degree evaluations gaining favour withworkers management. Arizona Republic, D1, 1999.
[25] L. Martınez. Sensory evaluation based on linguistic decision analysis. Interna-tional Journal of Approximate Reasoning, 44(2):148–164, 2007.
[26] L. Martınez, M. Espinilla, J. Liu, L.G. Perez, and P.J. Sanchez. An evaluationmodel with unbalanced linguistic information applied to olive oil sensory evalu-ation. Journal of Multiple-Valued Logic and Soft Computing, 15(2-3):229–251,2009.
[27] L. Martınez, J. Liu, and J.B. Yang. A fuzzy model for design evaluation basedon multiple criteria analysis in engineering systems. International Journal ofUncertainty, Fuzziness and Knowlege-Based Systems, 14(3):317–336, 2006.
[28] L. Martınez, D. Ruan, and F. Herrera. Computing with words in decision sup-port systems: An overview on models and applications. International Journal ofComputational Intelligence Systems, 3(4):382–395, 2010.
[29] J.M. Mendel, L.A. Zadeh, E. Trillas, R.R. Yager, J. Lawry, H. Hagras, andS. Guadarrama. What computing with words means to me: Discussion forum.IEEE Computational Intelligence Magazine, 5(1):20–26, 2010.
[30] Y.C. Shen, G.T.R. Lin, K.P. Li, and B.J.C. Yuan. An assessment of exploitingrenewable energy sources with concerns of policy and technology. Energy Policy,38(8):4604–4616, 2010.
[31] J.J. Wang, Y.Y. Jing, C.F. Zhang, and J.H. Zhao. Review on multi-criteria decisionanalysis aid in sustainable energy decision-making. Renewable and SustainableEnergy Reviews, 13(9):2263–2278, 2009.
[32] Z. Wang and G. Klir. Fuzzy measure theory. Plenum press, New York:, 1992.
[33] R.R. Yager. On ordered weighted averaging operators in multicriteria decisionmaking. IEEE Transactions on Systems, Man, and Cybernetics, 18:183–190,1988.
19
[34] R.R. Yager. An approach to ordinal decision making. International Journal ofApproximate Reasoning, 12(3):237–261, 1995.
[35] W. Yang and Z. Chen. New aggregation operators based on the choquet in-tegral and 2-tuple linguistic information. Expert Systems with Applications,39(3):2662–2668, 2012.
[36] L.A. Zadeh. The concept of a linguistic variable and its applications to approxi-mate reasoning. Information Sciences, Part I, II, III, 8,8,9:199–249,301–357,43–80, 1975.
[37] L.A. Zadeh. From computing with numbers to computing with words - from ma-nipulation of measurements to manipulation of perceptions. IEEE Transactionson Circuits and Systems I: Fundamental Theory and Applications, 46(1):105–119, 1999.
20