a fuzzy set theory solution to combining likert items into a single overall scale (or subscales)

Qual QuantDOI 10.1007/s11135-014-0021-z

A fuzzy set theory solution to combining Likert itemsinto a single overall scale (or subscales)

Maria Symeonaki · Catherine Michalopoulou ·Aggeliki Kazani

© Springer Science+Business Media Dordrecht 2014

Abstract Likert scales (or subscales) are usually constructed by summing up the items whenthe assessment of their psychometric properties has resulted in showing that the scale is bothreliable and valid. This paper presents a methodology for developing a fuzzy set theorysolution to combine Likert items into a single overall scale (or subscales). The proposedmethodology puts together information produced by construct validity assessment, statisticalanalysis and experts’ knowledge produced during theory development, in a fuzzy inferencesystem to develop a more accurate attitude measurement. The evaluation of the methodologypresented is tested on a Likert scale that was used in a large-scale sample survey for measuringxenophobia in Northern Greece conducted by the National Centre for Social Research. Themethodology can be applied with minor modifications to other data sets.

Keywords Likert scales · Fuzzy set theory · Fuzzy inference systems · Reliability ·Construct validity · Exploratory factor analysis

1 Introduction

Likert (1932) developed a scale for measuring attitudes that has been widely used in socialsciences, educational, medical and health research. Likert scales are composed of a numberof items (opinion statements) which are rating scales, and respondents are asked to placethemselves usually on five response categories of agreement/disagreement with a neutralmidpoint. A Likert scale assumes that the strength of an attitude or belief is linear, i.e. on a

M. Symeonaki (B) · C. Michalopoulou · A. KazaniDepartment of Social Policy, Panteion University of Social and Political Sciences, 136 Syggrou Av.,17671 Athens, Greecee-mail: [email protected]; [email protected]

C. Michalopouloue-mail: [email protected]; [email protected]

A. Kazanie-mail: [email protected]

123

M. Symeonaki et al.

continuum from strongly agree to strongly disagree. Thus, respondents are not asked to decidejust whether they agree or disagree with an item (also referred to as item-question), but ratherto choose between several response categories, indicating various strengths of agreement anddisagreement, allowing for degrees of opinions. The response categories are normally scoredfrom 1 to 5 (Hartley and Betts 2009; Lozano et al. 2008) and usually labelled strongly agree,agree, neither agree nor disagree, disagree, strongly disagree. Having shown that the scale isboth valid and reliable, the respondents’ attitude is usually measured by their total score whichis computed by summing up his/her responses for each of the items. For this reason, Likertscales are also known as summated and summated rating scales (Moser and Kalton 1971).Depending on the context and the application, composite scales may also be constructed basedon the results of factor analysis used to assess the scale’s construct validity by applying one offour methods: regression, Bartlett, Anderson-Rubin and Thompson factor scores (Thompson2005). However, these composite scales do not preserve the variation in the original dataas does the method of summing up the items (DiStefano et al. 2009). In most exploratoryresearch situations, summing up the items is extensively used and, when factor analysisshows that simple structure is present, then its application is straightforward (DiStefano et al.2009). Thus, when the typical methodology is used one assesses a respondent’s attitude byexamining the response categories he/she chose in a number of statements. However, thesescales are subjected to uncertainty being created in the boundaries of the response categories.In addition, central tendency, social desirability and acquiescence response bias interfere withthe assessment of the respondent’s attitude. There is also a matter of subjectivity, particularlywhen one wants to explore complex social attitudes, where the respondents may not be ableto assess accurately their attitude (Moser and Kalton 1971). Therefore, there is a need toseek for new methods to measure attitudes which, in the present paper, constitutes the primemotivating factor for considering fuzzy set theory and fuzzy reasoning with Likert scales.

There have been numerous attempts to apply fuzzy set theory to various social settings(see among others, Betti et al. 2011; Sah and Degtiarev 2005; Song and Chissom 1994;Symeonaki and Kalamatianou 2011). In recent years, there has been an increasing interestin the scale of fuzzy numbers with applications to social sciences, psychology, management,etc. (see for example Aydin and Pakdil 2008; Turksen and Willson 1994). Fuzzy conversionscales, generally accepted as the most frequent application of the scale of fuzzy numbers (Dela Rosa et al. 2013), convert each verbal response into a fuzzy number by following differentfuzzification processes. Lalla et al. (2005) proposed a fuzzy conversion scale to analysequalitative ordinal data produced by a course-evaluation questionnaire. Mogharreban andDilalla (2006) proposed an inference engine using fuzzy logic for the analysis of Likert-typequestionnaires with 128 respondents and two rules. Lazim and Abu Osman (2009) studied afuzzy set approach for measuring teacher’s beliefs about mathematics (23 subjects) and Lazimand Abu Osman (2011) suggested a fuzzy conjoint model in describing student’s perceptionson computer algebra system learning environment (164 students). A different approach isdemonstrated in Hesketh and Hesketh (1994); Hesketh et al. (1988) with the concept ofthe fuzzy rating scale. A fuzzy rating scale is an extension of a rating scale that gives theopportunity to respondents to point out a preferred category and the respective latitude ofacceptance on either side. Gil et al. (2011) used the fuzzy rating scale in questionnaires toexpress the responses and Gil and Gil (2012) provide guideline to design questionnairesallowing free fuzzy-numbered response format. However, the above mentioned studies havenot as yet addressed the question of how to assess the psychometric properties (reliabilityand validity) of the proposed scales developed by applying these methods. Moreover, fuzzyrating scales seem at the moment to be appropriate for respondents who can well understandthe concept of a membership function and follow the instructions of drawing the respective

123

A fuzzy solution to Likert scaling

trapezoidal fuzzy numbers. In this respect, the methods may be easily applied to Universitystudents (Gil and Gil 2012; Gil et al. 2011), but in most general social surveys it would beinfeasible to make use of such fuzzy rating scales (see also, De la Rosa et al. 2013). (Forexample, in the application presented in Sect. 5, the attitude measured is xenophobia where26 % of the respondents are aged between 60 and 80 years old and 40 % are illiterate orcompleted only the first grades of primary school. On the other hand only 13.7 % of therespondents are University graduates as is the case in most general population surveys. Toconsider fuzzification in such a setting requires appropriate training of all interviewers. Inthe field, the interviewers, after completing the main questionnaire, would have to instruct intheir turn the graduate respondents in constructing fuzzy rating scales; a process that wouldincrease the costs and would be difficult to supervise and control.)

Let us now describe briefly the proposed methodology. In attitude scaling, factor analysisis used mainly for three purposes: construct validity assessment, theory development andsummarising relationships in the form of a more parsimonious set of factor scores that can thenbe used in subsequent analysis (Thompson 2005). In previous work, following the methodintroduced by Lalla et al. (2005), Symeonaki and Kazani (2011) put forward a fuzzy systemfor measuring attitudes. Our objective here is to develop a useful methodology for measuringattitudes, that takes into account the results of construct validity assessment conducted usingfactor analysis, further statistical data analysis and experts’ knowledge on the attitude. We usethe term experts’ knowledge to describe the existing knowledge on the particular researchfield and more specifically on the attitude that can be incorporated in the forms of rules.This alternative approach considers the interconnections of constructs provided by factoranalysis with the attitude under investigation using symbolic knowledge representations andtakes advantage of the experts’ knowledge, which is usually available by theory. Theseinterconnections are normally strongly related to several population parameters not directlyencoded in the factors. Hopefully, some are available through additional information providedby theory and the questionnaire, but most have a fuzzy nature (e.g. age, church attendance,education, etc.) and consequently it is difficult to cover their influence in terms of classicalexpert systems. This is also the case for most factors which are inherently vague. In thesecases the application of fuzzy inference systems (FIS) can prove very useful:

– Here, the items (questions) are considered as fuzzy propositions and thus scores assignedto the item categories are interpreted as degrees of truth of fuzzy propositions (Duboisand Prade 1997), i.e. as fuzzy values representing the degree in which the respondentbelieves that the specific fuzzy proposition holds (by convention Likert scales assignvalues from 1 to 5, representing this strength of belief). Consequently, the fuzzy system’sinput concerning the items reveals a vague nature, takes values in the interval [0, 1],following the “degree of truth of fuzzy propositions” character, thus facilitating the rep-resentation of experts’ knowledge. Moreover, fuzzy inference is capable of capturing thevague nature of factors, since factor loadings can be considered as weights in a weightedsum aggregating each item’s influence. Suppose, for example, that the attitude to bemeasured is xenophobia and a factor produced by factor analysis has an interpretation asStereotypical Stigmatisation of Foreigners, that loads to questions like “Foreigners areresponsible for the increase in the crime rate”, “Foreigners increase unemployment” or“Foreigners who work in our country do harm to our economy” etc., through the respec-tive weights that are equal to the factor loadings (assuming that factor analysis is appliedfor assessing the construct validity). In this case, the answers to the specific questions arenumbers (usually from 1 to 5) representing the truth value (user belief) of the statementand they are weighted and added, supposing that the specific factor linearly influences

123

M. Symeonaki et al.

them. Thus, the value of the final result has clearly a fuzzy nature that can be covered byfuzzy partitioning.

– The proposed methodology allows us also to consider as input to the fuzzy system,a number of key parameters (mostly socio-demographic variables) that are related tothe factors (subscales) and simultaneously influence the attitude. The majority of theseparameters delineates a fuzzy nature and thus fuzzy partitioning can be used to address thevagueness introduced also in this case. If the attitude measured is again xenophobia anda construct produced by factor analysis is the Stereotypical Stigmatisation of Foreignerswe can include in our evaluation key variables such as age and education and use experts’knowledge concerning the way all these variables are related with the construct. If one isyoung and highly educated, for example, it is more likely to hold less stereotypes towardsthe “other”, as relevant research shows.

– Difficulties arise, moreover, if we attempt to construct models to simulate the associationbetween factors and the attitude. In an attempt to make this simulation feasible, experts’knowledge, easily represented in the form of rules, can be used. Most constructs involvedhave a semantic interpretation that is clear for social scientists and can easily be expressedwhen fuzzy partitions are used. For example, if one is young, highly educated, with nostereotypes towards the “other” then one is more likely to exhibit a less xenophobicattitude, as relevant research shows.

Therefore, classical Likert scaling theory may be extended to obtain a more accurate mea-surement of an attitude that preserves the original scoring and at the same time incorporatesthe methods traditionally used for assessing the scale’s construct validity, while using allavailable information from relevant research.

The paper has been organised in the following way. Section 2 provides an overview of thepreliminaries and the nomenclature used in the remaining of the paper. Section 3 describes theseries of steps to assess construct validity and reliability of a Likert scale. Section 4 presentsin detail the proposed methodology for evaluating an attitude by extending Likert scales.Section 5 examines the application of the proposed methodology on a Likert scale that wasused in a large-scale sample survey for measuring xenophobia in Northern Greece conductedby the National Centre for Social Research (Michalopoulou et al. 1998) in order to facilitatethe understanding of the proposed method. This data set is used mainly for methodologicalreasons because it provides the necessary for validation single questions used in the literatureas indicators of xenophobia. The derived scores of xenophobia were found significantlycorrelated with the scores produced with the traditional method. Further analysis showedthat, interestingly, fuzzy scores are more highly correlated with all xenophobia indicatorsused, than crisp scores. Section 6 lays out the conclusions and summarizes the findings ofthe presented work.

2 Preliminaries and notation

Let us first provide a brief introduction to the theory of Fuzzy Logic presented by L. A.Zadeh in 1965. Fuzzy logic is based on fuzzy set theory (Zadeh 1965) and is considered asa generalization of the classical (crisp) logic theory. In contrast with classical logic theoryassumption, where objects are dichotomized and identified as members or non-members ofa set, membership in a fuzzy set is not a matter of affirmation or denial but rather a matter ofdegree. When A is a fuzzy set and x is a relevant object, the statement, x is a member of A,is not necessarily either true or false, but it may be true only to some degree, being usually

123


1

0

Very young Middle Aged Old

Age (in years)

(a)0

100

Young Very old

Mem

bers

hip

Age (in years)

(b)

0 100[ )[ [) )[ [) )Very young Middle Aged OldYoung Very old

Fig. 1 Age in years with range [0, 100] conceived as a fuzzy variable; b a traditional (crisp) variable

a number in the interval [0, 1]. This degree is represented by the membership function ofthe fuzzy set A, mx (A). A membership function is a curve that defines how each point inthe input space is mapped to a membership value between 0 and 1. Different membershipfunctions may be used in different context for characterising fuzzy sets. Whether a particularshape (trapezoidal, triangular, etc.) is suitable or not can only be determined in the contextof each application. Fuzzy systems are systems in which variables have as domain fuzzy setsencoding structured, empirical (heuristic) or linguistic knowledge in a numerical framework.They describe the operation of the system in natural language with the aid of human-likeIF-THEN rules and thus provide a more reflective representation of human perception. FISsare extensions of the crisp point-to-point mappings into set-to-set mappings, i.e. mappingsfrom the set of all fuzzy subsets of the input space X (F(X)) to the set of all fuzzy subsetsof the output space Y (F(Y )). Several methods have been proposed (Klir and Yuan 1995)for the implementation of this mapping. We use one of the most widely used method, themulti-conditional approximate reasoning schema.

Several fuzzy sets representing linguistic concepts such as low, medium, high, and soon are employed to define states of a variable and often called linguistic values or states.Such a variable is usually called a fuzzy variable (Klir and Yuan 1995). In Fig. 1a, forexample age within a range [0, 100] is characterised as a fuzzy variable and it is contrasted inFig. 1b with a crisp variable. The states of the fuzzy variable are fuzzy sets representing fivelinguistic concepts: very young, young, middle-aged, old, very-old. They are all defined bymembership functions from [0, 100] → [0, 1]. Graphs of these functions have trapezoidalshapes, which together with triangular shapes, are most common in real applications (Klirand Yuan 1995). The partitioning presented in Fig. 1a is a fuzzy partitioning on the domain[0, 100] of order d = 5, i.e. the number of the linguistic values. States of the correspondingtraditional variables are crisp sets defined by the right-open intervals of real numbers shownin Fig. 1b.

A domain A of attitude measurements consists of three pairwise disjoint sets C, Q and V ,representing the constructs, items and values of the domain, respectively. Given a constructc ∈ C and a set Qc = {q1, . . . , qm} ⊆ Q, a scale for c is a mapping κc : {c} ∪ Qc → P(V),where P(∗) is the power set of ∗. Based on a function val(∗) for ∗ ∈ C∪Q, with val(∗) ∈ κ(∗),we say that an item q (a construct c) is measured as val(q) (as val(c)). A scoring λκc of c,

123

M. Symeonaki et al.

based on the scale κc, is a mapping λκc : κc(q1)×· · ·×κc(qm) → κc(c). If λκc is a scoring forc, then val(c) = λ(val(q1), . . . , val(qm)), where λ associates the values of q1, q2, . . . , qm

with the values of c, according to the specific measurement λκc . For simplicity, we may omitthe indicators of κ and λ, whenever their domains and ranges are apparent. Moreover, with alittle abuse of notation, we may denote by qi both the item and the value of the item (strictlydenoted as val(qi )), as clarified by the context. A measurement database D for a constructc ∈ C is a structure {q1 j , . . . , qmj ; M j } j=1,...,n , with qi j ∈ Qc, i = 1, . . . , m, where M j

is a structured description of the measurement j , representing additional information and ndenotes the total number of respondents.

Intuitively, given a survey for measuring attitudes and the relevant sample of respondents,the elements of C (the constructs) represent abstractions of concepts of the attitudes’ domainand the elements of Q (the items) represent questions (opinion statements) possibly used inthe survey, measuring the relevant constructs. Items are evaluated on the basis of subsets of thevalue domain (the scales defined by κ), and then used for the scoring of the relevant constructs(based on the measurements defined by λ). For example, Likert scales usually apply fiveinteger response categories as values of the items (val(qi ) ∈ κ(qi ) = {1, 2, 3, 4, 5}) and theresponses are summed up in order to obtain the respondents’ scores on the construct (val(c) =λ(val(q1), . . . , val(qm)) = ∑m

i=1 val(qi ), thus κ(c) = {m, m + 1, . . . , 5m}). Measurementdatabases are usually constructed from questionnaires. For a respondent j , {q1 j , . . . , qmj }represents the endorsement of the relevant items-questions according to his/her responses,while the additional information M j represents socio-demographic information (e.g. age,level of education etc.) for the specific respondent.

Two important psychometric properties of a construct’s measurement are reliability andvalidity. A measurement for a construct c ∈ C is reliable if ∀ j = 1, 2, . . . , n, val(c) isconstant over time, provided that the repeated measurements takes place under consistentconditions and no actual change in the attitude has occurred. Validity refers to the extent towhich what was to be measured was actually measured.

Semantic information interconnecting the constructs is usually available from the expertsof the attitude domain. It can be expressed in the form of (possibly weak) restrictions, definingthe dimensions of the constructs. Formally, a theory for a construct c ∈ C is a mapping ofthe form τc : κ(c1) × · · · × κ(cl) → κ(c), with c1, . . . , cl ∈ C. In this case, we say that c issemantically restricted by c1, . . . , cl that are the dimensions of c. Every scale for a dimensionof c is called a subscale for c. Intuitively, theories represent semantic rules of the domain andcan be used for the indirect scoring of construct, with the aid of the measurement of theirdimensions.

A measurement for a construct c ∈ C is unidimensional if and only if there exists a (reliableand valid) scale for c. It is multidimensional if and only if c is semantically restricted by aset of (reliable and valid) constructs that are measured by unidimensional measurements.

3 Assessing the validity and reliability of constructs

The first step of the analysis of attitude measurement is the assessment of the validity andreliability of the overall Likert scale or subscales. On completion of this assessment, theunidimensionality or multidimensionality of the Likert scale is decided upon.

3.1 Assessing construct validity

Construct validity assessment depends on whether the goal is theory development (subscalesare not predetermined as dimensions by theory) or theory testing (subscales are predetermined

123


as dimensions by theory). We therefore distinguish between the following cases (Tabachnickand Fidell 2007; Thompson 2005) described separately in the following subsections.

3.1.1 Theory development: subscales are not predetermined as dimensions by theory

If subscales are not predetermined by theory, then principal components analysis (PCA)or exploratory factor analysis (EFA) is performed to define components or factors as sub-scales and component or factor loadings are reported. Though PCA is exploratory in nature(Thompson 2005) it differs from common factor analysis defined as EFA (Bartholomew et al.2008; Fabrigar et al. 1999). The choice between PCA and EFA depends on whether the goalis to reduce the correlated observed variables into a smaller set of independent compositevariables (components) or to test a theoretical model of latent factors causing the observedvariables (Bartholomew et al. 2008; Fabrigar et al. 1999). In performing PCA or EFA thefollowing sequence of decisions is required (Tabachnick and Fidell 2007; Thompson 2005)which incorporate several rules of thumb—as is the case in all applications. The rules ofthumb are not presented as absolute but as our choice based on the relevant literature:

1. Initially, univariate statistics are computed for each item and their distributional propertiesare inspected (testing normality) to decide on the appropriateness of the methods tobe used. Missing values are investigated for size and randomness. Corrected item-totalcorrelations are computed and items meeting the criterion of correlations greater than0.30 are included in the analysis (Nunnally and Bernstein 1994).

2. Component or factor extraction method: In applications, both PCA and EFA are usedand their results are compared (Bartholomew et al. 2008; Fabrigar et al. 1999). (Lowextraction communalities, i.e. less than .4, indicate candidate items for exclusion fromanalysis.)

3. Decision on the number of components or factors to be extracted based on the eigenvaluegreater than 1.0 rule, scree test, parallel analysis and interpretability (O’Connor 2000).

4. Component or factor rotation method: In applications, both orthogonal and oblique rota-tions are applied. The choice between them is based on the correlations among compo-nents or factors and the simple structure criterion (Reuse et al. 2000).

5. Subscales are computed by summing up their defining items based on their factor load-ings. Average inter-item correlations for each subscale (as defined by components orfactors) and between subscales are computed in order to decide on whether subscales arewarranted or should be combined into a single overall scale (Clarkand and Watson 1995).Average inter-item correlations in the range of 0.15–0.5 that cluster near the mean valueare used to decide on unidimensionality (Clarkand and Watson 1995). If the condition ofaverage correlation between subscale items significantly greater than zero but substan-tially less than the average within-subscale values (i.e. 0.20) is met, then subscales arewarranted. If not, they should be abandoned in favour of a single overall scale (Clarkandand Watson 1995).

In applications, this sequence of decisions is performed several times before deciding onthe final solutions.

3.1.2 Theory testing: subscales are predetermined as dimensions by theory

If subscales are predetermined by theory, then if the sample size is adequate the sampleis randomly split into half and EFA is performed on one half-sample to define factors assubscales (reporting factor loadings) and confirmatory factor analysis (CFA) to the other-half sample to validate the multidimensional structure of the attitude under investigation. If

123

M. Symeonaki et al.

the structure of the attitude is proved to be multidimensional then subscales are constructed.Like PCA and EFA, CFA requires various decisions on the matrix of associations to beanalyzed, the estimation theory to be used and screening data for outliers to name but a few(Tabachnick and Fidell 2007; Thompson 2005). But the full presentation of a sequence ofdecisions for CFA is beyond the scope of the present paper.

3.2 Assessing the reliability of the overall Likert scale or subscales

In order to assess reliability, Cronbach’s alpha reliability coefficients for the overall scaleand subscales are usually reported (Fabrigar et al. 1999). A scale or subscale is reliable ifCronbach’s alpha coefficients are at least 0.70 (Nunnally and Bernstein 1994; Revelle andZinbarg 2009). Richardson (KR) formula, or spilt-half reliability coefficient are two otherof the many alternative methods for investigating internal consistency of tests (Revelle andZinbarg 2009). The overall scale or subscales should be both valid and reliable (Moser andKalton 1971).

In the case of theory development, if subscales are determined, and are found to bereliable, then to examine their construct validity PCA or EFA is performed for each subscaleand component or factor loadings are reported.

In classical Likert scale theory, in both cases of theory development and theory testing,having ascertained the reliability and validity of the scale or subscales, the overall scaleor subscales are normally computed by summing up (or averaging) the respective valuesval(qi ) of their defining items. Figure 2 presents an optimum flowchart diagram of a Likertscale construct validity assessment, which in real applications requires the repetition of theprocedure before deciding on the final solution. When a solution is proved valid, reliabilityassessment follows.

4 Extension of Likert scales or subscales

We now provide a detailed description of the proposed methodology for extending Likertscales using fuzzy inference. Let a survey for measuring an attitude with the use of Lik-ert scales and the relevant set of respondents j = 1, 2, . . . , n. The use of any statisticalanalysis requires assumptions on sampling procedures (probability sampling and effectivesample sizes), the level of measurement of the variables and an inspection of their distrib-utional properties. Likert items’ level of measurement is considered by many as ordinal. Inapplications, where the number of response categories used for each item is at least five,the ordinal categories can be understood as being interval and one could proceed to per-forming statistical analyses using these pseudo-interval variables (Bartholomew et al. 2008).Likert (1932) developed a scale theory requires items to be interchangeably worded posi-tively and negatively. Therefore, before carrying out an analysis, the ordering of the positiveor negative items should be reversed, depending on the definition of the negative-positiveends of the overall scale or subscales.

4.1 Extending Likert scales

Let c1, c2, . . . , cl ∈ C be the dimensions identified by factor (or component) analysis and li j

denote the factor (or component) loadings of qi on factor (or component) c j . The meaningof each dimension is inferred from the items that have their highest loading on the respectivecomponent or factor (Nunnally and Bernstein 1994; Steiner and Norman 2008).

123


Construct validity assessment of a Likert scale

YES

END

NO

NO

YES

Perform the respective steps of sequence 1 (theory

development)

Average correlation

between subscale

zero but substantially less than the average within-

subscale values (i.e. .20)

Compute a single overall scale by summing up all items

YES

NO

Theorydevelopment

END

Compute subscales by summing

their factor loadings)

Choice for rotation method based on the correlations among components and

factors and the simple structure criterion

Decision on number of components or factors (eigenvalue greater than 1.0 rule,

scree test, parallel analysis and interpretation)

Compute inter-item correlations for each subscale and between

subscales

END

Theorytesting

Sample size is

adequate

Split-half sample 1

Perform CFA to validate

multidimensionality

Structure is multidimensional

YES

NO

Subscales are warranted

Theory validated

END

Theory does not apply

Perform both EFA and CFA

Unidimensional scale

Component or factor rotation

(apply both orthogonal and oblique rotations)

Subscales are warranted

PCA or EFA to

or factors

One factor

Split samplein two halves

Perform EFA on the split-half

sample 1

Split-half sample 2

Compute a single overall scale by summing up all items

Average inter-item correlations in the range of .15-.5 that cluster near the

mean value indicate unidimensionality

Likert item analysis

END

Fig. 2 Process flowchart of a Likert scale construct validity assessment. The squashed rectangle representthe start point, diamond shapes represents decisions, rectangles processes (analysis, rotation, etc.) and paral-lelograms an output or result. We use trapezoids that usual stand for manual operation to represent a choiceor decision

We distinguish between the following possible solutions: (a) investigating the dimension-ality of the scale, though construct validity assessment results in a number of componentsor factors and they are proved to define separate dimensions or not or (b) they do not definemultiple dimensions and a unidimensional measurement of a construct may be defined.

4.1.1 Unidimensional measurement of a construct

Unidimensional measurement of a construct is considered in cases where construct validityassessment results indicate that there exists a single underlying dimension. This emerges, as

123

M. Symeonaki et al.

CONSTRUCT CONSTRUCT

PCA EFA

Fig. 3 A hypothetical ideal unidimensional factor solution produced with PCA and EFA in assessing thecohesion (construct validity) of the scale with no “cross-loading” items. Observed variables are representedby rectangles

earlier mentioned either when PCA or EFA results in one component or factor (Fig. 3) or ina number of components or factors, that are proved to define a single dimension (Fig. 4). Inthe first case the item-scores can simply be added.

The most usual outcome of construct validity assessment that emerges in real applicationsis the one depicted in Fig. 4, where “cross-loading” items, i.e. items with loadings greaterthan .30 in one factor and with loadings greater than 0.22 in another factor are considered toload on multiple factors (Stevens 2002; see also Anagnostopoulos et al. 2013).

Once the factor solution is produced, the meaning of each factor is inferred by its definingitems, i.e. the items that load higher on each factor, irrespective of existing “cross-loadings”.In the literature, scale scores are usually computed by summing up the respective valuesval(c j ). In an attempt to increase accuracy, all available information is of great matter andtherefore “cross-loading” items must not be overlooked. The purpose is to deal with numerousreal applications where an item loads, for example with 0.415 on one factor and with 0.399 onanother factor and only the first loading is considered in the computation of the relative factorscore. We estimate the scores for each factor for all respondents, val(c j )(i),∀ j = 1, 2, . . . , land ∀i = 1, 2, . . . , n, taking “cross loadings” into account:

val(c j )(i) =∑m

s=1,ls j ≥0.22 ls j val(qs(i))∑m

s=1,ls j ≥0.22 ls j. (1)

Since 1 ≤ val(qs(i)) ≤ p,∀s = 1, 2, . . . , m and ∀i = 1, 2, . . . , n it can easily be deducedthat val(c j )(i) ∈ [1, p],∀ j = 1, 2, . . . , l, i.e. the weighted score val(c j )(i) can take anyvalue between 1 and p.

123


Fig. 4 A hypothetical realistic unidimensional factor solution produced with PCA and EFA in assessing thecohesion (construct validity) of the scale with “cross-loading” items. Observed variables are represented byrectangles and the dimensions ci are enclosed in ellipses. Straight lines correspond to factor loadings anddashed lines to secondary loadings

Consider now the factor scores val(c j )(i), j = 1, 2, . . . , l and the additional informa-tion variables Variable_h, h = 1, 2, . . . , r . These comprise the input of the system for everyrespondent, i = 1, 2, . . . , n, i.e. x(i)=[val(c1)(i), val(c2)(i), . . . , val(cl)(i), Variable_1(i),Variable_2(i), . . . , Variable_r(i)]. We define fuzzy partitions A j of order d j on the domain[1, p] of factor score val(c j ), ∀ j = 1, 2, . . . , l and fuzzy partitions Al+h of order dl+h onthe universe of discourse [ah, bh] of the additional information variable, Variable_h, h =1, 2, . . . , r .

In addition, we define B, a fuzzy partition of order u on the domain [a, b] of the final scalefor the measured construct C and Bv are the linguistic values of C , v = 1, 2, . . . , u. Therelationship of the crisp universe of discourse is represented using linguistic rules that definea mapping of the fuzzy partitions Ai , i = 1, 2, . . . , l + r to fuzzy partition B. In so far as theelements of Ai and B have a linguistic meaning heuristic and empirical linguistic rules can beused in order to describe the input-output relationships. Combining these rules with the well-known generalised modus ponens (GMP) rule of inference (Klir and Yuan 1995; Stamou andTzafestas 1999), the multi conditional approximate reasoning schema (system rules) can beformulated. The FIS developed brings about the knowledge of the social researchers on thespecific field, concerning the way each factor and the key variables are related to the attitudebeing measured. Thus, experts’ knowledge is used not only in forming the questionnaire butalso in the development of the inference system which helps researchers evaluate the attitudeunder investigation in a more reliable and accurate way. It is important to note here that thesystem’s rules development for the attitude involves a combination of the experts’ knowledge

123

M. Symeonaki et al.

q4 qm.....

.....

.....

q5q1

q5(i) q4(i )

c1

A11 A1

2 A13

RULE1:IF (c1, c2, . . . , c l, I S ( A11 , A2

1 , . . . , A l1, Al +1

1 , . . . , Al +r1 )

OUTPUT: the i-th respondent's score is y in [a,b]

1

A21 A2

2 A23

c2 cl

0

pp

1

0

1

0

p1 1 1

q1(i) qm(i)

..... clc2c1

l11

l12 l51 l52

l5ll41

l42

lm2

lml

l4l

Al1 Al

2 A l3

B3B2B1

a bC

0

1

val(c1)(i) val(c2)(i) val(cl)(i)

Additional Data

1

0

Variable 1

Al+11 Al+1

2 Al+13

1

0

Variable ra1 b1

Al+ r1 Al+ r

2Al+ r

3

x(i) :

INPUT

Variable_1 Variable_r

Variable_1(i) Variable_r(i)

EXPERTS' KNOWLEDGE AND STATISTICAL ANALYSIS

THEN C IS B1

VALIDATION AND JUSTIFICATION

.

.

.

SYSTEM RULES FOR ATTITUDE C

ar br

Fig. 5 The operation of the system

produced by theory and other empirical studies and statistical analysis, expecting to provide amore precise measurement. See Gacto et al. (2011) for a summary of the current state-of-the-art in assessing the interpretability of linguistic fuzzy rule-based systems. Figure 5 illustratesthe operation and the structure of the system. We use one of the defuzzification methods(the centroid method) to estimate the i-th respondent’s score. Repeating this process, a setof scores for all respondents is produced.

4.1.2 Multi-dimensional measurement of a construct

When multi-dimensionality of a scale is proven or validated and subscales are warranted, thesubscales should also be reliable (Fig. 6) and the proposed methodology can be applied withsmall variations (no “cross-loadings”) in the case of subscales.

123


Fig. 6 A hypothetical ideal multidimensional factor solution produced with PCA and EFA in assessing thecohesion (construct validity) of the scale with no “cross-loading” items. Observed variables are representedby rectangles and the dimensions ci are enclosed in ellipses. Straight lines correspond to factor loadings

4.2 Validation of the proposed methodology

In order to validate the results, the fuzzy scores can be correlated with the scores producedby classical Likert scaling theory by assessing correlation coefficients between the fuzzyinference system’s output and the crisp scores. The former could be considered as a validmeasurement for the attitude, if it appears highly correlated with the latter. In order to justifythe proposed methodology the new scores can also be correlated with a number of variables,known by theory or used in other empirical studies as indicator variables for the attitude beingmeasured. Higher correlation between the new scores and the indicator variables reflects amore accurate measurement.

5 A fuzzy Likert scale for measuring xenophobia

In the present section the methodology presented in Sect. 4 is applied to a Likert scalethat measures xenophobia. This specific Likert scale was developed in the questionnaireof a large-scale sample survey conducted by the National Centre for Social Research(Michalopoulou et al. 1998), in order to investigate opinions and attitudes towards the “other”,foreigner.

123

M. Symeonaki et al.

Table 1 The socio-demographic characteristics of respondents (N = 1,200) and Greek adult general popu-lation (Census of 1991)

Variables Sample (%) Census (%) Difference (%)

Gender

Male 50.1 48.6 1.5

Female 49.9 51.4 −1.5

Age

18–29 24.2 24.9 −0.7

30–44 25.7 26.8 −1.1

45–59 24.6 25.9 −1.3

60–80 25.5 22.4 3.1

Marital status

Single 21.4 21.0 0.4

Married 73.1 70.0 3.1

Divorced/widowed 5.6 9.0 −3.4

Education

Illiterate/has left primary school 7.7 17.5 −9.8

Primary school certificate 32.3 40.2 −7.9

Lower secondary education certificate 17.0 10.8 6.2

Secondary education certificate 29.3 23.1 6.2

Higher education certificate 5.7 2.0 3.7

Higher (university) education degree 7.5 6.1 1.4

Postgraduate degree 0.5 0.3 0.2

Economically active population 55.4 48.2 7.2

Employed 49.6 44.2 5.4

Unemployed 5.8 4.0 1.8

Economically inactive population 44.6 51.8 −7.2

Adapted with permission from Michalopoulou et al. (1998)

5.1 The sample

A stratified, three-stage quota sample of 1,200 individuals was drawn to cover the adult(18–80) population of permanent residents in Northern Greece-Macedonia, using genderand age as quota controls. The detailed checks carried out on the analytic age distribu-tion did not reveal significant differences from the census data and the same applies tomarital status. The over estimation of education confirms the results observed in mostopinion surveys (Stephan and McCarthy 1974). Certainly, these checks do not justifythe sampling method used and therefore the results should be treated with the necessarycaution. In Table 1, the socio-demographic characteristics of the respondents are brieflypresented.

5.2 The Likert scale

The questionnaire used in this survey lasted about an hour and comprised a total of 57questions, determining 155 variables when the database was developed. It included factual

123


questions, opinion questions, an attitude scale and open-ended questions (Michalopoulou etal. 1998). From the 57 questions a series of 18 opinion questions defined as Likert itemsregarding foreigners, mainly economic immigrants, were used to measure xenophobia. Fiveresponse categories (values) were employed for each of the 18 items (strongly agree, ratheragree, neither agree nor disagree, rather disagree, strongly disagree) rated from 1 to 5. Thevalues of all negatively worded items were reversed in order to achieve correspondencebetween the ordering of the response categories.

At the time that the survey was conducted, according to the then theory and practice, itemanalysis indicated the rejection of one item and principal component analysis (PCA) andorthogonal rotation were used for assessing the construct validity of the scale (Fabrigar et al.1999). Therefore, it was decided that a reassessment of items’ analysis and the scale’s con-struct validity based on current theory and practice were necessary. Accordingly, univariatestatistics were computed for each item and their missing values and distributional propertieswere inspected. In this respect, though non-normality was not severe for most items (skew>2; kurtosis >7), principal axes factor analysis (PAF) was employed (Fabrigar et al. 1999).Missing values for most items were negligible, ranging from 0.1 to 1.8 % and only for Item8 they exceeded 2 %. Testing for randomness resulted that the data were probably missingcompletely at random (Little’s MCAR test, p = 0.631) and thus list-wise exclusion wasadopted (Tabachnick and Fidell 2007). The corrected item-total correlations of the threeitems (as is) were 0.253, 0.281 and 0.260, respectively, indicating their exclusion from theanalysis. Also, on the basis of their corrected item-total correlations the exclusion of threeitems (“Like Greeks, foreigners can be good or bad” (Item 3), “A foreigner’s children mustattend the same schools as our children” (Item 5) and “Foreigners who live in our countrygave us the chance to get to know new people” (Item 9)) was indicated. Therefore, 15 out ofthe 18 items were included in the analysis. In the present paper we consider the Likert scalecomprising of the 15 items which refer to a core of rights, stereotypes of a discriminatorynature or to relations on a personal level between others and us. The 15 items as they appearedin the questionnaire are presented in Table 2.

5.3 Construct validity and reliability assessment

Since the goal was the development of theory, there were no predetermined subscales andboth PCA and EFA (PAF) were applied to investigate the construct validity of the scale. Thisinitial application resulted in a three components and factors solution based on the eigenvaluegreater than 1.0 rule, implying different dimensions of the underlying phenomenon. Paral-lel analysis validated this initial solution showing that the retained three components hadactual eigenvalues (5.059, 1.677 and 1.174) that were greater than the randomly generatedones for both the average (1.211, 1.1687 and 1.137) and the 95th percentile (1.243, 1.197and 1.160) eigenvalues criteria. Parallel analysis showed the same results for retaining threefactors. Subsequently, the three components and factors solution were investigated for sim-ple structure and interpretability by applying both varimax and promax rotations (Table 3).“Cross-loadings” are presented as defined in Sect. 4.1.1. The correlations among compo-nents and factors are greater than 0.32 implying that there is approximately a 10 % (or more)overlap in the variance among components or factors, i.e. enough variance to indicate that anoblique rotation is warranted (Tabachnick and Fidell 2007). As shown, though both methodsresulted in a quite similar structure and interpretability, PAF provided a simpler structure thanPCA (Fabrigar et al. 1999). Based on these results the three factors may be interpreted. Thefirst factor is defined exclusively by items indicating favourable attitudes towards foreigners,expressing the need for the authorities to intervene on their behalf, i.e. social policy measures

123

M. Symeonaki et al.

Table 2 The opinion questions included in the analysis

Item Items-questions

1 Foreigners who live in our country must have equal rights with us

2 Many of the foreigners who live in our country are responsible for the increase in the crime rate

4 Foreigners must have lower wages even when they do the same job as us

6 The foreigners in our country increase unemployment for Greeks

7 The local authorities must organize events so we get to know the foreigners who live and work here

8 I would never marry a foreigner

10 I would never work for a foreigner

11 We should facilitate foreigners who want to settle in our country

12 Foreigners who work in our country do harm to our economy

13 The state must organize programmes for foreigners who live in our country

14 The more foreigners arrive, the lower the wages get

15 We must create reception departments in our schools for foreigners’ children

16 Foreigners should only come as tourists

17 Work permits must be given to foreigners who want to live here

18 We must close our borders to foreigners who come to work here

requiring intervention by public authorities (IPA-subscale). On the other hand, the secondfactor is defined mainly by items indicating unfavourable attitudes of stereotypical stigmati-zation of foreigners (SSF-subscale). The third factor is defined mainly by items suggestinga negative projection of self for family and working relations with a foreigner (PS-subscale)(Michalopoulou et al. 1998). Subscales IPA, SSF and PS were constructed by summing upthe defining items. PAF was performed for IPA, SSF and PS resulting in a single factor thatexplained 46.688, 49.504 and 56.927 % of the variance, respectively; not an outstandingperformance.

Cronbach’s alpha reliability coefficients for IPA, SSF and PS were 0.770, 0.793 and0.620, respectively, indicating that only IPA and SSF subscales were reliable. Split-halfreliabilities for IPA, SSF and PS were 0.773, 0.762 and 0.622, respectively. Correlationsbetween subscales were 0.551, 0.455 and 0.462. Average inter-item correlations within IPA,SSF and PS subscales were 0.358, 0.390 and 0.352 and between subscales 0.304, 0.295and 0.307, respectively. These results indicated that these subscales were not warranted andshould be combined into a single overall scale. Cronbach’s alpha and split-half reliabilitycoefficients of the overall scale using the 15 questions were 0.856 and 0.779, respectively.Statistical data analysis was performed using IBM SPSS Statistics Version 20.

5.4 Development of the fuzzy inference system

We now proceed with the development of the FIS using the GUI editor and viewer in theFuzzy Logic Toolbox of MATLAB R2012b, i.e. we define the membership functions, buildthe rules set and produce the measurement of xenophobia. First of all we identify the inputand output variables of the FIS. The input variables of the FIS are IPA, SSF and PS, corre-sponding to Factor I, II, and III, respectively and a number of other variables important forthe measurement of xenophobia (additional information), i.e. age, education, gender, politicsand religious practice. We denote by x(i) the input vector of the i-th respondent, i.e. x(i) =

123


Table 3 Loadings for principal components and factor using varimax and promax rotations

Rotation Item Principal components analysis(components)

Maximum likelihood factor analysis(factors)

I II III I II III

Varimax

Item 1 0.429 0.265 0.390 0.408 0.263 0.325

Item 7 0.693 0.001 0.107 0.568 0.044 0.137

Item 11 0.642 0.247 0.254 0.586 0.245 0.258

Item 13 0.757 0.114 0.101 0.681 0.129 0.123

Item 15 0.718 −0.034 0.096 0.590 0.016 0.130

Item 17 0.543 0.271 0.162 0.452 0.252 0.199

Item 2 0.031 0.639 0.079 0.072 0.489 0.115

Item 6 0.037 0.785 0.023 0.051 0.699 0.052

Item 12 0.259 0.656 0.292 0.265 0.609 0.282

Item 14 0.077 0.756 0.032 0.087 0.655 0.073

Item 16 0.354 0.524 0.437 0.354 0.496 0.363

Item 18 0.405 0.469 0.371 0.389 0.432 0.363

Item 4 0.169 0.018 0.661 0.233 0.088 0.424

Item 8 0.211 0.144 0.668 0.254 0.170 0.513

Item 10 0.050 0.115 0.810 0.093 0.118 0.731

Promax

Item 1 0.338 0.127 0.296 0.331 0.117 0.217

Item 7 0.792 −0.159 −0.066 0.673 −0.129 −0.039

Item 11 0.636 0.099 0.074 0.595 0.075 0.062

Item 13 0.846 −0.039 −0.116 0.805 −0.045 −0.126

Item 15 0.832 −0.202 −0.077 0.712 −0.167 −0.049

Item 17 0.542 0.168 −0.016 0.443 0.137 0.029

Item 2 −0.118 0.714 −0.055 −0.077 0.554 −0.012

Item 6 −0.123 0.895 −0.159 −0.131 0.843 −0.152

Item 12 0.085 0.636 0.137 0.075 0.599 0.111

Item 14 −0.072 0.852 −0.152 −0.082 0.772 −0.123

Item 16 0.181 0.430 0.317 0.160 0.388 0.303

Item 18 0.272 0.373 0.234 0.248 0.320 0.227

Item 4 −0.005 −0.176 0.757 0.105 −0.093 0.470

Item 8 0.018 −0.040 0.725 0.074 −0.027 0.562

Item 10 −0.212 −0.078 0.951 −0.214 −0.129 0.939

Correlations among components or factors

I – –

II 0.467 – 0.566 –

III 0.555 0.504 – 0.688 0.618 –

Loadings greater than 0.220 are presented in boldface

123

M. Symeonaki et al.

1

0

Middle Aged Old

Age (in years)18

80

Young

Mem

bers

hip

30 40 6050 70

Fig. 7 Fuzzy quantization of the range [18, 80] by trapezoidal-shape fuzzy numbers

[val(I P A)(i), val(SSF)(i), val(P S)(i), Age(i), Education(i), Gender(i), Poli tics(i),Religious Practice(i)]. A possible realization could be: x(i) = [1.82, 3.46, 2.67, 23, 4, 1,

4, 3], for the i-th respondent. Age is measured in years, Education in one of the categoriespresented in Table 1, Gender is either Male (1) or Female (2), Politics is a variable that takesvalues from 1 (Left wing) to 10 (Right wing) and Religious practice (how often do you go tochurch) takes values from Every Sunday or more often (1) to Never (5). The output variableis xenophobia taking values in [15, 75].

After identifying the input and output variables and the ranges of their values, we selectmeaningful linguistic values for each variable and express them by appropriate fuzzy sets.These fuzzy sets are fuzzy numbers representing linguistic values such as young, old, high,low, etc. The range of Age is, for example, [18, 80]. Figure 7 illustrates the proposedfuzzy quantization of the range [18, 80] by trapezoidal-shaped fuzzy numbers, where thelinguistic values selected for this variable are: Young, MiddleAged, and Old. The repre-sentation of the input variables as fuzzy variables is presented in Fig. 8. Fuzzy partitionsA1, A2, A3, A4, A5, A6, A7, A8 are defined on the domains of the fuzzy variables IPA, SSF,PS, Age, Education, Gender, Politics and ReligionPractice, respectively. B denotes the fuzzypartition on the domain of XENOPHOBIA.

In the next step, the knowledge pertaining to the given problem is formulated in terms of aset of fuzzy inference rules. The experts’ knowledge and the results of the statistical analysisare combined providing a fuzzy expert system for measuring xenophobia, as shown in Fig. 9.It is known, for example, that the elderly, those without formal education and groups withfrequent participation in religious practice have more pronounced elements of xenophobia.

An excerpt of the fuzzy rule base for XENOPHOBIA, which consists of 53 inferencerules of the canonical form, i.e. IF-THEN rules, is the following:

1. IF (IPA, SSF, PS, Age, Education, Gender, Politics, ReligiousPractice) IS (Favourable,NonStereotypical, Low, Young, High, Male, LeftWing, Rarely), THEN XENOPHOBIAIS Low,

2. IF (IPA, SSF, PS, Age, Education, Gender, Politics, ReligiousPractice) IS (Neutral,SomeStereotypes, Medium, MiddleAged, Medium, None, Centre, Occasionally), THENXENOPHOBIA IS Medium,

3. IF (IPA, SSF, PS, Age, Education, Politics, ReligiousPractice) IS (Unfavourable, Stereo-typical, High, Old, Low, RightWing, Female, Frequently), THEN XENOPHOBIA ISHigh.

We then select a suitable defuzzification method. The purpose of defuzzification is toconvert each conclusion obtained by the set of inference rules to a single real number (Klir andYuan 1995). The output of the system after the defuzzification process, is a score measuringxenophobia for each respondent. The data from 1,200 respondents (valid data missing values

123


Fig. 8 Representation of factors I, II and III produced in assessing construct validity of the Likert scalemeasuring xenophobia, as fuzzy variables intervention by public authorities (IPA), stereotypical stigmatizationof foreigners (SSF) and projection of self (PS)

excluded N = 1, 088) were introduced to the FIS developed. A set of scores for xenophobiaranging from 19.06 to 74.94 were produced with lower scores indicating a very little or notxenophobic attitude and higher scores a very xenophobic attitude. The output of the systemfor the first respondent, for example, with:

x(1) = (3.811 2.27 2.15 31 6 1 2 3) ,

123

M. Symeonaki et al.

1

0

1

0

1

01

1

1

PSSSFIPA AGE EDUCATION GENDER POLITICSRELIGIOUS PRACTICE

FavourableNeutral

UnFavourable

NonStereotypicalSomeStereotypes

Stereotypical

LowMedium

High

1

0

YoungMiddleAged

Old

Age18 80

1

0

LowMedium

High

Education 1 7

1

0

Male Female

Gender 1 2

1

0

LeftWingCentre

RightWing

Politics 1 10

1

0

1

FrequentlyOccasionally

Rarely

ReligiousPractice

RULE 1: IF (IPA, SSF, PS, Age, Education, Sex, Politics, ReligiousPractice) IS (Favourable, NonStereotypical, Low, Young, High, Male, LeftWing, Rarely) THEN Xenophobia IS Low

EXPERTS' KNOWLEDGE/THEORY AND STATISTICAL ANALYSIS

RULE 53: IF (IPA, SSF, PS, Age, Education, Sex, Politics, ReligiousPractice) IS (UnFavourable, Stereotypical, High, Old, Low, Female, RightWing, Frequently) THEN Xenophobia IS High

.

.

.

Correlation analysis:Comparison with crisp measure of Xenophobia

Indicators of Xenophobia

SYSTEM RULES

1

0

15

LowMedium

High

75Xenophobia

Fig. 9 The operation of the system

using as a defuzzification process a very popular defuzzification method, the centroid method(also called centre of area method or centre of gravity) results in scoring 49.19 on the xeno-phobia scale. The certain defuzzification method can be interpreted as an expected value ofxenophobia.

5.5 Validation

In order to validate the proposed method, fuzzy scores were correlated with five single itemsthat are considered in the literature as indicators of xenophobia (Eurobarometer 1989). Also,a combination of four of these items was used as an indicator of xenophobia (Michalopoulouet al. 1998). These indicators measure xenophobia based on the perception of the numberof “others” (of another nationality or religion) and the disturbance caused by their presence.

123


Table 4 Xenophobia (crisp andfuzzy) for the first 10 respondents

Respondent Xenophobia (crisp) Xenophobia (fuzzy)

1 50 49.19

2 29 25.11

3 35 39.95

4 51 64.63

5 54 64.41

6 53 55.65

7 52 54.24

8 48 45.00

9 39 37.76

10 37 35.55

Table 5 Pearson’s CorrelationCoefficients

N = 1, 088. Correlations aresignificant at p < 0.001

Xenophobia (fuzzy) Xenophobia (crisp)

Xenophobia (fuzzy) 0.860

Xenophobia (crisp) 0.860

Table 6 Xenophobia indicators

Indicator Question

1 During the last years individuals from other countries which are not members of theEuropean Union have come to live and work in Greece. According to your opinion, theseforeigners who live today in Greece are too many, many but not too many, not too many

2 How do you feel about the presence of individuals of another nationality. Disturbing ornot disturbing?

3 How do you feel about the presence of individuals of another religion. Disturbing or notdisturbing?

4 In your opinion these individuals of another nationality are too many, many but not toomany or not too many

5 In your opinion these individuals of another religion are too many, many but not too manyor not too many

6 Combination of indicators 2–5

Table 4 presents the scores of xenophobia (crisp and fuzzy) for the first ten respondents. Thefuzzy and crisp scores of respondents 4 and 5 are remarkably different. Investigating thecircumstances under which this has happened it was found that these cases responded in adifferent manner in the other variables used for the construction of the system’s rules.

The results of correlation analysis between xenophobia (fuzzy) and xenophobia (crisp)are presented in Table 5. As expected xenophobia crisp and fuzzy are highly correlated.The indicators used are given in Table 6, whereas Table 7 exhibits the correlation analysisresults between xenophobia fuzzy and crisp and all xenophobia indicators. As shown fuzzyscores are higher correlated with all xenophobia indicators, thus obviously producing a moreaccurate measurement of xenophobia.

123

M. Symeonaki et al.

Table 7 Spearman’s rhocorrelation coefficients

N = 1, 088. All correlations aresignificant at p < 0.001

Indicator Xenophobia (fuzzy ) Xenophobia (crisp)

1 −0.296 −0.242

2 −0.498 −0.439

3 −0.477 −0.381

4 −0.369 −0.216

5 −0.294 −0.215

6 0.573 0.422

6 Conclusions

Central to attitude measurement in social survey research is Likert scaling theory. The presentpaper aims to design a new methodology for measuring constructs, by combining Likert scal-ing theory, factor analysis and fuzzy set theory. The proposed methodology can be applied inthe construction of a single overall scale or subscales, considering “cross-loading” items andpreserving the original scoring as is the case in most explanatory social research situations.We are motivated by the fact that in almost all real applications, “cross-loading” items exist,but they are treated only in the context of composite scales that they do not preserve theoriginal data. The demonstrated methodology, additionally suggests that semantic informa-tion, usually available by the experts of the attitude domain, must also be taken into account,together with results of the statistical analysis produced by the current or previous studiesand therefore handle the uncertainty introduced to attitude measurement in social surveyresearch. The proposed method was tested on a Likert scale that was used in a large-scalesurvey for measuring xenophobia. The findings show that the measurement of xenophobiaproduced is valid and more accurate since correlation analysis revealed that (a) xenophobiascores (fuzzy and crisp) are highly correlated and more importantly, (b) fuzzy scores arehigher correlated with a number of xenophobia indicators. Thus, the proposed methodology,by incorporating the methods traditionally used for assessing the scale’s construct validityand using all available information and experts’ knowledge, extends current theory by pro-viding a more accurate measurement of an attitude. Furthermore, the method may be appliedto the multi-dimensional case, using the loadings resulting from factor analysis, justified byConfirmatory Factor Analysis.

References

Anagnostopoulos, F., Yfantopoulos, J., Moustaki, I., Niakas, D.: Psychometric and factor analytic evaluationof the 15D health-related quality of life instrument: the case of Greece. Qual. Life Res. 22(8), 1973–1986(2013)

Aydin, O., Pakdil, F.: Fuzzy SERVQUAL analysis in airlines services. Organizacija 41, 108–115 (2008)Bartholomew, D.J., Steele, F., Moustaki, I., Galbraith, J.: Analysis of Multivariate Social Science Data. Chap-

man and Hall/CRC, London (2008)Betti, G., D’Adostino, A., Neri, L.: Educational mismatch of graduates: a multidimensional and fuzzy indicator.

Soc. Indic. Res. 103, 465–480 (2011)Clarkand, L.A., Watson, D.: Constructing validity: basic issues in objective scale development. Psychol.

Assess. 7(2), 309–319 (1995)De la Rosa, S., Gil, M.A., López, M.T., Lubiano, M.A.: Fuzzy rating vs. fuzzy conversion scales: an empirical

comparison through the MSE. In: Kruze, R., Berthold, M., Moewes, C., Gil, M.A., Grzegorzewski, P.,Hryniewicz, O. (eds.) Synergies of Soft Computing and Statistics for Intelligent Data Analysis, Advancedin Intelligent Systems and Computing, vol. 190, pp. 135–143. Springer, Berlin (2013)

123


DiStefano, C., Zhu, M., Mîndrila, D.: Understanding and using factor scores: considerations for the appliedresearcher. Pract. Assess. Res. Eval. 14(20), 1–11 (2009)

Dubois, D., Prade, H.: The three semantics of fuzzy sets. Fuzzy Sets Syst. 90, 141–150 (1997)Eurobarometer: Racism and Xenophobia. Commission of the European Communities, Brussels (1989)Fabrigar, L., Wegener, D., MacCallum, R., Strahan, E.: Evaluating the use of exploratory factor analysis in

psychological research. Psychol. Methods 4(3), 272–299 (1999)Gacto, M.J., Alcalá, R., Herrera, F.: A double axis classification of interpretability measures for linguistic

fuzzy rule-based systems. In: Fanelli, A.M., Pedrycz, W., Petrosino, A. (eds.) Fuzzy Logic Applications,pp. 99–106. Springer, Berlin (2011)

Gil, M.A., Gil, G.-R.: Fuzzy vs. Likert scale in statistics. Comb. Exp. Theory Stud. Fuzziness Soft Comput.271, 407–420 (2012)

Gil, M.A., de Saa, S., López, M.T., Lubiano, M.A.: Comparing Likert and Fuzzy Scales Through SomeAtatistical Tools. ERCIM, London (2011)

Hartley, J., Betts, L.R.: Four layouts and a finding: the effects of changes in the order of the verbal labels andnumerical values on Likert-type scales. Int. J. Soc. Res. Methodol. 13, 17–27 (2009)

Hesketh, T., Hesketh, B.: Computerised fuzzy ratings: the concept of a fuzzy class. Behav. Res. MethodsInstrum. Comput. 26(3), 272–277 (1994)

Hesketh, T., Pryor, R., Hesketh, B.: An application of a computerized fuzzy graphic rating scale to the psy-chological measurement of individual differences. Int. J. Man-Mach. Stud. 29(1), 21–35 (1988)

Klir, G.J., Yuan, B.: Fuzzy Sets and Fuzzy Logic Theory and Applications, 1st edn. Prentice Hall, New York(1995)

Lalla, M., Facchinetti, G., Mastroleo, G.: Ordinal scales and fuzzy set systems to measure agreement: anapplication to the evaluation of teaching activity. Qual. Quant. 38, 577–601 (2005)

Lazim, A., Abu Osman, M.T.: Measuring teacher’s beliefs about Mathematics: a fuzzy set approach. Int. J.Soc. Humanit. Sci. Eng. 8(1), 39–43 (2009)

Lazim, A., Abu Osman, M.T.: Fuzzy set conjoint model in describing student’s perceptions on computeralgebra system learning environment. Int. J. Comput. Sci. Issues 8(2), 92–97 (2011)

Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. 140, 1–55 (1932)Lozano, L.M., García-Cueto, E., Muñiz, J.: Effect of the number of response categories on the reliability and

validity of rating scales. Methodology 4, 73–79 (2008)Michalopoulou, C., Tsartas, P., Giannisopoulou, M., Kafetzis, P., Manologlou, E.: Macedonia and the Balkans:

xenophobia and development, in Greek. National Centre for Social Research-Alexandria, Athens, Greece(Abridged English edition 1999) (1998)

Mogharreban, N., Dilalla, L.: Fuzzy inference in the analysis of non-interval data. In: Proceedings of theWSEAS International Conference on Simulation, Modeling and Optimization, Lisbon, Portugal, pp. 22–24(2006)

Moser, C., Kalton, G.: Survey Methods in Social Investigation. Heineman Educational Books, London (1971)Nunnally, J.C., Bernstein, I.H.: Psychometric Theory, 3rd edn. McGraw-Hill, New York (1994)O’Connor, B.P.: SPSS and SAS programs for determining the number of components using parallel analysis

and Velicer’s MAP test. Behav. Res. Methods Instrum. Comput. 32, 396–402 (2000)Reuse, S., Waller, N., Comrey, A.: Factor analysis and scale revision. Psychol. Assess. 12(3), 287–297 (2000)Revelle, W., Zinbarg, R.E.: Coefficients alpha, beta, omega, and the glb: comments on Sijtsma. Psychometrika

74(1), 145–154 (2009)Sah, M., Degtiarev, K.: Forecasting enrolment model based on first-order fuzzy time series. In: Proceedings

of the World Academy of Science, Engineering and Technology I, pp. 132–135 (2005)Song, Q., Chissom, B.: Forecasting enrolments with fuzzy time series: part II. Fuzzy Sets Syst. 62, 1–8 (1994)Stamou, G., Tzafestas, S.: Fuzzy relation equations and fuzzy inference systems: an inside approach. IEEE

Trans. Syst. Man Cybern. 99(6), 694–702 (1999)Stephan, F.F., McCarthy, P.J.: Sampling Opinions: An Analysis of Survey Procedure. Greenwood, Westport

(1974)Steiner, D.L., Norman, G.R.: Health Measurement Scales: A Practical Guide to Their Development and Use,

4th edn. Oxford University Press, Oxford (2008)Stevens, J.: Applied Multivariate Statistics for the Social Sciences, 4th edn. Lawrence Erbaum Associates,

Mahwah (2002)Symeonaki, M., Kalamatianou, A.: Markov systems with fuzzy states for the description of students’ edu-

cational progress in Greek Universities. Proceedings of the 58th World Statistics Congress ISI, Dublin,Ireland (2011)

Symeonaki, M., Kazani, A.: Developing a Fuzzy Likert Scale for Measuring Xenophobia in Greece. ASMDA,Rome (2011)

123

M. Symeonaki et al.

Tabachnick, B.G., Fidell, L.S.: Using Multivariate Statistics. Pearson Allyn and Bacon, Upper Saddle River(2007)

Thompson, B.: Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications.The American Psychological Association, Washington, DC (2005)

Turksen, I.B., Willson, I.A.: A fuzzy set preference model for consumer choice. Fuzzy Sets Syst. 68, 253–266(1994)

Zadeh, L.A.: Fuzzy sets. Inf Control 8(3), 338–353 (1965)

123

a fuzzy set theory solution to combining likert items into a single overall scale (or subscales)

Documents