Variable selection in PCA in sensory descriptive and consumer data

Frank Westada,*, Margrethe Hersletha, Per Leaa, Harald Martensb

aMATFORSK, Norwegian Food Research Institute, Osloveien 1, N-1430 As, NorwaybSensory Science, The Royal Veterinary and Agricultural University, DK-1958 Frederiksberg, Denmark

Received 30 July 2002; received in revised form 6 November 2002; accepted 12 November 2002

Abstract

This paper presents a general method for identifying significant variables in multivariate models. The methodology is applied onprincipal component analysis (PCA) of sensory descriptive and consumer data. The method is based on uncertainty estimates fromcross-validation/jack-knifing, where the importance of model validation is emphasised. Student’s t-tests based on the loadings andtheir estimated standard uncertainties are used to calculate significance on each variable for each component. Two data sets are

used to demonstrate how this aids the data-analyst in interpreting loading plots by indicating degree of significance for each vari-able in the plot. The usefulness of correlation loadings to visualise correlation structures between variables is also demonstrated.# 2003 Elsevier Science Ltd. All rights reserved.

Keywords: PCA; Descriptive sensory data; Consumer data; Variable selection; Validation

1. Introduction

In multivariate analysis where data-tables with sen-sory descriptive and consumer-related variables arestudied, it is important to extract the interpretable andstatistically reliable information. One objective may beto find significant sensory attributes in sensory descrip-tive analysis. Whereas descriptive analysis of sensorydata often yields high explained variance, consumer-related data such as preference data are less structured.There may be several reasons for this phenomenon(Lawless & Heymann, 1998). Firstly, the consumersmay not differentiate among the products at all, eitherbecause of too similar product samples or because theconsumers are indifferent to the attributes in the pro-ducts. Consumer ratings would consequently not fit wellinto the product space. Secondly, some consumers maybase their hedonic scores on factors (sensory or non-sensory) that were not included in the product spacederived from the analytical sensory data. Thirdly, someconsumers simply yield inconsistent, unreliable respon-ses, possibly because they changed their criteria foracceptance during the test. Unreliable responses can

also be the result of consumers who are not motivatedto take part and therefore answer randomly.Many different groups of background variables are

usually available for the consumers, such as demo-graphy, eating habits, attitudes, etc. Socio-economicvariables often serve as a basis for segmentation of theconsumers before relating them to sensory descriptivedata with some preference mapping method (Helgesen,Solheim, & Næs, 1997). However, the segmentationshould be validated, and removing non-relevant vari-ables is usually more essential for these data than forsensory data. The main focus in this paper is to findrelevant variables in sensory descriptive and consumerdata, although the method can be applied on any datatable.For the sensory data on K sensory attributes, where L

assessors in a trained panel have evaluated N products,analysis of variance (ANOVA) is usually employed toassess which individual sensory attributes are sig-nificant. These tests will reveal if assessors are able todistinguish among products on selected sensory attri-butes, and the average response over the assessors isoften computed before further analyses are per-formed. This data table of averaged responses is thebasis for the analysis as described below. As a resultfrom the ANOVA, one might exclude assessors ordown-weight some assessors for certain attributes to

0950-3293/03/$ - see front matter # 2003 Elsevier Science Ltd. All rights reserved.

doi:10.1016/S0950-3293(03)00015-6

Food Quality and Preference 14 (2003) 463–472

www.elsevier.com/locate/foodqual

* Corresponding author. Tel.: +47-64-970303; fax: +47-64-

970333.

E-mail address: frank.westad@matforsk.no (F. Westad).

yield more reliable average values in the matrix of sizeN�K. (Lea, Rødbotten, & Næs, 1995).Principal component analysis (PCA) is a frequently

applied method for multivariate overview analysis ofsensory data (Helgesen et al., 1997; Jackson, 1991). Themain purpose is to interpret any latent factors spannedby characteristics such as flavour, odour, appearanceand texture, and to find products that are similar ordifferent, and what differentiates them. This is doneby studying loadings and score plots. In this context,it is of interest to assess which variables are sig-nificant on the individual components to simplify theinterpretation. There exist rules of thumb such as acut-off for loadings at values higher than, e.g. 0.3 toassess which variables are important. However, thechallenge for such general rules is that the squaredloadings sum up to 1.0, so that the cut-off is dependingof the number of variables as well as samples (Hair,Andersen, Tatham, & Black, 1998). Also, one shouldconsider the amount of explained variance for the com-ponent studied. Guo, Wu, Massart, Boucon, and deJong (2002) applied feature selection from Procrustesanalysis to find the best subset of variables to preserveas much information in the complete data as possible.Work on finding important variables has also been doneby, e.g. Krzanowski (1987) and Rannar, Wold, andRussel (1996).

1.1. Model rank

The main purpose of assessing the model rank is toprevent spurious correlations to be interpreted asmeaningful information. Methods to assess the correctrank based on cross-validation have been addressedextensively in latent variable regression methods such asprincipal component regression (PCR) and partial leastsquares regression (PLSR) (Green & Kalivas, 2002;Martens & Martens, 2000). Model results from thesemethods include root mean square error (RMSE) froma validation procedure, which (preferably) decreases,and thereafter increases, or approaches some asympto-tic value. This behaviour is not necessarily to be expec-ted for the residual cross-validated variance in PCAsince the space into which the deleted objects are pro-jected is expanding with more components. A correctionfor the degrees of freedom consumed as more compo-nents are extracted might aid in assessing the rank. Inthe cross-validation for PCA the correction K/(K�A) isemployed, where K is the number of variables and A isthe number of components.Also, the explained variance for the component is of

importance, as one component may not be relevant tointerpret at all. In PCA, there exists an ensemble ofmethods (Jackson, 1991) to find the correct rank. Pre-ferably, a robust method should give the correct rankautomatically from the analysis. Cross-validation

(Wold, 1987), inspection of scree-plot, ratio of eigen-values and Bartlett’s test for model dimensionality areamong the existing procedures (Jackson, 1991). Theterm ‘‘rank’’ with respect to a multivariate modeldeserves some comments, as ‘‘rank’’ has variousfacets:

1. Numerical. This rank is the one based on

numerical computations, e.g. the number ofcomponents that can be computed without sin-gularity problems.

2. Statistical. The important issue here is to find the

optimal rank from a statistical criterion, pre-ferably based on some proper validation method.

3. Application specific. Since significant is not the

same as meaningful, this judgement is typically acombination of background knowledge, modelcomplexity, and interpretation aspects. In mostsituations, this rank is lower than the statisticalrank, i.e. the data-analyst tends to be moreconservative.

1.2. Uncertainty estimates

Significance testing based on uncertainty estimates inregression has been published elsewhere (Martens &Martens, 2000; Westad & Martens, 2000), and hasrecently been applied in a related method to PCA,independent component analysis (ICA; Westad & Ker-mit, in preparation). Uncertainties may be estimatedfrom resampling methods such as jack-knifing andbootstrapping. Jack-knifing is closely connected tocross-validation, the difference lies in whether the modelwith all objects or the mean of all individual modelsfrom the resampling should be regarded as the ‘‘refer-ence’’. We feel that it is more relevant to use the modelon all objects as the reference, since this is the model weinterpret in terms of scores, loadings and other relevantplots. Thus, this approach for estimation might benamed modified jack-knifing (Martens & Martens,2000), and it is applied in this paper. According tostudies by Efron (1982), the difference between thesetwo is negligible in practical applications, especially forlarge numbers of objects. The main objectives with esti-mating uncertainty in multivariate models are to assessthe model stability and to find significant componentsand variables.Model validation is essential in all multivariate data

analysis. The validation can be either model validationon the data at hand, such as cross-validation (Wold,1978), or system validation. One example of the secondtype of validation is where a survey is repeated at dif-ferent times or in different segments to confirm thehypothesis we might have about the system we are try-ing to observe.

464 F. Westad et al. / Food Quality and Preference 14 (2003) 463–472

2. Materials and methods

2.1. Example 1: descriptive sensory evaluation of icecream

Fifteen different samples of vanilla ice cream wereevaluated by a panel using descriptive sensory analysisas described in ISO 6564:1985. The sensory panel con-sisted of 11 panellists selected and trained according toguidelines in ISO 8586-1:1993 and the laboratory wasdesigned according to guidelines in ISO 8589:1988. Thesamples were described using 18 different sensory attri-butes (Table 1). The panellists were given samples fromboth extreme ends of the scale to acquaint themselveswith the potential level of variation for the differentattributes. A continuous, unstructured 1.0–9.0 scale wasused for the evaluation. Each panellist did a monadicevaluation of the samples at individual speed on acomputerised system for direct recording of data (CSACompusense, version 5.24, Canada). Two replicatedmeasurements were made for each sample of ice cream.The samples were served in a randomised order. Repli-cates were randomised within the same session, so thatno replicate effect is needed in the models (Lea, Rød-botten, & Næs, 1997).

2.2. Example 2: consumer preference mapping ofmozzarella cheese

The second data set was taken from Pagliarini, Mon-telleone, and Wakeling (1997), where nine commercialmozzarella cheeses where evaluated by a trained sensorypanel, and six of them were selected for a preference testby 105 consumers. The six cheeses were selected to spanthe sensory characteristics of the nine cheeses. Thesamples were rated on a nine-point hedonic scale by theconsumers. In this paper the focus is on analysing thepreference data with N=6 products and K=105 con-sumers.

2.2.1. PCA and significance testsFor a matrix, X, assume the bilinear model structure

X ¼ TPT þ EA ð1Þ

where X (N�K) is a column centred data matrix; T(N�A) is a matrix of score vectors which are linearcombinations of the x-variables; P (K�A) is a matrix ofloading vectors, PTP=I; and EA (N�K) contains theresiduals after A principal components have beenextracted.The uncertainty of the loadings, s2(pak), may be esti-

mated from (Efron, 1982; Martens & Martens, 2000)

s pakð Þ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXMm¼1

pak�pakð�mÞ

� �2 !ðM� 1Þ=Mð Þ

vuut ð2Þ

where M=the number of segments in the cross-vali-dation; s(pak)=estimated uncertainty variance of eachvariable k in the loading for component a; pak=theloading for component a using all the N objects;pak(�m)=the loading for variable k for component ausing all objects except the object(s) left out in crossvalidation segment m.pak and s(pak) may be subjected to a modified t-test

(pa/s(pak)=0 with M degrees of freedom) to give sig-nificance values for individual variables for each com-ponent, and can also be used as an approximateconfidence interval around each variable. The jack-knifebased estimates tend to be conservative due to theinherent validation aspect. Univariate tests might not bethe best way to assess significance for multivariatemodels. For one thing, there is a danger of false posi-tives when applying many tests. Another aspect is that avariable may be significant in a multivariate sense,although individual tests do not give significance. Theseaspects are not pursued in this paper, but explainedvariance >50% has shown to be a good ad hoc rule toaid the decision of significance.

2.2.2. Rotation of modelsIn PCA, cross-validation for individual segments

might give components that are mirrored or flippedcompared to the model on all objects. The componentsmay even come out in a different order when the corre-sponding eigenvalues are similar and/or close to eigen-values of the noise part of the data. The PCs from thecross-validation must therefore be rotated towards thePCs based on all objects before the uncertainties areestimated. Procrustes rotation (Jackson, 1991; Milan &Whittaker, 1995) can be applied to rotate loadings andscores. The aim of Procrustes rotation is to make amatrix A similar to B by estimating a rotation matrix Cso that the squared residuals D are minimised

A ¼ BC þD ð3Þ

In this paper, the rotation matrix C in each submodelis estimated from the scores for objects not left out inthat segment (Martens & Martens, 2001) with orthogo-nal Procrustes rotation, and the inverse of C is thenapplied in rotating the loadings. Applying the rotationmatrix directly allows rotation and stretching of thesubmodel in the direction of the main model. Thereby,the submodel may be closer to the main model than wewanted from the original objective of flipping, mirroringand ordering of the components. This may give toooptimistic significance values in situations with fewobjects and/or skewed distribution of samples. Onealternative is then to round the elements in C to integervalues (�1,0,1) before scores and loadings are rotated.This can, however, give a rotation matrix that is notorthonormal when the submodel is rotated with anangle close to 45 degrees. The norm of the rounded

F. Westad et al. / Food Quality and Preference 14 (2003) 463–472 465

Table 1

Sensory data for the ice-cream with significance values from ANOVA and jack-knife estimates

Object Whiteness Colour

hue

Int. of

colour

Int. of

flavour

Acid

flavour

Sweet

flavour

Vanilla

flavour

Creamy

flavour

Egg

flavour

Metal

flavour

Sun

flavour

Rancid

flavour

Packaging

flavour

C a

fl o

moothness Thickness Viscosity Fattiness

1 6.33 2.00 2.85 6.97 4.79 6.35 5.59 3.94 2.63 2.32 1.96 1.02 1.21 . 6.50 4.44 2.82 3.92

2 5.52 3.85 4.40 6.53 4.35 6.03 5.53 4.72 3.78 2.07 2.09 1.10 2.83 . 6.35 5.45 3.41 4.72

3 6.31 2.43 2.89 6.36 4.45 5.88 5.30 4.02 2.89 2.39 2.40 1.15 1.95 . 6.58 4.40 2.95 4.03

4 6.90 1.83 2.26 6.05 4.71 5.85 5.65 4.41 3.11 2.14 1.67 1.06 1.71 . 7.38 4.96 3.47 4.26

5 5.72 3.41 3.85 6.12 4.69 5.81 5.39 5.08 4.04 2.45 2.44 1.23 2.55 . 7.33 5.76 4.56 4.88

6 5.39 4.13 4.37 6.16 4.28 5.73 5.26 4.63 3.57 2.41 3.09 1.41 3.56 . 6.43 4.76 3.26 4.13

7 5.73 3.54 3.96 6.29 4.75 5.84 5.54 5.11 4.06 2.12 1.92 1.06 2.15 . 7.26 5.59 3.72 4.66

8 5.35 3.94 4.40 6.44 4.70 5.98 5.86 5.00 4.12 2.44 2.34 1.28 2.59 . 7.30 5.60 3.61 4.65

9 5.73 3.32 3.97 6.20 4.94 6.00 5.51 5.05 3.86 2.14 1.54 1.03 1.88 . 7.37 5.47 3.81 4.55

10 5.77 3.37 3.95 6.26 4.59 5.93 5.36 4.86 3.44 2.02 1.91 1.18 2.37 . 7.10 5.55 3.96 4.50

11 5.73 3.56 4.16 6.33 4.57 5.94 5.29 4.80 3.53 2.32 2.02 1.11 2.17 . 6.83 5.25 3.81 4.49

12 6.02 3.35 3.67 6.28 4.95 6.01 5.50 5.25 4.04 2.03 1.52 1.17 1.56 . 7.19 5.50 3.55 4.64

13 5.85 3.36 3.83 6.26 4.97 5.80 5.56 5.23 3.88 1.96 1.72 1.13 1.98 . 7.39 5.57 3.45 4.81

14 5.40 3.99 4.38 6.33 4.94 5.89 5.73 5.35 4.48 1.92 1.58 1.05 1.70 . 7.48 5.82 3.85 5.00

15 5.69 3.64 4.05 6.45 4.68 6.05 5.59 5.35 3.97 2.16 2.07 1.14 1.95 . 7.52 5.95 4.12 4.86

S.D. 0.42 0.70 0.64 0.22 0.22 0.15 0.17 0.44 0.50 0.18 0.42 0.11 0.58 . 0.41 0.48 0.44 0.32

p(ANOVA) <0.001 <0.001 <0.001 0.062 0.233 0.805 0.396 <0.001 <0.001 0.269 <0.001 0.064 <0.001 < . 0.001 <0.001 <0.001 <0.018

p(PC1) 0.090 0.002 0.033 0.557 0.894 0.447 0.777 0.000 0.000 0.361 0.881 0.128 0.029 . 0.262 0.000 0.006 0.000

p(PC2) 0.334 0.292 0.334 0.649 0.204 0.983 0.272 0.258 0.515 0.022 0.001 0.056 0.022 . 0.000 0.078 0.220 0.113

p(PC3) 0.277 0.420 0.173 0.001 0.504 0.052 0.448 0.556 0.366 0.803 0.654 0.564 0.583 . 0.888 0.649 0.959 0.556

466

F.Westa

det

al./

FoodQuality

andPreferen

ce14(2003)463–472

ar

av

5

2

2

1

2

2

1

2

1

1

2

2

1

1

1

0

0

0

0

0

mel

ur

S

38

14

32

54

11

01

74

22

97

99

10

13

99

89

90

89

001 <

371

700

253

rotation matrix is computed to monitor that the round-ing procedure does not give more than one value (�1 or1) related to each component in the main model. In suchsituations, the component with the highest correlation tothe main model component is chosen. This, and furtheraspects about rotation and uncertainty estimation inbilinear models are treated in Høy, Westad, andMartens(in preparation). There is also a danger of overfitting ifthe rotation is performed with too many components. Toavoid this, the rotation is restricted to the relevant com-ponents only, as found from, e.g. cross-validation.

2.2.3. Sensory data—scaling or not?When analysing descriptive sensory data, the question

of scaling (weighting) arises, where the two most com-mon options are: (1.) Centring, but no scaling. (2.)Scaling to unit variance over the attributes. One argu-ment for not scaling is that the attributes are in the samenumerical range and thus the modelling should beemployed on the absolute numerical differences. On theother hand, some attributes may not span a large partof the range, but still describe systematic differencesamong the products.The loading plot is often used to interpret the attri-

butes in terms of correlation. However, when attributesare not scaled, high correlations between attributes maynot be revealed because the numerical ranges arespanned differently. The correlation loadings (Martens& Martens, 2000) are useful in revealing such unwan-ted effects of the choice of scaling. They show thecorrelations between the attributes and the principalcomponents, and the interpretation of the variables inthis plot is invariant to the scaling applied in the modelitself. The co-ordinates of the variables in this plot arethe square roots of the explained variance for eachcomponent.

2.2.4. The issue of mean centring and validationAssume that a PCA on preference data is one step in

the data analysis. In preference mapping, this matrix isoriented as product�consumer (‘‘short-fat’’, I�K),viewing each consumer as one ‘‘instrument’’ or variable.Mean centring over variables means that only the var-iance for individual consumers is taken into considera-tion; not the average preference. Cross-validation in thissituation is rather conservative due to the low numberof products (often as few as 5–8), and because the pro-ducts are deliberately chosen to span the multi-dimensional product space, often defined by the sensorydata. Thus, each product is somewhat unique, andremoving one product during cross-validation may alterthe model direction to a large extent if the removedproduct is extreme in a specific sense. Having a productas a kind of a ‘‘centre’’ sample might reduce the effect ofchange in the model direction. This can be visualised inthe stability plot (Martens & Martens, 2000).

Still, a significance test can be used to find informativeconsumers, but a significance level of 5% is often tooconservative. We are not too concerned about keepingsome consumers, although they are not significant atthis level; a level of 20% has been recommended (Mac-Fie, personal communication, 2001). The value of 20%is of course just as arbitrary as that of 5% in classicalstatistics. Interpretation may be done in the correlationloadings plot, which is invariant to the consumers’individual use of the preference scale. It means that aslong as one consumer is systematic in the way he/she isassessing the products, the range within a scale of 1–9he/she uses will be of less importance. On the otherhand, if the mean preference of individual consumers isnot of particular interest, the mean centring does notoppose the objective of the data analysis.When PCA is employed on the N�K matrix (‘‘long-

thin’’) with mean-centring, the average preference of theproduct will not influence the analysis; only the variancearound the mean contributes to the position in theloading plot for the products.One might argue that removing the average of the

products by mean centring has an undesirable effect,which is more detrimental in the sensory than in theconsumer case. This often makes the loading plot ofconsumers rather uninformative in the first component,since all products might be liked to some extent. Sub-tracting the grand mean (Jackson, 1991) or double-cen-tring are alternatives, but is not pursued in this paper.

3. Results and discussion

3.1. Results from analysis of the ice-cream data

Analysis of variance (ANOVA) was employed withmodel structure: samples (fixed effect), assessors (ran-dom effect), the interaction samples�assessors (randomeffect), and unit (box of ice-cream) nested within sam-ples. The unit takes the place of replicate here, and isalso a random effect. This was done in order to identifythe sensory attributes for which there were significantdifferences among the samples. Results from theANOVA are given in Table 1. The arithmetic averageresponse over assessors and sensory replicates for eachsample are used in the rest of this paper.PCA with mean centring was employed on the sen-

sory data, and significance values were estimated fromjack-knifing after rotation with three components asdescribed in Section 2 ( p-values for significance on thefirst three dimensions can be found in Table 1). Wechose a significance level of 10% since erroneouslyaccepting an attribute as significant at this level was notconsidered to be critical. The score and loading plot forcomponents 1 and 2 are shown in Fig. 1a and b. Table 2shows the explained variance for calibration and

F. Westad et al. / Food Quality and Preference 14 (2003) 463–472 467

Fig. 1. Results from PCA on mean centred ice-cream data. Score (1a) and loading (1b) plots.

468 F. Westad et al. / Food Quality and Preference 14 (2003) 463–472

validation. From the explained variance and by inter-pretation of the loading plots, three components werefound to be relevant. The first three PC’s explained 55, 22and 15% of the variation, respectively. The cross-vali-dated variances were 27, 6 and 27%, respectively, with K/(K�A) correction for degrees of freedom. The relativelypoor explained validation variance is due to the highleverage for object number 1, which has a high value forthe attribute caramel flavour. It should be mentionedthat Bartlett’s test, which looks for the number of thelargest unequal eigenvalues of the covariance matrix ofX suggests 13 components for this data set.

As mentioned earlier, interpretation of the loadingplot may not reveal the actual correlation structureamong the variables when the variables have differentstandard deviations, as shown in Table 1. This is illu-strated in Fig. 1b where the attributes fatness andthickness are not clearly interpreted as having a highcorrelation although they lie in the same direction.Correlation loadings (Appendix) are useful to interpretthe correlation structure between the variables and thePCs, regardless of how they were scaled prior to themodelling. In this case, the correlation loadings plot(Fig. 2) reveals that fatness and thickness are highlycorrelated (correlation 0.96), but this was not obvious inthe loading plot from the model on centred data.Correlation is in general not a reliable measure forunderstanding the data-structure, and a plot of thevariables itself will show the distribution for the actualvariables.In Fig. 2 the significant sensory attributes are marked,

with the circles indicating 50 and 100% explainedvariance, respectively. We see that the attributes inthe middle, with less than 50% explained variance,are not significant, but this is also true for caramelflavour, although its position is far from the origin. The

Table 2

Explained calibration and validation variance from PCA on the ice-

cream data

ExpVar

ExpVar, PC ExpVarVal ExpVarVal, PC

PC1

55.3 55 27.3 27

PC2

76.9 22 33.2 6

PC3

91.6 15 60.3 27

PC4

95.3 4 62.6 2

PC5

97.0 2 76.6 14

PC6

98.0 1 77.9 1

Fig. 2. Correlation loadings plot from PCA on mean centred data. Significant variables are marked with ‘‘+’’ (PC1), triangle (PC2) or square (both

PCs).

F. Westad et al. / Food Quality and Preference 14 (2003) 463–472 469

Fig. 3. (a) Score plot of the six mozzarella cheese products. (b) Correlation loadings with significant consumers at 20% level indicated by marker

codes. Significant variables are marked with ‘‘+’’ (PC1), triangle (PC2) or square (both PCs).

470 F. Westad et al. / Food Quality and Preference 14 (2003) 463–472

explanation is that the distribution of this attribute isvery skewed, so the correlation between attributecaramel flavour and the PC is mostly due to object 1.Therefore, the model changes considerably when thisobject is kept out during the cross-validation, and theuncertainty estimate thus becomes high. In contrast tomethods mentioned earlier in Section 2, the variableselection method used here works for keeping samplesout, rather than variables. This ensures that the modelis validated in terms of stability towards taking someobject(s) out.As seen from Table 1, only two attributes are found

to be significant at the 10% level for PC 3. These arealso the two attributes with explained variance close to50% on this component. The marking of significantattributes helps the user in discarding attributes that arenot relevant for the components shown in the plot. Inpractical data analysis, one might want to make a newmodel without the attributes that are not significant onany relevant component.

3.2. Analysis of the mozzarella cheese data

The data was subjected to PCA with the consumers asvariables (N�K) (‘‘short-fat’’) where each consumer wasmean centred. The score and correlation loading plotsare shown in Fig. 3a and b. From the score plot it canbe seen that PC2 is spanned by product 4, and someconsumers are found to be not significant at the 20%level (black dots) although they have a high explainedvariance. Examples of such consumers can be seen onthe axis for PC2 near the 100% explained variance cir-cle. We might still want to keep these consumers in fur-ther analysis, but the purpose of the cross-validation/jack-knifing is to visualise that the model is not stabledue to the uneven distribution of the products alongPC2. A diagnostic tool to visualise the model stability isthe stability plot (Martens & Martens, 2000) whichshows how the model changes when one object is takenout during cross-validation. However, we may stilldecide to assign these consumers to component 2 sincethe objects in the study deliberately have been chosen tospan the subspace without redundancy, and therebyyielding low model stability. The uncertainty estimates,nevertheless, indicate a need to investigate the datastructures in more detail.The majority of the consumers inside the 50%

explained variance circle in the plot are not significant,or we may want to name them non-informative con-sumers or consumers with no systematic assessment ofthe products. This corresponds well with experiencefrom analysis of sensory data that variables with lessthan 50% explained variance are not significant.Fig. 2 shows that this also applies to the sensoryvariables for the ice-cream data. When the objectivein further data analysis is to segment the consumers,

one may want to take out the non-informative con-sumers and label them as a segment with no specificpreference.

4. Conclusions

In multivariate methods such as PCA, where inter-pretation of loading plots is the main objective, it isimportant to find which components are relevant andwhich variables are significant on the components.Finding the correct model dimensionality from cross-validation in PCA is not straightforward since the resi-dual validation variance does not necessarily have aminimum. The significance values based on uncertaintyestimates from jack-knifing are useful for visualisingwhich attributes are relevant to the interpretation, andfor finding informative consumers. Cluster analysis maybe employed in the correlation loadings plot after thenon-informative consumers are taken out as a clusterwith ‘‘no preference’’. The validity of such a procedurein segmentation of consumers will be discussed in aforthcoming paper.Mean centring, but no scaling of variables in sensory

and consumer data may give misleading loading plotswhen interpreting the structure of the data. The corre-lation loading plot is useful for visualising each vari-able’s correlation along each component and betweenthe variables themselves, regardless of how the variableswere scaled in the analysis.

Acknowledgements

The authors wish to thank Elin Kubberød andØyvind Langsrud for valuable comments. One of thereviewers is thanked for suggestions that led to animprovement of the part about rotation and uncertaintyestimates. This work was partially funded by The Nor-wegian Research Council (Project 132975/112).

Appendix. Correlation loadings

The correlation loadings may be computed from theformula

rka ¼ pka

ffiffiffiffiffiffiffiffiffiffitTa ta

q=ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffieT0;k e0;k

qðA1Þ

where rpa=correlation loading for x-variable #k; pka=conventional loading for x-variable #k; e0,k=mean-centred x-variable #k, xk-x; and ta=score vector (N�1)for PC # a (with suitable correction for any missingvalues in e0,k).

F. Westad et al. / Food Quality and Preference 14 (2003) 463–472 471

References

Efron, B. (1982). The Jackknife, the bootstrap and other resampling

plans. Philadelphia, PA: Society for Industrial and Applied Mathe-

matics (ISBN 0-89971-179-7).

Green, R. L., & Kalivas, J. H. (2002). Graphical diagnostics for

regression model determinations with consideration of the bias/var-

iance trade-off. Chemometrics and Intelligent Laboratory Systems,

60, 173–188.

Guo, Q., Wu, W., Massart, D. L. C., Boucon, C., & de Jong, S. (2002).

Feature selection in principal component analysis of analytical data.

Chemometrics and Intelligent Laboratory Systems, 61, 123–132.

Jackson, J. E. (1991). A user’s guide to principal components. New

York: John Wiley & Sons.

Hair, J.F, Anderson, R. E., Tatham, R. L., & Black, W. C. (1998).

Multivariate data analysis (5th ed.). London: Prentice Hall Interna-

tional.

Helgesen, H., Solheim, R., & Næs, T. (1997). Consumer preference

mapping of dry fermented lamb sausage. Food Quality and Pref-

erence, 8, 97–109.

Høy, M., Westad, F., & Martens, H. (in preparation) Improved jack-

knife variance estimates of bilinear model parameters. Journal of

Chemometrics (in preparation).

Krzanowski, W. J. (1987). Selection of variables to preserve multi-

variate data structure, using principal components. Applied Statis-

tics, 36, 22–33.

Lawless, H., & Heymann, H. (1998). Sensory evaluation of food.

Principles and practices. New York: Chapman & Hall (ISBN 0-412-

99441-0).

Lea, P., Rødbotten, M., & Næs, T. (1995). Measuring validity in sen-

sory analysis. Food Quality and Preference, 6, 321–326.

Lea, P., Rødbotten, M., & Næs, T. (1997). Analysis of variance for

sensory data. Chichester, UK: John Wiley & Sons (ISBN 0-471-

96750-5).

Martens, H., & Martens, M. (2000). Modified Jack-knife estimation of

parameter uncertainty in bilinear modelling (PLSR). Food Quality

and Preference, 11, 6–15.

Martens, H., & Martens, M. (2001). Multivariate analysis of quality.

An introduction. Chichester UK: John Wiley & Sons.

Milan, L., & Whittaker, J. (1995). Application of the parametric

bootstrap to models that incorporate a singular-value decomposi-

tion. Applied Statistics, 44, 31–49.

Pagliarini, E., Montelleone, E., & Wakeling, I. (1997). Sensory profile

description of mozzarella cheese ant its relationship with consumer

preference. Journal of Sensory Studies, 12, 285–301.

Rannar, S., Wold, S., & Russell, E. (1996). Selection of spanning

variables in PCA, In S. Rannar (Ed.).Many variables in multivariate

projection methods. (PhD thesis). Sweden: Department of Organic

Chemistry, Umea University.

Westad, F., & Kermit, M. Cross validation and uncertainty estimates

in independent component analysis. Analytica Chimica Acta (sub-

mitted for publication).

Westad, F., & Martens, H. (2000). Variable selection in NIR based on

significance testing in partial least squares regression (PLSR). Jour-

nal of Near-Infrared Spectroscopy, 8, 117–124.

Wold, S. (1978). Cross-validatory estimation of the number of com-

ponents in factor analysis and principal component models. Tech-

nometrics, 20, 397–406.

472 F. Westad et al. / Food Quality and Preference 14 (2003) 463–472