do people fake on personality inventories

7/29/2019 Do People Fake on Personality Inventories

1/21

DO PEOPLE FAKE ON PERSONALITYINVENTORIES? A VERBAL PROTOCOL

ANALYSIS

Chet RobieWilfrid Laurier University

Douglas J. Brown

University of Waterloo

James C. BeatyPreVisor

ABSTRACT: Research has focused heavily on whether individuals can fake onpersonality inventories. Research is less clear on whether individuals actually dofake on personality inventories. Verbal protocol analysis was used to trace themotivational processes for 12 participants as they completed a personalityinventory in an applicant context. Exploratory analyses suggested that individ-uals do fake on personality inventories; that individuals can be classed into one ofthree faking classes (honest responders, slight fakers, and extreme fakers); andthat honest fakers take less time to complete and make less corrections to theirpersonality inventories than faking responders. Study implications, limitations,and future research will be discussed.

KEY WORDS: faking; personality testing; verbal protocol analysis.

INTRODUCTION

Findings from years of research strongly support the use of per-sonality inventories in selection systems. Barrick, Mount, and Judge(2001) quantitatively summarized the results of 15 prior meta-analytic

Address correspondence to Chet Robie, E-mail: [email protected]

This research was funded in part by Social Science and Humanities Research Council ofCanada Grant # 410-2005-0207 awarded to Chet Robie and Douglas J. Brown.

Journal of Business and Psychology, Vol. 21, No. 4, Summer 2007 (2007)

DOI: 10.1007/s10869-007-9038-9

489

0889-3268/07/0600-0489/0 2007 Springer Science+Business Media, Inc.


2/21

studies that have investigated the relationship between the Five FactorModel (FFM) personality traits and job performance. Their primaryresults were that conscientiousness was a valid predictor across perfor-mance measures in all occupations studied; emotional stability was foundto be a generalizable predictor when overall work performance was thecriterion, but its relationship to specific performance criteria and occu-pations was less consistent than was conscientiousness; and the otherthree Big Five traits (extraversion, openness and agreeableness) did notpredict overall work performance, but they did predict success in specificoccupations or relate to specific criteria. Moreover, organizations areusing personality inventories in their staffing systems. In a survey of 959organizations in 20 different countries, personality inventories werefound to be the most often used method of selection (Ryan, McFarland,Baron, & Page, 1999).

One issue that continues to concern practitioners is the issue offaking (i.e., intentional or motivated distortion) on personality invento-ries. For the purpose of the present study, we define faking as a consciousattempt to manipulate ones responses to create a positive impression(McFarland & Ryan, 2000; Zicker & Robie, 1999). A survey of 77 asses-sors who conducted individual assessments for an international con-

sulting firm was recently conducted (Robie, Tuzinski, & Bly, 2005). Oneof the findings was that a majority of the assessors (70%) believed thatfaking was a serious threat to the validity of the personality inventory inthe assessment process. The current study is designed to examine whe-ther this concern by practitioners is justified; specifically, do people fakeon personality inventories? We will first briefly review the literature onfaking on personality inventories. We will then provide an overview ofthe primary methodology we used in the current study (verbal protocolanalysis) and how it has been used in prior studies. Finally, we willpresent our study hypotheses.

Faking on Personality Inventories

A great deal of research has been conducted on faking on personalityinventories. The majority of research has attempted to answer thefollowing research questions: (1) Can people fake on personality inven-tories?; (2) Does faking affect the measurement properties/validity ofpersonality inventories?; and (3) What can be done to reduce faking orthe effects of faking?

The research is most clear on the first research question. Meta-analytic research across 51 studies has found that individuals can fakepersonality inventories when instructed to do so (Viswesvaran & Ones,1999). Research also shows that individuals vary considerably in theirability to fake (McFarland & Ryan, 2000; Mersman & Shultz, 1998).

JOURNAL OF BUSINESS AND PSYCHOLOGY490


3/21

The research on the next research question is less clear. Researchconducted by (Ellingson, Smith, & Sackett, 2001; Smith & Ellingson,2002) found that faking does not affect personality structure, althoughother researchers suggest that faking does affect psychometric properties(Stark, Chernyshenko, Chan, Lee, & Drasgow, 2001). In a multi-samplestudy, Hough, Eaton, Dunnette, Kamp, and McCloy (1990) found thatpersonality validities remained stable regardless of possible distortionby respondents in either unusually positive or negative directions;whereas, other researchers have found faking to result in decrementsto criterion-related validity (Douglas, McDaniel, & Snell, 1996;Topping & OGorman, 1997). Many studies have found prima facieevidence of faking (cf. Bass, 1957; Becker & Colquitt, 1992; Dunnette,McCartney, Carlson, & Kirchner, 1962)that is, a group that onewould expect to score higher on a noncognitive test (e.g., applicants) onaverage does score higher than a control group (e.g., honest group).Several researchers suggest that faking may affect hiring decisions(Christiansen, Goffin, Johnston, & Rothstein, 1994; Mueller-Hanson,Heggestad, & Thornton, 2003; Rosse, Stecher, Miller, & Levin, 1998).

The final research question has arguably led to the most controversyand the least clarity. Research has found that measures that contain

items that are ipsative (Bowen, Martin, & Hunt, 2002); forced-choice(Christiansen, Burns, & Montgomery, 2005; Jackson, Wroblewski, &Ashton, 2000);1 or subtle (Holden & Jackson, 1981; Worthington &Schlottmann, 1986) may be better than normative measures for con-trolling faking. However, issues of applicant comparison for these typesof measures complicate hiring decisions compared to items from nor-mative measures. Researchers have also developed indirect measures ofpersonality (James et al., 2005); however, the development of theinstruments appears to be onerous and, after years of development, onlyone or two constructs have been developed. Research fairly conclusively

has shown that social desirability measures that are designed to catchfakers do not act as predictors, moderators, or mediators of various jobperformance outcomes (Ones & Viswesvaran, 1998; Ones, Viswesvaran,& Reiss, 1996). Researchers have also attempted to use item responsetheory techniques called appropriateness measurement to identify fakers(cf. Zickar & Drasgow, 1996); the results of this line of research havemostly been disappointing. Similarly, use of response latencies to identifyor ameliorate the effects of faking has not met with promising results(Holden, Wood, & Tomashewski, 2001). One experimental study foundthat a warning which identified that faking could be identified andthe potential consequences of faking impacted responding (Dwight &

1 A recent study by Heggestad, Morrison, Reeve, and McCloy (2006) calls into questionthe viability of using forced-choice measures to control faking.

C. ROBIE, D. J. BROWN AND J. C. BEATY 491


4/21

Donovan, 2003). Moreover, McFarland (2003) found that warnings do notnegatively affect test-taker reactions.

All of the above research rests on the premise that at least someindividuals do actually fake on personality inventories. However, noresearch to date to our knowledge has addressed the basic question ofwhether individuals fake on personality inventories. In the currentstudy, we used a methodology called verbal protocol analysis to providesome preliminary insights into whether and the extent to which indi-viduals fake on personality inventories.

Verbal Protocol Analysis

Verbal protocol analysis (VPA) is a method used to collect data on howpeople make decisions from a process perspective (Carroll & Johnson,1990). When utilizing VPA, experimenters request participants to saywhatever comes to mind while performing a task of interest. Concurrentreports are most often utilized because memory and other cognitive biasescan distort retrospective reports (Nisbett & Wilson, 1977). It is importantnot to cue the participant (i.e., decision maker) by asking for certain typesof information, particularly if that information would not normally be

salient while doing the task (Ericsson & Simon, 1984).VPA has been used for decades in the decision sciences; its use is

more recent in the organizational sciences. In one of the earlier organi-zational applications of VPA, Isenberg (1986) used VPA to examinemanagerial problem solving. Martin and Klimoski (1990) used VPA totrace cognitions associated with self- and supervisor evaluations of per-formance. Barber and Roehling (1993) used VPA to investigate the pro-cess of deciding whether or not to apply for jobs. Cable and Graham(2000) used VPA to examine the determinants of job seekers reputation.VPA has thus been used to examine a wide variety of organizational

issues. VPA has not (to our knowledge) been used to examine fakingissues. We will now present the hypotheses of the current study.

Study Hypotheses

As noted above, one of the basic premises upon which much of thefaking literature is based is that at least some individuals do fake onpersonality inventories. Therefore, our first hypothesis is:

H1: One or more individuals will fake on the personality inventory.

Few studies have speculated on how faking may be distributedwithin a given population. A multi-sample study by Zickar, Gibby, andRobie (2004) using mixed-model item response theory techniques foundthat individuals could be classed into three faking classes: regular



5/21

responding (or honest) class; a slight faking (or mixed) class; and anextreme faking class.

H2: Individuals will be able to be classed into honest responders,slight fakers, and extreme fakers.

A study on faking in the individual assessment process suggests thatindividuals who fake on personality inventories compared to those whodo not fake may: (a) make more corrections to their personality itemresponses (i.e., change their responses); and (b) take longer to completethe inventory. Specifically, in a study of individual assessors who regu-larly administer personality inventories for selection purposes (Robieet al., 2005), one assessor answered the open-ended question, Pleaseshare additional possible reasons for faking with the following answer:

I had a participant who completed the personality inventory andworked it over for two more times before he submitted the ques-tionnaire.

This comment suggests that participants who fake both change re-

sponses and take longer than those who do not fake. Thus:

H3: Individuals classed as honest responders will make fewer cor-rections to their responses on the personality inventory than individualsclassed as fakers.

H4: Individuals classed as honest responders will take less time tofinish the personality inventory than individuals classed as fakers.

METHOD

Participants

Twelve individuals not currently enrolled in university courses (i.e.,a community sample) participated in the study. The majority of partici-pants (seven) were female and all participants were Caucasian. Fourparticipants were between 36 and 40 years of age; five participants werebetween 41 and 45 years of age; and three participants were between 51

and 55 years of age. One participant possessed a high school degree orequivalent; four participants possessed some college or university; fiveparticipants possessed undergraduate degrees; and two participants



6/21

possessed post-graduate degrees (M.S., M.A., or M.B.A.). Three partici-pants were employed full-time; six participants were employed part-time;and three participants were unemployed and searching for work. Oneparticipant possessed 57 years of full-time work experience; oneparticipant possessed 1113 years of full-time work experience;three participants possessed 1719 years of full-time work experience;and seven participants possessed 20+ years of full-time work experience.The majority of participants (seven) reported holding a part- or full-timeretail sales position in the past. No participants currently held a part- orfull-time retail sales position. The employed participants held a variety ofjobs including: Administrative Assistant; Bus Driver; High SchoolGuidance Counselor; Laborer; Machine Operator; Manager; Packager;Paper Route Deliverer; and Technical Writer. Table 1 lists the demo-graphic data for each participant.

Procedure

Participants were recruited for the current study via a local dailynewspaper advertisement placed in the employment section on a con-secutive Wednesday and Saturday. The advertisement read:

Wanted

Twelve research participants

On a study examining personality at Wilfrid Laurier University.Three top-scoring participants on the personality inventory will eachreceive $100. Participants must be between 25 and 55 years of age,

possess at least two years of full-time work experience in anyindustry, and be willing and able to travel to Wilfrid Laurier Uni-versitys Waterloo campus for testing. Compensation for travel tocampus will be $10. Participants will be accommodated on a first-come, first-served basis.

Please contact Dr. Chet Robie at [email protected] (519) 884-0710ext.2965 to set up a one-hour appointment if you are interested inparticipating.

All participants were individually administered all materials by

Chet Robie. Participants were first asked to read and sign a consentform detailing the tasks that they would engage in and the respectiverights and responsibilities of the participant and researcher duringand after the appointment time. Participants were told on the consent



7/21

Ta

ble1

Da

taDis

pla

yo

fAllR

ele

va

nt

Stu

dy

Var

iable

sw

ithP

art

icip

an

tsR

an

k-O

rd

ere

db

yP

er

sona

lity

Score

GenderAge

Education

Employment

Status

Work

Ex

perience

(years)

Past

Retail

Experience?

CurrentJob

Personality

Score

Memory

Score(%)

#ofCor

rections

to

Inventory

Time

(min)to

Complete

Inventory

Fakeor

Honest?

Female4145University

Part-time

20

+

No

Highschool

guidance

counselor

214

27.27

6

28.75

Extreme

faker

Female3640Someuniversity

Part-time

17

19

No

Busdriver

212

22.73

0

14.42

Honest

Female3640Post-graduatePart-time

5

7

Yes

Technical

writer

199

63.64

1

0

41.00

Slightfaker

Male

4145Post-graduateUnemployed17

19

Yes

Unemployed

196

34.09

0

16.50

Slightfaker


Part-time

20

+

Yes

Paperroute

delivery

195

56.82

0

12.92

Honest

Male

4145Someuniversity

Full-time

20

+

No

Machine

operator

194

45.45

1

12.08

Honest

Male

3640University

Unemployed17

19

Yes

Unemployed

192

38.64

0

10.33

Honest


Part-time

20

+

Yes

Administrative

assistant

188

84.09

0

13.33

Honest


Unemployed20

+

Yes

Unemployed

188

90.91

1

16.67

Honest

Male

5155University

Part-time

20

+

Yes

Laborer

183

56.82

0

12.08

Honest


Full-time

20

+

No

Manager

168

68.18

0

16.92

Honest

Male

3640Highschool

Full-time

11

13

No

Packager

157

34.09

0

10.25

Honest

Note.Allparticipants:wereCaucasian;

werenotcurrentlyenrolledinun

iversity;anddidnotcurrentlyholdretailsalesjobs.



8/21

form that the research project examined how people go aboutresponding to personality inventories in work-type situations. The truepurpose of the study was not divulged to participants until a post-experimental debriefing was administered. Participants were then gi-ven the following directions (based on typical verbal protocol analysistechnique; cf. Ericsson & Simon, 1984):

As soon as you begin working on completing the personality inven-tory please start thinking aloud. The best way to do this is to be asspontaneous as possible. Tell me everything you are thinking as youare thinking it, even details or sidetracks that seem insignificant orembarrassing. If you think aloud spontaneously, you will soon forgetthat you are speaking at all. There is no need to explain to me whyyou are thinking what you are. You dont have to interpret or justifyyour approach to an item. Just tell me what you are thinking at themoment. If you are silent for more than a few seconds, I will remindyou by saying: Please tell me what you are thinking or Keeptalking.

While taking the inventory, keep in mind that how you score on thequestionnaire will determine whether you will be considered for oneof the three $100 prizes. Nothing that you say out loud will have anyimpact on whether you will be considered for one of the prizes. Thethree $100 prizes will go to the individuals whose personality testscores most closely match the requirements of the job detailed in thefollowing job advertisement:

Participants were then shown a job advertisement for the targetretail sales positions (see Appendix). The job advertisement was an ac-

tual job advertisement from the Human Resources Development CanadaJob Bank web page (http://jb-ge.hrdc-drhc.gc.ca/) with identifying infor-mation on the employer and address made fictitious. Participants wereinstructed to take as long they liked to read the job advertisement butthat they could not refer back to the advertisement once they beganfilling out the personality inventory.

Participants then filled out a paper-and-pencil personality inventorywhile thinking aloud. Their verbalizations were audio taped. After par-ticipants completed the personality inventory, they were asked to (in thefollowing order): complete a brief post-experimental manipulation check(see Measures); complete a demographic information sheet; read a post-experimental debriefing form that explained the true purpose of thestudy; read and sign a release form (with an option for opting out) for the



9/21

use of their quotations in subsequent reports; and complete several formsto ensure that they would be able to be contacted in the event that theywon one of the $100 prizes and to release the $10 travel compensation tothem.

Nine participants provided email addresses. One week after partic-ipation, six participants who responded honestly (see Determination ofResponse Set below) and provided email addresses were sent thefollowing brief post-experimental email-based survey:

Dear Participants:

If you feel comfortable with sending me a brief email describingpossible reasons as to why you decided NOT to match yourpersonality responses to the hypothetical job description (i.e., fakegood) on the personality inventory, I would very much appre-ciate it!

Sincerely,

Chet Robie, Ph.D.

For the three participants that faked to some extent (see Determi-nation of Response Set below) and provided email addresses, the emailwas changed to the following:

Dear Participants:

If you feel comfortable with sending me a brief email describingpossible reasons as to why you decided to match your personalityresponses to the hypothetical job description (i.e., fake good) on thepersonality inventory, I would very much appreciate it!

Sincerely,

Chet Robie, Ph.D.



10/21

Measures

Personality inventoryThe personality inventory used in this study was the Criterion

International Service InventoryShort Form (CISI-S) (ePredix, 2004).The CISI-S is a brief, standardized, self-report assessment designed to beused along with other job-related information to identify candidates withhigh potential for success in entry to mid-level positions. Three charac-teristics of the inventory demonstrate that it was developed specificallyfor use in a work context. First, all items reflect a work environment

frame of reference. Second, the span of contact in the items includes onesself, coworkers, supervisors, and customers. Finally, the items includework relevant situational demands such as dealing with an angry cus-tomer, checking work for errors, or getting to work on time. Test-takersrespond to statements by choosing one of four response categories labeledas definitely no, probably no, probably yes, and definitely yes.

Meta-analytic research has shown that effective customer serviceinstruments tend to measure three broad personality characteristics:Conscientiousness, Emotional Stability, and Agreeableness (Frei &McDaniel, 1998). The CISI-S was designed to measure two of those three

broad personality characteristics: Conscientiousness (31 items) andAgreeableness (26 items). Both factors were drawn from a previouslydeveloped and larger personality inventory based on the Five-Factormodel of personalitythe Criterion Occupational Behavior Inventory(COBI). Conscientiousness in the CISI-S inventory is defined as beingrelated to the likelihood that individuals will exert effort to completetasks ahead of schedule, tend to be well-organized and pay attention todetail in tasks, get to work on-time and attend work regularly, have astrong desire to achieve their goals, and tend to be dedicated employeesthat are committed to the success of the company. An example item fromthe Conscientiousness scale is: Sometimes when it is especially nice

outdoors, I will excuse myself from work. The internal consistencycoefficient for the Conscientiousness scale is .84. Agreeableness in theCISI-S inventory is defined as being related to the likelihood that indi-viduals will demonstrate enthusiasm in customer interactions, success-fully calm irate customers, react with sensitivity to others feelings,continue to work hard during busy or frustrating circumstances, andmaking time to find product information for customers. An example itemfrom the Agreeableness scale was: At work, I tend to offer my assistanceto others. The internal consistency coefficient for the Agreeablenessscale is .78. Because the two factors were found to be fairly highly cor-

related in a large customer service applicant sample (r = .66, N = 49,100)(Shepherd, Brown, & Robie, 2005), we elected to simply combine the twofactors for scoring purposes and select top-down when drawing for prizes.



11/21

The highest possible score on the personality inventory was then 57 (sumof 31 items from Conscientiousness scale + 26 items from the Agree-ableness scale) 4 (maximum points available per item) = 228.

Post-experimental manipulation checkParticipants were asked to complete a partially blank job adver-

tisement form after completing the personality inventory. The jobadvertisement form was similar in all respects to the job advertisementform that they were presented with prior to their completing with thepersonality inventory with the exception that the skill requirementsections had been removed and only the headings remained: Education;Credentials; Languages; Work Setting; Product Setting/Knowledge;Specific Skills; Security and Safety; and Essential Skills. Participantswere asked to remember as much as they possibly could from the firsttime they were presented with the information and write the informationin the appropriate fields. The task was scored by weighting a field by itsnumber of words. In this manner, the total score for the task was 44 (44words in all the fields). For example, under Specific Skills which con-tained 14 words, if a participant only remembered Work with MinimalSupervision, they would receive a score of 4 for that field.

Numbers of corrections to personality inventoryThe numbers of times that participants made corrections or changes

to personality responses on the personality inventory were recorded.Multiple corrections per item could be counted although this did notoccur.

Time to complete personality inventory

The time to complete inventory was recorded in minutes. Conversionto tape counter to time was: (one side of tape = 360 tape count = 30 min).

Thus, the tape counter was divided by 360 and multiplied by 30 to derivethe time in minutes to complete the personality inventory.

Determination of response set

Participants were classified as responding in one of the followingresponse sets: (1) honest, (2) slight fakers, or (3) extreme fakers.

We coded participants as honest based on the principle of traitactivationthat ...behavioral expression of a trait requires arousal ofthat trait by trait-relevant situational cues (p. 398) (Tett & Guterman,2000). To be classified in the honest response set, a participant mustclearly be responding to all of the items on the personality inventory fromthe standpoint of situations that appear to be eliciting trait-basedinformation from that individual. Several excerpts from actual audiotapes illustrate the coding principles:



12/21

...When the need comes, I will offer to work extra hours or weekendsor whatever if I feel it is really important.

...[reads item]...Ya, I think I do that.

Yes, I do complete jobs within a reasonable amount of time.

[reads item]...definitely no...because of my attachment to Christ.

To be classified in the extreme fakers response set, a participantmust clearly be responding to all of the items on the personality inven-tory from the perspective of a hypothetical ideal applicant (i.e., thepersonality inventory is not capturing any trait-based information). Inother words, it did not appear that the individual was using situationsfrom his or her experience to answer the personality items which thenwere activating trait-based information. Several excerpts from an actualaudio tape illustrate the coding principles:

I wonder what type of responses they want?

If I answer definitely yes to all of these, theyll think Im lying.

[reads item]...yes, that would be a plus for the job.

[reads item]...I dont remember reading anything about getting acommission...however, if I say definitely yes, they are going to be

thinking that I am going to be cutthroat...um, ya, Ill say yes, defi-nitely.

To be classified in the slight fakers response set, a participant mustappear to be using themselves as the referent in their initial verbaliza-tions for each item, but switch to how the hypothetical ideal applicantshould respond. Several excerpts from actual audio tapes illustrate thecoding principles:

[reads item]...Generally yes, so Ill say probably yes because its trueand it looks good on the sheet.



13/21

[reads item]...Oh, thats not me...ya, I get distracted, but you wouldhave to go with probablydont say definitely on any of the-se...ok...[reads same again]...probably no, because that means that atleast that you are open to people coming up to you and talking to youin retail.

RESULTS

Table 1 provides a data display of all relevant variables with par-

ticipants rank-ordered by personality score. Personality scores rangedfrom 157 to 214 and averaged 190.50 (SD = 16.09); this compares to arange of 66 to 228 and an average of 209.24 (SD = 11.87) for a largecustomer service applicant sample (Shepherd et al., 2005). Scores on thepost-experimental manipulation check (i.e., memory scores) ranged from22.73 to 90.91% accuracy and averaged 51.89% accuracy (SD = 21.99)and did not differ significantly by honest versus faking responders[t(10) = .92, p > .05]. The number of corrections to the personalityinventory ranged from 0 to 10 and averaged 1.50 (SD = 3.18). The time tocomplete the personality inventory ranged from 10.25 to 41.00 min and

averaged 17.10 min (SD = 8.99). One individual was classified asresponding in the extreme fakers response set; two individuals wereclassified as being in the slight fakers response set; and nine individ-uals were classified as responding in the honest response set. Giventhat at least one individual was classified as a faker on the personalityinventory and individuals were able to be classified into the three a priorispecified classes, support was found for the first two hypotheses.

The extreme fakers and slight fakers response set categorieswere collapsed into one category so that a response set category offakers could be compared to honest for testing the third and fourth

hypotheses. The discriminant function for the number of items correctedon the personality inventory was statistically significant (v2 = 7.16,p < .01), with fakers participants correcting 5.33 items on average andhonest participants correcting 0.22 items on average. The discriminantfunction for the time to complete the personality inventory was alsostatistically significant (v2 = 8.94, p < .01), with fakers participantstaking 28.75 min to complete the personality inventory and honestparticipants taking 13.22 min to complete the personality inventory.Thus, support was also found for the third and fourth study hypotheses.

Five of the nine participants who were classified as being in thehonest response set responded to the post-experimental email-based

survey question asking them why they did not fake on the personalityinventory. Following are excerpts from each responding participantsanswers:



14/21

It never entered my mind to not tell the truth. I answered thequestions as if you really wanted to know who I am. (bus driver)

What motivated me to be true and sincere can be summed up in oneword experience! I have worked with lots of achievers and otherso called ,hard workers and have realized that the truth alwayscomes out in the end. Up front, it may appear as if you have fooledsomeone, but as I always say, the worm always turns! If I have whatyou need, then Ill deliver, but I do not falsify just to get a job!(administrative assistant)

It never even entered my mind to fake my responses so as to en-hance my chance of winning the money. If it had occurred to me to doso, I most likely wouldnt have, for integritys sake. I already knowfrom experience that I am good with customers in a retail setting, soI was quite confident of being naturally suitable for the hypotheticalposition. Thus, even if I had been willing to fake my responses,fakery would not have seemed likely to produce any substantial

improvement. Finally, I am generally more concerned with processthan outcome, so the potential reward of $100.00 would not motivateme more than the intrinsic motivation of my own character just to betrue to myself. I am the sort of principled idiot who most likely wouldstand before a firing squad refusing to tell a white lie that would savemy life. Wanting to live would not be enough it matters too muchhow I live. Wanting the hundred dollars was not enough to entice meinto dishonesty. (laborer)

If the prize amount was greater, hypothetically a thousand dollars,then you would likely have seen changes in my results. This is be-cause a hundred dollars would make no appreciative difference inmy current situation, but a thousand dollars would help consider-ably, therefore I would be more motivated to conform to the per-sonality needed for the job. I suspect that if the prize amount wasgreatly increased there would be a lot more general competition.(unemployed female)

Not sure why I didnt lie to conform to the job description. IntegrityI would have to say...seems to trump exaggerations. Hindsight Ishould have lied...oh well integrity is of more value anyways.(packager)



15/21

Both of the participants who were classified as being in the slightfakers response set responded to the post-experimental email-basedsurvey question asking them why they did not fake on the personalityinventory. The participant who was classified as being the extremefakers response set did not respond to the post-experimental email-based survey question. Following are excerpts from each of theresponding participants answers:

I think the internal ambivalence and struggle I experiencedanswering the questions was in deciding whether it was the jobbeing applied for or the temporary monetary benefit that I wasgearing my answers towards. Initially, I approached the task withthe mindset that I wanted to best represent myself for the job. Ithink I felt obligated and it was easier to respond to a question ashonestly as possible, if it was a reflection of myself (maybe with justa bit of a stretch if it seemed a little more desirable one level up).(technical writer)

Though it was very clear, right upfront, that the ,winners would get

prizes of $100, I found that this information somewhat receded intothe background as I was continuing the questionnaire. Also, thedetails of the job also were rather lost on me after I got involved inthe questionnaire. I knew it was a retail position based on aWalmart job description, I believe but as I continued through thequestions, I actually took most of the answers based on my ownpersonal awareness. However, I was conscious about what employ-ers value in general, so I tended to focus on the positive aspects of mypersonality in relation to the job, and tempered my personality as-pects that were not in keeping with my understanding of employers

needs and wants. Now, a couple of caveats: Were this a real position,or if the ,prize money were considerably more, then my answers mayhave been different. As well, I would have likely taken much moretime in going through the questions, probably analyzing them muchmore (potentially too much!) than I did. (unemployed male)

DISCUSSION

This is the first study to provide direct evidence that individuals fakeon personality inventories. Prior studies inferred the existence of fakingthrough higher mean personality inventory scale scores betweenincumbent and applicant populations (Hough, 1998) or applicants



16/21

endorsement of nonexistent job analytic task statements (i.e., sayingthey have done something on a job when could not have possibly done it)(Anderson, Warner, & Spencer, 1984; Pannone, 1984).

Past research has found that individuals vary in their ability to fakeon personality inventories (McFarland & Ryan, 2000; Mersman &Shultz, 1998). The present research suggests that individuals also varyin their motivation to fake and that individuals may be categorized basedupon their faking motivation into one of three faking classes: honest;slight fakers; and extreme fakers. This more direct evidence of class-based faking motivation is consistent with less direct evidence providedby Zickar et al. (2004).

We also found that individuals classed as honest responders madefewer corrections and took less time completing their personalityinventories than individuals classed as faking responders. The results ofthe current study support McDaniel and Timms (1990) notion that lyingtakes time; however, other studies have found faster latencies associ-ated with faking (George, 1990; Hsu, Santelli, & Hsu, 1989). Further-more, Holden et al. (2001) has found that limiting response time does notaffect personality inventory validity. The discrepancies between thecurrent study and previous studies in examining completion time can

probably be accounted for the fact that previous studies used the item asthe level of analysis and the current study used the scale as the level ofanalysis.

An interesting finding from this study is the large number ofrespondents that were classed as honest and the concomitant low aver-age personality scale score in comparison to a large applicant sample ofcustomer service representatives (Robie et al., 2005). In fact, the differ-ence between the average personality scale scores in the current studyand the applicant sample in Robie et al. (2005) is over one standarddeviation. One could infer that the induced motivation in the present

study is not as strong as an employment context in which an individual isseeking a job. Several post-experimental respondent comments suggestthis to be the case (see Results section).

Study Implications

Several implications from the current study can be drawn. Onemajor implication is that, given that direct evidence of faking has beenfound in a fairly low stakes testing context, it is likely that more faking isoccurring in applicant contexts. Indirect evidence of such likely faking isprovided by the large mean differences in personality scale scoresbetween the experimental sample in the present study and the applicantsample in Robie et al. (2005).



17/21

Another major implication from the current study is the issue ofrank-order change. As can be seen in Table 1, the rank-order ofrespondents would likely change quite drastically if the individuals whofaked chose to respond honestly. As has been suggested by others but notdemonstrated directly, selection ratio would also be a determining factorin the degree to which faking impacts validity (Christiansen et al., 1994;Mueller-Hanson et al., 2003; Rosse et al., 1998). At a selection ratio of.25, one would hire two fakers and one honest responder. At a selectionratio of .50, one would hire three fakers and three honest responders. Ata selection ratio of .75, one would hire three fakers and six honestresponders.

Limitations

This study has several limitations. One limitation is the small size ofthe sample which may hinder arguments for generalization. VPA studiesgenerally utilize small samples (1015 participants) because of the timeintensiveness and costliness of the methodology. Future replications willsurely be needed using multiple methods to ensure generalization. An-other limitation of our study was also its strength; namely, that is was

experimental in nature. It is very difficult to model the motivationalcharacteristics of an applicant context. Respondent post-experimentalcomments suggest that with an adult community sample perhapsincreasing the motivational press over and above what was introduced inthe current study would be warranted.

Another limitation to the study was the Time to Complete Inventoryvariable. As one reviewer of this study noted, some individuals aresimply more talkative than others. We did not note any individuals thatwere off task to any major extent and talking about irrelevant material;that is, all participants were engaged in the task at hand and it appeared

to us that the individuals who were attempting to fake the measuresimply took longer to verbally work through the process.

Future Research

One of the most pressing avenues for future research is to examinethe extent to which honest responders, slight fakers, and extreme fakersproduce differential job outcomes and whether these outcomes may differby job context. Hogan (1991) has argued that faking does not really existand is instead an applicants attempt to manage how the applicant wouldlike for him or her to be seen which is likely how he or she will perform onthe job. On the other hand, some research suggests that faking behaviormay actually be positive for some occupations such as sales personnel(Tull, 1998).



18/21

One critical limitation of VPA, that individuals are many timesself-deceived about their own decision-making processes (cf. Nisbett &Wilson, 1977), suggests a future study to rule out alternative explana-tions for the results in the present study. For example, did the presentstudy identify more honest individuals than one would expect simplybecause an experimenter was monitoring the participants responses (cf.Asch, 1961) or is this a true state of affairs? A follow-up study can bedesigned that includes three matched groups: (a) a group that isadministered the VPA via a live experimenter; (b) a group that isadministered the VPA via a automated system; and (c) a group that isadministered a paper-and-pencil version of the measure. Issues ofconformity to experimenter expectations can then be addressed.

CONCLUSION

The current study is the first study to find direct evidence of fakingon personality inventories. People do fake on personality inventories;people can be categorized into one of three classes of responders; andhonest responders take less time and make fewer corrections to their

personality inventories than do faking responders.

APPENDIX

Job AdvertisementAdvertisement number: 1360658Title: Sales associateretailTerms of Employment: Permanent, Part Time, Weekend

Salary: To be negotiatedAnticipated Start Date: As soon as possibleLocation: xxxxxxxx, xxxxxxx (3 vacancies)Skill Requirements:

Education: Completion of high schoolCredentials (certificates, licenses, memberships, courses,etc.): Not required

Experience: Experience an assetLanguages: Speak English, Read English, Write EnglishWork Setting: Department storeProduct Experience/Knowledge: not applicable



19/21

Specific Skills: Operate cash register, Customer service oriented,Provide advice about merchandise, Work with minimal supervi-sion

Security and Safety: Criminal record checkEssential Skills: Writing, Oral communication, Working withothers, Problem solving

Employer: WalmaxPlease apply for this job only in the manner specified by the em-

ployer. Failure to do so may result in your application not being properlyconsidered for the position.

How to Apply:In Person between 9:00 and 16:30:1105 Felsure Blvd.xxxxxxxx, xxxxxxxxx xxxBy Phone between 9:00 and 16:30:(519) 555-7500

Advertised until: 2005/06/03

REFERENCES

Anderson, C. D., Warner, J. L.,, & Spencer, C. C. (1984). Inflation bias in self-assessmentexaminations: Implications for valid employee selection. Journal of Applied Psychology,69, 574580.

Asch, S. (1961). Issues in the study of social influences on judgment. In I. A. Berg & B. M.Bass (Eds.), Conformity and deviation (pp. 143153). Oxford, England: Harper.

Barber, A. E.,, & Roehling, M. V. (1993). Job postings and the decision to interview: A verbalprotocol analysis. Journal of Applied Psychology, 78, 845856.

Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at thebeginning of the new millennium: What do we know and where do we go next?.

International Journal of Selection and Assessment, 9, 930.

Bass, B. M. (1957). Faking by sales applicants of a forced choice personality inventory.Journal of Applied Psychology, 41, 403404.Becker, T. E.,, & Colquitt, A. L. (1992). Potential versus actual faking of a biodata form: An

analysis along several dimensions of item type. Personnel Psychology, 45, 389406.Bowen, C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative

approaches for ability to control faking in personality questionnaires. InternationalJournal of Organizational Analysis, 10, 240259.

Cable, D. M.,, & Graham, M. E. (2000). The determinants of job seekers reputation per-ceptions. Journal of Organizational Behavior, 21, 929947.

Carroll, J. S.,, & Johnson, E. J. (1990). Decision research: A field guide. Newbury Park, CA:Sage Publications.

Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choiceitem formats for applicant personality assessment. Human Performance, 18, 267307.

Christiansen, N. D., Goffin, R. D., Johnston, N. G., & Rothstein, M. G. (1994). Correcting the

16PF for faking: Effects on criterion-related validity and individual hiring decisions.Personnel Psychology, 47, 847860.Douglas, E. F., McDaniel, M. A., & Snell, A. F. (1996, August). The validity of non-cognitive

measures decays when applicants fake. Paper presented at the annual meeting of theAcademy of Management, Cincinnati, OH.



20/21

Dunnette, M. D., McCartney, J., Carlson, H. C., & Kirchner, W. K. (1962). A study of fakingbehavior on a forced-choice self-description checklist. Personnel Psychology, 15, 1324.

Dwight, S. A.,, & Donovan, J. J. (2003). Do warnings not to fake reduce faking?. HumanPerformance, 16, 123.

Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of socialdesirability on personality factor structure. Journal of Applied Psychology, 86, 122133.

ePredix. (2004). Criterion International Service Inventory (short form) technical report.Minneapolis, MN: Unpublished technical report.

Ericsson, K. A.,, & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cam-bridge, MA: MIT Press.

Frei, R. L.,, & McDaniel, M. A. (1998). Validity of customer service measures in personnelselection: A review of criterion and construct evidence. Human Performance, 11, 127.

George, M. S. (1990). The use of response latency to study accuracy and consistency in acomputerized lifestyle assessment. Unpublished doctoral dissertation, University ofToronto, Toronto, Ontario, Canada.

Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choiceassessments of personality for selection: Evaluating issues of normative assessmentand faking resistance. Journal of Applied Psychology, 91, 924.

Hogan, R. T. (1991). Personality and personality measurement. In M. D. Dunnette & L. M.Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 2, 2nd ed.,pp. 873919). Palo Alto, CA, US: Consulting Psychologists Press, Inc.

Holden, R. R., & Jackson, D. N. (1981). Subtlety, information, and faking effects in per-sonality assessment. Journal of Clinical Psychology, 37, 379386.

Holden, R. R., Wood, L. L., & Tomashewski, L. (2001). Do response time limitations coun-teract the effect of faking on personality inventory validity?. Journal of Personality andSocial Psychology, 81, 160169.

Hough, L. M. (1998). Effects of intentional distortion in personality measurement andevaluation of suggested palliatives. Human Performance, 11, 209244.

Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on thosevalidities. Journal of Applied Psychology, 75, 581595.

Hsu, L. M., Santelli, J., & Hsu, J. R. (1989). Faking detection validity and incrementalvalidity of response validity of response latencies to MMPI subtle and obvious items.

Journal of Personality Assessment, 53, 278295.Isenberg, D. J. (1986). Thinking and managing: A verbal protocol analysis of managerial

problem solving. Academy of Management Journal, 29, 775788.Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on

employment tests: Does forced choice offer a solution?. Human Performance, 13, 371388.

James, L. W., McIntyre, M. D., Glisson, C. A., Green, P. D., Patton, T. W., LeBreton, J. M.,Frost, B. C., Russell, S. M., Sablynski, C. J., Mitchell, T. R., & Williams, L. J. (2005). Aconditional reasoning measure for aggression. Organizational Research Methods, 8,6999.

Martin, S. L., & Klimoski, R. J. (1990). Use of verbal protocols to trace cognitions associatedwith self- and supervisor evaluations of performance. Organizational Behavior and

Human Decision Processes, 46, 135154.McDaniel, M. A., & Timm, H. W. (August, 1990). Lying takes time: Predicting deception in

biodata using response latency. Paper presented at the 98th Annual Convention of theAmerican Psychological Association, Boston, MA.

McFarland, L. A. (2003). Warning against faking on a personality test: Effects on applicantreactions and personality test scores. International Journal of Selection and Assess-ment, 11, 265276.

McFarland, L. A., & Ryan, A. M. (2000). Variance in faking across noncognitive measures.

Journal of Applied Psychology, 85, 812821.Mersman, J. L., & Shultz, K. S. (1998). Individual differences in the ability to fake on

personality measures. Personality and Individual Differences, 24, 217227.



21/21

Mueller-Hanson, R., Heggestad, E. D., & Thornton, G. C. III. (2003). Faking and selection:Considering the use of personality from select-in and select-out perspectives. Journal of

Applied Psychology, 88, 348355.Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on

mental processes. Psychological Review, 84, 231259.Ones, D. S., & Viswesvaran, C. (1998). The effects of social desirability and faking on

personality and integrity assessment for personnel selection. Human Performance, 11,245269.

Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personalitytesting for personnel selection: The red herring. Journal of Applied Psychology, 81,660679.

Pannone, R. D. (1984). Predicting test performance: A content valid approach to screeningapplicants. Personnel Psychology, 37, 507514.

Robie, C., Tuzinski, K. A., & Bly, P. R. (2005). Faking in the individual assessment process.Minneapolis, MN: Personnel Decisions International.

Rosse, J. G., Stecher, M. D., Miller, J. L., & Levin, R. A. (1998). The impact of responsedistortion on preemployment personality testing and hiring decisions. Journal of Ap-

plied Psychology, 83, 634644.Ryan, A. M., McFarland, L., Baron, H., & Page, R. (1999). An international look at selection

practices: Nation and culture as explanations for variability in practice. PersonnelPsychology, 52, 359391.

Shepherd, W. J., Brown, D. J., & Robie, C. (2005, April). Equivalence of tests administeredon computer versus interactive voice response. In F. Drasgow (Chair), Innovations incomputerized assessment: Research on practical issues. Symposium conducted at theannual meeting of the Society for Industrial and Organizational Psychology, Los

Angeles, CA.Smith, D. B., & Ellingson, J. E. (2002). Substance versus style: A new look at social

desirability in motivating contexts. Journal of Applied Psychology, 87, 211219.Stark, S., Chernyshenko, O. S., Chan, K., Lee, W. C., & Drasgow, F. (2001). Effects of the

testing situation on item responding: Cause for concern. Journal of Applied Psychology,86, 943953.

Tett, R. P., & Guterman, H. A. (2000). Situation trait relevance, trait expression, and cross-situational consistency: Testing a principle of trait activation. Journal of Research in

Personality, 34, 397423.Topping, G. D., & OGorman, J. G. (1997). Effects of faking set on validity of the NEO-FFI.

Personality and Individual Differences, 23, 117124.Tull, K. T. (1998). The effects of faking behavior on the prediction of sales performance using

the Guilford Zimmerman temperament survey and the NEO Five Factor Inventory.Unpublished doctoral dissertation, University of Akron, Akron, OH.

Viswesvaran, C., & Ones, D. S. (1999). Meta-analyses of fakability estimates: Implications

for personality measurement. Educational and Psychological Measurement, 59,197210.Worthington, D. L., & Schlottmann, R. S. (1986). The predictive validity of subtle and

obvious empirically derived psychological test items under faking conditions. Journal ofPersonality Assessment, 50, 171181.

Zickar, M. J., & Drasgow, F. (1996). Detecting faking on a personality instrument usingappropriateness measurement. Applied Psychological Measurement, 20, 7187.

Zickar, M. J., Gibby, R. E., & Robie, C. (2004). Uncovering faking samples in applicant,incumbent, and experimental data sets: An application of mixed-model item responsetheory. Organizational Research Methods, 7, 168190.

Zickar, M. J., & Robie, C. (1999). Modeling faking good on personality items: An item-levelanalysis. Journal of Applied Psychology, 84, 551563.


do people fake on personality inventories

Documents