effects of performance context on processing speed and performance ratings

Journal of Business and Psychology, Vol. 18, No. 4, Summer 2004 ( 2004)

EFFECTS OF PERFORMANCE CONTEXTON PROCESSING SPEED AND

PERFORMANCE RATINGS

Tiffany JenningsWidener University

Jerry K. PalmerEastern Kentucky University

Adrain ThomasAuburn University

ABSTRACT: The present study demonstrated that the presence of evaluativelypolarized context performances not only produces contrast and halo effects onjudgments of a target performance, but also causes judgments to be made muchfaster. Processing speed and positive halo were highly correlated, supporting thenotion that halo in performance ratings results from raters’ recall and use of asingle, general impression. Furthermore, regression analyses demonstrated thatprocessing speed mediates the relationship between context and halo. The rela-tionship between these findings, halo, processing speed, and general impres-sions, as well as implications for performance appraisals, are discussed.

KEY WORDS: context performance; contrast effect; halo; general impression;processing speed.

Contrast effects in performance appraisals occur when ratings of atarget performance are displaced away from ratings of contextual perfor-mances (Kravtiz & Balzer, 1992). For example, an average (target) per-formance may be rated as good when viewed and judged in the presenceof poor performances, or rated as poor when viewed and judged in thepresence of good performances. The typical laboratory study on contrast

Address correspondence to Jerry K. Palmer, Eastern Kentucky University, 127 Cam-mack Building, 521 Lancaster Avenue, Richmond, KY 40475-3102. E-mail: [email protected].

453

0889-3268/04/0600-0453/0 2004 Human Sciences Press, Inc.

454 JOURNAL OF BUSINESS AND PSYCHOLOGY

effects in performance appraisal judgments uses videotaped lecturers asperformance stimuli (ratees). Participants watch and rate either twoevaluatively good lecturers or two evaulatively poor lecturers, followedby a lecturer whose performance is considered average and whose spe-cific behavior consists of both good and poor behavioral instances. Rat-ings of the average, target lecturer are displaced away from the evalua-tive level of the context lecturers when compared to ratings of theaverage, target lecturer performed in absence of any experimentally pro-vided context (control). Thus, the typical study utilizes three conditionsfor the context factor: good context, poor context, and a context-free con-trol.

In studies using this design one explanation of contrast effects isthat raters pay more attention to the context-discrepant behavior of thetarget ratee (Maurer & Alexander, 1991). In other words, raters attendto poor information in a good context and good information in a poorcontext, and give less or no attention to the context-consistent behaviorexhibited by the target lecturer. When assigning ratings, raters attend-ing primarily to the poor (good) behaviors will recall poor (good) behav-iors and thus give evaluatively lower (higher) ratings when compared toratings of the target lecturer from the context-free control condition.Thus, context-induced attention to context-discrepant behavior drivesthe contrast effect.

HALO

The most easily remembered information about a person or perfor-mance is the general impression (Feldman, 1981; Srull & Wyer, 1989).The general impression is simply an evaluative summary of the individ-ual or the performance, such as good/poor or like/dislike. In other words,when individuals are asked what they think of a person or performance,the first thing that comes to mind is usually whether they like or dislikethe person or performance. With some effort, traits and behaviors maythen be recalled (Srull & Wyer, 1989). For example, after having recalleda general impression, the person might recall traits relevant to that im-pression, such as honest, caring, and dependable. With additional effort,the third type of information typically recalled is specific instances ofbehaviors (Srull & Wyer, 1989). After recalling the general impressionand more specific traits, the person might then recall behaviors relevantto specific traits, such as specific behavioral examples of honesty, caring,and dependableness.

In addition to being the most accessible of the performance/personrelated cognitions, the general impression is also sufficiently diagnostic(Feldman & Lynch, 1988) for almost all specific (trait) judgments. In

455T. JENNINGS, J. K. PALMER, AND A. THOMAS

other words, a rater need only recall whether an employee is good orpoor to make ratings on a variety of different performance dimensions.Additionally, raters’ use of a global, general impression when makingspecific ratings has received strong support as one potential explanationfor positive halo (Lance, LaPointe, & Fisicaro, 1994; Lance, LaPointe, &Stewart, 1994). Positive halo in performance ratings exists when (1) spe-cific performance ratings are correlated more highly than their theoreti-cal true correlation levels or (2) when the variability of the ratings is lessthan the theoretical true variability (Balzer & Sulsky, 1992). In short,positive halo exists when a rater does not differentiate properly betweenmultiple rating dimensions.

Palmer, Feldman, and Maurer (1998) demonstrated that a system-atic relationship between postive halo and contrast exists. They foundhalo to be more pervasive with increasingly contrasted ratings. They the-orized that because a strong performance context influences raters toattend to a context-discrepant subset of ratee behavior, it should alsoresult in a strong, highly accessible, and easily recalled general impres-sion based on a homogenous (good or poor) set of ratee behavior in-stances. In their control condition the target lecturer was rated in ab-sence of an experimentally provided context and received averageratings that varied considerably from dimension to dimension. Whenparticipants viewed the average performance in the presence of good orpoor context performances, they gave almost all good or all poor ratings(high positive halo) across the dimensions of performance. Furthermore,they demonstrated that the stronger halo was not simply a result of rat-ing range restriction due to the ratings being shifted toward the end-points of the scale. Feldman and Lynch (1988) theorized that individualsrespond to survey questions by recalling the most accessible of the mini-mally diagnostic judgments applicable to the questions. For person-im-pressions and performance judgments, the general impression, beingmost accessible, is most easily recalled.

PROCESSING SPEED

When cognitions and attitudes are highly accessible and easily re-called, judgments are rendered more quickly from memory (Fazio, Chen,McDonel, & Sherman, 1982). Fazio (et al., 1982) demonstrated that par-ticipants respond faster to attitudinal questions when the attitude isbased on direct experience. For example, when individuals are asked torender their attitudes on nuclear power, most will have a strong attitudeabout the issue. However, those individuals who live in close proximityto a nuclear power plant or who have direct experience with nuclear


power will respond more quickly to the questions, even though the nu-merical answers to a given question may be identical.

It should then be the case that when a rater uses a general impres-sion of a ratee, or when the general impression is more easily recalled,reaction times for those judgments should be faster than when the raterdoes not use the general impression. When a rater does not use a stronggeneral impression to make judgments and instead recalls more specifictraits and behaviors from memory, reaction times should be slower. If itis the case that halo results from the recall of a single general impres-sion, participants who commit high halo should return ratings withfaster reaction times than participants who do not commit high halo. Inaddition to supporting the theory that halo stems from the use of a singlegeneral impression, a demonstration of this would indicate that it is pos-sible to identify those individuals likely to commit halo.

Given the previous discussion, the current study was undertaken toaddress the following hypotheses:

1. Participants who make faster performance judgments will showgreater levels of halo.

2. Participants who view a target lecturer in the presence of astrong, polarized performance context will make faster judgmentsof that target lecturer.

3. Speed of processing will mediate the relationship between contextand halo.

METHOD

Participants

Ninety undergraduate psychology students participated, receivingclass credit. Sixty-eight were female and the average age was 20.34.

Design, Stimuli, and Measures

The present study consisted of three conditions, two context condi-tions (good and poor) and a context free (random) control condition, eachwith 30 participants. In the poor context condition, participants watchedtwo short videotaped evaluatively poor lecturers followed by an averagelecturer. Participants in the good condition watched two good lecturersfollowed by the same average lecturer. In the control condition, partici-pants viewed only the average lecturer. A seven-point Likert-type ratingscale ranging from very poor (1) to very good (7) used extensively in pre-vious research (e.g., Maurer & Alexander, 1991; Murphy, Balzer, Lock-hart, & Eisenman, 1985) was utilized for lecturer evaluation. As in the


previous research, a total of nine demensions of lecturer performancewere measured. These dimensions were confidence with the material,organization of the lecture, preparation of the lecture, coherence of thelecture, speaking ability, nonverbal communication ability, rapport withthe audience, informational content of the lecture, and animation of thelecture.

Procedure

Initially, participants were given a demonstration of how to rate thelecturer’s performance via computer and practiced responding to thecomputerized questions prior to watching any lectures. The order of thenine performance dimensions to be rated, as presented via computer,was randomized. To minimize any differential practice effects betweencontext conditions and the control, participants in the context conditionspracticed only once; participants in the control condition practiced twice.

Following this demonstration, participants in the context conditionsviewed the first context performance tape, and rated the lecturer’s per-formance by using the computerized rating scale. This procedure wasrepeated for the second context lecturer and the control lecturer. Controlcondition participants viewed and rated only the target lecturer.

RESULTS

Reliabilities and Manipulation Checks

Manipulation checks were performed in order to demonstrate a suc-cessful manipulation of context. Average ratings of each of the four con-text lecturers (2 poor and 2 good) were compared with those of the targetlecturer from the control condition (Kravitz & Balzer, 1992). All four t-tests were significant (p < .01) demonstrating a successful manipulationof the context. Coefficient alphas for the two poor, two good, and averagelecturers were consistent with prior research in that all exceeded .90.

Ratings

A oneway ANOVA was conducted with context condition (poor, good,control) as the independent variable and the average of the nine per-formance dimension ratings for the target lecturer as the dependentvariable (α = .97). The ANOVA was significant, F(2, 87) = 53.9, p < .05.and rating means were in the expected direction (XPOOR = 5.07, SD = 1.35;XCONTROL = 3.04, SD = .99; XGOOD = 2.34, SD = .73). The rating means arepresented graphically in Figure 1. As demonstrated in the figure a con-trast in ratings clearly occurred, consistent with prior research.


Figure 1Rating Means

Halo

As suggested by Balzer and Sulsky (1992), halo was operationalizedas the average absolute deviation of the specific rating dimensions fromthe mean of the rating dimensions. The hypothesis held that halo wouldbe higher (i.e. less variance among the dimension ratings) in the contextconditions. If the conditions are ordered 1) poor, 2) control, and 3) good,this amounts to a test of a quadratic trend in the data, with weights of1, −2, and 1. A second orthogonal, planned comparison, pitting ratingsfrom a poor context against those from a good context used weights of 1,0, and −1. Thus, the second comparison tests the post-hoc hypothesisthat halo is higher in a poor context condition than in the good contextcondition (Palmer, Feldman, & Maurer, 1998).

The first planned comparison was significant, t(87) = 3.89, p < .05,and the halo means were in the expected direction; Halo was higher inboth context conditions (XPOOR = .49, SD = .21; XCONTROL = .81, SD = ..25;XGOOD = .67, SD = ..33). The second planned comparison on halo means


was also significant, t(87) = 2.68, p < .05. Halo in the poor context condi-tion was higher than that in the good context condition, consistent withthe argument that poor information has a greater effect on processingand judgments (Wexley, Yukl, Kovacs, & Sanders, 1972; Skowronski &Carlston, 1989). Halo means are presented in Figure 2.

Response Latencies

The correlation between halo and reaction time was .321 (p < .05),supporting the hypothesis that participants who maker faster judgmentsexhibit greater levels of halo. Second, as expected, faster responses oc-curred for the context conditions (XPOOR = 3.89, SD = 1.11; XCONTROL = 5.31,SD = 1.10; XGOOD = 3.33, SD = .91). Planned comparisons, with weightsidentical to those for halo were used. The first planned comparison onresponse latency means was significant, t(87) = 7.29, p < .05. The secondplanned comparison found no difference between good and poor contexts(p > .05). Thus, hypothesis two is supported; faster reaction times oc-cured in context conditions relative to the control condition (Figure 3).

Figure 2Halo Means


Figure 3Response Latencies for Ratings

Tests of Mediation

It was hypothesized that processing speed would mediate the effectof context on halo. To test for mediation effects, a series of regressionanalyses were computed: halo regressed onto context (step one) followedby reaction time (step two), and halo regressed onto reaction time (stepone) followed by context (step two). Evidence of mediation would be asignficant reduction of context R2 when entered into the equation afterreaction time (James & Brett, 1984). When entered alone, context ac-counted for 20 percent of the variance in halo. When entered after reac-tion time, however, context accounted for only 14.5 percent of the vari-ance; the reduction was significant (p < .05). Both context and reactiontime were significant for both analyses. In other words, reaction timewas also significant when entered after context; this indicates that indi-vidual differences in processing speed account for halo variance not ex-plained by context, and provides additional support for hypothesis one.Hypothesis three, that processing mediates the context-halo relation-ship, was thus supported.


DISCUSSION

The present study demonstrated 1) that the speed with which ratersperform ratings is related to halo in ratings, 2) that the presence of anevaluatively polarized performance context increased the speed withwhich ratings were rendered, and 3) that the speed with which ratersrendered ratings mediated the relationship between context and halo.

Hypothesis one was supported. Participants who made faster rat-ings gave ratings with higher positive halo. Participants’ retrieval anduse of a general impression without recalling specific trait or behaviorinstances is the most likely explanation for this result (Srull & Wyer,1989). Furthermore, the mediation analyses indicate that significantvariance in halo is uniquely explained by individual differences in pro-cessing speed. This finding fits nicely with the findings of Palmer,Maurer, and Feldman (1999). Their study found that participants highin Need for Cognition (Cacioppo & Petty, 1982), a measure of individualdifferences in habitual, thoughtful processing, returned ratings with lesspositive halo. It is possible that high Need for Cognition individuals aremore likely to retrieve more rating-relevant memories when making rat-ings, rather than merely using the first diagnostic thought that comes tomind.

Participants in the context conditions had faster reaction times thanthose of the control condition, supporting hypothesis two. They also com-mitted higher positive halo. Both findings support the general impres-sion explanation. Context directs attention to a context-discrepantsubset of ratee behavior, resulting in a strong, accessible general impres-sion. This general impression is based on a homogenous subset of rateebehavior, is easily retrieved from memory, and is diagnostic for all ratingdimension judgments. Halo was higher in the poor context conditionthan in either the good context condition or the target performance; thisis also consistent with prior research (Palmer, Feldman, & Maurer,1998) and theory (Skowronski & Carlston, 1989). Poor information hasa stronger influence on processing than does good information. On theother hand, good and poor context conditions did not differ on targetrating processing speed.

Hypothesis three was likewise supported. The effect of context onperformance ratings was significantly mediated by processing speed. At-titude (or general impression) accessibility, indicated by speed of process-ing, is a function of the strength of the attitude and the evaluation therater holds for the object of judgment (Fazio & Williams, 1986). Whenthe participants viewed the target lecture after having viewed good con-text lecturers, the poor performance behaviors were glaringly salient,thereby attracting attention. Thus, when it came time to rate the perfor-mance, participants did so. When the target lecturer performance was


rated without a preceding experimentally provided context, reactiontimes were slower; raters in this condition had to integrate a mixture ofgood and poor behavioral information into the creation of an impression(Palmer, Maurer, & Feldman, 1999; Srull & Wyer, 1989).

CONCLUSIONS

Future research should examine the effects of rater training on reac-tion times. The present study demonstrated that reaction time is clearlyrelated to general impression use and halo. Therefore, research on ratertraining might benefit from its inclusion as a criterion. At minimum, itappears reaction time for ratings may be an indication of which individu-als habitually use general impressions for specific and distinct ratings.It may also be possible to train raters to make quality decisions withoutexpending an excessive amount of time in rendering decisions. In otherwords, reaction time as a criterion should shed light on how raters, withpractice, learn to automatically make performance ratings with little ef-fort (Feldman, 1992). The present research demonstrated that a signifi-cant relationship between processing speed and traditional performancejudgement dependent variables (e.g., halo, contrast error) exists. Assuch, it represents a promising step toward understanding the complexrelationship between reaction time, context, and halo.

REFERENCES

Balzer, W. & Sulsky, L. (1992). Halo and performance appraisal research: A critical reex-amination. Journal of Applied Psychology, 77, 975–985.

Cacioppo, J. T. & Petty, R. E. (1982). The need for cognition. Journal of Personality andSocial Psychology, 42, 116–131.

Fazio, R., Chen, J., McDonel, E., & Sherman, S. (1982). Attitude accessibility-behaviorconsistency, and the strength of the object-evaluation association. Journal of Experi-mental Social Psychology, 18, 339–357.

Fazio, R. & Williams, C. (1986). Attitude accessibility as a moderator of the attitude-per-ception and attitude-behavior relations: An investigation of the 1984 presidential elec-tion. Journal of Personality and Social Psychology, 51(3), 505–514.

Feldman, J. (1981). Beyond attribution theory: Cognitive processes in performance ap-praisal. Journal of Applied Psychology, 66, 127–148.

Feldman, J. (1992). The Case for non-analytic performance appraisal. Human ResourceManagement Review, 2(1), 1434–1469.

Feldman, J. & Lynch, J. (1988). Self-generated validity and other influences of measure-ment on belief, attitude, intention, and behavior. Journal of Applied Psychology, 73,421–435.

James, L. R., & Brett, J. M. (1984). Mediators, moderators, and tests for mediation. Jour-nal of Applied Psychology, 69(2), 307–321.

Kravitz, D. & Balzer, W. (1992). Context effects in performance appraisal: A methodologicalcritique and empirical study. Journal of Applied Psychology, 77(1), 24–31.


Lance, C., LaPointe, J., & Stewart, A. (1994). A test of the dependency of three causalmodels of halo rater error. Journal of Applied Psychology, 79(3), 83–96.

Lance, C., LaPointe, J. A., & Fisicaro, S. A. (1994). Tests of three causal models of haloerror. Organizational Behavior and Human Decision Processes, 57, 83–96.

Maurer, T. & Alexander, R. (1991). Contrast effects in behavioral measurement: An inves-tigation of alternative process explanations. Journal of Applied Psychology, 76(1),3–10.

Murphy, K., Balzer, W., Lockhart, M., & Eisenman, E. (1985). Effects of previous perfor-mance on evaluations of present performance. Journal of Applied Psychology, 70(1),72–84.

Palmer, J., Feldman, J., & Maurer, T. (1998). Contrast, halo, and accuracy: Systematicrelationships explored. Paper presented at the 13th annual conference of the Societyfor Industrial and Organizational Psychology, Dallas, TX.

Palmer, J., Maurer, T., & Feldman, J. (1999). Accountability in Performance Appraisal:Higher Accuracy and Reduced Errors. Paper presented at the 14th annual conferenceof the Society for Industrial and Organizational Psychology, Atlanta GA.

Skowronski, J. J. & Carlston, D. E. (1989). Negativity and extremity biases in impressionformation: A review of explanations. Psychological Bulletin, 105(1), 131–142.

Srull, T. & Wyer, R. (1989). Person memory and judgement. Psychological Review, 96,58–84.

Wexley, K. N., Yukl, G. A., Kovacs, S. Z., & Sanders, R. E. (1972). Importance of contrasteffects in employment interviews. Journal of Applied Psychology, 56(1), 45–48.

effects of performance context on processing speed and performance ratings

Documents