the influence oe role prescriptions on the performance appraisal process.pdf

7/29/2019 THE INFLUENCE OE ROLE PRESCRIPTIONS ON THE PERFORMANCE APPRAISAL PROCESS.pdf

1/8

1978 Schneier and Beatty 129Academy of Management Journal1978, Vol. 21, No. 1, 129-135

THE INFLUENCE OE ROLE PRESCRIPTIONS ON THEPERFORMANCE APPRAISAL PROCESS

CRAIG ERIC SCHNEIERUniversity of Maryland

RICHARD W . BEATTYUniversity of Colorado

The appraisal of individual performance is a pervasive organizationalissue. Perform ance appraisal (P A ) results typically augm ent the rationalefor several personnel decisions, such as promotion and wage and salaryadministration (see Cumm ings & Schwab, 19 73 ). As Cumm ings (19 73 )noted, however, the design and use of PAs are often based on prescriptivesuggestions rather than empirical field research. One such intuitively ap-pealing and often repeated prescription (see e.g.. Miner, 1971) advocatesthe use of multiple raters (i.e., superior, peer, subordinate, and/or selfratings) to evaluate an employee. The purpose of this paper is to examinethe proposition that the role prescriptions of the multiple raters are a majordeterminant of raters' judgments.

There are several potential advantages to using multiple raters in thePA process. A psychometric advantage is that multiple raters permit theassessment of convergent validity of multiple rating criteria through themultitrait-multirater matrix (Lawler, 1967). Content validity may also beenhanc ed as observations of ratee performance may be increased (Bo r-man, 1974), thus tapping more of the behavioral domain of the job. Theremay also be operational advantages because the use of multiple raterswidens the participation of relevant persons in the PA process and thusfosters interest and commitment.Behind these potential advantages lies the question as to whether ratersoccupying different roles would be expected to hold congruent perceptionsof ratee performance. That is, persons with different role prescriptionswithin an organization may develop different expectations for ratees' per-formance based on their (raters') own jobs.

Among those who reported agreement on ratings across roles are Fogli,Hulin, and Blood (1971) and Kavanagh, MacKinney and Wolins (1971).Differences in ratings of a group of ratees by raters occupying differentroles were reported in at least as many, and perhaps more, studies (e.g.,Bernardin & Alvares, 197 5; Henem an, 197 4; Th ornto n, 1968 , reviewed by


2/8

130 Academy of Management Journal March(e.g., Zedeck & Baker, 1 97 2) . Other studies have comp ared such ratingsto assess convergent validity and hence have assumed these to be differentmethods of measurement (e.g., Hen eman , 197 4; Lawler, 19 67 ). The as-sumption of similarity or difference of these methods of measurement isnot made explicit. The issue of whether multiple rater perceptions shouldbe expected to converge or diverge has thus been largely ignored in research .The Influence of Role Prescriptions on Ratings

Divergence between perceptions of those raters occupying different roleswould seem to come first from the job env ironm ent. Specifically, differencesin job duties and proximity, causing differing frequencies and/or durationof observation of ratee performance, could account for divergent ratingsgiven by, for example, superiors and peers. This thesis has some empiricalsupport (Borm an, 1974; Zedeck & Baker, 19 72 ).Differences in behavior of raters across organizational levels could alsobe due to the influence of rater role prescriptions on perceptions of rateeperformance. A role is that "set of prescriptions defining what the behaviorof a position member should be " (Biddle & Tho m as, 196 6, p. 2 9 ) . T henotion of role prescriptions (i.e., guidelines for proper behavior by roleincumbents) has been used as a powerful explanatory concept in generaltheories of interperson al perception (e.g., Jone s & Th ibau t, 1 95 8) as welas in theories of organization al behav ior (e.g., Grae n, 19 76 ; Ka tz & Ka hn1966) . Prescriptions for raters could include the role of evaluator, judge,or critic of subordinates' (ratees') performance. In terms of Katz andKahn's (1966) role episode, these roles are "sent" to supervisors actingas raters by their own superiors and by perceptions of organizational struc-ture and policies. Peers of raters, on the other hand, are not typically "sent"such formal, evaluator role prescriptions. As supervisors respond fromtheir critic roles, peers may respond from their roles as co-workers, colleagues and friends of ratees.

In support of this logic, Zedeck, Imparato, Krausz and Oleno (1974)reported that superiors rated the same behaviors as illustrative of lowerlevels of performance than did their subordinates. Schneier, Beatty andBeatty (1976) found that supervisors perceived undesirable ratee behaviors as occurring more often than did the ratees' peers. Additional supportis provided by Landy and Guion (1 9 7 0 ), Kirchner (1 9 6 5) , and Barret( 1 9 6 3 ) .But whether differing role prescriptions lead to divergent PA percep-tions is unresolved. Conflicting research h as ap peared (e.g., Dickinson &Tice, 19 73 ; Henem an, 19 74 ), and more importantly, research designs(e.g., Borm an, 1974 ; Dickinson & Tice, 19 73 ; Hene man , 19 74; Schneie


3/8

1978 Schneier and Beatty 13 1An Empirical Test

Four sequential areas of inquiry guided the researchtwo tests of ex-perimental controls and two research hypotheses. First, because the sub-jects had essentially the same primary job tasks or duties, it was assumedthat any two groups of raters would generate essentially the same set of jobdimensions upon which to evaluate the ratees. Second, an a priori assump-tion was made that the two rater groups had equal opportunities to observeratee performance. If both of these assumptions are confirmed, the ex-perimental controls would appear to be effective. With such controls, dif-ferences found either in expectations toward performance and/or in actualratings would thus seem to come from basic differences in role prescriptionsacross levels and not from differences in primary tasks or observation fre-quency (i .e., from the job environment).

Hypotheses Hypothesized differences across hierarc hical levels in termsof performance expectations could be tested as raters from two levels in-dicated the degree of performance illustrated by a group of critical in-cidents. It is also hypothesized that the superiors would rate the rateeslower than peers, due to their roles as evaluators or critics of performanceand dispensers of merit raises and o the r rewa rds (see e.g., Klimoski &London , 1974; Thornton , 19 68 ).Method

Sample The research was cond ucted in a medium-sized man ufactur-ing company. The two roles used were entry-level manufacturing workers(n=74) and their immediate superiors (n=15) who worked closely withtheir subordinates in teams.ProcedureIn order to assess perceptions of job-task similarity betweenthe two roles, small groups of members from each role were asked to gen-erate or "brainstorm" a list of PA criteria reflecting their job duties and

then to weigh them in degree of importance. Second, to assess assumedsimilarity in frequency of observation of ratee performance across roles,members of each role were to indicate how often they perceived each of aset of 183 critical incidents as actually occurring among ratees. Third, eachmember of the two roles was to indicate, using a Likert scale, the degree ofperformance (excellent to unacceptable) illustrated by each of the 183incidents. Finally, randomly selected ratees (n=31) were each evaluatedby three to five peers and two to three superiors chosen at random fromthose who had worked on the same shift as the ratee.Resnlts


4/8

132 Academy of Management Journal MarchTABLE 1

Mean Ranks for Job DimensionsJo bDimension

DryingSystem flo wRecord keepingGrindingAttendanceCentrifugationFiltrationInitiativeReaction controlSafetyDependabilityDistillationsHousekeepingCommunication

Mean SubordinateRank (n = 24)2.56.58.62.08.95.65.611.18.812.68.86.6S.611.0

Mean SuperiorRank (n = 12)2.66.88.02.18.75.15.511.08.612.29.97.011.011.5

t value - . 1 3 9 .492.884 .126.259.636.135.137.252.435- 1 . 1 4 9.574- 1 . 7 8 2 .579

All tests were two-tailed; critical t for n of 40 = 2.021.then ranked each of the 14 job dimensions for importance as PA criteriaTa ble 1 shows mean ran ks for each of the 14 dimensions, as well as r-testsfor differences between mean ranks. As none of the r-tests were significantthe two groups were assumed to be in agreemen t on the w eighting of the jobdimensions.

A /-test for dep end ent measures (pa ired differences) was used to assesdivergence in the two grou ps' perceptions of how frequently the 183 behavioral incidents actually occurred on the job. The result of this test indicatedgeneral agreement across rater groups on the frequency of observed behavior, r(1 82 ) = .270, ns.A /-test for dependent measures was again used to assess superior-subordinate differences given the behavioral incidents. The test indicated a difference between the groups, /(182) = 2.137, p < .025. Superiors wermore lenient in that they assigned higher scale values to incidents indicative of subordinate performance than did the subordinates themselves (i.e.they felt behaviors were illustrative of better performance than did theisubordinates).Mean ratings given each of the 31 ratees by the superior and peer raterat two different time periods four months apart were used as data pointin a two-factor, repeated measures analysis of variance (A N O V A ). Asignificant rater role effect (F [l ,9 0 ] = 55 .89 4; p < .001) and a significantime period effect (F [l, 9 0] = 16.1 52 ; p < .001) were found. The interaction effect was not significant ( F [l ,9 0 ] = .3 5 3 ; ns). The table of meanfor these data (Table 2) indicates that, as predicted, superiors gave loweratings at both time periods than peers.


5/8

1978 Schneier and Beatty 133TABLE 2

Mean Performance Ratings Across Ratersand Time Periods

Period Superiors PeersI 324.3 348 .!2 " 337.9 358.3 Time periods were separated by a four-month interval.

erally agree as to their identification of PA criteria and weights of thosecriteria; (b) given a priori assumptions of similarity in frequency of observa-tion of ratees, members of two roles generally agree as to their perceptionsof frequency of occurrence of specific ratee behaviors; (c) given equal jobtasks and observation frequency, occupants of two different roles disagreeas to their perceptions of behaviors desired or expected for successful per-formance; and (d) given equal job tasks and observation frequency, oc-cupants of two different roles disagree as to their actual ratings of ratees.The divergent perceptions of expected performance held by occupants oftwo different roles, as well as their divergent actual ratings, both point to afundamental difference in role prescription across levels which, if supportedby further research, would seem to have important implications for thepsychometric and operational aspects of the PA process.The divergence found here could in part explain the disappointing inter-rater reliability and convergent validity evidence found in psychometricstudies employing members of two different organizational levels as raters(e.g., Lawler, 197 6; Zedeck & Baker, 19 72 ). T hat is, to the extent th atthere are fundamental differences in orientation across different roles rela-tive to ratees, high interrater reliability correlation coefficients cannot beexpected.

The second implication of the findings of the present research concernspractical or operational aspects of appraisal. In this regard, the specificnature of the differences found here between hierarchical levels is note-worthy. This study found superiors giving higher scale values to incidentsthan subordinates, contrary to the hypothesis. However, in keeping withtheir role prescriptions as critics of performance, superiors reversed theirlenient orientation when actually evaluating the ratees and gave lower rat-ings than did the peer raters at each of two different time periods. This pat-tern of results has also been noted in past research (e.g., K limoski & Lo n-don, 1974; Zedeck et al, 1974). Because many unsupported suggestionsare offered to PA system designers concerning the utility of using ratersfrom mo re than o ne role in PA, differences between role perceptions should


6/8

134 Academy of Management Journal Marchthe same rating scale format and whether to anticipate agreement fromthem.Th e present study, as well as other rece nt PA research (e.g., Grey &Kipnis, 19 76; Hake l, 1 974 ; Scott & Hamner, 1976), signals a marked shiftfrom almost total concentration on appraisal formats and their psychome-tric properties to the investigation of various rater characteristics whichinfluence the results and operation of PA systems. This recent research em-phasis has advanced understanding of the PA process (i.e., recalling per-formance recall, trait attribution, human judgment and decision making,perceptual selection and bias), as well as the outcomes of the process (i.e.,the ratings themselves).

REFERENCES1. Barrett, R. S. "Performance Suitability and Role Agreement: Two Factors Related toAttitudes," Personnel Psychology, Vol. 16 (1963), 345-367.2. Bernardin, H. J., atid K. M. Alvares. "The Effects of Organizational Level on Percep-tions of Role Conflict Resolution Strategy," Organizational Behavior and HumanPerformance, Vol. 14 (1975), 1-9.3. Biddle, R., and E . J. Th om as. Role Theory (New York: Wiley, 1966).4. Borman, W. C. "The Rating of Individuals in Organizations: An Alternate Approach,"Organizational Behavior and Hu man Performance, Vol. 12 (1974), 105-124.5. Cummings, L. L. "A Field Experimental Study of the Effects of Two PerformanceAppraisal S ystems," Personnel Psychology, Vol. 26 (1973), 489-502.6. Cummings, L. L., and D. P. Schwab. Performance in Organizations (Glenview, 111.:

Scott Foresman, 1973).7. Dickinson, T. L., and T. L. Tice. "A Multitrait-Multimethod Analysis of Scales Devel-oped by Retranslation," Organiaztional Behavior and Human Performance, Vol. 8(1973), 421-438.8. Fogli, L., C. L. Hulin, and M . R. Blood. "Developm ent of First-Level Behavioral JobCriteria," Journal of Applied Psychology, Vol. 55 (1971), 3-8.9. Graen, G. "Role-Making Processes within Complex Organizations," in M. D. Dun-nette (Ed.), Handbook of Industrial and Organization Psychology, (Chicago: RandMcNally, 1976), 1201-1246.10. Grey, R. J., and D. Kipnis. "Untangling the Performance Appraisal Dilemma: TheInfluence of Perceived Organizational Context on Evaluative Processes," Journal ofApplied Psychology, Vol. 61 (1976), 329-335.11. Hakel, M. D. "Normative Personality Factors Recovered from Ratings of PersonalityDescriptors: The Beholder's Eye," Personnel Psychology, Vol. 27 (1974), 409-421.12. Heneman, H. G. "Comparison of Self and Superior Ratings of Managerial Perfor-mance," Journal of Applied Psychology, Vol. 59 (1974), 638-642.13. Jones, E. E., and J. W. Thibaut. "Interaction Goals as Bases of Inference in Interper-sonal Perception," in R. Tagiui and L. PetruUo (Eds.) Person Perception and Interper-sonal Behavior (Stanford, CaWt.: Stanford University, 1958), 151-178.14. Katz, D., and R. L. Kahn. The Social Psychology of Organizations (New York: Wiley,1966).15. Kavan augh, M . J., A. C. Mac Kinney, and L. Wolins. "Issues in Managerial Perfor-mance: Multitrait-Multimethod Analyses of Ratings," Psychological Bulletin, Vol. 75(1971), 34-49.16. Kirchner, W. K. "Relationships between Supervisory and Subordinate Ratings forTechnical Personnel," Journal of Industrial Psychology, Vol. 3 (1965), 57-60.


7/8

1978 Schneier and Beatty 13520. Miner, J. B. "Management Appraisal: A Capsule Review and Current References,"in W. L. French and D. Hellriegel (Eds.), Personnel Management and OrganizationDevelopment (Boston: Houghton Mifflin, 1971), 247-261.21. Schneier, C. E., R. W. Beatty, and J. R. Beatty. "An Empirical Investigation of Per-

ceptions of Rater Behavior Frequency and Ratee Behavior Change Using BehavioralExpectation Scales (BES)" (Unpublished paper. University of Maryland, 1976).22. Scott, W. E., and W. C. Hamner. "The Influence of Variations in Performance Pro-files on the Performance Evaluation Process: An Examination of the Validity of theCriterion," Organizational Behavior and Human Performance, Vol. 14 (1975), 360-370.23. Thornton, G. C. "The Relationship Between Supervisory and Self-Appraisals of Exec-utive Performance," Personnel Psychology, Vo l. 21 (1968), 441-4 55.24. Zedeck, S., and H. T. Baker. "Nursing Performance as Measured by Behavioral Expecta-tion Scales: A Multitrait-Multirater Analysis," Organizational Behavior and HumanPerformance, Vo l. 7 (1972), 457-466.25. Zedeck, S., N. Imparato, M. Frausz, and T. Oleno. "Development of Behaviorally An-chored Rating Scales as a Function of Organizational Level," Journal of Applied Psy-chology, Vol. 59 (1974), 249-252.

Academy of Management Journat1978, Vo l. 21 , N o. 1, 135-140

INFLUENCE SOURCES OF PROJECT AND FUNCTIONALM ANAGERS I N M ATRI X ORGANI ZATI ONS

EDW ARD J . DUNNE, J R.MICHAEL J. STAHLLEONARD J . MELHART, JR.

Air Force Institute of TechnologyA topic of interest to those who study and practice project management isthe influence or authority structure in an organization which contains proj-ects. Formal authority typically resides primarily with managers in function-al areasengineering, procurement, production, et ceterabut a projectmanager usually has responsibility for coordinating efforts across severalfunctional areas. The influence exerted must be based on more than formalauthority.French and Raven (1959) classify five different power bases as sourcesof influence: legitimate pow er, reward pow er, coercive power, expert p ow er,and referent power. This typology has been examined in functional organ-izations. Ivancevich and Donnelly (1970) found that expert and referentpow er were positively assoc iated with measure s of organ ization al effective-ness. Bachman, Bowers and Marcus (1968) found that legitimate powerwas important for complying with supervisors' requests and that expert pow-er was positively associated with subordinate satisfaction and performance.In project management organizational settings, Lucas (1973) and Hodgetts(1968) both interviewed project managers concerning sources of influence.


8/8

the influence oe role prescriptions on the performance appraisal process.pdf

Documents