The Next Generation of Leadership Assessments: Some Case Studies

Download The Next Generation of Leadership Assessments: Some Case Studies

Post on 08-Apr-2017




0 download

Embed Size (px)


  • Personnel Management online version of this article can be found at:

    DOI: 10.1177/009102600803700405

    2008 37: 435Public Personnel ManagementMarilyn K. Gowing, David M. Morris, Seymour Adler and Mitchell Gold

    The Next Generation of Leadership Assessments: Some Case Studies

    Published by:

    can be found at:Public Personnel ManagementAdditional services and information for Alerts:

    What is This?

    - Dec 1, 2008Version of Record >>

    at TEMPLE UNIV on December 5, 2014ppm.sagepub.comDownloaded from at TEMPLE UNIV on December 5, 2014ppm.sagepub.comDownloaded from

  • The Next Generation ofLeadershipAssessments: SomeCase StudiesBy Marilyn K. Gowing, PhD, David M. Morris, PhD, JD, Seymour Adler, PhD,and Mitchell Gold, PhD

    In this article, we provide a brief overview of leadership theory and research, somebackground on the traditional assessment center process, and detaileddescriptions of three of the latest approaches to leadership assessment, alongwith case studies drawn from public and private organizations. The threeapproaches include the Telephone Assessment Program, LEADeR, and video-based assessment centers. We conclude with some suggestions for newdirections in leadership simulation design.

    Great leaders. We know them when we see them, but how can we identifythose high-potential employees who will one day run their organizations?Workforce planners have been warning for years of the impending wave ofbaby boomer retirements. Many organizations are finally heeding those warnings andtaking steps to identify and nurture leadership talent.

    While professionals in psychology have been in the forefront of leadershipassessment for many decades, some fascinating new approaches have recentlyemerged that allow organizations to maximize the use of technology, and these aregaining broad acceptance in corporate America and in local, state, and federalgovernment organizations.

    In this article, we provide a brief overview of leadership theory and research,some background on the traditional assessment center process, and detaileddescriptions of three of the latest approaches to leadership assessment, along with casestudies drawn from public and private organizations. We conclude with somesuggestions for new directions in leadership simulation design.

    The first of the new approaches we shall describe is the Telephone AssessmentProgram (TAP), which was created by one of the authors of this review, Seymour Adler,PhD. TAP was one of the earliest attempts to take the complex and costly assessmentcenter process and make it more efficient.

    A later innovation that will be described here is LEADeR, which is a miniassessment center on computer. Aon Consulting was the first company to market thefully automated leadership simulation developed by Kirk Rogg, PhD. Sadly, Dr. Rogg

    Public Personnel Management Volume 37 No. 4 Winter 2008 435

    at TEMPLE UNIV on December 5, 2014ppm.sagepub.comDownloaded from

  • passed away in 2007, ending a brilliant career much too soon. This article is dedicatedto Dr. Rogg, as well as to Dr. Mark Lifter, the former head of the Aon Consulting TalentSolutions practice, who supported the LEADeR research.

    The third approach we describe, Video-Based Assessment Centers, is anotherattempt to enhance the efficiency and effectiveness of an assessment center process byusing videotape in creative ways. David M. Morris, PhD, JD, president of Morris &McDaniel and the designer of this innovative, streamlined assessment system, is alsoone of the authors of this review.

    What Does the Leadership Research Tell Us?We are fortunate to have an excellent summary of the history of leadership theory andresearch in the form of a 2007 special issue of the American Psychologist.1 This issuebegins with a compelling quote by Robert Sternberg: The United States became agreat nation because of the leadership skills of the Founding Fathers. Whether it willremain a great nation will depend, in large part, on the leadership skills of those inpower today.2 So, with the success of our organizations, our government, and ourfuture depending upon our leaders, what should those responsible for developingtomorrows leaders be assessing?

    Assessing the PersonZaccaro has summarized the evolution of trait-based leadership theories.3 He beganwith the earliest leadership studies and their emphasis on the unique attributes ofleaders that were inherited and part of leaders genetic makeup.4 Zaccaro indicatedthat this perspective, suggesting that leadership qualities were traits that were largelyimmutable and not amenable to development, guided the preponderance of theleadership research into the 20th century until Stogdill and others suggested that trait-based leadership was insufficient to explain leaders effectiveness.5 Zaccaro added,This rejection [of trait-based leadership] was widespread and long lasting, and itechoed in most of the major social and industrial and organizational psychologytextbooks for the next 30 to 40 years.6

    Zaccaro noted that in the 1980s, charismatic and transformational leadershipmodels again surfaced, emphasizing extraordinary qualities of individuals asdeterminants of those peoples effectiveness. Zaccaro concluded that recent studies,including his own research, have linked personality variables and other stable personalattributes to leaders effectiveness, providing a substantial empirical foundation for theargument that traits do matter in the prediction of individuals effectiveness as leaders.7

    Many theorists have gone beyond traits, emphasizing extensive lists of abilities,competencies, and skills that are essential for leadership effectiveness.8

    Assessing the SituationWhile many theorists have been concentrating on the identification of individualattributes that contribute to leaders effectivenesswhether they be stable personalitytraits, competencies, knowledge, values, or something elseothers have focused on

    Public Personnel Management Volume 37 No. 4 Winter 2008436

    at TEMPLE UNIV on December 5, 2014ppm.sagepub.comDownloaded from

  • the environment in which leaders operate. Vroom and Jago9 credited Stogdill10 withbeing among the first to conclude that an adequate analysis of leadership involves notonly a study of leader attributes, but also an investigation of the situations in which aleader operates.

    The Ohio State studies in the 1950s and 1960s, with Shartle, Stogdill, Fleishman,and others, focused on leadership behavior. Vroom and Jago suggested, Leaderbehavior research was a step in the direction of acknowledging the role of situation orcontext in leadership. Unlike traits, behavior is potentially influenced not only by theleaders dispositions, but also by the situations that leaders confront. Leaderbehavior can therefore be an effect of subordinate behavior as well as a cause of it.11

    There were also a number of pure situational theorists who found that very littleof the organizational outcome variance could be explained by changes in leadership.12

    Assessing BothVroom and Jago have summarized the current state of leadership theory best, writing,Most social scientists interested in leadership have now abandoned the debatebetween person or situation in favor of a search for a set of concepts that are capable ofdealing both with differences in situations and with differences in leaders.13 Theseapproaches have yielded contingency theories that try to describe what type of peopleand behaviors are effective in different situations.14 Vroom and Jago concluded thatleadership, which involves the process of influence, is represented in all aspects of thatprocess, including the traits of the source of influence, the cognitive process of thesource, the nature of the interaction that makes influence toward a goal possible, andthe situational context of that process. With so many variables influencing theeffectiveness of a leader, the leadership assessment had best be complex. What betterway to capture and manipulate these various aspects of leadership than in simulations?

    The Traditional Assessment Center ProcessFor more than 25 years, assessment centers have been seen as a validby some, themost validmethod for assessing leadership potential.15 A core element of theassessment center method is deploying a team of trained assessors who evaluate eachcandidates job-relevant leadership competencies by observing the candidate inmultiple simulation exercises. In addition, assessors often also draw on otherassessment tools such as tests and interviews. Traditionally, final candidate evaluationsare arrived at through a judgment process that involves discussions between theassessors of their individual evaluations and the formation of a consensus opinionregarding a candidate.

    It is important to note that a half century after Douglas Bray first introducedassessment centers in a nonmilitary context for leadership assessment at AT&T,16 it isstill not entirely clear why assessment centers are valid, as C.E. Lance has discussed.17

    What is clear is that assessment centers, when properly designed and administered,provide valid information for making hiring, promotion, and development planningdecisions for leadership positions.

    Public Personnel Management Volume 37 No. 4 Winter 2008 437

    at TEMPLE UNIV on December 5, 2014ppm.sagepub.comDownloaded from

  • For most of the past 50 years, leadership assessment centers have been physicallocations, with multiple candidates and multiple assessors present for periods rangingfrom one to four days.18 In the most typical setup, 10 to 12 candidates, six assessors,and an administrator come together at a central facility that has one or two conferencerooms and six or more smaller rooms for breakout sessions. Candidates come to thelocation for a day or two of assessment exercises, interviews, and tests. The assessorsstay a day or two longer to discuss each candidate and to develop a compositecandidate profile and final overall evaluation for each candidate. This traditionalstructure is still widely used, especially in the public sector.19

    This traditional bricks-and-mortar design for assessment centers is costly andcumbersome. The notion that this traditional design is the only acceptable way to doassessments has, over the past 20 years, led many organizations to move away fromusing assessment centers despite the centers proven effectiveness and towardalternative and often inferior ways of assessing employees and possible hiresleadership potential. Other organizations have increased assessment center efficiencyby videotaping participants performances on exercises and scoring the participants ata later time.20 Lievens and Thornton21 discussed the increased use of computerized inbaskets or video simulations in which participants provide responses to multiple-choice questions. They also pointed out that, while such uses of technology canincrease the efficiency and flexibility of assessment centers, those efficiencies andflexibilities come at a significant cost in terms of lowered fidelity of the simulationexercises and, thus, the credibility and validity of the exercise results.

    One of the key benefits of using assessment centers in which candidates andassessors interact face to face is the rich behavioral information the interactionsprovide about a candidate. That information can be critical in a developmentalassessment center when participants receive diagnostic feedback and work with amanager or coach to create a specific and actionable individual development plan. Itcan also be valuable in a selection/promotion assessment center for promotingperceptions of procedural fairness.22

    In the next sections, we describe three alternatives to the traditional assessmentcenter design that incorporate all the elements that make bricks-and-mortarassessment centers valid and behaviorally rich, but which leverage technology to makeassessments flexible, simple, and significantly less costly. We describe how each ofthese nontraditional approaches was developed; detail how they are designed,validated, and applied in real-world settings; and present case studies on theirapplication for leadership assessment.

    Breaking Out of Bricks and Mortar: The TelephoneAssessment ProgramIn 1979, a major financial services firm, under a court order to develop a moreobjective, job relevant selection procedure for hiring account executives, engagedAssessment Solutions Incorporated (ASI) to design an assessment center for evaluatingcandidates. At the time, the industry-leading firm Merrill Lynch had successfully

    Public Personnel Management Volume 37 No. 4 Winter 2008438

    at TEMPLE UNIV on December 5, 2014ppm.sagepub.comDownloaded from

  • implemented such an approach for account executive screening.23 The Merrill Lynchprogram used a traditional assessment center approach, using experienced managersand account executives as assessors and bringing groups of candidates to a branchoffice during off-hours to conduct the assessment.

    The new assessment center developed by ASI followed this established modelinitially, requiring candidates to come to a testing facility. Each candidate was assessedon their performance during six role-play exercises that were designed to represent therange of challenging, on-the-job situations that were identified during job analysis asbeing typical of those encountered by new account executives early in their careers.These situations reflected interactions with different types of prospects and customers.Four of these simulated interactions were conducted over the telephone by twoassessors who were situated remotely, and the other two simulations wereadministered on site and face to face by two assessors.

    Even in this first iteration of an assessment solution, ASIs design departed fromorthodox assessment center practice in two ways. Like the initial assessment centerused by AT&T in its famous Management Progress Study,24 but unlike virtually alloperational centers for the following 25 years, ASI used professional assessors whowere specially hired and trained to conduct role-plays and evaluate candidates usingassessment center methodology. Unlike the Management Progress Study assessmentprocess, though, the one developed by ASI...


View more >