the context group innovative methods of selecting new employees: simulations and in-baskets warren...
TRANSCRIPT
The Context Group www.contextgroup.com/hr.htm
Innovative Methods of Selecting Innovative Methods of Selecting New Employees: Simulations and New Employees: Simulations and
In-Baskets In-Baskets Warren Bobrow, Ph.D.Warren Bobrow, Ph.D.
The Context GroupThe Context [email protected]@contextgroup.com
(310) 670-4175(310) 670-4175
The Context Group www.contextgroup.com/hr.htm
Some History In Personnel Some History In Personnel SelectionSelection
• Personality Now Of Great Personality Now Of Great Interest to I/O PsychologistsInterest to I/O Psychologists
• Performance Testing Has a Long Performance Testing Has a Long History in I/O PsychologyHistory in I/O Psychology
• Computer-Based Tests Are Computer-Based Tests Are Increasingly PopularIncreasingly Popular
The Context Group www.contextgroup.com/hr.htm
Validity of Various Selection Validity of Various Selection ProceduresProcedures
(Schmidt & Hunter, 1998)(Schmidt & Hunter, 1998)
Personnel measure Validity (r) Personnel measure Validity (r)
Work Sample Tests .54 Biographical Data .35
Cognitive Ability (IQ) .51 Conscientiousness Tests .31
Interviews (structured) .51 Reference Checks .26
Peer Ratings .49 Job Experience (years) .18
Job Knowledge Tests .48 Education (years) .10
Integrity Tests .41 Interests .10
Job Tryout Procedures .44 Graphology (handwriting) .02
Interviews (unstructured) .38 Age -.01
The Context Group www.contextgroup.com/hr.htm
What We’ll Be What We’ll Be AddressingAddressing
• The Criterion Validity of The Criterion Validity of Various Innovative Various Innovative Procedures.Procedures.
• How Streamlining How Streamlining Assessments Can Increase Assessments Can Increase ROI and Usability.ROI and Usability.
The Context Group www.contextgroup.com/hr.htm
Comparison of Validity Comparison of Validity Between Live and Computer-Between Live and Computer-
Based SimulationsBased Simulations• Many organizations are looking Many organizations are looking
to leverage computers in to leverage computers in selectionselection– More efficiencyMore efficiency– More accurate data gatheringMore accurate data gathering– Projects a more positive image with Projects a more positive image with
applicantsapplicants
• Are they getting good value?Are they getting good value?
The Context Group www.contextgroup.com/hr.htm
Why is This Important?Why is This Important?
• If Computer-Based Simulations If Computer-Based Simulations Are As Effective as Live Ones, Are As Effective as Live Ones, Then:Then:
– Objective Way To Measure Skill Objective Way To Measure Skill and Abilityand Ability
– Cost Savings Can Be RealizedCost Savings Can Be Realized
The Context Group www.contextgroup.com/hr.htm
Live and Computer Based Live and Computer Based SimulationsSimulations
• Live Simulations Have Been Shown To Live Simulations Have Been Shown To Be Valid Predictors of Job PerformanceBe Valid Predictors of Job Performance
• Computers Are Limited in Their Ability Computers Are Limited in Their Ability to Analyze Open-Ended Responsesto Analyze Open-Ended Responses
• Is the Lower Cost of Computer Is the Lower Cost of Computer Simulations Worth the (Presumed) Simulations Worth the (Presumed) Lower ValidityLower Validity
The Context Group www.contextgroup.com/hr.htm
Item Writing Possibilities: Item Writing Possibilities: All Applicants Get the Same TestAll Applicants Get the Same Test
Item1
Item2
The Context Group www.contextgroup.com/hr.htm
Item Writing Possibilities: Item Writing Possibilities: All Applicants Potentially Get A Unique All Applicants Potentially Get A Unique
TestTest
Item1
Item2
Item4
Item3
The Context Group www.contextgroup.com/hr.htm
Item Writing Possibilities: Item Writing Possibilities: All Applicants Get Similar But Different All Applicants Get Similar But Different
TestsTests
Item1
Item2
Item3
Item4
The Context Group www.contextgroup.com/hr.htm
Simulation Simulation DemonstrationsDemonstrations
• Supervisor In-BasketSupervisor In-Basket
• Supervisor CoachingSupervisor Coaching
• The CSR Computer Based The CSR Computer Based Simulation TestSimulation Test
• The CSR Live Simulation TestThe CSR Live Simulation Test
The Context Group www.contextgroup.com/hr.htm
ComparisonComparison
• But Do These ‘Innovative’ But Do These ‘Innovative’ Methods work as well (and Methods work as well (and have the same ROI) as have the same ROI) as Traditional Ones?Traditional Ones?
• Same Construct Validity?Same Construct Validity?
• Worse/Equal/Better Criterion-Worse/Equal/Better Criterion-Related Validity?Related Validity?
The Context Group www.contextgroup.com/hr.htm
What the CBT MeasuresWhat the CBT MeasuresAbility Call A Call B Call C
Typing Call A XTyping Call B XTyping Call C XSpeed Call A XSpeed Call B XSpeed Call C XEmpathy Call A XEmpathy Call B XEmpathy Call C XListening Call A XListening Call B XListening Call C X
The Context Group www.contextgroup.com/hr.htm
Live SimulationsLive Simulations
• Designed as a Traditional Designed as a Traditional Simulation ExerciseSimulation Exercise
• Trained Role-Players/RatersTrained Role-Players/Raters
• 1 Practice and 2 Real Calls1 Practice and 2 Real Calls
The Context Group www.contextgroup.com/hr.htm
Live Simulation Factor Live Simulation Factor StructureStructure
Ability Call 1 Call 2Speed Call 1 XSpeed Call 2 XProblem Solving Call 1 XProblem Solving Call 2 XEfficiency Call 1 XEfficiency Call 2 X
The Context Group www.contextgroup.com/hr.htm
Validities:Validities:Computer vs. LiveComputer vs. Live
Live ComputerCost Per Applicant $70 $35Median Correlation with Productivity 0.24 0.23Median Correlation with Customer Satisfaction 0.21 0.22
Live k=3, Computer k=21
The Context Group www.contextgroup.com/hr.htm
Moral to the StoryMoral to the Story
• Computer Based Simulations Have Computer Based Simulations Have the Same Construct Validity as the Same Construct Validity as Their Traditional CounterpartsTheir Traditional Counterparts
• Multiple Choice Format Does Not Multiple Choice Format Does Not Lead to a Decrement in ValidityLead to a Decrement in Validity
• Innovative Methods Provided Innovative Methods Provided Higher ROIHigher ROI
The Context Group www.contextgroup.com/hr.htm
Improvements to Improvements to Computer TestsComputer Tests
•Better, More Better, More Sophisticated Branching Sophisticated Branching TechniquesTechniques
•Voice Recognition of Tone, Voice Recognition of Tone, Not Just WordsNot Just Words