linking instructional objectives to the design of instructional environments: the integrative...

Human Resource Management Review 16 (2006) 376–395www.socscinet.com/bam/humres

Linking instructional objectives to the design of instructionalenvironments: The Integrative Training Design Matrix

Eric Anthony Day ⁎, Cassie Blair, Suzie Daniels, Vykinta Kligyte, Michael D. Mumford

Department of Psychology, The University of Oklahoma, 455 W. Lindsey, Room 705, Norman, OK, 73019-2007, USA

Abstract

The purpose of this article is to present a system, referred to as the Integrative Training Design Matrix (ITDM), for streamliningthe design of instructional environments in a way that integrates the instructional design elements commonly discussed in thescientific literature. This system expands the use of job-analytic information to better link needs assessment to the design ofinstructional environments. Specifically, the ITDM shows how determining an instructional objective's standing along four trainingissues–extensiveness of training, location of training, nature of transfer performance, and learning difficulty–yields a compre-hensive plan for designing instructional environments. These plans involve design elements such as trainee-to-trainer ratio,feedback delivery, hands-on practice, modeling, structure of practice, amount of training, meaningfulness, pre- and post-traininginterventions, and general training methods. The ITDM then shows how organization and person analyses can be targeted tosupport these design elements.© 2006 Elsevier Inc. All rights reserved.

Keywords: Training; Instructional design; Instructional objectives; Training transfer; Needs assessment

Training plays a large role in organizations, and it iswell known that estimates of annual spending byU.S. organizationson training and development are in the billions. With economic trends showing decreases in the quantity and quality oflabor coupled with rapid technological advancements and the globalization ofmarkets, the importance of effective trainingand development programs will most assuredly increase. More bluntly put, training is big business.

The academic literature is predominantly grounded in a traditional model of systematic instructional design thatadmonishes practitioners to conduct needs assessment at the organization, job, and person levels. Nevertheless, thistraditional approach has been criticized for its linear structure and cumbersome processes. The crux of these criticisms isthat a traditional model of systematic instructional design is not responsive enough to today's fast-moving business world.Indeed, considering the trends in market globalization and technological advancements, customer demands are becomingincreasingly dynamic and in turn there are growing pressures for organizations to become more responsive to the needs ofcustomers. Although there is truth to the criticism that the traditional model of systematic instructional design is overlylinear and unnecessarily cumbersome, this does notmean that such criticismwarrants completely abandoning the approachoffered by the traditional model of systematic instructional design. In other words, it is important to adhere to the

⁎ Corresponding author. Tel.: +1 405 325 3237; fax: +1 405 325 4737.E-mail address: [email protected] (E.A. Day).

1053-4822/$ - see front matter © 2006 Elsevier Inc. All rights reserved.doi:10.1016/j.hrmr.2006.05.007

mailto:[email protected]

http://dx.doi.org/10.1016/j.hrmr.2006.05.007

377E.A. Day et al. / Human Resource Management Review 16 (2006) 376–395

fundamental tenet that needs assessment (i.e., front end analysis) is crucial to the effectiveness of training and developmentprograms; however, the approach to implementing needs assessment needs to be streamlined to be more compatible withthe pace of today's business world. This article presents an example of a more streamlined approach, and this approach isbased on expanding the use of job-analytic information to better integrate organization-, task-, and person-levelinformation in the design of instructional environments.

Within the existing training literature, there are numerous reviews of the training design elements that affect thelearning process and the transfer of learning to the job environment. Design elements include such things as trainee-to-trainer ratio, feedback delivery, hands-on practice, observational learning, structure of practice (e.g., distributed ratherthan massed), amount of time devoted to training, building meaningfulness, pre- and post-training interventions, andgeneral training methods. From an applied perspective, these reviews are incomplete in three respects. First, althoughthey provide recommendations as to how each design element can be optimized in training, these reviews fall short ofproviding a set of recommendations that take into account how the design elements should be integrated. Second, thereviews provide very little guidance on how these recommendations are specifically spawned from information garneredin the needs assessment process. Third, the reviews primarily focus on optimizing the instructional environment in termsof effectiveness, yet they tend to ignore issues of efficiency. That is, recommendations are often based on how designelements can be structured to increase the likelihood that trainees will acquire full proficiency, but they rarely addresshow design elements can be composed to efficiently bring trainees to levels of satisfactory proficiency. Yes, cost (orreturn on investment) is discussed in the training literature, but the focus of such discussions tends to be at a broader,training program or training technique level. Cost is often lost in recommendations for designing the actual instructionalenvironment. While one can find exceptions to this statement (e.g., Arthur, Day, Bennett, McNelly, & Jordan, 1997), theexceptions are few. Because the literature falls short in these three ways, it is difficult for the training practitioner to usethe scientific literature to develop a comprehensive training design system that can be swiftly and effectively appliedacross a full range of training needs.

Thus, the purpose of this article is to present a system for designing instructional environments that integrates theinstructional design elements commonly discussed in the scientific literature. This system, referred to as the IntegrativeTraining Design Matrix (ITDM), provides recommendations that are comprehensive, yet balanced with respect toeffectiveness and efficiency. The system essentially helps training practitioners use job-analytic information to betterlink needs assessment and instructional objectives to the design of training settings. Specifically, the system yields acomprehensive list of design recommendations based on a relatively short series of questions about the nature of aninstructional objective and its relation to performance in the job environment. Although the ITDM model is usefulacross organizational contexts, the impetus for developing the ITDM was to provide training practitioners from largeorganizations, whose training and development needs are widely varied, with a streamlined system for designinginstructional environments. It is our hope that the ITDM will have immediate prescriptive value for practitioners.Moreover, we hope that our presentation of the ITDM stimulates in the academic community more integrative thinkingthat is better connected to real-world training concerns.

1. An overview of the ITDM system

As previously stated, traditional models of systematic instructional design emphasize the role of conducting needsassessment at the organization, job, and person levels. The purpose of an organizational analysis is to make sure thattraining programs meet the needs of the organization and that the broader job and organization contexts are supportiveof the training programs. Training programs do not exist in a vacuum. They are very much a part of a largerorganizational system. An organizational analysis helps align a training program with the goals of the largerorganizational system and coordinates training with other subsystems of the organization. The purpose of a job analysisis to identify the tasks and responsibilities associated with a job as well as the knowledge, skills, abilities, and othercharacteristics (KSAOs) needed to effectively perform those tasks and responsibilities. The purpose of a personanalysis is to ensure the content and design of training are matched with the (KSAOs) of the trainees. Matching trainingto trainees is accomplished in several ways. First, a person analysis provides information regarding incumbents' or newhires' deficiencies in job-related knowledges and skills (and attitudes). Person analysis thus helps to ensure that trainingis meeting the job-related needs of the trainees. Second, it is also important that a person analysis be used to identify theextent to which trainees have the prerequisite KSAOs that are needed to learn the material covered in training as well asto transfer what is to be learned in training to the job environment. Third, a person analysis can also help identify any

378 E.A. Day et al. / Human Resource Management Review 16 (2006) 376–395

special interventions that could be utilized to better tailor the instructional environment to meet the needs of theindividual trainee.

Ideally, instructional objectives for training are derived from a combination of thorough organization, job, and personanalyses. In doing so, organization and person analyses precede the design of training settings in traditional models ofsystematic instructional design. However, addressing all the areas in which a needs assessment, particularly organizationand person analysis, could inform the design of instructional environments before developing an initial plan for theinstructional environment is an unwieldy process. This is certainly prudent with respect to issues such as determining theneed for training, garnering support for training, and identifying the persons in need of training. However, one of theassumptions of the ITDM is that in many ways it is more feasible to use organization and person analyses while planningthe design of the instructional setting. What is essentially advocated in the ITDM process is an approach to trainingdesign that involves a more comprehensive use of job- and task-based information in the initial design of a trainingprogram followed by targeted support from organization and person analyses. By using organization and person analysismore concurrently with the design of instructional environments, the training design process makes use of job-relatedanalyses and information in a more streamlined and targeted fashion.

Given an instructional objective (or set of closely interrelated instructional objectives), implementation of the ITDMsystem involves three general steps. The first step is examining how an instructional objective relates to four training issuesthat involve the learning and performance of the tasks associated with the instructional objective. These four issues arelearning difficulty, nature of transfer performance, location of training, and extensiveness of training. These issues areaddressed by obtaining ratings from subject matter experts (SMEs) that include and extend the typical task ratings–importance and frequency–that are obtained during the job analysis phase of a needs assessment. These ratings includemore detailed information about the tasks associated with an instructional objective as well as information about the jobcontext inwhich these tasks are performed and information about the expertise of the trainees. To a small degree, the ITDMsystem incorporates organization analysis up front in determining the location of training and the nature of transferperformance. At this point, the ITDM system also incorporates some aspects of a person analysis in determining theextensiveness of training by addressing the level of related expertise trainees will bring to training as well as the targetedlevel of expertise that trainees are to reach by the end of training. Obtaining these ratings greatly expands the role of job-analysis in the training design process and initially facilitates linking organization and person analyses to the design of theinstructional environment.

The second step is planning the design of the instructional environment. This step involves examining how the decisionsmade in the first step relate to each other in terms of instructional design. Specifically, design recommendations for a giveninstructional objective are provided by examining how the decisions regarding learning difficulty, transfer performance,location of training, and training extensiveness intersect in amatrix that yields six nonredundant cells—the actual ITDM (Fig.1). These recommendations are composed of a comprehensive set of instructional design elements. Although decisions abouteach issue alone have implications for instructional design, the ITDMshows how relating the decisions to each other providesa more comprehensive and integrated plan for designing instructional environments.

The third step involves determining how additional information about the trainees and organizational context can beused to support the instructional environment. Specifically, the ITDM provides recommendations for when additionalorganization and person analyses need to be conducted. In other words, the ITDM highlights the situations in whichextra resources are needed to produce successful training programs. In this manner the ITDM can be used tosimultaneously design instructional environments and customize organization and person analyses to meet the specificneeds of a training program.

In the sections below, we describe the steps to the ITDM in more detail. First, we review each of the four key trainingissues. Table 1 provides a brief review of the four training issues. We then present the actual ITDM, which shows howrelating the key training issues to each other yields comprehensive plans for designing instructional environments. Asstated previously, the ITDM is presented in Fig. 1. We then describe how the ITDM helps link organization and personanalyses to the design of instructional environments. Table 2 provides a brief description as to how organization andpersonal analyses can support specific instructional design elements.

2. Key training issues

The underlying structure to the ITDM is based on the assumption that there are basically four key issues that needto be addressed when designing the instructional environment for a given instructional objective. We also contend

Fig. 1. The Integrative Training Design Matrix. Contents reflect recommendations for instructional design. Boxes outlined with a dashed line indicate situations when the design of the instructionalenvironment especially needs support from organization and person analyses (see Table 2).

379E.A.Day

etal.

/Hum

anResource

Managem

entReview

16(2006)

376–395


that these key issues operate in a highly interactive fashion with respect to instructional design. As previouslymentioned, these key issues are the difficulty to learn the material in training, nature of transfer performance, locationof training, and extensiveness of training. Although other issues may also influence training, we believe thatconsidering these four issues together provides a fairly comprehensive basis for designing instructional environments.However, the ITDM system also assumes that the primary goal of training is to augment and enhance the knowledgeand skills needed to perform the tasks and responsibilities of a job. Other issues may affect the design of trainingwhen performance-related knowledges and skills are not the target of training. For example, the primary purpose oftraining may be to reward employees for good performance or to increase employee morale. Such is often the casewhen training involves sending employees on extended trips or retreats to exciting and perhaps even exotic locations.In other cases, training programs may be used for promoting awareness of a controversial issue (e.g., sexualharassment, affirmative action) and to show employees (and the public) the organization's concern for controversialissues (Salas, Cannon-Bowers, Rhodenizer, & Bowers, 1999). In both of these cases, the issues influencing trainingdo not relate to augmenting or enhancing performance-related knowledges and skills. Therefore, the ITDM systemdoes not apply to such cases.

2.1. Learning difficulty

Learning difficulty refers to the likelihood that errors will occur while learning to perform the tasks associatedwith an instructional objective. Addressing learning difficulty is important to building instructional environments that

Table 1General training issues that drive the determination of instructional design elements

Issue Definition Importance Questions addressed

1. Learning difficulty The frequency in whicherrors occur in thelearning process

To ensure timelyacquisition

• What is the likely frequency of errors while practicing thetasks associated with the instructional objective?

• How many subtasks or procedural steps are there typicallyin performing the tasks associated with the instructionalobjective?

• To what extent is the execution of the instructionalobjective comprised of multiple interdependent parts?

• How much practice is required to learn the tasks associatedwith the instructional objective?

2. Nature oftransfer performance

The extent to whichperformance needs to begeneralizable and resistantto decay

To ensure learningreflects the performancedemands of thejob setting

• To what extent will the individual have to adapt theperformance of the tasks associated with theinstructional objective to a variety of environments andcircumstances?

• How frequently will the individual be performing the tasksassociated with the instructional objective?

• To what extent will the individual have to perform the tasksassociated with the instructional objective in high stresscircumstances?

3. Location The location of training in termsof on or off the job setting

To minimize costlyerrors and maximizetransfer to the jobsetting

• How significant are the consequences of performanceerrors?

• While on the job and performing other job duties, howdifficult would it be to learn the tasks associated with theinstructional objective?

• How important is the job setting to learning the properway to perform the tasks associated with theinstructional objective?

4. Extensiveness The depth of training required To ensure traineesacquire appropriatelevels of proficiency

• What level of expertise do individuals bring into training?• What is the target level of expertise upon completion oftraining?

• How critical to the job are the tasks associated with theinstructional objective?

Table 2Linking organization and person analysis to the design of instructional environments via the Integrative Training Design Matrix

Type of analysis

Organization Person

Matrix cell Nature of support Rationale Nature of support Rationale

1. Feedbackand trainee-to-trainer ratio

• Invest in selecting, training,and rewarding trainers whowill be able to adjust to theindividual needs of traineesboth in terms of learningactivities and in deliveringfeedback

• Trainingeffectiveness is heavilydependent on livetrainers' understandingof how to balanceknowledge andskill acquisition with thedevelopment of adaptableknowledge and skill

• Assess trainee individualdifferences in ability,motivation, andlearning styles

• To ensure that trainers areaware of and able to adjusttheir feedback delivery anduse of learning activitiesto the needs of individualtrainees

• Training effectivenessis heavily dependent onmeeting the special needsof individual trainees

2. Vicariouslearning

• Include organization- andjob-context cues andconsequences in thevicarious experiences

• To enhance the relevanceof and identification withthe vicarious experiencesand consequently enhancefocus and learning in training

•Match model demographiccharacteristics to traineecharacteristics

• To enhance traineeidentification with modelsand consequently enhancefocus and learning intraining

3. Hands-onpractice

• Provide opportunities toperform what was learned intraining back on the job

• To increase the likelihoodthat knowledge and skillsare continually developedafter training

• Use ability and trainabilitytests to select trainees

• To ensure that traineesare capable of meeting thedemands of the trainingcontent

4. Meaningfulness • Invest in selecting, training,and rewarding trainers whowill have communication andmotivation skills

• Embed the transfer climate ina larger continuous learningculture that inspiresemployees to continuallydevelop themselves bysolving problems creatively,taking risks, and sharinginformation and experienceswith others

• Training effectiveness isheavily dependent on the skillof live trainers in impartingdepth of understanding andinspiring trainees to usewhat is learned in trainingback on the job

• To foster the developmentof adaptable knowledgeand skill sets

• Assess trainee individualdifferences related topersistence such asself-efficacy, internal locusof control, emotional stabi-lity, andmastery orientation

• To ensure that trainees aremotivated to transfer whatthey learned in training tothe job environment○ Through selection○ Through special

interventions

5. Time andstructure ofpractice

• Invest in selecting, training,and rewarding trainers whowill have skill in instructingand designing practiceconditions

• Training effectiveness isheavily dependent on livetrainers' understanding ofhow to balance trainingefficiency with thedevelopment of adaptableknowledge and skill

• Assess trainee individualdifferences related topersistencesuch as self-efficacy,internal locus of control,emotional stability, andmastery orientation

• To ensure that traineesare capable of adjusting tothe difficult and frustratingdemands of the practiceconditions○ Through selection○ Through special

interventions

(continued on next page)


Table 2 (continued)

Type of analysis

Organization Person

Matrix cell Nature of support Rationale Nature of support Rationale

6. Methods • Align performancemanagement and rewardsystems (formal andinformal) with the goals oftraining

• Foster an organizationalculture that is perceived asjust and supportive andspecifically as one that takesseriously the careerdevelopment of its employees

• To maximize andprotect the organization'sheavy investment in training○ To ensure transfer○ To ensure a long-term

retention of employeesafter training

• Assess traineeorganizational commitment,job involvement, and careercommitment

• To maximize andprotect the organization'sheavy investment intraining

• To help identify factorsin the broader job andorganizationalenvironments that couldundermine trainees' pre-and post-trainingmotivation

• To gain assurance thatemployees are motivatedto make the most out oftheir training opportunitiesfor both their own benefitand the benefit of theircompany


will better facilitate the speedy attainment of an instructional objective. That is, the timely acquisition of theknowledges and skills to be learned in training is better ensured when learning difficulty is explicitly addressedduring the needs assessment phase in the instructional design process. Although models of task complexity exist(e.g., Wood, 1986) and could be used to gather task-analytic information, we believe that framing instructionalobjectives in terms of learning difficulty captures issues of complexity more intuitively and likewise provides aninstructional design system that is more feasible to implement. Nevertheless, questions pertaining to complexity canbe used when addressing the difficulty to learn instructional objectives. Accordingly, learning difficulty can beaddressed by asking questions that refer to the likely frequency of errors while practicing the tasks associated with aninstructional objective and how much practice is required to learn the tasks associated with an instructional objective.Additionally, such questions can be supplemented by questions that more directly address complexity such as thosereferring to the number of procedural steps involved in performing the tasks associated with the instructionalobjective or the extent to which the execution of the instructional objective is composed of multiple, interrelated parts(Naylor & Briggs, 1963).

By addressing learning difficulty early in the instructional design process, both the effectiveness and efficiency oftraining can be better ensured. When instructional objectives are difficult to learn, it is important to devote more timeand resources to training. Otherwise learning at the levels required is unlikely to occur. On the other hand, wheninstructional objectives are easy to learn, less time and fewer resources are needed for trainees to acquire the requisitelevels of knowledge and skill. In short, learning difficulty is an issue that should be addressed not only to facilitate theeffectiveness of training in terms of the acquisition of knowledges and skills but also in terms of cost.

2.2. Nature of transfer performance

Transfer of training refers to the extent which trainees apply what was learned in training to their job settings(Goldstein & Ford, 2001; Wexley & Latham, 2002). The nature of transfer performance refers to the degree to whichthe performance of the tasks associated with an instructional objective needs to be resistant to decay and generalizableacross a variety of circumstances (i.e., stimulus-response patterns). The issue of transfer essentially comes down to theneed for adaptable performance and ensuring that what is learned in training accurately reflects the performancedemands imposed by the job setting (Goldstein & Ford, 2001). Recent research indicates that there is more to learningthan simply the acquisition of knowledge and skill. Likewise, tests of knowledge and skill at the end of training oftendo not adequately reflect learning and are insufficient in examinations of training effectiveness (Alliger, Tannenbaum,Bennett, Traver, & Shotland, 1997; Schmidt & Björk, 1992). Rather, performances later in time and under novel


circumstances may be better indicators of learning and more accurately represent the target outcomes of instruction.Accordingly, the complexity of learning as a construct and the need to emphasize adaptability to the demands of thejob setting have been increasingly recognized in the scientific literature on training (Goldstein & Ford, 2001;Kozlowski et al., 2001; Kraiger, Ford, & Salas, 1993).

High adaptability is needed when (a) there is an infrequent use of the knowledges and skills learned in training on thejob, (b) the knowledges and skills learned in training need to be executed across a variety of circumstances on the job (i.e.,generalizability), and (c) there are times when the knowledges and skills need to be executed under extreme duress.Adaptability is thus a combination of both retention (i.e., resistant to decay) and generalizability. With respect togeneralizability, adaptability is especially important when the job environment is so dynamic that individuals will need torecognize the relevance of the knowledges and skills they learned in training across a variety of circumstances as well asalter the execution of their knowledges and skills in innovative ways to meet the challenges imposed by novelcircumstances.

When the knowledges and skills learned in training are used frequently across a limited set of circumstances thatrarely involve high levels of stress, the nature of the transfer performance is considered stable. Thus, adaptability is nota concern. For instance, in some situations the job setting will require that what was learned in training be used on sucha frequent basis that decay is not even an issue. Similarly, generalization is also not an issue when the job setting isfairly constant and the possible circumstances that could arise are easily anticipated.

2.3. Location of training

The location of training refers to whether the training will take place on or off the job site. The distinctionbetween on- and off-site training is common in the literature, and texts frequently devote separate sections to each(e.g., Landy & Conte, 2004; Muchinsky, 2003; Wexley & Latham, 2002). In terms of learning and performance,determining the location of training is important for two reasons. The first reason is to minimize the cost ofperformance errors, and the second reason is to maximize the extent to which the knowledges and skills acquired intraining transfer to the job setting (Wexley & Latham, 2002). If the consequences of performance errors aresignificant, then off-site training is typically recommended in order to minimize the likelihood of performanceerrors and the damage caused by performance errors. When considering transfer to the job setting, trainingdesigners should determine the extent to which the job setting is important to learning the proper way to performthe tasks associated with the instructional objectives. If there are aspects of the job setting that provide critical cuesfor the proper execution of tasks and these aspects are difficult to simulate, then on-site training is recommended.However, it is also important to recognize that some jobs are so complex or involve such high levels of stress, it isdifficult to learn new knowledges and skills while on the job performing duties unrelated to the instructionalobjectives. Although training is taking place in the transfer environment, transfer will be undermined by thedifficulty of learning the instructional objectives imposed by the job setting. Therefore, conducting off-site trainingis recommended in these situations.

2.4. Extensiveness of training

Extensiveness of training refers to the depth of training required in order to ensure that trainees acquire appropriatelevels of knowledge and skill proficiency by the end of training. When addressing extensiveness, there are severalquestions that need to be answered covering both job- and person-analytic information. From a job analysisperspective, it is fundamental to determine the criticality of the tasks associated with the instructional objectives(Goldstein & Ford, 2001; Wexley & Latham, 2002). It is also important to determine the level of expertise that isneeded at the end of training in order to properly meet the job demands of the tasks associated with the instructionalobjectives. Ignoring the target level of expertise will frequently result in training programs that do not provide theexperiences necessary to develop the knowledges and skills needed to perform the job successfully (May &Kahnweiler, 2000). It iscommonly recognized that learning occurs in stages (Anderson, 1987, 1996), and a significantamount of resources must be devoted to training for individuals to advance to later stages of learning and acquire highlevels of expertise (Ericsson & Charness, 1994). As such, the more critical the tasks and the higher the level of expertiserequired upon the completion of training, the more extensive the training needs to be. However, the degree to whichextensiveness is determined by task criticality and needed expertise can be qualified by other factors, and perhaps the


most obvious qualifying factor is the level of expertise the trainees bring to the training program. The more expertisebrought to training, the less extensive the training needs to be.

2.5. Making decisions about an instructional objective's standing on the four training issues

In streamlining the design of instructional environments, the ITDM presents the four training issues as dichotomies:easy or difficult learning difficulty, stable or adaptable transfer performance, on- or off-site location, and low or highextensiveness. In determining an instructional objective's standing on these issues, SMEs answer a few questions perissue as shown in Table 1. To expedite the process, it is recommended that the ratings be provided by a small set (two tothree) of SMEs and that the ratings are averaged within each issue using equal weights. We recommend answering thequestions using five-point Likert scales. Scales should be anchored such that higher (lower) scores reflect difficult(easy) learning difficulty, adaptable (stable) transfer performance, off-site (on-site) location, and high (low)extensiveness. In this manner, higher scores reflect a higher degree of resources needed for training. The middle of eachscale should be anchored using neutral language. Thus, higher than average scores across SMEs and questions forlearning difficulty mean that an instructional objective has a high degree of learning difficulty. Likewise, a high averagescore for nature of transfer performance means that an instructional objective requires adaptable performance, a highaverage score for location means that an instructional objective should be trained off-site, and a high average scoremeans that an instructional objective requires a high degree of training extensiveness. Lower than average scores yieldopposite conclusions. In cases when average scores equal the neutral point of the scale, we recommend that a decisionreflecting lower than average scores be made for the sake of feasibility and cost effectiveness. After decisions havebeen made regarding an instructional objective's standing on every issue, then the training designer can use the ITDMto make decisions regarding the design of the instructional environment.

3. Instructional design elements and the ITDM

After addressing an instructional objective's standing on each of the four key training issues, instructional designrecommendations for that objective are provided by examining how the four issues intersect with each other in a matrixformat—the ITDM (Fig. 1). To help streamline the system, the training issues are dichotomized and the result is amatrix comprised of six cells, which in turn are each comprised of four boxes. Each cell represents a different designelement (or small set of elements), and each box represents a different set of design recommendations for each designelement given an instructional objective's standing on the pair of issues intersected in the matrix. In upcoming sections,we provide more details about the cells comprising the ITDM. Through this process, the training designer is providedwith a set of integrated design recommendations for a given instructional objective in a manner that is explicitly basedon job-analytic information. These recommendations are specific enough to provide guidance to training designerswhile at the same time broad enough to allow training designers to exercise their own professional judgment whenmaking final decisions about designing the instructional environment. It is important to note that this matrix reflects ourproposition that the four key training issues are relatively independent from one another and influence instructionaldesign in a highly interactive fashion. In other words, we contend that the optimal design of instructional environmentscannot be accomplished by addressing learning difficulty, nature of transfer performance, location of training, andextensiveness one at a time without considering how they jointly influence design requirements. Given this interactiveperspective, we believe that effective decisions about instructional design cannot be made by addressing the keytraining issues in a predominantly sequential fashion.

As an overview of how the ITDM works, consider the following contrasting examples while examining Fig. 1. Oneinstructional objective has a high learning difficulty, requires adaptable performance, should be trained off-site, andrequires a high degree of extensiveness. A second instructional objective has an easy learning difficulty, requiresstable performance, can be trained on-site, and requires a low degree of extensiveness. Overall, the first objective willrequire substantially more resources to train. With respect to feedback and trainee-to-trainer ratio (Cell 1), the ITDMyields the following recommendations for the first instructional objective: a small trainee-to-trainer ratio; a highpercentage of personalized feedback from a live trainer; a gradual transition from immediate, positive, and specificfeedback to more delayed, negative, and less specific feedback; and a need to adjust learning activities to the needs ofthe individual trainee. For the second instructional objective, efficient yet effective training would involve a largetrainee-to-trainer ratio with a low degree of personal feedback from a live trainer, feedback that is consistently


immediate, positive, and specific, and little need to adjust learning activities to the needs of the individual trainee.Regarding vicarious learning experiences (Cell 2), the first objective requires many rich examples and trainees wouldbe given the task of generating their own learning points. The second objective only requires one or two basicexperiences and learning points would be provided by the trainer. The first instructional objective requires that a largepercentage of training time be devoted to providing trainees with hands-on practice, whereas the second objectiverequires very little hands-on practice (Cell 3). To provide meaningfulness (Cell 4), the first objective requires a highdegree of elaborative rehearsal as well as a high need for both pre- and post-training activities. Buildingmeaningfulness is not required for the second objective, and therefore there is a minimal need for elaborative rehearsalas well as for pre- and post-training activities. The first objective requires variable and randomized practice sessionsthat are distributed over the course of several weeks or months of training (Cell 5). The second objective requires onlya few hours of training (at the most) held over a single session in which practice is massed, constant, and blocked.Finally, the first objective requires multiple training methods with an emphasis on off-site methods that provide acomprehensive coverage of the instructional objective with respect to the larger demands of the job and organization(Cell 6). The use of behavioral simulations and computer-adaptive training may need to be used in conjunction withclassroom and self-directed learning as well as with a degree of on-site experiences. The second objective does notrequire the use of multiple methods, and training can take place on-site without being comprehensive with respect tothe larger demands of the job and organization. In such situations, on-the-job training with or without job aids wouldmost likely be effective. As another example, Fig. 2 specifically shows the recommendations provided by the ITDMfor an instructional objective that has an easy learning difficulty, requires adaptable performance, should be trainedoff-site, and requires low extensiveness. Overall, training for this instructional objective would require a moderatedegree of resources.

As shown in the ITDM, it is important to align different elements when designing instructional environments withthe goal of optimizing the learning of instructional objectives and the transfer of learning to performance settings. Inparticular, building instructional environments around the interplay between observational learning, direct learning,and feedback serves as the foundation for the ITDM. That is, the ITDM assumes that vicarious learning, active practice,and feedback are necessary ingredients to the success of virtually all training programs. Although this assumptionseems common sensical, the importance of building instructional environments around these three factors is rarelymade explicit in training recommendations. All too often specific techniques and general methodological approachescapture more attention, and this is evident in the pervasiveness of fads in the training community. Furthermore, thesethree factors are often discussed with a strong focus on one without much attention to the others. For instance, vicariouslearning experiences are sometimes only referred to within the context of behavioral role modeling and treated as anactual approach or method to training for special circumstances like leadership development or interpersonal skillstraining (cf. Goldstein & Ford, 2001). Indeed, we believe that applying principles of social learning theory, specificallyobservational learning and behavioral role modeling (Bandura, 1977, 1986, 1991), should be incorporated in themajority of training programs, considering that observation is both a natural and powerful means of learning. Similarly,learning is not likely to occur unless individuals are given opportunities for practice accompanied by feedback abouttheir progress. However, the relative degree and the manner in which these design elements are delivered will not beidentical across all training programs, and the ITDM provides recommendations as to how these elements shouldappropriately differ across an array of training needs. Specifically, the difficulty of the training content drives howfeedback, vicarious learning, and active practice should be differentially incorporated in training and this is reflected inthe first column of the ITDM (Cells 1, 2, and 3) as shown in Fig. 1. In this respect, the ITDM ensures that these threefundamental processes to adult learning are given due attention in the training design process and are not lost in thelarge number of training fads, tips for enhancing the learning process, and transfer interventions that are found in thetraining and development literature.

The other cells in the ITDM (4, 5, and 6) represent the importance of building instructional environments beyondthese three fundamental processes in order to better ensure both learning and transfer. In fact, while general trainingmethods are frequently the first item considered in training design, the ITDM implies that decisions regarding generaltraining methods (Cell 6) should be withheld until after decisions regarding the other five cells have beenmade. Trainingmethods should be viewed as the vehicles bywhich the other instructional elements that more directly influence learningand transfer are delivered. Together, the six cells of the ITDM provide a training design system that speaks to theimportance of examining instructional objectives in relation to the learning process as well as performance in the worksetting. Explicitly developing instructional environments with a simultaneous focus on learning and transfer better

Fig. 2. Design recommendations using the Integrative Training Design Matrix for an instructional objective that has an easy learning difficulty, requires adaptable performance, should be trained off-site, and requires low extensiveness.

386E.A.Day

etal.

/Hum

anResource

Managem

entReview

16(2006)

376–395


ensures that trainees acquire the knowledges and skills that will directly improve their performance on the job. We nowprovide more details about the recommendations given by the ITDM as presented in Fig. 1.

3.1. Cell 1: feedback and trainee-to-trainer ratio

The difficulty to learn a specified instructional objective and the nature of transfer required jointly determine bothhow feedback should be provided during training and the trainee-to-trainer ratio. Feedback is a fundamental part of thelearning process. The absence of feedback and the presence of inappropriate feedback are both strong impediments tothe learning process (Kluger & DeNisi, 1996, 1998). Feedback provides knowledge of results, and feedback serves as asource of motivation.

As reflected in the ITDM, the more difficult an instructional objective, the more important it is that proper feedbackis provided to the individual trainee, and this importance is accented when adaptable performance is needed.Accordingly, for proper feedback to be delivered to the individual trainee, smaller trainee-to-trainer ratios arerecommended, especially in terms of adjusting feedback to meet the needs of the individual learner. Smaller ratiosmake it easier for trainers to determine as well as adjust to individual trainee needs. Not only does feedback need to beadjusted consistent with the rate of acquisition and competency of the trainee (Ilgen, Fisher, & Taylor, 1979), it is alsoimportant that trainers use different learning activities according to individual differences in ability (Snow, 1986; Snow& Lohman, 1984) and learning styles (Inglis, 1994). For difficult instructional objectives and a need for developingadaptability, it is also important that personal feedback be provided by live trainers. In such cases, feedback isabsolutely critical to the development of skill proficiency, and it is therefore critical that trainees perceive the feedbackas accurate and coming from a credible source (Ilgen et al., 1979). Working closely with trainers who provide personalfeedback will facilitate the acceptance and use of feedback.

There is a lesser need for highly personalized feedback and flexibility in learning activities for easy instructionalobjectives and stable performance demands. In such situations, larger trainee-to-trainer ratios are more sensible (a singletrainer may suffice) and feedback delivery mechanisms other than a live trainer, written feedback from a paper- and-pencil activity for example, are viable. In situations involving difficult instructional objectives and stable performancedemands, moderate trainee-to-trainer ratios are appropriate and a variety of potentially worthwhile feedback deliverysystems should be available. The same is true for situations involving easy instructional objectives and adaptableperformance demands. In such circumstances, a moderate trainee-to-trainer ratio will allow the use of a combination oflearning activities that, in addition to the modest degree of feedback that the trainers could personally provide, yielddifferent sources of feedback including written and computer-based sources.

Feedback can differ in many respects, but in particular the immediacy, direction, and specificity of feedback havereceived the most attention in the literature. Feedback that is less specific, negative (i.e., critical), and delayed has beenshown to increase retention and generalizability of knowledge and skill (Goodman &Wood, 2004; Goodman, Wood, &Hendrickx, 2004; Schmidt & Björk, 1992; cf. Earley, Northcraft, Lee, & Lituchy, 1990). Thus, when trainingindividuals to develop adaptable performance, it is important to incorporate less specific, negative, and delayedfeedback rather than solely relying on highly specific, positive, and immediate (i.e., continuous) feedback. By delayingfeedback, providing less specific feedback, and providing negative feedback, trainees are induced to engage in self-regulation, elaboration, and reconstruction of behavioral responses. Consequently, trainees become less dependent onexternal sources of feedback. More simply put, delayed feedback, negative feedback, and less specific feedback causeindividuals to engage in problem-solving and cause-and-effect thinking. Inducing these cognitive processes betterreflects transfer situations because external sources of immediate feedback are not common in many transfer situations;therefore, engaging in these cognitive processes facilitates performance when adaptable performance is required.Moreover, in novel circumstances, any normal source of feedback that might be available is not likely to produceaccurate feedback. However, like distributed, variable, and random practice, delayed feedback, negative feedback, andless specific feedback retard knowledge and skill acquisition. Therefore, for easy instructional objectives requiringstable performance demands, providing immediate, positive, and specific feedback is appropriate. The same isrecommended for difficult instructional objectives requiring stable performance demands. However, in situationsrequiring adaptable performance demands, a mix of feedback types should be used to appropriately balance the needfor acquisition during practice and performance in the job setting. For example, in situations involving difficultinstructional objectives and adaptable performance demands, the optimal feedback system is likely to be one thatinitially involves immediate, positive, and specific feedback and then gradually provides feedback that is more delayed,


more critical, and less specific. In this manner, knowledge and skill acquisition are facilitated but not at the expense ofretention and generalizability. Similarly, a transition from immediate, positive, and specific feedback to delayed,critical, and less specific feedback is likely appropriate for situations involving easy instructional objectives yetadaptable performance demands. However, this transition in feedback delivery may be faster compared to the previoussituation because the instructional objectives are not difficult to learn.

One important lesson learned from research on feedback is that the effects of feedback are not as straightforward aswas once thought and in some circumstances feedback can produce negative effects. Current thinking on theeffectiveness of feedback suggests that feedback should be used in conjunction with goals (Kluger & DeNisi, 1996,1998). Without clear goals, it is difficult to ensure that any feedback provided will guide trainee attention in theappropriate direction. That is, incorporating specific goals, both distal and proximal, will help provide the adaptiveguidance needed to maximally promote acquisition and adaptable performance. One of the assumptions underlying theITDM is that more difficult instructional objectives and objectives that involve adaptable performance demands requiresmaller trainee-to-trainer ratios in order to ensure that the learning activities used and the goals and feedback providedmeet the needs of the individual trainee.

3.2. Cell 2: vicarious learning

The difficulty to learn a specified instructional objective and the location of training jointly determine the extent towhich time in training should be devoted to vicarious learning as well the specific features of the vicarious learningexperiences. Vicarious learning experiences, similarly referred to as observational learning and behavior modeling,involve learning by watching demonstrations of performance and imitating the demonstrations. The importance ofimitation to learning is best articulated in Bandura's social learning theory (1977, 1986, 1991). One of the key points toBandura's research is that demonstrations shown leading to positive outcomes strengthen the vicarious experience andserve as salient reminders of appropriate behavior. Moreover, the saliency of these reminders is accentuated when themodel is viewed as having high status and high similarity to the observer.

Why should recommendations concerning vicarious learning be yoked to the site of training and learning difficulty?The rationale is based more on the specific features of the vicarious learning experiences and not so much the proportionof time spent on vicarious learning. Specific features to vicarious learning involve the vividness of the demonstrations,the spectrum of experiences displayed, and the degree to which critical behaviors are highlighted; all of which refer to therichness of the vicarious experience. Difficult instructional objectives demand maximally effective learning conditions,and off-site training typically does not contain all the situational cues that prompt the performance of trained skills.Consequently, when training is off-site and the learning difficulty is high, it is important to develop vivid demonstrationsin which the critical behaviors are highlighted across the spectrum of performance variations for an instructionalobjective. The opposite is true when training is to be conducted on-site and learning difficulty is low. In such situations,multiple rich demonstrations are not necessary. In off-site training for instructional objectives low in difficulty and viceversa, moderately rich demonstrations are fitting. For easy instructional objectives, off-site training may require a littlemore than a single simple demonstration in order to ensure adequate comprehension.When training is conducted on-sitefor difficult instructional objectives, placing a high degree of effort on developing rich demonstrations is not completelynecessary because the situational cues prompting appropriate performance are naturally present, but the high level oflearning difficulty requires more assurance that performance errors are minimized and skill is acquired. Thus,moderately rich demonstrations are fitting for the on-site training of difficult instructional objectives.

Developing rich vicarious experiences includes (1) using multiple live demonstrations versus video, computersimulated, or paper demonstrations to ensure vividness, (2) modeling a range of positive as well as negative behaviorsto ensure a spectrum of experiences is displayed, and (3) articulating key learning points to ensure critical behaviors areproperly highlighted. Furthermore, more time should be devoted to vicarious learning experiences when richerexperiences are required. For difficult instructional objectives, rich vicarious experiences that show similar othersovercoming difficulties and achieving success will be especially important not only for conveying appropriatebehaviors but also for boosting self-efficacy (Bandura, 1982, 1986; Wood & Bandura, 1989). Although research hasindicated that highlighting key learning points is crucial to vicarious learning (Mann & Decker, 1984), the extent towhich key learning points are highlighted will differ as needed. For instance, because trainee-generated learning pointshave been shown to better facilitate learning compared to trainer-generated learning points (Hogan, Hakel, & Decker,1986), we would argue that training time should be devoted to allowing trainees to generate and share their learning


points when difficult instructional objectives are being trained off-site. To maximize learning in such cases, multiplelive demonstrations should be utilized when feasible or at least a combination of video and live demonstrations toprovide a spectrum of vicarious learning experiences that will likely facilitate transfer (Baldwin, 1992; Decker &Nathan, 1985; Goldstein & Sorcher, 1974). When richness is not important, multiple demonstrations are not required.When live demonstrations are not feasible, video or simulated demonstrations should be considered. Paperdemonstrations (i.e., case studies) are also a viable means of providing demonstrations for easy instructional objectives.Paper demonstrations are probably the most cost effective as well. In general, it is important to match the use ofvicarious learning experiences to the needs of the training program. Nevertheless, vicarious learning should be builtinto all training programs due to the fact that learning through observation is a natural form of learning that can yieldprofound results often times with little expense, even for complex skills (Arthur et al., 1997).

3.3. Cell 3: hands-on practice

The difficulty to learn a specified instructional objective and the extensiveness of training required jointly determinethe extent to which time in training should be devoted to hands-on practice. It is generally considered necessary thattrainees be given opportunities to actively practice what is to be learned during training, whether the instructionalobjective is predominantly physical or cognitive in nature (Goldstein & Ford, 2001; Wexley & Latham, 2002). Activepractice, sometimes referred to as the production of a response or direct learning, is what we refer to as hands-onpractice. We prefer to use this nomenclature because we believe it better connotes the importance of affording eachindividual trainee the opportunity to personally engage in the performance of tasks that represent the given instructionalobjective.

The rationale for hands-on practice is fairly straightforward. The more important it is that trainees reach the requiredproficiency levels at the conclusion of training, the more important it is that trainees are given adequate time for hands-on practice. Active practice facilitates knowledge and skill acquisition, particularly in terms of transitioning fromacquiring knowledge to the application of knowledge and more complex problem solving (Anderson, 1987, 1996).Unfortunately, most training programs do not afford trainees with the opportunities to fully develop their knowledgeand skills (May & Kahnweiler, 2000). Within the ITDM, however, the need for hands-on practice is made moreprominent and is shown to be exacerbated by the difficulty of the training content. With increased difficulty comes anincreased likelihood of errors. Difficult instructional objectives tend also to be associated with tasks that have manyprocedural steps with the performance of later steps dependent upon the successful performance of preceding steps.Performance errors are thus more likely to occur when learning difficult tasks. Furthermore, performance errors aftertraining are least tolerated when high levels of proficiency are demanded. With this in mind, it is imperative that higherproportions of training time are devoted to hands-on practice when extensiveness and difficulty are both high. Not onlydoes hands-on practice help foster appropriate behaviors, it also minimizes the development of inappropriate behaviorsand harmful errors in the post-training environment.

Within the ITDM, hands-on practice is operationalized in terms of the proportion of time in training devoted tohands-on practice. Specifically, the need for hands-on practice is minimal when extensiveness is low and theinstructional objectives are relatively simple to learn. In these cases, only a small proportion of time in training (e.g.,less than 20%) needs to be devoted to hands-on practice. When either extensiveness or learning difficulty is high andthe other is low, then moderate amounts (e.g., between 20% and 39%) of hands-on practice are recommended. Whenboth conditions are high, a large amount of training time (e.g., 50%), and in many of these cases the vast majority of thetime, should be devoted to hands-on practice.

3.4. Cell 4: meaningfulness

The nature of transfer required and the location of training jointly determine howmeaningfulness of training should bepromoted. Meaningfulness frequently refers to the extent to which training facilitates associations among different piecesof information within training as well as with prior knowledge (Wexley & Latham, 2002). Meaningfulness translates intofostering elaborations of training content and depth of understanding (Craik & Lockhart, 1972). The ultimate goal ofmeaningfulness is to enhance trainees' understanding of how different pieces of information related to an instructionalobjective are both independently and interdependently important to performance. Thus, the perspective of meaningfulnessreflected in the ITDMmodel is that the purposefulness of an instructional objective in relation to the performance demands


is primarily an issue of transfer. Accordingly, buildingmeaningfulness is more critical to training that is conducted off-site,and this is especially truewhen the performance setting requires adaptable performance. By taking a transfer perspective tomeaningfulness, it is important to incorporate both cognitive and motivational approaches when building meaningfulnessin training. However, this does not mean necessarily that cognitive and motivational approaches are mutually exclusive.Indeed, interventions can be designed that jointly foster the understanding of training content, the motivation to learntraining content, and the motivation to use what was learned in training back on the job.

There are a number of cognitive approaches identified in the training literature that can be used to promotemeaningfulness during training, and several of the approaches that we describe have been described previously (e.g.,Wexley & Latham, 2002). An overview of the content at the start of training helps promote a general understanding of thetraining content which in turn helps trainees better understand how key concepts are related to the instructional objectives.There are manyways of providing an overview to trainees. Examples include discussing instructional objectives, outliningthe content of training, or illustrating and demonstratingwhat performance at the end of training should look like. Effectiveoverviews should provide trainees with a common theme connecting key concepts and components of an instructionalobjective together. Both at the beginning and continually throughout training, it is important to describe concepts usinglanguage familiar to the trainees. Meaningfulness will additionally be facilitated by sequencing material in a logical order.Simpler material should be presented before complex material, and the context of an instructional objective should bepresented before practicing the tasks and subtasks associated with an instructional objective.

Training that is conducted on-site that involves stable performance demands will generally be more inherentlymeaningful for trainees. Nevertheless, all of the above examples are worthwhile for both off- and on-site training aswell as for instructional objectives involving either stable or adaptable performance demands. Additional techniquesfor building meaningfulness will be needed when training concerns instructional objectives that involve adaptableperformance demands; techniques that foster greater elaboration and deeper understanding of material are particularlycritical. In this vein, meaningfulness should be promoted by using multiple examples that reflect the full range ofperformance situations associated with an instructional objective. Examples familiar to the trainees should be used atfirst followed by more novel examples. It is also important that the situational examples be specific to the job. To betterensure that trainees understand the principles underlying various situations, techniques such as analogical reasoning(i.e., comparing and contrasting; Thompson, Gentner, & Loewenstein, 2000) and error training (i.e., using mistakesexplicitly to teach key learning points; Dormann & Frese, 1994; Frese et al., 1991) should be used to help trainees learnhow the underlying principles can be adapted across different situations. Fostering a deeper understanding of materialscan also be facilitated by representing more abstract concepts using visual aids. For example, presenting diagrams ofmental models, the inter-relatedness of concepts and components comprising an instructional objective, will promotemeaningfulness by highlighting important concepts and helping trainees assimilate the key components of instructionalobjectives with prior knowledge and new information (Goldstein & Ford, 2001). Presenting diagrams of mental modelscan be done at the beginning of training to facilitate general understanding as well as later in training to help clarify anymisunderstanding of the training material. Lastly, using a combination of the above techniques in creative ways canfurther promote elaborative rehearsal and depth of understanding. For instance, relating mental models to a variety ofsituational examples could be a powerful technique for promoting meaningfulness.

Pre- and post-training interventions are other mechanisms for promoting meaningfulness, particularly from amotivational perspective. In other words, in many training situations simply providing trainees with information aboutthe content of the training before training will more than likely not have a profound effect on trainees' motivation tolearn or motivation to use the training on the job. For instance, incorporating more intensive pre- and post-traininginterventions is especially worthwhile when training is conducted off-site, and the need for such interventions isstronger when the instructional objectives involve adaptable performance demands. The use of pre- and post-traininginterventions including applications of work motivation theories to training and development has been discussed moreextensively by others with many specific and worthwhile examples (e.g., Goldstein & Ford, 2001; Wexley & Latham,2002). However, the differential need for these interventions has not been explicitly identified previously in theliterature, and the ITDM fills this void by explicitly distinguishing cases of greater need for such interventions.

Training conducted off-site will typically have a greater need for pre- and post-training interventions because traineeswill generally perceive the training as less relevant compared to training that occurs on-site. Furthermore, it will bedifficult for trainees to fully grasp the extent to which training is applicable across a range of situations and problemswhen off-site training involves instructional objectives that require adaptable performance. For these cases a moreintensive use of pre- and post-training interventions is recommended in the ITDM. Although a number of theories could


be used in conjunction, it is specifically recommended that principles from goal setting theory (Locke & Latham, 1990)and expectancy theory (Vroom, 1964) be used for training programs that are conducted off-site and principles fromsocial learning theory (Bandura, 1977, 1986, Wood & Bandura, 1989) be used for instructional objectives that requireadaptable performance demands. It is important to note that conducting an organization analysis is paramount tobuilding pre- and post-training interventions, because such interventions require substantial support from supervisorsand co-workers both in terms of development and implementation. Not only is it important to show trainees how thecontent of training is linked to the performance management and compensation systems, it is important to create a workclimate that supports training and provides trainees with opportunities to performwhat they learned in training on the job(Ford, Quiñones, Sego, & Sorra, 1992; Rouiller & Goldstein, 1993).

With respect to off-site training, pre- and post-training interventions based on goal and expectancy theories can be usedto build meaningfulness by increasing trainees' motivation to learn and use their training back on the job. These kinds ofinterventions will help trainees understand the importance of training, thus ultimately facilitating transfer. For adaptableperformance demands, applications of social learning theory are needed in order to build trainees' self-efficacy in adaptingwhat they learned to a variety of circumstances that are both novel and stressful. Skill retention and generalization can befacilitated through elaborative rehearsal strategies as previously discussed and by more effortful practice conditions likethose discussed below concerning Cell 5. However, ignoring the motivational factors to skill retention and generalizationcould undermine these more behavioral and cognitive approaches. Indeed, building the motivation to adapt one'sknowledge and skills to novel and demanding situations is critical to increasing the retention and generalization ofknowledge and skills. Therefore, training that encourages cognitive as well as motivational self-regulation is needed fordeveloping adaptable performance (De Corte, 2003; Kozlowski et al., 2001). Thus, interventions based on social learning,especially those that build self-efficacy and mastery goals (Kozlowski, Gully, Brown, Salas, & Nason, 2001), areworthwhile when training for adaptable performance demands. Similarly, interventions consistent with building acontinuous learning culture (Senge, 1990; Tannenbaum, 1997; Tracey, Tannenbaum, & Kavanagh, 1995) will alsofacilitate adaptable performance. For instance, bringing trainees together after training to share and discuss their post-training experiences in relation towhat they learned in training can foster innovativeways to adapt the knowledge and skillinitially developed in training to better meet the demands of the job setting.

3.5. Cell 5: structure of practice

The nature of transfer required and the extensiveness of training jointly determine how training sessions will bestructured as well as the amount of time to be devoted to training specified instructional objectives. The structure ofpractice refers to several design features all of which have been linked to enhanced knowledge and skill acquisition,retention, and generalization: distributed versus massed practice sessions, variable versus constant practice, and randomversus blocked practice (Schmidt & Björk, 1992). The distinction between distributed versus massed is a matter ofdividing practice into several sessions with breaks in between versus having one continuous practice session. Variablepractice involves introducing a variety of stimulus-response patterns (i.e., variation in actual performance demands),whereas constant practice involves no variation in stimulus-response patterns. Random practice refers to training severaldifferent instructional objectives during the same practice session; in contrast, blocked practice involves training onlyone instructional objective per training session. We dichotomize these three design features simply for the sake ofproviding easy to understand definitions. In actuality, it is best to consider these design features along continuums withthe descriptions of each dichotomy representing opposite ends. These structural features promote more effortfulcognitive processes that are present in transfer and retention settings, namely the reconstruction, elaboration, and self-regulation of behavioral responses. Although they facilitate retention and generalization, these structural features tend toretard knowledge and skill acquisition.

For training situations involving stable performance needs coupled with low levels of extensiveness, the ITDMshows a set of recommendations comprising of massed, constant, and blocked practice with a relatively short period oftraining time, several hours at most. Conversely, adaptable performance needs coupled with high levels of extensivenessrequire training sessions that are distributed over weeks or even months and involve variable and random practice. Insituations requiring little extensiveness but adaptable performance, it is recommended that training involve several hoursup to several days of variable and random practice that is more massed than distributed. On the contrary, for situationsrequiring high extensiveness but stable performance, it is recommended that training involve several days or a fewweeks of constant and blocked practice.


In situations when it has been determined that adaptability is primarily a function of infrequent use on the job and theneed to perform under high duress, then overlearning (see Driskell, Willis, & Copper, 1992) should be addressed moreexplicitly when designing the instructional environment. Overlearning refers to performing tasks repeatedly well after theperformance of the tasks has been mastered. Overlearning strengthens stimulus-response connections to the point whereperformance becomes reflexive, which put another way means performance requires little use of executive cognitivefunctions. Consequently, when it has been determined that the performance of tasks associated with an instructionalobjective will be infrequent but in situations of high duress, then the time in training devoted to practice should beincreased. Doubling the time in training devoted to practice could very well be warranted.

One might think that the sheer amount of time devoted to training a specified instructional objective should bedetermined, at least partially, by learning difficulty. However, we believe the amount of time in training is more affectedby training individuals to successfully acquire knowledges and skills that are resistant to skill decay and generalizable.We believe this to be true because both overlearning and exposing trainees to a variety of stimulus-response patternsrequire more time in training. As previously stated, promoting retention and generalization also tends to retardacquisition, thus more time is recommended in order to simultaneously ensure acquisition, retention, andgeneralization. We would also argue that acquiring difficult to learn knowledges and skills is more dependent uponopportunities to observe appropriate behaviors, engage in hands-on practice, and receive individualized feedback,which in combination are akin to coaching and deliberate training practices associated with acquiring expertise incomplex skill domains (Ericsson & Charness, 1994). In other words, practice does not make perfect, but perfectpractice does. This philosophy is reflected in the ITDM by the training design features shown in the column for learningdifficulty.

3.6. Cell 6: training methods

The location of training and the extensiveness of training jointly determine the methods that are appropriate for theinstructional objectives. Methods are specific instructional techniques involved in training workers on job relevantinstructional objectives. Although methods are distinguishable from each other, they nevertheless are broad enough toallow a variety of learning activities and media to be used when training workers. That is, methods provide the generalframework for how training will be conducted. Depending on the specific instructional objective, logistical constraints,and individual savvy of trainers, different learning activities and media will be used within the framework of aparticular training method (for a comprehensive review of learning activities and media and how their use can betailored to the type of instructional objective, see Gagné & Medsker, 1996; Gagné, Wager, Golas, & Keller, 2005).While the other cells in the ITDM address specific instructional design elements, Cell 6 addresses the importance ofselecting appropriate training methods through which the specific instructional elements would be operating. In otherwords, as the other cells address elements that directly have an impact on learning and transfer, Cell 6 primarilyaddresses how these elements are functionally implemented.

Cell 6 of the ITDM helps the designer focus on the location of the training in relation to how comprehensive thetraining needs to be. By considering location and extensiveness together, decisions can be made regarding thesuitability of using different training methods either singly or in combination. For instance, when extensiveness is lowand training is to be conducted on-site, methods like on-the-job training (OJT) and job aids can be used, and it is likelyunnecessary to combine these methods with other methods. When training is to be conducted on-site for instructionalobjectives that require a high degree of extensiveness, methods like OJT and job aids should be combined with othermethods involving a larger variety of learning activities. In this manner, it is important to combine methods that providea comprehensive perspective to the instructional objective that no single method or pair of methods can provide. Bycomprehensive we mean the depth to which tasks associated with the instructional objective are learned in relation tothe broader knowledges and skills to be learned as well as in relation to other tasks and more general informationregarding the job and organization. To provide greater comprehensiveness using on-site training, OJT and job aids canbe combined with action learning, mentoring, coaching, job rotation, or vestibule training.

For off-site training with low extensiveness, classroom training or self-directed learning are appropriate either singlyor in combination. Self-directed learning refers to providing trainees with written, video, or computer-based materialsto study on their own. However, if training is to be conducted off-site for instructional objectives that require a highdegree of extensiveness, classroom training and self-directed learning should be combined with more intensivemethods like low-fidelity simulations, high-fidelity simulators, or computer-adaptive instruction.


Although the ITDM in Fig. 1 shows the off- and on-site distinction as non-overlapping classes of methods, we donot necessarily mean to say that off- and on-site methods cannot be used together. For example, if on-site training isdeemed more appropriate in general but it is also determined that the job setting is fairly important to learning theproper way to perform the tasks associated with the instructional objectives, then combining off-site methods with OJTwould probably be the optimal combination of training methods. Similarly, if trainees need to develop a high degree ofexpertise but it is also difficult to learn new knowledges and skills while on the job performing duties unrelated to theinstructional objectives, then methods like action learning, job rotation, and mentoring can be combined with moretraditional off-site methods like classroom training. Overall, the message communicated by the ITDM is to choose themethod or combination of methods that will best meet the needs of training in the most cost-effective manner. In doingso, the training designer should consider which method or combination of methods is best suited to implementing theinstructional design recommendations addressed by the other five cells of the ITDM.

4. Linking organization and person analyses to the design of instructional environments

After using the ITDM to help plan the design of instructional environments, the ITDM can also be used to helplink organization and person analyses to the design of instructional environments. The ways in which organizationand person analyses support the design of training will differ based on the design elements needed to make trainingeffective. Not all training programs are alike, and linking organization and person analyses to the design of trainingenvironments should therefore be matched to the specific needs of each training program. The ITDM helps clarifythese links by showing the circumstances when the design of instructional environments especially needs supportfrom organization and person analyses. These circumstances are shown in Fig. 1 by the boxes outlined with dashedlines. Basically, the bottom right-hand corner of each cell represents a design circumstance when support fromorganization and person analyses is especially needed. Table 2 provides a brief description of how organization andperson analyses can be linked to these circumstances to maximize training. The total number of these dashed boxesthat are identified for a training program also marks the overall need for organization and person analyses. The moredashed boxes identified, the more likely training will fail without careful attention to organization and personanalyses.

Concerning organization analysis, Table 2 highlights the importance of how support from the job environment isrequired for training to be successful. This support is frequently referred to in the training literature as the transferclimate (Rouiller & Goldstein, 1993; Tracey et al., 1995), which translates specifically into the degree to which trainingis supported socially by supervisors and peers and institutionally by the performance management and reward systems.Building a positive transfer climate is vital to training success, but the rationale for building positive transfer and thespecific ways in which positive transfer is built will differ depending on the particular training design needed. Withrespect to person analysis, Table 2 shows which KSAOs should be emphasized in trainability assessments. Trainabilityassessments are thus tailored to the needs of the training program and can be used for two purposes. One, the individualdifferences could be viewed in terms of pre-requisites and therefore be used to select the trainees that would get themost out of training. Two, the trainability assessments can be used to design training interventions to better support andmeet the needs of individual trainees.

5. Conclusions

The ITDM system presented in this article should be viewed as an initial step towards developing more integrativetraining design systems that are better connected to the fast-paced nature of the business world. Given that this is aninitial step and that this system deviates from traditional models of systematic instructional design, validation and furtherdevelopment of the ITDM system are warranted. While many of the recommendations stem from specific empiricalresearch, some of the recommendations are based more on professional experience and a broad understanding of thescientific literature. For example, recommendations regarding the operationalization of hands-on practice (Cell 3) interms of the percentage of total training time devoted to hands-on practice are not based on specific empirical findings.Therefore, validation of the ITDM is well warranted.

In addition to conducting validation studies, there are aspects to the ITDM that are in need of further development. Forexample, the present ITDMmodel presents recommendations based on a dichotomization of the four key training issues. Itcould be argued that dichotomization yields inappropriate conclusions when an instructional objective's standing on an


issue lies more in the middle rather than clearly on a side of an issue. However, it is important to recognize that the purposeof the ITDM is to serve as a decision-making aid for training designers to use rather than as a replacement for trainingdesigners. One can retranslate the matrix to represent issues as continuums rather than dichotomies, which in turn wouldyield a blend or compromise of design recommendations for each cell.

In addition, the design elements included in the model, although important to training effectiveness, do not fullycapture the conditions required for effective learning. The ITDM does not include how specific instructional activitiesand events should be structured, nor does the ITDM provide recommendations for how the design of instructionalenvironments should differ depending on the nature of the learning outcomes (e.g., verbal, motor, and attitudinal).Comprehensive details regarding the design of specific instructional activities and how they can be tailored to thedemands of different types of learning outcomes have been provided by others, Gagné and his colleagues (Gagné &Medsker, 1996; Gagné et al., 2005) in particular. Future work is needed to bridge the ITDM model with models ofinstructional activities and events.

In conclusion, the goal of this article was to present a streamlined system for designing instructional environments thatintegrates the instructional design elements commonly discussed in the scientific literature. This system expands the use ofjob-analytic information to better integrate needs assessment in the design of instructional environments and providestraining practitioners with training design recommendations that are balanced with respect to effectiveness and efficiency.Although evaluation and development of the ITDM are needed, we believe the ITDM can presently be applied to theprocess of designing instructional environments. We also hope the ITDM stimulates more integrative thinking abouttraining design. We believe this to be true not only in terms of how design elements can be packaged together to form thefoundation of instructional environments, but also in terms of how the use of job-analytic information can be expanded tobetter link organization-, task-, and person-level information in the design of instructional environments.

References

Alliger, G. M., Tannenbaum, S. I., Bennett Jr., W., Traver, H., & Shotland, A. (1997). A meta-analysis of relations among training criteria. PersonnelPsychology, 50, 341−358.

Anderson, J. R. (1987). Skill acquisition: Compilation of weak-method problem solutions. Psychological Review, 94, 192−210.Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American Psychologist, 51, 355−365.Arthur Jr., W., Day, E. A., Bennett Jr., W., McNelly, T. L., & Jordan, J. A. (1997). Dyadic versus individual-based training: Loss and re-acquisition of

a complex skill. Journal of Applied Psychology, 82, 783−791.Baldwin, T. P. (1992). Effects of alternative modeling strategies on outcomes of interpersonal skills training. Personnel Psychology, 77, 147−154.Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122−147.Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248−287.Craik, F. I., &Lockhart, R. S. (1972). Levels of processing:A framework formemory research. Journal of Verbal Learning andVerbal Behavior,11, 671−684.De Corte, E. (2003). Transfer as the productive use of acquired knowledge, skills, and motivations. Current Directions in Psychological Science, 12,

142−146.Decker, P. J., & Nathan, B. R. (1985). Behavioral modeling training: Principles and applications. New York: Praeger.Dormann, T., & Frese, M. (1994). Error training: Replication and the function of exploratory behavior. International Journal of Human–Computer

Interaction, 6, 365−372.Driskell, J. E., Willis, R. P., & Copper, C. (1992). Effect of overlearning on retention. Journal of Applied Psychology, 77, 615−622.Earley, P. C., Northcraft, G. B., Lee, C., & Lituchy, T. R. (1990). Impact of process and outcome feedback on the relation of goal setting to task

performance. Academy of Management Journal, 33, 87−105.Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725−747.Ford, J. K., Quiñones, M., Sego, D., & Sorra, J. S. (1992). Factors affecting the opportunity to perform trained tasks on the job. Personnel

Psychology, 45, 511−527.Frese, M., Brodbeck, F., Heinbokel, T., Mooser, C., Schleiffenbaum, E., & Theimann, P. (1991). Errors in training computer skills: On the positive

function of errors. Human Computer Interaction, 6, 77−93.Gagné, R. M., & Medsker, K. L. (1996). The conditions of learning: Training applications. Belmont, CA: Wadsworth.Gagné, R. M., Wager, W. W., Golas, K. C., & Keller, J. M. (2005). Principles of instructional design (5th ed.). Belmont, CA: Wadsworth.Goldstein, A. P., & Sorcher, M. (1974). Changing supervisor behavior. New York: Pergamon Press.Goldstein, I. L., & Ford, J. K. (2001). Training in organizations: Needs assessment, development, and evaluation (4th ed.). Belmont, CA:Wadsworth.Goodman, J. S., & Wood, R. E. (2004). Feedback specificity, learning opportunities, and learning. Journal of Applied Psychology, 89, 809−821.Goodman, J. S., Wood, R. E., & Hendrickx, M. (2004). Feedback specificity, exploration, and learning. Journal of Applied Psychology, 89, 248−262.Hogan, P. M., Hakel, M. D., & Decker, P. J. (1986). Effects of trainee-generated versus trainer-generated rule codes on generalization in behavior-

modeling training. Journal of Applied Psychology, 71, 469−473.


Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organizations. Journal of AppliedPsychology, 64, 349−371.

Inglis, S. (1994). Making the most out of action learning. Brookfield, VT: Gower Publishing Limited.Kluger, A. N., & DeNisi, A. (1996). Feedback interventions: Toward the understanding of a double-edged sword. Current Directions in

Psychological Science, 7, 67−72.Kluger, A. N., & DeNisi, A. (1998). Effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary

feedback intervention theory. Psychological Bulletin, 119, 254−284.Kozlowski, S. W., Gully, S. M., Brown, K. G., Salas, E., & Nason, E. R. (2001). Effects of training goals and goal orientation traits on

multidimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes, 85, 1−13.Kozlowski, S. W. J., Toney, R. J., Mullins, M. E., Weissbein, D. A., Brown, K. G., & Bell, B. S. (2001). Developing adaptability: A theory for the

design of integrated–embedded training systems. In E. Salas (Ed.), Advances in human performance and cognitive engineering research, Vol. 1.(pp. 59−123). Amsterdam: Elsevier.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Applications of cognitive, skill-based, and affective theories of learning outcomes to new methods oftraining evaluation. Journal of Applied Psychology, 311, 311−328.

Landy, F. J., & Conte, J. M. (2004).Work in the 21st century: An introduction to industrial and organizational psychology. New York: McGraw-Hill.Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall.Mann, R. B., & Decker, P. J. (1984). The effect of key behavior distinctiveness on generalization and recall behavior in behavior modeling training.

Academy of Management Journal, 27, 900−910.May, G. L., & Kahnweiler, W. M. (2000). The effect of a mastery practice design on learning and transfer in behavior modeling training. Personnel

Psychology, 53, 353−373.Muchinsky, P. M. (2003). Psychology applied to work (7th ed.). Belmont, CA: Thomson/Wadsworth.Naylor, J. C., & Briggs, G. D. (1963). The effect of task complexity and task organization on the relative efficiency of part and whole training

methods. Journal of Experimental Psychology, 65, 217−224.Rouiller, J. Z., & Goldstein, I. L. (1993). The relationship between organizational transfer climate and positive transfer of training. Human Resource

Development Quarterly, 4, 377−390.Salas, E., Cannon-Bowers, J. A., Rhodenizer, L., & Bowers, C. A. (1999). Training in organizations: Myths, misconceptions, and mistaken

assumptions. Research in Personnel and Human Resources Management, 17, 123−161.Schmidt, R. A., & Björk, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training.

Psychological Science, 3, 207−217.Senge, P. (1990). The fifth discipline. New York: Doubleday.Snow, R. E. (1986). Individual differences and the design of educational programs. American Psychologist, 41, 1029−1039.Snow, R. E., & Lohman, D. F. (1984). Toward a theory of cognitive aptitude for learning from instruction. Journal of Educational Psychology, 76,

347−376.Tannenbaum, S. I. (1997). Enhancing continuous learning: Diagnostic findings from multiple companies. Human Resource Management, 36,

437−452.Thompson, L., Gentner, D., & Loewenstein, J. (2000). Avoiding missed opportunities in managerial life: Analogical training more powerful than

individual case training. Organizational Behavior and Human Decision Processes, 82, 60−75.Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. (1995). Applying trained skills on the job: The importance of the work environment. Journal of

Applied Psychology, 80, 239−252.Vroom, V. H. (1964). Work motivation. New York: Wiley.Wexley, K. N., & Latham, G. P. (2002). Developing and training human resources in organizations (3rd ed.). Upper Saddle, NJ: Prentice Hall.Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 37, 60−82.Wood, R., & Bandura, A. (1989). Social cognitive theory of organizational management. Academy of Management Review, 14, 361−384.

linking instructional objectives to the design of instructional environments: the integrative...

Documents