optimizing learning & performance management – part 1 & 2

Optimizing Learning &

Performance Management – Part 1 & 2

www.eLearningGuild.com

Bryan Chapman, Chapman Alliance

November 11-14, 2008 San Jose, CA

Page 1Session TIA1 & TIA2 – Optimizing Learning & Performance Management – Bryan Chapman, Chapman Alliance

Optimizing Learning & Performance Management: Beyond the Standard

Blend

Facilitator:

Bryan ChapmanChief Learning Strategist

Chapman [email protected]

Benchmarking Exercise

Learning and PerformancePractices

the spotlight is on you

How the Customer explained it.

How the Project Leader understood it.

The compromise solution.

How the vendor interpreted the design.

How the Learning Strategist envisioned it.

How the project was documented.

What operations installed.

How the customer was billed.

How it was supported. What the Customer really needed.

3 Macro Strategies

CLIC

QUIP

People, Process, Technology

© 2008

implement

INNOVATE

INVENTORY

IMPROVE

IMPRESS

Initiate

Learning Strategy Design

3‐yea rImplementation

Plan

(revised annually)

LMS and Learning

Technology Selection

Courseware Development for Major Initiatives (learning design and development)

Competency Modeling and Skill Gap Analysis

Informal Learning strategy development

Single‐Source, multimode learning strategy and development

Talent/Performance management strategy

Learning Alignment Health Check

Simulation strategy development

Learning Governance Optimization

Revised Strategies

Team Skill Development

Sharing Best Practices

Executive and Learning Leader Presentations

Project & Vendor Management for Learning Platform Implementations

Be a part of the “CLIC”

Legacy Content Conversion

Open source assessment and customization

Rapid Development Extreme

Class Generated

QUIP Approach

Q = Question

U = Understand

I = Investigate and Interpret

P = Propose



Better than “Quibble” method

Q = Question

U = Understand

I = Initiate Discussion

B = Belabor

B = Backoff

L = Lose Interest

E = Evaporate

People, Process, Technology

People + Process + Technology

© 2008

implement

INNOVATE

INVENTORY

IMPROVE

IMPRESS

Initiate

Learning Strategy Design

3‐yea rImplementation

Plan

(revised annually)

LMS and Learning

Technology Selection

Courseware Development for Major Initiatives (learning design and development)

Competency Modeling and Skill Gap Analysis

Informal Learning strategy development

Single‐Source, multimode learning strategy and development

Talent/Performance management strategy

Learning Alignment Health Check

Simulation strategy development

Learning Governance Optimization

Revised Strategies

Team Skill Development

Sharing Best Practices

Executive and Learning Leader Presentations

Project & Vendor Management for Learning Platform Implementations

Be a part of the “CLIC”

Legacy Content Conversion

Open source assessment and customization

Rapid Development Extreme

Learning Strategists answer the “tough” questions

What is the best LMS?

How many people does it take to run a learning platform?

What is the optimal blend?

How do we get learners to complete courses?

Which is better from a learning perspective – ILT or e-learning?

Should we buy or build it ourselves?

…more

Learning Management SystemLMS

Virtual ClassroomSynchronous Learning

Self-Paced, Self-Service Online Learning

Informal Learning, Knowledge On Demand

Off-the-shelf, pre-built courses

Authoring Tools

Learning Content Management

• Ready-to-use content (courseware)• Easy to find: IT skills, leadership, safety• Hard to find: very specific, job-related skills

• Custom development• Screen design and layout• Interactive exercises• Tests, Quiz, Assessment• Simulations

• Manage large-scale development (workflow)

• Reusable learning content• Searchable repository of source

material

• Central Access to Learning• Individualized Learning Plans• Reporting & Completion Tracking• Instructor-Led Training Scheduling• Launch and track online learning• Certification Management• Competency Management

• Instructor-led, Real time, Online• Connecting geographical disperse

learners• Groups of learners meet together

• 24 X 7 X 365 access to courses• Completed at learner’s own pace• Remediation and feedback are

automated• Developed once, used many times• Automated scoring and completion

status

• Just-in-time, anywhere, anytime• Capture and retain organizational

knowledge• Facilitates collaboration

Modalities of Learning (Current State)



Modalities of Learning (Current State)

International Airline

Geographically disperse

Mix of live, virtual classroom (personal interaction)

Self-paced e-learning (conceptual, procedural)

Construction Company

Big issue: regulatory compliance (safety)

Off-the-shelf courses & LMS

Medical Association

Thousands of medical, teaching case studiesOn CD, on shelf; photos, charts, graphs, narrativeManage and distribute (no patient names) for teaching (25,000 association members)

National Restaurant Chain

Large population, customer-facing employeesNot large groups, needs to be self-paced

Simulation, role-play exercises (empowerment)



Industrial/Manufacturing

100,000+ employees; lack of information about other parts of the companyLearning Wiki – invited 200 authors to write articles

Ended up with over 7,000 content contributors

Cost vs. Functionality of learning platform

Skill-Gap Analysis

Launch e-learning

Advanced Classroom Management

Content Development Tools

Learning Content Management

Happy Sheet

Talent Management

Analytics

Language support

3rd party content

Deep BackofficeIntegration

Certification

Testing

Advanced Question Types

Reporting

Collaborative Learning

E-commerce

Optimizing the Use of Blended Learning

Definition

Blended Learning

1) To combine or mix various instructional delivery modes and methods to meet a common set of learning goals or objectives.

2) Blending the best characteristics of instructor-led training with other, innovative forms of instruction.

3) A phrase frequently used as “buzz words” to try and persuade buyers to choose e-learning solutions (sometimes synonymous with “hype”).

How organizations view “Blended”

Constructivism – “learners create their own reality, based on past experience.”

Behaviorism – “cause and control of behaviors that can be measured and observed.”

Cognitivism – “Mental maps, macro thinking, cognition”



IBM Learning Model

White Papers

Product Knowledge

PowerPoint

Word

.PDF

Student Guide

Scenario-BasedSimulations

GamesExercises

Instructor-Led“culminating experience”

Instructor-Led“culminating experience”

Practice

Pilot Training

Step 1: Ground School

Step 2:Simulator

Step 3: Fly with a Mentor

Step 4: Fly alone

Facts, Concepts

Application More Application, Analysis, Synthesis and Evaluation

Same, with little or no guidance

Blended Learning Ameriprise(Minneapolis, MN)

Business issue: how do you offer consistent training to learners in Atlanta, Boston, Minneapolis, and Seattle? Previously used regionally located trainersLearning community (by date of hire) meet together for training for 10 weeks; 90 minutes, 3 times each week; 15 virtual sessions each dayEmbedded “war stories” from experts in the field (pre-recorded Webex mini-scenarios)Before $4.1 Million; After $1.3 Million95% pass rate; 60,000 completions from in 16 months

Learners: Financial Advisors; geographically disperse

Best Practice: War stories, blended learning, regular sessions with learning community

Time Zones



Cathay Pacific Airline (Hong Kong)

Learners: 14,000 staff worldwide; very geographically disperse

Best Practice: Rapid delivery of quickly changing content; innovative use of application sharing for interactivity

Difficult to pull people off the job for location-based trainingRapidly changing visa requirements (China)Cathay was fined millions for misunderstanding of visa requirementsNeeded quick, hard-hitting training in 32 countriesSynchronous learningWide use of pictures of documents for interactive discussion; passing application sharing control for others to point out parts of documents

Traditional Model Real-Time, Blending Learning

Service as a Product

University of Phoenix (½ virtual, ½ on-campus University)

1300 virtual online sessions per week; 17,000 instructors

Typical course 5 to 6 weeks in duration

Class size 10-13 people to “make the course manageable for instructor and to facilitate collaboration”

Evaluations of course effectiveness indicate that there is “no significant difference” in either the cognitive or affective results

Learners: 180,000 students; 106k attend classes at physical locations; 74,000 take courses fully online

Main Business: Higher Education, For-Profit University

Best Practice: Optimization of class size through experience; research-based findings comparing classroom and virtual

Blended Curriculum

One Lecture per Week. Live virtual session or recorded session.Group and Individual Projects Given Weekly.Small group of 3 to 6; many projects done through work; assignment uploadedThreaded Discussion. Students are graded on their level of participationCourse Resources. Articles, worksheets, online textbookKnowledge Tests. Multiple choice, true/false and essay questionsPost Course Evaluation. Level 1

Blended Learning in an LMS Blended Learning in an LMS



Kirkpatrick – 4 Levels of Evaluation

• Level 1 – Reaction (did they like it?)

• Level 2 – Learning (did they learn something?)

• Level 3 – Behavior (do they now perform better?)

• Level 4 – Results (are we meeting organizational goals?)

Donald KirkpatrickAuthor

Who owns performance Management in your

organization?

Convergence of Learning and Talent Management

LMS

TalentManagement

Compensation Management

Talent Acquisition

Time and Attendance

Performance Appraisals

CompetencyManagement

Career and Succession

PlanningWorkforce Planning

Classroom Management

Onboarding

e-Learning

PerformanceManagement

Certification &Assessment

Learning ContentManagement

Learning ReportingAnd Tracking


LMS

TalentManagement


Talent Acquisition

Time and Attendance






Onboarding

e-Learning





Performance Management

Performance Management

Goals of Performance Management

• Align employee behaviors with organizational goals and guiding principles

• Ensure employee motivation and accomplishment of objectives

• Improve interactions between managers and employees about performance

• Document employee performance objectively for use in development and rewards


LMS

TalentManagement


Talent Acquisition

Time and Attendance






Onboarding

e-Learning





Solutions originating from HR Automation Sector

• Taleo• BrassRing• Vurv• Authoria• Ceridian• Halogen• SuccessFactors• Cezanne• Softscape• SAP• Oracle




LMS

TalentManagement


Talent Acquisition

Time and Attendance






Onboarding

e-Learning





Solutions originating from the Learning Space

• GeoLearning• Saba• Cornerstone • Learn.com• KnowledgePlanet• Plateau• TEDS• SumTotal

Levels of Performance Management Treatment

Level 1. Paper-based, filing system (pharma non-example)Level 2. Spreadsheet or other standard application, tracking compliance and qualificationsLevel 3. Using isolated tools for performance management (i.e. forms for performance appraisals, stand-alone 360 degree assessment.Level 4. Learning Management System (LMS) with robust tracking, competency management, blended learningLevel 5. Convergence of LMS and Talent Management

Levels of Performance Management Tracking Typical Evolution of Performance Management

New CompanyHuman Resources Training

Deploy system to track employees, benefits, compensationOversee annual performance reviewsProvide learning content in the areas of management, regulatory/compliance, and software applicationWork with “competencies”

Skills, knowledge and attitude trainingManage classroom and online learningProvide a catalog of learning content to teach people how to “do their jobs”Creates reports showing “learning gains”Measures through “competencies”

Learning Management System

LMS

Does the word “competency” mean the same thing to both groups?

New CompanyHuman Resources

Radiology Technician:Operates X-ray equipmentGood people/patient skillsPractice patient safetyTeamwork/Collaboration

Radiology Technician:Follows correct procedure to calibrate X-ray equipmentPractices seven steps to mixing chemicals without endangering patientsWhen encountering a difficult patient, practices proper engagement strategy

Training

Learning Management System

LMS

Assessing Competencies



Gap Analysis40 Technical Skills

Gap Analysis

Invo

lvem

ent

Time on job

HRTraining

RecruitRecruit OnboardOnboard

Line of Biz

LearnLearnAppraiseAppraise

CareerCareer

CompensateCompensate

PromotePromoteCathay Pacific Airline (Hong Kong)

Learners: 14,000 staff worldwide; very geographically disperse

Best Practice:Connecting annual performance review with actual, observed behavior as opposed to arbitrary pay increase. Using an LMS

Previous approach: annual reviews almost always lead to annual pay raises, based on simple, scored formThe process was sending the wrong message about what constitutes good performanceGoal: Build an environment that encourages the practice of desired “behavior”LMS serves up a combination of learning and performance events, then creates a profile to reward performance.

Tips for optimizing performance management using an LMS

• Open communication between HR and Training; find a way to work together on competencies

• If you already have some form of competency management in place, link it; rather than replace

• Blended learning is more about on-the-job training, than it is about mixing classroom and online. Don’t be afraid to have your learners “perform.” It is the key to performance management.

• Take advantage of the testing capabilities of your LMS to measure knowledge. Pass this information along during performance reviews

Tips for optimizing performance management using an LMS(cont)

• Use Analytics to measure Kirkpatrick levels 3 and 4. It will take some thought, but you can do it.

• Try taking on “onboarding” as a task for you LMS. You can mix forms and learning into a single environment. Great proof-of-concept for what is possible.

• The most important thing you can do is to create a unified, learning delivery system to bridge the content from HR, training and other content providers into a unified, individualized learning plan.



Unified SystemUnified

InterfaceIndividualized Learning Plans

Common messaging

Comprehensive Transcript

Questions???

Facilitator:

Bryan ChapmanChief Learning Strategist

Chapman [email protected]

1

Case Study – Ameriprise Financial Company Profile: For more than 110 years, Ameriprise Financial has been offering financial planning services in banking and cash management, insurance, investment products such as mutual funds, and lending products such as home equity loans. Ameriprise Financial one of the largest financial planning services in the United States with approximately twelve thousand financial advisors nationwide. Challenges: Like many large companies with a geographically dispersed workforce, Ameriprise Financial needs a training solution that is:

• Reusable and repeatable – to leverage consistent, quality learning solutions across the country as individuals join the company and advance in their careers

• Flexible - to accommodate time zones and advisor/client scheduling, as well as varying levels of expertise, from new hires to experienced financial planners

• Scalable – to support increasing numbers of advisors joining the company

• Trackable – to monitor and report on completion of prescribed training

In 2005, the company set out to design the Ameriprise Financial Transformation Learning strategy. The overall goal was to provide a consistently high level of training for all advisors nationwide, from new hires to experienced financial planners.

Previous Methods of Deployment: Previously, Ameriprise Financial utilized a large network of field sales leaders assigned to key locations throughout the country. As part of their responsibilities, the sales leaders delivered training at the local level in a traditional classroom setting. The consistency, level of instruction, curriculum, materials, and effectiveness of the programs varied dramatically from one location to the next. While sales leaders were involved in training, they could not focus on business development opportunities.

Solution: Ameriprise Financial introduced the “Ameriprise Financial Transformational learning strategy” which encompasses a blended approach including Virtual (WebEx) synchronous classrooms, Online (asynchronous) interactive courses, and DVD job aids all administered through Ameriprise University learning management system (LMS). The competency-based Advisor Career Framework provides the curriculum structure for the Ameriprise University LMS, which guides advisor learning development throughout their careers, starting with foundation knowledge and skills for Pre-Appointment new hire training, then basic 100 level, advanced 200 level, and mastery at the 300 level.

2

Tactical Approach: To deliver a consistent, repeatable, and prescribed curriculum, Ameriprise Financial created a technology infrastructure with a Learning Management System and a blended learning strategy. 1) Learning Management System (LMS): The learning management system is the technology platform for the Advisor Career Framework curriculum. Advisors access all training resources in the prescribed curriculum, the online web courses, and the virtual (WebEx) from a one-stop location. The course enrollment, tracking, monitoring and reporting features help ensure that all advisors have completed their prescribed training. 2) Virtual simulated classroom (WebEx tool): To achieve the goal for community and networking, the new hire advisors attend 30 synchronous Pre-Appointment training sessions (1.5 hours per session) with their hire class and the same trainers for 10 weeks. Professional trainers provide interactive training through white board activities, audio discussion, role play, chat, and polls to over 100 offices with 15 live simulated classes per day. Ameriprise hired the following resources to manage the virtual classroom events:

• Master Scheduler: manages the instructor and class schedules for delivery of 15 live classes per day.

• Master Producer: supports the virtual instructors during class on the WebEx tool.

• Market Group Trainers: 13 professional regional market group trainers deliver training sessions daily.

3) Online web-based courses: The asynchronous web-based courses are organized according to the Advisor Career Framework Ameriprise University in the LMS. These 23 courses represent a portion of the prescribed curriculum. 4) DVD job aids: New advisors have access to seven hours of best practice demonstrations for interacting in sales situations. The Ameriprise Financial training team used a phased approach to ensure a successful rollout:

• Pilot: Ameriprise piloted the entire 10-week blended Pre-Appointment program in two market groups. They conducted on-site training for leaders and staff on the new curriculum, delivery methods, and schedule. They monitored the results through program surveys, individual course surveys, and two on-site visits to observe and interview leaders, staff, and advisor participants. Pilot evaluation and results were used to revise the training appropriately.

• Early roll out: Following the pilot, the program expanded to four more market groups. Leaders and staff were trained through virtual sessions.

• National roll out wave 1: Expanded to eight more market groups. Repeated the virtual training for leaders and staff.

• National roll out wave 2: Expanded to ten more market groups.

3

Business Driver/Results: By delivering consistent, high quality training to all advisors across the country, Ameriprise advisors are better prepared to successfully market their services to new prospects and to provide a higher level of financial planning expertise to their existing clients. The technology supports a scalable solution, leverages professional instructors for national virtual delivery, and delivers consistent curriculum which is tracked and reported. Specific training results include:

• Cost reduction: By switching from a model of using 110 local sales leaders to train in their markets to a centralized model using 13 regional market trainers, Ameriprise Financial saved $3 million in payroll costs.

• Increased business opportunity: By eliminating training responsibilities, local sales leaders are able to instead spend an additional four hours per week developing business opportunities or increasing advisor productivity by coaching advisors.

• Learner satisfaction: Survey results indicate that advisors are receptive to the new technology and innovative delivery method. For example, Virtual Instructor Led course satisfaction is 4.6 on a scale of 1-5.

• Course completion: It is required for advisors to complete all 10 weeks of training and to pass all courses in order to be appointed. Virtual course attendance is 98% which exceeds Ameriprise’s goal of 90%.

About the Author Bryan Chapman is Chief Learning Strategist at Chapman Alliance; a provider of research-centric consulting solutions that assist organizations to define, operate and optimize their strategic learning initiatives. As a veteran in the industry, he has over 20 years experience and has worked with such organizations as American Express, Shell, Kodak, Sprint, Sharp Electronics, Honda, IBM, Microsoft, Avon, UNICEF, The Food and Drug Administration, U.S. State Department, and many others; to help them optimize learning efficiency through the use of innovative learning techniques and technologies. Bryan was formerly the Director of Research and Strategy for independent research and consulting firm Brandon Hall Research, where he served as the primary author and researcher on high profile projects such as the LMS Knowledgebase, LCMS Comparative Analysis Report, Comparison of Simulation Products and Services, and a comprehensive study of custom content developers in the industry. In addition, Bryan was responsible for structuring Brandon Hall Research’s consulting practice. He continues to provide technology-selection services in partnership with Brandon Hall Research as a registered Associate. Chapman Alliance (801) 568-7011 [email protected]

Blended Learning Profile SAP Education Page 1

SAP Education Company Description SAP AG is the world’s leading provider of business software solutions and the third-largest software supplier in the world (ranking after Microsoft and Oracle) with 32,000 people located in 50 countries and annual revenues of $10.5 billion. The company has 12 million software users (in 120 countries), 100,600 installations at 28,000 companies, and 1,500 partners. Standard Blend SAP Education has gone through an evolutionary path in creating its blended learning model. A few years ago, the company made a dramatic shift toward e-learning and was “underwhelmed with the results.” It found that e-learning was (1) too expensive to create for the results, (2) difficult to update and maintain, and (3) lacking the personal touch needed by customers. Their blended learning model was born out of a gradual reintroduction of asynchronous e-learning as a component of blended learning courses. The SAP parent company uses different blended learning models in different parts of the company and different audiences, although supported through a set of common learning development resources. SAP Education’s primary focus is on training partners and external customers. The following outlines the blended learning strategy used for its Partner Academy Courses that has effectively eliminated one full week out of five weeks of training.

E-learning programs as prerequisite material. Without totally abandoning e-learning altogether, SAP Education is using it to fill the role of providing basic information and concepts upfront, before the classroom events begin. Learners self-enroll in the course and can work at their own speed to master the prerequisites prior to becoming involved with course instructors. By the time the instructor meets the learners, they have generally taken up to four e-learning modules (about 20 percent of the overall blended course). Simulations as prerequisite material. In addition, simulations provide opportunities for the learners to “try” the concept learned in a simulated environment. These are self-paced activities that must be completed before the learners arrive in the classroom. Classroom delivery. After completing all the e-learning and simulation-based prerequisites, the learners then come to a classroom setting, complete with a computer lab, so that they can learn on the actual system. This is the first time that the SAP trainer meets with the student. The basic information and simulations have prepared them for simulated OJT tasks on the computer. Instructionally, this allows the course to become more customized and adaptable as they enter the classroom portion of the course. Case Study. The last day of the course is called a “case study,” which is a learning activity that draws on all prerequisite skills to complete a project-based assignment. The exercises are very prescriptive and very structured to ensure success.

Certification Test. The final event of the blended course is the certification exam. These tests are delivered in paper-based format because of the need for security in issuing the proctored exam. The tests include multiple-choice and true/false questions.

Post-Course Evaluation. Students fill out an online survey to provide feedback on the class, instructors, environment, etc.


Collaboration Through Communities of Practice. Although the blended learning model doesn’t require collaboration to complete the course, each certification track has a parallel “community of practice” where students and graduates can go to share information and answer each other’s questions. The community of practice is not currently used as an assigned resource for the course (perhaps in the future).

Additionally, SAP Education still has some courses that are entirely based on e-learning. E-learning is used for shorter courses or courses that require complete consistency. Results of the Blend SAP Education has had success in shortening the length of existing courses using a blended learning model. For example, for several end-user, customer courses, they have reduced the amount of in-class time from five days to two and a half days (with the remainder of the course available in self-paced study mode). For the Partner Academy, courses that were previously five weeks in length have been reduced by one full week. In comparing the new blended courses to the previous format, SAP Education has discovered that there is no difference in meeting the course objectives (as monitored through pass rates on certification exams). Both prepare learners for the certification, yet the blended learning model is more efficient and less time consuming. Resource and Support Requirements

KPS – shared, learning development team. KPS stands for “Knowledge Production Services.” This group of 175 people provides internal learning development resources for all of SAP (servicing all 32,000 SAP employees and numerous partners and end users in the case of SAP Education). The group manages and maintains the LMS, manages the server farms used to support content delivery, runs help desk support for trainers and end users, [[needs verb here?]] instructional design, and develops blended learning, including e-learning courseware. Since SAP Education is a for-profit training organization, courses must be designed to the highest quality standard. KPS provides this service. Outsource Development. SAP Education used a combination of internal and external development. For some of the e-learning modules, external vendors are used to create specific types of simulations and interaction.

Help Desk. 100 percent supported by KPS. Instructional Designers. SAP Education often uses its own instructional designers to create detailed designs for courses, using KPS or external vendors for the production work only. They have found this to be a good model to ensure that the designs meet customer needs.

Tools and Technologies Used for the Blend The core of the self-service, blended learning model is SAP’s TEM module (training and events scheduling system). TEM is used for scheduling and managing the classroom events (whether inside the SAP training facility or at the customer’s location). The group is planning a migration to SAP LS (Learning Solution) in the near future. In the meantime, it has created its own proprietary system for launching and tracking the completion of e-learning prerequisite material. In addition, it has created its own internal solution for administering tests and collecting post-course feedback.


RWD InfoPak simulator is used for software simulations used as prerequisite training. The company previously had an entire curriculum based on the use of virtual classroom. Today, it is using virtual classroom software only on a very limited basis, although it is working on a new model to provide blended virtual classroom courses in the near future. Lessons Learned

One of the most important lessons learned is that, in a blended learning model, everyone needs to perform his/her role and do his/her part. For example, instructors rely on the fact that the e-learning modules cover certain aspects of the course that don’t need to be repeated in the classroom setting. If instructors try to do it all, “we lose the benefits of blended learning.”

There are definitely cultural issues in blended learning. For example, “Germans naturally have a reluctance to want self-paced learning, while in Latin America, they are very open to self-paced.” You need to have a plan for adapting the blend for specific types of audiences and in different languages.

“We have learned that e-learning alone is not the way to go.”

Although Web-based learning represents only 20 percent of the blended course, it takes 50 percent of the development resources for creating the course. SAP Education is experimenting with using recorded, live sessions for some of the information-based prerequisites. This may be a good way to reduce some development costs.

optimizing learning & performance management – part 1 & 2

Documents