informal learning, line responsibility: audit to impact
TRANSCRIPT
E-Learning, Informal Learning,
Line Responsibility: Audit to Impact
Will Thalheimer, Work-Learning ResearchHillary Leisten & Sal Venegas, Walgreen Co.
103
March 11 - 13, 2009Orlando, FL
Page 1Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
March 2009Retail Best Practices
Sal VenegasCorporate Manager Walgreens Learning [email protected]‐914‐8188
E‐Learning, Informal Learning, Line Responsibility: Audit to Impact
Will Thalheimer, PhDPresidentWork‐Learning Researchinfo@work‐learning.com617‐666‐9637
1
Sal VenegasCorporate Manager, Program DevelopmentWalgreens Learning Services
2
Which of the following
characteristics make it difficult to excel in
the learning enterprise?
Which are opportunities?
Reflection Questions
3
• Long history of store‐based learning
• More than 6000 stores across the US
• Expanding into non‐traditional healthcare
• Tight margins, competitive industry, increased regulation
Walgreen Co.
4
• Geographically dispersed with over 200,000 employees
• 10 – 15 job roles per store
• Very top‐down management, very task focused
• No training budget – managers expected to fit training into budget
Walgreens Retail Stores
5 6
• Large support function, split into two departments: Learning Services & Systems Training
• Learning largely e‐learning, with some classroom training for management
• Main driver of what gets built – requests from Operations
• Learning group is centralized in Chicago
2
Walgreens Learning Landscape
March 11 - 13, 2009Orlando, FL
Page 2Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
What are the issues with a geographically‐
dispersed audience?
What is likely to happen if stores don’t have a
training budget?
Reflection Questions
7
• What was the situation like at Walgreens that prompted this effort?
• What was the driving issue?
• Why was someone from the outside called in?
How Did This Get Started?The Walgreens Perspective
8
Will Thalheimer, PhD
9
MissionË Help Clients Build More Effective Learning Ë Utilizing Wisdom from Research & PracticeË Compile Research, Share Findings
WorkË Consulting (Learning, Performance, Assessment)Ë Workshops & SpeakingË Learning Audits, Work‐Learning AuditsË Research and Writing
10
• What had you been doing with Walgreens?
• What seemed to be the driving issue?
• What were some of your biggest worries getting started?
• What were the biggest plusses getting started?
How Did This Get Started?The Work‐Learning Research Perspective
11
• How Well is Learning Supporting Store/ Pharmacy Success?
• What More Can be Done?
• What’s Wrong with the E‐Learning?
Note: E‐learning programs are called PPL’s at Walgreens
Framing the Audit Goals
12
March 11 - 13, 2009Orlando, FL
Page 3Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
LearnerFulfillment
LearningResults
LearnerApplies
The Learning Landscape
Learner Learns
LearnerRetrieves
LearningIntervention
PerformanceSituation
LearningOutcomes
© Copyright 2008-2009 Will Thalheimer, www.work-learning.com
13
Making Learning WorkThe 4 Factors
A. Preparation
Are Learners Able & Willing to Learn? Does the Culture Encourage and
Support Training?
A1. Do workers have time for learning?
A2. Do workers want to learn the topics taught?
A3. Do learners’managers encourage learning(vs. completion)?
A4. Do learners’coworkers encourage learning?
B. Training
Do Learning Programs Create Understanding
and Enable Remembering?
Do learning programs:
B1. Promote motivation to learn?
B2. Enable understanding?
B3. Focus on relevant tasks?
B4. Provide sufficient practice?
B5. Focus on key concepts, and avoid overload?
C. After-Training
Does the On-the-Job Situation
Support & Reinforce What was Learned?
C1. Do managers reinforce what was learned?
C2. Do workers have incentives and/or motivation to use what they learned?
C3. Do workers have sufficient authority, time, competence, and resources?
C4. Can learners remember what they learned?
D. OTJ-Learning
Does the Workplace
Support On-the-Job Learning?
D1. Do workers know good/poor performance when they see it?
D2. Do managers provide good coaching and feedback?
D3. Are there clear overarching job goals?
D4. Can workers learn from each other?
D5. Is there a learning attitude/culture?
14
• Maximize the benefits of Walgreens formal learning practices, within the business model’s typical cost and time constraints.
• Maximize the benefits of Walgreens store/pharmacy workplace‐learning environment to ensure store/pharmacy success.
Overarching Goals
15
• Phase 1 – Gather Data to Determine Needs
• Phase 2 – Build Consensus About the Issues
• Phase 3 – Bring Stakeholders Together to Make Plan
• Phase 4 – Begin to Implement Plan
• Phase 5 – Continue Momentum, Fix Problems
• Phase 6 – Ensure Integration, Look for Opportunities
Project Phases
16
l Interviews in Boston & New Hampshire with 20+ employees
l Focus Groups in Minneapolis, Tucson, and Southern California
l Store Observations and Shadowing
l Learning Audits of E-Learning Courses (PPL’s)
l In-class review of training for executive assistant managers
l Review of company artifacts (newsletters, magazines, etc.)
l Interviews with training management and staff
l Interviews/Discussions with key business leaders
l Attendance at Learning Task Force meetings
Methodology
17
Strengths of the
Methodology? Limitations of the
Methodology?
Reflection Questions
18
March 11 - 13, 2009Orlando, FL
Page 4Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
TheFindings
19
• Focus on competency, not completion
• Improve e‐learning quality
• Retool Walgreens learning function
• Develop better metrics
• Grow “performance development managers”
Recommendations
20
LearningCulture
21
A lesson well learned ‐ and never forgotten…
In a series of jobs with Chicago's leading pharmacists ‐ Samuel Rosenfeld, Max Grieben, William G. Valentine and, most importantly, Isaac W. Blood ‐Walgreen grew increasingly knowledgeable ‐ and increasingly dissatisfied ‐ with what he saw as old‐fashioned, complacent methods of running a drugstore. Where was the desire to provide superb customer service? Where were the innovations in merchandising and store displays? Where was the selection of goods that customers really wanted ... and could afford? Where was the sense of trying to understand, please and serve the many needs of drugstore customers?
Exact Quote from the Walgreens Website
Learning
Analysis
Vision
Action
LAVA
Charles R. Walgreen, Sr.
22
Formal Training
1. PPL’s2. PPL In‐store Worksheet Activities3. ADSM4. Tutoring by DTSS and DPTC, etc.5. Workshops (e.g., Beauty Trainings)6. Public Seminars7. Seminars for District Staff at
Corporate8. Emerging Leader
Meetings
1. All‐store meetings2. 5‐minute shift meetings3. Store management meetings (store
managers with their EXA’s and MGT’s)4. District meetings5. Task Force & committee meetings6. Management Seminar (Vegas)
Out‐of‐Store Work Experience
1. Rotations to other stores2. Time spent in training stores3. MGT’s get 1‐day orientation in one
store4. MGT’s spend 20 days in a training
store5. Task force work, (or more generally,
committee work)6. Special assignments for DM7. Special assignments for Region or
Corporate
Reports, Documentation, Databases
1. Reports2. KPI’s3. StoreNet4. Policy and Procedures5. Walgreens World
23 24
March 11 - 13, 2009Orlando, FL
Page 5Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
25 26
Formal Training1. PPL’s2. PPL In‐store Worksheet Activities3. ADSM4. Tutoring by DTSS and DPTC, etc.5. Workshops (e.g., Beauty Trainings)6. Public Seminars7. Seminars for District Staff at Corporate8. Emerging Leader
Meetings1. All‐store meetings2. 5‐minute shift meetings3. Store management meetings (store managers with their
EXA’s and MGT’s)4. District meetings5. Task Force & committee meetings6. Management Seminar (Vegas)
Out‐of‐Store Work Experience1. Rotations to other stores2. Time spent in training stores3. MGT’sget 1‐day orientation in one store4. MGT’s spend 20 days in a training store5. Task force work, (or more generally, committee work)6. Special assignments for DM7. Special assignments for Region or Corporate
Reports, Documentation, Databases1. Reports2. KPI’s3. StoreNet4. Policy and Procedures5. Walgreens World
Worker‐Initiated Learning1. People ask peers questions. Peers answer questions.2. People ask managers questions. Managers answer questions. 3. People ask for new challenges.4. People ask for feedback from manager(s), peers,
or direct reports.5. People ask to shadow someone with more experience.6. People take notes and refer to them later.7. People take notes to remember to ask questions later.8. People review the PPL’s (online or from printouts).9. People ask to do different tasks or work different shifts.10. People look for opportunities to work with (learn from) “experts.”11. People review StoreNet, KPI’s, and/or other reports12. People read Walgreen’s World, Applause, etc.13. People create a vibe that lets others know they are open to feedback.14. People step back, and observe and reflect on what is going on.15. People take advantage of open‐door policy.16. People gossip and listen to gossip.17. People evaluate their own strengths/weaknesses and look for opportunities to parlay their strengths and improve on
their weaknesses.18. People notice their tendencies and try to be more flexible.19. People look for opportunities for repetitions to solidify their learning and refine their skills.20. People make a special effort to remember who knows what.21. People contact the Help Center.22. Managers ask for feedback on their management behavior.
Manager‐Driven (Expert‐Driven) Learning1. DM’s do walkthrough’s pointing out good and bad things.2. Managers do walkthroughs pointing out good and bad things.3. Assistant Managers do walkthroughs pointing out good and bad things.4. Managers ask workers to complete tasks on task list, highlighting what is important.5. Managers assign tasks to be done, enabling workers to learn new tasks.6. Managers assign tasks to be done, reinforcing previous learning.7. Managers show how a task is done.8. Managers show how a task is done and then assign the task and give feedback. 9. Managers provide constructive criticism and positive feedback at critical moments.10. Managers assign workers to (a) Observe Task Being Done, (b) Do Task with Close Supervision, (c) Do Task with Far
Supervision, (d) Do Task On Their Own. 11. Managers leave problems, and see how long the problem will stay (w/o worker intervention).12. Managers intentionally pair “learners” with people with the right attitude, knowledge, and teaching skills.13. Managers let employees try experiments (i.e., merchandising) andthen look at data to see what effect it had.14. Managers give people new challenges. 15. Managers encourage an “experimental attitude,” encouraging workers to wonder what would happen if changes were
made, trying things out, learning together through trial and error.16. Managers assign people to take over certain responsibilities when they are off.17. Managers assign people to take over other people’s responsibility when they are off. 18. Managers answer questions, or find out who to ask.19. Managers give people a notebook to (a) write down questions, (b) take notes on things to remember. 20. Manager encourages others to approach him/her, encouraging questions, discussions, etc.21. Manager models appropriate behavior.22. Manager coaches/mentors store employees.23. Manager sets guidelines for each worker to know who to ask.24. Manager educates workers on who knows what.25. Manager educates workers on the location of critical information.26. Manager provides employees with detailed performance reviews.27. Managers give others responsibility for doing walkthroughs.28. Store manager assigns people particular departments or areas in the store to manage, and together they observe the
results (from a sales perspective).29. Store manager keeps track of each employee’s development, either through a mental or actual checklist, and provides
coaching, training, and learning opportunities as appropriate.
Hands‐onLearning
27
How many of you get to look at on‐the‐job learning/ performance?
A.ExtensivelyWe fully
understand it.
Reflection Questions
B.Significantly
We understand
some
C.Occasionally
We understand a
little
D.None/LittleWe don’t really get it
28
LearnMost
Learn a Lot
LearnSome
Learn a Little
Learn Least
“Off theboard”
Rough Example – Pharmacy Tech
PPL’s
Experience
Self Learning
StoreNet
WalgreensWorld
Customers
InsuranceCompanies
Corporate 3rd Party
WalgreensCorporate
Pharmacist
PharmacyManager
Other Pharmacy Techs
Senior Pharmacy
Tech
Pharmacy Supervisor
Store Manager
DPTC
EXA
MGT
29
Does your business side understand that people learn from those closest
to them?
Reflection Question
30
March 11 - 13, 2009Orlando, FL
Page 6Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
How Learning Works in Stores
31
OK
Expert
Weak
How Learning Works in Stores
32
Weak
Beauty Advisor
ExpertWeak
OK
HeadCashier
ServiceClerk
H. Beauty Advisor
Expert
StoreManager
OK
EXA
Weak
MGT
Expert
SIMSCoord’r
Weak
WeakWeakOK
PhotoTech
ServiceClerk
ServiceClerk
ServiceClerk
OK
H. PhotoTech
Moderate‐Skill Store
33
Weak
Beauty Advisor
ExpertWeak
HeadCashier
ServiceClerk
Expert
StoreManager
Weak
MGT
Expert
SIMSCoord’r
Weak
WeakWeakOK
PhotoTech
ServiceClerk
ServiceClerk
ServiceClerk
Expert
Expert
Expert
H. PhotoTech
H. Beauty Advisor
EXA
Strong‐Skill Store
34
Weak
Beauty Advisor
Weak
OK
ServiceClerk
H. Beauty Advisor
Expert
StoreManager
OK
EXA
Weak
MGT
Weak
WeakWeakOK
PhotoTech
ServiceClerk
ServiceClerk
ServiceClerk
OK
H. PhotoTech
OK
OK
SIMSCoord’r
HeadCashier
Weak‐Skill Store
35
Implications
1. On‐the‐job Learning is easier at some stores than others.
2. Causing a spiraling up/down effect:
• the more learning, the more experts, the more learning, etc.
• the less experts, the less learning, the less experts, etc.
36
March 11 - 13, 2009Orlando, FL
Page 7Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
Is this notion widely considered in your organization?
Reflection Question
37
E-Learning38
Store Manager:
“The problems before PPL were greater than they are now. I think PPL has created additional issues, so I don’t think it’s fair to say, get rid of PPL get rid of our problem. We just go back to the same problem we had before we didn’t know most stores probably weren’t taking the time to communicate with the new employees, which created a lot of problems.”
Exact Quote from the Focus Groups
Get Rid of PPL?
39
“We become more focused on getting people through, tab, tab, tab, tab, test. So that they’re done instead of them learning material. It’s an unrealistic timeline that a lot of times gets thrown out there…”
Manager: “I never asked somebody, you know, did you learn anything from the PPL, just points us to get it done that’s it.”
“The thing that’s important with the PPL is is like make sure that you like learn it right before it comes in. If you got that knowledge fresh in your mind than you got the hands‐on then boom you pick it up like that. If you learn the PPL and then six months later the new equipment comes you’re like well I better just go back and read that again and you’ve wasted time and we don’t have time to waste.”
PPL Feedback
40
“I’ll be real honest. Ever since the first day first time I got here and did PPL’s it was great and I learned a lot on the computer but I really, except for the legal ones, I haven’t really paid attention to a lot of them except one time a couple of years ago when they switched over to the new [pharmacy] system, the whole system got completely revamped, then I really paid attention to that. So, it’s that stuff that is really critical in knowing how to run the computer. The other stuff I really just look over it. I don’t look at it.”
“As I say that, part of me feels guilty actually. I feel guilty saying that because there is somebody out there who actually puts the time in to make sure that people should know this stuff and, all of a sudden I’m realizing, are putting it out there not just to put it out there, maybe somebody out there that feels this is important maybe I should look at it. I’m feeling a little guilty about that.”
PPL Feedback
41
What People LIKE About PPL’s
42
March 11 - 13, 2009Orlando, FL
Page 8Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
What People Do NOT Like About PPL’s
43
l Perceptions: Lots of strongly-felt complaints. Many see benefits too.
l PPL’s have been evolving to more hands-on application; better design.
l PPLs have a brand problem.
l PPL completion mandate is problematic
l PPL’s create wrong frame about what training/learning is
l Managers may spend less time on coaching, or thinking about development
l Due to corporate staffing model, training department was driven to hire less senior, lesser-skilled staff
l Technology base created limitations
Some Perspective on PPL
44
Re-Brand PPL’s, and revise them to better support hands-on learning.
Improve the PPL learning environment
Energize the courses
Rethink every PPL: Eliminate? Shorten? Redesign?
Add a “Steve Jobs” or “Oprah”hardnosed visioning and editing role
Gather better data on how PPLs are implemented
Improve E‐learning
45
TheCompetency
Way46
Jane Smythe MGT Store # 67384
Course Name Completion? Last Date
Tobacco Handling Yes 10‐1‐2007
Merchandising Introduction Yes 10‐4‐2007
Merchandising 1 Yes 10‐15‐2007
Merchandising 2 Partial 10‐17‐2007
Merchandising 3 No ‐‐‐
Customer‐Service Introduction Yes 10‐2‐2007
The LMS Way
47
The Competency‐Management Way
48
March 11 - 13, 2009Orlando, FL
Page 9Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
The Competency‐Management Way
Expert
StoreManager
Store#98102
Store#98106
Store#98104
Store#98105
Store#98103
Store#98101
Expert
H. Beauty Advisor
I need to train my beauty advisor. I wonder who has an expert beauty advisor?
Expert
H. Beauty Advisor
StoreManager
OKHey Joe, can Sally train my beauty advisor?
Sure Sue, can you help me with my Performance‐Development stuff? How about sending your advisor over next week?
Store#98108
Store#98107
49
The Competency‐Management Way
Store#98102
Store#98106
Store#98104
Store#98105
Store#98103
Store#98101 Store
#98108
Store#98107
DistrictManager
Oops. I’m getting an alarm. The system tells me I might have a weakness in photo skills in my district? Let me see.
Weak
PhotoTech
Expert
H. PhotoTech
OK
H. PhotoTech
Expert
H. PhotoTech
Weak
PhotoTech
Weak
PhotoTech
Weak
PhotoTech
Weak
PhotoTech
Ouch. It looks like we need to plan some extra hands‐on training? I’ll get those two expert Head Photo Techs to cycle around the district for the next month until we fix this.
50
Does your organization have anything like this?
Would this work for you?
Reflection Question
51
LearningFunction
52
• Training is a request‐driven operation for the most part.
• Training constantly educates Ops that issues may not be a training issue.
• Business strategy was to bring in entry‐level staff and grow from within, rather than hiring talent from outside.
• Learning materials are sometimes created without Training input (or knowledge).
Retool Walgreens Learning…
53
• More Focus on Workplace Learning, Management Development, and Performance Consulting – Less Focus on Developing Courses.
• Develop a better partnership with operations, more proactive.
• Seek more highly skilled talent.
• Spend more time on the ground, in stores, with audience
Retool Walgreens Learning
54
March 11 - 13, 2009Orlando, FL
Page 10Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
Do we need to retool our learning function from time to time? What’s your experience?
Reflection Question
55
Metrics &Feedback Loops
56
Have store personnel rate their developmental opportunities
Rate store development culture, rate store manager, assistant managers.
Have store managers, asst. managers rate district learning support and coaching
Create better measurement for PPL effectivenessLet each learner rate each course, develop competency-management metrics
Do formal control-group evaluations of 1% of courses every year
Develop Better Metrics
57
Are you able to do the learning
measurement you want to?
Reflection Question
58
LeveragingManagers
59
What Sets Them Above?1. Gets Both Store & People Results2. Tracks Employee Competencies3. Looks for Development Opportunities for his/her People4. Is Open for Questions, Discussion. Is Approachable. Is Inviting.5. Makes Developmental Assignments, even when it temporarily reduces store performance.6. Assigns Responsibilities, not just Tasks7. Gives People Practice in Noticing what is wrong and what is right8. Enables Experimentation and Follow-up Data Analysis an Decision-Making9. Uses PPL’s to Drive Hands-on Learning10. Is a Good Role Model11. Creates a Trusting, Caring, Respectful Environment
Grow Performance Development Managers
60
March 11 - 13, 2009Orlando, FL
Page 11Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research
Has your organization been able to leverage its managers to push
on‐the‐job learning?
Reflection Question
61
TheFollow-up
62
Current State
• In first year of a 5 year plan to change the learning culture.
• As part of larger corporate initiative, we’re re‐evaluating the jobs and skills needed for the learning function and reorganizing the department.
• We’ve been bringing in higher level more experienced people, to manage and staff the function.
63
• As part of a larger talent management initiative, we are building a competency‐learning system.
• As part of a larger initiative, we are putting more training resources into the field.
Current State
64
Will Thalheimer, PhDPresidentWork‐Learning Research, Inc.2 Belmont TerraceSomerville, Massachusetts, 02143, USA
617‐666‐9637
Mail: info@work‐learning.comWebsite: www.work‐learning.comBlog: www.willatworklearning.com
Sal VenegasCorporate Manager, Program DevelopmentWalgreens Learning Services304 Wilmot Road, MS #3165Deerfield, IL 60015
(847) 914‐8188
Mail: [email protected]@venegas.info
65