tncpe examiner orientation...4. report out a lesson learned. 61 1.1 2.1 3.1 6.1 •criteria–read...

66
TNCPE Examiner Orientation 2018 Board of Examiners 1 Congratulations! 2 “Serving as a TNCPE examiner is the most powerful investment of time you will ever make.” – Dr. Lyle Ailshie Deputy Commissioner TN Dept. of Education

Upload: others

Post on 19-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

TNCPE Examiner Orientation

2018 Board of Examiners

1

Congratulations!

2

“Serving as a TNCPE examiner is the most powerful investment of time you will ever make.” 

– Dr. Lyle AilshieDeputy CommissionerTN Dept. of Education

Page 2: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

What Is an Examiner?

TNCPE Examiners serve…• The Southeast region• TNCPE• Applicant organizations

…through

• Reviewing and scoring applicants• Performing site visits

• Writing feedback reports

• Serving as ambassadors

3

As a member of the BOE…

• You will gain valuable skills, knowledge, and experience

• Contribute to the economic growth of applicants, the Southeast, and the USA

• Meet and work with some of the finest professionals in the state

• Enjoy a profound sense of achievement

• Have fun!

Page 3: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Introductions

• Name

• Employer

• What do you want to learn today?

• Adjective that best describes you

5

Alice: “Which way should I go?”

Cat: “That depends on where you are going.

Alice: “I don’t know where I’m going!”

Cat: “Then it doesn’t matter which way you go!”

6

Through the Looking GlassLewis Carroll, 1872

Page 4: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Agenda/Objectives

• Getting Grounded• Baldrige Performance Excellence Program

• Tennessee Center for Performance Excellence

• Baldrige Excellence Framework™

• Getting Started• Understanding the pre‐work assignment

• Step‐by‐Step Instructions

• Using Scorebook Navigator™

7

Baldrige and TNCPE Overview

8

Page 5: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Baldrige Performance Excellence Program

• Established by Congress in 1987

• Purpose: to enhance U.S. competitiveness in a global economy by:

• Identifying and recognizing role‐model organizations

• Establishing criteria for evaluating improvement efforts

• Disseminating and sharing best practices

• Internationally recognized and emulated 

• National resource:  www.baldrigepe.org

9

Mac Baldrige

Endorsed by PresidentsPresident Ronald Reagan

“The Malcolm Baldrige National Quality Award offers companies a standard with  which to compare their own progress to that of the country’s very best.” 

President Bill Clinton

“Every year I feel more strongly that this is the way America ought to work. This is the way all of our organizations ought to work.”

President George W. Bush

“These organizations have been selected as exemplifying the qualities of excellence and ethics. They are being recognized for their continual improvement, their commitment to employees and community, and their enterprising spirit.”

President Barack Obama

“The road to greatness in American has been, and always will be, traveled by thosewho embrace change and work hard every day to be the best; the organizations we honor today with the Baldrige National Quality Award exemplify that spirit.”

President Donald Trump

“This program is about much more than recognizing successful organizations or winning a single award … When companies implement the ground‐tested Baldrige approach, they create organizations that employees and customers love, that continually improve, and that produce innovative and outstanding results.” 10

Page 6: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

How Organizations Use the Baldrige Excellence Framework

• Internal assessment & management framework

• State/Regional/Industry Baldrige‐based program

• Ease of entry into process

• 5, 15, 35, or 50‐page application

• National Baldrige application

• 50‐page application

• Same requirements as Level 4 TNCPE application 

11

Benefits of “The Journey”

Helps answer 3 questions:

1. Is your organization doing as well as it could?

2. How do you know?

3. What – and how – should your organization change to get better results? 

12

Page 7: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Why TNCPE?

• “Easy way in” to understanding and applying the Baldrige Framework• 4‐tiered application program

• All receive site visit & written feedback report

• Train internal experts as examiners

• Learn & share best practices as a member 

• Pre‐requisite to Baldrige application• Must win your state’s or region’s top award to be eligible to apply to Baldrige

13

Mission  To drive organizational excellence in Tennessee and the Southeast region

Purpose  To strengthen the economy of Tennessee and the Southeast region 

Vision To be the partner of choice in leading organizations to world‐class performance excellence.  

Together…building a state of excellence!

Values Customer Focus; Integrity; Respect;Collaboration; Leadership; Excellence; Innovation; Continuous Improvement

Core Competencies

Building loyal relationships with key stakeholders;Operating an effective Award program

TNCPE Culture

14

Page 8: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

TNCPE Board of Directors  Dr. Lyle Ailshie, Deputy Commissioner, TN Dept. of Education

Mark Bainbridge, VP, Caterpillar Financial Services

Chris Beaulieu, Global Operations Leader, Cummins Business Services 

Hal Beckham, VP & CFO, Medtronic Inc., Spine Division

Tony Benton, CEO, Tennova Healthcare East Market

Lee Brown, General Manager, Erwin Utilities

Janell Cecil, SVP & CNO, University of Tennessee Medical Center

Dr. Dennis Depew, Dean, College of Business and Technology, ETSU

Dr. John Dreyzehner, Commissioner, TN Dept. of Health

Kevin Grayson, Asst. Dir. Technology Management, North Carolina State University

David Hart, CEO, TRICOR

Rebecca Hunter, Commissioner, TN Dept. of Human Resources

Paula Jacobs, VP Process Improvement, Methodist Le Bonheur Healthcare

Doran Johnson, SVP, NHC South Central Region

David Jones, CEO, EnableComp

Wes Kelley, President and CEO, Huntsville Utilities

Patrick Lawton, City Administrator, City of Germantown

Stacey Max, Director of Engineering, Marvin Windows and Doors

Paul Saylor, President & COO, Caris Healthcare

Chuck Shoopman, AVP, UT Institute for Public Service

Amy Shreve, EVP, First Tennessee

Jennifer Slayton, Ex Dir, Quality Improvement, Vanderbilt University Medical Center

Van Wardlaw, EVP, TVA

Alan Watson, CEO, Maury Regional Medical Center

Rodney Woods,  VP, BlueCross BlueShield of Tennessee

15

Stair Steps to Excellence

16

Level 4

Level 3 Excellence

Level 2 Achievement

Level 1 Commitment

Interest

Org Profile Yes Yes Yes Yes

ApplicationFocus

OrgProfile

Basic item requirements

Overall item requirements

Multiple item requirements

Max Length 5 pages 5 + 15 pages 5 + 35 pages 5 + 50 pages

Site Visit ½ Day 1 Day+wkshp 2 Days 3 Days

Feedback Focus

7 Categories Overall requirements

Multiple requirements

Multiple requirements

Page 9: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Tennessee Baldrige Winners

17

www.TNCPE.org

• Founded in 1993

• Based on a national standard – Baldrige Excellence Framework – with a focus on continuous improvement

• 501c(3) corporation

• Powered by a volunteer workforce

• Funded through corporate and individual dues and award participant fees

• National and international benchmark

• Partnership with NCAfE

• Recognize – Train – Share 

18

Page 10: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

TNCPE Results (1993 – 2017)

• 1,469 applicants

• 642,736 Tennessee employees impacted

• 3,840 examiners trained

• 340,012 hours of service

• $34 million pro‐bono consulting services

19

Why Baldrige/TNCPE?

20

Page 11: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

The Baldrige Excellence FrameworkTM

Business/Nonprofit | Health Care | Education

21

Baldrige Excellence Framework

• Used to view organizations from a systems‐perspective

• “Management by asking really good questions”

• Includes the Criteria, Core Values, and Scoring Guidelines

• Offered in 3 versions – Business/Nonprofit, Health Care, and Education

• Updated every two years

• “Leading edge of validated management practice” 

22

Page 12: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

23

Category StructureCategory

1 – LeadershipItem1.1 – Senior Leadership

Areas to Address1.1a – Vision and Values1.1b – Communication1.1c – Mission and Organizational Performance

Item1.2 – Governance and Societal Responsibilities

Areas to Address1.2a – Organizational Governance1.2b – Legal and Ethical Behavior1.2c – Societal Responsibilities

24

Page 13: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Item format (page 2)

25

Applicant Assessment Process

26

Self assess

Apply

Examiner review & site visit

Award

Receive Feedback Report

Address OFIs

2

3

45

1

Page 14: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Examiner Evaluation Cycle

IndependentReview

Consensus Review

Site Visit Review

Finalize Feedback Report

Complete Pre‐work & Training

27

Getting Startedon Pre‐work

Understand the Assignment

Follow Step‐by‐Step Instructions

Use Scorebook Navigator

28

Page 15: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Case Study Preparation

• Schedule adequate time (20 – 25 hours) 

• Assemble documents and tools:• Scorebook Navigator (examiner software system)

• 2017‐2018 Baldrige Excellence Framework (Business version)

• Green Gateway Case Study

• Pre‐work Assignment Packet

• Links for downloads

• Step‐by‐Step Instructions

• Examples & Resources

• Pens, highlighters, clock, etc.

29

Pre‐Work Assignment

Key Factors

Item Evaluation

+ 1 OFIComment

P.1 a

P.1 b

Item 1.1

Item 2.1

Item 3.1

Item 6.1

P.2 a

P.2 b

P.2 c

Item 7.1

Item 7.2

30

Page 16: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Remember…

Trust the process!

31

Log on to the system

Review the Framework

Read the Application

Create Key Factor List

Print 2 copies of assignment

Note # of hours

Complete pre‐work evaluation

Attend 3‐day training

1.12.13.16.1

•Criteria

•Key Factors

•Application: PROCESS

•6‐10 Comments (ADLI)

•Score

•1 Feedback‐Ready OFI

7.17.2

•Criteria

•Key Factors

•Application: RESULTS

•6‐10 Comments (LeTCI)

•Score

•1 Feedback‐Ready OFI

Pre‐work Assignment

32

Page 17: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Log on to Scorebook Navigator

• Go to https://scorebooknav.org

• Enter your username and password firstnamelastname(all lower case; no special characters)

• Accept license agreement

• Upon initial login, you will be prompted to update your password 

• Use “Forgot password?” button to email password to address on file after first login

33

Page 18: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors
Page 19: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Review the Framework

• The Criteria (p. 4‐30)

• Category and Item Commentary (online)

• Scoring System and Scoring Guidelines (p. 31‐36) 

• Criteria Response Guidelines (p. 37‐39)

• Core Values and Concepts (p. 40‐44)

• Changes from 2015‐2016 (p. 45‐46)

• Glossary (p. 47‐54)

• Index (p. 69‐71)

38

Page 20: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Read the Application Case Study

• Read the application all the way through, from beginning to end

• Yes – really!

• Get a broad overview of Green Gateway

• Highlight and take notes on key processes 

• Note results that you expect to see

39

40

Page 21: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

4141

42

Page 22: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

43

4444

Page 23: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Create a List of Key Factors

45

• What is a “key factor”?

• Where do I find key factors?

• How and where do I record key factors?

A key factor is an attribute of an organization or its environment that influences the way the organization operates and the key challenges it faces. Examiners will use key factors to focus their assessments on what is important to the applicant.

46

Page 24: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

47

48

Page 25: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Sample Key FactorP.1a

Product Offerings

Applicant provides 3 main lawn tractor products: Commercial (50%); 

Homestead (30%); & Putting Green (20%)

49

Exercise 1 – INDIVIDUAL (step 1)

• Read the assigned area from the Green Gateway Organizational Profile.

• Independently identify three key factors for the assigned area.

50

Page 26: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Exercise 1 – TABLE (step 2)

As a team, agree on at least 3 key factors for your assigned area and write them on your flip chart. 

Prepare to discuss a lesson learned.

Write BIG (not little)

51

Page 27: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors
Page 28: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Log on to the system

Review the Framework

Read the Application

Create Key Factor List

Print 2 copies of assignment

Note # of hours

Complete pre‐work evaluation

Attend 3‐day training

1.12.13.16.1

•Criteria

•Key Factors

•Application: PROCESS

•6‐10 Comments (ADLI)

•Score

•1 Feedback‐Ready OFI

7.17.2

•Criteria

•Key Factors

•Application: RESULTS

•6‐10 Comments (LeTCI)

•Score

•1 Feedback‐Ready OFI

Pre‐work Assignment

55

1.1

2.1

3.1

6.1

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application to identify processes and gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using process evaluation factors (A/D/L/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

7.1

7.2

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application; group results and identify gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using results evaluation factors (Le/T/C/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

Item Evaluation Steps

56

Page 29: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

58

Page 30: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

59

60

Page 31: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Exercise 2 ‐ TABLE

Select key factors for item 5.1

1. Review the overall requirements for item 5.1.

2. Using the master list identified by the class, select 

the 4‐6 most relevant key factors to keep in mind when 

evaluating the applicant’s response to item 5.1.

3. Write these key factors on the                                                     

flipchart.

4. Report out a lesson learned. 

61

1.1

2.1

3.1

6.1

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application to identify processes and gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using process evaluation factors (A/D/L/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

7.1

7.2

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application; group results and identify gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using results evaluation factors (Le/T/C/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

Item Evaluation Steps

62

Page 32: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Identify Key Processes and Gaps

• Identify the processes or approach the applicant uses to meet item requirements

• Gaps exist when the applicant fails to address either

– A key requirement of the Criteria, OR

– A relevant key factor

• Easiest to do one area at a time!

63

What If…

It sounds too good to be true?

64

Page 33: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

The Tinkerbell Principle

65

“Clap your hands if you believe!”

– Peter Pan

Page 34: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

5.1b(1) GG ensures workplace health by providing health assessments, flu vaccinations, health screenings, coaching for wellness, and Wellness Committee activities. GG ensures workplace security through automated security devices for doors and gates that require unique codes or GG‐issued entry badges, security monitoring systems, surveillance systems, and promptly removed access for terminated associates. Electronic accessibility to the GG network is provided by IT staff members who assign each employee a unique login, with password changes required quarterly. In addition, associates who need remote access to GG servers and files are provided access by the IT Department. The GG facility also has only one point of common public access to ensure workplaces security. GG ensures workplace accessibility by ensuring that the facility is handicapped‐accessible by meeting the International ISO Accessibility Standards, added in the 2016 cycle of improvement. 

Performance measures and improvement goals for workplace environmental factors are compliant with the OSHA 18001 Standards for Occupational Health and Safety. Results are indicated in Figure 5.1‐3, with additional results available on‐site. 67

Page 35: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Exercise 3 ‐ INDIVIDUAL

Step 1:

• Read the Criteria for your assigned area

• Consider the key factors you selected for item 5.1. 

• What would you expect to see in the application response?

Step 2:

• Read the application response.

• Independently identify one process that Green Gateway has in place that addresses the item requirements – or note a gap.

Table  Assignment

1 & 5 5.1a(1)

2 & 6 5.1a(2)

3 & 7 5.1a(3)

4 & 8 5.1b(2)

69

1.1

2.1

3.1

6.1

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application to identify processes and gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using process evaluation factors (A/D/L/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

7.1

7.2

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application; group results and identify gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using results evaluation factors (Le/T/C/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

Item Evaluation Steps

70

Page 36: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Evaluating Key Processes

• Analyze applicant’s response:

– Approach (methods)

– Deployment (extent to which the approach is applied)

– Learning (cycles of improvement)

– Integration throughout organization

• Record your thoughts on scratch paper or in the “As Evidenced By” box in Scorebook Navigator

• Note any results you expect to see in Category 7

71

Process Evaluation FactorsA = Approach: Is the process appropriate and effective? Is it systematic (repeatable and based on reliable data an information)?

D = Deployment: Is the approach used consistently by all appropriate work units?

L = Learning: Is there a method to improve the process? Does it encourage change, innovation, and best practice sharing?

I = Integration: Is the process aligned with what the applicant said was important? Does it connect to other processes or support the Strategic Plan?

72

Page 37: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

An Example

73

5.1b(1) GG ensures workplace health by providing health assessments, flu vaccinations, health screenings, coaching for wellness, and Wellness Committee activities. GG ensures workplace security through automated security devices for doors and gates that require unique codes or GG‐issued entry badges, security monitoring systems, surveillance systems, and promptly removed access for terminated associates. Electronic accessibility to the GG network is provided by IT staff members who assign each employee a unique login, with password changes required quarterly. In addition, associates who need remote access to GG servers and files are provided access by the IT Department. The GG facility also has only one point of common public access to ensure workplaces security. GG ensures workplace accessibility by ensuring that the facility is handicapped‐accessible by meeting the International ISO Accessibility Standards, added in the 2016 cycle of improvement. 

Performance measures and improvement goals for workplace environmental factors are compliant with the OSHA 18001 Standards for Occupational Health and Safety. Results are indicated in Figure 5.1‐3, with additional results available on‐site. 74

Page 38: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

A‐D‐L‐I AssessmentProcess: Workforce Climate

• Approach: + The applicant provides:

• Health assessments and screenings, flu shots, a reimbursable fitness center, and wellness coaching. 

• Security is ensured through automated security devices for entrances, entry badges, and surveillance systems. 

• Other benefits include 401k, health insurance, safety shoe reimbursement, and tuition reimbursement.

• Deployment: ‐ Not clear how these options are provided to temporary workforce members

• Learning: +/‐mentioned 2016 cycle of improvement but not clear how that process works

• Integration: +/‐measures for health, security, and accessibility (see Figure 5.1‐3); not clear how offerings and security differ for various work environments (headquarters vs. shop floor)

76

Page 39: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Exercise 4a ‐ INDIVIDUAL

Using the worksheet, independently evaluate a key process: A‐D‐L‐I

77

Selecting Comments

• Review the gaps you noted in the applicant’s response to the Criteria• Are any of the gaps significant to the applicant? • If so, these are good sources of OFIs.

• Review your assessment of the applicant’s processes.• Think vertically (responses to the areas of the Criteria)…• And horizontally (A‐D‐L‐I)

• Choose 6‐10 strength and OFI comments that are most valuable to the applicant

78

Page 40: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Outlining Comments in Scorebook Navigator • Select the relevant key factors for the comment

• Is it a double strength or OFI?

• Write the “Topic Sentence” in the “Strengths” or “Gaps and OFIs” box• What is the main point of your comment?

• Is it based on the Criteria?

• How does it link to the evaluation factors?

• Give supporting “Evidence” in the following box labeled “As Evidenced By…”

• Check A‐D‐L‐I

• Indicate the item reference (for example, 5.1b(1))

• Don’t worry about the key… for now  79

A‐D‐L‐I AssessmentProcess: Workforce Climate

• Approach: + The applicant provides:

• Health assessments and screenings, flu shots, a reimbursable fitness center, and wellness coaching. 

• Security is ensured through automated security devices for entrances, entry badges, and surveillance systems. 

• Other benefits include 401k, health insurance, safety shoe reimbursement, and tuition reimbursement.

• Deployment: ‐ Not clear how these options are provided to temporary workforce members

• Learning: +/‐mentioned 2016 cycle of improvement but not clear how that process works

• Integration: +/‐measures for health, security, and accessibility (see Figure 5.1‐3); not clear how offerings and security differ for various work environments (headquarters vs. shop floor)

80

Page 41: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

An Example5.1b(1)

Strength:  The applicant has a systematic approach for ensuring a positive workforce climate

As evidenced by…

• It provides health assessments and screenings, flu shots, a reimbursable fitness center, and wellness coaching. 

• Security is ensured through automated security devices for entrances, entry badges, and surveillance systems. 

• Performance measures for the workplace environment include percent of buildings with engineered access methods and workforce affirmation of workplace health. 

• Other benefits include 401k, health insurance, safety shoe reimbursement, and tuition reimbursement.

81

An Example

5.1b(1)

OFI:  There is little consideration of the applicant’s varying 

workforce environments or temporary workforce members

As evidenced by…

• Not clear how OSHA requirements or air quality 

processes differ at headquarters vs. shop floor

• Not clear which of the organization’s workplace benefits 

(health assessments, etc.) apply to temporary workers

82

Page 42: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Exercise 4b ‐ TABLE

Outline a strength or OFI comment in Scorebook Navigator:

• As a table, select one strength or OFI topic sentence and and write it on the poster.

• Include supporting evidence.

• Note appropriate evaluationfactor(s) and item reference.

• Report out.

83

84

Page 43: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

85

86

Page 44: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

1.1

2.1

3.1

6.1

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application to identify processes and gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using process evaluation factors (A/D/L/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

7.1

7.2

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application; group results and identify gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using results evaluation factors (Le/T/C/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

Item Evaluation Steps

87

Score the Item

• Refer to Scoring Guidelines in Framework or in Scorebook Navigator on “Scoring” panel  

• Begin with 50‐65% range for “approach” and move up or down

• Pay attention to “basic,” “overall,” and “multiple” requirements

• Identify range that is the “best fit,” i.e., most descriptive

• Repeat for “deployment,” “learning,” and “integration”

88

Page 45: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Item format (page 2)

89

90

Page 46: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Score the Item

• Determine the “best fit” scoring range considering all four factors 

• Select a score within the range (in 5 point increments) 

• Record the score

91

Exercise 5 – TABLE 

1. Based on the class’s evaluation of item 5.1, discuss the appropriate scoring ranges for approach, deployment, learning, and integration.

2. As a table‐team, determinethe “best fit” scoring range for item 5.1 overall.

3. Select a percentage score for item 5.1 (in a 5% increment) and write it on the chart paper.

92

Page 47: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

9393

94

Page 48: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Write 1 OFI Comment

95

1.1

2.1

3.1

6.1

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application to identify processes and gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using process evaluation factors (A/D/L/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

7.1

7.2

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application; group results and identify gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using results evaluation factors (Le/T/C/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

Item Evaluation Steps

96

Page 49: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Finding the “Sweet Spot”

97

Organization (examples + key factors)

EvaluationFactors

Criteria requirements

=  value‐added feedback

Making Your Comment “Feedback‐Ready”

Your Outline

• Is in bulleted phrases

• Is based on factual evidence from the application (usually related to an evaluation factor)

• Is descriptive

• Communicates your insights to other examiners on your team

Feedback‐Ready Comments

• Are complete thoughts

• Are unified, coherent, well‐developed and provide value to the applicant

• Connect the Criteria, key factors, the applicant’s response and your analysis

• Use grammatical sentence structure

98

Page 50: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Comment Writing TipsIs the comment Criteria‐based? 

Is it insightful?

T = Start with the “topic sentence” or main point 

E =  Give 1 – 2 supporting examples from the application

S = Explain the “so what” (relevance) to the applicant by linking the comment to a key factor 

T = Try it out loud!

99

Comments Should…

• NOT exceed 75‐100 words

• NOT include dense Criteria language

• NOT be judgmental or prescriptive

• NOT focus on how well the application is written

• Let the applicant know 

what it does well

• …and what it needs to improve

• Include examples from the application

• Use complete sentences

• Contain a clear “so what”  

100

Page 51: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Sources of “so what”

101

• Key factors• Item notes

• Category and item commentary

• Core Values and Concepts 

• Glossary of Terms

• A‐D‐L‐I progression or gaps

• Le‐T‐C‐I progression or gaps

• Scoring guidelines

Constructing a Comment5.1b(1) There is little evidence of consideration of differing

workplace environments. For example, there is no mention of addressing OSHA compliance or maintaining air quality (Figure P.1‐7) differently in various areas of the facility, such as the shop floor, shipping/receiving or office areas. No health or security distinctions are made for different workplace environments or for workforce segments, such as temporary workers (Figure P.1‐5). Consideration of the various workplace environments may assist the applicant in addressing its strategic challenge associated with technical workforce retention.

102

Page 52: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Exercise 6 – TABLE 

As a table:

• Review the process or gap that you identified in item 5.1

• Write one feedback‐readystrength orOFI comment

103

104

Page 53: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Feedback

•Move to your assigned table.

• Review the written comment and provide feedback:– Is the comment grounded in the Criteria?

– Is it insightful?–What is the topic sentence?

– Is the comment supported with examples?

–So what?  Does the comment link to a key factor?

105

Evaluating Results Items

Where the rubber meets the road!

106

Page 54: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Results Items

• Results are outputs and outcomes achieved by an organization in addressing a Criteria item

• Results align with key requirements from customers, stakeholders, markets, processes, and the strategic plan

• Results items are in Category 7

107

Page 55: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

1.1

2.1

3.1

6.1

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application to identify processes and gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using process evaluation factors (A/D/L/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

7.1

7.2

•Criteria – Read the Criteria item requirements•Key Factors – Select 4‐6 most relevant key factors from master list•Application – Read relevant section of the application; group results and identify gaps

•6‐10 Comments– identify and outline 6‐10 strength and OFI observations using results evaluation factors (Le/T/C/I)

•Score the item •Feedback‐Ready OFI – Write one OFI comment

Item Evaluation Steps

109

Results Items• Group a set of results, rather than 

evaluating each graph separately

• Identify gaps (results not provided) from:

‒ Criteria

‒ Key factors

‒ Key processes you noted in categories 1‐6

110

Page 56: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Results Evaluation FactorsAnalyze applicant’s response:

Performance Levels (Le)Validity or meaningful scale

Trends (T) *Rate of improvement or sustainability

Comparisons (C)Appropriateness; benchmarks

Integration (I)Importance, harmonization, segmentation

* Note: Unfavorable trends may be explained in the narrative

111

Green Gateway Results

7.3a(2) GG provides a work environment that is safe, healthy, and productive. Figure 7.3‐8 shows top‐decile (against IW Best Plants comparison data) satisfaction with health, safety, security, accessibility, and benefits.

Figure 7.3‐9 (RTB BSC – People) outlines the turnover rate, which shows a stable workforce. Figure 7.3‐10 (RTB BSC – People) shows a favorable trend in associate and salaried absenteeism.

Page 57: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Green Gateway Results

113

GG Results

114

Page 58: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Outlining Results Comments• Based on your analysis, identify 6‐10 strength and OFI comments that are most valuable to the applicant

• Think vertically (Criteria items) and horizontally (evaluation factors) 

• Note relevant figure number(s)

• Note whether results are favorable or unfavorable

• Note item reference, relevant key factors

115

Score the Item

• Refer to Scoring Guidelines in Framework or Scorebook 

Navigator

• Begin with 50‐65% range for “levels” and move up or 

down

• Repeat for “trends,” “comparisons,” and “integration”

• Pay attention to “basic,” “overall,” and “multiple” 

requirements

• Identify range that is most descriptive (best fit)

• Assign a final score (multiple of 5)

116

Page 59: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

117

Page 60: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Constructing a Comment

7.3a Many results do not present data by workforce 

segments noted as important by the applicant. For 

example, other than segmentation for hourly vs. 

salaried vs. management, none of the results in this 

item are segmented by the diversity of the workforce 

profile (Figure P.1‐5). Additionally, there are no 

results for temporary workforce members, who 

represent 5% of the total. Tracking results by these 

important segments may help the applicant build on 

its core competency of relationship building and its 

organizational value of respecting others.

119

120

Page 61: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Scorebook Navigator™

121

Software Tips

• A Scorebook Navigator Tip Sheet has been provided online and in your packet

• Scorebook Navigator has been enabled to work with Internet Explorer, Firefox, Safari, or Chrome

• SAVE YOUR WORK AFTER EVERY ROW

• The system will time out after a period of inactivity

• When in doubt, call for help! TNCPE Office (800) 453‐6474 or (615) 889‐8323

122

Page 62: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

123

Two Quick Notes

• Copy/Paste – you MUST use the system copy/paste icon to transfer your work from Word

• Font Size CTRL + increases the font size (magnifies your screen)

CTRL – decreases the size CTRL 0  (zero) returns to normal

Page 63: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Wrap‐UpReview | Questions?

125

Log on to the system

Review the Framework

Read the Application

Create Key Factor List

Print 2 copies of assignment

Note # of hours

Complete pre‐work evaluation

Attend 3‐day training

1.12.13.16.1

•Criteria

•Key Factors

•Application: PROCESS

•6‐10 Comments (ADLI)

•Score

•1 Feedback‐Ready OFI

7.17.2

•Criteria

•Key Factors

•Application: RESULTS

•6‐10 Comments (LeTCI)

•Score

•1 Feedback‐Ready OFI

Pre‐work Assignment

126

Page 64: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Trust the Process!

I hear ... and I forget.

I see ... and I remember.

I do ... and I understand.

Ancient Chinese Proverb

127

Bring to Training• Two copies of your pre‐work

• Completed pre‐work evaluation form– Include time you spent on pre‐work

• All of your materials– Green Gateway Case Study

– Baldrige Excellence Framework– You will be given a copy of the Framework of your choice

– Your pre‐work packet

• Your TNCPE name badge (return examiners)

• Enthusiasm and a positive attitude!!!128

Page 65: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

2018 Examiner Training

Orientation

May 8, May 10*, May 15, May 17 and May 22

Three‐day training course

• June 5‐7  Nashville

• June 12‐14  Oak Ridge

• June 12‐14 Greensboro, NC

• June 19‐21 Memphis

• June 26‐28 Bristol

• July 10‐12 Nashville II129

Award Program Calendar

Intent to Apply due April 15(NO Organizational Profile!)

Application due (Level 2, 3, 4)* July 16(with Criteria Response and Org Profile)

Teams formed  late July

Examiner Assessment Aug. 1–Oct. 30

Judges review mid‐November

Deliver Feedback Reports by Dec. 31

Conference & Banquet Feb. 27‐28, 2019

* Level 1 Applications are accepted year‐round 

130

Page 66: TNCPE Examiner Orientation...4. Report out a lesson learned. 61 1.1 2.1 3.1 6.1 •Criteria–Read the Criteria item requirements •Key Factors –Select 4‐6 most relevant key factors

Katie Rawls, President & CEO

Sue Alexander, Manager, Administration

Tennessee Center for Performance Excellence

2525 Perimeter Place Drive, Suite 122Nashville, Tennessee  37214

Phone: 615‐889‐8323or 800‐453‐6474

E‐mail: [email protected]

www.TNCPE.org

131