copyright 2002-2004 minnesota council for quality1 the 2004 mqa site visit
Post on 26-Dec-2015
215 Views
Preview:
TRANSCRIPT
Copyright 2002-2004 Minnesota Council for Quality 1
The 2004 MQA Site Visit
Copyright 2002-2004 Minnesota Council for Quality 2
Today’s Agenda
• Purpose of the Site Visit• Elements of a Successful Site Visit• Site Visit Roles and
Responsibilities• Site Visit Preparation• Site Visit Process and Products• What Happens Next
Copyright 2002-2004 Minnesota Council for Quality 3
Baseline Questions
• Is this applicant use the Narrative or Baldrige-Express?
• Which Criteria are used?• How long is the site visit? (Usually 3
days.)
Copyright 2002-2004 Minnesota Council for Quality 4
Purpose of the Site Visit
• Understand key organization factors• Verify and clarify facts and site visit
issues• Prepare the feedback report• Score the organization• Assess commitment, sustainability,
and momentum• Present exit presentation
Copyright 2002-2004 Minnesota Council for Quality 5
Elements of a Successful Site Visit
People
Process Products
Roles,Ground rules,
Communication
Assignments,Controlled agenda,
Consensus building
Exit presentation,Feedback report,Judges’ presentation
Copyright 2002-2004 Minnesota Council for Quality 6
Evaluator Team Role & Responsibilities
• Gathers and documents facts for feedback report
• Communication with applicant is equally critical
– Take time to explain and educate – You represent the Council and help
sell the Council– Help make it a fun, pleasant
experience
Copyright 2002-2004 Minnesota Council for Quality 7
Applicant Representative Role & Responsibilities• Contributes information for findings• Arranges availability of interviewees and
materials• Helps team complete its agenda• Applicant Representative usually provides:
• List of key contacts• Organization chart• Facility layout• Optional requested material
Copyright 2002-2004 Minnesota Council for Quality 8
Presentation to Judges
Information Flow
Copyright 2002-2004 Minnesota Council for Quality 9
Site Visit Sequence
Advance Preparation
Final Preparation
Site Visit
1b.Exit
Presentation
1a.Executive Preview
2.Feedback
Report
3. Presentation
to Judges
Copyright 2002-2004 Minnesota Council for Quality 10
No
Yes
<= DAY 33
<= DAY 34
<= DAY 41
<= DAY 44
Stage 2: Consensus & Site Visit Planning
2.01Team Leads
assign Category / Itemleads & back ups
2.02Standard Message #3:
Team Leads CommunicateConsensus assignments& meeting arrangements
to team memberscc: Lead Judges &
Chief Judges &Council
2.04Team Leads
& Lead Judgesconfer
2.05 Are matrices
OK?
2.06Team Lead
coachesEvaluator(s)
2.07Team Lead
notifiesEvaluator
2.09Evaluators e-mail
completedConsensus Matrix
& Expected Results Tableto team
2.08Evaluators completeConsensus Matrices
& consolidatedExpected Results Tables
for assigned Items
2.03Evaluators completeConsensus Matrix for
first assigned Item& e-mails to Team Leads &
Lead Judges
2.11Team meets from8AM-3PM to reach
consensus on comments& draft KTs
2.10Evaluators review all
materials beforeConsensus meeting &
identify possiblediscussion points <= DAY 47<= DAY 45
<= DAY 44 2.14Council removes
Evaluator(s) to SVsize if needed;Council sendsSV material
2.17 Are comments
& SVIs OK?
<= DAY 48
SV Strategy WorksheetsExit presentation template
Judges presentation templateFeedback Report template
2.15Category/Item Leads
prepare SV Issuesheets & e-mail to team
& Lead Judges
No
Yes GO TOSTAGE 3
2.18Team Leads
revise Commentsand/or SVI Sheets 2.16
Team Leads& Lead Judges
confer on:comments, SVI's,
storyboardissues, KTs,walkarounds
2.12Participant rep.arrives (3PM);
trainers provideJIT SV training
2.13Category Leads
e-mail Consensuscomments to
team &Lead Judges
<= DAY 30
We are at step 2.12
Copyright 2002-2004 Minnesota Council for Quality 11
Advance Preparation• Dates set• Expectations set• Site visit issues compiled – SV Issue sheets
– Prepare list of documents needed and individuals to be interviewed on site – use a matrix to help coordinate requests for meetings and documents
• Site visit agenda set• Logistics arranged
– Hotel– Emergency contact number 612-462-3577– Name tags (as needed)– Expense accounting
Copyright 2002-2004 Minnesota Council for Quality 12
Site Visit Planning WorksheetSite Visit Date:________________ Site Visit Location:________________ Evaluator:________________
Category:2 and 5Item Ref #: 2.2 and 5.2
Issue: Verify the content of the Human Resource Plan (2.2) and clarify its use in identifying the strategic training needs for the organization (5.2)
Person(s) to Interview: HR Manager
Document(s) or Information to Review:Human Resource Plan
Sample Questions:Tell me about HR plans you create. How do you use these?What is an example of training that was computed in the past year?
Findings:
Verify or Clarify
Copyright 2002-2004 Minnesota Council for Quality 13
Focus of Site Visit Issues
• Information that is missing or vague.• Strengths or weaknesses identified from
narrative, Baldrige Express report, results data, or organizational profile.
• Deployment of the required practices.• Cross-cutting issues or themes• Site Visit Issues should be written on the
“vital few” – there should not be more than 3-4 per item.
Copyright 2002-2004 Minnesota Council for Quality 14
Site Visit Techniques• Acceptable Techniques
• Reviewing data, reports, documents hard-copy or electronic as provided directly by the applicant
• Interviewing individuals and teams• Receiving presentations by participating
organization
• Unacceptable Techniques• Conducting focus groups or surveys with
customers, suppliers, or dealers• Disrupting work processes• Interviewing sources or collecting data beyond
that provided by the applicant • This includes conducting library or internet This includes conducting library or internet
research and visiting web sites.research and visiting web sites.
Copyright 2002-2004 Minnesota Council for Quality 15
“ Typical” Site Visit IssuesVerify/Clarify• Role and activities of senior management (driving customer
focus, empowerment, innovation, learning)• Degree of involvement and empowerment of employees • Reliable and accessible data and information • Use of facts in decision-making• Focus on customers• Systematic approaches to management• Training effectiveness, on-the-job reinforcement• Alignment of recognition and rewards to support strategy,
customer focus, and key business outcomes• Use of strategic objectives, action plans, and related measures
to align work at all levels—links to performance feedback and employee rewards
• Evidence of evaluation and improvement• Integration of all processes—both core business and support• Supplier involvement in performance improvement activities
Copyright 2002-2004 Minnesota Council for Quality 16
“Typical” Walk-around Questions
• What are the values, mission, and vision of the organization?
• What are the action plans for your work unit? • What are the goals?• What role do you play in achieving them?
• What kind of training do you receive?• Is it useful to your work? • How is training reinforced on the job?
• How are you empowered to take actions needed to carry out or improve your work? • How are you involved in decision-making about
work?
• What is recognized or rewarded here?
Copyright 2002-2004 Minnesota Council for Quality 17
<= DAY 57
Stage 3: Site Visit
3.04Team arrives
at site
3.01Team arrives
at site visit hotel
<= DAY 583.03
Team preparespreliminary
Story Board *
3.02Team conducts
final prep meetingevening before(see agenda)
3.06Team conducts
preliminary caucus
3.05Participant KickOff presentation& introductions(optional tour)
3.07Team conducts site visit:
interviews, data review, walkarounds;frequent team caucuses; daily revision
of storyboard (reviewed with HRO at EOD)and comment worksheets/SVIs
<= DAY 60
3.09Team Leads/Council
meet w/Participant HRO
3.08Team prepares
exit presentations
<= DAY 60<= DAY 60<= DAY 60
3.10Team makes
exit presentation
3.11Team finalizes
Item comments and scores
3.12Team completes
Executive Summary(KTs), PGPs, Judges'
questions
3.14Team provides improvement
input for process, completes 360reviews, conducts final checklist,
& leaves site/hotel
<= DAY 60
GO TOSTAGE 4
* The story board process is coveredin the site visit training.
3.13Team Leads
appoint feedbackwriter (optional)
Copyright 2002-2004 Minnesota Council for Quality 18
• Final prep meeting held evening before site visit– Review SV issue sheets– Set up key themes worksheet/storyboard– Establish walk-around questions– Finalize the agenda, assignments, pairs
• Each Evaluator brings draft of site visit issue worksheets, to prepare a composite set
• Applicant representative meets with the team• Aid in planning the time on site• Share general schedule conflicts
Final Preparation
Copyright 2002-2004 Minnesota Council for Quality 19
Final Preparation (continued)
• Review and update documentation list as needed
• Review and update list of individuals to interview on site • This will help you identify and manage
hand-off issues.• Finalize the matrix of document and
meeting requests the team created earlier in the processes
Copyright 2002-2004 Minnesota Council for Quality 20
Site Visit Interview Matrix
Individual to Interview Site Visit Issue Ref.
Sheila Knight, Marketing 1.1.1, 2.2.1, 3.1.1, 3.1.2, 3.1,3, 3.2.1, 3.2.4, 4.1.1, 4.1.3, 4.2.1, 4.2.2, 7.1.3
Ron McDonald, Administration 1.1.1, 1.1.3, 1.2.1, 1.2.2, 7.2.3
Mary Stewart, Employee Development
1.1.1, 2.1.1, 5.1.1, 5.1.3, 5.2.1, 7.3 all
Alex Hamilton, Operations 1.1.1, 2.1.1, 6.1.3, 6.2.1, 6.2.2, 7.5 all
John Jones, Finance 4.1.4, 4.2.1, 4.3.2, 7.2 all
Leader/manager of Process Improvement Team & members of PI Team
6.1.5
Individual responsible for coordinating leadership communication
1.2.5, 2.2.2
Individual responsible for Support Services Process Improvement Meetings with Leadership
6.2.5
Copyright 2002-2004 Minnesota Council for Quality 21
Sample Site Visit Schedule
Monday
Vanessa (1), (2) 7.2Walt (3, 7.1)
Roy & (4, 5, 7.3) Deana (6, 7.4)
7:00 Breakfast
7:30 Travel to Site
8:00 Opening Meeting
9:00 Ron McDonald, CEO & Leadership Team Meeting
Worker Services Team Team Meeting
10:00 Category Lead Meeting, then Quick Caucus
11:15 One on ones with Eos during lunch breaks
1:30 Nate O. Forsight Planning VP
Mary Stew, Dir IT Development
2:35 Caucus
3:00 Customer Team * Sheila Knight (Mktng)
Alex Hamilton, Operations
4:15 Walk around, Shift Change Interviews
5:00 Caucus, revise plan in detail for Day 2
Copyright 2002-2004 Minnesota Council for Quality 22
Site Visit Agenda• Opening meeting
– Introductions to the applicant (MCQ/Team Leads)– Participant presentation (up to one hour)
• Facilities tour (optional)• Scheduled meetings with key applicant
resources – Allow time for travel when necessary– Schedule applicant interviewees
• Scheduled times for walk-around interviewing of applicant employees
• Scheduled times for team discussion (caucuses)
• Executive preview and exit presentation
Copyright 2002-2004 Minnesota Council for Quality 23
Managing Issues and Handoffs• Create 7 flip charts, one for each
category.
• Ask Evaluators to use yellow sticky notes to record issues they have to hand off to the category owner in the “open” area and red for new issues.
• When the information has been collected, record the answer and move the sticky to “closed” portion.
• This way everyone can keep track of open and closed issues and effectively manage handoffs.
Category 1
Open
Closed
Verify deployment
between shifts
Ask CEO about employee focus (4)
Copyright 2002-2004 Minnesota Council for Quality 24
Key Themes Worksheet/Storyboard• A key theme is a Strength or OFI that is
common to more than one Item or Category, is especially significant in terms of the applicants Key Factors, and/or addresses a Core Value of the Criteria.
• Provides key points and an overall summary of the team’s evaluation of the applicant.
• Divided into three sections to address three questions concerning important Strengths, significant OFIs, and key Results
• Will be used to help prepare the exit presentation
Copyright 2002-2004 Minnesota Council for Quality 25
Evaluator Team Caucuses
• Compile findings• Review feedback comments• Update SV issue sheets and key themes• Hand-off any outstanding questions• Remember - applicant representative
does not participate except for logistical issues and planned review of key themes
• Be sure to schedule enough time for caucuses plus some personal break time
Copyright 2002-2004 Minnesota Council for Quality 26
Structure for Interviewing• Begin the interview with:
– Introduce yourself and why you’re there•We will respect confidentiality of information
•We will be looking for both Strengths and OFIs
– Ask if they have questions– Explain why you are taking notes– Describe the time limits
• End the interview with:– Do you have any questions for me?– Thank them for their time and honesty
Copyright 2002-2004 Minnesota Council for Quality 27
Effective Interviewing Tips
• Refocus when they ramble• Ask open-ended questions, probe when
necessary, bring it to a conclusion• Script your lead and follow-up questions• Be frank and respectful
– Use the applicant’s terminology– Actively LISTEN to the applicant
• Be patient with shy interviewees
Copyright 2002-2004 Minnesota Council for Quality 28
Asking QuestionsUse open-ended questions•Tell me about …•How do you …•Help me understand …•Please walk me through …
Avoid Leading questions.• Why don’t you – • At my office we do it this way. What do you do?
•Have you always done it this way?•Why did you change?•How did you know what to change?
•Don’t you think it would be a good idea to re-engineer your manufacturing process?
•How do you determine how frequently to check this process?
•Why do you only audit this process every two years?
•How do you know customer requirements are correctly defined?
•Why don’t you involve customers in your design-build teams?
Copyright 2002-2004 Minnesota Council for Quality 29
Typical Daily Progress (Day 1)
• Advanced level of understanding of business factors
• Close some Site Visit Issues (typically Cat. 1 and 2)
• Establish remaining agenda• Brief applicant on progress and
outstanding issues• Category Owners begin editing their
comments
Copyright 2002-2004 Minnesota Council for Quality 30
Typical Daily Progress (Day 2)
• Confirm key themes• Close 100% of remaining Site Visit Issues• Identify remaining information/interviews
needed (if any – there should not be any)• Finalize plan for Day 3• Brief applicant on progress, outstanding
issues, and plans for Day 3• Finalize 100% of the feedback comments
Copyright 2002-2004 Minnesota Council for Quality 31
Typical Daily Progress (Day 3)
• Close remaining issues (if any)• Prepare and deliver executive preview and exit
presentation• Finalize feedback report
– Category Owners recommend final wording for comments– Entire team reviews and signs off on items as completed
• Identify any Pretty Good Practices• Score• Assess commitment, sustainability, and momentum• Complete logistics (hotel checkout, return of
applicant materials, collection of shredding, 360 feedback)
Copyright 2002-2004 Minnesota Council for Quality 32
Preparing the Exit Presentation
• Review the key themes worksheet– Significant Strengths, including Pretty Good
Practices– Significant Opportunities for Improvement
• Prepare the PowerPoint presentation• Identify presenters
Copyright 2002-2004 Minnesota Council for Quality 33
Executive Preview
• Applicant’s option• Team Leads only• Council representative also present• Summary review of exit presentation
content• Opportunity for clarifying questions• Expression of thanks for cooperation
and participation
Copyright 2002-2004 Minnesota Council for Quality 34
Exit Presentation
• One hour• Moderated by MCQ representative• Audience determined by applicant• Review key themes
– Significant Strengths and Pretty Good Practices
– Significant Opportunities for Improvement
• Clarifying Q&A• Thanks
Copyright 2002-2004 Minnesota Council for Quality 35
Scoring• Assign scoring facilitator• Determine order for scoring items
– Allow 5-10 minutes maximum per item
• Review scoring guidelines, terminology(refer to Clarifying Requirements for Basic-, Overall-, and Multiple-
Level Scoring)
• For each item– Category Owner reviews high points of comments– Team members individually determine scores– Team displays scores on facilitator’s signal– Team negotiates variances
• Be mindful of item point values• Don’t bog down
• Record percent scores on scoring matrix Remember--It’s not over till it’s over
Copyright 2002-2004 Minnesota Council for Quality 36
Preparing the Feedback Report
• At site– Category owners prepare final wording of comments– Team reviews and signs off on comments– Team reviews judges’ questions
• After site (within one week)– Feedback editor collates comments into feedback
template– Feedback editor writes executive summary based on Key
Themes– Feedback editor distributes report to team for feedback– Team leader forwards finished feedback to lead judges
• Next steps (approximately two weeks)– Judges review and edit final report and forward to Council– MCQ forwards finished report to applicant
Copyright 2002-2004 Minnesota Council for Quality 37
Reminder About Comments
• Review writing rules for all comments (refer to your training manual)
• Category 7 comments must:– Address results, not approach, deployment,
integration, or learning– Reflect the measures the applicant says are
important (expected results table)– Be stand-alone and quantified– Address specifics of levels, trends, and
performance relative to comparisons
Copyright 2002-2004 Minnesota Council for Quality 38
Stage 4: Post-Site VisitIMPROVEMENT PLANNING/JUDGING
4.02Council sends
post-site scheduleto TLs, Judges
<= DAY 62
4.03Feedback Writer completes
draft FR & e-mails TL(optional, if FW assigned)
4.04Team Leads editreport & e-mail to
Lead Judges
<= DAY 72
<= DAY 79
4.07Council sends final FRto TLs, Lead Judges, & Participant contact
(as appropriate)
<= DAY 81
4.08Council finalizes IPagenda/approach
w/Participant
4.09TLs/Council prepare
for IP Session
4.11Council sends thank youand satisfaction survey toParticipant (OCP, HRO)
4.01Council confirms
time/location for IPSession w/Participant& sends to full team
4.10TLs/Council participate
in IP Session: review FR,prioritize opptys, create
draft action plans(as appropriate)
4.05Lead Judge(s)work with TLsto create final
draft report
<= DAY 82
<= DAY 89
4.06Council makes
final edits
4.13TLs work withLead Judge(s)
to create Judges'Presentation; send to
Council
4.14Council sends agenda,Judges' presentation,
Org Profile, Cat 7,& FR to full POJ
4.15Council preparesscoring profiles &
loads PPT Judges'Presentation(s)
4.12Council confirmstime/location forJudging Session;
sends invite toEval. team(s)
4.18Panel of Judges
deliberates & reachesconsensus on level;if questions, contact
TLs by phone
4.17Lead Judges
present case to POJ& recommend
MQA level
4.16Team Leads
present to POJ(see agenda)
4.20Team Leads
inform team membersof decision (not forpublic consuption)
4.19Council communicatesrecognition level & key
messages to ParticipantHRO & Team Leads
NOTE: Improvement Planning sessions arescheduled individually with each applicant;
Judging Sessions are batched throughout theyear. The process above is typical (IP before
Judging), though on occasion Judgingprecedes the IP Session.
END OFJUDGING PROCESS
END OFASSESSMENT
PROCESS
Copyright 2002-2004 Minnesota Council for Quality 39
Judges’ Presentation Agenda
• Key Factors – 2 minutes• Key Themes: Process Strengths – 3
minutes• Key Themes: Process OFIs – 3 minutes• Key Themes: Results – 6 minutes• Significant Differences (if any) – 1 minute• Commitment, Sustainability, and
Momentum – 5 minutes• Questions – 10 minutes
Copyright 2002-2004 Minnesota Council for Quality 40
Judges’ Presentation - CommitmentThe extent to which leaders are willing to
invest time, resources and energy in making the journey.
An example of low commitment: leaders spend little time on quality or planning for improvements – few resources are made available for improving quality systems and approaches
An example of high commitment: leaders consistently encourage and lead efforts to improve the organization – leaders plan for high levels of performance – leaders allocate resources to the effort
Copyright 2002-2004 Minnesota Council for Quality 41
Judges’ Presentation - SustainabilityThe extent to which the organization is likely
to continue making progress on its journey
An example of low sustainability: The quality journey may be seen as just a management fad – “something the boss wants us to do.” If the leader of the quality effort were to disappear from the scene, the journey would stop.
An example of high sustainability: “Managing quality is everyone’s job. We need to excel as an organization in order to survive and prosper. It’s not just the boss’ thing.” If there were a change in senior leadership, progress would continue.
Copyright 2002-2004 Minnesota Council for Quality 42
Judges’ Presentation - Momentum
The direction and rate of progress on the quality journey.
An example of low momentum: There may be slow or no progress overall from year to year, although some pockets of rapid or steady growth may exist. The organization may be little aware of its opportunities for improvement or how to close the gaps. Perhaps, improvements in the past may have been lost and are no longer apparent.
An example of high momentum: Innovation and improvement in business practice is frequent and pervasive. Baldrige scores may have been rising each year through multiple cycle of self-assessment and improvements.
Copyright 2002-2004 Minnesota Council for Quality 43
What Happens After the Site VisitFinal judging Applicant receives feedback reportImprovement Planning SessionCelebration!
top related