©2014 mfmer | slide-1 unlocking a quality management system’s potential – key components and...

©2014 MFMER | slide-1

Unlocking a Quality Management System’s Potential – Key Components and Self-

Monitoring Tools

©2014 MFMER | slide-2

Agenda

• Introduction to the Research shield at Mayo Clinic

• Circumstances that led us to a Quality Management System (QMS)

• Key components of a QMS

• Self-monitoring tools

• Control limits and defined escalation

©2014 MFMER | slide-3

Objectives

• Explain key components of an effective QMS

• Illustrate helpful tools that can be used to self-monitor

• Discuss importance of control limits and defined escalation processes

©2014 MFMER | slide-4

Mayo Clinic Research Shield

• Physicians actively engaged in research ~ 640

• Individuals certifying research effort > 4000

• Research space > 900,000 sq. ft.

• Grants and contracts > 4700

• In the top 25 NIH grant funding by institutions

• Active Institutional Review Board-approved human research studies > 8,900

• Research and review articles in peer-reviewed journals > 5,700

©2014 MFMER | slide-5

Research Funding Sources

TOTAL RESEARCH BUDGET

$634 million

Total extramural $386 million

Mayo institutional support $248 million

39%

EXTRAMURAL FUND-ING

Federal & state $304 million

Industry $44 million

Other extramural $38 million

79%

10%

61%

11%

©2014 MFMER | slide-6

Research Finance at Mayo Clinic

• Research Finance is an enterprise wide division with staff working in Arizona, Florida, and Rochester. The team is devoted to ensuring compliant financial processes for post award activity for all funding types.

• Organizationally in the Finance Department

• Research Finance has employees working across 7 functional teams

• Award Management• Award Compliance • Award Revenue• Cores• Program Services• Reporting• Quality

©2014 MFMER | slide-7

Mayo Clinic Research Finance Historical Staffing

• Staffing numbers• Pre Department of Justice (DOJ)

investigation – 2003 – 25 FTE

• Post implementation of controls and launch of QMS – 2008 – 100 FTE

• QMS additions and enhancement, Self-monitoring in place, Working Differently initiative – 2013 – 88 FTE

©2014 MFMER | slide-8

DOJ Investigation and Settlement

Whistleblower (qui tam) suit filed – Agents visit

DOJ press release and investigation closed

Mayo settled for $6.5M

Reviewed NIH funds spent from 1992-2003

November2003

May 2005

June 2005

November 2003 - May 2005

©2014 MFMER | slide-9

Transition

• In June 2005, NIH sent a letter of inquiry asking for compliance related information and assurances related to 4 areas:

• Cost Transfers• Financial Management Systems• Written Policies and Procedures• Roles, Responsibilities, and Training

©2014 MFMER | slide-10

Mayo Clinic Response (2005-2008) Building the Framework

• Conducted comprehensive gap analysis

• Identified opportunities for improved compliance controls

• Self-disclosed to NIH self-identified compliance issues

• Met with NIH officials in Bethesda, multiple times to provide comprehensive updates

©2014 MFMER | slide-11

Mayo Clinic Response (2005-2008) Building the Framework

• Executed 26 projects• Automation• Policies & Procedures• Roles & Responsibilities• Tools

• Implemented new Enterprise Resource Planning (ERP) system, including Grants Management module

©2014 MFMER | slide-12

Strengthening the Framework (2008-2013)

• In 2008, adopted a Quality Management System, modeled after International Organization of Standardization (ISO) framework

• Cornerstones remain; • Added continuous improvement,

competencies, and customer satisfaction elements

• Enhanced training, measurement and analysis and our documentation system

©2014 MFMER | slide-13

Quality Management System - Key Components

Continuous Improvement

Customer Service &

Satisfaction

Culture

QMS Administration

Documentation System

Management Commitment

Measurement & Analysis

Competence, Awareness & Training

Start

©2014 MFMER | slide-14

Research Finance Mission Statement

• Mayo Clinic Research Finance believes in quality management. Research Finance aspires to use quality as the vehicle behind all it does in an effort to meet and exceed our customers’ expectations. The Research Finance Quality Management System emphasizes a proactive approach to meeting our customers’ needs and to ensure continuous improvement.

Management Commitment

©2014 MFMER | slide-15

Establishing a Quality Culture

• Work environment

• Readiness

• Employee engagement and empowerment

• Expectations

• Accountability

• Promote teamwork

• Promote continuous improvementCulture

©2014 MFMER | slide-16

Self-Monitoring Tools

©2014 MFMER | slide-17

QMS Annual Plan

• Critical part of a Quality Management System

• Identifies long term goals and current year objectives

• Commits required resources in writing

• Provides staff with direction for coming year

Management Commitment

CultureQMS Administration

©2014 MFMER | slide-18

Annual Plan Content

Management Message:

Communicates expectations, incorporates QMS elements and conveys that the QMS plan is owned by the department

Define Scope:

Identifies area that QMS supports

Purpose of the Department’s Quality Plan

Describes actions taken to ensure core business processes and QMS are continually improved

©2014 MFMER | slide-19

Annual Plan ContentGoals:

Communicates long term purpose for plan, doesn’t need to be measureable

Objectives:

Define time frame, objectives are specific and measureable

Activities by QMS Element

Align QMS work identified within the plan with the eight elements

Quality Plan Administration

Identifies responsibilities of the plan administrator

©2014 MFMER | slide-20

• Policies, Procedures and Work Instructions

• Policy – organization’s official position setting an expectation of behavior

• Procedure – high level of roles and responsibilities explaining how it happens

• Work Instruction – documents how to perform the work

• Record Retention

Documentation System

Documentation System

©2014 MFMER | slide-21

• Stores policies, procedures and work instructions

• Automated emails or monthly reports • Identifies person/area responsible for

documents• Check in/out, version control, and historical

content available

Content Management System

©2014 MFMER | slide-22

Competence, Awareness & Training

• Employee performance appraisals

• Learning management system

• Training • Education based format (utilize Bloom’s

Taxonomy)• Competency assessment

Competence, Awareness & Training

©2014 MFMER | slide-23

Bloom’s Taxonomy

http://nashvilleisd.groupsite.com/uploads/files/x/000/04b/a15/performance_verbs_for_learning_objectives.png

©2014 MFMER | slide-24

Standard Training Format

• Title

• Objectives utilizing Bloom’s Taxonomy• Define Cost Sharing

• Introduce and discuss each objective

• Test your knowledge questions• “____________or matching means that portion of

project or program costs not borne by the Federal Government.”

• Summary restating objectives• Cost sharing is the portion of project costs not borne

by the Federal Government.

©2014 MFMER | slide-25

Bloom’s Taxonomy

©2014 MFMER | slide-26

Standard Training Format

• Title

• Objectives utilizing Bloom’s Taxonomy• Define Cost Sharing

• Introduce and discuss each objective

• Test your knowledge questions• “____________or matching means that portion of

project or program costs not borne by the Federal Government.”

• Summary restating objectives• Cost sharing is the portion of project costs not borne

by the Federal Government.

©2014 MFMER | slide-27

Competency Plan

1. Conduct needs assessment

2. Identify competencies addressed by job

3. Write goals and objectives

4. Determine teaching methods

5. Determine assessment methods

6. Determine program improvement methods

©2014 MFMER | slide-28

Measurement and Analysis

• Balanced Scorecard

• Outcomes • Financial • Processes • People

Strategic Plan

Measurement & Analysis

©2014 MFMER | slide-29

Measurement and Analysis

• Compliance - Internal Control Evaluation (C-ICE)

• Scoping• Flows & Controls• Testing• C-ICE Metrics

©2014 MFMER | slide-30

Customer Service & Satisfaction

What is customer service?

Customer service is exceeding the needs of both internal and external customers.

Tools:

• Event Management

• Voice of the Customer (VOC)

Customer Service & Satisfaction

©2014 MFMER | slide-31

Continuous Improvement

1. Establish Scope

2. Identify & Define

Reportable Categories

3. Report Results

4. Outline

EscalationQuality

Assurance

ContinuousImprovement

©2014 MFMER | slide-32

Quality Assurance Controls

90 % of all QA findings are process issues

©2014 MFMER | slide-33

Continuous Improvement

1. Establish Scope

2. Identify & Define

Reportable Categories

3. Report Results

4. Outline

EscalationQuality

Assurance

©2014 MFMER | slide-34

Quality Assurance Control Chart

©2014 MFMER | slide-35

Quality Assurance Tools

©2014 MFMER | slide-36

Project Management vs. DMAIC

When to use DMAIC

Improves an existing process

Required steps for DMAIC

CharterStakeholder AnalysisCommunication planProject plan/timelineClosure

Value Stream MapEstablish and validate measurement systemData collection

When to use Project Management

Establishes a new process

Required steps for Project Management

CharterStakeholder AnalysisCommunication planProject plan/timelineClosure

©2014 MFMER | slide-37

Project Management (PMBOK)

©2014 MFMER | slide-38

Linking DMAIC and Change

* Many change management tools can (and should) be used in more than one phase.

©2014 MFMER | slide-39

Control Limits and Defined Escalation Processes

©2014 MFMER | slide-40

Self-Monitoring QMS

• Identify controls • Visual • Easily understood

• Defined escalation• Communicated• Administered

©2014 MFMER | slide-41

• Policies, procedures and work instructions warehoused in content management system

• Expectations and due dates communicated

Documentation Controls

©2014 MFMER | slide-42

Balanced Scorecard

©2014 MFMER | slide-43

Documentation Controls

Definition: Monthly summary of Research Finance documents that have been filed by the due date

Calculation Description: Total Records Retained (paper & electronic) divided by Total Records Requiring Retention

Long Term Goal: 100 % resolution - 95% or greater is green, less than 95% is yellow and less than 90% is red

Metric Steward: John Smith

Source: Record Retention Metrics

Analysis Summary: For the month of January this metric falls within the desired range

Record RetentionPercent of Research Finance Documents filed On Time

139 140 142 141

128 129 125 130140

132 135 129139 140 141

138128 129

125130

140132 135

129

100% 100% 99% 98% 100% 100% 100% 100% 100% 100% 100% 100%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0

20

40

60

80

100

120

140

160

180

200

Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13 Aug-13 Sep-13 Oct-13 Nov-13 Dec-13 Jan-14

Total Electronic + Paper Total Completed on Time % Compliant

©2014 MFMER | slide-44

QMS Metrics

• Timely Federal Financial Report (FFR) submission

• Past due employee performance appraisals

• Completion of assigned compliance training

• Successfully passing the Competency Assessment

• Completion of C-ICE flows, controls and testing

• Timely filing of documents for Record Retention

©2014 MFMER | slide-45

Quality Assurance Control Chart

©2014 MFMER | slide-46

Quality Assurance Controls

90 % of all QA findings are process issues

©2014 MFMER | slide-47

Does it really work?

Continuous Improvement

Customer Service &

Satisfaction

Culture

QMS Administration

Documentation System

Management Commitment

Measurement & Analysis

Competence, Awareness & Training

©2014 MFMER | slide-48

QMS – Before and After

Vis

ion

Mission

Goals / Objectives

Vis

ion

Mission

Goals / Objectives

Before QMSQMS

Implementation After QMS

©2014 MFMER | slide-49

Indicators of Success

• Staff Costs:• FTE reduced nearly 15% over the past two

years as a result of process improvement• Process Improvements:

• 35 process improvement projects were completed in the last 9 months of 2013

• Accomplishments

©2014 MFMER | slide-50

Mayo Clinic A-133 Audit Trend(Research Specific)

©2014 MFMER | slide-51

Summary

• Implement a Quality Management System

• A QMS consists of eight key components

• Self-monitoring tools help maintain an efficient and effective QMS

• Control limits and defined escalation processes need to be understood

©2014 MFMER | slide-52

Questions/Discussion