AN INTEGRATED MODEL FOR PERFORMANCE

MANAGEMENT BASED ON ISO 9000 AND BUSINESS

EXCELLENCE MODELS

Shannon Macey

Submitted

in partial fulfhent of the requirements

for the degree of

MASTER OF APPLIED SCIENCE

Industrial Engineering

at

DALHOUSIE UNIVERSITY

Halifax, Nova Scotia August, 2001

Copyright by Shannon Macey, 2001

National Library Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibfiographic Services senrices bibliographiques

395 WeUington Street 395. rue Wellington OltawaON KlAON4 Ottawa ON K1A W Canada Cam&

The author has granted a non- exclusive licence allowing the National Library of Canada to reproduce, Ioan, distribute or sell copies of this thesis in microfom, paper or electronic formats.

The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fkom it may be printed or otherwise reproduced without the author's permission.

L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cetie thèse sous la forme de microfiche/film, de reproduction sur papier ou sur format électronique.

L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni Ia thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation.

Table Of Contents

List of Figures ................... ..............................m....~.........em...........................~......... x

List of Abbreviations . . . . . - . m . . . . . m ~ . . . . m ~ m ~ * - ~ ~ ~ m - . ~ m m ~ ~ ~ m ~ ~ ~ ~ ~ ~ ~ ~ m ~ ~ o o o o o xi

1.0 Introduction ................... .......~.......o...~....~....ooo..~~~.e...............................~e...o...... 1

Performance Management ...... ..... . .......................................................................................... . ...... 1

Models for Performance Management . . .. . . , . . , .. ... . . . . . .. . .. . . . .. . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -4

Performance Measurement ......................................... ... . ............................................. 5

Performance Management in R&D ............................. ,.. .... . ............. . ................... . ......... . 7

Case Study Organisation (CSO) ............................... ,. .......... . ................................... . . . 9

Ornanisation of the Thesis ............................................. . .................................... . . . . 1 0

2.0 Literature Review ............. ...............*................~o.....sm..........*.~*...........*........~... 12

2.1 Introduction ............................... ........ ............................ .......... ............... . ................ . . . . 12

2.2 Performance Management ................................ . ........... . . . . . . ................. . . ..............12

2.2.1 Models for Performance Management ..... ........................... ....................................... . 13

2.2.2 Performance Measurement . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . .. . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . . -27

2 3 Performance Management in R&D .................................... ....... ........... ..................................... -46

2.3.1 Addressing Issues Unique to R&D ....................................................................................... 46

2.3.2 Benefits of Implernenting Quality Management in an R&D Environment ........................... 49

2.3.3 ISO 9000 in R&D .................................................................................................................. 51

Motivation for Proposed Research ...................... ... ............................................................... 52

Objectives of Proposed Researcb ................................................................................................ 53

An Integnted Mode1 For Perf~~nance Muiagemcnt ................................. 5 4

Introduction ................ ............. .. ...... .......................................................................................... 54

Development of the OPIM ................... ............ .......... .............w........9.9........ .......................... 54

3.2.1 Framework Development ...................................................................................................... 54

3.2.2 Criteria Development ............................................................................................................ 57

3.2.3 Development of the Assessment Questions .......................................................................... 66

3.2.4 Establishment of the Scoring Scheme ................................................................................... 66

3.2.5 Scoring Scheme Cornparison ................................................................................................ 73

3 3 Cbapter Summary ....................................................................................................................... 76

4.0 Seff Assessrnent .............................................................................................. 77

4.1 Introduction .................................................................................................................................. 77

....................................................................... 4.2 Self-Assessment at the Case Study Organisation 77

................................................................................................................................ 4.2.1 Planning 78

4.2.2 Assessrnent ............................................................................................................................ 83

4.2.3 Final Score ............................................................................................................................ 89

4.2.4 Outcomes .............................................................................................................................. 90

4.2.5 Evaluation o f Assessrnent Process ........................................................................................ 93

5.0 Performance Measurernent ........e.................................................................... 96

5.1 Introduction .................................................................................................................................. 96

Cornparison of Assessrnent ResulW to Quriity ~ u d i t Results ..................... ... ...................... 96

Implications for Simultaneous Auditing and Assessrnent ................................................... 102

Result-based PerCormance Measurement ..................... ... ............. ...................................... -103

Review of Organisational Performance Measures ............................................................. 103

Review of the Performance Measures at Case Study Organisation .................................. 104

Self.assessment, Auditing and Result-based Performance Measuremeat ............................. 117

Comparing Self-Assessrnent and Result-Based Performance Measurement ....................... 117

The Roles of Self.Assessrnent, Quality Audits. and Result B a d Perfomance Measurement

in the Organisational Performance Masurement System ...................................................................... 119

........................................................................ ....................................... 5.5 Chapter Summary .. 120

6.0 Performance Management In R&D .............................................................. îîl

6.2 The R&D Environment at the CS0 ........................................................................................ 121

......................... 63 Modifying the OPIM for Use in an R&D Environment ......................... ... 122

.......................................................................................................................... 6.3.1 Framework 123

................................................................................................................................ 6.3.2 Criteria 125

................................................................................................................................ 6.3.3 Scoring 128

6.3.4 Addressing R&D Issues with the Modified OPIM .............................................................. 130

...................................................................................................................... Chapter Summary 131

Conclusions .................................................................................................. 132

................................................................................................... Contributions of the Research 132

.................................................................. Scope for Further Research ... ...... ..... ................. 134

References .................................................................................................... 136

Appendices ................................................................................................... 144

APPENDlX A . OPIM Questionnaire ............................................................................................. 1 4 5

APPENDIX B . Modified OPIM for R&D .................. -..... ................................................................. 160

APPENDIX C . Triiiium Results ...................... ..,............. ............................................................... 169

APPENDIX D - Cornparison of Performance Management Mode1 Criteria ...................................... 170

APPENDIX E - Approach. Deployment and Results Adapted from the EQA ................................... 175

List of Tables

....................................... Table 2-1: Channeristics of Performance Management Models -21

...................... Table 2-2: Strengths and Weaknesses of Performuice Management Models 22

..................... Table 2-3: A Cornparison of the S c o ~ g Schernes from the Quairy A w d 24

............................................................ Table 2-4: Selection of Self-Assessrnent Method -31

Table 2-5: Evaluation and Scoring Methods of the Performance Management Models ....... 33

Table 2-6: Description of Evaluation EIements Used in the EQA ..................................... 34

Table 2-7: Scoring Guidelines from the MBNQA ........................................................... -35

.......................................................... Table 2-8: Sample Score Calculation for MBNQA 3 6

Table 2-9: Plan-Do-Check-Act Cirde of Audit and Self-Assessment Programs .................. 45

Table 3-1: Inma Caregories of the OPIM ........................................................................ 57

Table 3-2: Initial fit of Mode1 Giteria to OPIM Framework ............................................. 58 Table 3-3: Changes to the OPIM Categories ............................................................... 60

............................................................. Tabie 3-4: Application of Cnteria to Frarnework 6 1

Table 3-5: Final OPIM Framework with Comespondmg Critexia from Xntegrared Models .. 64

Table 3-6: Calculauon of Scores from Preferences ............................................................ 71 . .

Table 3-7: OPIM Scoring Cntena .................................................................................... 72

Table 3-8: Distribution of Points Arnong Criteria of OPIM, MBNQA, and EQA ............. 74

Table 4-1: Sample from the Question Mauix ................................................................... 82

.................................................................. Table 4-2: Tirnelines for Self-Assessrnent 8 3

................................................................................. Table 4-3: Final Score Calculauon 8 9

Table 4-4: Prioritising Areas for Improvement ................................................................ 90

Table 5- 1 : Comparing Outcomes from Self- Assessment and Q d t y Audit ....................... 96

.... Table 5-2: Recomrnendations, and Noncornpliance Issues and Areas for Improvement 98

Table 5-3: Comparison of CS0 Scores on Self-Assessmenr to Audit Findings ................ 101 Table 5-4: Results Categories from MBNQA and EQA ............................................... 103

....................................................... Table 5-5: Performance Measwes Used at the CSO 105

................................. Table 5-6: Performance Measurement System Interview Questions 107

Table 5-7: Applying the BaLnced Scoreard to the Performance Measurtment Syscem of the

Cs0 ..................................................................................................................... 110

Table 5-8: Performance Mea~ures According to QuiLry, TirrP, Cos, and Flexibility ........ 111

Table 5-9: Performance Arev Addresseci by the OPIM ................................................. 113

Table 5-10: Suggested Performance Measures to Add Relating to Resources ................... 115

Table 5-1 1: Suggested Pedomiance Meyures to Add Relating to Society ........................ 116

Table 5- 12: Suggested Perfonnuice Mevures to Add Relathg to Employees .................. 116

Table 5- 13: Characteristics of Perfoxmance Measurement Methods ................................. 117

................. Table 6-1: Chamteristics of QwLty Management Addressing Issues in R8rD 123

Table 6-2: Strategy Criteria Gom the OPIM ................................................................. 127

Table 6-3: Moditied Strategy Criteria fiom the OPIM ................................................... 128

................ Table 6-4: The Categories of the Modif1ed OPIM Addressing the R&D Issues 129

Table 6-5: Scoring Sdieme for the Modifieci OPIM ................................................... 130

Table 6 4 Modifieci OPIM Addressing R&D Pro blems ................................................. 131

List of Figines Figure 1- 1: P e r f o ~ c e Management ...................... ... ............................................... 1

Figure 1-2: Methods for Performance Improvemmt .......................................................... 3

Figure 1-3: The Relationship h e e n Performance Management and Performance

Measurement .......................................................................................................... 3

Figure 2- 1: Un-g Framewoxk for the Bddrige National QuaLF/ Award Model .......... 18

Figure 2-2: Enablen and Re& of the EQA ................................................................. 19

Figure 2-3: The Self-assessrnent Process ........ ............................. .................................... 2 9

Figure 2-4: Sehssessment Process Rrgour Versus Data Qua<ty ...................................... 32 . . .

Figure 2-5: OveMew of audit amvines ........................................................................... 3 8

Figure 3-1: OPM Framewotir ............................................................................... 5 6

Figure 3-2: Measuring Preference in Triilium ................................................................... 70

Figure 3-3- A Cornparison of the Point Distributions Among the Models ........................ 75

Figure 4-1: Steps in the Self-Assessrnent of the CSO ....................................................... 77

Figure 4-2: Test Operations & Svategic Sourcing ............................................................. 80

Figure 4-3: MvkRing and Sales ................................................................................ 8 1

Figure 4-4: Product Development ................................................................................... 81

Figure 4-5: Excerpt from Find Repon ..................................................................... 8 8

Figure 5-1: Cornparison of Scores on Self-Assessment to Audit Findings ........................ 100

Figure 5-2: Comparing Low Self-Assessrnent Scores to Non-Cornphces ...................... 102

Figure 5-3: Perfomance Measwement Categoxy from OPIM .................................... 104

Figure 5-4Measuring E fficiency and E ffectiveness in Self- Assessrnent ........................... 1 18

Figure 5-5: Relationship of OPIM and Derforniance Management System ....................... 120

Figure 6-1: Deployment of O ~ a t i o n a l and R&D Goals ............................................ 122

Figure 6-2: The Modified OPIM Framework ................................................................. 124

List of Abbreviations

CS0 - Case Study Organisation

TQM - Total QuaLty Management

BEM - Business Exceilence M&

MBNQA - Malcolm Baidnge National QuaLty Award

EQA - European QuaLty Award

DP - Deming Plize

NPD - New Product Development

OPIM - Organisational Performance Improvement M d

C E 0 - Chef Executive Officer

Go0 - Chief Operating Officer

VP - Vice President

QA - Qdity Asnirance

PFM - Preference Function M o d e b g

Acknowledgement s

1 would like to thank my supervisor, Dr. St&v Karapevovic, for invithg me to take part

in this research project and for all the guidance and support that he provideci throughout the

process. His knowledge of and experience in rhis field proved Livduable and was a great

source of inspiration.

1 wish to give thanks to the case study oqpkation for pamcipating in this project and for a i l

the CO-opemion 1 received whik performing my research there. Without its fwcial

support, this project would not have been possible. I would specifidy like to thank bot -

Mark DeKoning and Max Jaffer for their involvement and advice. Their support was of

great benefit to the successful completion of this project.

1 would Idce to thank Dr. John Blake for his valuable feedbadc and tirne spent reviewing rhis work and Dr. Eldon Gunn for his encouragement and support. Thanlu to both for serving

as memben of my guiding comminee. Thank ou to Dr. Walter Wdbom for his Ume spent

reviewuig some of the w o i in the thesis and for travelling to Halifax for my defence.

Thanks also to Dr. Car1 Sandblom for seming as a member of my guidance cornmittee.

Thanks also to the Department of Industrial Engineering at Dalhousie University for

providing funding as well as the opportunity to serve as a teaching assistant. Thanks to ail of

the Industrial Engineering deparunent staff, especially to Cindi Slaunwhite and Linda

Seamone for their cheer and fnendship.

1 would like to express my sincerest gratitude to Greg Delbndge for his inexhawible

encouragement, patience and support throughout my d e s .

In today's global economy, o&tions mua ourperform &eir cornpetitors in order to survive. They must set goals and objectives chat will move them into leadership positions in their indusrries. To ensure that these goals are being me5 cornplnies are developing performance management systems to help them measure and improve the efficiency and effectiveness of theu activities. S e v d modek have been developed to assist organisations in these efforts. Some of these models, such as ISO 9001, provide the minimum requirements for a quality management system, while orhen, like the National Quality Awards, provide a broader set of criteria that extend the prinaples of management to al1 organisational activities. Companies natudy question which model is best suited for their performance management needs. It has been suggested that the model should be selected based on the quélity maturity level of the organisation. The ISO 9001 standard for quality management provides a necessaxy foundation for the journey towards business excellence- The Business Excellence Models can then be used to build onto this foundation.

In this paper, a model that integrates ISO 9001, the Malcolm Bddnge National Quai;ty Award, the European QuaLty Award, and the Deming Pxize is developed The Organisational Performance Improvernent Modei (OPM) is considered to be a combination of quality assurance standards that provide the minimum recpirernents for sU1i;ty improvement, and business excellence models that illustrate the opportunities for improvement.

Using a Canadian hi& technology Company as a case midy, a p h self-assessrnent is conduaed againn the OPIM. This self-assessrnent pmcess is then compared to the two other performance meanirement methods used within the organisation to cietennine its usefulness as a tool for p e r f o m c e irnprovement. Because the case study organisation is highly focused on its research and development activities, modifications were made to the OPIM to create a more appropriate model for application and use in an R&D environment.

The OPIM was developed specifically for the case study organisation (CSO) but can be easdy adapted to be used by other organisations. A comparry cm employ this model to evaluate its current quality management system against the minimum requirernents in the model and rhen apply the opportunities for improvement as a guide for continuous improvement. Self-assessrnent against this model wili be beneficial in enhancing the organisation's quality auditing program as well as its redt-based perfoxmance measurement system.

1.0 Introduction

1.1 Performance Management

The purpose of any organisation is to niccessfully achieve in gods and objectives. To

remain cornpetitive, these goals need to be achieved in the most effiaent and effective

manner possible. Effectiveness and effiaency constitute the two fundamend dimensions of

performance (Neely et al, 1995). Effectiveness refers to the exrent to wbich goais are met,

while efficiency is a m m of how economidly the h ' s resouras are udised to redise

these goals. Organtations musr continually improve their processes and the use of th& resources to realise improved o v e d performance. Perfoxmance management is the overali

process of managing aa organisation's effiaencyuid effectiveness in the adiievernent of its

corporate and funaional strategies and objectives.

Performance management is a dosed loop process. Figure 1-1 illustrates this d e .

............................ 1 Audit .....................

Self-Assessrnent

Set Goais and Objectives 4

I lui for I.mprwemtnr I

Yes

As a firsr s e p in this process, the organisation musc set irs and objectives. These cm

be baseci on benchmarking, standards or other such refemce aiteria.

Once goals and objectives are determineci, performance measurement is used to nadr the

progress towards these goals. Performance measurement is the quantification of the

effiuency and effectiveness of an action (Neeiy et ai, 1995).

To narrow the gap becween the m e n t lwel of perfomunce and rhe levels required to meet

its objectives, an organisation must improve. Performance impmvement is the systematic

process of idennfying and implementing changes to oqpisauonal processes in order to

more efficiendy and e ffectively achieve set organisational objectives. Performance

improvement can be achieved in two ways. The fvst is to elimioate problems or

deficiencies. The second û to initiate actions that Mn aaualky result in improved

characterïstics.

There are rwo ways in which companies can elimliate problems. These are corrective and

preventative actions. Corrective actions are those which are taken to eliminate the cause of

an existing non-conformity[the non-fuifilment of a requirement (ISO 9000: 20W)J or other

undesirable situation (ISO 9000: 2000). Preventive actions are those taken to elMinate the

cause of a potential nonconformiry or other potentially undesirable situation (ISO 9000:

2000).

Improved characteristics can be achieved through Kaizen or Quantum Leaps. Kaizen means

smd incremental irnprovernents towards the achievement of objectives. Quantum Leaps or

breakthroughs are larger irnprovements. Both of these reduce the average nurnber of

defects or reduce variation.

Figure 1-2 illusvates the concept of performance irnprovement.

Quaiify = k d o m h m deficiencies ( 0 ) + imP'0ved characteristics (+)

ELIMINATE PROBLEMS GET BETTER

F e , once a goal or objective is reaiised, new goals should be set to continue the cycle.

Both performance measurement and improvernent are essenria if an organisation wishes to

sirndtaneous~ achieve its objectives and maintain its cornpetitive edge.

MANAGEMENT

The measurement synem un be seen, according to Bitichi et al. (1997), as the

heart of the performance management qmem.

ACT +

PLAN + 4

l IMPROVEMENT v b

DO MEASUREMENT

L

There are xveral wys in which an organisation can mevure its performance and plan for

improvement. One such method is to compare its puformance to a given reference model. There are many such modek used by o ~ t i o n s today. Some of these models are

focused on specific processes within the organisation such the ISO 9001: ZOOO standard

for sUai;ty management. Others such as National QuaLty A w d often referred to ar;

business excellence models (BEM) have a broader focus and uy to encompass all aspects of

an organisation. All of these models address the main aspects of performance management:

performance measurernent and performance improvement.

1.1.1.1 ISO 9000: 2000 Standards for Quli;ty Management

The ISO 9000 series of @ty management syste.cn standards is an international set of

nanduds that provide the minimum requkements for an organisation's quality management

system. The core standard of the series, ISO 9001, addresses the creation and

implementation of a quaiity policy, the standardisation of procedures, the identification and

elimination of defects, cor-ve and preventive action, and the management review of the

quality management systea The standard is process onenteci, aïnu at customer satisfaction

asniance, and focuses an organisation's attention on the effectiveness of its quaiiy

management sysrem. It was conceived as a set of essential requirements for building a

qualiy management synem that fits the company's ne& in relation to its products, services

and customers.

1.1.1 -2 National Quaiity Awards

Q d y awards are models for business excellence. These models consist of viteria to which

organisations can compare their activities and results. They have been developed based on

the p ~ a p l e s of Total QwLty Management FQM). TQM is a quality-oried approach

chat consisu of applymg quality management techniques t h r e o u t the organisation. The

evhtion criteria provide puideluies to organisations to measure their progress in these

efforts.

1.1.1.3 Integration of ISO 9001 and the Business Exceknce Models

Bo& the ISO 9000 series and the quality mat& have been adopted by orgaaisations to

measure and improve perfomiuice. Many oqpisations wonder whether one mode1 is better than another, and whether the choice of model should depend on the type of

organisation. Many authors have mggesteci that both mo& are important to an

organisation (van der W d e et ai, 2000; Dale, 1999; Karapetrovic and Wdborn, 2ûûlB). The

ISO 9001 standard is seen as providing the minimum reS;rements for a &ty management

syaem or the basics to stan the joumey towards orgauisational exceilence. The quality

awardr, provide guidance for the n a sep on this joumq..

Combining these modeis would provide an integrared set of criteria that codd help an organisation transition smooddy from its quaity management system t o w d the use of the

business exceilence modek for improvernent. There are however, many differences between

these models and as a resuh, they cannot simpiy be cur and pasteci together (Dale, 1999).

1.1.2 Performance Measwernent

To meet a company's organisational goals, performance measurement must be used to

evaluate, control and improve organisational processes (Ghalayini and Noble, 1996). This

process is used to interpret and descnbe quantitatively the criteria used to mesure the

effectiveness of the organisation.

As noted previously, performance measurement can be defmed in terms of the rwo

fundamental dimensions of performance. Performance measurement is thus the process of

quanufyuig the efficiency and effectiveness of an action (Neely et al, 1995). ISO 9000

(2000) defmes effciency as the relationship bmeen results achieved and resources used and

effecriveness as the extent to which planned aaivities are realised and planned results

achîeved

Three main performance mevurcment methods are used byo+tions today: self-

assessment, auditing and renilr-based performculce rneasurement.

As organisations srrive to implement perfomunce management models, there is a redting

need to meanire progress and improvemenc of the impIementation efforts. Self-assessrnent

involves the regular assessment and monitoring of organisational activities to determine what

is working well, what needs to be improved, and what is missing (Dale and Bunney, 1999).

Instead of compliance with the criteria of the business excelience model, self-assessment

idendes an organisation's strengths and opportunities for improvement in each area of a

predefmed set of criteria. Self-assessrnenu ask "how doesthe organisation meet the

criteria?" and whether this approach is used consistendy (Le. well deployed) throughout the

enterprise.

Self-assessrnent using am-ard models is focused on finding impmvement opportuniries and

new ways in which to drive the organisation down the road of business excellence (van der

Weile et al., 1995).

The performance meanirement merhod used to evaiuate an organisation's qUa;ty

management system is the quaiity audit. The audit is used to evaluate the compliance of the

system widi the criteria of the ISO 9001 standard. The outcome is either compliance or

non-cornpliance. Audits are designed to provide independent, objective and documenteci

evaluation of the evidence that the quality system &S.

Quaity au& are used to measure the effectïveness of an organisation's quahqr system

against the selected ISO 9000 model. Audits do not however, meanire the efiaency of the

system and therefore miss one essentd aspect of perfomance.

The audits used with ISO 9000 series registration look for noncornpliance and assess

whether the quJiry mern and its undeliying procedures are king followed. In other words,

+ty audits ask 'if the organisation meeu the deria", wheeas self-assessrnena ask 'how,

and how well, does the organisation meet the criteria".

1.1.2.3 Result-based performance measurement

Result-based performance measurernent is another method organisations use ro meanire the

effuency and effectiveness of their processes. This involves the collection of information

and data to analyse process performance uid/or input and output characteristics (ISO 9000:

2000). To do this, companies design or adopt performance measures that ~rovi& valuable

information about the current performance lwels of organisational activities. Performance

measwes are meuics used to quanafv performance.

The set of performance measures used by an organisation fomis the performance

measurement system. Firms m m carefully select these m e m e s to provide a view of key

performance areas to control and improve performance lwels. It is the seleaion of this set

of meanires that has proved chailenging.

1.2 Performance Management in R&D

Research and development (R&D) is an area in which there is much variability and

uncenainty. The ability to innitute systematic decision-making continuous improvement,

experimentation with new ideas, and tearnwork are cntical elements of the TQM philosophy

that have been found to be important for a success ful RBrD work environment (Gamin,

1993, Kiella and Golhar, 1997). This area of research however has been found to be weak

as, for the most part, the lirerature deals with the manufamiring and service sectors.

Due to the l w tangibildy and repetiuon found in R m &pmnents, manufamiring-based

quality practices when applied to R&D my prove dislsrrorrr (Kumar and Boyle, 2001).

There are several issues, specific to an R&D organisation, t&at shoulcf be considered.

Individuai achievement, as opposeci to teamwork, has been recognised and awarded in

the R&D envkonments fostering a very inciividualiswc workethic.

The goals and objectives of the R&D funaion are often emphasised over the o v d

company goals, leadhg to a system rhat is not f d y conducive to the achievement of

these organisational goals.

In todn/s competitive marketplace, companies are faced with the prospect of

introducing new products at a fnghtening Pace in order to survive.

In a shon-sighted effort to remain competitive, companies reduce the number of

resources avaiiable. The R&D department is often the fim target. The expectation is

that researchers will continue to perform at the same level with fewer resources.

W1th increased cornpetition, organisations must recognise that R8rD/new product

development is one of the moa important sources of competitive advanrage for a f m

aayawama and Pearson, 200 1).

The emergence of new technology gready impacts the R8d) environment. R%D mus

view technology and scientific advances as tools to assin the organisation in meeting

company goals (McLaughhn, 1995).

Its responsiveness to and influence on changing customer needs musr drive the R8rD

organisation.

The company used as a case study (case mi+ organisation) for the research in this thesis is

very mu& f a e d on its research and development efforts to maintain its competirive

position in the marketplace. Many of the issues lined above apply to the R&D departments

in this organisation. The n a section provides an overview of the case study organisation.

1.3 Case Study Organisation (CSO)

The companyused for this work, which we wiU caiI the case study organisation (CSO) is a

Cinadian hrgh technology company which desigm, manufactures and markets eleamnic

components, p@ &con integrated circuits (ICs) and thick-film hybrid circuits, for

specialised applications.

The organisation has rwo faciliaes with total plant space of 70,000 square feet, encompassing

all aspects of integrated circuit development, muiufacnviag and marketing. Revenue

exceeds f 107 million per year. The CS0 employs ahost 500 people.

The company bas recendy reorganised into rhree divisions. The divisions are based on

produa families and each indu&s marketin& sales, R&D, and manufacturing functions.

The CSO recognises that in today's fast-changing economy; initiative and ingenuity are

essenùal elements of success. In 2000, it renewed its focus on RSrD, ramping up spending

to approximately 17 percent of des.

The organisation is dedicated to quality. To ennue hi& quality produas and services, the

organisation's Quality Program is registered to ISO-900 1-94 and is constantiy evaluated

through interna and extemal audits. The quai~ty system is currendy being updated to meet

the ISO 900 1: 2000 standard for quality management.

W e the quality synem is being rnaùitained in accordance with the ISO 9001-94 standard,

the CS0 is aware that these efforts are no longer adequate to maintain their cornpetitive

advantage. The company needs to continue its quality journey and expand its continuous

improvement efforts. Several National Quality A . suggest cnteria that should move the

CS0 into this nez phase of excellence.

1.4 Organisation of the Thesis

The remainder of this thesis is organised as follows.

Chapter Two is a nwqr of existing iitennire to develop a background for the problems

addressed in this thesis. These top& indude:

Performance Management

Performance Management in R&D

Issues Uniqye to R&D

ISO 9001 in R&D

It dso indudes an o v e ~ e w of the motivation for and objectives of the research in this

thesis.

Chapter Three describes the development of an integrated model for performance

management based on the ISO 9001: 2000 Standard for QwLty Management and severai

business exceilence models.

Chapter Four subsequendy reports on a pilot self-assessrnent performed in a business

division at the case study organisation. This self-assessment was conducteci against the

proposed model for performance management.

Chapter Five compares diree me&& of perfomance measurement used by the CSO.

These include the self-assessment, the p h t y audithg prognm, and the result-based

performance measurement system.

Chapter Six describes the modification of the OPIM to be more appropriate for use in an

R&D environment.

Chapter Seven condudes the thesis with a nimmary of r e d a and convibutions of the work.

The scope for htrther research is also discussed.

2.0 Literature Review 2.1 Introduction

A nwqr of existing literature has been conducted to review the foliowing aspects of

performance management:

the main elemenu of performance management iaduding mewrement and

improvemenr;

existing frarneworks and modeis for organisationai performance management;

the integdon of the minimum requirements for a quality management synem with die

cntexia frorn the more holinic national @ty awards;

three organisational performance measurement methob: self-assessment, audiring, and

redt-based performance measurement;

performance management in a research and devdopment environment;

Performance Management

Success of an organisation's business depends on its ability to set svafegy and objectives and

then plan for the resources and activities necessary to achieve them To ensure that these

pluuied activities acnially occur and lead to the fulfhent of the set objectives, the

organisation needs to msnage and improve its performance. Perfomiance management is

"the process by which the Company manages its performance in line wirh irs corponte and

functional strategies and objectivesn (Bititci et al., 1997). Performance management is

essentiai to the successful operation of the enterprise. It has become a key business process.

The performance management process is a dosed loop deployment and feedback ryaem

that sans with the company's vision, broken down into business objectives and strategic

goals. Action plans are created based on the achievement of these goals and objectives. To

monitor the progress towuds the accompiishment of these goals and objectives,

performance measurement is used to collect information, indicating where the nirrent

performance gaps exist. This information is then fed into the performance improvement

process, which narrows the perfomunce gaps and then feeds information back into the

process. In SV, the main elements of performance management are:

establishment of goals and objectives

pertonnance measurement

performance Unprovernent

These interdependent processes, hctioning hannoniousiy, usix~g various resources to

achieve goals related to the effiaency and effeaiveness of organisational processes, make up

the performance management system. (Karapetrovic and Wi11born, 1998A)

There are numerous rnodels for performance management. The next section reviews several

of these.

2.2.1 Models for Performance Management

This section outlines the frameworks and models used by organisations today to design,

implement, maintain, and improve their performance management systems. These range

from models of the basic requirements for a quaRy management system such as the ISO

9000 standard for q d t y management to more holistic models for business excellence. The

latter includes the Malcolm Baldnge National Q d t y Awad, the European Q d r y Award,

and the Deming Prize.

2.2.1.1 ISO 9000: 2000 Standard for Quality Management

Customen in today's marketplace require assurance of the quaiity of the products or services

that they are buying. To provide this assurance, organisations look to the ISO 9000 series of

standards for q d c y management to provide guiciance in the planning, implementation and

maintenance of an appropriate quality system.

A @ty system is a set of interdependent processes tbat h & o n hamoniody, using

various resources, to achieve the objectives relateci to quality (Karapeuovic and Wiibom,

1998A). The quai~ty qmem is a sub-system of the business system of an organisation.

The ISO 9000 senes of standards was developed in 1987 as a set of models that stipdated

the requirements upon which an e x t e d pany could assess an organisation's quality system.

The senes consisted of guidance nandvds and conformance standards. The guidance

standards ISO 8402, ISO 9000, and ISO 9004, provideci interpretations for using the

conforniance standards. The conformance standards ISO 9001, ISO 9002, and ISO 9003

represented three distinct f o m of funaional capability suitable for extemai qyality

assurance purposes. Companies applied the applicable standard to their organisation to

assure their customers of the @ty of their products and senices and to make their quality

efforts more uaasparent.

The original ISO 9000 senes was revised in 1994. The ISO 9001 standard in this version of

the model consisted of 20 elements that addressed all aspects of a q d t y srjem. This

sysem induded the organisation, allocation of responsibilities, procedures, processes, and

resources that joindy led to the provision of goods and seMces in accordance with the firrn's

quality policy and objectives (PG, 1995).

The standards emphasised the asniance of quaiity of products and services through:

clarification of management poiicies and cornmiunent

identifilcation of roles, responsi bility and authority (Elmuti, 1996)

establishment of clear instructions to ali personnel affecting qualiry

development of precise procedures and instructions in al1 areas of operationai activity to enwe consistenq and uniformity (kadi et ai., 1996)

This model has been used extensively by organisations 1 over the world since its incepuon.

Today, over 400,000 companies world-wide are registered. There have been however,

severai criticisms made of the model:

tao much emphvis is piaced on documentation, creating bureauctacy in convol

rnechanisms and bodenecks when documents need to be c h g e d or updated

(Karapeuovk, 1999; Dale, 1999)

la& of emphasis on impmvement (Chin et al., 1995; Pun, 1998; Najmi and Kehoe, 2000)

la& of a suong focus on nistomer satisfaction @odinson, 1991)

frequendy mistaken as a guarantee for produa quahty (Bum, 1990)

In December 20d0, the ISO 9000 senes was updated for the second time. The 1994

version, which emphasised +ty assurance, has now been replaced by ISO 9001:2000 that

was developed to assis organisations implement and operate effective quality management

systems. The sUa;ty asnuance standard provided confidence that the requirements for

quaiity will be met (Karapetrovic and Wilibom, 1998A) through quality planning and

conuol. The new series adds the process of qwLty impmvement into a broader model

emphasising quality management. Q d t y management is the combination of quality

planning, q d t y conuol, and quaiity impmvement. This new emphasis on improvement

renilted in a section of the ISO 9001 standard devoted to measurement, an+ and

improvemenr processes.

The changes manifest themselves in the underiymg p ~ c i p l e s of the model. Ir is based on

eight q d t y management principles.

1. Customer focused organisation

2. Leadership

3. Involvement of people

4. Process approach

S. System approach to management

6. Continual impmvement

7. F a d approach to decision making

8. M u e beneficial supplier relationships

By connecting the prinaples of qu+ management to organisational processes, the 2000

version of ISO 9ûûû provides a greater orientation tawvds continual improvement and

customer satisfaction, thus widening the scope and the use of the standard

The ISO 9000 senes now consists of a pair of quality management system standards: ISO

9001 and ISO 9004.

ISOSM)I ISO 9001 s p d e s requiremerits for a qu?lity managrment system that c m be used for infernal application by oqpnktions, or for cdca t i on , or for contracnui purposes. It focuses on the effectiveness of the qu;Jity management system in meeting customer qukments. (ISO 9004:2000)

ISO 9004 gives guidance on a wider range of objectives of a qu;ility management system than does ISO 9001, particulariy for the continuai impmvement of an organisation's o v d performance and efficiency, as wd as its effectiveness. ISO 9004 is recommended as a guide for o ~ a t i o n s whose top management wishes to move beyond the requirements of ISO 9001 in punuit of continual impmvement of performance. However, it is not intendecl for certification or for c o n d purposes. (ISO 9004:2000)

These IWO standards can be used alone or a complementaxy models for an organisation's

quality management system ISO 9001 sets the essenrial requirernents and ISO 9004 shows

the way for future developments. To maintain registration status, an organisation mus

perform intemal and extemal audits against the ISO 9001 model to show that it has met

these requirements. It can however, enhance the audit process by performing self-

assessments againn the ISO 9004 model. This process can help identify strengths and areas

for irnprovement in the quaiity management system in texms of its efficiency as well as its

effectiveness. If the issues identified in the self-assessrnent are addresseci, the results of the

next quality audit should show irnprovement.

While the ISO 9004 standard is a model for improving the efficiency and effectiveness of the

quality management system, other models have been ke loped that address an even broader

set of organisationai activities. These are the National QuaLty Awarb or Business

Excellence Models (BEM) as they are often referred to.

2.2.1.2 National Quality Awardc

QuaLty awards are modeis of business excellence consking of uitexia to whidi

organisations cm compare their activities and resuits. This compvison process, r e f e d to

as self-assessment, involves the regular assessrnent and monitoring of organisational

activiries (Dale and Bunney, 1999). hstead of cornpliance with the criteria of a business

excellence model (BEM), these assessments idenrify an organisation's suengrhs, weaknesses,

and oppormnities for impravement in each area of the criteria.

Three of the most pop& q d t y awards are the Malcolm Baldnge National Qua;ty Award,

the European Qua;ty Award and the Deming Prize. Each of these a w d has its own focus, famework, mteria and scoring method

This award is named for Malcolm Baldrige, who served as Secretary of Commerce from

198 1 und 1987. His managerial excellence conuibuted to long-term irnprovement in

e fficiency and effectiveness of goverment (NIST, 200 1)

Organisations that aim at k ing world-class focus on insrilling core values such as good

leadership, amorner focus, respect for employees, and continuous improvement

(Pannirselvam and Ferguson, 2001). These form the basis of the MBNQAaiteria. The

model embodies these essential values in a framework of seven main categones.

1. Leadership

2. Strategic P l k g

3. Customer and Market Focus

4. Information and Anaiysis

5. Human Resource Focus

6. Process Management

7. Business Results

There are two key assumptions thax devin the modeL The fim is that the leadership

provided by top management is the primvy driver of the business. Top management m e s

organisational values and goals and direcrs orp.aiutional improvement. The second is that

the basic goal of the slUi;ty process is the delivery of mer-impruving q d t y and d u e to

customers (Malcolm Baldnge National Quatty Awad, 2000). Figure 2-1 provides the

framework comecting and integrating the categories.

4 Information and Adysis

Organisations are evduated against a given set of criteria and a maximum total of 1000

points are allocated among seven award criteria Each critenon curies a different number of

points in accordance with its relative value within the a w d

The European Quality Award (EQM) was introduced in 1991 by the European Foundation

for Quality Management (EFQM).

The EFQM Excellence moQl is based on a set of fundamental concepts that formulare the

definition of excellence (European Quality Award, 1999). These are:

results orientation

amorner focus

leadership and consistency of purpose

management by processes and facrs

people dwelopment & involvement

continuous leaming innovation & improvement

partnership development

public responsibility

The mode1 incorporates these Fundamend Concepts in a set of nine criteria; five of which

are 'enablers', four of which are 'redts'. The 'enablers' criteria address what an organisation

does while the 'renilrs' cover what an organiution achieves. Figure 2-2 ilhwates the model with the 'enablers' and 'results'.

1 Enablers Results

Leadership Processes

& Resources u=LJ People

Key Performance

R e d t s

Innovation and Leaming I

organisations are evaluated agaian a given set of criteria and a maximum total of 1000

points are allocated among nine awd criteria. Each mterion carries ii different number of

points in accordance 4 t h its relative value withiu the a w d

The Deming Prize was established, in honour of Dr. W. Edwards Deming, by the Board of

Directon of the Japanese Union of Saentists and Engineers W E ) in 1951. Dr. Deming,

one of the fathen of quaiity, is best known for his work in Japan, d e r e from 1950 onwad,

he taught top management and engineen' methob for the management of qualiry. The

impact of Dr. Deming's teachings on American manufacturing and semice organisations has

been profound

The orignal objective of this prize was to assess a company's use and application of

statisticd method (Dale and Bunney, 1999).

The prize consins of a cheddisr with ten major categories. These are:

1. Policies

2. The organisation and its operation

3. Assembhg and disserninating information

4. Standardisation

5. Human resources

6. Quaiiry asnirance

7. Mallitenance

8. Improvement

9. Effects

10. Future plans

Each pricnary category has six sub-categories (except for q d t y assurance, which has

twelve). The cheddist expiiady identifies the factors and procedures that constitute the

Company-Wide Q d t y Convol and Totai QwLty Control (CWQCrTQC) process.

Because it names specific techniques and appm&es, the ch& Y inherendy presuiptive.

There are no pre-designateci points allocarad to the individd sukategories.

The Dmimg PNc ir not b d on an udaiying fmmework linking concepts, aaiwies, processes and resuits together. Furthemiore!, it does not assume an undcrtying causality. It si.& ~rovides a lisr of desirable or good gu;ility-orieûted management pncticcs. Its focus is on poliaes and plans; implement;iOon of p h ; information collection, a d + ;md control; resulrs flowing frorn the implemenmtion of poliaes and thtir effects; and fuwc improverncnt p h .

(Ghobadian and Woo, 1994)

2.2.1.3 Comparing the Models

Each of these rnodels har its own focus, scope, approach to evaluarion, and improvement

strate W. Table 2- 1 illusvates these major differences.

Responsibility for F spnwlists and top management Idmufication of conformance and non- conformances, resulting in a pasdfd verdia

ISO 9000 Conformanu (CO p r o c h and CO

speafi&ons)

Qu&y assurance

Identification of the existence of -ch dement through an auditing process

MBNQA Iniprovan~~t (of rmthods and of o v e d business d u ) ExeCUfive

EQA lmprovanenc (of mcthods a d of o v d business r d t s ) ExCCutive

nruigths and oppominiria for improvemcnt, and scom i n d i h g the m;inuity of the

Deming Rizc Statisud Mcrhods (prevaition of @ty ~ m b l m )

Executive

svcngths uid oppomuiitics for improvanent, and scores indiaring the cnanuity of the

Design and production processes, and dimdy- associateci support activities

managunmt v a n nie entire management synem, includlig ludcrstiip, pluuung and the use of infornucion and meuurunuits, the dorelopmenr and management of people, the mcrhods used for understanding clrstomcr and market rtquirrmenrs, and the management of kcy inuiness proceSKs.

managemcnt syscem The entire managanent system, induding leadership, people, policy and nntegv, puuiuships, mourccs, proceses, and the management of key business processes

Process orientcd nrong emphasis on proces management

Giteria confmed to the application of p r o ~ e ~ &, statisticd methoch and q d t y c i d a

1 ISO 9000 Fnmcwodc I Baseci on the vroccss

nonconformanccs and to pxwent m>ccurrrnce of pmblans

MBNQA B a 4 on an undcrjuig fnmework linlunp conceps, mkkks, processes and rrnihs togerher

Appendix D provides a detailed cornpuison of the elemems covered by the ISO 9001

EQA Based on an mdatying fnmervorklinlung concepts, &es, proceses and re& togtthff

CoaunlKnlS ilxpvmmfuung PIAN-DO- CHECK-ACI' cyde.

standard, the MBNQA, the EQA and the Deming Prize, in each of their cntena Table 2-2

DrmingPrizt Does not unime

anun- u d t y , u m p I y provides a list of &sirable or good quality-oricnted management pncrices ( G h o b a b and Woo, 1996)

Quality lmpmvemcnt t h r w g h d d conml

presents an overview of the svengths and weaknesses of each model.

Model

1- 9001

MBNQA

Stmlgths M i n focus is on the processes wnhia the quality management system Quaiity system model is prescriptive and therefore the way forward is clear and unambiguous End point (achieving registration) is weii defineci and externaüy valiàated (Najmi and Kehoe, 2000) Constinites an integrai pan of an organisation's o v e d management systern (Pun et ai, 1999) Addresses the identification and provision of resources with a focus on idkanmm, the work environment and the conml of documents Forces an organisation to describe its key processes and make thern more transparent (van der WÏde et al., 2 0 )

Broad model addressing an organisation's o v e d activities, not oniy those relateci to the quality management system Asks how the organisation meeu the criteria Focused on cunomer management

~

--

we;ikncsscs Weak in the area of humui

Provides ody the minimum requirements for the quaiity management system Does not specificafy address technology or financiai resources Primanlyaàdress iimited areas of the enabler si& of the BEM's Little emphasis on areas that directly impact on business or organisational effectiveness (Poner and Tanner, 1996) Process criteria goes into less depth than the ISO 9001 stanàard Does not spetlftcalty îddress the use of resources in its cnteria

s t m l g t h s B d modei ;iddrcssing an orginisuion's o v d activitics, not ody those r c i a d to the qu;ility management sysrem Asks h m the oqpimion meets the criteria Addresses the efficient and effective use of resourres within the o ~ s i o n Focused on the relations with the cornmunity, and customer's and ernployee's satisfaaion

Focused on q d t y control Focuseci on compuiy wide qu?lay efforts, conMwus improvement Provides a List of desirable or good qu;ilisr- onented management pnctices

Weaknesscs Process critaia ~ o e s into less dephrbrnthefs09001 stvldvd Does not &as how the orguiis;ition manages customer rehionships and satisfacrion or how it contribuus to socieiy and miaimises environmentai impact Gter ia are confined to the application of process an&&, mtisticd methob and q d t y d e s Crimia such as Company poiicy and p h m g d t s , or future p h are p- concenieci with quîlity wmce activiries and q d t y resuits Not b d o n a n u n d e r j i n g h e w o r k linking concepts, aaivities, processes and d f s

t ogether

The major points arisiig from this cornparison process are:

The ISO 9000 standard provides oniy the minimum requkmeno for the q d t y

management sysrem but is seen as the foundation of an organisation's total &ty

efforts.

The MBNQA is a broad model addressing an organisation's overaii activities, not ody

those related to the quality management system. It does not specifiulh/ address the

organisation's use of resoumes.

The EQA is also a broad model for business excelience. Ir does not address customer

management in the same detail as the MBNQA.

The Deming PNe is a model focused on quality conaol. The criteria is not as broad as

those in the MBNQA or the EQA but provides some important pieces conceming the

use of quaiity control in a company's continuous impmvement efforts.

Table 2-3 provides a cornparison of the major categories addressed by Bdi of the qudity

award models as weli as th& respective point values.

MBNQA 1

I (%) Ladership 12.5 Leadership 10

Strategic Planning Customer and Market Focus Information and

Human Resource Focus

1 1

8.5 Policy and Stntegy 8

8.5 People 9 1

Partnenhips and 9 Resources

Process 8.5 Processes 14 Manaeement

Business Results 7 Custorner Resuits jf society People resuits Resulu

Key Perfomiance Results

Deming Rue

Orpiution and iu '1

Assembling and

information Qu;ility-= MÎiatenuice 10 Standardisation 10

Irnprovement Future Plans

Ir is evident that each of the awards places emphasis on different areas of their criteria. Both

the EQA and the MBNQA however, assign the most points to the Results sections of their

model. The Deming Prize does not single this category out as being the most important in

rems of its scoring xheme (each is valwd at 10%). The MBNQA assigns 12.5% of its

points to Leadership and distributes the ren of its points among Strategic Planning,

Cusromer and Market Focus, Infornuon and Analysis, and Human Resource Focus

assigning 8.5% for each. The EQA however, wip ody 10% to Leadership d e givhg

Processes 14%. The People category is worth 9%, ody slghrh/ higher thao in the MBNQA.

This cornparison process has rwealed that each of the performance management madels

have muiy nrengths. Each one could compensate for aaother's weaknesses. The next

section examines, in further detail, the research that has been done conceming their

2.2.1.4 Integration of ISO 9001 and the Business Exceilence Models

The ISO 9001 standard for quality management is seen by many as a requed foundation for

a company's @ry and performance improvement efforts (van der Weile et al., 1997, Dde,

1999, Karapmvic and Willborn, 20O1B, Russell, 2000, Meegan and Taylor, 1997). For

exampie, van der Weile et al (1997) make the following comment:

Continuous improvément ody m;ikes sense if an organisation h m what is going on in reiation to the processes, which are undedying the things that need to be improved. The ISO 9000 series forces an organisation to desaibe the key processes and make them more transparent.

Once the requirements for rhis standard have been met however, companies need to b d d

on these foundations to extend the principles learned to ail organisational activities and not

just to the activities relating to the quaiity management *en This means irnproving a

quality synern beyond the level of simply meeting spec&cations to creation of the proper

conditions for business excellence.

The synem required for accreditation WU outline the organisationai structure, responsibilities, procedures, processes, and resources needed to knplement quaiity management. The m i t i o n zone [beyond ISO 90001 begins then at the intersection b e e n the operationai and management infrastnrctures (Meegan, 1997)

The transition from quaiity management to business excellence need not involve leaving ISO

9000 behind to move on to somRhing supenor (Meegan, 1997). To facilitate the movement

from ISO 9001 quaiity management to the o v e d business excelience embodied in the

quality awards, researdien niggest that these models be integrated (Mann and Voss, 2000,

Tonk, 2000, Dale, 1999, van der Wkle et al., 2000, Pun et al., 1999).

Pun et al. (1999), describe a self-assessed qual~ty management sysrem using the framework of

the Baldnge M d for the integrarion of ISO 9000: 1994, ISO 14000, and the Baldrige

criteria. In a case stuciy, the assessrnent against rhis f r a m e d d l consisted of the regukr

auditkg and performance assessments performed by the organisation.

When there is a co11ection of afk systems or stan- fiam different ueu, efficiency is difficult and misconcepuoas and coafurion often occur. It is &us ben to elimbae possible confusion and sub-optimisation from the outset by establishing and maintaining a t o d systems approach. (Puri et 4 1999)

Finai conclusions from Sun (2000), suggea that TQM and ISO 9000 srandards musr be

completely and systematically implemented and in tegrad Researchers must detemine a

method of c o m b i i g ISO 9000 and TQM, and bridging the gap bnween the two.

Mann and Voss (2000) present an innovative approach used by one company IO integrare its

ISO 9000-certified management system with the Baldrige model. The company concemed

developed its ISO 9000 qmem to address al l elements of the Baldnge criteria. Of parti&

note is its process improvement approach that prioritises improvemenr projects based on

their expected impact on the company's Baldnge score.

Companies, who are ar present working for an "integrated quality management sysremw

where ISO is a major contributor, go beyond the requirements of the standard ro M e r

irnprove the performance of the organisation in various dimensions. The ISO standard is

thus the comerstone of TQM and, as such, presents both oppotrunities and challenges.

The challenges involved in integrating ISO 9000 and TQM are well reviewed in the

literantre. We have seen that ISO 9000 has been advocated as the fxst step towards total

quality management. The ISO 9000 series and the excellence models clearly have different

goals and perspectives and define total quai~ty management at different levels of maturity

(van der Wiele er ai., 2000). It is obvious then that the uiteria from the ISO 9000 series

cannot simply be incorporateci into a model for business excelience. The criteria represented

in the reference models are different, ISO 9000 representing the minimum requirements and

the quaiity award models king more holisric paie, 1999). If rhis were to o c w , it would

lose much of its potency in the building of the foundatîon of an organisation's TQM

activities (Dale, 1999).

The quaiity a d models and the ISO 9001 standard use self-wessment and quality

auditing respectively for evaluation of rhe otganisauon against them. To integrate the

models would require the integration of these measurement methods. Karapetrovic and

Wiübom (2001B) explore the integration of quality audits and seif-assessment processes.

Another difficulty in integrating the ISO 9000 and the @ty award modeis is the different

'scoring" schernes used by eadi. For example, when Tonk (îûûû) describes the integration

of the MBNQA and the ISO 9000 system, he suggesu:

The Bddrige scoring system uses percenrages, w h m s the ISO 9000 systern uses, in effect, a pus/fd (conform;ince/nonconformuice) system Decisions need to be made as to whether to retain the MBNQA percenage scoring sysrem and if so what scoring levels will be reqwred in each of the uiterion in the new model.

2.2.2 Performance Measurement

Performance rneaswement is the process of coileccing data pertaining to al1 aspects of the

company's activiues. This data can then be used to help control and correct waning

performance areas and to set new targets for improved performance.

According to Sinclair and Zain (1999, rhis meanvernent is exuemely important to

organisations for several reasons:

Meanires can be analysed to enable appropriate decision making.

Measurement provides feedback and reinforces behaviour.

Measurement is an integral part of continuous impmvement; "if yai m o t meanve an

activity, you cannot irnprove it".

Measwement "helps an organisation direct its scarce resources to the mon aMcnve

impmvement opportunities*.

It has been shown that perfo~llil~lce improves if individuas are givea targets, and is . .

maxunised if targets are seen as cbillenging but achievable.

Measurement is impoxtant in the maintenance of longterm focus.

Measurement aiso plays an important role in quality and producOvYy improvement

(Oadand, 1993). It:

ensures that customer requirrments have been met

provides standards for establishiog cornparisons

provides visibility and a scoreboard for ~eople to monitor th& own performance levels

highlighrs quality problems and deremines which areas require priority attention

gives an indication of the costs of poor q d t y

justifies the use of resources

provides feedback for driving the improvement effort

Each of the National QluLty Awub provides guidelines for perfonning organisational self-

assessments agalin the given set of criteria for business excellence. MBNQA provides a

Guide to Self-assessrnent and Action. The EQA provides detailed guidance to organisations

and suggests a self-assessrnent approach depending on the quaiity rnaturity of the

organisation.

These organisations usually perform self-assessrnent in one of rhree ways (van der Weile et

a., 1999):

taking the a d guideha and foiiowing these for intemal use in the sarne way as

prescribed in the respective guidance booklets for those organisations who are putting

forward a formal application for the chosen award

adjusting the cnteria and/or the scoring to fit the specifîc situation and goals of the

organisation

speQfying the parh and seps to be t&n h m the present situation ro the end goal.

Conti (1993) suggescs that organisations should use already existing data in the assessment

process and as additional information for the goal setting process. Some examples of this

data are:

cunomer survgs, amorner cornplaint statistics, marketing strategies

employee wqs, work safety statistics, illaess rates, absenteeism and fluctuation,

human resources development plans

processes e.g. processing Ume or process liability, ~rojects and gained improvements

producr e.g. e m r rate. life duration and failure rates

fmancial e.g. trends conceming profits, sales volume, renim on investment (ROI), qwlity

coas &ink and Schmidt, 1998)

2.2.2.1.1 Self-Assessrnent Process

The self-assessrnent process usually follows the Deming PLAN-DO-CHECK-ACT qde. Figure 2-3 has been adopted from the EFQM (1995). It provides an o v e ~ m of the self-

assessment process.

Wbde the unciersiag pmcess re& the same, the ISO 9004 standard, the MBNQA and

the EQA each has its own m a for conducOng the self-assessment.

The assessment method suggested in the ISO 9004 standard involves the evaluation of the

mamrity of the @ty management synem for each major dause in ISO 9004 on a scale

ranghg from 1 (no formal system) to 5 (best-inclas performance). The standard provides

exvoples of typid questions that cm be asked duriag the self-assessment. It is suggested

that self-assessment can be used in a flexible way according to the needs of the organisation.

Two recommended m d o d s are to:

perfoxm the self-assessrnent on an individual basis for all or part of the quality

management system and then to pursue improvement.

have a cross-function group of people perfonn self-assessment on ail or part of the

quality management system, followed by group review and an*, then consensus

building, to detemilie improvement prioriries and action plans.

MBNQA

The Baldrige National Qwltty Progra. provides a Gui& to Self-Assesunent and Action.

This document ourlines ten seps ro conducting a self-assessment.

1. Idenufy the boundaries of the organisation to be assessed. -7 2. Decide on a format for the assessment and action plan. I 3. Write the business/organisaüon oveMew

l - 4. Select seven champions and teams, one for each criteria categoty

5. Practice self-assessrnent techniques with the seven category

6. Càtegory teams prepare a response for their assigned teams

7. Share responses among te- and fiiulise the fmdings J 8. Assess the organisation's strengths and o p p o d t i e s for improvement e-4-I

u 9. Develop and implement an action plan for improvement 1 - 10. Evaluate and improve the self-assessrnent and action process JLl .~m 1

EQA According to the EFQM, selectïon among assesmient methods should be dependent on the

quality ma- of the organisation. Table 2-4 presenu self-wessment approaches

correspondmg to the maturity of the oqpnkation and the desired effort level.

Facihateci workshop

Award simulation

Pilot a w d simulation

Pro-forma and wofkshop

Verv detaïleci questionnaire

This approach involves the creation of a questionnaire used to gather infomtion about the

perceptions of people within the organisation. Some questionnaires can consisr of simple

y e h o questions. Othen provide a set of possible a m e r s nich as A-Fullyachiwed, B-

Considerable progress, C-Some progress, or D-Not started. These questionnaires are

dinnbuted and used alone, or as nippon for more elaborate self-assessment approaches.

In this approach, an achievement ma& is developed For each category of the award,

statements perraliing to possible achievernents by the organisation are set on a scaie from 1

(minimum achievement) to 10 (greatest achievement). This is used as a basis for

management discussion and sening objectives for improvement.

Pro-forma Here pro-formas are created for each sub-category. The description of the critexia in the

sub-category is printed at the top of the page with areas to address beneath it. The

assessrnent entails f&g in suengths, areas for improvement, and evidence collected on the

pro-forma itself.

Woikshor> The management team is responsible for gathering dam from their unit and presenting the

results to their p e r s in a workshop environment.

Thic appmach is effectively a simulation of entering for the award. The unit undenaking the

self-assessrnent wntes a full submission document. A team of trained assessors then

evduates this report.

The EFQM dso indudes in the EQA document, a figure that i i l m t e d the relationship

between the rigour of the different approaches and the quaiity of data captured. (See Figure

Process Rigour 4

Low

Q r n h u t i

Pro- f o n c3 <workrhop>

Based on Opinion

Table 2-5 presents the evduation and scoring methods used by ach modeL

Mode1 ISO 9004

~ a&le S y s t d c pmess-bued appmpC4 Fufy =%e of system~tlc rmprovement , &ta avaiiable on c o n f o m c e to objectives and existence of improvement vends

Evaluation and Scoring Evalu?tes the m;uunty of the qu;rlity managtznenr system for cach major ciause in ISO 9009 on a s d e rianging from 1 (no formai system) to 5 (best-inîlus performuice).

hpmvement process in use; good resuits and susuineci improvement

I I trends

r Details 1

2

100%. Evaluates each elexnent in the award criteria based on

No systematic approach cvident, no results, poor d t s or unpredimbIe d t s

Problem or systemaric appfo;idr; minimum data on improvement resuits

Evaiuates each element in the a d criteria bved on the appmach, the deplayment of appmch, and resuits re?lised These are evaiuated on a d e from 0% to

these elemenu. These are evaiuated on a s d e fkm OYo to 100%.

5

the approach, the depIoyment of approach, the resuits r e d i d , and the assessrnent and review (RADAR) of

I

Cfiecklist is evaiuatd Each category is worth the same.

Strongiy integrated impmvement process; best-in&s results dernonstrateci

See Table 2-7 for details.

See Table 2-6 for details

Table 2-6 desaibes the evaluation dimensions used by the EQA.

Results r-- Assessrnent & Review r

Embodies evduation, improvemendleaming

1s b d on diable inforrmtion and &ta

Dcsaiptim Mallod(s) d to ;iddress item requvcmcnu

Alignmeat with o ~ t i o d needs Eviduice of innovation

Elernmt is cvaiuatcd based on Appmpri?teness of the methods to the rrgUiranmfs Effectivcness of the use of the methods

Exrem CO which a p p r d is applied to 4 the in each item

1 . breacith, and importance of ~our

Use of the approach in addressing item requtements relevant to your organisation Use of the a p p r d by 1 appropriate work units

OutcomesinachieVing the purposes of the item

CuITentperformance Performance relative to appropriate compvisons or bencbmarh

The scope of evidence

Gtpn from measurement and leamhg is / anw and useci KI identify, prioritise, plan

performance impmvements Remilu me;uurement of the effectiveness of

of measurernent, leaming and Unprovernent activities

1 and impIernent improvementr

t h b r o a c h and deplayment is carrieci out Levniag acùvities are used to idenufy and share best practice and improvement opportuniues

Table 2-7 illusuates the wduation dimensions used by the MBNQA

I m a c h and Dcployment no systematic approach aident; ando t a l

l eariy stages of a transition from re?cting to problems to generai improvernent orientation major gaps exist in deplayment dut wouid inhibit progress in ?chi+ the prinury

a sound, systematic a p p d , responsive to the primîry purposes of the item

I a faa-based improvement process in place in key areas; more emphasis is piaced on improvement than on reaction no major gaps in deployment, though ( someunrmayixineariyrvges

a sound, systermtic approach, responsive to overail purposes of the item

1 a fact-based irnprovernent process is a key management toil; dear evihence of

I refinmient and improved integrauon as a r e d t of improvement @es and adysk approach is well-deployeâ, with no major gaps; deployrnent may Vary mong some areas or work units

a sound, systematic approach, fully responsive to al1 the requirements of the item

1 a very nrong, fut-based improvement

I pm&s is a key managem&t tool; nrong refinement and integrauon - backed by excellent axmlysis approach is f d y deployed without any sipificuit weaknesses or gaps in any ueas

R c d s no resuits or poor results in areas reponed

evfy sages of âeveloping trends; some improvements d m eariy good performance levels in a few areas results not r e p o d for mvry to most areas of importance to the cornpuiy's key business requirrments

improvement trends di good performance levels reporteci for m;ury to moa ueu of importince to the company's kqr business fequirements no p a m of adverse trends d m poor performance lweis in areas of importance to the appliant's key business requinmenu some trends and/or current perfommce lwels-evaiuated against relevant cornparisons d m bencbmuks-show areas of svengrh and/or good to very good relative performance levels

current performance is good to excellent in most key ares of importance to the company's key business requirements most improvement trends andlor current performance lwels are sustained

muiy to rnost trends and/or current performance levels-evaluated a g a , relevant cornparisons and/or benchmuks- show areas of leadership and very good relative performance lwels current performance is excelient in most ares of impoxtance to the company's key business requirements excellent improvement mds and/or sustained excelent performance in most a r e s mong evidence of industry and benchmvk leadership dernonsvated in many areas

In self-assessment, once the evduation of each element is complete, the cmmhtive scores

are used to calculate a fnal score. In both the EQA and the MBNQk this is a score out of

lûûû. These IWO points have k e n distributed among the caregories of the award by the

developers of these modeis based on the perceived imponanœ of each.

Table 2.8 uses the MBNQA categones to illusvate how a final score is calculated. First, each

category is evaluated out of 100% according to the process described previously. Nem, each

score is calculated based on the total poim value of the categoty and the evaluation v h .

For the Leadership category, the evaluation wu 60%, the total point value for the category is

125 so the score for Leadership is 75 (125 x 60%).

In an initial assessment such as this, mon organisations do not score more than 250 out of

1000 points. World class perfoxmance would achieve a score of around 700 points, while a

C;rtegory Leadership Srrategic Phnning Custorner and Market Focus Information and A d y m Human Resource Focus

score of 500 is exception@ good (Public Service Office, 2001)

Evduation 60% 45% 55% 60% 52%

Point Vaiues

85 450 IO00

L

A score of 200 is reporte* typical for an initiai assessment of an organisation at the

beglining of a Baldxige-based quality effort (McWaid, and Gale, 1995).

Aau;il Score 75

38.25 46.75

51 43.35

It is important to emphasise though, the importance of improvement over the a d score

M h u m Score 125 85 85 85 85

34 157.5

370.85

Process Management Business Resulfs Totai

achieved in any specific assessment.

40% 35%

Qrulity auditing is another method rhP o e t i o n s un use to meastue their performance.

A quality audit is

A systematic, independent, and documenteci proces for ob- audit M d m c e , and waluati~ it objecüvely to determine the extent to which agreed aiteria ue fulfilled

(ISO 9000: 2000)

Severai elernents are essential to the performa~lce of an effective audit Fint, there is the

audit standard. Standards provide guidance on the prinaples of auditing, the management

of audit programs, the conduct of a& as weil as the competence of auditors (ISQ 19011:

2002). ISO 1001 1 is the m t standard for @ty management system audi*. A new

auditkg standard however, ISO 1901 1, is being drafted to integrate quality and

environmental management system au&.

There are three ~ / p e s of audits that c m be p e r f o m d These are first, second and rhird

parry audits. First puty audits, o t h e W e known as intemal audits, are performed by the

organisation on itself. Second p a q audits are initiated by a customer (die dient) to venfy

that the organisation meas their quality requirernents. Third party audits are refemed to as

exvinsic audits. These are performed by an outside f m calied a registrar and are prompted

by the organisation directiy (Goetsch and Davis, 1998).

Before performing the audit, its scope m m be defmed. This should be based on the

objectives or information desired byrhe client and should be communicated to the auditee

pnor to the audit (ISO 14010,1995). It is prudent ro do so to ensure that the auditee is

prepared wirh necessary documents or objective evidence to demonstrate cornpliance to the

cnteria being auditeci.

Audit criteria are the set of policies, procedures or requirements used as a reference (ISO

1901 1,2002). Audit evidence are the records, statements of fact or other information, which

evaluation of the coiiected a d t evidence a g d the audit &ria.

Audits should not be stand-done processes, but part of an overali auditing system (Goetsch

and Davis, 1998; Karapetrovic and Wdbom, 2ûûlA). Wdibom and Cheng (1994) suggew

thar a badine audit (a more cornprehensive and inteasive audit) establishes the starting

effectiveness of the sysrem and t h du+ the eariy pend of implementation, regulu audits

should be scheduled at reg& intervals to mainuia the inaeases in effectiveness of the

system. As the system matures and the effectiveness stabilises, audits should be scheduied

less frequendy.

The execution of any audit indudes sweral steps. ISO 19011 prodes a dugram to illustrate

the activities involved in the auditing process (See Figure 2-5)

T In i t ibg the audit Appointhg the audit team leader Defining objectives, scope and criteria Determinhg the feasibility of the audit Estabidmg the audit team

Establishing i n i d contan with the auditee

Conduaing document revicw 1 Review relevant management system documents, induding record, and determining their adequacy

w

Prcparing for in-site audit actRritics 1 Preparethe audit p h I / Arrigning woik to the audit team I

Gmducting on-site wdit actinties ConducMgopeningmccting Communication during the audit Roks and responsibilRies of gui& and observen Cdlectiag aud verifying informaion Genvllring audit fin+ Prepuing audits conciusions Conducting closing mecting

Preparing, approving and distributhg the audit rtport

Prepariq the audit repon Approving and distributhg the audit report

I Completing the wdit RRÎinuigd~cuments

F i the audit I ...-... ---...-...- --- ....

Conducting audit foilow-up

The objectives of a quality audit, as outlined by the ISO IO0 1 1 standard, are to:

determine the conformity or nonconformity of the quality system elernents with specified requirements

determine the effeaiveness of the implemented quality v e m in meeting specified

quality objectives

provide the auditee with an opportunity to improve the @ty systern

meet regulatory requirements and

permit the listing of the audited organisation's quality system in a regiaer

Audits of management sysrems are invaluable in assuring that a system is perfomhg

effenively and meeting its objectives. It is also a necessaxy component in the continuous

improvement process. The achievemem of continu ou^ improvement can o d y occur if

quality audits are used in a dynamic and proactive rnanner (w;ubom, 1990; Banhelezlly and

Zairi, 1994; and Peters, 1998). If organisation use a u d i a only as a tool to verify

cornpliance widi agreed standYb wïthout dieduog the aptness of the standards thernselves

or seeking information about the effectiveness of the q d t y system in meeting objectives

(Beeler, 1999), auditiug will not lead to impmvement.

Result-bued perfomunce measurement is a third m n h d for meaniring performance.

Redt-based measurement system is an integnl pan of an organisation's performance

management system. It is a means of gathering data to support and CO-orciinare the process

of making decisions and taking action throughout the organisation (Schallcwyk, 1998)

Organisations, and researchers in the performance management field, have developed many

performance mevurement models. These models range from a balanced set of indicators

that periodicdy show an organisation how it is performing in various areas of the business,

to integrated models that provide real t h e information with respect ro how thqr are

performing cmently and forecasts how thq. mry do in the future. As competitiveness

increases, organisations require measurement synems that provide not onty up to date

performance idonnation but also examine how these meanires interrelate so they can make

more informed decisions. Performance measurement systems should be aligned with

organisationai goals and objectives. This helps to ensure that the organisation is measuring

the efficiency and the effectiveness of the activities that have the greaten effect on the

achievement of these goals.

The set of measures selected by a parcicular organisation should provide m e n t information

penaining to the health of the organisation and also link direcrty to suategic objectives.

Traditional perfomiance meaSuTemenrs systems are based m a d y on f inand performance

measures 6.e. retum on invesunent, renun on sdes, productivity and profit per unit

production) that were once sufficient to s u c c e s s ~ express a compmy's perfomunce.

However, organisations need more than a badrwvd giance at how w d rhey have done over

the past p e n d Lgging metrics, apparent on firunad reports, were the r e d t of past

decisions and are not seen as usefd in operational performance asessrnent (Ghalayuii and

Noble, 1996). An organisation neeck to see how w d ir is doing now, in t e m of con, t h e ,

flexibility and quality. Not oniy do they need to k m their current perfomimce but also

how thqr CUI improve this perfomiance. Neely (1999) suggests that the mevures in today's

Company need to be:

focused on organisational strategy

provide data on @y, responsiveness and flexibility

encourage managers to se& improvement continually

provide information on what cunomers want and how cornpetitors are performiag

indicate what would happen in the short r e m and the long term

Selecting a set of measures that will fulfil these requirements can be a daunting task. One

method that researchers have used is to i d e n e uiticai performance areas and

corresponding meanires to coiiea the important data This data is then used to control and

improve performance in these key areas.

It is becoming increasingly important for companies to understand how all areas of their

organisation are perforrning. If properly consvucted, performvv~ meanires provide

information to management to enable fact-based decision making. They are also important

for effective communication. Their use increases constructive problern solving, increases

influence, monitors progress, and gives feedback and reinforces behaviour (Sinclair and

ZaLi, 1995A). Performance measuremenr helps an organisation allocate scarce resources to

processes with weaker perfomiance because it highghts areas that require these resources

the most.

If an organisation dwelops performance mevurrs for each process, this may result in a very

large lia of measures. The question is, how does one select a few kqr measures that wili

ensure manageable data collection but yet provide a good indication of the naus of the

organisation's health.

M q authon have trieci to develop guidelines or approaches that will help in this selection

process (Keegan et al., 1989, Fitzgerald et ai., 1991, Kaplan and Norton, 1996, Dixon et al.,

1990, Neeiy et al., 1995, Globenon, 1985). To d s e the contributions of these

authors, performance measures should:

be linked to strategy

stimulate action

support long-terni improvement

emphasise whar is important to the organisation currently

cover ali the appropriate elements of perfonnance, not jus the finanaal ones

not be in conflict with each other

enable cornparison with other organisations in the same business

simple and easy to use

be mainly ratio-based Uistead of an absolute nwnber

be under convol of the evaluated organisational unir

be desiped based on discussions with the people involved (cunomers, employees,

managers)

be mady objective

Vary berween locations - one meanire is not suitable for all departmenu or sites

change as circumaances do

provide fan feedback

stimulate continuous improvement rather than simply monitor

Nurnerous performance measurernent frameworks have been developed since the late 1980s.

These framewoh provide information about the types of measures an organisation c m use

to track perfoxmance in a "baiancedm way. This ensures the Company is not just lwking at

its financiai situation as an indicator of its perfomunce but that all elements of performance

are being m e d and m o n i t o d Two of such h e w o & are the Balanced Scorecvd

developed by Kaplan and Norton (1996) and the model from Neeiy et al. (1995), which

consists of four elemenrs of perfoimance.

The Balanced Scorecard anernpts to capture an organisation's performance from four

different perspectives. The first is a f i n a n d perspective, Le. how the organisation looks to

th& shareholden. The second is an intemal business perspective, which asks at what the

organisation should excel. The third looks at how the organisation appean to the cwomers

and the founh perspective is one of innovazion and leaming which asks how the Company

cm Lnprove and create value.

This nructure is baseci on the premise t h to achieve finuiciai success, the dt;matr? purpose of a q organisation must be to satisfy customers. So orginisltons mut optimise their intemai vaiue-creating processes. To do so, they mua l m and th& employees must grow in their individual capabilities. (APQÇ 2 w

Neeb et al (1999) suggest that an organisation's m a n u f a h g performance measures

should fd into four categones. These are quality, Ume, cost and flexibility. These cm be

extended to cover the performance aspects of other deparmients as wd. The performance

of design and development processes for instance can be viewed from these four

perspectives.

2.2.2.4 Cornparison of Performance Measurement Methods

Three methods of performance meanirement have been presented in this lirerature review.

This section wdl review the work done to compare these methods in terms of their

applicability and usefulness to organisations.

2.2.2.4.1 Auditing and Self-Assessrnent

Auditing and self-assessrnent are systematic approaches used to measure performance in an

organisation. The audits used with regard to ISO 9000 series reginnuon are mady looking

for non-compliance and wess whether the system and the undedying procedures are being

followed. O n the other hanci, self-asessrnent against the a w d modek is focused on

fmding improvement opportunities and new wys in WU to drive the organisation d o m

the road to business excellence (van der Weile et al., 1995).

Both audits and self-assessments shouid be dosed loop processes that foiiow Deming's

PLAN-DO-CHECK-ACT loop. Thk mearis planning for the audit or assessment,

perfomiing the a d t or assessment, reviewing rhe auditing or assessment process for

adequacy and effeheness, and making impmvements to the process as identifed in the

review.

Each of these processes is used as an evaluation method to detemine the aatus of the

current syscem A series of audits can be used to meawe the achiwed improvement of an

organisation's quaIity system but do little to indicate how fast it is going in a chosen direction

of improvement (effuency) (Karapetrovic and Wdbom, 2001B). Self-assessments are able

to measure this efficiency through the a-s of current approaches to and deployment of

these approaches satisfying a given set of BEM cnteria. The uialysis should indicate

whether the current approaches are sound and integrated and whether there is systematic

and implemented deployment of these approaches. The remit is the ability to predict

whether these are sufficient to lead to improvement of the system. Thus, the objective of an

audit is to venfy compliance with a set of cnteria and self-assessment is used to examine the

driven for improvement (Karapetrovic, and Wdborn, 2001A).

The r e d t of a quality audit is either compliance or non-cornpliance with the criteria. Either

the organisation meets the criteria or it does not In a self-assessrnent process, the objective

is not to cornply with the cntena but to use it as a frarnework for idendjing svengths and

oppominities for improvement in each area of the BEM cxkeria.

When it cornes to perfonning the audit or assessment, the procedure is sirnilar. Both

processes are designeci around DemLig's Plan-Do-Che&-Act d e .

I T ~dentify o ~ o x d s u u c n i r e and mq&d cornpetence for audit program management

1 1 Define audit .. pr- -. . objectives, wope, mler

I 1 0 Ennve cornpetence and suitab* of auditors

DO

I 1 Cummuniute audit pian

and respoasibdiaes Identifywdit teams

I 1 Manage the planning and suitability of auâitors I 1 Gxnmuniute audit pian I I Manage the piamhg and arecution of

I 1 Perfm foÏlow-uP actions and/or audits

Select approach and scope

A m

Establish d-assessrnent teams

Execute wess ment

perfofxxLlnce Review and irnprove audit prognm h u g h feedback and recommendations from the dent. a& and auditor

Identify r e q d actions 0 Incorponte actions into p h

Implement pians

Karapevovic and Wdbm (2001A) compare the principles and practices of qualicy audits

and self-assessments for the purpose of examinlig th& compatibihty and providing the

basis for integration.

The authors suggest severai suategies for increasing the capability and potential aiigriment of

audits and self-assessrnenu. These include:

Integration of reference models. This smtegy may involve a g r a d d conversion from

the basic requirements of ISO 9000 to the all-encompassing criteria of excelience

models, suggesred in literafufe. In that sense, self-assessments incorporate quality audits.

Integration of evduation merhodologies. Qwlity audits w o d be used for assessing

'hard controls", while self-usessments would be used for 'softer" issues and idenufying improvemenu. In pnctice, this strategy is already applied.

Goss-use. Quaiity audits can be applied to enhance self-assessments, and vice-versa.

2.3 Perfo~naflice Management in RkD

A large portion of the quîliry problems rhat vise on the manufactuxing floor could have

been avoided completely with proper attention to the design process. Quaiity cannot be

tested into a manufactureci p d u a , but musc be designecl imo the produa. Most of the

focus of quality management research however, &als Mth the manufacniring and s e ~ c e

secton and lacks RSrD focus. The quality management pracrices that are successfd in

rnanufacniring environments are wd known and widely publishd The application of these

manufacnuing-based d t y pnctices in an R8rD environment is not as successful due to

the lads of replication and high level of variability in RBd) projeas (Kumar and Boyle,

2001). There are also high levels of uncextainry in terms of the impact of the produa on the

market and the revenue it wiU generare over the followhg years. It is more difficult to apply

@ty practices in such an environment however, the undedying concepts of quality

management are essenria to decreasing this variab'ility in the R&D processes. Research is

needed to examine an R&D operation and idennfy pertinent quality management aspects

that are applicable in nich an environment. To adequately address this problem, there needs

to be an understanding of issues that are specific to an R&D organisation.

2.3.1 Addressing Issues Unique to R&D

Individuai Recognition of Achievement and Organisational Goais vs. R&D

Objectives

EspecialIy in North Amencan industry, the manner in which researchers and scientists in

R&D roles have been recognised and awarded has fostered a very individualistic work ethic.

Researchers often prosper by phcipating in a system that rewards individual achievement

and discourages alliance building and CO-operation (McLaughlui, 1995). This individual

recognition crûares an amiosphere plagued with individds working toward goals that will move hem ahead without due consideration for the overall goals of the RgrD funaion itself.

This is counterpfoductive to the achievement of these goais. Individual performance and

mards need to be replaced with systems that reward team accomplishments.

As individuah are rewarded for their personal contributions to R&D activities, the

environment becomes one that encourages the achievemat of persona1 goals

The (R&D) environment is academic and contributions to and support of o%;iniszuonai goals and objectives are minimai. In fact, mon researchen are unrwve of w h t the o ~ u o d goah and objectives are.

(Swyt, 1999)

It is exuemely important that r e s e d organisations ensure they are focused on projects that

address the saaregic needs of the organisation and that rhese needs are being met effuendy.

Fast Pace and Factors that Increase Cycle Tirne

In today's competitive marketplace, companies are faced d the prospect of introducing

new products at a fnghtening pace. The ability to reduce one's cyde rime in product

development is viewed as the key to innovation nuicess and profitability (Cooper and

Kleinschmidt, 1994). The r ed t is a need for efficient and effective new produa

development (NPD) processes that will ailow the Company to mainuin its competitiveness

within its indunry. What R&D facilities are facing now ody increases the pressure while

acting as a barxier sometimes to the achievement of good processes and competitive outputs.

There is a need for a systematic approach to irnproving design and development processes,

and the management systems surrounding hem, to provide the bea environment for

&taking cornpeutive advanrage.

Many of the factors that influence NPD cyde Ume are outside the R&D department's

control, making the process that much more difficult. Coprate decision making, customer

needs, and orpan;Sational goals are all factors that impact the t h e to market of many new

products (McLaughLR 1995).

Fewer resources

As companies downsize as a meam of c o s CU&& the R%D department is ofren the hm target.

Poor or uncertain economic conditions cui cause havoc with R&û funcihg and personnel. These uncertain conditions have a dmmxic and powerful psychologid effect on projects, innovation, and creativity.

@kLaughttr, 1995)

Executives expect that the R&D function will perform successfullywith reduced resources

and unrealisric objectives (Kumar and Bayle7 2001). This reduction in resources affects the

spirit of the R8rD teams d o meanwhile must nniggle to keep improving cyde times to

keep up with the market.

W& moa research organisations there is a constant pressure to produce more results with

less budget in less rime. It is imperative therefore t h research organisations ensure their

cnticd resources, the research naff, are focused on proieas that address the strategic needs

of the organisation and that these needs are being met efficiendy (Endres, 1992).

Increased competition

R&D or new ~roduct development is one of the mon important sources of cornpetitive

advantage for a f m (Jayawama and Pearson, 2001). With increased competition in today's

global marketplace, organisations need to not oniy satisfy customers, they need to M e

them widi their ability to foresee and fulfil all their requirements. Gmpanies that do diis the

best are dependent on the^ R&D departments to recognise the needs of their target markets

and translate them into produas and services efficiendy and effectively.

Emergencc of ncw technology

The unergence of new technology gready impacts the RBrD environment. RSrD mua view

tedinology and scient& advames as tools to assis the o~anisation in meeting Company

goais (McLa.gbi.i, 1995). Inevitabiy, tedinology must be manageci within the R%D

organisation to benefit all employees/shuaiolders. The ability to manage effectively can

best be achieved through the implemenmion of quîlay management priuciples

(McLaughlin, 1995).

Capturing the customa's changing needs

The R&D organisation mus be driven by its responsiveness to changing cunomer needs.

This is one of the greatest challenges for RBrD hinctions. At the initial stages of a new

product development project, researchers and scientists musr accurat* and precisely

ascertain customer requirements. These are then translateci into a set of specifications chat

defme the proâuct to be designed. A new challenge arises when customers request changes

ro rheir on& requirements in the rnidst of the design stage. This may only necessitate

s m d changes to the design, but could result in the design team starcing again from scratch.

These changes increase cyde rime, costs, penon houn and affect the RSrD schedule, rying

up employees that are requireà on other projects.

As a r e d t of these factors, the need for teamwork, efficient and effecrive management

sysrems, and appropriate management of technologies has become critical (McLaughLn,

1995).

2.3.2 Benefits of Implementing Quality Management in an R&D Environment

One of the main benefits of irnplementing quality management in an R&D environment that

authors have reponed is the darification of the defdtion of 'quality in RBrD' (Pa&o, 1997,

Kumar and Boyle, 2001, Pearson, et ai., 1998). Several &finitions have been suggesteù. A

review of litaamre by K w and Boyle (200 1) provides the following components of

'quaiity in R&D':

an understanding of who the R8rD client is and his or her values and expectations

what the key technologies are and how they can be used to meet R&D clients'

expectations and the needs of the entire organisation

who the R&D cornpetitors are and how t h q will respond to emerging R&D clients

needs

Qing diu igs nght once you know you are working on the nght things? c o n c e n t h g on

continually improving the system.

enabling people by removing barrien, and encoUrap;ig people to make their maximum

connibution

Many authors have written about the need to have systemauc processes for managing quality

to gain and remain in a cornpetitive position (Cooper et d, 1990, Colurao, 1998, Auer et al.,

1996). The result of irnplementing such a q d t y approach is a hm that continuody

serves the needs of its customers (both internai and extemai) in an efficient and effective

manner (McLaughhn, 1995).

Q d t y awarb can be w d as a yardstick for measuring 'quality in R&D' progress and

idenufylng gaps for improvement, as well as providuig a comrnon focus or direction for

the whole Company. (Pearson, et al., 1998)

Systematic decision making, continuous improvement, experirnentation with new ideas

and teamwork are important for a successful research and developnieit environment

( G d , 1993).

Several reasons have been provided in the literature for the limited use of quality

management practices in R8rD. These are:

concem that on of a qrulity myiagement prognm in RBrD nu& d e creative

woik and be counterpductive (Brown, et d., 1994)

the dificulty of meamring performance in R%D (Montma, 1992)

Lck of vainiog and acceptame by RBrD personnel of the "sofiw side of @ty, such as

teamwork and participation management (Montana, 1992)

the feu that it will raise u~vea~onable expectations (i.e. ahuays dohg it nght the frrn thne) (PaGo, 1997)

concem thar it will d e proper recognition of individual accomplishment (Parino, 1997)

ciifferences in inrerpretation as to what 4 t y realiy is and how it is to be pmued

(Francis, 1992)

PaGo, (1997) suggesfs that some of these barriers can be overcome if there is sufficent

dialogue and darity as to whar ' M t y in RBrD' means.

Several authon have discussed the implications of implementing ISO 9000 in RgrD

environmenu (Cooper et ai, 1990, Davidson and Pniden, 1996, DeWeed-Nederbf et al.,

1995, McLaughlin, 1995).

Process driven quality synem mch as ISO 9001 are potenwüy of great vdue to control

the quality of the R&D process (Auer et al., 1996).

ISO 9001: 2000 provides an "ideal" oppominity for R%D managers to achieve the

benefits of total quality management and thus improvement in various kqr performance

dunensions (Jayawama and Peuson, 2001).

W1thin the RBrD context, ISO 900 1 is becomuig more popular as a way of making

people think and plan ahead and as providing a disciplineci way of capniring the R8rD

redts (Cooper et ai., 1990).

The revised version of ISO 9001 look into the dyaamic chamter of organisations m

which issues such as leadership, people involvement, v e r n approach to management,

continuous ;.iprovernent and a fact -basai approach to decision-making for ewnple,

receive speciai attention. The need for more dynaxmc approach ro RBrD management

has been emphasised for yeus (De Weed-Nederfiof et al., 1995).

ISO 9001 addresses both the issues of better convol and continuousty raising the R&D

standards (Iayawama and Pearson, îûûl).

ISO 9001 is a good means of providmg a framework for developing a quality syscern in

R&D. It outlines the organisational nnicnup, management responsibilities, procedures

and processes r e q d to set the base for a "hoiistic @ty management system"

(Davidson and Pruden, 1996).

The Company can adapt (select) from the specifications in the new standard, enabhg

them to ensure a good fit of the standard to their R&D activities.

The main advaatage of ISO 9000 in RBrD is that it provides a structure and consistency to

an otherwise chaotic process.

'The issues relevant to the application of quaiity management pnctices in an R&D

envitonment have been addressed in this section. The n a section provides an o v e ~ e w of

the motivation for the proposed researdi.

2.4 Motivation for Roposed Research

In light of the topics reviewed in this litenture search, the following points outline the

motivation for the proposed research.

7he ISO 9001 standard for q d t y management systerns is a stepping aone on the path

to business excellence. Extending these minimum requirements for a quality

management system to encompass ail organisational processes can be a daunting task

Methodologies for evduation of a d vernis desirable systern statu have been

developed for quality management systerns (quality auditing) and for overd business

performance (self-assessment). Both of these mdodologies have been proven to be

essentiai to an organisation's compet;tiveness. There is a need to analyse the

commonalties and differences between these method01ogies in order to provide a

theoretical baclspufld for th& b e n t and possible integtation (Karapetrovic and

~ b o m , 2001A).

Result-based performance masurement k a method01ogy for measuring an

organisation's performance. Organrwtions are Ehallenged ro select a rneaningful set of

performance measures thar wili provide a broad view of organisational performance.

Whde the abject of q d t y management in a manufacturing environment has been wd

addresseci, the application of these methodologies and theu implications to an RBrD

environment hu not. Further research is needed to explore the applicability of quhty

management frameworks in rhis type of environment.

2.5 Objectives of Proposed Research

This section describes the objectives of the proposed research. These are to:

develop an integrated model for performance management based on the ISO 9001

standard for @ty management and three national @ty awards to provide a reference

model for organisations wishing to move from the minimum requirements for their

quaiity management system to overall business excellence

condua a self-assessment against diis new reference model in a case study organisation

compare the self-assessrnent renilrs to the outcornes of a quality audit to furcher

investigate the opportuniries for integration of these masurement methods

review the redt-based performance measurernent system within the case mdy organisation and compare rhe outputs to the self-assessrnent and the quality audit

moddy the reference model for more appropriate evaluation in an R&D environment

3.0 An Integrated Mode1 For Performance Management

3.1 Introduction

This diapter addresses the development of an integrated model for performance

management combining the concepts and criteria from the ISO 9000: 2000 standards and

three nationai quality a d . The chapter describes the development of the framework for

the rnodel and correspondhg criteria, and the establishment of a scoring scheme.

3.2 Development of the OPIM

Organisations roday must reach bqrond the minimum requkmenrs for their qualiry systws

using a more holisric approach to improve aii elements of their business. The minimum

standard ISO 9001 for an organisation's @ty system is d considered to be essential to

the foundation of all quality improvernent efforts (Karapeuovic and Wiübom, 2001B; Dale,

1999; van der Wiele et al., 2000; Russel, 2000). As these requiremenu are achieved

however, new lwels of improvement need to be reached. Organisations musr continue to

ensure that thv are meeting the requirements of the standard and also are moving forward

with their improvement effoxts. The integration of elements of the ISO 9001 standard and

the criteria from the more holistic quai~qwliry award models for business excellence provides a

framework that can be used for organisatid improvement.

This section describes the development of the new mode1 for qualiry management named

the Organisational Performance Irnprovement Mode1 (OPIM). The Grrr part illustrates its

framework or underlymg suucture. The next sub-section discusses the application of crireria

from the other models to this framework Fin*, the development of an appropriate

scoring scherne is described.

3.2.1 Framework Development

The framewok for both the ISO 9001: 2000 standard and the &ty a w d are based on

severai undertying principles (see section 2.2.1). The fm stage in the Qvelopmem of the

A system is a set of interdependent processes that function haxmoniousiy, using various

resources, to achieve established objectives. Processes transform inputs h o outputs. The

effectiveness and efficiency of the system's operations and use of resources, is quantdieci in

the system's performance results. Applying this definition to any set of processes results Li a

"system's view" (Kuapetrovic and Willbom, 1998A).

A framework based on this 'system's view" wadd result in a comprehensive model

incorponthg all of the essential elernents of the organisation.

The OPIM is a model that describes the C S 0 7 s quality system. Ir focuses on the continuous

meanwmenq monitoring and control of the sysrem's performance and its impmvement

(Kanpetrovic and Macey, 2001). Figure 3-1 represenrs the systems concept for applicability

to the OHM. This concept is based on previous research by Kvapetrovic and Wdborn

(1998A). The dotted lines nirrounàing the objectives and resources are meant to emphasise

rhe active role of the processes in the system and to indicate that these elemenrs are used as

inputs to the F e m not to de-emphasise their importance.

OBJEC77vES i MANAGEMENT

PROCESSES

PROCESSES MANAGEMENT

RESOURCES

......................... .......................................... V II 1 : I RESULTS I

PERFORMANCE MEASUREMENT, MONITORING, CONTROL & IMPROVEMENT

Each element of the synem portrayeci in Figure 3-1 is a main category of the OPIM

framework These are:

1. Objectives

2. Resources

3. Processes

4. Results

5. Performance

The fm sep in the integrauon of other models to fonn the OPIM was to use these

categones and to devise sevenl sub-categones that were thought to encompass the purpose

of each main category. The subsategories were created from a review of the content and

purpose of each of the a w d models un& evduation and the ISO 9001: 2000 nuidud

Table 3-1 shows the i n i d set of caregoris and ~~b-categories of the OPIM.

1.0 o b j j v e s 1.1 Intacncd parlia

1.1.1 Cunoma 1.1.2 Ownas/ i n v a t o n 1.1.3 People 1.1.4 Supplias d parTnen 1.1.5 Society

1.2 Mui;ig~ll~llt syacm 1.2.1 Muugtmmt rtsponsibïhy 12.2 Mur;igcment mnew

1.3 Legd 2.0 Rrsouras

2.1 People 2.1.1 Acquisnion 2.1.2 Deploymait 2.1.3 Worksynmis 2.1.4 Wdl-king and saisfaaion 2.1 .S TrJning

2 2 L?addip 2.2.1 lLsponsibilicics 2.22 Policy and nrrtegv

2.3 Infomution 2.4 infnnnicnuc 2.5 TechnoIogy 2.6 Supplim/purnaships 2.7 I 4 k c d s 2.8 N d resourccs 2.9 F i c e s

3.0 P r o c e ~ ~ s 3.1 Design and/or dcvelopmuir

3.1.1 PlaMlIlg 3.1.2 Lnprtts 3.1 -3 Aaiviues 3.1.4 Chpus 3.1.5 Revicw 3.1.6 Verification 3.1.7 Validation 3.1.8 Convol o f changes 3.1.9 Design and developmcnt o f p r ~ ~ e ~ ~ s

3 3 Product =&or ~ t rv i ce 3.2.1 1drnrif;caxion of producc mUor service processes 32.2 Idmrifiminn and c r a c d i 32.3 Produa d / o r m i c e ourput 32.4 Graama propeny 32.5 Vdidah of pnxases 3.2.6 Gntrol o f me;;llairing a d monitoring devices

3.3 RvEfuung 3.3.1 Idemifiution of nced 3.32 kifofmaricm 3.3.3 Vdarion o f purchascd product and/or services

3.4 Support Proccsscs 3 .S. 1 Idear.i6&0n

3.5 Pluinulp 4.0 Renrhs

4.1 Cunoma 4.2 F d and uurket 4.3 People 4.4 Supplia and m e r 4.5 Soàery 4.6 G p n k i o d effectivena

5.0 lmprovanent 5.1 monitoring and control

5.1.1 Synmi performance 5.1.2 Cuscoma satisfaction 5.1.3 i n t d audit 5.1.4 Processes 5.1.5 Prodm a d o r service 5.1.6 GICIM~ of nonconformhy

5.2 An;ityPs o f clam 5.3 6 d v e anion 5.4 Prrvcntive acûon

3.2.2 Criteria Development

The OPIM shodd be considered as a combinarion of q d t y asnvance standards that

provide the minimum requirernents for q d t y improvement and business excellence

rnodels, illustating the opportuniues for improvement. The criteria in each categov of the

mode1 are split into two parts. The fmt presents the minimum requkments for the quality

management system. The second provides opportunities for improvement over and above

these minimum requirements, descnbing the continuous improvement of the category. This

split wiU enable ui organisation to maintain ifs quaiiv management system as a foundvion

for its conkuouc impmvement efforts. niis nsn section describes the dwebpment of the

criteria for the OPIM.

Once the categories of the OPIM were created, each awad model was reviewed to provide

an i n i d fit of rheir categories and sub-categories into the OPIM framework Table 3-2

shows these matches.

OPIM 1 ISO9ûûi: ( IS09004-. 1 MBNQA I EQA I DP

OPIM

d o r u n i i c c ~ 3 1 2 khtificPioa d traceabdÜ'y 33.3 Produa d o r savice

ISO 9001: rOOO

32.4 propaty 32.5 VIliduion of prousses 3.2.6 Cbtmf of masuxing and

After ensuring that moa of the categories from the models were mapped to the OPIM, the

contents of the awards and ISO 9001 were reviewed to ensure that the categorymatches

were appropriate. This process resulted in the rearrangement and renarning of some of the

caregones in the OPIM to provide a more streamllied set of cnteria.

ISO 9001: 2060

7-53 7.3.8

conml 5.1.1 S ~ c m p a f o ~ c e 5.1.2 G~~amn satisfaction

Category 5.0, labelied Irnprovement, is provided as an example. Irnprovement cannot corne

from corrective (5.3) and preventive (5.4) actions done because they just remove the causes

(existing or otherwise potential) of problems. They do not lead to Breakthroughs, which break new ground in improvement. Breakchroughs cm be graduai (Kaizen) or innovation /

EQA

32 Rohua d o r service

MBNQA

75.3 7.55 7.6

DP

5d

I 7.53

1

82.4

6.7 32.1 Idadkaion of ~roduct

75.1 , 75 I 1

7.6

4.1

6.2

6.7

5.1.3 hami audi 1 8 2

73.9 1

8.2.13 5.1.4 Proces~s 5.1.5 l'mduci urd/or savicc 5.1.6 GmmI of noaconfonnity

5.2 AiillyPs of d m 5.3 Cormxivc action 5.4 Prcvmuvc action

1

6.6 6.5. 6.4 6.7

8 2 2 82.3 83 8.4

83 1

Sb,% Sb, Se

8.4 8.52 8.53

3.3.8.3 6.5.62 IO

4.2 6.1.6263 6.1.6.2, 6 3

Anorher review of the category 5.0 rwealed that it &O indudeci analyus of data, intemal

auditing, and measurement and monitoring These elements are all a pan of an

organisation's performance management system. The category was renamed Perfozmance

under which three sub-categories were creatd Management, Monitoring and Conmiling,

and Improvement. Matching elexnenu from the other models were reshuffled to fit into the

new structure of the OPIM.

Table 3-3 illustrates the process as describeci above. The first parc of the table shows the

initial sub-categones in category 5.0 and correspondmg criteria The second pan shows the

modXed mb-categoies and their comespondlig criteria.

5.0 hprovemcnt 5.1 Mcinirrmair, monitoring and convoi

5.1.1 S y n a n pafomunce 5.12 Luorner satisfaction 5.1.3 I n t e r d audit 5.1.4 P m r s ~ s 5.1.5 Product and/or semice 5.1.6 Conml of nonconformiry

5.1 An3ysir of duî 5.3 Conmive: action 5.4 Prcvmtive action

5.0 Perfonnancc 5.1 Management

5.1.1 Spem 5.1.2 Key Pcrformuicc Outcomcs 5.1.3 K y Perfonnancc Indiuton

5.2 Monitoring and Controlluig 5.2.1 Audits 5.2.2 Self-Assasrnaits 5.2.3 Puformancc M a s u m e n t

5.3 Improvanent 5.3.1 C o k v e and Prnteative Actions 5.3.2 Kaizcn d Quuinun Laps

8.2.1.3 82.1.5 8.1,8.2.1. 8.4 8.5.1 8.52,8.5.3 8.5.4, Anna

MBNQA EQA

illustrates the caregones and comsponding de~cn~tiom for categoxy 5.0 Performance.

F i OPIM 5.0 Performance

5.1 Mui;igcxnent 5.11 Systan -

Describe the +an applied to masure org;iniutionai performance. Illuruve the inurrclîtioriships mong the internai audit, d-assusment and rcsuit -bd syncms for performance rnasurrment.

O

Describe haw rht i.urrcl;uionships of i n d audits, d-wessmcats and rrsuh m~surrmcnts are managcd and imprwd

5.1.2 Key Performance Outcornes (EQA, 91) Rczpciramnr Desaibe the +cm appiieci to nmmre the ky paforrmnce outcornes idurtified in section 4.6.3 of OPM

Describe how the mcmmmnt v a n for ky performance outcornes is continuaily impmtd 5. t .3 Key Perfoxmance Indicltors (EQA, Sb) -

Describe the system applied to mcasurc the key performance outcornes idendicd in Kctions 4.6.1 and 4.62 of OPM

O

Describe how the meuwpient spem for key performance indiators is contll idy impmved

5.2 Monitoring and Conmlling 5.2.1 Audits

Repmaens (ISO 1,8.2.2) Dexnbe the syscun applied to perfonn internai and extemai audits against the rquircmcnts of the ~lected management nandards, induding ISO 9001: 2Cûû and/or ISO 14001:1996. Qpmozm ( I D , 8.2.1.3) Describe h m the effectiventss, effiaency and consistency d the audit sysrun are continuaüy ;mpmvcd.

5.2.2 Self-Assessments (IS04,8.2.1.5) - Describe the system applied to perform rcguiar self-assessrnenu against the The CS0 Performance improvcment M d

0 qyJmmm3 Describe h m the effmiveness, efficiuicy and consisturcy of the self-wcssmcnt synern are concinuaiiy improveci

3-4 I-

5.2.3 Performylce~wumncnt~SOI,8.1,821,8.4) (MBNQA, 4.1al and 42)

Describe the performance mevurrmait systan d to collea, moaitor, adysc, synthcsise and 1 act upon puvpioüs (cnan;J) and performance indiators (iuitamal) &ring to:

- C U S t O ~ - shareholdus - people - suppkandputnas - -ccy

(MBNQA, 4.1d) Descrik h m the effecUvcness, efficimcy aud connsVncy of the performuice masurement systcm arc contindy impmved

5.3 Improvemcut(ISO1,8.5.1; IS04,8.5.1;DP,9) 5.3.1 Corrrcuve and Pmr& AcÛons

i?qmmm ( m l , 8.52 and 8-53) Describe the v a n applied to m o ~ o r , idenufy, +, and rcmzve the cxisting ;and potenriai auses

l of nonconformiries in in- pmccws, outprru, objectives, rcsources and d u . (IS04,8.5.2 and 8-53)

Desaibc how the conrctive and prevcntive actions ur used in the business piau and how theu effcctivaress is continuîlty improvcd.

5.3.2 Kaizen and Quannun b p s (IS04,8.5.4 and Anncx B) - Describe the syncm applied to c o n ~ u c d y impme the feuurcs in inputs, processcs, outputs, objectives, NSOWCS and rrnilu.

Describe how the effecWenus of the continuai impmamnt +an is i n d

To better integrate the requirements of the ISO 9001 standard and the non-prexiipnve

criteria in the qualiry a w d , each category of the OPIM was split into rwo aspects, the

'Minimum Requirements' and 'Opportunities for Improvement'.

The OPIM should be considered as a combination of quaiity assurance srandards that provide the minimum requinmenu for quality improvement and business excellence modek, iliustnting the oppomuiities for improvernent.

(Karapetrovic and Macey, 2001)

Table 3-5 outlines the fuial OPIM and the comesponding cnteria of the MBNQA, EQA,

Deming Pnze, and ISO 9001, ISO 9004.

Please note that there is no entry in element 5.2.2 from the MBNQA or EQA. The reason

for this is that these a w d do not specifically address self-assessrnent in their uiteria but

provide further guidance in sepante documents on this process. Please refer to the

Approach to Evduation t h supplement the EQA and Guidelines for self-assessment

that accompanies the MBNQA for comsponding elemems.

Once the framework and ait& for the OPIM were cornplme, a document cded 'OPIM Fundamentas" was created to describe the undertying frarnewods of the OPIM, its content,

terminoiogy and application (Karapev~Mc and Macey, 2001).

This document is meant to accompany the OHM Criteria and contains general information

about the OPIM induding its relationship with the CSO's performance management system,

the framework of the OPIM, the content of its criteria, a description of terminology used,

and notes pertaining to its application.

3.2.3 Development of the Assessment Questions

Once the model wu complete, it was ready to be wd as a reference for conducting self-

assessments. To couen appropriate information for a self-assessment, the criteria were

translated into a set of questions to be used in assessment interviews. The questions were

based on the cnteria of the OPIM and were specific enougb to capture suffiuent

information with which to evaluate each element in the OPIM.

The questions were designeci to be open ended to obtain as much information as possible.

Appendk A contains the set of questions for the assessment. Please note that category four

is the r ed t s section that is used to evaiuate the results obtained in all other sections of the

model. For this reason, there are no questions for this category in the questionnaire but

evidence of the results in each pertinent category are collecteci from interviewees when

questions pertaining to these categories are asked. These questions ask the inte~ewee to

provide perception measures or performance indicaton that indicate performance relative to

the respective category critena.

3.2.4 Establishment of the Scoring Scheme

This model was developed as a set of criteria against &ch a self-assessrnent could be

conducted in a praaicai sertlig. Although, not the main puipose of self-assessment, it is

ofren beneficial to calculate a score from the assessment that can be compared from one

assessrnent to the next IO monitor i r n p r o v ~ ~ ~ l t ia a quantitative muiner. As seen in

Chapter 2, each of the refemce models has its own scoring scheme and evduation methods.

An appropriate scoring xheme w u required for the OPM. The rwo a w d considered as

models for the scoring xheme for the OPIM were the MBNQA and the EQA. These have

the most complete and comprehensive methods of evaluation. The evduation scales are

detailed, enabling the wesson to be more p r k in their evaluation The evduation of

each element is based on the three or four criteria outlined (approach, deployment, r e d s ,

assessrnent and review).

To create a scoring scheme for the OPIM, it was fim necessvy to assign point values to

each of the categories of the OPIM. The OPIM was developed to help an organisation

improve its business performance. It wu therefore important to incorporate the perceiveci

contribution of each categoq to the organisation's perfomiuice into the final disuibution of

points. A multi-criteria decision making methodology, was used to determine this

distribution of point values. This approach enables each organisation uàng the OPIM to

incorporate the impacts of each category to their specifc business.

3.2.4.1 Preference Function Modeliing

Mdti-cnteria decision making involves the selection of one alternative from a given set of

altemarives based on a set of criteria. Preference Function Modelling is a decision

analysis method010gy for mulUsriteria decision makiog. It is used to create a mathematicai

mode1 of the decision-maker's preferences for alternatives based on a set of critenk more

detaii see Banilai (1998, 1997).

A typical multi-cnteria decision involves the cornparison of several alternatives based on a

set of criteria. Take for example the purchse of a car. A decision-makerwould have a set

of cars to choose from. To make this choice, several elements rnight be considerd colour,

pnce, fuel economy, wa~anty, etc. These are the cri& for the decision. The outcome of

the decision would the selection of one car out of the set of cars.

The Trillium software (Badai, 2001) MS developed to encapsdate preference function

modeliing rhmiy into a foxmat for practical use. To use Trilliu111, rwo inpurs are requked.

These are a set of alternative outcornes for a decision, and a set of aiteria upon which the

deusion wiu be made. Once these are fed imo the model, decision-makers rank the

alternatives on each uitenon by compaxing tnplets. The result of diis process is a

positioning of these aitematives on an interval sale (a preference function). N a , the usen

indicate uade-offs between the criteria This allows waghu to be detennined for each

critenon. The set of weights is then used to resolve e d preference function (or rankings

on each interval scale) into one overail preference function.

R e f e h g again to the car example, once the altemacives are listed in the software, the

decision-maker is then asked to describe the set of criteria that wili form the basis for the

decision. Next each of the altemative are compared on one criterion. For instance, if we

have a blue Ford, a red Chiysler, and a green Toyota, the decision-maker is asked for his or

her preference for a blue car when compared to the altematives for green and r d This is

done for each criterion. Once this is done, the decision-maka provides mde-offs between

the criteria. For example, how many units of price would one trade for how many units of

fuel economy?

This methodology is used to select one out of a set of alternatives. However, in this

application, the software was used for a slighdy different purpose. The hope was to develop

a function of values to be applied to the OPIM categories. The software was used to rank

the caregones of the mode1 based on their relative contributions to business performance.

These ranks would then be trandated into a set of vaiues for each of the categories. Thur instead of selecting one car, we are creating a set of weights that would dow us to evaluate

any given car on a single, uni-dimensional scde that weights the car's characteristics

The foilowing describes the development of this function of values for the OPIM.

TO detennine the dimibution of scores among the categones of the OPIM, the Trillium

software was iwd In this o v e d function, the htghest ranked element f~anslates into the

category that is worch the mon points.

The relative contribution of each category to business performance w u captured using the

opinions of top management and those who had experience with BEM and self-assessrnent

process. A review of literature rwealed several instances where authors that have vied to

determine the relationship betweai quality management practices and business performance.

In 1999, Teniovski md Samson used several performance ourcomes as representations of

business perfomiance. These agreed with the connnicrs used by other researchers. The

performance outcomes were produaivity, competitive advantage, emplayee monle, coa of

quaiity, customer relationships, and product &ty. These outcomes were adopted to help

defme business perfomance in the decision modelling process.

To dwelop the peference function for OPIM, the foUowiq process was followed. The

alternatives are the categones in the OPIM and the criteria are the six components of

business identifid Sweral people, independent of one another, ranked the set

of alternatives in sets of five on each decision cnteria Figure 3-2 shows the scale used in the

Trillium software with severai elements from the OPIM positioned on it. After these

rankings were complete it was necessary to determine the uade-offs between the cntena to

indicare to the software which cnteria was the moa important to the organisation at the

present Ume. The Chief Executive Officer, Chief Operathg Officer, Vice President of

Finance, and the Vice President and General Manager of Division A were asked to indicate

the importance of each of these to the CSO in the Trillium software.

Appendix C contains the raw data taken from the Trilliurn software.

The end renilr of this anal* was a rmkiq of all the OPIM categories on an intemai scale

from 50 to 150.

It was decided that 1000 points would also be the t o d point value to facilitate cornparison

with the other award assessments.

The following process was used to change the ranks of the alternatives on the preference

function into scores out of 1000 for the OPIM.

1. The preferences on the scale were summed.

2. The sum was divided by 1000 to give a f- ~rctor.

3. Each individual preference value was divided by this factor to uuisform them into

scores. The sum of al1 the resulting values wu 1000. Table 3-6 shows the results of this

process.

4. The resulting scores were rouded to give the result in Table 3-7.

I Pmfamcw 1 Scotc out of 1000 1 People

* m t customcr Rdalionships

Performance Managamt

Performmce h p m e m c ~ l t

Strvegy ikvelopment and Deploymtnt

Performance Monitoring and Cent*

1%

148.77

144.44

48.49

48.09

46.69

132.05

131.94

13 1.75

131.12

l

42.69

42.65

42.59

42.3 8 L Peopk Resulu

De* Proccsxs

lnfonnuion Resources

h* Resourccs

F~cialResources

St;ikehol&r Objectives

Raiisation Ptoccrxs

Cwomer Wts

Supplier and P m a M u

Shareholdcr Results

System R d t s ~ ? n i r e ~

Support Processes

l'urchasing Processes

Society Rmilts

Total

Faaor

41.90

41.61

40.49

39.6 1

38.79

38.46

38.46

35.65

33.54

32.16

30.69

28.89

28.06

26.77

16.16

ZOO0

129.62

128.73

12526

122.54

120

118.99

118.98

110.3

103.n

99.48

94.95

89.38

86.8

82.8

50

3.09356

3093.56 r

4.4 SU&&~S and Pumas 33.50 4.5 SOUm 16.00 -

/

1 4.6 Svstem I 1

I 1 30.50 1

The process of g a t h e ~ g information or preferences from the decision-makeis in the CS0

cz

' 5.2 Monitoring and G n u o l h g I

was difficult. niey quenioned the impact of their decisions and were initially concemed that

they did not f d y understand the Triilium software. Decision-makers understood the

5.3 Improvement

concept of favouMg one cntenon over another but were a little unsure of the merhod used

42.00 43.00

in the software to capture these uade-offs. Once this was clarified, thqr were more

comfortab1e wRh entering their preferences into the mode1 for an+. The most dZficult

T-.-I I r -

element to comprehend wu the idea of uade-offs.

nnn

the MBNQA criteria in the OPIM Processes utegory are p e r than the poinrs in the

EQA The uiteria from each of these a d that fd into the Results catego'y of the OPIM make up between 43 and 45% of the respective award points. The aiteria from each of these

awads that fu l inro the Performance category of the OPIM make up between 7.5 and 8.5%

of the respective awad points.

Objecllves

T o d Resources

T o d Rwults

12.9%

32.8%

10% 8% 7.8%

20.8% 9%

7.2%)

18% 20% 9% 6%

O r g a n h o n a l T p Public Responsibility and Citizenship Sv?te&~ Development Smtegy Deployment Custorner and Market Knowledge

Work Systerns Employee Education, Tninllig and Development Employee Well Being and Satisfaction

borner Focused Results Ficiai and Mvket Results Humas Resource R d u Supplier and Partner Resuits

Tod Perfol.;mance

~ t i o n a i Effenivmess R e d a 1 11.5% 1 Results ( l job) x L 12 1 7.541, 1

T o d Processes

8.5% 4% 4% 4% 4% 24.5% 3.5%

2.5%

2.5% 8.5% 11.5% 1 1.5% 8% 2.5%

Total

1

Leadership Policy and Smtegy Processes (14%) s 1/5

People Pmerships and Resources (9%) s - I / S

Customer Results People Results SocietyResula Kcy Pcrfommcc

1 -

12.8%

19Yo 1

22.5%

Kcy pcrtormancc Resulrs ( 15%) >)s 1 / 2

Analysis of Organisationîl Performance

Customer Satisfaction and

42.5%

7.5% Measurement of Organisationai Performance

Relationships Cusromer md Market Knowledge P d u a and Service Processes Supplier and Pamer Processes Suppon Processes

45%

4%

4.5% 8.5%

4.5% 4% 5.5% 1.5% 1.5% 17%

Pracesses ( 1 -4%) s 4 / . V 7.5%

1 3 , Pannerships and Rttsuurces (9%) s 1 / 5 1.8%,

13%

Corn parison of Category Point Values

i O OPIM 1 H MBNQA 1 O EQA

l Objectives Resourceç Processes Results Performance 1 OPIM Categories !

4 1 1

There are siight differences between the models in ternis of the preference given to each

category. The EQA and the MBNQA are both very focuzed on the results categones. A

large percentage of the total points have been assigneà to this category in these cwo models.

The s c o ~ g scheme developed for the OPIM has points that are more evenly disvibuted

among the categories. n i e point value of the resources categoiy in the OPIM is aiso ver-

different from those of the other models.

The Resources category was deemed to be of greatest importance to the organisation's

perfoxmance. This is very 1ogica.I. It is often forgotten that no systern exists widiour

resources. A V e m is not jus a set of processes thar interact to produce an output.

Resources are ememely important as input into these processes. The q d t y of these

resources needs to be high to obtain quality outputs. One cannot cast inarmpetent

employees, unreliable equipment, or poor materials into a sysrern and expea satisfacto'y

results. It is interesting to note that resources are worth the lean in the scoring scherne of

the MBNQA.

This redt is a h v e y important to companies moving beyond ISO 9000 towub business

excellence. The main focus of IsO 9001 is on pmesses. TO move beyond this standard,

companies musr begin to concentrate on &ek resources as a next sep.

The distribution of these points in the OPIM is based on the opinions of people in the CSO.

This distribution of points my have a great effect on the final score in a seif-assessment.

Chapter 4 describes the performance of a self-assessrnent against the OPIM. A sensitiviry

analysis is done to answer this question.

3.3 Chapter S u m n w y

This chapter described the development of an integrated model for performance

management based on ISO 9000 and business excelience models. This involved the creation

of the model framework and corresponding criteria. A scoring scheme was also set up

capnuing the specific relationships h e e n the elernenu of the modeL The next chapter

describes how this proposed model was used to conduct a self-assessrnent 4 t h a division

at the CSO.

4.0 Self Assessrnent

4.1 Introduction

The development of the OPIM was describeci in Chapter k. This chapter illustrates

how a self-assessrnent against this proposed mode1 was used as a meanire of organisational

performance.

4.2 Self-Assessrnent at the Case Study Organisation

The CSO's quality system has been registered to the ISO 9001 standard since 1995. Since

thar tirne, quality audits have been conducted to ensure the continwus cornpliance of this

system. The organisation had not however, conducted a self-assessment against a mode1 for

quality management or business excellence. Sweral seps were involved in conductiag the

fwst self-assessment a g h the OPIM (See Figure 4- 1). This induded planning, garhering

idonnation, evduating the r e d s and the assessment process, and incorporating the r d t s

into the business planning process. This secrion describes the seps involved in the pilot

assessment of one of the (30's business divisions.

4.2.1 Planning

The planning phase of the self-assessrnent induded definmg the scope, self-messment

method, resources, and tim$ines. The scope refen to the areas within the organisation rhat

wili be assessed This can ange from one depanment to the whole organisation. The self-

assessrnent method concerns how the infonnation will be gathered to complete the

assessment. (Chapten 2 and 3 ourline several nich methods). It is important to defme the

resources required to conduct a self-assessrnent before it begtas. These consist of

personhoun for coUecring and analysmg the assessment info~nation. The tirneline for the

assessrnent must also be detennined in advance. This incldes times for information

coileaion, a d y s ~ ~ , and presentation of the resuits to senior management.

4.2.1.1 Scope

It was decided that the self-assessrnent wouid be c o n d u d in one division (Division A) of

the cornpany's business. This decision was based on several factors. Fim of all, this self-

assessment effort was the fm in the organisation's hisrory. Seleclhg a d e r ~ b s e t of the

organisation wu deemed to be a more realistic underilking than a t t e m p ~ g to assess all

areas of the business. Seconcüy, the rime and resources available limited the scope of the

assessment. Fin*, this particular division was selected because it was the fastes growing

business area of the Company.

4.2.1.2 Self-assessrnent Method

It was necessary to select a self-assessrnent method that would be appropriate for the

organisation as wd as meet the requirements and expectations of the CSO.

There are sevenl choices for appropriate self-assessment method, as seen in Chapter Two.

The seleaion of a self-assessrnent mode1 involves wsessing the maturity level of the

Company.

The CS0 had progressed well in the implementation of its quaiity system. This suggests that

it has srarted its joumey towaràs excellence, and w u 'on its wy". As a redt , a more

rigorous assessment approach was deerned to be appmpxiate. The main objective of this

assessment exerQse was to collect information from the organisation to identify the

suengrhs and areas for improvement in each area. The initiators of this process were also

interesteci in gaining as much information as possible from the assessment. Because of the

amount of infoxmation required and the quality manuin/ level of the organisation, a variation

on the pro-forma approach (see section 2-22) was selected.

The pro-forma approach involves the creation of a set of pages where each page

corresponds to one sub-critenon. The description of the sub-critaion is printed at the top

of the page with areas ro address beneath it. The information requested on the page

indudes suengths, areas for improvement and evidence. In te~ews are conducceci to collect

evidence for rhis process. Once these are finisbed, the assesson then complete the pro-

formas.

Ln this self-assessment, a set of questions relating to the sub-criterion was asked of the

inte~ewees ro uy to capture its essence. The assessor then identifieci the strengths and

areas for improvement based on the responses of the interviewees. Thus, the pro-forma

was completed by the assessor through the collection of information and evidence from the

interviewee.

Ir was necessary to select a subset of emplcyees with which the interviews would be

conduaed. The set of 159 questions was disuibuted among a number of interviewees,

seleaed by the managers of the division This redted in a set of 25 interviewees from the

following areas and with the following memben of the executive:

Marketing * Research and Dwelopment Test Operations & Strategic SourSng

- Materials Planning

Vice President of Finance Chief Executke Officer (CEO) Chief Operating Officer (COO) Vice President of Division A

The following organisational charcs indicate which representatives fiom eadi are? were

intervieweci.

A mauix contaïning the question set and the names of the inten?ewees w u created. This

was used to ensure that dl questions in the set were asked at least three times to three

different individuals. Questions were distributeci to employees through a nntified randorn

assignment (Mendenhall and Sinach, 1993). The employees were srratifid into

management and non-muugement to ennve that the questions would be asked to the

appropriate parties, appropriate for their function. Table 4-3 is a piece of rhis mauix.

Interviewees are listed at the top and the questions run down the length of the ma&.

'Yes" in a cell indicates that a question wrr asked of a corresponding inte~ewee. The 1st

row in the marriv monitors the number of questions asked each interviewee in total to

ennue an equal distribution The second last colwnn indicates the number of times a

particular question had been asked of an inrehewee to ensure at least rhree. The lasr

column indicates if ail the intewiewees asked the question were able to provide a usable

answer to the question. If this number wu not at leas three, the question was posed to

more inteMewees to get the three answers required.

,

Question How do leaders communiate the importance of meeting cunomer as weii as reguiatory and legal requirements? Whar is the process for iden* and targeting cusromers, customer groups and market segments? How does the CSO address and improve workplace heaith, safety and ergonornic factors? How is design validation incorporated into the design and deveiopment processes? How do you detennine key suppon process requirements? What is the systern applied to measure your key performance indicators relating to organisationai effectiveness (meeting key objectives)? How do you &à& what key compaxtive data and information to use? How do you continuaiiy improve the effectiveners, effiàency rad consistency of the performance mevurement systern? Number of questions for each interviewee

Ys

Yes

Ys

Yes

Yes

Yes

1 2 1 9 5

Yes

Yes

Yes

Yes

3 3 4 3 3

Yes

Yes

Yes

Asked 3

3

3

3

3

3

3

Yes

Yes

Yes

Aaswer 6

Yes

Yes

Yes

3

4.2.1.3 Timelines

A Mieliae was developed to indude al stages of the assessment process. These were

developed based on previous experience in other organisations. Table 4-2 ilhinrates the

suggested and a d rimes for the assessment.

St;ise m Esum;ited Ti i

Discussion with wessment ini&ors re: scope, wessment team, assesscnent process, tirnelines, evaiuilfion methods

Presentation to managers in the division

1 p* 1 ~eve lo~mmr of the &miew manix I 1 1 1

IdV

l &

1%

ld?y

1M 1 l d y 1 1 M d up session with (QA manager and 000)

1 Compare interviews and d u

A d Time

6.00

Conducc and analyse interviews

1 1 Week 1 I weekl

3

10

l d y

Persons Involved

12.00

20.00

2-3W&

Person Hours

2d;iys

1

I Total Person Hours 1 377.25 1

5 Wrrkr

Report

- --

4.2.2 Assessrnent

4

Idenyfy OFh and r a s (3 OFI'r and 3 +'s Der careeonr)

After the ~luining was complete, the next phase of the assessment involveci conducring

interviews, reviewing and e v a l b g responses, reviewing resuits, and crearion of the final

report.

8.00

25

Write the final reporc

Presentation to Management

75.00

week

2-3 W&

1 da^

wIFk 2

100.00

30.00

-

3 we$rr

1 h

80.W

1

15

InteMews were scheduled through intemal e - m S Each interviewe was sent a List of

questions to feview before his or her scheduled interview Mie. The interviews were

conducteci by the aurhor and were recorded dire+ ont0 a laprop computer. Aher each

meeting, the notes taken were rwiewed and if any dacificarion was necessary, a phone c d or

e-mail was made to the interviewee.

Objective evidence wu also coiiected from the interviewees. This evidence included the

following:

customer satisfaction nwqr re&

employee satisfaction nwqr results

monthly performance data from the result-based performance memement system

This evidence was used to evaluate the Redts section of the OPM The data was reviewed

ro see if trends from the pas few yean have shown continuous impmvement.

4.2.2.2 Review of Responses

At the end of the interview process, the ansvers from each interview were reviewed. It was

here that the i n i d analysis of the responses took place. The three answers to each question

were midied to determine if thqr described the same approach. If al1 three answers were

sirnilar, this would indicate that there was good deployment of the approach across and

down through the division. If an inte~iewee in a management position described a strong

approach but other respondents did not concur, diis indicated that the deployment of the

approach was weak. If, after getting these dvee answers, they wue ail different, the question

was posed to yet another h t e ~ e w e e . If al1 four of these responses were different, this

niggested that there was no consistent approach to the element in question throughout the

division.

After ensuring that each question had been arked a SuffiCient number of thes, the author

and a member of the quaity assurance team ma to discuss and evaluate the amvers. The

three or four responses to each question were Nmmanred before the evduation Each

summahd question was evaluated first accordllig to the approach describeci. Appendix E,

adapted from the EQA, darifies the scores given for approach, deployment and results.

Each evaluator scored each question independen* as per these definitions If the scores feu

within 20% of each other then the average score was taken for that question. This range was

felt to be acceptabie to the assessrnent team since, as we can see in Appendix E, the range

for each score level is 20%. If however there w u a gap greater than 20%, then discussion

ensued This was a vduable part of the evaluarion process. A large gap occurred onlytwice.

The discussion that followed led to a re-scoring and a s d e r gap.

The following is provided as an examp1e of the evaluation process. It describes the

evduation of element 5.1, Performance Management.

These questions were asked in order to mess this aspect of the organisation's business.

What are the systems used to mevure organisationai performance?

How are these synems interrelated?

How are these relationships manageci and improved?

The responses to these questions provided enough information to evaluate the approach and

deployment cf the methods to satisfy the given element.

Approach:

There wu some evidence of a sound approach to internai auditing and remit-based

performance meanirement.

There was no self-assessrnent process in place at that t h e .

There w u some evidence the performance rneasurememt systern suppoxted policy

and strategy and that the interna1 auditing process wu well defined

Each department tracked its own perfonavrce measures but had its own merhod of daetmiaing which performance areas needed monitoring and how this information w u

colected.

Ther- was no evidence that the relationship berween the intemal auditing systern and the

performance measurement system is managed or improved at d

The average score given was 20%. Refeming to Appeadk E, we can see that the CSO had

oniy some evidence of systemauc approaches to satisijmg the criteria in the given category.

Deplqment:

The audit resulu were used u a performance measure in the qd tyusu ru i ce

department.

There was some evidence that the approach to internai auditing and remit-based

performance measurement was implemaued and deployed in a systematic way.

There was no evidence that any type of self-assessment process was implementedor

deplayed

There was no evidence of a weil-deployed and implemented method of remit-based

performance meanirement.

The average score given was 13% Again, the assessrnent reveaied that the CS0 had little to

no evidence of systematic deployment of approaches satisfyuig the criteria in the given

category.

4.2.2.4 Results

M e r the xoring w u complete for each sub-category, a lia of svengths and areas for

hprovement w u compiied. These were based on a bendimuking ana+ to derennine the

ben p&ces within arez Some of &e m g t h s and areas for improvemenr were

evident from the cornparison of the organisation's a p p r d e s to each part of the critek If

there were apparent gaps in approaches or the deployilent of these approaches through the

division, these were mentioned as part of this list as w d

A final presentation of the results of the assessrnent was made to the managers and directors

of the division The results were also communicated in a Gnal reporr that wu left wirh the

CSO. Each page of the body of the report conrained the sub-category title, the fmal score

for that section, the criteria in that category, the list of strengths and areas for irnprovement,

and some examples of what excellent companies do. Figure 4-5 is an excerpt from th is

repon.

5.1 PRfo1111;~1ce Management

mwuranmts are cxlaqcd and irnproved

Dcsaik the s)rrtan applied t o m t v u n t h c ~

p e r f 0 ~ O u t C O m c s &nt&d in section 4.63 of OPIM. Describe hoprthe

muuraeait systan forkcy

Each depumient is mponsiblc for u?dring iadiriclual iodicaton of effe&eness

Resuits-bwd performance mevures ur mricwed mon* in the FORM

Thcre is a system of ESD audits and i n d ISO qu?lny auditz

The R&D process masures cover proccss h, ùm, efficicncy, and planning quai~ty

RSrD mevurrs are avaiiable on-iine (the inmet)

Areas for lmprovement

There is no formal recognition of the in td t ionsh ips between the interd audit, self-assessrnent and =suit- based synems

There are sepante systuns in cadi department for u?cking r e d t s - b d mevures

It is undear which performance measures convibute the mosr to which keycompany goais

There is no evi&nce of a sysccmatic approach to impmving the measunment synem for kcy performance outcornes, kcy process, and mource performance indicaton

The= is a Lck of rncmms dating to the effective use of RU)UTCeS

5-13 Key Performance indiaton

Describe the qaan applied to masure thc krypafomlalce OutcOmCs

identifid in sections 4.6.1 and 4.6.2 of OPM Describe h m the mtuutemtni syffem for ky performance indicvon is contindy improvcd.

wbat Excellent Compuiics Do

Finand mevia covcr the past, prcxnt and fucurr pcrformuice

The o q p i d o n lmows the uuc cons of its processes and producrs/senrices

Cycie Umc for ail ky processes is me;isurcd

Key processes have ben identifid in each unit, funaion, and depvuncnt of the organisation, and pmcess masuns have b e n & f i for a c h b p -

Process mesures uc codatecf dire+ with product./servicc charameristics or performance factors thît uc of prime imponuice to cwomen

*ll)LC organisation bu dcveloptd and o v e d safety in& thu is u;idrtd u lus t once a month, and consists of several output masures Wre timc-lon accidents as wd as the numbu of pmrcnt?tive or behaviod musure

4.2.3 Fimd Score

Once earh categoxy was evduated, the scores were uuisfured h o the final scoring template

to caiculate a f;nal score for the self-assessment.

4-3: F-

OPIM Citcpory I 2000 Score O b j d v a 1 50

Pemat Score 1 Maximum Score 39% 1 W

1 1 I

Raccsses

Proces Managanent

Ginorna rùationships

Design ~ Q V S

~ g P r o c c s s c s

1

Pcrfonnîna Monitoring and Gnuolling Perforxuncc knprovanmt

Ova;ill OPIM Scac

60

10

12

13

7

309b

10%

5%

5%

Shucholder Rcsulrs

People Rcnilts

Supplier ancl Pvrncr Rcnrlu

Society RcsulKs

30

44

33

10

9

4

2

1

1 1

27% l 220

23%

25%

30%

3096

43

43

IO00

12

12

U

49

13

23

27%

28%

281 1 28%

A score of 284 on the first assessrnent indicates that the CSO and the division that

undenvent the d-assessmuir have embarked on the joumey towarb excellence. This

score is not to be taken as an absolute accurate 'proportionw of business excellence (Le. the

Company being about 28% excdlenr), but as an indicator of which areas within the

organisation are further dong than others (Le. some categories received a score of 50h

whereas others scored 30%, meaning the fint category is not as developed as the second

one). It is not the absolute score in each category that is the most important eitha but the

translation of scores into priorities for impmvement that is the most value a d d d

4.2.4 Outcornes

Once the fmai scores of these categones were caldateci, it was important to prioritise the

results. This would assist the in the process of selecting the area on which to focus

improvement efforts.

This prioritisation was based on the score achiwed in a speciiic catego'y and the value of the

category. Table 4-4 illustrates the prioritisation process. The categones are ordered by score

achieved resulting in a score index. The categones are then ordered by total point value

giving a value index. These index values are then multiplied resulting in the total index.

for

IGtcgory / Score 1 Score Index ( Vaiue of 1 Vaiue Index 1 T o d I n d

People Results Process Management Information Resources 1Techno10gy Resources &orner relationships

Achiwed 4 10 9 11 12

Perfomiuice Management Supplier and P m e r Rcsults Resoufce Management Perfommce Monitoring and Conuolimg RÊ;ilis?tion Prucesses Desien Processes

12 25 7 10 14 8

11 2 14 12 10 13

23 15 17 13 11

44 33 47 43 37 43

Gtegory 44 44 4 1 45 49

I

19 20 13 22 24

18 8

23 16 10 17

456 300 289 286 264

2 16 200 161 160 140 136

The self-assessrnent process providecl the CSO with severai important outcornes. Firn, the

- 0 ~

Performance Impmvement Infr;isuucnueResources Work Environment NaturalResources Shareholder Renilrs System Red t s Strategy Development & Deployment Support Processes People fiesources &orner Results Pudashg Processes Financiai Resources SocietyResults Management

self-assessrnent process itself was impomt. The pilot assessrnent will provide valuable

information to the organisation regardmg how best to conduct their self-assessrnent process

Score Achieved

12 8 14 4 9 4 17 8

25 16 7 18 1

27

in the future. A h , a review of the strengths and areas for improvement found in the

assessrnent will be beneficial as an input to the m u a i business planning process. The final

score on the assessrnent wili be a usefd benchmarkuip tool as weli as a good quantitative

Totai Index

135 133 126 120 96 88 56 54 52 45 40 36

indicator of the statu of the quaiity management system and its level of business excellence.

' Value Index

15 7

2 1 5 6 4 14 3

26 9 2 12

Scorekidcx Vaiue of

The observations and recommendations resulting from this wessment were the following

9 19 6 24 16 22 4 18 2 5 20 3

Objectives

1 1 26 25 1 25

Gccgory 43 30 45 29 30 28 42 24 53 35 23 40

26 1

Suppliers and partners are not involved in the strategic planning pmcess.

10 5 1

There is no syscematic process to follow-up on action items to determine if they have

been completed and to evaluate the gaps.

Processes

Lack of syste~llittic incorporation of leamhg and customer feedback into design

processes.

No systernatic process for incorporation of current and projected infernal Nsromer

requirements and priorities into the design of support processes.

No review of customer related processes to ensure they are d meeting custorner's

requkemenu.

Collecf and use data from day to day contact with customers to enhance customer

satisfaction levels.

Resources

Provide managers with a report of only those budget items that thqr have control over.

0 Develop meanires for detennining the effectiwness of employee development and

training.

Develop a formal process for providing timely feedback to suppliers and parmers on

their performance.

Develop a synem for waiuariag the uade-off berween maintaining 'old" techdogy

and investing in new technology.

Results

Review the reporting of perfomance measures ro ensure that vends for the pan three

years are evident and that rargets are explicitly xared.

Performance

Irnplement a review of performance measures used to track rhe performance in each

deparunent to enwe thqr are reflecting the key drives of your business.

Focus on closing the loop. Irnplement processes to e n m that outputs from co~~ective

and preventive actions become inputs to design and process control activities.

4.2.5 Evaluation of Assessrnent Rocess

As this self-assessrnent was the k t for the CSO, the organisation and assessors reviewed

the process to evaiuate its effectiveness and to suggest impmvements. This should become

a reg& pan of the assessment proceu. Severai questions should be asked within this

process.

Was this process perceived to be successful by the employees, managers,

assessors, and otha interested parties?

This process wu perceived to be successfd by the managers, and directon. The M presentation was proceeded by many questions and exceiieat feedback This indicated that the

suggestions arising from the self-assessrnent would be reviewed and used as pur of the business

planning process. There was a lack of inte- on the part of some of the interviewees. This

wouid suggest that more information and support coming from management to encourage active

participation in the process by employees should orin before the n m self-assessrnent is

conduaed.

Was the tirneframe appropriate for pmper feedback into the business planning

process and 0 t h processes that would require input fiom this process?

This self-assessrnent did not oc= at the proper t h e for the results to be fed into the business

p l h g process. These self-assessments should be perfomied doser to the beginning of the

new planning q d e for the year.

Did the question set provide sufficient information to enable good collection of

information?

The questionnaire developed for use in this pilot assessment enabied good coiieccion of data and

information. The questions were clev to the interviewees as evidenced from the responses.

Did the selection of inte~ewees result in a good representation of a cross-section

of the organisation or unit king assessed?

This proces nee& to be reviewed and imPmved in the next assessment. The inte~ewees were

selected by department managers and directom. A better cross-section of employecs including,

for instance, more people involved in the re;ijisatiion processes, could be selected.

Was the scope of the assessment appropriate?

For this pilot wessment, the xope was entirely appropriate. For the n w assessrnent however,

rhis should be thought out before it is conmicred and reviewed again once it is complete.

Was the number of assessors appropriate and sufficient to conduct a good

assessment?

It is benefiaal ro have a number of assessors from al1 areas of the business. This would result in

a more efficient and effective coiiection of information, since thete would be a better

understanding of the s p d c proceses Mthin the Qputmenfs. This would eluninate Ume spent

on understanding the organisational processes.

Was the management cornmitment to the process sufficient?

Management commiunent w u fady weak No xnanagen above the lwel of k o r were

present at the finai presentation of the assessment redts. Self-asessrnent redts also showed a

la& of management commiunent.

This self-assessment process involved many people wirhli the CSO. The tirne involved was

lengthy. As this was the first assessment of ths kind performed within this organisation, it is

useful to evduate its value to the Company. One wayto do rhis is to compare the process

and red t s to those of a q d t y audit performed in the same rime LamP. This anparison

indudes:

the resources required to perforrn the self-assessrnent and the quality audir

the processes themselves

the type of r e d s obtained from each process

the value of idonnation obrained from each to the o v e d performance of the business

The ne= chapter deais Mth such a comprison proces. It also lodrs at remit-based

perfommnce measufement in &on to self-assessrnent and quality a u d i ~ g .

5.0 Performance Measurement 5.1 Introduction

The CSO uses several methods for measuring organisational perfonnance. The self-

assessrnent desuibed in the previous chapter is now compareci to wo other types of

perfonnance measurement used by the organisation: a quality audit conducted in the same

time frame, and the red-based performance mevureman used to continuously monitor

the processes 4 t h the organisation.

5.2 Cornparison of Assessrnent Results to Quality Audit Results

At the same time as the self-assessrnent was conducted, an internd audit of the C W s

quality system was performd In the year 2000, the self-assessrnent w u conducted h e e n

October 23d and December 15* (8 weeks) and the q d t y audit occurred during the week of

October ZP to November 2" (1 week). The internal audit team within the Quality

Assurance deparunent perfonned this audit with the assistance of an extemai consultant

acting as lead auditor.

Both the audit and self-assessrnent resulted in a set of recornmendauons for improvement.

The audit's recommendations resulted from the non-confonnities and opportunities for

improvement identified by the lead auditor. The recommendations from the self-assessrnent

arose from the areas for improvement found throughout the process. Table 5- 1 shows the

main areas requiring attention that were identified in each of these processes as a remit of

the prioririsation process oudines in Chapter Four.

Self-assessrnent (Oct 23-DeclS. MOO) I I n t d Quaüty Audit (Od 27 - Nov 2,2000) Highest ladar People Resuits Process Miusagement Information Rtsourca Tcchnology Rsourccs ûrnomcr relatioiuhips Stak&ol&r Objecrives

Eiemenrs Requinag Major Attcnuon 4.4 Design Conml 4.6 Rirch;rsing 4.9 Process Gntml 4.1 1 Inspection, Masuring And Test Equipment 4.14 CoRMiVe And Prcvcntawe Action 4.17 internai Qu?liry Audits

At first glance, the elements identified in the self-assessrnent seem to be ver- different from

those identified in the qualiry audit This is not surprising. The elements identifiecl in the

self-assessment are mîinly categones that are not covaed by the ISO 9001-94 standard The

CSO's quality system is not concenuated in these areas and as a result, these may need

M e r development .

The Process Management category h m the OPIM however was found to be the weakest

element. This corresponds well to the audit findings since Process Management

incorporates elements 4.4,4.6,4.9, and 4.1 1 of the ISO 9ûû 1 standard.

The category identifid in the self-assessment that does not seem to have been identifieci in

the quality audit is Customer Reiationships. Elemmts 4.14 and 4.17 were identiûed in the

audit but no related elements stood out in the self-assessment. Customer Relationshps in

the OPIM indudes element 4.3 of the ISO 9001 standard and a srnail part of element 4.4

deaiing with identification of cusromer requirements. Element 4.3 was not identifieci as

requiring major attention but f d e r an+ indiates an agreement h e e n the weaknesses

within Customer Relationships and those found within element 4.4 Design Conml.

Table 5-2 shows the main recormnendations, and non-cornplance issues and ares for

improvement idendied in the self-assessrnent and audit respecciveiy. The table lins the

main caregories in the OPIM with corresponding recommendations. T h e non-compliuice

issues and opportunities for improvement redting from the audit have been listed

accorciingly. The issues underlined within the table have been identified as si& issues

arising from the audit and self-assessment. As was illustrated in Table 5-1, many of the

recommendations from the self-assessrnent do not correspond to any in the audit as the

audit covers onty those items in the ISO 9001: 1994 standard. Those that were identified as

major issues in the audit may not have been major issues in the self-assessment. This is

because other issues arising in the self-assessrnent were ranked at a higher impomnce level.

Objectives

Focus G devcloping a process ro follow-up on action items to dccumine if thcy hne been completcd md to ev;ihiue the gaps Review the timing of rhe digment m e y to ensure t h R is opirml to pmvide input into the ;inrnul

Provide mui;igen with a report of ody thosc budget items that they hne control over Devclop mevurs for detuminùig the effecllveness of employee devtlopmc~it and uaining Wew ROI pioccss to msurc consistent use ;uross

the division Cdlm and mriew dadinfomrztion coaccrning the effiuency and effmiveness of informuion systems and availability of data Deveiop a synem for e v a i w the uaâc-off betwecn mîintaining 'oldw technology and investing in new technology

Processes

Develop a process to incorporace cumnt and projected i n t e d cusromer requkmcnts id priorities into the design of nippon proces~s Implement a rcview of amorner rctted processes to ensure th y ue aiil mceting customa's rcquircments Coüm and use data from day to day contact with custorners to enhance customer satisfaction levek

for . . . to -s on k

IKtfofminrC

Revicw the rcponing of performance rnclsurrs to ensure ttut uuids for the put thne ycvs am cvidcnt

M Qurlity Audit (Oct 27 - NOV 2,2000)

Qriltrysysta Use ISO 90012m direction for dcploying stratcgic ptnn;np prinC;Plcs to &me bmu iinkage berweai svPegic objectives & the systcm dtslgned to achieve tbmi

N o compiiancc issuCs

No compiiancc issucs

N o compluace issues - No c o m p ~ c e isnics - N o compLUnce issues

Design Control

Consider using quahy funaion depiuyment to idaiufy the uue cunoma speufications & not what tbey think thy wuit

Consider ;idopUng suicca DFMEA & PFMEA during the e V t y p k s Use t- feasibiity Jgn-off a p p n d i

Move t d tnic supplier qu;ilif~cation in pkcc of product quaMcations Consider a formal sys~cm for achicving the above

lmplemcnt a rcview of perfomunce mesures used to crack the performance in #ch dcpvunent to ennuc thcy arc rrflccting the key &ers of your business

i n t a d Qu?lity Audit (Oa 27 - NW 2,2000)

Conrtrive and Prevenuve Action

k t cause ui;3yUs is inadapte Corrrcllvc actions not responded to in a timeiy muuicr, and somc n a rcspondcd to at dl Mcthod for esublishing effecrivcness of solution is

Intenul Audits

Prioritise audit ;ictivillcs b w d on due-added and effective w of reu,urces Numaous comcWe isnies ourstuiding. Consider an &on Clause No objective evidence that mut?gcmuit personnel rwiewcd audit fndings Indepuidence of auditors not evident in di cases

Statist id Techniques

Several issues can be ewacred from this comparison. A mjor problem identifid in the

q d t y audit (element 4.17) was that there was no objective evidence that management

personnel reviewed the audit fmdings. This was also emphasised by the fact that there were

numerous corrective issues found to be outstanding. This is one of the main differences

berween the self-assessrnent and auditing processes. It is evident that the outcornes or

recommendatïons from the quality audits are not reviewed and acted upon to an acceptable

level. The results are not fed back into the business planning process. The self-assessrnent

fmdings on the other hand were presented to senior management and were considered

usefil to the planning process.

As discussed previously, the renilrs of an audit are simple pass or fd judgements. The

assessrnent performed against the OPIM was given a score based on the approach and

deployment of the approaches that the organisation has in place to mea the given criteria of

the model. It is difficult, therefore, to compare the f i score or outcome of these two

processes. If there were a methoci for cornpuison, the findings would show that the score

on an assessment againa rhis model Qcreases with fhe number of non-conformances and

elements requiring major attention in a qualicy audit performed in the same rime frune.

The OPIM has attempted to incorporate the basic reqykments for the organisation's q d r y

system. If there were any non-conformances arising from the audit, it should follow that the

corresponding element in the OPIM would be judged as inadequate. If the minimum

reguirements have been met, the score for the eiemem in the self-assessment should be

hrgher. It does not indicate however, that ail the critexia in the corresponding element of the

seIf-assessrnent have been met. Figure 5-1 iliumtes this concept. The ana of the triangle

represents the relative percentage of elemenu with:

no non-conformances with a low score on the assessment

no nontonfonnances with a high score on the assessment

non-conformances with a low score on the assessment

nonsonformances with a hi& score on the assessment

Self-Assessrnent 4 Low W b

I Wouid h?\e non- conformances whilc 0bt;ilring a hi& score on the self-asxssment.

Audit

The r e s h of the pilot self-assessrnent and the @ty audit with respect to the concept in

Figure 5-1 are illustrateci in Table 5-3. A l m score on the seif-assessrnent is a score in the

bottom @e of the xores on the self-assessment. A hi& score is one in the top quanile

of the scores.

This corresponds well with the hypothesis presented in Figure 5- 1. The highest percentage

of elemenrs is in the quarter for low scores and non-conformances. The lowest percentage

of elements is in the quarter for high scores and non-conforniances. The other rwo quarters

are almost equal.

Figure 5-2 compares the scores on the OPIM to the elemenu of the quality audit that weie

found to have non-conformities. On average, the self-assessrnent xores were lower when

corresponding ISO 9001 elements were found to have non-conforniinces. The results do

Vary. Keep in rnind that the elements of the OPIM in moa cases cover more than

corresponding elements of ISO 9001. It is possible to have a low score on the OPIM men

if there were no non-conformances found in the audit. There should not be however, mun,

elements in the OPIM that achieve a high score if rhere are non-conforniances found in the

corresponding element from the ISO 9001 standard

Nontonformance

6 1

QualityAudit SeIf-assessrnent Score Low

Conforrmnce

4 3

5.2.1 Implications for Sirnultaneous Auditing and Assessrnent

The self-assessment perfoxmed againa the OPIM provided much information regarding the

effectiveness and the efficiency of al of the organisation's processes. In particular, suengths

and areas for improvement were identifieci in processes related to quality management. A

comparison of the resuits of the self-assessrnent and the quality audit rwealed that similar

issues were idenufied in each process. The implications of this are far reaching.

It is evident from this exercise that performing self-assessrnent on the elements related

to q d t y management c m provide valuable information regardhg rhe effciency of

these processes. This c m cl+ the quality audit, which aims to judge the effectiveness

of such processes.

The men& and weaknesses identified in the self-assessrnent process are incorporated

into the business pknning pmcess. This ensures that the areas for improvement, related

to the quality management system, are iopns h o rhis planning process as w d . This

compensates for the fact that the results of quality audits are not incaporated into the

business planning process.

Self-assessrnent of the processes relating to +ty management, especdy if perforrned

a opposing times of the year from the interna1 quality audits, would lead to further

improvements to the efficiency and effectiveness of these processes.

5.3 Result-based Performance Measurernent

In addition to quality audiring and self-assessment, the organisation uses resulr-based

performance meanires to undemd the curent perfomiyice lwel of its processes. This

section fm describes a review of the measures cumntly tracked by the CSO and a

comparison of rhis set of measures with measures suggested by researchers and in the

national quality award models. Fin* a cornparison of these three performance

meanirement me&& is described in the final part of the chapter.

5.3.1 Review of Organisationai Performance Measures

Self-assessmenr against the MBNQA and the EQA involves the evduation of organkationd

results. These reniks are assessed based on the history and trends of m m e s that have

tracked perfomiance in each of several areas. Each a w d suggests areas of performance

that should be monitored by an organisation. These are found in the Results caregory of the

EQA and the Business R e d t s categoxy of the MBNQA (See Table 5-4).

MBNQA (Business Rendu) Gstomer Finuicial and Market Human Resource Supplier and P m e r Omnisauond E ffectiveness

EQA (Result4 Customer People SRY Performance

It is the a d results (trends, levels, ben* in each of these areas rhat are evaluated

in a seIf-assessment. The self-assessrnent does not appraise whether the organisation

amdlymeasures perfonnuice in each of the areas. If there is no measure for a speciGc

element of performance, the element cannot be wessed.

Gregory 5.2.3 of the OPIM, Performance Meanirement, uks the organisation to describe

its perfomiance measurement V e m (see Figure 5-3). It is at this point that the

organisation's resuit-based perfomiuice measurement system is wessed It is also here that

the organisation has the opportunity to i d e n e any gaps in ternis of areas it needs to be

meamring.

5.2.3 Performance Mcuuremait (IS04,8.1,82 1, 8.4)

hpmmus (MBNQA, 4.1a1 and 42)

Describe the performance measurement systcm uscd to collect, monitor. analyse, synthesise and act upon perceptions (extcrnal) and performance indicators (intemal) relating to: - ~0~

- people - supplias and partnen

- Society

qptm- WNQA, 4 . 1 4

Describe how the effectiveness. emciency and consistency of the performance measurement system are continually improved.

OPIM

5.3.2 Review of the Performance Measures at Case Study Organisation

The CSO has IWO main divisions. Each of these has its R&D, marketing, and manufacninng

components but are assisted by the same support departments. These are cornputer

services, quality conuol, quality asniance, hurnan resources, and finance. Division A

focuses mady on new produa development. This means the design of new integrated

circuits @Cs) to meet the needs of its customen. Most of the new products developed are

manufactureci out of house. Division B has both new product development and on-site

manufacturing. The divisions are quite different h m one another but both must conuibute

to the ovedl adiwement of profitable gr&

Most departments have dwdoped their perfoxmance measures from the objectives of the

depanment, which in tum are developed h m the CSO's overall objectives. To improve

the C D ' s performance measurement system, it is important to identify gaps where aspects

of performance are not being measured.

The CSO's performance measurement system is made up of several perfomiance meanires

uacked in each depanment of the organisation. The following measures are reportecl on

monddy (See Table 5-5).

Qualicy Assurance

NP1 - New Roduct Introductions

Sales

(to QA) in pans & - d o n (AIQ @pm) ) Producr Group 1 P d u s t Group 2 Prodm Group 3

(AOQ ( P P ~ ) ) CAOQ (cuxoma average outgoing quaiity) Qwlny Assurance Inspection cvdc umc

revenue Pmrntage effort for top ten . .

-

Explmation The pcrcmtage incmasc in des over the pmiious ycu (5 veu compound) (Prr-tuincomc-invesmicntinromc-tucs) /~ori l~-cuh + ST- Ilbiliries) (Incomc/ average equity) (incorne beforc tax + d a r i a and bcnefits)/(numba of crnployces) Percaitigc inarve in shvt pice over the pmrious yar (5 yar compound) This is the numba of defects o k c d on di batches submitted to QA for -pli%

This is the nurnber of defms observai in thc acceptable batcùa a the QA

This is che number of defective renirns observd in the total quuitiry s d c i

This is rime from buch urivai ro arrivai u stores

Clrnomcn d g produas want to knaw the 'rooc cause" of fdurc A d r m e d life testing - QuîLfying . . Coafirmcd dcfecûve from cunoma by QA This is the p e r c a i w of corponte revenue genented from pducts

i n v c u a i i n ' ~ ~ ~ This is the pcruntagr of anplqrcc t h e spenr on the cumnt top tai priority

1 invmrory (m produns) Product mU vs. MPS 1 The actuai mix of produas compvcd to the sch& mix of produas in

V

-&on

Testing (Extemal

and Intemal)

Producrion rate vs. MPS (%) 1 Amui pfodunion vasrrr the w produccion schcdulc Wdcr Fab. yield (%) 1 The d e r yield before elcarial unLig (physicd ody) BasiiiccyderLN 1 T i from the stzrr of firn operrtioa in production to end prochicr into

1 nwerproducrionschcdulc - Asscmbly Opcrations/Test /Extemal

. AIQ (ppm) Prototype cyde timc in- howc/offshorc (w-bys)

Asscmbly

uiventory The amai mù of produns çompued to the d d c d mix of pro& in 1

The actual mix of produar testcd cornpucd to the scheduicd mir of turing in ~produaionschcdulc

Master Production Schedulc (MPS) Rue (%) Yicld (%)

' Cyde (w*)

Accuai prafuaion vernir the miner produaion schcdule

Number of finished pro du^ v a w numba of produns stvtcd T i from the nin of firn operaion in production to end prodricr into

Several interviews were conmicted to gain information about the current measures being

used. These measures were reviewed with the manager of QA, the VP of Research and

Developmen~ the VP of Manufaauring, and the VP of Finance. Each one was asked a

number of questions pe~aining to these measures.

Quesuons Which performance indicarors are most important to you in determinhg the performance of your department?

measures thu you feel are not quite as useful as they could be?

meaSures are imponant with the possible exception of the Quai~ty Assurance Inspection Cyde Time, which is done for customer satisfaction reasons. These meilSufes are a good indication of the performance of the QA deriment.

measures are manufacnuing COR

on-rime delivery q d t y ( % defects) production of appropriate volumes

Other healthand safety

VP Finance

although these need to be ex plained thoroughiy to the attendees at the montMy performance meetings so that di measures used are u n d e r s r d

Quesuons Do you fed thît yourperformuice indiuton are in line with the go;ils/objectives of

How ue these

How does the performance of p u r deparunent relate to the o v e d perfommce of The organisation?

How often are these measures tracked and reported on? How is the information from these measured used?

How are targeu for these measures set? 1s bencharking done? If so how is t h done?

Yes

The performance of the QA depaKmenf determines the outgoing qu;ikty of products and the reliability of new desim. This has a g r 2 effm on The organisation's

Measures are reporteci on monthiy

Measures are used to fmd areas w here d t y performance is not up to targets The targets for aU these measures are 25% improvement over previous year. Target was 50°h but after improving the 'low huiging fruitn, it was more difficult to achiwe this 50% improvement. ~

Yes

There is a database for tmcking these measures (on-line NP1 &tabue)

Yes

Mon of these nwswes are tracked in the QIS. Finaud mesures are uacked in aflother inforniwon sysrem.

The performance The performance of the R&D department determines how fast the Company grows It is the primvy cost driver of the business and deterrnines the quality of produm.

of the ~llil~lufacfuring dep;i~ment affects cusfomer satisfaction and ~lll~lufacniiing COSU

Measures are coiiected and reponed on

Me;isures are collected weekly and reporteci on

mon* mon* This information is Measures are used used to focus improvement efforts

CO frnd problems in the production pmcess

Set realisticaiiy Targecs are set based on historiai baseci on

forecasteci dernad, some benchmarking such as on-time delivery. Benchmuking is done m d y from a process perspective not just to determine tuners. ~

Yes

M m e s are CalcUlated by accounting d e r k from information in another information syst=.

The finuicial measures are the m?in indiaors of The orgmktion's performance

coiiected monthly

Measures indicate how the organisation is performing o v e d

Targets are set based on an increase of 20 in saies and revenues each year

Based on this set of interviews, the foflowing lin of strengrhs and areas for improvement

was created.

Strengths

The measures used in the Quaiity Assurance, R8rD and Manufactwing are felt to be

adequate to capture perfoxmance in each depanment.

The meanires are felt to be in-line with the organisation's strategy.

The CS0 uses its performance measurement infommion in monddy meetings.

Areas for impravemcnt

Most measures are monitored monrhly. Some may need to be mcked more o h .

There is no systematic review of these measures to ennue that they are d l meeting the

needs of the organisation.

Littie bendimuking is done to set competitive targets for improvement.

Targets are set according to historiai performance in R&D.

5.3.2.1 Identifying Performance Measurement Gaps

Several perfoxmance measurement models were reviewed in diaprer 2. It is useful to

compare the set of measures used by the CSO to these models to idendy a . areas whem

the organisation does not meanire performance.

Table 5-10 places the CSO's perfoxmance measures wirhin the four penpecrives of the

Balanced Scorecard (BSC).

Fimacial Perspective Sales growth RRurn on Average Net Assets (ROANA) Rcturn on Equity (ROE) Vaiue addecl per pcnon per year EPS growth

Customa Perspective Customer Satisfaction Survey Fdure Assessrnent (FA) success nte Customer renvns technid CAOQ (Cunomer Average Outgohg Qdq) Prûdua reliabhy Ievels On-Ume shipments Backorden Càil response Ability to accept orders

I n d Business Perspective Design cycle time De* 4 w Devclopmait effort Milestones on time Manufaawingcyde Urnes Design-ins Yield Supplier on-time delivery Qu?lity of supplier products Billingacmmq

Innovation and Lcaming hvestment in NP1 New produa initùtiom New producc introductions Tnining hours per employee Training costs as a % of salary

Ir is evident that the C D ' s measures fit well inro the BSC fîamework The weakest category

seems to be Innovation and Learning. The CSO should find several more measures

penaining to its performance in this area Sorne suggestions follow in the next section.

Neeiy et ai (1999) niggest that an organisation's manufacturing perfomiance measures

should fd into four caregories. These are qyahty, time, con and flexibility. These can be

extended to cover the performance aspects of other deparunents as wd. The performance

of design and development processes, for instance, can be viewed from these four

perspectives. Table 5-8 shows how the CSO's measures can be assigned to Nedy's elements

of performance.

Mmuf?aiPing AIQ ( P P ~ ) AOQ ( P P ~ CAOQ (customer average outgohg quality) Product reliabiikty levels Customer rcnuns technical Production m e vs. MPS (%) P& Yield Prohuct mix vs. MPS Sdety FA m e R&D Mask d o % First tirne right Humui Rcsourccs ProductiY;ty/E mployee Training H o m per employee Hit me (%) Quîlityof hire

Muiufacnuing Lead T i e Rare of production

' Tcning and Strategic Sourcing On time shipments from Suppliers

Rétention rate Testing and Stntcgic Sourcing Quality of WO accepted

Muiufacturlllg Qu;ility Assurance Inspection Production Cycie times

i n d & o n Deliver lead time Due-Date performance Frequenq of delivery

Manufaauring cost Vdue added

O n T Ï e Deh iy R&D Design Cyde Uaie Development effort Milestones on time Average v k c e Human Rcsourccs Ovenime - # of employees > 100 hourdyear

S e h g pice Running con Service con Cost of Qu;iLty Design con Distribution con Cost dative to cornpetitors Materiah Inventory

Percentage effort for top ten priority projects Design-ins 1 Sales

Flembiliry

Other

This set is comprehensive. The con category however, is quice weak. This uea should be

reviewed to ensure that the organisation uuly understands its costs. O n e important aspect is

the identification of the c o s of quaiity. This is not addressed at al l by the CSO's

performance measurement system. This indudes looking at preventative, appraisal and

I I l Investment in NPI NP1 investment as % of forcas menue

MPaGlquîliry 0urPutQu;itty N-vproduct M o d i f y ~ h D&mbility Volume Mix Resourcemix

Those from the C S O ' s set of pafonxunce me?surrs thu do not fit in these ategories

fadure costs to show management the costs of poor quaiïty, to idenufy opporninities for

improvement, and to idenafy hiden costs.

Manufrcturing B;ickordas Rtw Ncw p& ;n;t;lt;ons (MG) New prohua imroducaom (MR) Humm Resourccs Ov& - # of cmployecs > 100 hours/year Tcsting and Stmîegic SourQng RespoWveness of supplies (1 low - 5 hi&) Sales growth R O M ROE Value addcd per person per year EPS growth % Revenue h m NPI projects iaunchcd within last 3 years

A third set of measurement categories is found in the Resuhs sections of the OPIM. The

OPIM incorpontes the meanires suggested by the MBNQA and the EQA. Table 5-9

compares these measures to those that are mcked bythe CSO.

People

i OPlM

Supplien and Parmen

Case Study Orguiisation

Society

1

Reput;ition

Ourpum

Interfaces

-orner Satisfaction Survey

Fdure fina+ Success Rate -0rnerRcnims CAOQ On-the shipments Backordcrs GIl Response Rare (Applications)

Financial

Non-Finuiciai

Interfaces Motivation Satisfaction

Achiwemeats

Services Lnputs

Interfaces CRizenship Cornmunity Environment Hdth and S a f q

Sdes Growth ROANA ROE value Added EPS Growth Design-hs New Prociuct Inm>micuons Tirne to rnarket (N'PI cyde Urnes)

Hours of Tnining Employee Alignment Survey RRenaon Absen& -of Hire Enuy level/expenence % Empluyee Productivity Accession (% hedcount)

AiQ for Offshore paru Offshore cyde tirnes On-tirne deliveries Quai~ty of WO accepted Responsiveness

Health and Safety Accidents Incidents

Interfaces Days lost

Design and Development

Management Finances

[nformauon [nfnsuucturr People

Nature Work EnWonment

IC cyde Ume Development Wty T o d Development Effort AIQ for Offshore P- Offshore cyde h e s On-Ume deliveries Qu;ility of WO accepted Responsiveness Production rate vs. MPS Produa mix vs. MPS Production cycle times Yield AIQ Quality-ce Ins~ection T i e

RONA ROE

Produ&+ per employee Design Effon - --

EPS Net Profit On-tirne shipments Schedule Accuracy FA Success Rate Mes Design-ins

Accordhg to the utegories of perfonnuice in the OPM, the CS0 is missing perfomiance

measures that reflect the effiaent and effective use of the organisation's r emces . Table 5-

10 provides suggesred meanires that shodd be incorporateci into the CSO's m t set of

masures to monitor the use of resomes. The suggested meanires have been taken from

several sources induding the EQA

Pcrformancc Ar#

Exterd Resources

1 1 Recoenition of m e r s ' conmbuàons 1 EOA 1

- Suggested Pcrfamancc Measuns to Add to Existing Sct

C 1 %

Source

Supplier psice Numkr and vaiue added of p d p s

Inventor- leveh and timing of deliveries Number and vliut added of innovative product solutions

Maintenuice corn EQA M t ratings EQA Defea rates of mu& EOA

EQA EQA

Shaw, 1999

Financial Resources

Resources

EQA

c. . Cîsb flowitems Balance sheet items Depreciation

-. \

EQA EQA EQA

inventory tumover Utility consumption Lhihation

1 EQA EQA EOA

Technology Resources ~&i i i~ of toolr Value of i n t d d property Patents - - -

EQA EQA Information Resources

kit egrity Relwance Timeliness Sharine and usïm knolk'ledee

In addition to the resource indicators, the CSO does not have indicators that show what the

organisation is adUeving in relation to s o c i q . Some of the measures that are s-ested in

I

Royalties Accessibility

EQA EQA EQA EOA

Y .

- - -

Ba*, 1994 EQA EOA

Vdue of intellecd capitd

1

EQA

literanue are k e d in Table 5-1 1. The CSO should inco'pome seved of these mevures

into its m e n t set of performance measures.

Pcrform;uice Arca 1 S u g g c d Pcrfœmance Masures to Add [ Source 1 Performance as a responsible a&

The CSO dso does not report on services for employees. Table 5-12 indudes a lia of

in volve men^ in the communities where it operates

Reporting on activities to wist in the presemation and SUSt3iZI;Zbility of resources

suggested performance measurrs covering rhis perfomiance area.

- - toExistingSet

Disdosures of information relevant to the EOA cornmunRy E q d oppominities practices Impact on Id and nationai economies Involvement in education and &&g

' Volumvy work a d phiianthropy Support for sport and leisure Choice of avupon Ecologid impm Reduction and eiimktion of waste and

In general these tables address the three areas where the CSO's perfomiance meanirement

system is weak Thus, performance measures should be designed or adopted to reflect

performance in the following areas:

innovation and learning

-x--

EQA EQA ,

EQA EQA EQA EQA EQA

Services for Employees

use oI resources

interaction with society

evaluation of empluyee services

Benefit lwels Provision of facilities and services

EQA EOA

The 0 should a h rmiew the suengdis and areas for improvement related to its remit-

based performance measurement system identifid in this chapter.

5.4 Self-rssessment, Auditing and Resuh-based Performance Measurement

It is evident that self-assessment, audiring and result-based performance meanirement are ali

benefiaal to an organisation's performance improvemem efforts. Each one requires a

different cype of input and each redts in outpins that become inputs into the performance

management system of the organisation. Table 5-13 recaps the characteristics of these

performance mesurement methods.

Inputs

' Effort

Outcornes

These methods each have its specific role to play but are obviousiy quite interdependent.

Self-assessrnent

Opinions Obiective evi&nce

Measure

This next section describes the relationship arnong these meanirement methods and how

1

they can be used together to improve organisational perfoxmance.

Audrnng Objective evi&nce

Possible hi& effon lwei Quîlttative Subjeaive (Based on opinion) Holistic Effiaency anci Effecûveness

5.4.1 Cornparhg Self-Assessrnent and Result-Based Performance Measuremcnt

Result- baseci pe dormance measunment

Numeric &ta

The OPIM, like the EQA and the MBNQA, contains cntena that address the Enablers and

Results of the performance management system. In self-assessment, the efficiency of the

Enablers is judged. Their effectiveness is mevurd by the Redts. Figure 5-4 illusmates this

Medium effon ievel Qualitative Objective Pfocess-based

Effectiveness

relationship.

Dependent on system Quantitative Ma+ Objective Specific

E fficiency and E ffectiveness

SELF-ASSESSMENT .

Efficiency of Effectiveness of

Ir is the R e d t s category, in fact, that represents the organisation's result-based performance

meanirement system. Performance measures are used to detennine the nirrent effecWeness

of organisational activities (Enablen). In other worb, to what exrent these activities are

contributhg to the overall performance (Results) of the organisation.

The CS0 has a well-developed result-based perfomance meanirement synem. The

meanires in this system provide quantitative and qualitative information concerning the

efficiency and effectiveness of activities. For instance the CSO uses Quahy Asnirance

Inspection Cyde Ume as a measure of the how efficiendy the QA department performs its

inspections. It also racks customer satisfaction, which is an indiwor of how effectively it is

meeting cunomer requirements.

O n e wodd ask then, why wouid an organisation need self-assessment. The anmer lies in

the limited abiiity of these meanires to evaiuate the approaches and deployment of

approaches used to perform organisational activities. It is hem that self-assessment becomes

important. This process imrolves a more in-depth a d p ~ of why the activity is at the

current performance lweL Self-wessment, for instance, can provide information about

how the QA department performs its inspection processes thus shedding light on why it's effiaency is where it is.

Resuit-based performance measurement provides a snapshot of performance levels. Self-

assessrnent evaiuates the causes of these performance levels. It is &dent that these nvo

performance meanurnent methods are complementary. Used together they can provide

valuable information for improvernent.

5.4.2 The Roles of Self-Assessment, Quality Audits, and Result Based Performance Measurement in the Organisational Performance Measurement System

This chapter has illustrated the importance of each type of performance measurement to an

organisation. Figure 5-5 illusvates how these fit into an organisation perfonnance

measurement system. The result-based perfonnance measurement system consins of

perforrmnce meanues. The OPIM exisrs as a reference mode1 for self-assessrnent and

encompasses the elemenu that are audited against ISO 9001. The results of self-assessments

against OPIM will provide an indication of the CSO's overall performance, and therefore

will be added to the set of existing performance memes.

PERFORMANCE MEASUREMENT w

Assessrnent Audit

i

5.5 Chapter Suimnary

Li this chaprer, the processes of self-wessment and @ty audit* were cornpared. niis

reveded the several implications for the simuitaneous use of audits and self-assessments

within an organisation. The chapter dso described the review of the CSO's perfonnance

meanvernent system and rhen compared the use of redt-based perfonnance meanirement

to the use of audiring and self-assessment.

6.0 Pdormance Management In RdrD 6.1 Introduction

This diaprer outlines the modification of the proposed model, the OPIM, for use in an

R&D environment. These modifications were based on issues raised in the literature review

and practical experience gained t h u g h the self-assessrnent conducteci at the CSO.

6.2 The R&D Environment at the CS0

The CSO, as an organisation, is very much focwd on its R&D efforts to compete in the

fast paced marketplace. There are several R&D reams woiking in each of irs three business

areas. After interviewhg several of the managers in each of these areas, it was dixoverd

that mon of the issues mentioned impact the efficiency and effectiveness of the R&D

function.

The R&D management is very much focused on reducing q d e Urnes and ensuring that

products reach the marketplace as dose to fun as possible. It is currently woriung on

increasing the awareness of the vaiue of time amongst R&D ream members. Through

several benchmarkhg mdies reporteci on in the CSO's strategic pianning meeting, it was

discovered that Ume delays con much more in the long rem than does exceeding the project

budget to get the work done on cime.

One author found, mggesteci that in a market with 20% annual growth and 12O/0 pnce drop

per year chat a delay of six months while on budget wdl earn 33% less profit over life but

introducing the same produa on rime with a 50% budget overrun cuts profits by only 4%.

(From the 0 ' s strategic planning meeting)

The CSO is very aware of the need to malitain its RgrD efforts. About 20% of net revenue

is invested back into R&D. The CS0 is constantly exploring other markets and expanding

its opporruniaes.

Division A is involved in the design of integrated Circuits for th& application to a particular

segment of indu-. Most of these products, once designed, are manufacnued off-site.

7'he mlin activities of the division are therefore R&D relatecl.

6.3 Modifying the OPIM for Use in an R&D Environment

The OMM is based on standards and models that have not speaf idy been taiiored for use

in an R&D environment. To be effective as a mode1 for R%D, several modifications needed

to be made.

R&D goals and objectives shodd be defined in a waythat shows de+ how each

researcher's activities contribute to the company's goals. These should be aligned from the

a m with the goals of the organisation. If organisational objectives are undear, then the

nom is generally for individwls to outperfom othen through a system that r e w d vaiues

diat are not conduave to organisational goals (McLaughLn, 1995).

Overaii Organisational Goais

#

l R&D Goals l

Organisationai goals should be translated into goais for the R&D effort and subsequently

individual goals for each researcher.

The literature review outlined a number of issues that F&D groups are facing in today's

marketplace. It is essential to consider each of these when designing a qwLty management

system for the R&D environment. Table 6-1 iflusvates how the characteristics of a wd- designeci quality management system could vldress each of the issues.

Issues Affccting the RdtD Enviranmcnt Individual recognition of achicvcmcnt

Oqpisationll goais vs. R&D objectives

ChPrctaStics Emphask on developmcnt of team baseci recognition R e w d s - - based - on - the dievernent of

Fast pace

I I d e rimes I

o ~ o n a l goals An environment that is focused on flexibiky and

Incrclsing cycle time adîptabiiity and empowered rcsarchcrs A systan t h encoqes the systemîtic

1 resources Inaeased cornpetition A syaan that piaces the o r p k i o n in a position

'

to unckmnd ascorner's current and future needs

I

Fewcr rcsourccs A system that leaàs to improvernent in the effectiveness and efficiency of the R&D proceses which enables improved performance with fewer

GQfLving the customa's changing nccds

6.3.1 Framework

to continuously iinprove its ab* to fuifil these neeâs and remin competiùve I

Emcrgence of ncw technology

The R&D hinction can be viewed as a mbsystem of the organisational management sysrem.

In this way, R&D should develop its own quaiity management system chat is aiigned and

easdy integrated with the o v e d organisational management system. This will ensure that

the R&D mbsystem is operating to achieve goals that are w d aligneci with the organisation's

overall goais. The system is comected to the overall system thmugh balances and checks to

ensure continuous alignrnent of goals and objectives. Figure 6-2 illustrates the framework

for the modified OPIM to which an RBrD subsystem has been ad& The mows in the

figure depict the interrelationships between this subsystem and the o v e d OPIM to ensure

continuhg aignment. For example, compare Figures 3- 1 and 6-2. ï h e OPIM can be applied to the qmem as a whole but the items can atm be tailoreci to be used for the R&D

A sysrem that enables the effective management of technology

system.

i MANAGEMENT i

PROCESSES

PROCESSES

MANAGEMENT

R&D PROCESSES

RESOURCES

The research and development funaion can be viewed as a system of processes whose most

important resources are technology, people and understanding of the customers'

requirements and expectations.

Once the f~amework for the modified OPIM was created, the next sep was to modify the

cnteria to reflect the specific requirement of a quaiity management system in R&D.

The categones of the OPIM should address the follo*

Obieaives The Objectives section of the modifieci OPIM should focus on management, R&D nategy

fomulation and alignment with organisational suategy, and the main nakeholders of the

R&D function. These stakeholders include interna and external astomers, senior

management, people, suppiiers and Pamiers, and society.

Resources This section should cover all resources indudeci in the origiuai OPIM but should focus on

how the RSrD function plans for, acquires, deplay$ and controls these resources Li order to

develop produas and new technology effiaendy and effectively.

Processes The Processes section should aàdress process management and the planning and conuol of

customer related processes, design and development processes, purchashg processes, and

support processes. Realisation processes originally addressed in the OPIM are not required

in this version.

This section should determine how customer requirements are completely defmed. These

requirements 'drive" the design convoi process so customers are aswed that the

organisation will design the appropriate processes to ensure cornpliance. The R8rD

professes should be planned and designed frorn the customer's point of view. This indudes

everyrhuig normally associateci with design, development, test, prototyping, and

evaluation/assessment.

The R&D organisation cian re~uest rhu supplien demonstrate evidence that pudaseci

product will m a all quality reS;rements. This permits the R&D organisation to ensure

that purchased items and seMces meet the needs of the depanment.

The key support pnxesses r e q d for effective and efficient R&D should be planned and

conuolled.

Results In the R8rD environment, the rime lag becween action and outcome is prolonged This

makes difficult the development of a comprehdve set of masures that accurateiy reflect

the outcornes of the R&D activities.

Sevenl authon have addrrssed this problem and have corne up with suggestions for

meanires that can be used in R&D organisations.

The funaion of the R&D organisation is not only to develop but &O to transfer technology to o p t i o n s

and the marketplace, to enable the compuiy to meet its business goais and therefore maximise stieholder

value. R&D's conuibution, therefore, is n a to be m d simpiy by i n t d success mevurements but

by its performance against interoll ;and e x t d customer satisfaction short and long-term pals. (Patiiio,

199 5)

In general though, these meanires should reflect the achievement of set objectives relating to

customers, people, suppliers and pannes, society, and the synem itself. The meanires

should be reported in terms of the a d level of performance, targeted levels of

performance, trends over the past three to five years, uid performance relative to

cornpetiton. - This section should address the systems for perfomiance management, performance

monitoring and conmlling, and performance improvernent related spediuUy to the R&D

depumients. It wu not necessvy to modify this section h m the O& OPIM since the

departmental performance management, monitork and controulng and improvement

should be si& to that of the CSO.

Modifications were made to the OPIM to dow it to be more applicable to an R&D

depanment within an organisation that develops, mukers, and manufactures new produm.

AppendM B conuins the m&ed version of the model. The 'organisation" refers to the

whole organisation induding marketing and manufacturing departrnents. The "R&D

funcrion* refers to the RBrD department or mit.

The following is provideci as an sumpte of the modification made to the criteria of the

OPIM.

Section 1.2 of the onguial OPIM perrains to the establishment of quaLty policies and

objectives and the o v e d suategic ph.ning process of the organisation. Table 6-2 illusuates

the critexia from this category of the OPIM.

1.2 Strategy

1.2.1 Development

i?q&mm& (ISOI, 5.3 and 5.4.1)

Describe the sysiem for establishing quaùty poliaes and objectives.

(IS04,5.3; MBNQA, 2.1; EQA, EQA, la and 2c; DP, 1:l-1:3)

Describe rhe strategic planning process appLed to improve organisational performuice and set appropriate strategies and pliaes.

1.2.2 Deployment

h p m m n k (iS01,5.4.2,5.5 and 5.6)

Describe the processes of management planning and review, as weii as the wociated respoasi bilities and authorities.

qnpimomies @Sa, 5.4-5.6; MBNQA, 1. I b and 2.2; EQA, 2d and te; CP, 1:4 and 15)

Descnbe h m suacgic pians are deployed, reviewed and improved upon.

The developrnent and deployment of strategy in R&D ne& to be linked to the overail

business planning process. This requires uuisktion of the organisatiod goals and svategic

objectives into R8rD department goals and objectives. This transformation enables the

R%D funaion to better understand the corporate stratefies and to ensure the aignment of

its suategic goals and purpose.

Table 6 3 illusvates the modifications that have been made to this categoiy of the OPM.

OPM

Rqukwa @SOI, 5.3 and 5.4.1)

Describe the syscem for rransiaMg organisational quîlity policies and objectives into p h and objectives for the R&D function.

0 Qpamm (IS04,5.3; MBNQA, 2.1; EQA, EQA, la and 2c; DP, 1:1-1:3) Describe the stmtegic pluroùig process applied to improve R&D performance and set appropriate strategies and policies.

1.2.2 Deplayment

ibptmm @Sol, 5.4.2,5.5 and 5.6)

Describe the processes of R&D management pluining and review, as wd as the associated responsibilities and authorities.

(IS04,5.4-5.6; MBNQA, l . lb and 2.2; EQA, Zd and 2e; DP, 1:4 and 1:s)

Describe how strategic plans are deplayed to the R&D function. Describe how the strategic plans, translated to the R&D function, are reviewed and irnproved upon to ennire continuing alignment wirh organisationai p h

The scoring scheme for the OPIM wu developed from the opinions of people w i t h the

organisation. These people undemuid the importance of each part of the criteria to the

performance of the business. This sarne process could be foilowed to modify the scoring

scheme for R&D application. Expen opinion from those within the R8rD depments

could be c a p d using the PFM software and a new distribution of points could be done.

This wu not possible in this case because of tirne collsvaints, so a change in the disuibution

of points was made to address the issues identifieci in the litennue nwqr.

Table 6-4 shows the categoxies of the modified OPIM that mon &cdy effect these issues.

Issues Affectiag the R&D Envimmcnt Individual recognition of achievement

0rpUi;sauonal goais vs. R&D objectives

Fast pace

Fewer resources

Increased cornpetition

Capturing the customer's changing neeb

Emergence of n m technology

1.1 MaMganait 1.3 Stakeholders 2.5.1 Hunan Resource Sysvm 2.5.3 Satisfaction, Involvernent and Empowerment 4.3 Peonle Rcsults 1.1 Managrnient 1.2 S m q y 4.6 System Results 2.5.1 Human Resoutre System 3.3 Design and devdopment processes 4.6 System Results 2.5.1 Human Rcsource Systern 3.3 Design and devdopment processes - 4.6 System Rcs& 2.5.1 Hunan Resource System 3.3 Desigu and development procases 4.6 System Results 3 2 Customer duionships 3.3 Design and development processes 4.6 System R d t s 3.2 Customer reiationships 4.1 Customer Fkdts 2.6 Technology 4.6 System Results

These categories should be vaiued the highea within the modified OPM model. Table 6-5

presents the s c o ~ g scheme for the modified OPIM. The items in boldface rype are those

caregories that have become the moa important in the rarised s c o ~ g scheme.

Criteria

1.0 Objectives 1.1 Management 1.2 Stntcgy 13 Stakeholdm

L

2.3 Information 2.4 Lnfnrrnicnire 2.5 People 2.6 Tcchnoloev

I 2.7 Nature 2.8 Work Environment

OPIM Modif~cd OPIM Cîtcgoy Po*

129

40.50 40.00 48.50 44.00

--

29.00 43.00

1 - - - - - - - ' 3 3 Design and a dcveloprnent

m~g0ry Points

146

Item Points

129.00 48.00 43.00 38.00

39.00 39.00 50.00 46.00 28.00 40.00

3.4 Purchasing 3.5 Relis;ition

28-00 220.00 50.00 30.00

4.3 People 4.4 Suppliers and Partners 4.5 Sociew

Item Points

146.00 50.00 48.00 48-00

b

-

4.6 System 5.0 Performance

6.3.4 Addressing R&D Issues with the Modifed OPIM

42.00 26.00 38.00

L

3.6 Support 4.0 Resulu 4.1 Customa 4.2 Shareholders

42.00 33.50 16.00

a

5.2 Monitoring and ConuoUing 5.3 hnprovement

Totai

The moci&ed mode1 for perfomiance management addresses many of the issues we have

seen that RgrD funcrions face today. Table 6-6 outhes how the modifications respond ro

these problems.

47.00 26.00 0.00

28.00 190.00 36.00 32.00

190

48.00 3 1.00 16.00

128

220

Io00

3 0 3 128.00

42.00 43.00

Io00

- - - - -

110 - - - -

40.00 41.00

45.00 110.00

Mocüfied Elcmcnts Design of work and job functions

Addfcssing R&D hblems Improvement of this design udi support co- ~

The system used to reward and recognise R&D opecation, collaboration and innovation Irnprovement of this system to encourage team-

pem&el -

The development and deploymor of RdrD stmegic objectives The use of technology to achieve R&D

I tïmes and maintain comOefiuVeness

b d r e w d and recignition -

Improvement to ensure the alignxnent of R&D objectives with organisational goals Improvement to ensure efficient and effective -

objectives

The identification of nistomer reqllemenfs and the Licorporuion of these recpkments into the &sip process n i e management and improvement of design and development processes

&of technology in R&D projms to meet organisationai goais Impmvement to the pnxesses of idendj&g customer qukements to ensure thar changing customer needs are being met Impmvement to increase the efficiency and effectiveness of RBrD processes to shortai cycle

A -

I 1 performance management system

The rneasuremenr of results reiated to people, customers and the system as a whole

6.4 Chapts Summary

Improvernent of the mednument to ensure the effectiveness of the R&D

This chapter illusuateci the modification of the OPM for use in an R8rD environment. The

modifications were made to address the issues outlined in the literature review. The next

section closes this thesis with a summiuy of the conclusions arising from the studyas well as

some oppomuiities for future research.

7.0 Conclusions This chapter discusses the main conuibutions of the worir presentd in the thesis, followed

by recommendations for future research.

7.1 Contributions of the Research

In Chapter Three, a proposed m d for performance management that inteptes the new

ISO 900 1 standard for +ty management wirh three modek for business excelience wu

developed. The resulting model is based on a sysrems view of organisational performance.

It incorporates the requirements for a quality manageaient systern from ISO 9001 with the

identifid strengths of each model for business excellence. The redting set of aiteria

represents a path towvds business exceiience. The organisation can evduate its c u r w t

&y management system against the minimum requirements in the model and then use

the opportunities for improvement as a guide for conMuws improvement.

A scoring scheme was developed for the model using a decision an+ model that captured

the opinions of senior management regarding the relative importance of each element in the

model for business performance. The resource element was considered to be the mon

important, representing the transition from the process-based ISO 9000 to a more system-

based business excellence.

Chapter Four describes how the proposed model was used to perform a seif-assessrnent of

one division in the case study organisation. A set of questions was developed to obtain

information through interviews with a cross-section of employees from the CSO. The self-

assessrnent was successful in obtaining mudi idonnation penainiag to the organisation's

acrivities. It provided the CS0 with a comprehensive lia of areas in which to focus

improvement efforts.

Chapter Five compared the three perfomunce measurement rnethods used by the case study

organisation. This induded a cornphon of the seif-assessrnent outcornes with the results

of a quality audit perfonned at the urne rime as a m- of investigating the compatib* of

the IWO measurement methods. The chapter ais0 compareci the use of result-based

performance measurement to the ourcomes of the self-assessment and audit. This process

reinforcd the importance and interdependence of each method in an organisation's

performance measurement efforts: result-bd performance measurement is used to capture

the CUrrent performance levels withli the organisation; @y a u d i ~ g is used to detemiine

if an organisation is meeting the requkements for its managenent system. Self-

assessrnent is used to enhance both of these measurement processes, as it requires that a

Company evduate the effiaency of its activities. This reveals why a process is perfonniflg at

its current lwel and suggests why a qualRy au& has found non-cornpliance.

The cornparison of the auditing and assessrnent showed the foIlowing

Perfomiing self-wessment on eiements related to qeddty management can provide

valuable infomiation regarding the efficiency of these processes. 'Iliis enhanas the

@ty audit, which aims to judge the effectiveness of mch processes.

The svengths and weaknesses identified in the self-assessrnent can be incorporated into

the business planning process. This ensures that the veas for improvement, rekted to

the quality management system, are considered, compensahg for the absence of quality

audits results, in the business planning process.

Self-assessrnent of the processes relating to quality management, especiaily if p e r f o d

at opposing times of the year from the internai quality audits, would lead to f d e r

Mprovements in the efficiency and effectiveness of these processes.

Chapter Six ouùined how modifications were made to the OPIM to create a more

appropriate model for use in an R%D environment. The modiGed model addresses many of

the issues facing the CSO's R&D departments in today's marketpIace. This is a renilr of the

new focus on the most important aspects of the OPIM to W. These areas are:

Suategy dwelopment

Stakeholders' objectives

Human resounces

Techn01ogy resources

Oisromer related processes

Design and development processes

People renilu

Syaemresults

The scoling scheme was +Lo changed ro reflect the new focus of the OPM for R8rD. The

vaiue of the above listeci catepies wu increased Using rhis proposed model, the

organisation can assess its R8LD activities in more detaiL

In summaxy, the main contributions of the wotk in this thesis are the:

development of an imegrated mode1 to move beyond the minimum requirernents of ISO

9001 to create the proper conditions for business excellence

funher investigation into the compatibiity of self-assessment and audithg for future

alignment or integration of these performance measunment methods

examination of the roles of self-assessment, audiMg, and redt-based performance

measurement in the o v e d performance mevurement system

a modifieci version of the OPIM which provides a good framework for future

development and application of this model in an R&D environment

7.2 Scope for Further Research

The following are recommended issues for further research:

the use of the modified OPIM to conduct a self-assessment in an R&D environment

investigation into the use of self-assessment to enhance the @ty audithg process

smdies on the integration of self-assessment and audit te. using an integareci evaluation

methd for measurernent against the OPM.

funher smdy of the relationships between redt-based performance measurement and

self-assessrnent and their impact on business performance

funher use of the OPIM for organisationid self-assessrnent and investigation into the

effectiveness of the m d for improvernent

8.0 References

Auer, A., Karj*e!n, J., Seppanen, V. (1996), 'Improving R%D processes by an ISO 900 1- based quality management system", jar mal^^ Ahaime, Vol. 42.

Bde lemy , J. L., Zairi, M., (1994), 'Makmg ISO 9Wû Work: The Role of Auditingw, TQM Mag& VOL 6, No. 3, pp. 44-47.

B a d & J., 'Preference Function Modeliagn, ~://~~~.Scien~c&asiom.com/, (6 Augus, 2001).

Bardai, J., (1998),"Mea~urement Foundations for Preference Funaion Modelhg," l h ~ d n g ~ o f t h l o l E E E In&nduzdciqkbxz<n S+ttms, hiha, a n d C j b m ~ ~ ~ , pp.4038-4044.

Bandai, J., (1997), "A New Methodology for Deaiing with Contradicting Engineering Design Criteria," l + t x d p tfi4.e 18th A d Mehg fPbe Ammkan h e t y f i En- M' pp. 73-79.

Beeler, D. L., (1999)," Intemal auditiag the big liesw, QuakyAug>ea, Vol. 32, No. 5, pp- 73- 78.

Biutu, US., Cimie, A.S., McDevïtt, L., (1997), 'Intepteci performance meanirement systems: a development guide ", 1- J d t f O p e r i & ISulrrIio2 Manugemt, Vol. 17, NO. 5, pp.522-534.

Bodinson, G. W., (1991), 'Waming: ignoring ISO standards may be h d co your company's fumen, .i& Manrqgeney Vol 33, Pt. 2, pp. 11-12.

Bohoris, G. A, (1995), 'A comparative assessment of some major qulliy awardsn, r ~ / d o f Q & y r D d R $ t r t t n l i ~ ~ , Vd 12, No. 9, pp. 30-43.

Brown, M. G., Hitchcock, D. E., and W h 4 M. L., (1994), Why ïQM Fa& and Whit mdo A b It, Irwin Professional Publishg Bum Ridge, IL.

Brown, M. G., Svenson, R. A., (1988), 'Meanuing R&D Pruductiviry", Resed T i Mm~q3etiem, Vol. 31, No. 4., pp. 11-15.

Burr, J. T., (1990), 'The future necessisr, Quriby h p s , June, pp. 19-23.

Chin, K. S., Yam, C. M., and Rui, K. F., (1995), 'Continuou Qwi;ty Improvement - beyond ISO 9O0on, Total Qaky RaiRu, VOL 5, No. 1, pp. 53-59.

Glurcio, M., (1998), 'ISO 9000 quaiity v e m and product innovation: new evidence from shoe manufaauring factories", Pnmzhfirm PSP 4 E U S M InaornatidlPl- M--

Conti, T., (1999), "Vision 2000: positionhg the new ISO 9000 standards with respect to total +ty management modek", T d Qdty Vol. 10, Nos. 4/5, pp. 454-464.

Conti, T., (1993), Taal @&y: A A& M d a q p m ~ , Ch;anui & Hall, London

Cooper, R G., Kleinxhmidt, E. J., (1996), ' Wiming Businesses in P d u a Development: The Gitical Success Factors", R d T- M ~ ~ E P z , Jdy-Au- pp. 18-29.

Cooper, R G., Kleinschmidt, E. J., (1994), 'Deterniinants of Tiieliness in Produn Dweiopment" , Jd cf- ~ ~ U Z & P Z -, vd 11, pp.38 1-396.

Cooper, R D., Dormuth, K. W., Gug, 1. D., Sherman, GR., Truss, K. J., and Wlborn, W., (1990), "App& @ty assurauce to R&D projectsw, Q z u k y h p s , J u k

Dale, B. G., Bunney, H., (1994), T d @&y UmqgRlait, Blackwel Publishers Ltd, Mass.

Dale, B. G., (19991, M e - , 3d Ed, Blackwell Publishers Ltd, Oxford.

Davidson, J., Pruden, A., (1996), "Quality deployment in R&D organisationsw, ResearrS T-&=W=4 Jan--

Deming Application Prize (1996), DenilgPme Culcilojbr(ksnnr Co>lpilier, Japanese Union of Scienàsts and Engineen.

De Weerd-Nederhof, P.C., Fisher, O. A. M., and Kerssensvan-Drongelen, 1. C., (1995), 'Managing quality in R%D: social-dynamic aspects and the importance of generating new organisational configurations", P d g s , 3" EIASM I . E9oduct Amkpmt Mmqpmm Gpyfbmz In Jayawarna, D., Pearson, A. W., (2001), 'The Rde of ISO 9001 in ManagLig the Qu?Lty of R&D Activities" , n e 7QM M a g e Vol. 13, No. 2, pp. 120- 128.

Dixon, J. R, Nanni, A. J., Vollrnann, T. E., (IWO), The Neru I+gbmm C h h g - Mes* qmmim~& Wd-Glrrs Ccnpmk, DOW Jones-Irwin, Hommvood, Il.

Eknuti, (1996), 'World-class standards for global competitiveness: An overview of ISO 9000" I . M h p m a t , Sept/Oct, pp. 5-9.

Endres, A. C., (1992), 'Resdts and Conclusions from Applyiig TQM to R e s e h " , 464 A d @&y Ca?g>ar, May, pp. 426-430.

European Foundation for QuîLty Management (1999). Asssmqql6r Exc$lerre - A P r . S$Asses911~~, E~uropean Fouadation for QuaLty Management, Bnissels.

Eutopean @ty A w d (1999), EFQM E x d b m M d , Empean Foundation for Quality Management

European Foundaaon for Q d t y Management, (1995), S e ~ A s s e s s n r ~ ~ a G d h s , 1996, Bnissels.

Fktgedd, Le, Johmon, R, Bngnail, S., Silvestre, R, Voss, C., (1991), Meaamrrnt in Sm& bu si ne^^, CM& London.

Franck, P. H., (1992), ' P&g Qd~v h o the R&D Process", R e s d T- Man<sanem, Jdy-Aug~n, pp. 16-23.

Garvin, D. A., (1993), 'Building a learning organisationw, U n d Burins July- August, pp. 78-91.

Garvin, D., (1988), M-ug Q&y & Strmirgicd Gmplze Et&, Free Press, New York, NY.

Ghalayini k M., Noble, J.S., (1996), "nie changiog basis of performauce measurementn, r w j d 4 - h 6 hxhufim IçlroLgenar, Vd. 16, No. 8, pp.63-80.

Ghobadian, A, Woo, H. S., (1994), "Ch~aceristics, benefits and shortcomings of four major q d t y awarbn, hnemiiaird J i x m d ~ Q u h t y a d R e M z & t y ~ , Vol. 13, No. 2, pp. 10-44.

Globenon, S., (1985), 'Issues in developing a performance crite& system for an organisation", I - I d o f B u h m &sud, Vd. 23, No., 4, pp. 639-646.

Goetsch, D L , Davis, S.B., (1998), U-md ISO 9000 and I S O M , Prentice-Hd, Inc., New Jersey, 1998.

Grandzol, J. R, Gershon, M, (1998), 'A Survey Innnunent for Srandardising TQM Modehg Research", Intemamal Jd cf- S c i k ~ , VOL 3 No. 1, pp.80- 105.

Ho, S. K. M., (1999), "Change for the better via ISO 9000 and TQM", Manu--, Vol. 37, No. 4, pp. 381-385.

Hutdiins, D., (1990), In P d tfQuhy, Plmian Pubiishing, London.

ISO 1001 1, (1990) G d & n e s j b Q & y Azuhq Qdty Systm~, I n t e d o n a i Organisation for St andardkation, Geneva, Switzerland

ISO MO 10, (1996), Gd&t15h E + v m d A h g - Gemd Phncqde cfEmSiIIPtlSttrtl Al<drtmg, Intemational Organisation for Standardisation, Geneva, Switzerland

ISO DIS 19011, (2W1), G ~ m q ~ a r r d / a r m ~ ~ ~ y n s > u m d m n g , Intemational Oqpkation for Standardisation, Genwa, Switzerland

ISO FDIS 9000 (2000), QulLty i k k q p m t .ymmC - FunbmMk a d V e , International Organisation for Standardisation, Geneva, Swkzrland

ISO FDIS 900 1 (2000), Qudrty Ahqpmzt Spam - R e q k m t s , hternauonai Organisation for Standardisation, Geneva, Switzehd.

ISO FDIS 9004 (2000), Qdty IFysrem - GU*^ l b j b ~ ~ ~ h p m a m ~ , International Orguiisation for Standardisation, Genwa, Switzerland

Izadi, M., Kashef, A. E., dr Stadt, R W. (1996). 'Quaiity in higher educauon: Lessons leamed from the Baidnge Deming prize, and ISO 9000 regktration". /ou+ I m k d T& ~~ VOL 33, No. 2, pp. 60-76.

Jayawama, D., Peuson, A. W., (2001), 'TheRole of ISO 900 1 in Muiaging the Quality of R&D Activities", ïk 7QM VOL 13, No. 2, pp. 120- 128. Kaplan, R S., Norton, D. P., (1999, TT* S t r - ~ Z Z O Action: ihe Balmnd S 4 , Harvard Business School Press, Boston, Mass.

Karapetrovic, S., (1999), 'ISO 9000: the system emerging h m the viaous cirde of compLanceW, ?he 7QM M m VOL 1 1, No. 2,1999, pp. 1 11- 120.

Karapetrovic, S., Wdbom, W. (200 lA), "Audit system: Concepts and Pmctices", TotalQclLry M-, Vol. 12, No. 61, pp. 13-28.

Karapetrovic, S., Wdborn, W. (200 l), "Audit and Self-Assessment in Qualiry Management: Cornparison and Compatibility", Managend Audithg Journal, Vol. 16, No. 6, pp. 366-377

Karapeuovic, S., Macey, S., (2001). ' A Model for Perfoxmance Management and Assessment", Working paper, Department of Mechanid Engineering, University of Alberta.

Karapetrovic, S., Wdbom, W. (2000), "Genenc Audit of Management Systems: Fundamentals", M W A&mg/<lmill, Vol. 15, No. 6, pp. 279-294

Karapeuovic, S., Wdbom, W., (1998A), "The Systems View for the Clarification of Quality Vocabulary", J d q f Q d z t y & R$tabiltry-, Vol. 15,No. 1, pp. 99- 120.

Karapetrovic, S., Wdbom, W., (1998B), "Integated audit of management synemsn, r i jddwv1odR$iabi l iq ~m<qpmm, vd 15 NO. 7, pp. 694-71 1.

Kielia, M. L, Golhar, D. Y., (1997), 'Tod quaiity muiagemait in an R&D environment", I n t e m a t C Q n a l J d c f O p e r h ClllZIAndrrIaar UPulp"int, V d 17, NO. 2, pp. 184-198.

Keegan, D. P., Eïler, R G., Jones, C. R, (1989), 'Are your performance mevures obsoke?", Mamapmz Ac- June. pp. 45-50.

Kumar, V, Boyle, T., (2001), 'A Qualiry Management Implementation Framework for Manufacnving-based RBrD EnWonments" , I n d J a r m r J o/Q&y dnd Re* M m Vol. 18, No. 3, pp. 336-359.

L a w , C.W., Noble, J.S., (1998), "A Framewotk for Business System and QuaLty Management Integrationw , I u J a m i r d dQdzty & R$latrlny Mbqpmü, Vol. 15, No. 6, pp.566-581.

Laszlo, G. P., (1997), 'U.S. and Canadian national quality awards: increased emphasis on business resulu", 7Z.w 7QM Ahqpzzw? VOL 9, NO. 5, pp. 38 1-383.

Lazlo, G. P., (1996), 'Quahy a w d - recognition or model", The ïQMIti;zp&, Vol. 8, No. 5, pp. 14-18.

Loomba, A. P. S., Johannessen, T. B., (1997), 'Makoh B&ge Na t iod Q d t y Awad Gitical Issues and Inherent Valuesn, -@&y Màmgmnt & Tiirhmlogv, Vol. 4, No. 1, 1997, pp. 59-77.

Malcolm Baldrige National Q d r y Award (2000), Cri8tzjbrAqhmzm Exdmœ, NIçT, Gaithers bug, VA.

Mann, R, Voss, M., (2000), 'An innovarive process irnprovement approach rhat integrates ISO 9000 with the Baldnge frameworkW , Berhdmg An Irrtematio?al Jd, Vol. 7, No. 2, pp. 128-145.

McLaughlui, G. C., (1995), T d @&y bz Reemrhrmd-, The St. Lucie Press, Total Qdty Senes, Florida.

McWaid, T., and Gale, R, (1995) ''Summyr of the quality self-assessrnent of the precision enginee~g division", Planning Workshop of the NIST Pr~ision E n g i n e e ~ g Division, Defence Systems Management CoIIege, Fort Belvoir, VA

Meegan, S. T., (1997), 'A model for muiaging the transition from ISO 9000 to TQM", T r i rQi*rlP/, Vol. 5, No. 1, pp. 35-39.

Meegan, S. T., and Taylor, W. A., (1997), 'Factors iduencing a successfd transition h m ISO 9000 to TQM the influence of understanding and motivation", Intencmd J&qf @&y & RdddqiMnqpm~, Vol. 14, No. 2, pp. 100-117.

Montana, A. J., (1992), 'If it isn't Perfect, Make it Better", R e s d TeoSndogy M- Jdy-Augun pp. 38-41.

Najrni, M., Kehoe, D. F., (2000), 'An integrated fkmework for post-ISO 9000 +ty developmentn, I izasn<IuotrJ]OUtTtd@M 6 R$rabJiry Adkgmnt, Vol 17, No. 3, pp.226- 228.

Neely, A., (1999), "The Perfomiance Measurement Revolutiorx Why and What Next?" 1- Jd of OpPr& ard&u.htim -, V d 19, No. 2, pp. 205-228.

Neely, A., Gregory, M., Plans, K., (1995), "Performance Measurement Synan Design; A Literature Review and Research Agendan, In- Jd of*& rOd h z h m M-, Vol. 15, No. 4, pp. 80-116.

Oakland, J. S., Total Qdtylçtouigrnien- 7he Rwte t>hpvwgA+mun, 2nd eà, Bunerworrh-Heinmann, Oxford, 1993. In Sindair, D., Zain, M., (1995), "Performance Measurement as an O b d e to TQM", % ïQM M@, V d 7, No. 2, pp.42-45.

Pannirselvam, G. P., Ferguson, L. A., (2001), 'A audy of the relationships bmeen the Bîldrige categoriesn, ItttemdtMrtal Jd &&&y dltdR$ia6Jiru hîkmpmz, Vol 18, No. 1, pp. 14-34.

Patiao, H., (1997), ' A p p b g Total Quahty to R&D at Coors Brewing Company", Resead Tkhinkgy Mnqpm, September-Oaober, pp. 32-36.

Pearson, A. W., Vaughan, N., Butler, J. (1998), 'The implemenration of TQM in R&Dn, r i J d o f T-- VOL 16, NO. 4/5/6, pp. 405-432.

Peters, J., (1998), ' Some Thoughts on Auditing", TQMMagazzne, Vol. 10, No. 1, pp. 4-5.

Porter, L. J., Tanner, S. J., (1996), Assess* b i n e s Exo$letrp, Bunewod-Heinemmq Oxford

Puay, S. H., Tan, K. C., Xie, M., Goh, T. N., (1998), 'A comparative study of nine national quality awaràs" , 7fa 7QM ~~, Vol. 10, No. 1, pp. 30-39.

Public SeMce Office, 'Next Steps Reportw, hnp://www.offiddocumatsco.uk /document/an38/3889/chap-2b.htm (25 July, 2001).

Pun, K. P., Chm, K. S., Lau, H., (1999) 'A self-assessed quality management system based on inregration of MBNQA/IW 9000 and ISO 14ûûûm, I n t ~ ~ d . f Q d t y r O d R&h@ Ultrgenea, Vol. 16, No. 6, pp. 606-629

Pun, K. F., (1998), "Implementhg an integrad quality m e m e n t system", ProLmeduigs of the 52 Annual Quality Coagress, PPhiLdphia, PA, m, pp. 356-362.

hi, S. C., (1995), ISO 9000 e>tgEumc.rd T d @&y Standards-Qualiy Management Group, Ottawa-Nepean, Ontario, Canada, pp. 18 1 - 183. Russell, S., (2ûûû), 'ISO 9000: 2000 and the EFQM Excellence Model: cornpetition or co- operation?", T d @&y VOL l l, Nos. 4/5 & 6, pp. 657-665.

Schalkwyl5 J. S., (1998), 'Td Quahty Management and the Perfomiance Barrierw , The I p M Magazk, Vol. 10, No. 2, pp. 124- 13 1.

Shaw, A, (1999), A Gude toR+mam? Meaammeitad N m - F i d lidEurans, Mariison Public Relations, The Foundation for Performance Measurement, S u q

Sinclair, D., Zaki, M., (1995), " Effective pprocess management through performance meastuernent - Part 1 ", BusmeJs Anses Rem- & M m q p m m ] d , Vol 1, No. 1, pp.75-88.

Sinclair, D., Zai~$ M., (1995). " Effective process management through performance measurement - Part II", Burinss RUBs R e - e n m & M m q p m t J d , VOL 1, No. 2, pp.58-72.

Sinclair, D., Zain, M., (1995), " Effective process management rhrough performance measurement - Part III ", B h Arrss Re-engkmkg & M- J d , Vol. 1, No. 3, pp.50-65.

Sun, H., (2000), "Total quaiity management, ISO 9000 certification and performance

Swyt, D. A., (1999), "Experiences in initiating Baldrige-based quality in a NIST technid divisionw, 7be 7 Q M MaguzZne, Vol. 1 1, No. 3, pp. 142-1 56.

Texziovski, M, Samson, D., (1999), 'The Mc b e e n total q d t y management pnctice and organisationai performancen, I d Jd 6Qafity & Rd&& kîbqpmt, Vol. 16, N0.3, pp. 226-237

Tonk, H S., (2000), 'Integrating ISO 9001: 2WO and the Baiàrige Criteria: A pmpod for a 2 in centuy @ty paradgmw , QrdIy Aqpa, August, pp. 5 1-55.

Tummala, V. M. R Tang, C L, (1996). 'Stmegic qwlity management, Malcolm Bddnge and European quaiity awards and ISO 9000 certification: Core concepts and c o m p d v e a.nalyssn, Intanmuxlal J d o f Q d k y a d R $ t a b r l r t v ~ , Vol 13, No. 4, pp. 8-38.

van der Wiele, T., Bmmm, A, MiIlen, R, Wh& D., (ZOOO), 'hprwement in Orgamsational Performance and Self-Asse-nt Practices by Selected Amencan Firmsw ,

Abu- J t VOL 7, NO. 4, pp. 146- 157

van der Wiele, T., Brown, A, (1999), 'Seif-assessrnent practices in Europe and da", 1 , Jdqf@&yrIdR$iability- V d 16, No. 3, pp. 238-251.

van der W~ele, T., Dale, B. G., and W k , A. R T., (1997), "ISO 9000 series registration to t o d quaiity management the transformation journey", I n t d Jd dQ&y ScEslao, Vol. 2, No. 4, pp. 235-252.

van der Wiele, A., Williams, A. R T., Dde, B. G., Gner, Ga, Kolb, F., Luzon, D. M., A. Schmidt, A., Wallace, M., (1996), 'Self-assessment: A midy of progress in Ewope's leading organisations in @ty management practices", I n W Jd of Qdty dR$iabilrty Mlm<qge>le~, Vol. 13, No. 1, pp. 84-104.

van der Wiele, A., Williams, A. R T., Dale, B. G., Grtu, G., Kolb, F., Luzon, D. M., A. Schmidt, A., Wallace, M., (1995), 'State-of-the-an snidy on self-assessment", 7he TQM M a g k , VoL7, No. 4, pp. 13-17.

WiIIborn, W., Cheng, T.C.E., (1994, G i a M<maameit qfQuhy A s w i Systena, McGm- HiIl, Inc., New York. Li Karapetrovic, S., Wdbom, W. (2W1), "Audit and Self-Assessrnent in QuaLty Management: Cornparison and Compatibilky", Managerial Auditing J o u d Vol. 16, NO. 6, pp. 366-377

Wdbom, W., (1990), "Dynamic Auditing of Quahty Assurance: Concept and Method", 1- J d o f QdtymrdR$ tabr l s~~ M i q p r m ~ , VOL 7, No. 3, pp. 35-4 1.

Zink, K. J., Schmidt, A., (1998), "Practice and Implementation of self-assessment", Zn&m&ud J o d of C&dity Sarna: Vol. 3, No. 2, pp. 14% 170.

Zink, K. J., Schmidt, A., VoB, W., (1997), " T o d quality "down under": the Aunnlan Quality Award - an innovative mode1 for Business Excellencew, The 7QM Mug& Vol. 9, NO. 3, pp. 217-220.

9.0 Appendices

APPENDIX A - OPIM Qucstior~~zirc

APPENDIX B - Modified OPIM for R&D

APPENDrX C - Trillium Results

APPENDXX D - Cornparison of Pafomunce Management M d Criteria

APPENDIX E - Appach, Deployment and Results adapted from the EQA

ORGANISATIONAL PERFORMANCE IMPROVEMENT MODEL Q U E S T I O N N .

1. Objectives

1.2 Management

1.2.1 system

1. How do you idennfu the processes needed for your +ty management

systern? Whu are these processes?

2. How do you determine the sequence and interaction of these processes?

3. How do you continually impmve the processes and the management of

the relationships h e e n processes of the +ty management system?

1.2.2 Leadership

4. H m do leaders cornmunicate the importance of meeting customer as

wd as regdatov and legd r e q k e n t s ? (IS04,S.l)

5. How do leaders ensure the availability of resources to meet these

requirements? (IS04,S. 1)

6. What is the management review process and at what frequency is it

conducted? (IS04,5.1)

7. How are leaders involved with customen, parmers and representatives of

society?

8. How are these processes continually improved?

1.2 Suategy

1.2.1 Development

9. What is The organisation's suategic planning process?

10. How are strategic plans reviewed and impmved upon to ensure

alignrnenr with the nuTent and funire business environment?

How is the oqgmktion's mtegy t r m s w into actionable items and

meaninble goals?

H m are! these communicated and deployed to each management level to

assure individual contribution to achievement?

How are responsibilties for these items eaablished and communicated

to the people in the organisation?

H m is progress agakt these plans rneasured?

How are the p r o c e s for deplayment of strategy conUnwiiy impmved?

1.2 Stakeholders

1.2.1 Customers

16. H m do you determine and review your customers' key requirements?

17. How is rhis process continually improved to keep up with current and

future business needs?

18. How are these needs and expeaations deployed and communicated to

people throughout the organisation?

19. Please provide any perception measures andior performance indicaton

that indicate performance relative to the needs and expectations of

customen. State their current levels and trends. List the targeted levels of

these performance indicators.

1.2.2 Shareholders

How does the organisation idenufy and review shareholders' needs and

expeaations?

How is this identification process continualiy improved?

How are these needs and expectations deployed and communicated

throughout the organisation?

Please provide any perception measures a d o r performance indicaton

that indicate perfomiuice relative to the needs and expeaations of

shareholders. Include both financial and non-financial iadicators. State

th& current levek and uenb. List the targeted levels of these

performance indicators.

1.2.3 People

How does the organisation identify and review people's needs for

recognition and wok satisfaction? (ISO), 5.2)

How does the organisation idenrify and review people's needs for

cornpetence and knowledge developmem? (IS04,5.2)

How are these processes continuaUy improvdi

How are these neeck and expeaations deplayed and communicated

h u g h o u t the organisation?

Please provide any perception measures &or performance indicaton

that indicate perfonnatlce relative to the needs and expectations of

people in the organisation. State their current levels and d. Lin the

q e t e d levels of these performance indicators.

How does the organisation dwelop partnerships that iink to its arategic

objectives? (CABE 6.1)

How does the organisation idenufy the needs of these pamerships?

How are these pannerships reviewed to ensure rhey are still meeting

objectives of the relauonships?

How are these processes continually improved?

How are these nec& and expectations deployed and communicated

throughout the organisation?

Please provide any perception measures and/or perfomunce indicaton

that indicate performance relative to the neeb and expectations of

piumerships. State their current lwek and trends. List the tvgered levels

of these performance indicators.

1.2.5 Society

How do you i d e n e and manage the impacts of your products, seMces

and operations on society?

How do you identify other needs and expectations of society in the

process of sening your objectives?

How are these needs and expectations deployed and communicated

throughout the organisation?

How are these needs and expectations waluateci?

How are these processes continually improved.?

Please provide any perception masures d o r performance indicaton

that indicate performance dative to the needs and expectations of

society. State their current levels and trends. List the targeted levels of

these performance indicators.

2. Resources

2.1 Finances

4 1. How does your department manage its budget?

42. How do you monitor your accual spending relative to your budget?

43. How do you continually irnprove this process?

44. Please provide any performance indicaton relating to fmanùal resources.

Stare their current levels and trends. List the t qe t ed lwels of these

indicators. (OHM 4.6.2)

45. How does your department manage idonnation to achieve set

O bjectïves?

46. How do you monitor and convol the use of information?

47. How do you contindly improve these processes?

48. How do you ensure avdabilty of information?

49. How do you detexmine the effectiveness of the information system.'

50. Pl- provide any performance indicators r&ting to information

resources. State di& current lwels md mnds. ~ i s t the targeted levels of

these performance indicators. (OPIM 4.6.2)

2.3 Infrastructure

5 1. H m does your deparunent manage infasmicntre resources?

52. How do you monitor the use of infrastructure resources?

53. How do you continualiy impmve these processes?

54. Please provide any performance indicaton relating to infrasrncture

resources. State rheir curent lwels and trends. List the targeted levels of

these performance indicators. (OHM 4.6.2)

2.4 People

2.4.1 Human Resource System

How do you idenufy characteristics and skills needed by potential

employees that match the organisation's values and needs?

How are work and job funcrions designed to support CO-operation,

collaboration, individuai initiative and innovation?

What is the process for hiring and reuuiting new employees?

How is this process continually irnproved to meet future business needs?

How do you link reuwd and recognition of individuai's performance to

your suategic direction? (CABE 4.1)

How do you link reward and recognition of team performance to yuur

strategic direction? (CABE 4.1)

Please provide uiy performance indicators relating to the human

resource system. State their w e n t lwels and trends. Lin the targeted

lwels of these performance indicaton. (OPIM 4.6.2)

2.4.2 Competenues, Training, and Development

H m do YOU as~ess/evaluate departmentai skill levels to idenufy u e a ~ for

vainiog and education?

How do you wess/evduate individual skills and areas for individual

naining/ development and career progression?

H m do you seek and use input from employees and th&

s u p e N i s o r s / ~ e r s on education and uining needs, expectations, and

design? V N Q A 5.2)

How do you design or idenrify educational and vainiog prognms that

support individual and organisation needs?

How do you derexmine if an educationd or training has been effective or

successful? What type of follow-up is thml

H o - does nianagement reinforce and encourage employees to use the

skills, tools and techniques that thqr have leam&

How do you idennfy ski11 requimnents for future business needs?

Please provide any performance indicaton relating to cornpetencies,

tminhg, and development. State their current levels and trends. List the

targeted levels of these perfomiance indiators. (OPM 4.6.2)

2.4.3 Satisfaction, Involvement and Empowerment

How are people's suggestions and ideas captured, encouraged and

implernented? (CABE 4.2) How many suggestions have you received to

date? How have these suggestions improved the division's performance?

How are people encouraged to innovate and take Rsks in order to

achïwe goals? (CABE 4.2)

How does The organisation detexmine the key factors that affect

employee well-being, satisfaction and motivation?

Mease provide uiy performance iadicators relating to employee

satisfaction, wd-being, and motivation. State their m e n t lwels and

trends. Lin the targeted levels of these performance iadicators. (OPIM

4.3.1, OPM 4.3.2)

2.5 Technology

74. How does your deparmient manage technology to achieve set objectives?

75. How do you monitor and control the use of tedinology?

76. How do you contindy "prove these processes?

77. Please provide any performance indicaton relating to tedinology

resources. State their current levels and trends. List the targeted levels of

these performance indicators. (OPIM 4.6.2)

H m does The organisation address and improve workplace health,

s a f q and ergonomie factors?

How do emplcryees take PUS in idenafying these factors and in

ixnproviog wodspiace safery!

Please provide any performance indicators relating to health and s a f q

results. State their current levels and trends. List the targeted levels of

these performance indicators.

How does the organisation determine when to upgrade/improve the

work area, decor, and equipment in your location?

Please provide any performance indicaton relating to the work

environment. State their m e n t levels and trends. List the targeted levels

of these performance indicators. (OPIM 4.6.2)

2.7 Nature

83. What is the process for idenufyuig naturai resources that can influence

the organisation's performance? (IS04,6.7) What are your plans for rheir

continued availability or contingency plans to prevent or minimise

negative effects on the organisation's performance? (IS04,6.7)

84. How do you optimise your connimption of utilities? (EQA, 4c)

85. What do you do to d u c e and recycle waste? (EQA, 4c)

3. Processes

3.1 Management

86. How do you p h and develop the key pmcesses within your depamnent

and how they intedate? (IS04,7.1)

87. How do you manage these pmcesses and their iaterrelationships?

88. How do you improve the efficiency and effecriveness of these processes?

89. Please provide any performance indicators relating ro the key processes

within your depment . State their current lwels and trends. List the

targeted levels of these performance indicators. (OHM 4.6.1)

ûinomer relationships

90. How do you design processes that affect the organisation's rekrionships

with customers?

9 1. Hm are these processes implemented and maintaineci?

92. How do you continuaily improve these processes?

93. Please provide any performance indicators relating to cunomer

relationships. State their cunent levels and uends. List the targeted levels

of these performance indicators. (OPIM 4.1.4)

3.2.1 Knowledge

94. What is the process for idenufying and targeting customers, customer

groups and market segments?

95. How do you consider customers of cornpetitors in this process?

96. How do you continuaDy hprove the process of idennfying customen?

3.2.2 Satisfaction

97. What processes are used for determinhg Nnomer satisfaction?

98. How do you follow up with customen on produas to receive prompt

and actionable feedback?

99. How do you obtain customer satisfaction information relative to

cornpetitors a d o r benchmarks?

3 -3 Design and/or development

3.31 Planning

101. How do you plan design and development processes to ensure that they

respond to the neeb and expectations of the organisation's cusromers

and other interested parties? (IS04,7.3.1)

102. How do you implanent and maintain these processes?

103. How do you contindy improve these processes?

3.3.2 Inputs

104. H m do you ensure that inputs to design and development are accurate

and complete? (IS04,7.3.2)

105. How is rhis process continua& impmvd

3.3.3 Activities

106. How do you irnplement and maintain design and development

processes?

107. How are these processes conrinuah/ improved?

3.3.4 Outputs

108. How do you validate output againn input requirements and acceptance

criteria to ac hieve int erested p a q satisfaction? (iS04,7.3.2)

109. HOW is this process continually improved?

1 IO. How is âesign &ew incoqmrated into the design and development

processes?

î i 1. How is the design review process continualh/ improvdi

3.3.6 Vedication

112. H m is design verification incorponteci into the desigri and development

processes?

1 13. H m is the design veficatîon process contin.aUy impfoved?

3.3.7 Validation

114. H m is design validation incorporateci into the design and development

processes?

1 1 S. H m is the design validation process conUnually impmved?

3.3.8 Changes

116. H m do you incorporate changing customer/market requirements into

your designs and development processes?

1 17. How is the process of making design changes continualiy improved)

3.4 Purchasing

3.4.1 Planning

118. How do you i&nufy the resources needed, which cannot be obtained

interna& in order to defme a purchasing strategy? (IS04,7.4.1)

1 19. How do you idenufy and select capable suppliers so as to ensure

consistent quaiity and dependable nipply? (IS04,7.4.1)

120. How are these processes continualiy hproved?

121. How do you define processes to ensure that ~~ecification~ for purchased

product meet the requirements of the orpi&on and interested parties?

(IS04,7.4.1)

122. How do you implement and maintain these processes?

123. How do you continually improve these processes?

3.5 Realisation

3.5.1 Pianshg

124. How do you plan and design realisation processes to ensure that they

respond to the neeb and expeaations of the organisation's customers

and other interened parties? (IS04,7.3.1)

125. How do you contiauaUy improve the design of these processes?

3.5.2 Convol

126. How do you implement and maintain realisation processes?

127. How do you improve the effciency and effectiveness of these realisation

processes?

3.6 Support

3.6.1 Planning

128. How do you determine key support process requirements?

129. How do you plan and design suppon pnxesses to meet ail key

requirements?

130. How do you conth& improve the design of rhese support processes?

3.6.2 Conuol

13 1. How do you impiement and maintain key support processes ensure

meeting key perfomiance mquirements?

132. How do you improve the effiuency and effectiveness of these processes?

4.1 Customer

4.1.1 Reputation (OPM 32.2)

4.1.2 L~~&~(OPhil3.2.2)

4.1.3 Outputs(OPIM1.3.1)(OPhil3.2.2)

4.1.4 Interfaces (OPIM 3.2)

4.2 Shareholders

4.2.1 Financial(OPM1.3.2)

4.2.2 Non-Finanaal (OPIM 1.3 2)

4.2.3 Interfaces (OPIM 1.3.2)

4.3 People

4.3.1 Motivation (OPIM 2.4.3)

4.3.2 Satisfaction (OPIM 1.3.3)

4.3.3 Achievemenu (OPIM 2.4.2)

4.3.4 SeMces (OPIM 2.4.3)

4.4 Suppliers and Panners

4.4.1 Inputs ( o p h l 3.4.2)

4.4.2 Interfaces (OPIM 1.3.4) (OPIM 3.4.2)

4.5 Society

4.5.1 Citizenship(OPIM1.3.5)

4.5.2 Gmmunity(0PIM 1.3.5)

4.5.3 EnWonment (OPIM 2.7) (OPIM 1.3.5)

4.5.4 Heaith and Safety (OPIM 2.6) 4.5.5 Interfaces (OPIM 1.3.5)

4.6 System

4.6.1 Processes (OPIM 3.1)

4.6.2 Remurces (OPIM 2)

4.6.3 Objectives (OPIM 1.1.1)

5. Performance

5.1 Management

5.1.1 System

133. What are the systems used to measure organisauond perfoxmance?

134. How are these gcitems interrelafed?

1 35. How are tbese relationships manageci and improved?

5.1.2 Key Performance Outcornes

136. What is the system applied to mevure your key perfoxmance indicators

reiating to organisational effectiveness (meeting key objectives)?

137. How do you continwUy improve this measurernent system?

5.1.3 Key Perfomiuice Indicators

138. What is the syscem you use to measure your key resource performance

indicat ors?

139. How do you continuah/ improve rhis measurernent system?

140. What is the system you use ro meanire your key process performance

indicators?

14 1. How do you continuaUy "pmve this measurnent system?

5.2 Moni to~g and Controllhg

5.2.1 Audits

142. What is the process for the performance of interna1 audits againn the

requirements of the seleaed management standards indudiag ISO

9001:2W0 OS0 9001:1994) and/or ISO 14001:1996?

143. How do you continuaMy improve the effectiveness, efficiency and

consistency of this process?

144. What is the process for the perfo~nance of exremal audits against the

reS;rements of the selected management standarb induding ISO

9001:Zûûû O S 0 900 1: 19%) uid/or I S O 14001: l996?

145. How do you cont indy @mve the effectiveness, efficiency and

consistency of this process?

5 -2 -2 Self-Assessments

146. What is the process for perfomiing regular self-assessments against the

organisation Perfomunce impmvement Model?

147. How do you cont indy b p o v e the effecriveness, efiiency and

consistency of rhis process?

5.2-3 Performance Measurement

How do you design or select the indicaton used to masure and analyse

the performance in your department?

How do you idenufy and manage the interrelationships bmeen rhese

measures?

How do you decide what key comparative data and information ro use?

How do you support the organisational performance review and planning

with analysis of data and information?

How do you link the organisationai level results to R&D depamnend

and project tearn operations to enable effective support for decision-

rnakllig?

How do you conr indy irnprove the effectiveness, efficiency and

consistency of the performance measurement system?

5.3 Improvement

5.3.1 Coftective and Prwentive Actions

154. How do you monitor, iden*, &, and remove the exisUng causes of

nonconformities in inputs, processes, outputs, objectives, resources uid results?

155. How do you indude plans for and renilu of corrective and preventative

action in the business plan?

156. How do you evaiuate the effectiveness of these actions for rheir

continuw improvement?

5.3.2 K a h n and Quantum Leaps

What û the process for initiating and c e out quautun leap projects

that lead to the revision and irnprovement of existing pmcesses or the

irnplementation of new processes?

What is the process for initiating and ovrying out Kaizen (small

incremend impmvement efforts) for the continual improvement of

features in inputs, processes, ourpufs, objectives, resources, and results?

How do you wduate and improve the effectiveness of these

improvement processes?

APPENDIX B - Mded OPTM for R&D

ORGAMSAnONAL PERFORMANCE IMPROVEMENT MODEL CRITEIUA FOR-

1. Objectives

1.2 Management

1.21 System

Describe haw the R&D funaion establishes, "plements, and maintains its quality procedures and plans to meet the qwiity policies and objectives set by the organisation.

OS@, 4.land 6.1; DP, 2:l-25) Describe how the R&D function continuously improves its &ty pians and the effectiveness and effiaency of its qu?liry procedum. Desuibe how qu&y is defineci for the R&D fuaction (Kumar ad Boyle, 2001)

D&be how the cornmitment of R&D management to achieve policies and objectives set by the organisation is ensureci.

(MBNQA, 1.1% EQA, 1 b-d; DP, 1:6 and 10) Describe how leaders of the R&D function conmbute to the development, implementation and continuous improvement of the organisational management system, as well as their Uivolvement with di sdteholders.

1.2 Stategy

1.2.1 Dwelopment

hpmmts (ISOI, 5.3 and 5.4.1)

Describe the syscern for translating organisational quahty policies and objectives into plans and objectives for the R&D funaion.

(IS04,5.3; MBNQA, 2.1; EQA, EQA, la and 2c; DP, 1: 1-1:3) Describe the nntegic planning process applied to irnprove R&D performance and set appropriate suaregies and poliaes.

Rqmmm (iS01,5.4.2, 5.5 and 5.6)

Describe the processes of R&D management p h h g and review, as well as the associateci responsibilities and atithorities,

qDportrmrtJs (IS04,5.4-5.6; MBNQA, 1.1 b and 2.2; EQA, 2d and 2e; DP, 1:4 and 15) Describe how m e g k p h uc deplaycd to the R&D function. Describe how the nntegic pians, mdaed to the R&D function, ue revicwed and improved upon to ensure continuing ;ilignment wi3i organisational pians.

Describe how st;ikeholders are iden~ed, and how th& needs and requiremeau [iudiUiinp legal, regultory, MWP-tal, producr and protes standuds, as well as openriod) are identiûed, deployeâ, communicated and evaiuated.

@xmmtm ( iS04 ,52 ; MBNQ& 1 2 uid 3.1; EQA, 21 and 2b; DP, 2:6 and 4:l- 4:s) Descrik how the process of identification, deplayment, communication and evaiuation of stakeholders and th& a& and requirements is continualiy irnproved

1.2.1 Customers

1.2.2 People

1.2.3 Orpwat iod Leaders

1.2.4 Suppbers and Puniers

2.1 Management

h p t w m ~ ~ (IS01,6.1)

D e s d e how the resources listed below in d o n s 2.2 to 2.8 and 3.4 of OPIM are managd Describe the intdtionships among these resources and how they are used to support processes listed in section 3 of OPIM.

(IS04,6.1) Describe how the acquisition and deployment of resources is optimiseci to ensure effectiveness, efficienv and consistency of system operation and achievement of set objectives. JJiustrate how the use of the resources is continuously improved Describe how the commitment of long-tenn resources to R&D effom is ensureci. ('da & G o h , 1997)

2.2 Finances OS04,6.8; EQA, 4b)

Desuibe how finulciai resources are planned, acquired, deployed and convolled to achieve R&D objecrives.

QF== Describe how the efficiency and effectiveness of the use of fmancial resources is impmved.

2.3 Information

Describe how idonnation resources are pianneci, acquired, deployd and conmiled to adiiwe RStD objectives.

(IS04,4.2 and 6.5; EQA, 4e; DP, 3) Describe how the efficiency and effecfiveness of the use of information resources is

;mprovCd

2.4 Infrvvucturc

OS01,6.3)

Dexribe h m resources are pianned, acq;ed, deployed and contrdled to d e v e R&D objectives.

(IS04,6.3; EQA, 4 4 Describe how the efficiency and effecciveness of the use of ' ' buucture resources is

2.5 People

2.5.1

2.5.2

2.5.3

Describe how characteristics and skiils needed by potend employees are identifieci to ensure h t the current and future ne& of the R&D function will be met. Describe h m work and job functions designcd to suppon CO-opedon, collabontion, i.adividu?l hithive and innovation.

Qpfwztk (MBNQA, 5.1; EQA 3% 3 4 3e) Desuilx how the human resourre system is conUnuoUS, improved.

Campetenaes, Training and Development

~ ( I S O 1 , 6 2 ) Describe the systems used to ensure continuing competency of R&D perso~eL

C&nmumm (IS04,6.2.2; MBNQA, 5.t; EQA, 3b) Desuibe how k q cornpetencies are developed and sustained to achieve set levels of R&D perfomiuice. Describe how the efficiency and effectiveness of these systems are continuously improved

Satisfaction, Involvement and Empowerment

&panmm (ISOI, 6.2) Describe the system used to reward and recognise R&D personnel.

9lpnrrrniher @ S a , 6.2.1; MBNQA, 5.3; EQA, 3c) Iliunrate how a work environment that conmbutes to employee satisfaction, involvement and empowerment in O& to improve effecfiveness and effiaency is developed, maintaineci and enhancd

Describe how technology is used to achieve R&D objectives. Describe how business and technology strategies are integrated, communiated, and understood by aU depuunents (Kumar and Bayle, 2001)

9eipomotaier Describe how the efficiency and effectivcnss of the use of technology is improved

Describe how av;Utbility of nîtunl resourœs in cnnved in order to prevent adverse impacts on the a w i m n f n ~ t and negatbe effecu on the R&D perfonnuice.

q . p B d ? 5 Describe h m the use of n a 4 resources is optùaised to prevent negative environmentai impacts and improve organisationai performance.

Describe how work environment is to achiwe R&D objectives and provide a healthy and d e workpiace.

(IS04,6.4; EQA, 4c) Desaibe how a positive influence of the work environment on human resources is maintaineci and impmved.

3. Processes

Rqamm# @SOL, 7; EQA, Sa and 5d)

Describe how R8rD pmcesses listeci Mow confom to the criteri? stated in ISO 9001: 2000 standard, section 7.

(IS04,7; EQA, Sb; DP, 6 and 7) Desuibe how the effectiveness, continuoudy improved, referxing to 7.

efficiency and consistency of processes are the guidelines mted in ISO 9004: 2000, section

3.1 Management (ISO1,7.1; IS04,7.1)

Describe how R&D processes are managed

3.2 Customer relationships (EQA, Sc and Se; MBNQA, 3.2% ISO1,7.2.3; IS04,7.2)

Describe how processes rhat affect the R8rD function's relauonship 6 t h customers

are desiped, implementd, xmintained, and impmved

3.2.1 Knowledge (iSO1,7.2.l and 7.2.2; MBNQA, 3. la)

Describe how prucesses for idendymg internai and extemai customers and their

requirements are designed, implemented, maintained, and improved.

3.2.2 Satisfaction (MBNQA, 3.2b)

Describe how processes for determining internai and externai customer satisfaction

are designcd, implemented, maintained, and irnpmved

3.3 Design ancilor development 3.3.1 Pl?nning (IS01,7.3.1; I n , 7.3.1)

Describe how design and developrnent processes are p h e d , implernented, maintaineci and improved to ensure t&at thev resmnd to the neeb and exuectations

Inputs (iS01,7.32; ISW7.3.2) Describe how the R&D function ensurcs th t inputs to design and deveiopment are accurate and compled. Describe h m this process is conrinuaily improvd

Activities (MBNQA, 6.11) Describe how design and development processes are implemented, m;iintained and impmved

Outputs (ISOI, 7.3.3; IS04,7.3.2) Desaibe how ourput requinmenu are validateci against input requirements and acceptana criteria to achieve interesteci pvty satisfaction. Describe how this process is conMii?Uy impfoved

Review (Isol, 7.3.4; I n , 7.3.3) Desaibe how the design review is incorporateci h o the design and deveiopment processes. Describe how this process is continuousfv improved.

Verification (ISOL7.3.5; IS04,7.3.3) Desaibe how the design verification is uicorpod into the design and Qvelopment processes. Describe h m this process is continuody improved

Validation (IS01,7.3.6; IS04,7.3.3) Describe how the design vaii&tion is i n c o r p o d into the design and dweiopment processes. Describe how this process is c o n ~ u o u d y improved.

Changes (IS01,7.3.7; IS04,7.3.3) Describe how changing customer/market requirements are incorponteci into R&D design and developrnent processes

3.4 Purchasing (EQA, 4a) 3.4.1 Planning(IS01,7.4.1;IS04,7.4.1;MBNQA;6.3aland6.3a2)

Descnbe how resources, which m o t be o b d intemaiiy are idenUfied in order to define a purchasing strategy. Describe how capable suppiiers are identified and seiected so as to ensure consistent quaiity and dependable nippky. Describe how are these processes continua& improved

Describe how processes ro ensure that specifications for purchad producc meet the requirements of the organisation and infefened pmies are defined, implemented, maintaineci and improved

3.5 Support

Describe how requirements for key support processes are determined. Desuibe how these support processes are pknned, &signe4 and hproved

3.5.2 Conml @Sol, 7.5.3-7.6; IS04,7.5.2-7.6; MBNQA, 6.214-5)

Describe how key support processes are implemented, maintained and impfoved to ensure meeting kqr performance reguirrments.

4. Results

List the perception measUres and performance indica~ors relating to the following four componcnts of customer r d t s . S m c thci, current levels and uends.

QP-- List the targetcd leveis of perception mevurcs and performance indicators.

4.1.1 Reputaion (EQA, 61 1 and 6b 1; MBNQA, 7. Id)

4.1.2 Luydty (EQA, 6a4 and 6b4; MBNQA, 7.1a2)

4.1.3 Outpufs (ISO 1, 8.2.4 and 8.3; IS04,8.2.3 and 8.3; EQA, 612 and 6b2; MBNQA, 7.1a3)

4.1.4 Interfaces (EQA, 613 and 6b3; MBNQA, 7. lal)

4.2 People (MBNQA, 7.3; IS04 , 8.2.41)

List the perception mevures and performance indiators &ring to the following four componenu of people results. State their current levels and m d s .

List the targeted leveis of perception measures and perfonnuice indiaton.

4.2. l Motivation (EQA, 711 and 7b2)

4.2.2 Satisfaction (EQA, 7d and 7b3) 4.2.3 Achievements (EQA, 7bl)

4.2.4 Services (EQA, 7b4)

4.3 Suppliers and Panners (IS04,82.4c)

List the perception measures and performance indicaton reiating to the foiiowing cwo components of suppliers and panners relationships. State their current levels and trends.

9.lpnaprras Lin the targeted leveis of perception mesures and p e r f o m c e indiators.

4.3.1 Inputs(IS01,8.2.4;IS04,8.2.3)

4.3.2 Interfaces (MNBQA, 7.4)

4.4 Society ( I D , 8.2.4d)

List the perception measures a d p e r f o m c e indiaton reiiating to the foiiowing five components of the rektionship to Society. State th& current leveIs and trends. - List the tvg~ed levels of perception mesures and performance indiCuos.

4.4.1 Citizenship (MBNQA, 7.512; EQA, 811)

4.4.2 Community(EQA.812)

4.4.3 Enknment (EQ& 814)

4.4.4 M t h and S a f q (EQA, 81))

4.4.5 htcrfaces(EQ&8b)

4.5 System

4.5.1 Processes (EQA, 9bl; MENQA, 7.511, IS04, 8.2.2) - List the key perfomance indiators rekang ro processes stated in section 3 of OPM Sute their -t leveis and vends.

Qv-leJ List the tvgeted lweis of key process perfommnce indicators.

List the key performance indiators relating to r e s o m s stated in section 2 of OPIM. State their current levels and trends.

9biantrorias List the targeted lm& of key resource performance indiutors.

4.5.3 Obj&es(EQA, 91) - List the key performance indiutors reiating to R&D effectiveness. State their current lwels and trends. - List the targeted lwels of key R&D performance indicators.

5. Performance

5.4 Management

Describe the system applied to measure RSrD performance. Iiiustrate the interrelationships among the intemal audit, self-assessrnent and result-based V e m s for performance measurrment.

Describe how the intedxionships of in terd audits, self-assessments and result rnevurements are manageci and improved.

Key Performance Outcornes (EQA, 91)

Describe the system applied to mcasure the key performance outcornes identifieci in section 4.6.3 of OPM.

Describe how the measmmmt synmi for key performance outcornes is conUnually impmved.

Describe the system applied to masure the key perfonnuice outcornes identifid in sections 4.6.1 and 4.62 of OPIU

Describe how the me;isuremcnt sysrem for key pcrfomance indiators is c o n ~ w i l y improved.

5.5 Monitoting and Controiiing

5.5.1 Audits

Describe the V e m applied to perforrn internai and extemai auclits against the requirements of the selected management stui-, including ISO 9001: 2000 and/or ISO 14001: 1996.

Qpzwntm (IS04,8.2.1.3) Describe how the effectiveness, effiaency and consistency of the audit system are cont indy improved.

5.5.2 Self-Assessments ( I D , 8.2.1.5)

Describe the systern applied to perfonn regular self-assessments against the Oqpmauonal Performance Impmvement Model.

0

Descnbe how the effectiveness, efficiency and consistency of the self-assessment system are cont indy impmved

5.5.3 Performance Measutment (iS04,8.1,8.2.1,8.4)

(MBNQA, 4. la 1 and 4.2)

Describe the performance measurement synem used to cokct, monitor, an*, synthesise anâ act upon perceptions (extemal) and performance indicators 'nternai) relating to: - customers

- suppliers and pvtners

- soaety

Describe how the effecciveness, efficicncy and consistency of the performance measunment system are continuaüy improved.

5.6 hpmvcmait(ISO1,8.5.1;IS04,8.5.1;DP,9)

5.6.1 Corrective and Prevcntive Actions

R t q m m m (Bol, 8.52 and 8.5.3)

Describe the systmi applied to monitor, idene, analyse, and remove the exisung and

potend causes of nonconformiries in inputs, pfocesses, outputs, objectives, resources

and d f s .

(iS04,8.52 and 8.5.3)

Desuibe how the c o d e and preventive actions are used in the business pian and how

th& effectiveness is continuaiiy imprwed.

5.6.2 Kaizen and Quantum Ir?ps (IS04,8.5.4 and Annex B) - Describe the systern applied to continuousiy improve the features in inputs, processes,

outputs, objectives, resources and resuits.

Desuibe how the effecUveness of the conUnd impmement systern is incre;ised.

REFERENCE ClUTERK DP Demùig Application Prize (1996), Dsnirg F%E Gd Orsnoiv C k p z i s , Japanese

Union of Saentists and Engineers EQA European Qu;ility A d (1999), EFQM ExadlmP M d , European Foundation for

Quatty Myiagement ISO 1 1!50 FDIS 9001 (2ûûû), Q&y h h p a m S y a m - Rqbmmzs, Intemational

Organkation for Standudiz;ition, Geneva 1s04 rso mrs 9004 (20001, - G - ~ W -,

International O ~ u o n for Standardization, Genwa MBNQA Maicolm Baidrige Natiod Quaiity Award (2W0), Crieiiafi A+nmx? E x d h ~ , Mn,

Gaithers burg

Kieiia, M. L, G o k , D. Y., (1997), 'Td Quaiity Management in an R8cD Environment", inrenraakll J d o f O p e r m i c m & R d d m Mmqpmt, Vol 17, No. 2, pp. 184-198.

Kumar, V., Boyle, T., (2001), 'A quality management implementation framework for manufactuing-based

R&D envkonrnents", I M JuanafofQuhy & R$raEilrtyMhqpmt, Vol. 18, No. 3,2001, pp. 336-

359.

APPENDIX C - Trillium Resuits Trilliwn Resdts

Scoring for OPIM

98.77 8 1.75

68.99 87.49

lûû 85.19 39.38 84.38 84.83

94.44

78.73 32.8 68.98 36.8 60.3 49.48 79.62 53.77

O 44.95 82.05

8 1.12

Management Su;itegy Development and Deplaymcllt Stakeholder Objectives Management of Rwources

RÊsources People Technology Nature WorkEnWonment Management of Processes customeir - Relationships Design Processes Purchashg Processes &uon Processes Suppon Processes Customer Redu

'Sheholder R e s u l c People Results Supplier and Panner Renilts SociqResults System Results Performance Management Perforrmnce Monitoring and Conuoltng

75.46 61.72

56.41 76.17

97.7 87.43 3 1-01 89.75 83.8

79.33

89.2 88.49 87.1 72.36 37.98 30.47 100 50.06

O 67.2 1 72.86

59.52

87.58 100

97.87 86.09

85.75 81.41 59.37 53.86 87.37

99.5

89.84 61.66 51.83 30.76 68.29 58.45 69.6 1 71.4

O 42.06 88.67

69.44

100 81.47

50.5 90.02

91.46 74.14 19.13 94.47 77.82 --

48.67

52.52 O

46.26 48.21 51.11 54.63 61.79 28.82

37.65 63.03 68.46

89.37

90.55 72.92

71.84 86.72

89.93 91.7 66.39 80.06 92.97

78.02

100 80.56 89.39 12.47 70.03 74.99 93.49 69.65

O 35.34 88.91

91.1

68.74 67.77

68.86 58.26

100 38.39

10.55 71.03

71.42 55.63 48.32 60.13 48.72

-- ---- 100

37.9 O

51.85 50.5 1 62.94 49.16 48.41 51.41

37.17 43.72 53.31

54.56

77.2 1 72.19 49.43 79.06 79.56

63.42

95-36 59.66 75.78 55.09 62.8 28.53 72.49 72.98

47.53 46.5 1 92.56

89.05

APPENDLX D - Cornparison of Perfknance Muiagemmt Model G i t m a

Catcgory 1 ISO9001:2OOO 1 MBNQA 1 EQA 1 Dcming Prize

h k i n g for evidence of management coIIMifment to the gwlity management system and that top managanent eaSureS

The needs and expectwons of interesteci pmies are met The quaiity poiicy is d e h d w objectives and the qiulity management system (qms) is pianned Responsibility and authority is defmed The review of the qms

senior leaders gui& the or&;inis;ition and revicw or&;inisîtional perfolmance

Examines how leaders develop and facihate the achievement of mission and vision, develop values, and ensure th the o ~ o d management systern is developed and maintained

Addresses undersraading, knowledge and enthuskm of leaders, hm- effectively leaders impiement TQM, and senior management's role in the management v-

1 1 Examines how the 1 Examine how the 1 Examines how the

Generai

Generai

Looking for evidence of a

p o h and the planning of

q d t y objectives the quaiity management m e m

Suaregy and policy Examines the

company addresses its responsibilities to the public and how it practices g d cihnship

company's strategy deveiopment and deplayment processes

Examines how the compvry unplements its mission and vision

compaxxy's leaders are involved with representatives of

society

concerned with quaiity assurance miVities and q u ; i l i ~ ~ ~

company fulGIs its s d mponsibilities

&den& that the ~ q p k t i o n determines and provides the resourccs nceded to implement and &tain the qu;ility II1;Ul?gement system and to enhance customer satisfaction

Not speaficaliy induded

Lx>okmg for evidence t h t the compuiy &termines, provides and maintains the illhsmcture needed to achieve

Not specifically induded

Not specificaiiy included

Na specificaiiy included

Examines how the compv~y ui;iEyses i n f o r d o n on qu;ility, customen, opemionai and finuicial performance to support decision

Not included

Not included

Exunuies h m the company pians and manages its intan?l resources in order to suppon its policy and strategy and the effective operation of its processes

Examines how the compauy 's information sysrem is m a q e d to ensuredata validity, integrity and security

Exvnines how the company pians and manages its buildings, equipment and d

Examines how effectively the Company manages its miued resources and supplies so that material inventories are

m e is mkiimised Examines how the Eompany evduates and exploits techno1og)r Examines how the ~ompany 's manages its Il~lilllcialrrSOurceS

Examines how e x t e d information is collecteci and =bed to support decision making

- - .-

Not induded

Not induded

Not induded

Catcgory 1 ISO9601:ZOOO 1 MBNQA 1 EQA ] Dcming f i e

Eduution and Training

Employee well- being and

satisfaction

Looking for evidence that the company ensures the necessary cornpetence for personnel performing woiir ;iffectingproduct qu;iliv

Not induded

uaLlreSOurtem;inîge Examines how the

paformuice Examines how the Company secks input h m ernployecs and their supervisors on educaüon and . . trvnrngnceds Examines h m the Company support business objectives

Examines how the organisation maintains a work enWoument that contributes to employee wd- king and satisfaction

ent Examines how the compaq -es, develops and reieves the full potend of its people Examines how the company promotes the involvement and empowerment of its people

Exvnincs how people's kncswiedge and cornpetencies are idenaed, deveioped and susuineci

Examines how people are m;uded, recognised and cared for and how the company promotes the dialogue between i d and the w o ~ l e

Company appiies Qu;ility concepts into its human resource docation and management

Not induded

Examines h m the companyf edua t iod poliaes and plans consider QC management and how it reviews the effectiveness of the educationai Prognms, and whu eciucatiod

compuiies

Not induded

Cjtegory 1 ISO900t:2OOO 1 MBNQA 1 EQA 1 DcmingPrizc Processclualiw

evidence that the company pians and develops the processes a& for pduct

h o m e r management

Customer satisfaction

Not inciuded

cwomer requirements are deterrnined and fulfied with the aim of enhancing customer satisfaction b evidencë h t the company monitors inforniluon

l perception of fuifiient of

Examines how the l- ComPvIy m=%= its key product and service design, delivery, and suPpo*

Resuits Examines the compq ' s performance and Unprovernent in the folluwing key ue?s

Gmomers F i n u i d u i d market Humui resoufces organisational e ffecriveness

r management and s;i Examines how the compvry determines requiremenw expectation, and preferences of cunomers and markets

Examines the compuiy's perfonnuice and improvement in customcr sarisfaction

Examines how the company designs, -a, =d improves its processes to support its policy and strategy

Examines what the company is achieving in &on to:

Exterd customers People Society Key Performance R 4 t s

Great emphasis on the quaiity assurance of products and services

Prim+ concerned with quality assurance activities and WtyfeSults

Not included

Examines what the zomp;uiy is ichieving in relation to its exterd customers

Not inciuded

Not inducfed

Cpcgory 1 ISO9001:2OOO 1 MBNQA 1 EQA 1 DcmingPrizc SuppIiers/pumtn mamgamc a d performvlce

Genenl

Lcdnng for &ce rhîr the company cv;ilu?us and selecu nippliers based on th& ab&y to W P P ~ P& in accorcfance with the company's requiremcnfs

Focused on excension of

management CO qu?lity

suppliers of the fum

Ervnincr how the corn- manages its supplia and me ring processes

Examines how the corn- p h and manages its e x t a d pvtnerships to support.itspolicy and stnregy and the effective operation of its proccsses

APPENDIX E - Apprt Score

Attributes Sound:

Approach h u a clear rationaie There are wcii àefined and àeveloped processes Approach focuses on nakeholder needs

Intcgnttd: Approach suppons policy and nrategy Approach is linked IO othcr approaches as appmpriate

Approach is implemented

Syncrnatic: Approach is deplyed in a nniaured way

T d Trends:

T m & are positive and/or there is sunained g d p c r f o m c e

Tugcts: Targcts arc achievcd

a Targets ue appropriate

ich, Deploymi 0%

No evidence or anecdotal

No eviâence or anecdotai

No evidence or anecdod

No evidence or anecdotal

No redts or mecdow]

information

No r e d s or anecdotai

information

it and Results Ada~ted lrom the EQA 1 75%

Some evidence

Some evidencc

Somc evidencc

Some cvidence

Positive trenb and/or satisfactory

pcrfomiiuicc on somc muhs

Favourable and appropriate in some

areas

Positive trends and/or sustaincâ food pcrformzncc o n m a n y d s

over at leas ihrcc Y'=

FavowbIe and appropriate in

mvry-

dcar evidence

Strongly positive trends md/or

sustaincd excellent pcrfomunce on

mon rcsutts over at lm 3 y m

Favounble and appropriate in mon

arcas

Gnnprehcnsive cvidencc

St rongiy positive vmds and/or

sustaincd excellent performance in aü vcls over at leas

5 Y-

Excellent and appmpriate in

mon