process quality in ons rachel skentelbery, rachael viles & sarah green methodology directorate,...
TRANSCRIPT
![Page 1: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/1.jpg)
Process Quality in ONS
Rachel Skentelbery, Rachael Viles & Sarah Green
Methodology Directorate,
Office for National Statistics
![Page 2: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/2.jpg)
Aim of Presentation
This presentation aims to give the audience an appreciation of the importance of process
quality within the ONS and how we are looking to implement this within the
organisation
![Page 3: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/3.jpg)
Overview
• Background to Process Quality at ONS
• ONS Process Quality Initiatives
- ONS Process Quality Handbook
- Process Quality Measures
• Implementation
![Page 4: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/4.jpg)
Process Quality at ONS
• Leadership expert group (LEG) final report (Eurostat (2002)) states that:
‘in theory, good product quality can be achieved through evaluations and rework’
• This is costly and time consuming and so instead, it is believed that:
‘product quality will follow from improvements in process quality’
![Page 5: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/5.jpg)
Impact of Process Quality at ONS
Process Quality
Reputation
Customer Perceived
QualityProduct Quality
Business Quality
![Page 6: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/6.jpg)
Definitions
• Process:- A series of actions or steps towards achieving a
particular end
• Process Quality:- An assessment of how far each steps meets
defined criteria
• Process variables:- Factors that can vary with each repetition of the
process
![Page 7: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/7.jpg)
ONS Process Quality Initiatives
• Process Quality Project in ONS
1. Summary of the European Handbook on Improving Quality by Analysis of Process Variables
2. A defined set of process quality measures
![Page 8: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/8.jpg)
ONS Process Quality Handbook
• Aims:
- To assist the implementation of Continuous Quality Improvement techniques into ONS
- To provide guidance on applying the technique for each step in the Statistical Value Chain
![Page 9: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/9.jpg)
Overview of Process Quality Handbook
• Key Process Variables are factors that:-can vary with each repetition of the process-have a large effect on key characteristics
• Monitoring the process variables is split into 3 stages within the handbook:
Identify keyprocess variables
Measure keyprocess variables
Analyse keyprocess variables
![Page 10: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/10.jpg)
Steps for Continuous Quality Improvement
1. Identify key characteristics of the processes that will to make the output fit user requirements
2. Develop a process map3. Determine key process variables that will impact on
the key characteristics4. Evaluate how to measure the key process variables5. Monitor stability of the key process variables6. Determine if the process produces an output that over
time continuously meets user requirements
• Establish a system for the continuous monitoring of the process
![Page 11: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/11.jpg)
3. Determine Key Process Variables
• Not efficient or effective to measure all process variables
• Identify those key process variables
• Useful Tools:- Cause & Effect diagrams- Pareto Charts
![Page 12: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/12.jpg)
Pareto Chart of Contribution to Total Item Non-response by QuestionPareto chart of contribution to total item
non-response by question
0%
10%
20%
30%
40%
50%
60%
70%
80%
11 16 27 25 All other
Question
Con
trib
utio
n
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Cum
ulat
ive
%
33%
Example from ONS E-Commerce Survey
![Page 13: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/13.jpg)
4. Measure Key Process Variables
• How accurately key process variables can be measured?
• Measurement should require minimum effort
• Key points: - what can be measured that is useful to understanding
and monitoring the system - can the measures be used to assess changes in the
quality- what does success look like- can the measures lead to potential for improvement
![Page 14: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/14.jpg)
Example of Measuring Key Process Variables
Example from ONS UK Census: Occupation Coding
• Key process variable:- accuracy of coding (consistency rate) for each
Estimation Area (EA)
• 2% sample of codes was verified, for each EA, independently by coders
• Sample data were used to estimate process variables:
number_correctConsistency_rate=
number_checked
![Page 15: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/15.jpg)
5. Monitor Stability of Key Process Variables
• Need to analyse the key process variables
• Process stability:- a state where the process variation consists
entirely of random components - i.e. the variation is not systematic
• Useful Tool:- Control charts
![Page 16: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/16.jpg)
Control Chart of Coding Consistency Rate for Occupation Question
0 .88
0 .89
0 .90
0 .91
0 .92
0 .93
0 .94
0 .95
0 .96
0 .97
0 .98
0 .99
01
-Ju
l-2
00
1
01
-Au
g-2
00
1
01
-Se
p-2
00
1
02
-Oct
-20
01
02
-No
v-2
00
1
03
-De
c-2
00
1
03
-Ja
n-2
00
2
03
-Fe
b-2
00
2
06
-Ma
r-2
00
2
06
-Ap
r-2
00
2
S c anning E nd D ate
Ra
te
C o ns is te nc y
L o we r b o und
Up p e r b o und
C e ntre L ine
Example from ONS UK Census
![Page 17: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/17.jpg)
The way forward
• Processes fall into two categories:- those amenable to the whole process quality
approach- those where it is difficult to identify measurable
process variables
• Identify those areas where this approach can be used
• Support output areas implementation
![Page 18: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/18.jpg)
Process Quality Measures
• Enable producers to measure and monitor quality of processes over time
• Responsive to problems, highlight areas for improvement
• Quality assurance for users
• Standardised approach, comparability
![Page 19: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/19.jpg)
Process Quality Measures in ONS
• Currently developing a comprehensive list of process quality measures that will link to:- Each step of Statistical Value Chain- The Eurostat Process quality attributes:
- Efficiency - Effectiveness- Robustness- Flexibility- Transparency- Integration
• Will look to derive a subset of Key Process Quality Measures
![Page 20: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/20.jpg)
Implementation Across ONS
• Pilot use of Handbook
• Bespoke support to output areas from ONS Quality Centre
• E-learning course for process quality tools
• Expert led training sessions
• Quality strategy targets
![Page 21: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/21.jpg)
Review
• ONS specific Process Quality Handbook:- Developed- To be rolled out across ONS after piloting
• Process Quality Measures:- Key Quality Measures
• Blended approach to implementation- E-learning- Expert led training- Support and help to areas- Quality strategy targets
![Page 22: Process Quality in ONS Rachel Skentelbery, Rachael Viles & Sarah Green Methodology Directorate, Office for National Statistics](https://reader036.vdocuments.mx/reader036/viewer/2022070411/56649f415503460f94c60b2c/html5/thumbnails/22.jpg)
Further Reference Points
• Eurostat Handbook on Improving Quality by Analysis of Process Variables
Contact me directly at in ONS Quality Centre: