corrective actions trend analysis (polish experience 2005...
TRANSCRIPT
1
Corrective actions
(Polish experience
2005 – 2012)
Tomasz Cencek
Miroslaw Rozycki
Department of Parasitology and Invasive Diseases
Trichinellosis in Poland
Trichinelosis is a
notified disease in
Poland with average
incidence in the last 11
years of 79 cases per
year (vary from 4 to
292 cases/year). With
more than 1000 cases
for the last 11 years.
Human Trichinellosis 1999-2011
0
50
100
150
200
250
300
350
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
No
cases
Pigs trichinellosis in Poland
Trichinella positive pigs 2003-2012
0
10
20
30
40
50
60
70
80
90
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Trichinella positive wild boar
in Poland
Trichinella positive wild boar 2000 - 2011
0
100
200
300
400
500
600
700
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Trend analysis of human trichinellosis, farmed pigs
and wild boars in Poland in 2000-2011
Trend analysis
0
100
200
300
400
500
600
700
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
Year
No
of
cases
Wild boars
Pigs
Humans
Wykł. (Wild boars)
Wykł. (Pigs)
Wykł. (Humans)
Number of laboratories
• 2005 – less than 300 carried out examination with digestive method (in first PT participated 60 labs.)
• 2006 – over 220 new laboratories introduced the digestive method (PT – organised for 25 labs.)
• 2007 - 592 digestive/498 compresor (1090)
• 2007 PT test 564/455 passed/ 109 failed
• 2008 - over 1100 declareed to use digestive
• 2008 PT 790 / 675 passed/115 failed
• 2009 PT 822 / 753 passed/ 69 failed
• 2010 – 700 / 666 passed / 34 failed
• 2011 – 754 / 716 passed/ 38 failed
• 2012 old methodology of PT - 153/149 passed/ 4 failed
2
PT results 2007 – 2012.06.31
Unsatisfactory results in %
0,0
5,0
10,0
15,0
20,0
25,0
2007 2008 2009 2010 2011 2012 old
methodology
%
New Regulation 2075/2005 and results of
PT 2006 shows necesity to accelerate the
work on the implementation of new method
• Laboratories that have obtained unsatisfactory results were obliged to take corrective action.
• According to Polish low the Local Veterinary (LVO) Officer is responsible for examination of pigs in the county.
• He is also responsible for the laboratory staff, laboratory facilities and equipment.
• The first question raised after PT was:
• Do the LVO know the metod and its requirements sufficiently?
Meeting with LVO and Main
Veterinary Officer 2006
• The meeting showed necessity to implement training courses for veterinarians LVO.
• Action I:
– 16 courses were organised by NRL for LVO in order to present the method requirements, the need to adapt and implement digestive method
– To introduce properly requiraments set in legislation and standards.
2006 – the result of PT shows:
• The need to undertake Corrective Action on the
System.
• Corrective Action II:
The diagnosis of the real situation in laboratories
– Human factor.
– Competence of personnel.
– Technical requiraments:
– Accomodation and environmental condition.
– Equipment.
Solution: Checklist • The checklist described step by step:
• Human factor: – Personel education.
– Skills.
– Training.
– Training of staff - date of certificates?
– Cascade training?
3
The checklist described:
• Apparatus and reagents (knife, scissors balance with an accuracy d = 0.1 g)
• Type of trichinoskop and the date of last technical verification.
• The number of sets for digestion.
• The number of slaughtered animals.
• The strengh of pepsin used for examination.
• Fridge etc…
Analysis of checklist shows:
• Lack of understanding of the management for
changes and high requirements.
• Cascade trained laboratory personel:
The blind leading the blind.
• In some cases personel was cascade trained by
„skilled technicians carrying out examination for
many years” this lead to duplication of routine
errors .
• Sometimes modifications were realy terrifying.
Laboratory facilities
• Inadequate laboratory facilities for examination:
– too small,
– without proper ventilation system,
– ucontroled temperature and humidity,
– insuitable sinks.
• Generally laboratories were significantly underfunded.
• Incomplete equipment 60% of labs.
Control of laboratories in place
Lets start with sampling:
• Trays marked into 50 squares, each of
which can hold samples.
• Samples should be large enough.
• Samples should be taken from a pillar of
the diaphragm at the transition to the
sinewy part.
• Pictures shown on the next slides were
taken on the begining of 2007.
Visiting no coments:
5
Equipment 5.5
• ISO 17025 effectively requires a complete history of each piece of equipment.
• This should start with details of the checks and calibrations carried out before the equipment is placed in service and continue with a detailed record of all calibrations, repairs, routine maintenance and performance checks.
• In this context, ‘equipment’ should be understood as any items which may affect the validity of measurements or calibrations, including reference standards of measurement, such as standard weights and reference thermometers.
6
The are just 2
answers:
1) Training
2) Equipment
maintaining
How many
corrective action
should be
undertaken in this
lab.?
Outbreak of trichinellosis in north-
western Poland, June 2007
catalised changes
• Laboratories received significant financial
support to adapt to the requirements of the
new method.
• NRL has received financial support for
training and visits in laboratories.
• LVO-s were pushed to implement the
digestion method.
Result of 2007 PT as a tool
to recognise situation but not for single
laboratory
• Corrective action in Malopolska region: – Training (in few cases retraining).
– Laboratories were retested and visited.
Ranking PT results 2007
0
5
10
15
20
25
Lubuskie
Warm
ińsko-m
azurskie
Podlaskie
Wielkopolskie
Podkarpack
ie
Dolnośląskie
Pomors
kie
Śląskie
Świętokrzy
skie
Kujawsko -
pomors
kie
Łódzkie
Mazowieckie
Zachodniopomors
kie
Lubelskie
Opolskie
Małopolsk
ie
Corrective actions 2008
• In order to take care on laboratories performing meat inspection and Trichinella testing in 2008 were established 16 regional laboratories.
• They were responsible for introduction of the Quality Assurance System in field laboratories.
• To achieve that goal new 16 persons experts in QAS were hired.
• With personality strong enough to handle veterinarians.
PT - 2008 • 1 - NRL
• 16 - voivodship
laboratories (regional)
• Over 300 of them were at
that time under the
process of accreditation
(after the I audit of
accreditation body)
• PT samples levels were:
– 0
– 8-10,
– 15-18,
– 20-25.
• 826 – declared to
participate in PT
– 22 didn’t sent the
results of
examination
– 14 sent
incomplete results
– 115 failed
Corrective action: Shock therapy By the decision of MVO for the laboratory received unsatisfactory result,
its activities was stopped and personel was fired.
Results of PT and corrective
actions • 2009 PT 822 / 753 passed/ 69 failed
• Four samples - levels: 0, 5-7, 8-10, 10-15. – Manual for laboratories preparing for accreditation
• 2010 – 700 / 666 passed / 34 failed – Atlas of the artifacts 2010
– Conference and workshop in 2010
• 2011 – 754 / 716 passed/ 38 failed
• Retesting of laboratories 35 passed/ 2 laboratories closed their activity.
• On the next slide, you can see the changes that have occurred in the laboratory previously presented.
8
The disadvantage of this system is the bureaucracy.
It’s a tar bucket to what was achieved – documentation. Results of PT in subsequent years
show the efficiency of corrective actions
Unsatisfactory results %
0,0
5,0
10,0
15,0
20,0
25,0
2007 2008 2009 2010 2011 2012
%
What can we expect in the future?
9
Lets see how it looks in 2013
Unsatisfactory results %
0,0
5,0
10,0
15,0
20,0
25,0
30,0
35,0
2007
2008
2009
2010
2011
2012
2013 2
013
unsatis
facto
ry
i level o
f 1
larv
ae
%
In 2013 new methodology of PT was introduced. The preliminary study on
selected laboratories shows a high number of unsatisfactory results.
• Will that increase of requirements help to improve the quality of examination?
Can we expect such results in subsequent years?
0
5
10
15
20
25
30
35
2013 2014 2015 2016 2017
%
NO WHY?
Evaluation of results
• Detection (or not detection) of 1 larva
becomes accidental and is undependent
on the quality of the examination.
Good laboratories Level of contamination 1 larva / 100g
Expected results:
75% positive : 25% false negative
100% of positives will never be
achieved
Can we say that the negative result is „unsatisfactory”?
10
• Please look closer to obtained previously
results.
• The obtained results in this preliminary
study confirm our thinking.
• Proficiency testing at 1 larvae level
generates at least 20% of unsatisfactory
results.
• The randomness of PT result does not
stimulate laboratories to rise the quality of
examination.
Is it then impossible to evaluate
laboratory performance with
samples spiked with 1 larva ?
This is called an "impossible object",
which means that it's impossible to build.
But drawing it is not impossible, as you see.
Evaluation is theoretically possible
• Yes, but only if we use it to confirm the
proportion of positive to false negative
results 75% to 25%.
• For such calculation with statistic
confidence of 95% and accepted error of
5% it is neceserry to use at least 280
samples.
In our opinion • Laboratory should perform tests at 1 larva level. To pay attention
• Results can be used to evaluate the efficiency of the entire system of meat
examinations or as tool to assess the performance of PT.
• This level shouldn’t be used to evaluate the single lab. since PT on this level
generates unexpected unsatisfactory results.
• With numerous consequences like:
– stopped examination,
– suspended accreditation,
– laboratory has to inform the customer about the unsatisfactory results
(what than - shell we also call the RASFF?)
– re-audit of the unit by external accreditation body
• Corrective actions? (but what kind - retesting its not a corrective action)
• Testing of samples at the level of one larva is a great tool for evaluation of
the system in country (if 75% of the laboratory is able to detect one larva in
the sample), the system works.
• The results can be used for PT evaluation.
Z score value Even at level of 5 larvae is insufficient for such
calculation Histogram pole4
lukasz 2v*128c
pole4 = 128*1*normal(x; 3,2891; 1,329)
0 1 2 3 4 5 6
pole4
0
5
10
15
20
25
30
35
40
Lic
zba o
bs.
Test Shapiro – Wilka Wykres normalności pole4
lukasz 2v*128c
-1 0 1 2 3 4 5 6 7
Wartość obserwowana
-3
-2
-1
0
1
2
3
Oczekiw
ana n
orm
aln
a
pole4: SW-W = 0,9377; p = 0,00002
The hypothesis of normal distribution of results should be rejected
(p value is less than the accepted level of significance of the test α = 0.05).
11
OUR PROPOSITION
• Samples on 1 larva contamination level
should be used in PT for evaluation of PT
• Level 3/5 could be useful for evaluation of
sensitivity of the method in the labs.
• Samples with 20-50 larvae may be used to
evaluate Z-score
• Samples on „0” level to assess specificity
of the method
Achievements
• All of this is what we have achieved over the six years we owe the great help from the EURLP.
• This applies to training, visits to laboratories and numerous consultations.
• It wouldn’t be achieved without the guidance, ideas and the exchange of experiences presented by EURLP and other National Rreference Laboratories.
Thank you for your attention
We are not afraid of little exercises