validation une-en iso 16140 method qfast® … › laboratoriodocumentos › qfast...une -en iso...

VALIDATION UNE-EN ISO 16140

METHOD QFAST®

DETERMINATION OF SALMONELLA spp.

CUALITATIVE METHOD

Edition 3 (14-07-2016)

de 56

INDEX

1.- INTRODUCTION 3

2.- ALTERNATIVE METHOD AND REFERENCE METHOD 4

2.1.- PRINCIPLE AND PROTOCOL OF THE ALTERNATIVE METHOD

4

2.2.- REFERENCE METHOD

9

3.- COMPARATIVE STUDY OF THE METHODS 9

3.1.- RELATIVE ACCURACY, RELATIVE SPECIFICITY AND RELATIVE SENSITIVITY 9

3.2.- RELATIVE DETECTION LEVEL (LDR) 26

3.3.- INCLUSIVITY AND EXCLUSIVITY

28

4.- COLABORATORY STUDY 2014 29

4.1.- ORGANIZATION OF THE STUDY 29

4.2.- RESULTS OF ANALYSIS 31

4.3.- CALCULATIONS 4.4- INTERPRETATION OF THE RESULTS OF THE COLLABORATIVE STUDY

32

38

5.- COLABORATORY STUDY 2013

41

6.- AUDITS 50

7. APLICABILITY 51

8.- FINAL CONCLUSION 53

ANNEXES

Annex 1 – Protocol of the alternative method EASY QFast®

54

Annex 2 – Reference Method ISO 6579:2002-Microbiology of food and animal feeding

stuffs. Horizontal method for the detection of Salmonella spp

55 Annex 3 – Reference Method ISO 6579/A1- Microbiology of food and animal feeding stuffs. Horizontal method for the detection of Salmonella spp. Amendement 1: Annex D. Detection of Salmonella spp. in animal faeces and in environmental samples from the primary production stage.

56

de 56

1.- INTRODUCTION

All the data shown in the current report forms part of the initial report, issued on the 17-03-14, those

obtained in the “Extension” performed during 2014-2015 and those obtained in the extension of the

Meat category.

The results obtained that appear in the corresponding tables as well as their treatment have been

performed according to the norm UNE-EN ISO 16140.

Manufacturer: iMICROQ, Integrated Microsystems for Quality of Life, S.L.

Polígon Industrial Riu Clar

C/ Ferro 6 (nau 7)

43006 Tarragona, Spain

Expert Laboratory: AENORlaboratorio

Miguel Yuste, 12, 4ª planta.

28037 Madrid

Method to validate: Method QFast®, Fast method for detection of Salmonella spp.

Validation reference: UNE-EN ISO 16140: 2003. Microbiology of foof and animal feeding stuffs. Protocol

for validation of alternative methods.

UNE-EN ISO 16140:2003/A1:2012

Reference Method: UNE-EN ISO 6579: 2003. Microbiology of food and animal feeding stuffs. Horizontal

method for the detection of Salmonella spp

UNE-EN ISO 6579 (2003). Erratum: 2007 V2. Horizontal method for the detection of Salmonella spp.

UNE-EN ISO 6579:2003/A1:2007. Microbiology of food and animal feeding stuffs. Horizontal method for

the detection of Salmonella spp. Amendement 1: Annex D. Detection of Salmonella spp. in animal faeces

and in environmental samples from the primary production stage.

Scope of the validation:

Veterinary samples originating from the poultry industry.

Samples of animal food.

Environmental samples originating from primary production (poultry industry).

General Food: fruits and vegetables, dairy products and various products, including spices, mayonnaise

and eggs; and meat.

Certification Body: AENOR

de 56

2.- ALTERNATIVE METHOD AND REFERENCE METHOD

2.1.- PRINCIPLE AND ALTERNATIVE METHOD

PRINCIPLE

a) Pre-enrichment in media 1

Add to 225 ml of pre-enrichment media the adequate quantity of sample and incubate at 37ºC

± 1ºC for 16 ±1 hours.

b) Enrichment in media 2:

Take 10 µl of the pre-enriched sample and incubate at 41.5 ºC ± 1ºC for 7 hours ± 1 hour.

c) Immunocapture

Add 1 drop of immunocapture solution and incubate, with agitation, at 37 ºC ± 1 ºC for 30

minutes. Place the eppendorf in the magnetic separator for 3 minutes; remove the supernatant,

then add the washing solution and place the eppendorf back in the magnetic separator for 3

minutes. Remove the supernatant.

d) Reading

Add to the eppendorf tube, 1 drop of reaction solution; mix and vortex and incubate at 37 ºC ±

1ºC for 30 minutes. Add 10 µl in the sensor electromagnetic reader.

PROTOCOL

a) Pre-enrichment

1- Preparation of the samples, in the laminar flow cabinet or when it is not available, a

Bunsen Burner.

Veterinary Samples (faeces, neck, shoe covers)

• Faeces:

Weigh, with a precision of ± 1%, 25 g of sample in the flask that contains Hard media 1. If the

media Hard does not contain supplements, add them.

• Neck:


Hard media does not contain supplements, add them.

• Shoe covers:

Submerge the 4 shoe covers in the flask that contains Hard media 1. If the Hard media does not

contain supplements, add them.

de 56

Animal Food Samples

Weigh, with a precision of ± 1%, 25 g of sample in the flask that contains Soft media 1. If the Soft

media does not contain supplements, add them.

Environmental Samples (supports, Box bases)

• Supports:

Submerge the support in the flask that contains Hard media 1. If the Hard media does not

contain supplements, add them.

• Box bases:

Cut the sample to obtain 25 g of sample, with sterile material, submerge in the flask that

contains Hard media 1. If the Hard media does not contain supplements, add them.

Food Samples

• All the matrixes except meat:

Weigh, with a precision of ± 1%, 25 g of sample in the flask that contains Soft media 1. If the Soft

media does not contain supplements, add them.

In the case of foods that contain inhibitor substances, such as spices, the preparation of the

samples is carried out according to that indicated in the norm UNE-EN ISO 6887-4: 2003

“Microbiology of food for human use and animal food. Preparation of the samples for analysis,

initial suspension and decimal dilutions for microbiological examination. Section 4: Specific

rules for the preparation of products different to milk and lactose product, meat and meat

products and fish and fished products”.

• Meat:


Hard media does not contain supplements, add them.

2- Once the samples are prepared as indicated in point 1, proceed in the same manner for

all of them.

Agitate manually and horizontally the flask that contains the sample and the Pre-enrichment

media.

Incubate the sample at 37ºC ± 1 ºC for 16 hours ± 1 hour.

b) Enrichment in media 2

Wait until the enrichment media reaches room temperature (between 18 and 25ºC) before use.

Remove the sample from the incubator and agitate the flask manually and horizontally.

de 56

Then, take 10 µl, with a fixed pipette and add to media 2. If after agitating the flask manually, the

sample has foam or particles in suspension that make it difficult to take the 10 µl, then take previously 1

ml from the flask and add to a sterile eppendorf tube; subsequently agitate and vortex and from here

take the 10 µl, with a fixed pipette and add to media 2. Agitate with the vortex. Incubate at 41.5 ºC ± 1ºC

for 7 hours ± 1 hour, in a shaking incubator iMICROQ. In the case of the faeces samples, the incubation

time is 8 hours.

c) Immunocapture

Wait until the Immunocapture solution reaches room temperature (between 18 and 25ºC)

before use.

After this time, take out the eppendorf tube and add a drop of the Immunocapture solution.

Agitate with the vortex and subsequently incubate at 37 ºC ± 1ºC for 30 minutes.

When the specified time period has lapsed, take the eppendorf tubes out and place in the

magnetic separator. Agitate manually five times (soft rotating movements of 120º) supporting

the samples with the hands so that they do not fall out of the separator. Leave to stand for 3

minutes, using a timer.

Remove the supernatant from the eppendorf tube with a Pasteur micropipette, without

removing the tube from the magnetic separator. Keep the tube in the support (but remove the

magnet), add 1 ml of wash solution. Agitate manually approximately 5 times. Subsequently,

place the magnet and leave to stand for 3 minutes, control the time using a timer.

Keep the eppendorf tube in the separator, remove all the supernatant with a Pasteur pipette,

and when there is a small fraction left, remove with a 100 µL micropippette.

d) Reading Reaction

Wait until the solution reached room temperature (between 18ºC and 25ºC) before use.

Add a drop of the reaction solution to the eppendorf tube, agitate with the vortex and incubate

at 37ºC ± 1 ºC for 30 minutes in the shaking incubator iMICROQ.

Remove the eppendorf from the skaking incubator and take 10 µl with a fixed micropippette

and add to the sensor, that has been previously cooled down, that has previously been fixed in

the electronic reader. Read by pressing the start button.

In the case that the results of the reader indicates Positive, take with inoculating loop, from the

same eppendorf tube and inoculate on a plate with selective media (for example XLD agar).

For the extension of the study, it was inoculated in two selective media: XLD and ASAP.

de 56

e) Expression of the results

The PRESENCE of Salmonella spp. is considered in the part of the analysis (specifying the weight

in grams, of the sample analysed or surface of the sample), when:

The electrochemical sensor indicates: Positive

Note: if the sensor indicates “retest” measure again with another sensor with the remaining

sample in the eppendorf, taking the second measurement as the definitive one.

A result with a positive value indicates that the sample is contaminated with Salmonella.

In this case, the result should be confirmed.

The ABSENCE of Salmonella spp. in the part analysed (specifying the weight in grams, of the

sample or the surface sampled), when:

The electrochemical reader indicates: Negative

A result with a Negative test indicates that the sample does not contain Salmonella at a

concentration lower than the detection limit

Confirmation of positive results

All of the positive results of the method QFast® Salmonella should be confirmed.

The confirmation should be carried out using the liquid reaction prior to reading and should be started

after the read. Isolate specific plates, as for example XLD and ASAP. Incubate the plates at 37ºC for 24

hours. Confirm the suspicious colonies via adequate biochemical and/or serological tests.

The following table presents a summary of the description of the method together with the application

field and the corresponding reference method.

Description Application Field Reference method Alternative Method QFast® Salmonella HARD for fast detection of Salmonella spp

Veterinary samples from the poultry industry Environmental samples from primary production (poultry) Meats

UNE-EN ISO 6579:2003/A1:2007 UNE-EN ISO 6579:2003

Alternative Method QFast® Salmonella SOFT for fast detection of Salmonella spp

Animal Feed Samples

Food samples except meat

UNE-EN ISO 6579:2003

de 56

Conservation of the enrichment media and the reagents of the QFast® method

Media 1: Pre-enrichment media (HARD):

Presentation in flasks with 225 ml of media con supplements. Conservation between 2 and 8 ºC.

Expiration date 15 months from production.

Presentation in flasks with 225 ml of media without supplements. Conservation between 2 and

8 ºC. Expiration date 15 months from production.

Presentation bottles of dehydrated pre-enrichment media without supplements. Conservation

between 4 and 30 ºC. Expiration date 3 years from production.

Media Supplement 1 in glass packaging with a dropper. Conservation between 4 and 30 ºC.


Media 1: Pre-enrichment media (SOFT):

Presentation in flasks with 225 ml of the media with supplements. Conservation between 2 and

8 ºC. Expiration date 15 months from production.

Presentation in bottles of dehydrated pre-enrichment media without supplements. Conservation

between 4 and 30 ºC. Expiration date 3 years from production.

Supplement media 1 in glass packaging with a dropper (only for the dehydrated format).

Conservation between 4 and 30 ºC. Expiration date 1 year after production.

Media 2: Enrichment media

Presentation in eppendorf tubes with 1 ml of media. Conservation between 2 and 8 ºC.


Dehydrated format in aluminium bags of 2.8 g. Conservation between 4 and 30 ºC. Expiration

date three years from production.

Immunocapture Solution: Presentation in 3 ml bottles with dropper. Conservation between 2 and 8 ºC.

Expiration date two years from production.

Wash Solution: Presentation in bottles of approximately 50 ml with dropper. Conservation between 2

and 8 ºC. Expiration date one year after production.

Reaction Solution:

Presentation a bottle of 5 ml with a dropper. Conservation between -21 y -5ºC. Expiration date

one year from production.

Presentation in two bottles of 3 ml with a dropper; one with the dehydrated reaction solution

and another with the diluent solution. Conservation between -21 and -5ºC. Expiration date one

year from production.

Sensor: Presentation in plastic packaging with silicagel. Conservation between 2 and 8 ºC. Expiration

date one year from production.

de 56

2.2.- REFERENCE METHODS

The reference methods are the following:

UNE-EN ISO 6579: 2003. Microbiology of food and animal feeding stuffs. Horizontal method for

the detection of Salmonella spp.

UNE-EN ISO 6579 (2003). Erratum: 2007 V2. Horizontal method for the detection of Salmonella

spp.

UNE-EN ISO 6579:2003/A1:2007. Microbiology of food and animal feeding stuffs. Horizontal

method for the detection of Salmonella spp. Amendement 1: Annex D. Detection of Salmonella

spp. in animal faeces and in environmental samples from the primary production stage.

3.- COMPARATIVE STUDY OF THE METHODS

They have performed the following studies:

o Relative Efficiency (AC), Relative Specificity (SP) and Relative Sensitivity (SE).

o Relative detection level

o Inclusivity and Exclusivity

3.1.- RELATIVE EFFICIENY, RELATIVE SPECIFICITY AND RELATIVE SENSITIVITY

3.1.1 Matrix for performing the validation

In the study carried out in 2013 they analysed 284 samples, both with the alternative method QFast® as

well as with the REFERENCE method (UNE-EN ISO 6579 for animal food and collars; and UNE-EN ISO

6579/A1 for the environmental samples and primary production). In table 1, the distribution of the

samples by category is established.

Table 1- Distribution of samples by category in the validation (2013)

Category Positive Samples (1) Negative Samples Total

1. Veterinary (faeces, shoe covers and necks) 45 74 119

2. Animal Food (corn, rye and soya) 30 65 95

3. Environmental (support dust and box bases) 30 40 70

TOTAL 105 179 284

(1)Samples positive by one method or another

de 56

In the extended study carried out in 2014 and 2015 they included three categories of food as well as

increasing the number of samples analysed in the Veterinary matrix and Environmental primary

production; and of Animal Food. In total they have analysed 407 samples, both with the alternative

method QFast® as well as with the reference method (UNE-EN ISO 6579 for food, Animal Food and

collars; and UNE-EN ISO 6579/A1 for the Environmental samples and primary production). In table 2 the

distribution of the samples by category are established.

Table 2- Distribution of samples by category (2014-2015)

Category Positive Samples (1) Negative Samples Total

Veterinary (faeces, shoe covers and necks) 36 25 61

Environmental (support dust and box bases) 40 30 70

Animal feed (corn, rye and soy) 33 30 63

Fruits and vegetables 46 30 76

Dairy products 42 30 72

Others (spices, eggs and mayonnaise) 37 28 65

TOTAL 234 173 407

(1) Samples positive by one method or another

In 2015 they have validated the meat group; in total they have analysed 66 samples both with the

alternative method QFast® as well as with the reference method (UNE-EN ISO 6579). In table 3 the

distribution of the samples in the meat group are established.

Table 3- Distribution of samples in the meat group

Category Positive

Samples(1)

Negative

Samples Total

Meat 36 30 66

(1) Samples positive by one method or another

In table 4 the distribution of the samples by category in the extended study have been established

(2014-2015).

de 56

Table 4- Distribution of samples by category (2014-2015)

Category Positive Samples

(1)

Negative

Samples Total

Veterinary (faeces, shoe covers and necks) 36 25 61

Environmental (support dust and box bases) 40 30 70

Animal feed (corn, rye and soy) 33 30 63

Foods (Fruits and vegetables, Dairy products, others and

meat) 161 118 280

TOTAL 270 203 473

(1)Muestras positivas por uno u otro método

3.1.2 Natural samples with Salmonella spp.

In the study carried out in 2013 it was necessary to work exclusively with natural samples inoculated

with Salmonella spp.

In the extended study carried out during 2014-2015, they have analysed 40 naturally contaminated

samples, which amounts to 17 % of the total number of with Salmonella spp. analysed. In the study that

included meat, they have analysed 46 naturally contaminated samples that amounts to 17 % of the total

number of samples with Salmonella spp.

When it has not been possible to obtain a sufficient number of natural samples with Salmonella spp., the

natural samples are inoculated, as described in the UNE-EN ISO 16140.

According to that indicated in the UNE-EN ISO 16140, prior to the inoculation of the natural samples,

the strain is put under stress.

3.1.3 Preparation of the sample for analysis

The UNE-EN ISO 16140 has established with regard to the reference and the alternative method that the

validation should be carried out, whenever possible, with the same sample. In this case, it is not possible

and they have proceeded according to the described in the second case of the point 5.1.1.2.3 of the

UNE-EN ISO 16140.

de 56

3.1.4 Results of the tests

The raw results are shown in annex 4.

In the following tables the paired results of the reference and alternative method are shown; In

addition, they have also calculated the parameters relative sensitivity (SE), relative specificity (SP) and

relative efficiency (AC).

The following shows the definitions of each of the abbreviations:

A+ = Total number of Positive results by the alternative method

A - = Total number of Negative results by alternative method

R+ = Total number of Positive results by the reference method

R - = Total number of Negative results by the reference method

PA = Concordance of Positive results

NA = Concordance of Negative results

PD = Positive Deviation

ND = Negative Deviation

In the following tables the results by category and the global results are shown.

de 56

Table 5- Global Results (2013)

All the matrices Method ISO 6579 and ISO 6579/A1

Reference method (R+) Reference method (R-)

Qfast®

Positive alternative method

(A+) Positive concordance (A+/R+)

PA=101

Positive deviation (R-/A+)

PD=1

Negative alternative

method (A-) Negative deviation (A-/R+)

ND=4 Negative concordance (R-/A-) NA=178

Table 6- Global Results (2014-2015)



Qfast®



PA=229 Positive deviation (R-/A+) PD=4



ND=5

Negative concordance (R-/A-)

NA=169

Table 7- Global Results including the meat



Qfast®



PA=363


PD=6



ND=12


NA=376

de 56

CATEGORY 1: VETERINARY SAMPLES

Table 8- Results of the category 1: veterinary samples (2013)

Category 1: Veterany samples Method UNE-EN ISO 6579 and UNE-EN ISO 6579/A1


Qfast®

Positive alternative

method (A+) Positive concordance (A+/R+)

PA=44

Positive deviation (R-/A+) PD=1



ND=1

Negative concordance (R-/A-) NA=73

Table 9- Results of the category 1: veterinary samples (2014-2015)



Qfast®



PA=35




ND=1


Table 10- Results of the category 1: veterinary samples



Qfast®



PA=79




ND=2


de 56

CATEGORY 2: FOOD ANIMAL SAMPLES

Table 11- Results of the category 2: Food animal samples (2013)

Category 2: Food animal Method UNE-EN ISO 6579


Qfast®



PA= 27


PD= 0



ND= 3


NA= 65

Table 12- Results of the category 2: Food animal samples (2015)



Qfast®



PA= 33


PD= 0



ND= 0


NA= 30

Table 13- Results of the category 2: Food animal samples



Qfast®



PA= 60


PD= 0



ND= 3


NA= 95

de 56

CATEGORY 3: ENVIRONMENTAL SAMPLES

Table 14- Results of the category 3: Environmental samples (2013)

Category 3: Environmental

samples

Method UNE-EN ISO 6579/A1


Qfast®



PA= 30


PD= 0



ND= 0


NA= 40

Table 15- Results of the category 3: Environmental samples (2014-2015)


samples



Qfast®



PA= 38


PD= 0



ND= 2


NA= 30

Table 16- Results of the category 3: Environmental samples


samples



Qfast®



PA= 68


PD= 0



ND= 2


NA= 70

de 56

CATEGORY 4: FOODS

Table 17- Results of the category 4: Foods

Foods Method UNE-EN ISO 6579


Qfast®



PA=156


PD=5



ND=5

Negative Concordance (R-/A-)

NA= 113

3.1.5 Calculation and interpretation of the results and Confidence intervals.

Calculation of the relative efficiency (AC), relative specificity (SP) and relative sensitivity (SE).

The results of the validation are shown in table 18 and table 19.

Table 18- Results of the relative efficiency (AC), relative specificity (SP) and relative sensitivity (SE).

NA ND PA PD Total Efficiency Specificity Sensitivity

Food Animal 95 3 60 0 158 98,10% 100,00% 95,24%

Foods 113 5 156 5 279 96,42% 95,76% 96,89%

Fruit and vegetables 30 0 46 0 76 100,00% 100,00% 100,00%

Dairy Products 30 0 42 0 72 100,00% 100,00% 100,00%

Others 24 2 35 4 65 90,77% 85,71% 94,59%

Meat 29 3 33 1 67 93,94% 96,67% 91,67%

Environmental samples 70 2 68 0 140 98,57% 100,00% 97,14%

Veterinary samples 98 2 79 1 180 98,33% 98,99% 97,53%

Total 376 12 363 6 757 97,62% 98,43% 96,80%

de 56

Table 19- Results of the relative efficiency (AC), relative specificity (SP) and relative sensitivity (SE) in the

different categories

NA ND PA PD Total Efficiency Specificity Sensitivity

Food Animal 95 3 60 0 158 98,10% 100,00% 95,24%

Foods 113 5 156 5 279 96,42% 95,76% 96,89%

Environmental samples 70 2 68 0 140 98,57% 100,00% 97,14%

Veterinary samples 98 2 79 1 180 98,33% 98,99% 97,53%

Total 376 12 363 6 757 97,62% 98,43% 96,80%

Confidence Intervals

In the table 20, the results obtained, the AC, SP and SE (%) are shown for all the categories with regard to

their lower confidence limit (LCL) and higher confidence limit (LCS)

Table 20- Results of the AC, SP y SE with their lower confidence limit

Matrices AC (%) LCL (%) SP (%) LCL (%) SE (%) LCL (%)

Category 1. Veterinary samples 98,33 96,66 98,99 97,98 97,53 95,02

Category 2. Food animal samples 98,1 96,2 100 - 95,24 91,73

Category 3. Environmental samples 98,57 97,14 100 - 97,14 94,71

Category 4: Food samples 96,42 93,63 95,76 92,52 96,89 94,35

All of the category 97,62 95,43 98,43 97,38 96,80 94,20

de 56

3.1.6 Discordant results

Statistical Test according to annex F of the norm UNE-EN ISO 16140

The test used for the study of discordant results discordant is:

After counting the total number of discordant results of Y in the following manner:

Y= PD + ND

Check if the two methods can be different by the balance of the sensitivity versus the specificity:

- Case 1: For Y < 6, less than 6 discordant results there is no test available.

- Case 2: For 6 ≤ Y ≥ 22, between 6 and 22 discordant results, the m is calculated as the smaller of

the two values PD and ND and the binomial law is applied according to the following table 21:

If m ≤ M for a given Y, the two METHODS are different a α < 0.05 (bilateral).

Table 21. Values for disagreements for case 2.

Disagreements

Y= PD + ND

6 to 8

9 to 11

12 to 14

15 to 16

17 to 19

20 to 22

M= máx (m)

for α 22, more than 22 discordant results, the MCNemar test is used with the chi-

square distribution for 1 degree of freedom:

X2 = d2/Y, con d= |PD - ND| e Y= PD + ND

The two METHODS are different for α 3,841.

In the validation, the Y value obtained is 18 thus the second case applies where m is 6 and the

tabulated value of M is 4; thus in function of the results obtained, the QFast® method is equivalent to the

reference method.

The results of the statistical test obtained is shown in the following table 22.

de 56

Reference method

Alternative method Qfast®

Y= ND+PD Y=12 + 6 Y=18

Conclusion The methods are equivalent

After performing the calculation established in the UNE-EN ISO 16140 for the parameters of Relative

efficiency, Relative specificity and Relative sensitivity, as well as the study of the discordant results, the

conclusion was made that both methods (reference and alternative) are equal and, thus, the discordant

results do not put the validity of the alternative method in doubt.

de 56

3.1.6 Study of discordant results

Negative Deviation (ND)

In relation to the negative deviation, there were twelve. The information about these samples are shown

in table 23.

Table 23- Discordant Results obtained: negative deviations

Nº Sample

Matriz Qfast Result Reading sensor

XLD ASAP ISO 6579/A1 or ISO

6579 Result

IQ-699 Shoe covers Absence/ 4 shoe

covers - Positive

it was not done

Presence/ 4 shoe covers

560 Support sponge

powder Absence/ support 451 nA

Positive (1 colonie)

Positive ( 2 colonies)

Presence/ support

562 Support sponge

powder Absence/ support 383 nA

Positive (2 colonies)

Positive (2 colonies)

Presence/ support

161 Shoe covers Absence/ 4 shoe

covers 440 nA Positive Negative

Presence/ 4 shoe covers

IQ-149 Barley Absence/ 25 g - Positive it was not done Presence/ 25 g



503 Tabasco Absence/ 25 g 315 nA Positive

( 1 colonie) Negative Presence/ 25 g

504 Tabasco Absence/ 25 g 748 nA Negative Positive

(low level of growth)

Presence/ 25 g

1314 Chopsticks Absence/ 25 g

354 nA Positive Positive Presence/ 25 g

1315 Chopsticks skirt

Absence/ 25 g 403 nA Positive Positive Presence/ 25 g

1282 Entrecot

Absence/ 25 g 596 nA Positive Positive Presence/ 25 g

nA: nano amperiros

Comments:

Comparative Study 2013

In the samples IQ-699, IQ-149, IQ-153, IQ-157 analysed in the initial validation of the alternative

method, they have proceeded to, regardless of the results given by the reader, to spread on an XLD

plate. In the four samples with negative deviations there was growth characteristic of the Salmonella on

the XLD plate. The colonies characteristic of Salmonella spp. on the XLD agar have a black centre and a

slightly transparent reddish coloured area due to the change of the colour indicator; the Salmonella

variants H2S negative that grow on the XLD agar are pink with a darker centre; and the Salmonella

lactose positive that grow on the XLD agar are yellow with or without darkening.

The sample IQ-699 was contaminated with the strain (CV04), this strain is used at the same time to

inoculate other samples of the same matrix of which had favourable results.

de 56

The samples IQ-149, IQ-153 y IQ-175 were inoculated using the strain (CV14), this strain is used at the

same time to inoculate other samples of the same matrix of which had favourable results.

The morphology of the colonies grown on the XLD plates of the 4 samples correspond to the typical

morphology of the inoculated strains.

The negative result given by the reader of the four samples could be due to that fact that, at the time of

reading, the sensor was inoculated with a drop, it did not reach the mark indicated due a lack of

capillarity.

Note validation 2014-2015: The manufacturer of the alternative method has performed the modification of the design of

the reader to rectify this deviation.

Comparative Study 2014-2015

In all the samples analysed in the extended validation of the alternative method, they proceeded to

plate an XLD and ASAP plate, independently of the result given by the reader.

The characteristics of the colonies of Salmonella spp. in the chromogenic media ASAP are coloured

pink/purple.

In the two Environmental samples (560, 562) there was a low level of growth of colonies characteristic

of Salmonella spp. in the two plates; after the biochemical and serological confirmation, they were

confirmed as Salmonella spp.

The samples were inoculated with 2.7 cfu/ powder sponge support, the characteristics of this matrix

does not allow them to ensure the homogeneity of the contamination to such low levels. Thus, it was

determined that the quantity of microorganisms at the time of reading was not sufficient for the reader

to detect them and 24 hours later, after the incubation at 37ºC ± 1ª C, the microorganisms would have

been visible on the XLD and ASAP plates.

In the veterinary sample (161) there was growth of colonies characteristic of Salmonella spp. in the XLD

agar and there was no growth of characteristic colonies in the chromogenic media ASAP. The absence

of growth of characteristic colonies in the chromogenic ASAP media indicates a low concentration of

Salmonella in the inoculated sample, that has produced the negative result in the alternative method. The

discordance between the two methods is due to that fact that the analysis was carried out from a

different part of the sample in both methods.

In the two samples analysed of the tabasco species (503 y 504) there was only growth of Salmonella spp.

de 56

characteristic colonies, on one of the two confirmation plates; in the sample 503 there was only one

colony characteristic of Salmonella spp., which was confirmed subsequently via the appropriate

biochemical and serological tests, on the XLD plate. In sample 504 there was only low growth of

Salmonella spp., confirming subsequently via the appropriate biochemical and serological tests, on the

ASAP plate. Both samples were inoculated using the strain (CV14), this strain is used in the inclusive

study, and was satisfactory.

The Chile Tabasco species and garlic have natural substances with an anti-microbial effect. The norm

UNE-EN ISO 6887-4: 2003 “Microbiology of food for human consumption and animal food. Preparation

of the samples for analysis, initial suspension and decimal dilutions to examine microbiology”

establishes that the preparation of the samples of species for subsequent microbiological analysis by

ISO methods, require the addition of potassium sulphite to the peptone dilution water (APT) with the

aim of diminishing the anti-microbial activity of inhibitor substances that contain this type of matrix.

This addition is carried out in the Tabasco for subsequent analysis by the reference method but not for

the samples analysed by the alternative method. This explains that the two tabasco samples analysed by

the alternative method obtained false negative results. As a consequence of these results, the protocol of

the method should include that in the case of this type of sample, the preparation should be carried out

according to the instructions in the norm UNE-EN ISO 6887-4.

Comparative Study 2014-2015 (extension of meat matrix)

The three meat samples indicated with negative deviations were packaged in a modified atmosphere,

which minimises the growth of microbes. In these types of samples, with a low level of microorganisms,

in those where the hard media is used such as pre-enrichment, the Salmonella spp. can be metabolically

inactive due to the potent inhibitor effect of the media. This means that the enzymatic reaction does not

happen at the time of the read, giving a negative read but producing subsequently growth of

Salmonella spp. on the XLD and ASAP plates.

Positive Deviation (PD)

With regard to the positive deviations, there were six: one in the faeces matrix, four of these in the

species matrix and another in the beef black pudding. The information about these samples are shown

in table 24.

de 56

Table 24- Discordant results obtained: positive deviations

Nº sample Matriz Qfast result Reading sensor

XLD ASAP

ISO 6579/A1 or ISO 6579 Result

IQ-180 Faeces Presence / 25 g - Positive - Absence / 25 g

204 Granulated

garlic Presence / 25 g 3440 nA Negative Negative Absence / 25 g

205 Granulated

garlic Presence / 25 g 6910 nA Negative Negative Absence / 25 g

208 Meat dressing Presence / 25 g 1730 nA Negative Negative Absence / 25 g

282 Salad dressing Presence / 25 g 1,75 uA Negative Negative Absence / 25 g

1285 Beef black pudding

Presence / 25 g (1)

1,66 uA Negative Negative Absence / 25 g

nA: nano amperiros

uA: microamperios

(1)The subsequent identification with API 20E.

Comparative Study 2013

With regard to the IQ-180 analysis in faeces, the confirmation performed is the growth on XLD plates; in

this case, the colonies grown were similar to the colony characteristics of Salmonella, and could have

been a true positive. The discordant result with the alternative method was due to the heterogeneity of

the sample, as both methods are taken from a different portion.

Comparative Study 2014-2015

With regard to the positive deviations obtained for the species (204, 205, 208 and 282), there was no

growth of characteristic colonies in XLD agar or in the chromogenic media ASAP.

The species are complex matrices that due to their intrinsic characteristics could initially give rise to false

positives; after the compulsory confirmation concluded that they were confirmed assumed positive

negative.

Comparative Study 2014-2015 (extension of meat matrices)

With regard to the positive deviation obtained in the sample of beef black pudding, there was no

growth of characteristic colonies on the XLD or ASAP plates; however, the colonies present on both

plates were identified via the biochemical tests of API 20E and the following identification was

obtained:

• XLD Plate:

a. Yellow colonies Enterobacter cloacae

• ASAP Plate:

a. Turquoise colonies: Enterobacter cloacae

de 56

In the exclusivity study they analysed the strain of the Enterobacter cloacae collection (CECT 194) which was

not present on the soft and hard media. In this case the strain identified in both media was a wild strain

present naturally in the sample of beef black pudding analysed; it could be that in this case, the wild

strain had a different behaviour to that of the strain collection.

However, after the compulsory confirmation, the conclusion was that it was a suspect confirmed

positive negative result.

de 56

3.2.- RELATIVE DETECTION LIMIT (LDR)

The relative detection level is the smallest number of culturable microorganisms that can be detected in

the sample in 50% of occasions by the alternative and reference methods. The artificial contaminations

are performed in accordance with the requisites of the norm UNE-EN ISO 16140, previously described.

The determination of the relative detection limit was performed again in the validation carried out

during 2014-2015. For this, seven different matrices were studied, each encompassed a category. For

each one of the categories they used a different target microorganism.

They have used 4 levels, as a minimum, of target microorganisms per matrix, including the negative

control. The analysis was performed six times by both methods, alternative and reference.

These matrices as well as the Salmonella spp. strains were:

• Faeces: Salmonella typhimurium

• Dried powder: Salmonella london

• Dairy products (crude milk): Salmonella dublin

• Fresh Eggs: Salmonella enteritidis

• Vegetables (fruit): Salmonella virchow

• Corn: Salmonella derby

• Meat: Salmonella arizonae

3.2.1 Contamination protocol

The UNE-EN ISO 16140 establishes with regard to the reference and alternative method that a

validation should be carried out, with the same sample. In this case it was not possible and they

proceeded according to that described in the second case of the point 5.1.1.2.3 of the previously

mentioned UNE.

3.2.2 Results

The raw results are shown in annex 7.

The results of the detection limit are shown below, they were calculated with the Spearman–Kärber1

test. 1 “Hitchins A. Proposed Use of a 50 % Limit of Detection value in Definig Uncertainty Limits in the

Validation of presence-Absence Microbial Detection Methods. Draft 10th December, 2003”.

de 56

Table 25- Relative Detection Level Study: Results obtained

MATRIZ

Qfast® (cfu/ 25) UNE-EN ISO 6579/A1 (cfu/ 25 g) or UNE-EN ISO 6579 (cfu/ 25 g)

FAECES Relative detection level 0,8 1,38

Interval 0,42-1,50 0,98-1,90

DRIED POWDER

Relative detection level 0,46 0,61

Interval 0,18-1,15 0,27-1,38

FRUIT Relative detection level 0,8 0,22

Interval 0,48-1,32 0,15-0,35

EGGS Relative detection limit 0,85 0,3

Interval 0,55-1,30 0,20-0,50

CRUDE MILK Relative detection level 0,48 0,35

Interval 0,28-0,80 0,28-0,45

MEAT Relative detection level 0,65 0,4

Interval 0,40-1,02 0,22-0,72

CORN Relative detection level 0,4 0,4 Interval 0,25-0,625 0,25-0,625

Conclusions:

The alternative method and the reference method have relatively similar detection levels.

In the relative detection level of the alternative method there are between 0.18 and 1.5 cfu and in the

case of the reference method between 0.15 and 1.9 cfu.

de 56

3.3.- INCLUSIVITY AND EXCLUSIVITY

The inclusivity is the capacity of the alternative method to detect the analyte target between wide

groups of strains. The objective of the study is to verify that all the strains are detected by the alternative

method as there are two culture media, one for simple matrices (Soft) and other for complex matrices

(Hard), the study is performed with both media.

The exclusivity is the absence of interference in the alternative method of an adequate group of non-

target strains. The objective of the study is to verify that the non-target strains are not detected by the

alternative method and this verifying the selectivity of the alternative method. As with the inclusivity the

study is performed with both media.

3.3.1 Test protocol

The strain selected for this study are in cryoballs, which are allowed to grow in BHI media at 37ºC for

24h. Once this time has passed, the corresponding decimal dilutions are performed in maximum

recovery diluent (MRD), and 0.3 ml of the adequate dilution is added to 225 ml of the corresponding

media. To control the inoculum added to the sample, 1ml of the corresponding dilution is spread on the

5 PCA plates and they are incubated for 72h at 30ºC.

3.3.2 Results and conclusion

The inclusivity and exclusivity studies were carried out in the initial study in 2013 and were extended in

the study performed in 2014-2015. The results of the inclusiveness and exclusivity, both in the Hard as

well as the Soft media, are shown in Annex 5 for the Exclusivity and Annex 6 for the Inclusivity.

Initial Validation (2013):

Of the 30 target strains inoculated in the Soft media1: the results of all of them were ok. All the non-

target strains inoculated in the media were negative.

Of the 30 target strains inoculated in the Hard media 1: the results of all of them were correct. All the

non-target strains inoculated in the media were negative.

Extension of the validation (2014-2015)

The results obtained in both the Hard as well as the Soft media were correct, both for the inclusivity as

well as the exclusivity.

de 56

4.- COLABORATIVE STUDY 2014

Within the initial validation in 2013 a collaborative exercise was performed using faecal samples. In this

exercise, 14 laboratories participated. In 2014 and within the extension of the sample validation of

human food, a second validation was considered necessary. In this exercise carried out in powdered

milk, 11 laboratories participated.

4.1.- ORGANIZATION OF THE STUDY

The collaborative exercise study was organised in November and December 2014. The number of

participating laboratories was 11.

The chronogram of the collaborative is detailed below:

• 18-21 November 2014: Reception of the samples of powdered milk in the laboratory

• Week of the 24th of November 2014: start of test lot A (Samples M1 to M12)

• Week of the 1st December 2014: start of test lot B (Samples M13 to M24)

There were no incidents in the in the planned chronogram.

4.1.1.-Nature of the preparations

A sample was selected that originated from a single lot, consistent in skimmed powdered milk.

With regard to this sample, the absence of Salmonella spp., was verified via the application of the

Reference method (UNE-EN ISO 6579). In these tests, the presence of Salmonella spp. was not found in

any of the samples analysed.

Before being sent, a sufficient quantity of skimmed powdered milk packages were prepared to allow

the addition prior to sending a quantity inoculated with Salmonella spp. prepared with powdered milk,

obtaining a final weight of 25 grams.

de 56

4.1.2.-Inoculation rate

The samples were inoculated in three different levels:

Table 26- Levels of inoculation of Salmonella spp.

Level Salmonella spp. media count in 25 g

L0 0

L1 35 cfu

L2 76 cfu

The inoculums were prepared with the addition of Salmonella entérica from the CECT 4594.

The stability tests carried out for the period of the test confirmed the survival of the microorganism in all

the samples analysed that had been inoculated.

4.1.3.- Problems with the temperature and the transport at the time of reception

The samples were coded as M1 to M24, with each laboratory receiving two packages of each code,

which were used randomly for the test with the Reference method or the alternative method.

The samples and inoculum for sending to the laboratories were stored in the freezer (-30ºC) until

dispatched and the stability of the inoculum was confirmed when these were kept in the fridge for up

to 2 weeks.

The samples were sent on the 17th of November, in an isothermal packet with dry ice and all the

laboratories received them on the 18th of November. All the preparations arrived on time.

The laboratories should maintain the samples and inoculum in the fridge until starting the test: from the

24th of November for the codes M1 to M12, and from the 1st of December for the codes M13 to M24.

The codes corresponding to the inoculum and their corresponding level of contamination are those

shown in Table 27.

de 56

Table 27- Codes of the samples and their inoculation level

Level of contamination Sample codes

L0 M1, M2, M7, M8, M13, M14, M19, M20

L1 M3, M4, M5, M6, M9, M10, M11, M12

L2 M15, M16, M17, M18, M21, M22, M23, M24

4.2.- RESULTS OF THE ANALYSIS

The summary of the results obtained by the laboratories are shown in table 28.

Table 28- Coincidence of the results for each of the methods applied and the inoculation level

Laboratory

Alternative method Reference method

Nº Positive samples Nº Positive samples

Level 0 Level 1 Level 2 Level 0 Level 1 Level 2

1 0 / 8 8 / 8 7 / 8 1 / 8 8 / 8 8 / 8

2 3 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

3 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

4 0 / 8 8 / 8 7 / 8 4 / 8 8 / 8 8 / 8

5 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

6 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

7 0 / 8 6 / 8 8 / 8 0 / 8 8 / 8 8 / 8

8 2 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

9 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

10 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

11 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

FP TP TP FP TP TP

5 86 86 5 88 88

N 88 88 88 88 88 88

FP: Number of false positives

TP: Number of true positives

N: Number of analyses carried out in each level

de 56

4.3.- CALCULATIONS

4.3.1.- CALCULATION OF THE PERCENTAGES OF SPECIFICITY (SP), SENSITIVITY (SE) AND RELATIVE

EFFICIENCY (AC)

Calculations of Specificity (SP) and Sensitivity (SE)

The values of specificity and sensitivity in both methods are shown in table 29, including the value of

the inferior limit (LCL).

For the calculation of specificity and sensitivity, the following formula is used.

For Level L0:

For levels L1 and L2:

Being:

FP: False positives

TP: True positives

N: number of samples

Table 29: Specificity (SP) and Sensitivity (SE) of the alternative and reference method

Alternative method

QFast® LCL

Reference method

UNE-EN ISO 6579 LCL

Specificity (SP)

SP (Level L0) 94,32% 89,38% 94,32% 89,38%

Sensibility (SE)

SE (Level L1) 97,73% 94,55% 100,00% 100,00%

SE (Levell L2) 97,73% 94,55% 100,00% 100,00%

SE (Levell L1+L2) 97,73% 95,48% 100,00% 100,00%

de 56

Calculation of Relative Accuracy (AC)

For each contamination level and the total of the results, the alternative and reference method were

compared with the objective of calculating the relative efficiency and studying the discordant results.

The comparison of the pairs of results of the different levels of contamination are summarised in table

30.

Table 30: Results obtained by both methods

Level Alternative method

QFAST® Salmonella

Reference method

UNE-EN ISO 6579

Positive samples Negative samples Total

Level L0

Positive samples PA= 0 PD= 5 5

Negative samples ND= 5 NA= 78 83

Total 5 83 88

Level L1

Positive samples PA= 86 PD=0 86


Total 88 0 88

Level L2



Total 88 0 88

Level L1+2



Total 176 0 176

Note-PA: positive concordance; NA: negative concordance; ND: negative deviations; PD positive

deviations.

The values of relative efficiency (AC) for the different levels of contamination are shown in table 31,

together with the corresponding confidence intervals (LCS and LCL).

The relative efficiency is calculated according to the following formula:

de 56

Being:

PA: Number of positive concordance

NA: Number of negative concordance

N: Total number of samples analysed

Table 31: Values of the Relative efficiency (AC) and confidence intervals

Level Parametre Result

Level 0

AC 89,64%

LCS 95,40%

LCL 81,87%

Level 1

AC 97,73%

LCL 94,55%

Level 2

AC 97,73%

LCL 94,55%

Level 1+2

AC 97,73%

LCL 95,48%

de 56

4.3.2.- STUDY OF THE DISCORDANT RESULTS

The discordant results are examined with the statistical test according to that described in annex F of

the norm UNE-EN ISO 16140.

The results of the statistical test obtained are shown in table 32.

Reference method

Alternative method Qfast®

Level 0

Y= ND+PD Y=10

Conclusion

m lower value of PD and ND is 5 (m)

M=1 for Y from 9 to 11

Both methods are considered as equivalent

(m>M)

Level 1

Y= ND+PD Y=2 (Y

de 56

Study of deviations

False positives

5 false positives found in level 0 with alternative method and 5 false positives more detected with

reference method ISO.

The 5 false positives obtained with reference method were found in the following samples of the afore

mentioned laboratories:

Laboratory 1, sample M19: the laboratory issued the result as Presence of Salmonella spp., after receiving

the results from the biochemical tests carried out.

Laboratory 4, samples M2, M7, M8 y M14: the laboratory issued the result as Presence of Salmonella spp.,

after receiving the results from the biochemical tests carried out.

The 5 false positives obtained via the alternative method were obtained via the samples prepared by

the afore mentioned labs:

Laboratory 2, samples M7, M8 and M20.

Laboratory 8, samples M19 and M20.

The biochemical confirmations carried out by the participating labs allow them to assign the 10 false

positives to cross contaminations.

Laboratory 1 indicates in the observation section that as soon as they obtain the results from the

collaborative exercise organizer “it was possible to suffer cross contamination when the samples were

processes”.

Taking into account that the same laboratory has obtained a false negative (no growth in any of the two

plates) in sample M24, processed on the same day, it is possible that the samples could be interchanged

by the lab.

de 56

False negatives

Two false negatives detected in level 1 with the alternative method and two more in level 2 with the

alternative method.

The 4 false negatives obtained via the alternative method took place in the following samples of the

aforementioned laboratories:

Level 1: two samples in laboratory 7: M6 and M12.

Level 2: one sample in laboratory 1 (M24) and another one in laboratory 4 (M24).

With regard to the result obtained in 1 for level 2 in sample M24, the sensor indicates the absence of

Salmonella spp. Besides, there is no evidence of growth of colonies on the XLD plate (it does not

indicate Growth Results during the second test). Taking into account that the same laboratory obtained

a false positive (with growth in the two plates and biochemical confirmation) in sample M19 processed

on the same day, it is possible that the laboratory could interchange samples.

Results of the two samples from lab 7, level 1 1: The results obtained during the reading of the sensor

indicate the absence of Salmonella spp.; it shows growth of colonies characteristic of Salmonella spp on

the two plates used. The laboratory indicates that with regard to these 2 samples, an anomaly was

found due to the incubation during the first stage of Qfast which does not comply with the temperature

range established 37ºC ± 1ºC. This deviation due to the incubation process explains that it has not

reached the sufficient number of microorganisms to be detected by sensor. That subsequent incubation

of the inoculum in the XLD and ASAP plates as well as in the biochemical tests carried out confirms the

presence of Salmonella spp.

With regard to the result obtained by 4, level 2 in sample M24, the sensor indicates the absence of

Salmonella spp. and a possible growth of characteristic colonies on the XLD plate and second test. The

laboratory confirmed correct results in all samples linked with Salmonella spp. except from the sample

from the second stage. In this case the laboratory did not send additional information which allow to

clarify the reason of this negative deviation.

de 56

4.4- INTERPRETATION OF RESULTS FROM THE COLLABORATIVE SUTUDY AND COMPARISON OF THE

METHODS

4.4.1. COMPARISON OF RELATIVE EFFICIENCY (AC), RELATIVE SPECIFICITY (SP) AND RELATIVE

SENSITIVITY (SE) VALUES

In theTable 33 show the results obtained during the collaborative study carried out in powdered milk

and global validation.

Table 33. Comparison of collaborative study carried out in powdered milk and comparison of

methods

Collavorative study

Comparasion of methods

Relative Effciency (AC) 93,68 % 97,62 %

Relative Sensitivity (SE) 97,73 % 96,80 %

Relative Specificity (SP) 94,32 % 98,43 %

4.4.2.- ACCORDANCE, CONCORDANCE AND CONCORDANCE ODDS RATIO(COR)

The Accordance is the percentage chance of finding the same result from two identical test portions

(both positive and negative), in the same laboratory under repeatibility conditions.

In order to find this information regarding the data obtained in a collaborative study, the probability

that two samples give the same result is calculated based on the labs participants. Once probabilities

are obtained this are averaged.

The Concordance is the percentage chance of finding the same result for two identical samples analysed

in two different laboratories.

For the calculation, they take the replication of each laboratory, matching the identical results with the

others obtained in the laboratory. The concordance is the average of all data matched from the same

results with all the possible pairs of data.

The Concordance odds ratio: Although in a qualitative method, it is difficult to calculate the grade of

collaborative variation, as the conformity and the concordance are strongly influenced by the efficiency

grade, it is possible to calculate what is called the relative concordance opportunity (COR), calculated

according to the following equation:

de 56

In table 34 the results obtained in the collaborative performed with the powdered milk

If the concordance is smaller than the accordance, it indicates that two identical samples are more likely

to give the same result if they are analysed by the same laboratory than if they are analysed by different

ones, suggesting that there can be variability in performance between laboratories.

Table 34: Results of Accordance, concordance and concordance odds ratio in the collaborative of the

powdered milk

QFast® Salmonella UNE-EN ISO 6579

Level

Accordance

Concordance

COR

Accordance

Concordance

COR

Level 0 92,33% 88,98% 1,49 93,47% 88.86% 1,79

Level 1 96,59% 95,45% 1,35 100,00% 100,00% 1,00

Level 2 96,02% 95,51% 1,13 100,00% 100,00% 1,00

Conclusions:

The results de efficacy, selectivity and sensitivity obtained in the collaborative study are good, all those

above 90%. The sensitivity is greater than the specificity due to the fact that the exercise has produced

more false positives than negatives (5 false positives compared to 4 false negatives).

This also explains that the grade of correspondence between the response of the alternative method

and the reference method is greater for the samples inoculated with an efficacy (AC) of 97.73% than for

the non-inoculated samples with an efficacy (AC) of 89.64%,

de 56

As the accordance is smaller than the conformity, the probability indicates that two identical samples

have more possibilities to give the same result if they are analysed by the same laboratory than if they

are analysed by different ones.

The positive deviations produced in both methods in the level 0 are included in the COR values, these

values are raised in both methods and indicate a high inter-laboratory variation, due in this case to the

cross contaminations.

In the detection of inoculated samples there does not appear to be a significant influence in the inter-

laboratory variability as the COR is close to 1.

In accordance with the statistical study of the discordant results, we concluded that the methods are

equivalent.

de 56

5.-COLLABORATIVE STUDY 2013

5.1.- ORGANIZATION OF THE STUDY

The collaborative exercise study was organised in September 2013. The number of participating

laboratories was 14.

The chronogram of the collaborative was the following:

11th-12th September 2013: Reception of the faecal samples in the laboratory

Week of the 16th of September 2013: start of the tests lot A Faeces. (Samples H1 to H12)

Week of the 23rd of September 2013: start of the tests lot B Faeces. (Samples H13 to H24)

There were no incidents in the chronogram.

5.1.1.-Origin of the preparations

A sample was selected that came from a single lot, consisting of farm faeces from ecological production

that underwent a process of dehydrated.

With regard to this sample, the absence of Salmonella spp. was verified, via the application of the

reference method. In this test the presence of Salmonella was not found in any of the samples analysed.

There are data available of the natural contamination of the faeces obtained via the count of the

enterobacteria in VRBG with the result of 1.9 x 106 cfu/g.

The faeces sample was sent in aliquots of 6 grams, in a sterile anaclin flask (to perform the test of the

alternative method QFast® and in a sterile bag (to analyse with the reference method), with the

objective of facilitating the handling of the sample. These samples should be re-hydrated according to

the instructions that are sent with the samples immediately before being analysed.

At the same time, the powdered milk is prepared, as an inoculation substrate of the Salmonella. The

absence of Salmonella spp. in the powdered milk was also checked.

de 56

5.1.2.-Inoculation rate

The inoculation of powdered milk that should be used by the laboratories to add to the faeces, was

prepared in a way to obtain three different levels:

Table 35- Levels of inoculation of Salmonella spp.

Level Media Count of Salmonella spp. en 25 g Confiance interval (cfu/ 25 g)

L0 0 -

L1 49 cfu [34-70]

L2 130 cfu [90-190]

The inoculum was prepared with the addition of Salmonella entérica from the strain CECT 4594.

The stability tests carried out during the test period confirmed the survival of the microorganism in all

the samples analysed that had been contaminated.

5.1.3.-Problems with the temperature during the transport and the time of reception.

There were no problems registered at the time of transport of the samples to the different laboratories.

On the other hand in some laboratories, the reaction solution of the alternative method QFast® which

should be stored between -21 and -5ºC, did not reach these temperatures and/or in the laboratory it was

stored in the fridge instead of the freezer.

de 56

5.2.- RESULTS OF THE ANALYSIS

The summary of the results obtained by the laboratories are shown in table 36. The results for the

alternative method are after the confirmation.

Table 36- Coincidence of the results for each one of the methods applied and level.

Laboratory Alternative method QFAST Reference method UNE-EN ISO 6579/A1

Nº Positive Samples Nº Positive Samples

Level 0 Level 1 Level 2 Level 0 Level 1 Level 2

1 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

2 0 / 6 2 / 2 8 / 8 0 / 8 8 / 8 8 / 8

3 2 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

4 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

5 0 / 8 8/ 8 8 / 8 0 / 8 8 / 8 8 / 8

7 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

8 4 / 8 7 / 8 8 / 8 0 / 8 8 / 8 8 / 8

10 1 / 8 5 / 8 8 / 8 0 / 8 8 / 8 8 / 8

11 0 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

12 2 / 8 7 / 8 8 / 8 0 / 8 8 / 8 8 / 8

13 3 / 8 8 / 8 7 / 8 0 / 8 8 / 8 8 / 8

15 2 / 8 8 / 8 8 / 8 0 / 8 8 / 8 8 / 8

16 1 / 8 8 / 8 7 / 8 0 / 8 8 / 8 8 / 8

FP TP TP FP TP TP

15 93 102 0 104 104

N 102 98 104 104 104 104

The laboratory nº 6 and 9 did not participate in the end in the faeces collaborative study.

The laboratory nº 2 did not analyse all the samples due to a misconfiguration of the reader,

according to that declared by the participant.

The laboratory nº 14 was excluded from the interpretation

FP: Number of false positives

TP: Number of true positives

N: Number of analysis carried out at each level

de 56

5.3.- CALCULATIONS

5.3.1.- CALCULATIONS OF THE PERCENTAGES OF SPECIFICITY (SP), SENSITIVITY (SE) AND RELATIVE

EFFICIENCY (AC)

Calculations of the Specificity (SP) and Sensitivity (SE)

The specificity and sensitivity calculations of both methods are presented in Table 37, including the

upper (LCS) and lower (LCL) intervals, for values obtained below 90%, as applicable.

Level L0:

Level L1 and L2:

Being:

FP: False positives

TP: True positives

N: Number of samples

Table 37: Specificity (SP) and Sensitivity (SE) of the alternative and reference methods.

Specificity/

Sensitivity

Alternative

Method

QFast®

LCS LCL

Reference

Method

UNE-EN ISO

6579/A1

LCL

SP (Level L0) 85,15% 93,4% 76,90% 100,00% 100,00%

SE (Level L1) 94,90% - 90,45% 100,00% 100,00%

SE (Level L2) 98,08% - 95,38% 100,00% 100,00%

SE (Level L1+L2) 96,53% - 93,96% 100,00% 100,00%

de 56

Calculations of the Relative Exactitude (AC)

The comparison of the pairs of results of the different levels of contamination are summarised in table

38.

Table 38: Results obtained for both methods

Level Alternative Method

QFAST® Salmonella

Reference Method

UNE-EN ISO 6579/A1

Positive Samples Negative Samples Total

Level 0

Positive Samples PA= 0 PD= 15 15

Negative Samples ND= 0 NA= 87 87

Total 0 102 102

Level 1

Positive Samples PA= 93 PD=0 93


Total 98 0 98

Level 2



Total 104 0 104

Level 1+2



Total 202 0 202

Note-PA: positive concordances; NA: negative concordances; ND: negative deviations; PD positive

deviations.

The values of relative efficiency (AC) for the different levels of contamination, are shown in table 39,

together with the corresponding inferior critical detection limit LCL.

de 56

The relative efficiency is calculated according to the following formula:

Being:

PA: Number of positive concordance

NA: Number of negative concordance

N: Total number of samples analysed

Table 39: Relative efficiency (AC) values and confidence intervals

Level

Evaluation

Parameter

Result

Level 0

AC 85,15%

LCS 93,40%

LCL 76,9%

Level 1

AC 94,32%

LCL 89,38%

Level 2

AC 96,00%

LCL 92,08%

Level 1+2

AC 95,21%

LCL 92,10%

de 56

5.3.2.-STUDY OF THE DISCORDANT RESULTS

False positives

The participants do not have extensive experience in the development of this method. This lack of

experience has provoked a cross contamination between the spiked samples and non-spiked samples

(possibly at the time of the read when the sensor was placed by hand). The practice, experience and the

skill in the development of the procedure avoids this problem.

Note extension 2014-2015: The selected laboratories are technically competent to carry out both methods (reference and

alternative), as shown in the results obtained. The level of experience indicated in the previous paragraph does not put the

technical competence of the collaborating laboratories in doubt.

False negatives

The laboratory taking into account these discordant results (negative) carry out the corresponding

investigation of the possible causes arriving at the conclusion that the negative result given by the

reader in these artificially contaminated samples could be due to the fact that at the time of inoculating

the dropper in the sensor, it did not reach the mark indicated due a lack of capillarity.

Note 2014-2015: These deviations were reported to the manufacturer, who took necessary actions to improve the design of

the sensor to avoid the problems of a lack of capillarity. This action was carried out once the validation report was finished,

thus it was not included in the report.

de 56

5.4- INTERPRETATION OF THE RESULTS OF THE COLLABORATIVE STUDY AND COMPARISON METHODS

5.4.1. COMPARISON OF THE RELATIVE EFFICIENCY (AC) VALUES, RELATIVE SPECIFICITY (SP) AND

RELATIVE SENSITIVITY (SE)

In table 40 the results obtained in the collaborative exercise in faeces and in the initial validation are

shown.

Table 40. Comparison of the collaborative study and comparison of the initial methods

Colaborative Study Faeces Comparison methods 2013

Relative Efficiency (AC) 96,53% 98,3%

Relative Sensitivity (SE) 96,53% 99,7%

Relative Specificity (SP) 85,71% 96,00%

In the data collected in this table, it appears that the values AC, SE and SP are higher in the Comparative

Study than in the collaborative study. The main experience of the organising laboratory when

performing the method explains the small deviations between the collaborative study and the

Comparative Study.

5.4.2.- ACCORDANCE, CONCORDANCE AND COR

In table 41 the data of accordance, concordance and COR initial collaborative (2013) are shown.

Table 41: Results of Accordance, concordance and COR

QFast® Salmonella UNE-EN ISO 6579/A1

Level

Accordance

Concordance

COR

Accordance

Concordance

COR

Level 0 82,00% 75,32% 1,50 100,00% 100,00% 1,00

Level 1 93,03% 90,13% 1,46 100,00% 100,00% 1,00

Level 2 96,63% 96,19% 1,14 100,00% 100,00% 1,00

de 56

Conclusions:

In level 0 there were 15 false positives. After collecting the information about the handling of the

alternative method, we established that the cause of the false positives is cross contamination between

the spiked samples and the non-spiked samples (possibly at the time of reading when the sensor is

touched with the fingers).

The design of the sensor could explain the negative deviations that could be caused at the time of

reading when inoculating the dropper of the sensor as they must verify that it reaches the marked line;

when the dropper does not reach the line, it does not enter due to the capillarity.

Note 2014-2015: These deviations were reported to the manufacturer, who took necessary actions to improve the design of

the sensor to avoid the problems of a lack of capillarity. This action was carried out once the validation report was finished,

thus it was not included in the report.

de 56

6. AUDITS

Introduction

AENOR performed an annual audit that verifies the maintenance that complies with the requisites

established in the “Regulation of the certification of validated alternative methods for the quality control

and food safety (RP B 59)”.

On one hand, the quality management system is audited in the manufacturing facilities with the

objective of verifying the maintenance conditions established in the rule (RP B59).

On the other hand, annual contrast tests have been performed, in a way that during the 5 years, the

certified method is verified in all the matrices at least once. Prior to the follow-up audit, AENOR

prepares a sample that is divided into aliquots. The auditor gives one of the aliquots to the

manufacturer, starting the analysis of the test in the presence of the auditor. Once the analysis is

finished, the results obtained are sent to AENOR laboratory. In parallel, AENOR laboratory performs the

tests on the second aliquot. The results are compared and report is prepared.

Results of the audit

Year 2013

Audit: audit was performed successfully.

Procedure test 2013: Faeces matrix. Result correct.

Year 2014


Procedure test 2014: mixing matrix with Salmonella spp. added. Result correct.

Year 2015


Procedure test 2015: yoghurt matrix with Salmonella spp. added. Result correct.

de 56

6. APLICABILITY

Reagents and consumables of the kit, storage conditions and shelf life for the ready to use format

Soft Media 1 (pre-enrichment liquid media) in flasks (225 ml). Store between 2 and 8 ºC. Expiration date 15 months from production.

Hard Media 1 (pre-enrichment liquid media) in flasks (225 ml). Storage between 2 and 8 ºC. Expiration date 15 months from production.

Supplement Hard Media 1 in glass packaging with dropper (50ml). Storage between 4 and 30ºC. Expiration date 15 months from production.

Media 2 (pre-enrichment liquid media) in eppendorf tubes of 1.5 ml. Storage between 2 and 8 ºC. Expiration date 15 months from production.

Immunocapture Solution in bottles of 3 ml with dropper. Storage between 2 y 8 ºC. Maximum expiration date approx. 2 years from production.

Wash Solution in bottles of approximately 50 ml with dropper. Storage between 2 and 8 ºC. Expiration date 1 year from production.

Reaction Solution in plastic bags. Contains two droppers with dispensers (3 ml), one with the dehydrated solutions and another with the diluent solution. Store between -21 and -5ºC. Expiration date 1 year from production.

Sensor in plastic packaging with silicagel. Storage between 2 and 8 ºC. Expiration date 1 year from production.

Reagents and consumables of the kit, storage conditions and shelf life for the dehydrated format (reconstruction of the culture media required)

Media 1 Soft TBR (pre-enrichment dehydrated media) in plastic packaging (500 g). Store between 4 and 30 ºC. Expiration date 3 years from production.

Supplement Soft Media 1 in glass packaging with dropper (50ml). Store between 4 and 30ºC. Maximum expiration date approx. 1 year from production.

Hard Media 1 TBR (pre-enrichment dehydrated media) in flasks (225 ml). Store between 2 and 8 ºC. Expiration date 3 years from production.

Hard Supplement Media 1 in glass packaging with dropper (50ml). Store between 4 and 30ºC. Expiration

de 56

date 15 months from production.

Media 2 TBR (pre-enrichment dehydrated media) in plastic bags (2.8 g). Store between 4 and 30 ºC. Expiration date 3 years from production.

Immunocapture Solution in bottles of 3 ml with dropper. Store between 2 y 8 ºC. Maximum expiration date approx. 2 years from production.

Washing Solution in bottles of approximately 50 ml with dropper. Store between 2 and 8 ºC. Expiration date 1 year from production.

Reaction Solution in plastic bags. Contains two droppers with dispensers (3 ml), one with the dehydration solution and another with the diluent solution. Store between -21 and -5ºC. Expiration date 1 year from production.

Sensor in plastic packaging with silicagel. Store between 2 and 8 ºC. Expiration date 1 year from production.

Specific Equipment Specific for the method QFast®: Magnetic Separator iMICROQ Incubator with agitation iMICROQ Electronic Reader iMICROQ Normal material that exists in a microbiology laboratory: Bunsen Burner, heaters, balances, vortex, etc. For the reconstitution of the dehydrated media and required for the autoclave and plate heater.

Communal Operations Time 1 sample (min) Time 24 samples (min) Sample. Weight, mix 3 75 Operations QFast Transfer media 2 0.5 10 Inmunoseparation and wash 4 96 Reaction Solution 0.1 2 Insertion reader + reader 1.1 24 TOTAL 5.7 132

de 56

7.- FINAL CONCLUSION

The conclusions of the Comparative Study are:

The method QFast® has a satisfactory relative efficiency, relative specificity and relative sensitivity. With

regard to the results obtained, we conclude that the method QFast® is equivalent to the reference

method.

The levels of detection of the alternative method QFast® and the reference method are similar.

The exclusivity and inclusivity are totally satisfactory for both media used and the alternative method.

The conclusions of the collaborative study are:

The results obtained in the exercise carried out in 2014 improve those obtained in the exercise carried

out in 2013.

The results obtained in the exercise 2014 of efficacy, specificity and relative sensitivity are similar to

those obtained in the Comparative Study. The accordance and concordance of the alternative method, is

above 90%.

With regard to the results obtained in the present validation, we establish that both methods

are equivalent.

The conclusions of the audit performed are:

Both the results of the audit performed as well as the performance tests have been satisfactory

in the years 2013, 2014 and 2015.

Madrid, 14/07/2016

Agustina Sanchez Díaz

Tecnhical Director

de 56

Annex 1–Protocol of the alternative method QFast®

de 56

Annex 2 – Reference Method

Reference Method ISO 6579:2002-Microbiology of food and animal feeding stuffs. Horizontal method for the

detection of Salmonella spp

xg or ml of product

Dilution to 1/10 in APT

Incubation at 37°C 1°C for 18 h 2 h

Add 0.1 ml of mother solution incubated at 1 ml mother solution incubated at

10 ml RVS broth 10 ml MKTTn broth

Incubation at 41.5°C 1°C, 24 h 3 h Incubation at 37°C 1°C, 24 h 3 h

XLD and ASAP Agar (as second chromogenic media)

Incubation 37°C 1°C a 24 h 3 h

Isolate 1 characteristic colony

(For each agar media) and 4 colonies more in case the first one is negative

Nutrient Agar

Incubation 24 h 3 h a 37°C 1°C

Interpretation of the results

Serological Confirmation serológica

Biochemical Confirmation

de 56

Annex 3 - Reference Method

Reference Method ISO 6579/A1- Microbiology of food and animal feeding stuffs. Horizontal method for the

detection of Salmonella spp. Amendement 1: Annex D. Detection of Salmonella spp. in animal faeces and in

environmental samples from the primary production stage.

Pre-enrichment: APT

Dilution 1/10

Incubation at 37°C 1°C for18 h 2 h

Deposit separately

3 drops on the MSRV agar

(0.1 ml in total)

Incubation at 41.5°C 1°C for 24 h 3 h

Read plates:

Positive Test: migration area white-greyish from the drop inoculated.

Negative Test: re-incubate the other plates 24 h 3 h

Pick with a loop of 1μl in the area outside of the migration area

Isolate on XLD agar

Incubation at 37°C 1°C, 24 h 3 h

Read

Select a characteristic colony for isolation

Isolate on nutrient agar

Incubate at 37°C 1°C

24 h 3 h

Biochemical confirmation (miniaturised gallery) and serological confirmation

validation une-en iso 16140 method qfast® … › laboratoriodocumentos › qfast...une -en iso...

Documents