Validation of Alternative Methods Food Labs in Crystal Ball – Future Challenges in Food Analysis, 21-22 May 2015 AOAC EUROPE - NMKL - NORDVAL INTERNATIONAL SYMPOSIUM 2015

Dr Gail Betts Campden BRI, Chipping Campden, Gloucestershire, GL55 6LD, UK gail.betts@campdenbri.co.uk

Why am I interested in Method Validation? • I work in the Microbiology Department at Campden BRI in UK

• We hold ISO 17025 accreditation for a range of microbiological tests and test for a large number of clients

• We have 2400 member companies in 75 countries who rely on us for advice. Particularly on testing.

• We help companies develop and implement new test methods • We are an expert laboratory for validation bodies such as

NordVal, MicroVal & AOAC RI (30 validations)

Validation of Alternative Methods -

• There are currently many different test methods available, – for example in a review of rapid methods we found over 70

methods available for Salmonella!

• So how does a laboratory decide which method to use…..In many cases the validation status is the most crucial factor of choice

• This talk will examine – What is method validation – Why it is important to validate – How is validation done (according to ISO 16140) – Things to look out for when doing a validation

What is method validation?

‘validation’ establishment of the performance

characteristics of a method and provision of objective evidence that the performance requirements for a specified intended use are fulfilled

EN ISO 16140-1 Microbiology of food and animal feed - Method validation Part 1: Vocabulary

How is method Validation done? ISO 16140 – part 2

Why is method validation important?

• European Food Law : Food Business Operators (FBOs) are obliged to comply with the microbiological criteria set out in Commission Regulation 2073/2005

www.fsai.ie/uploadedFiles/Consol_Reg2073_2005.pdf

CRITERIA •Food category •Microorganism •Sampling plan •Limits •Method to be used •Stage of production where criterion applies •Action to be taken if results unsatisfactory

2073/2005- Alternative methods equivalency Article 5—2073/2005 • The use of alternative methods is acceptable when the

methods are validated against the reference method in Annex 1 and if a proprietary method, certified by a third party in accordance with the protocol set out in EN/ISO 16140 or other internationally accepted similar protocols

• If the FBO wishes to use analytical methods other than those validated and certified as described in paragraph 3 the methods shall be validated according to internationally accepted protocols and their use authorised by the competent authority

Third Party Validation Systems No single validation system that satisfies users

throughout the world – although there are many ‘harmonisation talks on-going

Europe NordVal AFNOR MicroVal USA AOAC International Official Methods Program AOAC Research Institute Performance Tested Methods

Program

• ISO Standard (EN ISO 16140): Microbiology of food and animal feeding stuffs- Protocol for the validation of alternative methods

EN ISO 16140

SCOPE •This part of ISO 16140 specifies the general principle and the technical protocol for the validation of alternative, mostly proprietary, methods in the field of microbiological analysis of food, animal feed, and environmental and primary production stage samples.’

•‘Validation studies according to this standard are intended to be performed by organisations involved in method validation

ISO 16140:2013 : Quantitative Methods Method Comparison • 5 food categories for a ‘broad range of foods’ Relative trueness • 5 samples of 3 food types per 5 categories = 75 samples Accuracy profile • 3 contamination levels per category with 2 samples at

each level with 5 replicates = 150 samples Inclusivity/Exclusivity (not applicable to total count

determinations) • 50 target strains - 30 non target strains Inter-laboratory Study • Min - 8 sets of lab results • 1 food, 3 levels, 2 replicates (min)

Food Categories (ISO 16140)

15 food categories e.g. • Meat –raw, RTE, RTRH • Poultry – raw, RTE, RTRH • Fish - raw, RTE, RTRH • Produce, fresh, processed, dried • Dairy – raw and heat processed • Chocolate/bakery/confectionary • Multi-components foods • Pet food/Animal feeds • Environmental and Primary production

ISO 16140‘new’ version : Quantitative Methods Method Comparison : Relative trueness • 5 samples of 3 food types per 5 categories = 75 samples

Category Types Number of samples Dairy products Dairy desserts e.g. chilled custard, trifle 5

Soft cheese 5 Hard cheese e.g. cheddar 5

Fruits and vegetables Seasonings e.g. spices 5 Sprouts e.g. mung beans 5 Leafy greens e.g. parsley, lettuce 5

Raw poultry and meats Fresh chicken cuts 5 Fresh mince 5 Frozen patties 5

Ready to eat foods Ready to eat poultry e.g. turkey fillet, chicken sausage 5 Cooked fish products e.g. prawns 5 Cooked meat e.g. ham 5

Multi component foods Composite foods with raw ingredients e.g. sandwiches, pasta salads.

5

Mayonnaise based salads 5 Cooked chilled foods e.g. rice products 5

ISO 16140 ‘new’ version : Quantitative Methods

Method Comparison : Accuracy profile • 3 contamination levels per category with 2 samples at

each level with 5 replicates = 150 samples

-0,60

-0,40

-0,20

0,00

0,20

0,40

0,60

0,00 2,00 4,00 6,00Bia

s

Reference Median

Fresh produce

Bias

β-ETI

AL = +/- 0.5

Category Types Item Level Test portion

Dairy products

Dairy desserts

Chilled custard

Low:100cf/g 1-5 Batch1 1-5 Batch2

Medium : 1000cfu/g 1-5 Batch1 1-5 Batch2

High : 10,000cfu/g 1-5 Batch1 1-5 Batch2

Fruits and vegetables

Leafy greens e.g. parsley, lettuce

Parsley

Low: 50cf/g 1-5 Batch1 1-5 Batch2

Medium : 1000cfu/g 1-5 Batch1 1-5 Batch2

High : 50,000cfu/g 1-5 Batch1 1-5 Batch2

Raw poultry and meats

Fresh beef Fresh steak Low: 50cf/g 1-5 Batch1 1-5 Batch2

Medium : 1000cfu/g 1-5 Batch1 1-5 Batch2

High : 50,000cfu/g 1-5 Batch1 1-5 Batch2

Ready to eat foods

Cooked fish products e.g. prawns

Fish pate Low: 50cf/g 1-5 Batch1 1-5 Batch2

Medium : 100cfu/g 1-5 Batch1 1-5 Batch2

High : 1000cfu/g 1-5 Batch1 1-5 Batch2

Multi component foods

Composite foods with raw ingredients

Pasta salad Low 500cf/g 1-5 Batch1 1-5 Batch2

Medium : 5000cfu/g 1-5 Batch1 1-5 Batch2

High : 50,000cfu/g 1-5 Batch1 1-5 Batch2

ISO 16140 : Qualitative Methods overview Stage 1: Comparison study • 5 food categories • 60 samples per category (~ 50% positive) Inclusivity/Exclusivity • 50 target organisms

(100 for Salmonella) • 30 non target organisms • specificity, sensitivity and relative accuracy Relative Level of Detection Stage 2: Inter-laboratory Study • 10 sets of lab results • 1 food, 3 levels, 8 replicates

Responses Reference Method Positive (R+)

Reference Method Negative (R-)

Alternative method positive (A+)

+/+ positive agreement (PA)

-/+ positive deviation (PD)

(R-/A+) Alternative method negative (A-)

+/- negative deviation (ND)

(A-/R+)

-/- negative agreement (NA)

NordVal AOAC RI MicroVal AFNOR Technical rules

√ √

√ √

Based on ISO 16140

√ √ √ √

Food Claims Broad range Specific foods

Broad range

Broad range

ILS √ × √ √ Protocol written by

Expert lab Technical consultants

Expert lab Expert lab

Acceptance criteria

In NordVal protocol

Not defined ISO 16140:2013

Not defined

Certificates renewed

2 years 1 year 4 years 4 years

Be aware that

• Validation of water methods is not covered in ISO 16140

• Separate standard covers waters – ISO 17994

• Different experimental design and different statistics.

Compact Dry ETC for water Campden BRI & NordVal 2014

• Combined experimental design and statistics of ISO 16140 and ISO 17994

• (i) Campden BRI as the expert lab did typical 5 products x 5 replicates x 5 contamination levels

• (ii) 8 collaborative labs tested 3 levels x 2 replicates • ISO 16140 statistics was done on linearity and

accuracy data from (i) • ISO 17994 statistics was done on whole data set

from all the labs

Compact Dry ETC for water

Validation of confirmation methods

• Is currently not covered in ISO16140 but part 6 will cover this

• ISO16140-6 'Microbiology of the food chain - Method validation - Part 6: Protocol for the validation of alternative methods for microbiological confirmation and typing'

• AOAC : Appendix J – does include Confirmatory Identification methods

Things to look out for – expert laboratory

• Even though all accreditation schemes are based on ISO 16140- each one has its own technical protocol so make sure you get the most up to date copy before you start writing a protocol

• Manage the expectations of the kit manufacturer if they are not familiar with the validation time-frame. It takes on average 12 months for a validation to be completed

• If doing a joint validation, start discussions with both schemes as early as possible to design a single acceptable protocol

Things to look out for : kit manufacturer • Make sure the kit is finalised before the validation

starts : do not use a validation as a method development exercise

• Do not change test method or kit instructions during a validation. This could cause delays or extra costs or could invalidate the study

• Be familiar with the scheme rules as these will highlight the manufacturers responsibility

• If existing validation was done under ISO16140 :2004, then you may need to do further studies when it is due for renewal to meet ISO16140 :2013

Things to look out for : end user • If testing under legislation and wanting to use a proprietary

method – Don’t just look at a Third party website and use any ‘validated’ for

the organism of interest.

• Check the validation certificate and report and ensure that the scope of the validation covers both the food types and organisms you wish to test for. – Issues- examples

• There is at least one method in a certification bodies list of validated methods that is noted not to detect non-motile Salmonella– but is validated and certified to detect motile Salmonella.

• Always check the food groups you wish to test have been included as a part of the validation – ‘all foods’ or specific matrix claim

• Always verify a validated methods use within your laboratory

Final thoughts........Importance of Validation to an end-user

“An official validation based on ISO 16140…is a must for our laboratories…. the food categories are very important for us…therefore we work with suppliers to validate our matrices…As for inclusivity test, because of some historical data, we are interested to have some specific strains on the list”

Further information on third party schemes:

Please get in touch.... gail.betts@campdenbri.co.uk

NordVal : www.nmkl.org

MicroVal: www.microval.org

AFNOR: www.afnor-validation.com

AOAC: www.aoac.org