sop how to assess a pqs testing laboratory

8/10/2019 SOP How to Assess a PQS Testing Laboratory

1/22


2/22

SOP No FCH/ IVB/QSS/PQS/007 Date of issue: 08/07/2004 Revision Date:

06/01/2007

Version: 01.05 Effective date: 08/07/2004 Page: 2 of 22Title: How to assess a PQS testing laboratory

Table of Content

1.0 Purpose.......................................................................................................................... 3

2.0 Scope............................................................................................................................. 3

3.0 Responsibility ............................................................................................................... 34.0 Documentation required................................................................................................ 4

5.0 Procedure ...................................................................................................................... 4

5.1 Assessment Register ........................................................................................... 5

5.2 Make initial inquiries and check ISO/IEC 17025 accreditation ......................... 65.3 Send assessment information pack ..................................................................... 6

5.4 Information dossier ............................................................................................. 6

5.5 Evaluate facilities and expertise.......................................................................... 65.5.1 ISO/IEC 17025 accreditation:..................................................................... 6

5.5.2 ILAC accreditation: .................................................................................... 7

5.5.3 References:.................................................................................................. 75.5.4 Facilities and CVs:...................................................................................... 7

5.6 On-site inspection ............................................................................................... 8

5.7 Evaluation results................................................................................................ 8

6.0 Distribution ................................................................................................................... 9Annex 1 Definitions....................................................................................................... 10

Annex 2 References ....................................................................................................... 10

Annex 3 Standard letter A: Assessment information pack............................................ 11Annex 4 Standard letter B: Request for missing information........................................ 13

Annex 5 Standard letter C: Request for reference ......................................................... 14

Annex 6 Standard letter D: Request for laboratory inspection visit .............................. 16Annex 7 Model checklist for a laboratory inspection visit ............................................ 17

Annex 8 Evaluation checklist ........................................................................................ 18

Annex 9 Standard letter E: Assessment rejection.......................................................... 19Annex 10 Standard letter F: Acceptance ....................................................................... 20

Annex 11 Format for listing a PQS-assessed testing laboratory on the PQS website .... 21

REVISION HISTORY FORM ......................................................................................... 22


3/22


06/01/2007


1.0 Purpose

Before a product or device can be added to the PQS database, it must be verified.

Verification establishes whether a specific product from a specific manufacturer satisfies

the requirements of the relevant PQS performance specification1. There are three distinctmethods:

1. Type-examination.

2. Independent type-testing.3. Full Quality Assurance.

An accredited testing laboratoryis an organization that can perform one or more of thefunctions outlined above. In addition it must be able to demonstrate its competency by

conforming to appropriate internationally or nationally accepted standards or codes of

practice and must be able to show to a competent third-party accreditation body that it

does so; usually by peer assessment2.

All independent type-testingmust be carried out by an accredited testing laboratory. In

addition accredited testing laboratories may sometimes be employed to carry out type-examination orfull quality assurance of PQS products. This SOP describes how to assess

whether a testing laboratory has the relevant competencies. Laboratories that satisfy the

assessment process may then be accepted by WHO to test specific categories of PQSproducts.

The procedures set out in this SOP will be followed by the PQS Secretariat(Secretariat),by the PQS Steering Group (PQS-SG) and by any WHO employee or consultant

appointed to inspect a testing laboratory.

2.0 Scope

This SOP is applicable whenever a testing laboratory is being considered for assessment

under the PQS initiative.

3.0 Responsibility

Responsibilities and tasks will be assigned as follows:

1. The PQS Secretariat (Secretariat) will identify potentially suitable testinglaboratories and will send out an assessment information pack.

1Refer to SOP No FCH/ IVB/ATT/VML/004:How to develop and publish a PQS product verificationprotocol.2Adapted from Testing, Quality Assurance, Certification and Accreditation. ILAC-I2:1994.


4/22


06/01/2007


2. The Secretariat will analyze the information dossier supplied by the testing

laboratory. If the dossier is satisfactory, the Secretariat may then arrange for asuitably qualified WHO employee or consultant to visit the laboratory.

3. Based upon the visit report, the Secretariat will make a recommendation to thePQS Steering Group(PQS-SG).

4. The PQS-SG will direct the Secretariat whether or not to accept the test

laboratory.

5. The Secretariat will notify the test laboratory of the PQS-SGs decision.

4.0 Documentation required

Associated documents: Refer to the following:

SOP No FCH/ IVB/QSS/PQS/001: How to develop and publish a PQS product

performance specification.

SOP No FCH/ IVB/QSS/PQS/002: How to review and revise a PQS product

performance specification.

SOP No FCH/ IVB/QSS/PQS/003:How to withdraw a PQS product performance

specification.

SOP No FCH/ IVB/QSS/PQS/004: How to develop and publish a PQS product

verification protocol. SOP No FCH/ IVB/QSS/PQS/005: How to review and revise a PQS product

verification protocol.

SOP No FCH/ IVB/QSS/PQS/006:How to withdraw a PQS product verification

protocol.

SOP No FCH/ IVB/QSS/PQS/008: How to re-evaluate PQS-assessed testing

laboratory.

5.0 Procedure

Each of the following task headings includes (in brackets) a description of the person or

group responsible for the task. Refer to Annex 1 for the definitions of certain critical

terms used in this SOP. Figure 1 outlines the procedure described in detail below.


5/22


06/01/2007


Figure 1 Procedure for pre-qualifying a testing laboratory

no

yes

Laboratoryresponds

Facilitiesand expertise

suitable?

Accreditationbodyan ILAC

signatory?

On-site inspectionoptional

Check against currentlist at www.ilac.org

yes

no

no

yes no

yes Rejectlaboratory

Laboratoryacceptable?

Make initialcontact

Start

On-site inspectionessential

LaboratoryISO/IEC 17025

accredited?

Test laboratory may applyto WHOor WHO may invite applications

PQS Secretariatsends ApprovalInformation Pack

Acceptlaboratory

5.1 Assessment Register (Secretariat): The Secretariat will establish and maintain a

register of testing laboratories which includes a comprehensive list of testing laboratories

and which records details of every application for assessment. Copies of allcorrespondence with testing laboratories will be kept in the register.


6/22


06/01/2007


5.2 Make initial inquiries and check ISO/IEC 17025 accreditation (Secretariat):Candidate testing laboratories may formally apply to WHO for assessment. Alternatively

WHO may choose to make the initial contact. The initial inquiry will establish the

laboratorys accreditation arrangements and its general area of expertise. No testinglaboratory will be accepted to carry out type-examination, independent type-testing or

full quality assurance unless it is already accredited in accordance with ISO/IEC

17025:1999 General requirements for the competence of testing and calibration

laboratories3. This document sets out all of the requirements that testing and calibrationlaboratories have to meet to demonstrate that they operate a quality system, are

technically competent, and are able to generate technically valid results4. If initial

enquires show that a laboratory is not ISO/IEC 17025 accredited then it must be rejected.

5.3 Send assessment information pack (Secretariat): All testing laboratories that are

considered for assessment will be sent an assessment information packby the Secretariat.This will contain, as a minimum, the following information:

PQS performance specification(s) and PQS verification protocol(s) relevant to the

laboratorys expertise;

a covering letter outlining the assessment process and listing the information

required from the testing laboratory. In addition the letter will set out formal

terms and conditions. Standard letter A will be used as the basis for thiscorrespondence (Annex 3).

5.4 Information dossier (Testing laboratory): All testing laboratories who seekassessment will submit an information dossier to the Secretariat. This must contain all

the information requested in standard letter A. The Secretariat will screen the dossier for

completeness before it is evaluated. If the dossier is incomplete the laboratory will becontacted in writing and will be given a single opportunity to provide the missing

information or material. Standard letter Bwill be used for this purpose (Annex 4). If,

after a reasonable period has elapsed, the laboratory fails to supply the missinginformation, the dossier will be automatically rejected.

5.5 Evaluate facilities and expertise (Secretariat): Completed information dossierswill be evaluated. The key checks will be as follows:

5.5.1 ISO/IEC 17025 accreditation: Check that the ISO/IEC 17025 accreditation isvalid. Only laboratories that have a valid accreditation can be considered for assessment.

3Under the terms of the ILAC agreement, laboratories operating under earlier standards (ISO/IEC Guide 25and/or EN45001) were required to comply with ISO/IEC 17025:1999 by 31stDecember 2002.4Quoted from the introduction to ISO/IEC 17025:1999.


7/22


06/01/2007


5.5.2 ILAC accreditation: Check whether the accreditation body that issued the

ISO/IEC 17025 accreditation is an ILAC signatory. The International LaboratoryAccreditation Cooperation (ILAC) arrangement came into effect on 31st January 2001.

Its stated aim is to develop a global network of mutually recognized and accredited

testing and calibration laboratories that can be relied upon to provide accurate results sothat products can be tested once and accepted everywhere. As of December 2003 forty

five laboratory accreditation bodies had become signatories to the ILAC arrangement.

All these signatories have been peer-reviewed and have been shown to meet ILACs

criteria for competence. If the accreditation body is an ILAC signatory it may beunnecessary to carry out an inspection of the laboratory, although an inspection is still

advisable. If the accreditation body is not an ILAC signatory, then an inspection is

essential. Laboratories that are accredited by ILAC signatories should receive asurveillance visit every 12 to 18 months and a reassessment visit every 4 to 5 years5.

Other accreditation bodies may have different arrangements.

5.5.3 References:Check the references listed in the information dossier using standard

letter Cand the attached questionnaire (Annex 5). There should be a minimum of at least

three replies.

5.5.4 Facilities and CVs:Check the information in the dossier to establish whether the

testing laboratory has adequate facilities and suitably qualified personnel to perform the

tests shown in the product verification protocol(s) attached to the assessment informationpack. An acceptable laboratory will:

Have the testing equipment described in paragraphs 1.3, 1.5, 1.6, 1.7, 1.8 and 1.9

of EPI Performance Specifications and Test Procedures - Annex 1: Testinginstitutions February 1999 revision.

(not specifically covered by ISO 17025)

Have a laboratory structure and management system that is clearly defined and

which is organized in such a way that the integrity of its staff and operation can be

judged.(ISO Section 4 Management requirements)

Employ suitably qualified laboratory staff, experienced and technically competent

for the work to be undertaken.

(ISO Section 5.2 Personnel)

Use laboratory equipment required for testing against the appropriate WHOproduct verification protocol and which is properly installed, maintained and

calibrated. Adequate records of calibration and servicing must be maintained.

(ISO section 5.4 Test and calibration methods and method validation)

5See ILAC-G10:1996 Harmonised Procedures for Surveillance and Reassessment of Accredited

Laboratories. www.ilac.org


8/22


06/01/2007


Have a testing environment and laboratory suitable for the tests undertaken.(ISO section 5.3 Accommodation and environmental conditions)

Employ laboratory practices that are demonstrable and meet WHO

requirements, such as:

- sample identification; (ISO Section 5.7 Sampling)

- test methods and procedures; (ISO Section 5.4)- supervision of staff; (ISO 5.2.1 and 5.2.3)

- checking of results and calculations. (ISO 4.12.2.1 to .3)

Operate a secure laboratory record system containing full details of all tests

undertaken.(ISO Section 4.3 Document control)

Supply test reports and documents which are accurate, clear and unambiguous and

contain all the relevant information.

(ISO section 5.10 Reporting the results)

The above requirements are those identified as being the optimum requirement;nevertheless, failure to fully comply will not preclude consideration .

If an inspection is necessary the Secretariat will write to the laboratory using standard

letter D (Annex 6).

5.6 On-site inspection (Secretariat/consultant): If an on-site inspection is indicated,

the Secretariat will arrange for a suitably qualified member of staff, or for a consultant, tovisit the testing laboratory. Annex 7 gives a model checklist for such an inspection. The

appointed staff member or consultant will prepare a brief trip report setting out his/her

observations and conclusions.

5.7 Evaluation results (Secretariat): The Secretariat will prepare a brief report and

recommendation on the evaluated testing laboratory, appending the on-site inspectionreport where relevant. This document will be submitted to the PQS-SG. Annex 8 shows

a checklist which may be used to record the results of the evaluation.

5.8 Acceptance (PQS-SG): The PQS-SG, will review the report and recommendationand will direct the Secretariat whether or not to add the testing laboratory to the list of

accepted testing bodies.

5.9 Publication (Secretariat): If the PQS-SG rejectsthe laboratory, it will be notified

of this decision using standard letter E (Annex 9). If the PQS-SG acceptsthe laboratorywill be notified in writing using standard letter F (Annex 10). A copy of the relevant

letter will also be sent to UNICEF-SD and a further copy will be filed in the

Prequalification Register.


9/22


06/01/2007


Relevant details of every accepted testing laboratory will be published electronically in

.pdf format on the PQS website in the format shown in Annex 11. In additionnotification of publication will be posted on the TechNet21 and SIGN e-forums.

6.0 Distribution

This SOP is to be distributed to the following individuals and groups:

PQS Secretariat;

all members of the PQS Steering Group;

any WHO employee or consultant who is appointed to inspect a testinglaboratory.


10/22


06/01/2007


Annex 1 Definitions

email Correspondence by electronic mail. A hard copy will be kept onfile.

IEC International Electro-technical Commission

ILAC International Laboratory Accreditation Cooperation

in writing Where the phrase in writingis used this means correspondence betransmitted by mail or by fax. A hard copy will be kept on file.

ISO International Standards OrganizationQA Quality Assurance.

Annex 2 References

ILAC-12:1994 Testing, Quality Assurance, Certification and Accreditation.

www.ilac.org ILAC-13:1996 The Role of Testing and Laboratory Accreditation in International

Trade. www.ilac.org ILAC-G10:1996Harmonised Procedures for Surveillance and Reassessment of

Accredited Laboratories. www.ilac.org

ILAC-G15:2001 Guidance for Accreditation to ISO/IEC 17025. www.ilac.org

ISO/IEC 17025:1999 General requirements for the competence of testing and

calibration laboratories. www.iso.org

The ILAC Mutual Recognition Arrangement. www.ilac.org WHO/EPI/LHIS/97.03EPI Performance Specifications and Test Procedures -

Annex 1: Testing institutions.February 1999 revision.


11/22


06/01/2007


Annex 3 Standard letter A: Assessment information pack

Dear Sirs,

Application for assessment of an independent testing laboratory under the PQS

initiative: Information dossier

We refer to your dated in which you expressed an

interest in offering the following testing services: (Strike out items that do not apply)

type-examination;

independent type-testing;

full quality assurance

We enclose the relevant performance specification(s) and product verificationprotocol(s) relating to the range of products that we consider you may be qualified to

evaluate: (Strike out items that do not apply)

Type-examination:

< Performance specification(s) description and reference, including revision ref.>

Independent type-testing:< Performance specification(s) description and reference, including revision ref.>

Full Quality Assurance

< Performance specification(s) description and reference, including revision ref.>

We also attach setting out some keyitems of test equipment that will be required for the specific purposes described.

If you wish to be considered for assessment you must provide us with an informationdossier containing the following:

full details of accreditation under ISO/IEC 17025, including next reassessment date;

full details of your accreditation body;

the names and contact details of at least five well-established organizations that haveused your testing services during the past three years.

full details of the laboratory facilities and equipment that are relevant to the enclosed

performance specification(s) and product verification procedure(s);

full details of your quality system;


12/22


06/01/2007


CVs for the personnel who would carry out the work prescribed above;

Confirmation of the range of products you would be able to test, classified by type ofservice (e.g. Type-examination, Independent type-testing or Full Quality Assurance);

If your response to this letter is satisfactory we may wish to inspect your laboratory and

to interview key members of staff before we can confirm acceptance under the PQS

initiative. We also refer you to the general terms and conditions attached to this letter.

Yours faithfully,

TERMS AND CONDITIONS

1. Examination of information dossier: The information dossier will be screened by WHO for

completeness prior to its evaluation. The dossier can be rejected on grounds of incompleteness.Complete dossiers will be retained for evaluation purposes. Incomplete dossiers may be retained for

record purposes.

2. Evaluation results: Completed dossiers will be evaluated. WHO will then write to the testing

laboratory confirming EITHER: that the application for is unsuccessful, OR: that WHO wishes to

make an inspection of the laboratory before reaching a final decision, OR that WHO proposes to

accept the testing laboratory as being suitable to carry out the tasks specified in the letter.

3. Meaning of PQS-assessed status: The conferment of PQS-assessed status means that WHO considers

the testing laboratory to be technically capable of carrying out certain specified work on its behalf.

WHO does not undertake that the testing laboratory will be commissioned to carry out any such work.

4. Payment for testing services: WHO does not itself undertake to pay for testing services. In most

instances testing fees will be paid by the manufacturer of the product under test.

5. Re-evaluation: The testing laboratory will be subjected to a WHO review once a year. Once every 12

months testing laboratories will be required to submit evidence of periodic surveillance or

reassessment of any relevant accreditations or certifications, including ISO/IEC 17025 accreditation,

together with details of any changes in facilities or personnel which may have an impact on the qualityof the service offered by the laboratory.

6. Periodic inspections: Laboratories will also be subject to monitoring and reassessment visits byWHO. The purpose of periodic monitoring and reassessment is to determine whether a laboratory is

continuing to comply with required procedures. Such surveillance visits will usually be undertaken

and completed within one day.

7. Withdrawal of qualification: If PQS-assessed status is withdrawn the reason for this will be

communicated to the laboratory by the WHO. It will be necessary to make formal application for

reinstatement. This application must detail the corrective actions taken to warrant the laboratorys re-

qualification.

8. Confidentiality undertaking: WHO will treat, and will require evaluators of information dossiers to

treat all information to which they will gain access during the evaluation, or otherwise in connection

with the discharge of their responsibilities in regard to the assessment of testing laboratories as

confidential. In addition, the evaluators of information dossiers will be required to sign a Declaration

of Interest. A sample of the confidentiality and declaration of interest undertaking for evaluators of

information dossiers can be obtained on request. If based on this Declaration of Interest, it is felt thatthere is no risk of real or perceived conflict of interest and it is thus deemed appropriate for evaluators

to undertake this work, they will discharge their functions exclusively as advisers to WHO.


13/22


06/01/2007


Annex 4 Standard letter B: Request for missing information

Dear Sirs,


initiative: Information dossierYour reference:

We refer to your letter dated enclosing the information dossier requested in

our letter dated .

We have now made a preliminary examination of the dossier and find that the following

material is missing:

If we do not receive the missing items by we will reject your application.

You will receive no further correspondence from us.

If you have any queries, please contact us as soon as possible.

Yours faithfully,


14/22


06/01/2007


Annex 5 Standard letter C: Request for reference

Dear Sirs,

: Request for reference

WHO is evaluating for its suitability to carry outindependent testing of the following types of product that are used in the Expanded

Programme for Immunization:

The laboratory has given the name of your organization as a reference. We would bemost grateful if you would complete the attached questionnaire and return it to us within

the next four weeks. Your reply will be treated as confidential and will be used solely for

our own internal evaluation of the named testing laboratory.

Please use the other comments section to highlight any issues that are not covered

elsewhere by the questionnaire. We recognize that the response to questionnaires of thistype can be somewhat subjective. If you would like to send us additional material to

support your observations and conclusions we would be grateful to receive it.

Yours faithfully,


15/22


06/01/2007


Annex 5 (cont) Standard letter C: Reference questionnaire

Reference questionnaire for:

Questionnaire completed by: ....................

On behalf of: ....................

Date: ....................

1.For how long have you usedthe testing laboratory? years

2.

Will you continue to use thelaboratory?

If no, please say why:

Yes No

....................

3.What types of product have

been tested for you and forwhat purpose?

Type of product

Purpose of test

4.

Where tests generally

completed on time?

If no, please comment:

Yes No

....................5.Where tests generally

completed on budget?

If no, please comment

Yes No

....................

6.Please rank the following

qualities on a scale of 0-4:

0

(very poor)

1 2 3 4

(excellent)

Quality of test facilities:

Quality of laboratorymanagement:

Quality of staff responsible

for testing:

Quality and accuracy of test

report(s):

Other comments:


16/22


06/01/2007


Annex 6 Standard letter D: Request for laboratory inspection visit

Dear Sirs,


initiative: Request for inspection visit

Your reference:


our letter dated .

We have now made a full examination of your dossier and would like to arrange a visit to

your laboratory.

In particular we wish to inspect the following:

We would also like to interview the following people:

The visit should take no more than one working day. We propose to send and we suggest . Please

advise whether this is convenient and confirm that the relevant members of staff will beavailable for interview. If the proposed dates are inconvenient, please contact us to

arrange an alternative time.

Yours faithfully,


17/22


06/01/2007


Annex 7 Model checklist for a laboratory inspection visit6

Inspection checklist for:

Checklist prepared by:

Checklist completed by (a): :

Date of inspection:

1.Test equipment (b): Comments:

2.Personnel (c): Comments:

3.Reports (d): Comments:

Sample type-examination

report

Sample independent type-testing report

Sample Full QualityAssurance report

4.Overall assessment of the

facility:

Comments

General organization

Quality of buildings

Quality of test equipment

Quality of record-keeping

Orderliness

Cleanliness

Quality of senior staff

Quality of technical staff

Other comments:Notes:

(a) The inspector should have a reference copy of ISO/IEC 17025 available during the inspection.

(b) Make a list of items to be inspected, based on the performance specifications and product verificationprotocols attached to the assessment information pack for example: +43C test chamber.

(c) By reference to the CVs already supplied interview members of staff with the relevant key skills.

(d) Ask to see sample test reports for relevant products and for all three different types of verificationprocedure (or those that are relevant).

6As the laboratory will already be ISO 17025-accredited, the purpose of the exercise is NOT to carry out

an in-depth global inspection. It should only be necessary to look at facilities, to interview staff that aredirectly relevant to PQS activities and to gain a general overall impression of the facility and its

management.


18/22


06/01/2007


Annex 8 Evaluation checklist

Evaluation checklist for: Checklist completed by:

Date:

1. Is laboratory ISO/IEC17025 accredited? Yes No (letter of rejection issued )

2. Date of next re-assessment

3. Name of accreditation body

4.

Is accreditation body anILAC signatory? Yes No

5. If the answer to Q4 is no,

is the accreditation body

acceptable? Yes No

6. Details of references: Ref recd. Date Mean overall result (scale of 0-5)

7. Was an on-site inspection

necessary? Yes No (go to Q9)8. If the answer to Q7 is yes

has an on-site inspection

been carried out? Yes on No

Who carried out the on-site

inspection?

Has an inspection report

been submitted? Yes on No Is the inspection report

appended to this checklist? Yes No (an inspection report is mandatory)

9. What services have beenoffered by the laboratory?

Type-examination: Type-testing: Full Quality Assurance:

10.General comments:

11.Is acceptance of thislaboratory recommended? Yes No

12.Which services should beaccepted?

Type-examination:

Type-testing: Full Quality Assurance:


19/22


06/01/2007


Annex 9 Standard letter E: Assessment rejection

Dear Sirs,


system initiative

Your reference:


our letter dated .

We have now evaluated the dossier and regret to inform you that we are unable to

recommend your laboratory for inclusion on the database of PQS-assessed independenttesting laboratories. A summary of the reasons for this decision are as follows:

We thank you for your application. We regret that we are unable to enter into any

further correspondence on this matter.

Yours faithfully,


20/22


06/01/2007


Annex 10 Standard letter F: Acceptance

Dear Sirs,


system initiative

Your reference:


our letter dated .

OPTION:

We also refer to our inspection visit on .

We have now completed our evaluation and are pleased to inform you that your

laboratory has been recommended for inclusion on the database of PQS-assessedindependent testing laboratories to carry out the following work:

We draw your attention again to the terms and conditions set out in the assessment

information pack sent to you on . These terms and conditions apply to allindependent testing laboratories listed in the PQS database. Note particularly the

requirement to inform us of the periodic renewal of any relevant accreditations or

certifications, including your ISO/IEC 17025 accreditation, together with details of anychanges in facilities or personnel which may have an impact on the quality of the service

offered by your laboratory.

Yours faithfully,


21/22


06/01/2007


Annex 11 Format for listing a PQS-assessed testing laboratory on the PQS website

PQS record for assessed independent testing laboratory

1. LABORATORY NAME:

2. Assessment date:

3. Last re-evaluation date:

4. Next re-evaluation date

5. MAILING ADDRESS

Telephone no.

Fax no.

Email address

Website:

Contact name:

6. ASSESSED SERVICES:

Type-examination

Type-testing Full Quality Assurance

7. SHIPPING ADDRESS

Telephone no.

Fax no.

Email address

Website:

Contact name: 8. DOCUMENTATION REQUIRED FOR SENDING EQUIPMENT FOR TESTING


22/22

sop how to assess a pqs testing laboratory

Documents