siteresources.worldbank.orgsiteresources.worldbank.org/exthsd/images/assessmenttool... · web...

ASSESSMENT TOOL

Based on the Model Quality Assurance System1

INSPECTION OF A PROCUREMENT AGENCY

1. MQAS. WHO TRS 937

Assessment Tool: Procurement Agency. Version 1. 2013.

Table of contents

1. Document history2. Introduction3. Purpose4. Scope5. Assessment tool6. Revision history

1. Document history

Action Date

Preparation of first draft May 2013

2. Introduction

The Expert Committee on Specifications for Pharmaceutical Preparations of the World Health Organization (WHO) adopted a Model Quality Assurance System for Procurement Agencies (MQAS) during a meeting in Geneva, Switzerland in 2005. This was subsequently published as Annex 6 in the Technical Report Series (TRS 937), in 2006.

The Global Fund Secretariat coordinated this project with the aim to prepare a harmonized assessment tool based on the World Health Organization document: Model Quality Assurance System for Procurement Agencies (MQAS); WHO guidelines on Good Storage Practices (GSP) and WHO Guidelines on Good Distribution Practices (GDP). (For current versions, see www.who.int/medicines).

This harmonized tool was developed by a working group consisting of representatives from the following organizations: QUAMED, PFSCM, UNICEF, MSF, IDA, Crown Agents, GDF, MSH, UNION, UNOPS, USAID, ICRC and CHMP.

3. PurposeThis harmonized tool was developed by the working group with the objective that it could be utilized with the aim of better use of resources by coordinating procurement agency (PA) assessments; and working towards mutual recognition of PA assessment findings.

4. Scope

The assessment tool is based on the 6 Modules in the MQAS:Module I General requirements for Procurement AgenciesModule II PrequalificationModule III PurchasingModule IV Receiving and storageModule V DistributionModule VI Re-assessment


http://www.who.int/medicines

The tool covers these topics in different modules below. The logical flow considered is the quality system and infrastructure of the PA under assessment, how the PA performed prequalification, then purchasing of the products followed by the receiving and storage thereof. The last two modules then focus on the receiving of orders and dispatch of products followed by the re-evaluation concept.

5. Assessment toolThe tool should be used by qualified, experienced persons when assessing PA (including wholesalers, distributors) for compliance with recommended international standards. It can also be useful for a PA when doing a self-assessment.

The tool is not checklist, but serves as a document to help and remind inspectors as to what should be assessed during inspections of PAs.

Module I: General requirements for Procurement Agencies

This Module covers general requirements for procurement agencies including premises, equipment, transport, and documentation (such as SOPs, confidentiality, code of conduct and complaint handling).

Area of operation

Note Critical aspects

Premises, Equipment, Furniture, Transport

General Space (offices for personnel,

products, documents, samples etc.)

Suitable conditions Necessary furniture Working office equipment Stationery and consumables Telephone and e-mail access Appropriate transport available

There must be a sufficient and functional infrastructure to enable the PA to perform its activities

Compliance with legislation (license)

Human resources

Personnel Number of people Training, education and

experience Support staff Key personnel - quality assurance,

prequalification, purchasing, storage and distribution

Compliance with national legislation (e.g. responsible person)

Quality assurance/prequalification and purchasing independent of one another

Compliance with legislation

Quality assurance/prequalification and purchasing independent of one another

Organization Job descriptions


Written job descriptions Signed and dated

Organization chart Authorized, current organization

chart In accordance with the job

descriptions

Ethical considerations

Code of conduct Designed, authorized and

implemented Covers conduct of staff All to comply with a code of

conduct

Confidentiality Relevant product information kept

confidential Confidentiality agreements

Conflict of interest Policy on conflict of interest is

observed Signed declaration of interest No vested interests

No conflict of interest and adherence to confidentiality

Computers Appropriate hardware and software Sufficient capacity and memory Reliable and accurate quality and

management information Data transfer Data kept (e.g. hard copies) Back-up at defined intervals,

storage, access, readable Virus protection programme and

firewall Technical support and

maintenance available Maintenance and upgrading plan Trained personnel

If used, reliable data management

Financial systems

National and international financial transactions

Funds available Adequate banking facilities Signatories of bank accounts

appointed Accounting system in place Regular financial audits are

performed. Financial transactions performed


without delay

Documentation Comprehensive documented system Covers policies, guidelines, norms,

standards, manuals, procedures, records and related documents

SOPs for activities

Quality manual Reflects quality policy Evidence of QM implemented,

maintained, reviewed and amended as necessary

Standard operating procedures Written, clear, detailed SOPs for

activities Quality Risk Management

principles applied SOP for writing an SOP followed SOPs are reviewed periodically

Style and layout SOPs in PA format Signed and dated Clear, unambiguous language

Activities described in SOPs which are implemented and followed

Activities to be covered by standard operating procedures

Handling of complaints Document/record control Self inspection Handling of recalls Monitoring of environmental

conditions (e.g. temperature) Monitoring supplier performance Identifying and reporting

substandard/spurious/falsely-labelled/falsified/counterfeit (SSFFC) products;

Evaluating offers received; Ordering product(s) from supplier

or manufacturer; Distribution and retrieval of SOPs Change control Variations CAPA

Written SOPs followed for complaints, recalls, handling sub-standard products, change control

List of prequalified products and manufacturers

Current, authorized, access controlled list

Based on the outcome of evaluation

Product- and manufacturing site-specific

A key person responsible

A list is maintained


The list should contains required information

Maintenance of records

Records of all operations kept Sufficient space Access restricted Retention period appropriate

Records are kept

Contract arrangements

Written contracts for delegated activities

Written agreements in place


Module II: Prequalification

Area of operation


Principles Documented policy and procedures for prequalification

Include assessment of product and manufacturers/suppliers

If delegated - written agreement

Key persons and responsibilities

Responsible personnel identified

Independent from the purchasing personnel

Job descriptions Communication between

evaluation and inspections

Evaluation of product information (Evaluators)

List of evaluators Suitable qualifications and

experience Job descriptions Contracted external evaluators

used (confidentiality, conflicts of interest and financial resources, references)

Inspection of manufacturing sites (Inspectors)

List of inspectors Job descriptions Qualified, trained, experienced Contracted inspectors –

confidentiality and no conflict of interest

Qualified, trained personnel perform prequalification activities (including assessment and inspections)

Key steps in prequalification defined

Soliciting information Procedures for preparation of

detailed, clear specifications; soliciting information; receiving and processing of the information

Policy and procedure for handling late submissions

Record of data received Procedure for submitting

product information publicly available and accessible

Product information (as a

Evaluation of product data and information

Ensuring compliance with GMP


minimum, see product questionnaire)

Step 2: Receive product information Written procedures for

receiving, identification, marking files, containers and samples, and sufficient space for unpacking and storage

Procedure to ensure traceability of the product information

Personnel

Step 3: Screen product information SOP: Screen for completeness A screening form used Record of screening kept Communicated to supplier

Step 4: Evaluate product information Written procedure – evaluate

against requirement Time frames Patent status considered Evaluation report for each

product Outcome communicated to the

manufacturer Response invited where needed Outcome accepted or rejected Evaluation report kept as record Samples analysed if needed

(See also monitoring below)

Step 5: Plan, prepare and perform inspections

Evidence of GMP compliance Batch-to-batch consistency Site of manufacture known Site inspections Contract manufacturing sites Control over APIs (inspection

risk based)

Procedure and recording system for inspection planning

Procedure and data reviewed as part of preparation for inspection (e.g. Site master file)

Written procedure for conduct


of inspection Scope: Data and information

verified and GMP compliance assessed

If not done - conditions for waiving inspections

Inspection report Inspection report for each site

inspected Outcome communicated CAPA and review Conclusion Copy of report kept

Step 6: Finalize assessment process Written procedure followed Covers product evaluation plus

laboratory results and inspection outcome

Responsible persons (decision taking) and reasons for decision

Outcome communicated List of prequalified products

and manufacturers Agreement between PA and

supplier List reviewed and updated at

regular intervals

Monitoring Random samples of batches supplied analyzed (risk based)

Independent testing - reliable quality control laboratory

Certificates of analysis where appropriate

General analysis - access to a quality control laboratory

Status of the laboratory (e.g. authorized, accredited)

Access to raw data Handling of out-of-specification

results

Handling out of specification results

Cost recovery Transparent procedure Fee-for-services structure


Module III: Purchasing

Procurement should be done with the aim of purchasing effective, quality assured products, and not be focused on price alone. The term “procurement” in this Module relates specifically to the purchase of health sector goods from manufacturers or suppliers. The module goes on to describe the key activities in purchasing pharmaceutical products, as well as the recommended organizational structure of the procurement agencies which carry out these key activities.

Area of operation


Procurement strategies

Strategic objectives for good pharmaceutical procurement include:

• selection of reliable suppliers of quality products;

• procurement of the most cost-effective pharmaceutical products in the right quantities and

meeting the quality standards;• mitigating possible risks;• timely delivery; • achievement of the

lowest possible total cost.

Efficient and transparent management

responsibilities; appropriate standardization, selection, specification and quantification of pharmaceutical products, monitoring

financial management procedures

competitive procurement methods

Procedure to calculate lowest possible total cost

Ensure that the following principles are applied:

prequalified products are purchased from approved manufacturers;

procurement and purchasing

Selection of reliable suppliers

Procurement of products meeting the defined quality standards


procedures are transparent; activities follow formal written

procedures throughout the process, including explicit criteria for awarding contracts;

independent contract review; purchasing is based on defined

procurement policy of the procurement agency;,

purchasing and tender documents list all pharmaceutical products by their INN or national generic names;

suppliers are selected and monitored through a process that takes into account product quality, service reliability and performance, delivery time, ethics, legal status, financial viability and minimum order quantities;

intellectual property rights are respected in accordance with best practice and national law.

Procurement methods

Responses examined from invited suppliers

Responsive to the defined terms and conditions

Informed of the outcome

Restricted tender Prequalified products and

suppliers Competitive negotiation minimum of three suppliers Prequalified products and

suppliers Direct procurement Limited use

Use a defined, transparent procurement method

Quality Assurance

QS to ensure that pharmaceutical products are of the quality required for their intended use

Quality system in place and functioning

Ensuring quality products are procured

Key activities Develop a list or catalog of products (INN)

Develop specifications for the products

Key activities are defined in SOPs that are followed


Quantification Accurate quantification of needs Quantities purchased based on

reliable estimate Methods of product

quantification

Procurement method According to their policy and

procedures

Organization and responsibilities

Personnel with appropriate qualifications and training

Established procedures Job description Independent from those

responsible for prequalification and quality assurance

Transparent, written procedures Explicit criteria for awarding

contracts Confidentiality agreement Declarations of conflict of

interest Reliable financing for

procurement Good financial management

procedures Procurement planned Procurement performance

monitored

Suitable personnel

Written procedures followed

No conflict of interest

Good procurement practices

Monitoring of the performance or prequalified suppliers

Procedure for continuous monitoring of the performance of the manufacturers and suppliers

Monitoring may include:

analysis of quality control results verification that the product

batches supplied have been manufactured in compliance with standards and specifications accepted in the product information through inspection;

pharmacovigilance; monitoring of complaints; outcome of inspection of

manufacturing sites; outcome of reassessment of

product information; monitoring of direct and indirect

Monitoring performance of manufacturers and suppliers


product costs; monitoring of adherence to

delivery schedules

contract terms and conditions.

Tracking system (values of contracts awarded, total purchases, performance)

Country legislation

Destination country’s legislation on registration/licensing

Donations Written procedure


Module IV: Receiving and storage

Area of operation


General arrangements

Received and stored correctly Quality and integrity is preserved Batch traceability Stock rotation Unidirectional flow Security of materials and

products Access to a quality control

laboratory Subcontracting

Procedures followed for receiving and storage

Pre-shipment quality control

Batches tested by the manufacturer prior to supply to PA

Batches received meet specifications

Receiving of stock

Receiving and dispatch bays Incoming goods cleaned,

quarantined Released for use or distribution Review of COAs

Checks on receipt: Order, delivery note, labels and

transport conditions, integrity of packages and seals, and for uniformity of the containers

Visual inspection for: Contamination; tampering and

damage; expiry date, compliance with labeling, packaging instructions;

Suspect containers and damaged containers - recorded and investigated.

Goods received and checked according to an appropriate SOP

Post-procurement control

Random sampling for independent laboratory analysis

SOP and national legislation Representative samples Appropriately trained and

qualified personnel Sampling plans and instructions

(risk assessment)Rejected materials

SOP, rejected products not used Separate storage or validated

computerized system Action approved by authorized

Rejected materials kept separately, access controlled and handled appropriately


personnel and recordedStorage of materials/products

Personnel Trained Personal hygiene and sanitation Job descriptions Working garments

Storage areas

Space Segregation of rejected, expired,

recalled or returned stock Narcotics/psychotropic

medicines as per national legislation

Adequate ventilation, temperature and relative humidity

Conditions checked, monitored and recorded

No unauthorized access SOP for fire control Fire detection and fire-fighting

equipment No smoking observed Clean and free of vermin and

accumulated waste Sanitation programme Pest-control SOP for cleaning up of any

spillages

Storage conditions

As established by the manufacturer

Orderly, batch segregation, stock rotation, FEFO

Stored off the floor Spaced to permit cleaning and

inspection Pallets in a good state of

cleanliness and repair Stacking of products without

damage Storage conditions (see GSP) Freeze-sensitive products -

“freeze-watch” monitoring device

Monitoring of storage conditions

Access controlled and sufficient space

Appropriate conditions for storage

Pest control


Temperature mapping protocol and report

Calibrated sensors / devices Ongoing monitoring with records Out of limit and out of trend

results investigated

Miscellaneous and hazardous materials

Rodenticides, insecticides, fumigating agents and sanitizing materials

Toxic substances and flammable materials

Re-packaging and re-labelling

Where re-packaging or re-labelling - WHO GMP

Stock control Validated stock control system Batch number control and expiry

dating Periodic stock reconciliation

done Significant stock discrepancies

investigated Records maintained Damaged containers handled

Control of obsolete and outdated materials and products

SOP Regular checks Outdated products not issued

Recalled materials and products SOP for recalls Written records of actions with

signatures Products identified, recorded,

reconciled, and stored separately Appropriately qualified and

experienced member of staff

Returned goods SOP for returned goods Quarantined and assessed Re-sale conditions Destruction in compliance with

national requirements Records

Waste materials

Stock control in place (reconciliation, obsolete materials, recalled products, returned goods, FEFO and waste)


SOP Safe storage while awaiting

disposal Toxic substances and flammable

materials No accumulation Safe disposal, intervals, national

regulationsDocumentation: written instructions and records

SOPs activities Handling of expired stock Ensuring batch traceability Records for storage conditions,

precautions, retest dates National regulations concerning

labels and containers Retention of records Comprehensive records of all

receipts and issues of products

Record keeping ensuring traceability, receiving, issuing, expired goods etc


Module V: Distribution

The PA (or contracted party) should have a well-managed distribution system meeting the objectives of ensuring constant supply of quality medicines.

Area of operation


General Constant supply of medicines Minimize medicines losses

(spoilage and expiry) Accurate inventory records Reduce theft and fraud Forecasting needs

Constant supply assured

Transport conditions

Transport process has no negative impact

Required storage conditions maintained

Temperature excursions - risk assessment

Appropriate transport conditions

Cold chain Special care taken where needed

Appropriate containers Packaging procedure Cooling agents used Calibrated monitoring devices Monitoring records maintained Validated process

Cold chain maintained, monitored, validated

Dispatch procedures

Procedures in place Special packaging requirements

observed where needed Dispatch and transport after

receipt of a delivery order Authorized recipients Compliance with legislation

Authorized recipients

Follow procedures

Dispatch containers

Provide protection Appropriately labeled Prevent theft (e.g.

locked/wrapped)

Suitable containers used

Dispatch records

Detailed records kept (e.g. date, customer name and address; product name and batch number and quantity)

Products and batches traceable Discrepancies investigated

Records ensure traceability of goods

Port of entry Storage conditions met Temperature-sensitive products Security measures (theft, fraud

and bribery)

Appropriate conditions ensured at port of entry


Module VI: Re-assessment

Quality of products and services should be continuously monitored. This process includes re-assessment.

Area of operation


Re-evaluation of manufacturers

Re-inspection frequency based on risk assessment

Within 5 year cycle Change control Mechanism for suspension and

withdrawal

Re-inspection policy and procedure followed

Re-evaluation of products

Re-evaluation procedure Within 5 year cycle Variations procedure

Re-evaluation of product policy and procedure followed

Monitoring performance of contractors

Written procedure Covers continuous monitoring,

periodic review and renewal of contracts

System for documenting service problems

Procedure followed for monitoring performance

6. Revision history

Date Version number Reasons for revision

February 2013 1.2013 Original


siteresources.worldbank.orgsiteresources.worldbank.org/exthsd/images/assessmenttool... · web...

Documents