risk based computer validation - cbi | powering … 4_rivera...23 legacy documents –strategies for...
TRANSCRIPT
2
Agenda
• What is risk based computer validation?
• Key components of a risk based computer validation
program
• Strategies for aligning current practices into a risk
based computer validation program
• Risk Based Computer Validation Strategies
3
Before we begin…
• Quick Census:
○ Who is in the quality organization?
○ Who is in the engineering organization?
○ Who is in the IT organization?
○ Others?
4
Let’s talk Risk
• What is Risk?
• Definition:
○ /noun/: (1) possibility of loss or injury (2) someone or something that
creates or suggest a hazard
○ /transitive verb/: (1) to expose to hazard or danger
5
For Computerized Systems…
• ISPE GAMP5
○ Section 5 – Quality Risk Management
○ “Systematic process for the assessment, control, communication and
review of risk”
6
Questions for Attendees
• Is the risk associated with a training system the same
as a process control?
• Should we validate a training system the same way as
a process control system?
• What about a document management system?
• Which one has the higher level of risk?
7
Risk Based Computer Validation
• Risk based validation is an approach for validating
computer systems that is based on the system risk
level
• This approach is also based on the risk level of the user
requirements and their impact to the product and
patient
9
Components of a risk based CSV approach
• Must have:
○ Document hierarchy that enables a risk based approach
• The document hierarchy requires the following:
○ Validation Policy
○ Computer Validation Standard
○ Computer Validation Procedures
12
Validation Policy
• High level document that describes the requirements for a risk
based validation program
• Provides alignment with regulatory requirements and industry
standards
• Provides the high level roles and responsibilities for each
functional area
13
Validation Policy
• Includes the following:
○ Process Validation
○ Equipment Validation
○ Risk Based Computer System Validation
○ Shipping Validation
○ CIP & SIP Validation
15
Computer Validation Standard
• Describes the specific requirements for the risk based
computer validation program
• Provides alignment with Regulatory requirements and Industry
standards
16
Computer Validation Standard (cont.)
• CSV standard should include the following information:
○ Purpose
○ Scope
○ Process Requirements
○ Roles and Responsibilities
○ Validation Planning
▪ Compliance Assessments
▪ Validation Plan
▪ Risk Assessment (system level & requirements)
▪ Supplier Assessment
17
Computer Validation Standard (cont.)
• Requirements Definition
○ User Requirement Specification
○ Functional Requirement Specification
• Design & Coding
• Development Testing
• Risk Based Qualification Activities
○ Commissioning
○ Qualification: IQ, OQ, & PQ
○ Supplier Assessment
18
Computer Validation Standard (cont.)
• Deviation and Discrepancy Handling
• Traceability Matrix
• Validation Reports
19
Computer Validation Standard (cont.)
• Discuss at a high level Operation and Maintenance processes:
○ Incident Management
○ Change Management
○ Configuration Management
○ Backup and Restore
○ Disaster Recovery
○ Security
○ Periodic Review
○ Decommissioning
21
Computer Validation Procedure
• Document that describes the specific steps to validate computer systems based on their level of risk
• Translates standard requirements into “how to” actions that promote a risk based approach
• Provides roles and responsibilities for all impacted functional areas
• Provides specific instructions and detailed requirements for the creation, review and approval of risk based validation deliverables
22
Computer Validation Procedure
• Describes the Risk Based Process
• Right Balance
○ Too General – Doesn’t provide enough guidance
○ Too Specific – Hinders the process
▪ Specific Details can be moved to Document Templates
23
Legacy Documents – Strategies for Alignment
• Validation policy and standards need to be revised to align with
risk based computer validation strategies
• Validation procedures need to be revised to align with the risk
based standards
• Validation templates need to be revised to eliminate
duplication and non-value activities and content
• Assessment tools need to be created to implement the new
risk based approach
24
Strategies for aligning legacy practices
• The outcome of the system assessments are an input into the
risk based validation strategy
• Assessment tools
○ GxP assessment
○ Part 11/Annex 11 assessment
○ System level risk assessment
○ Requirements risk assessment
26
System Assessments
• How do you determine the GxP and overall system impact?
○ Meetings
○ Endless debates
○ Relying on SME’s
27
System Assessments - Benefits
• Enables and document the agreed upon risk based approach
• Intended to provide consistency, efficiency and compliance
• Intended to provide consistent interpretations of the applicable regulatory requirements
• Eliminates misinterpretations
• Provides inputs to validation planning phase
• Supports the creation of User Requirements
• Cost efficient approach
28
GxP Assessment
• Initial assessment to determine whether the system has GxPimpact
• Identifies applicable regulatory requirements (high risk?)
• Supports the creation of URS statements related to regulatory requirements
• Provides input to the validation planning phase and risk assessment
29
GxP Assessment - ExampleQuestions Yes/ No
Process/Production
Does the computer related system perform functions or calculations affect any of the critical parameters of a product manufacture such as mixing times, heating/cooling temperatures, flow rates etc.?
Does the computer related system interface to any Inputs or Outputs (I/O) that control or monitor manufacturing parameters of a component or drug product?
Does the computer related system generate and/or store alarms, events or transactions that are generated during the manufacturing or packaging of a component or drug product?
Does the computer related system store data about the actual manufacturing parameters for a given component or drug product? (e.g., quantity of components added, mixing time, temperature, rpm, pressure, equipment used etc.)?
Does the computer related system perform manufacturing process control? (DCS, PLC, etc)
Does the computer system, either automatically or based on user inputs, formally initiate any actions that affect the processing of a component or drug product? (E.g. Does the Systems generate a work order that causes someone to make a change to the process?)
Does the computer system, either automatically or based on user inputs, formally approve any actions that affect the processing of a component or drug product?
Packaging and Labeling
Does the computer related system generate labels or formally identifies a component or drug product? (ex. Label printer)?
Does the computer related system store data about the actual packaging parameters for a given component or drug product? (e.g., quantity of components added, mixing time, temperature, rpm, pressure, equipment used etc.)?
Facilities, Equipment and Utilities
Does the computer related system control, monitor and/ or store data of manufacturing equipment configuration data over time or per batch?
Does the computer related system generate schedule, track and/or store equipment maintenance or calibration data?
Does the computer related system record the lots or batches that have been produced using a given set of equipment or procedures?
Does the computer related system control/monitor or store data that support manufacturing equipment operation or performance including Cleaning and Sterilization?
30
Part 11 & Annex 11 Assessments• Initial assessment to determine whether the system has Part 11
and Annex 11 impact
• Identifies applicable Part 11 and Annex requirements○ Open or Close System
○ E-records
○ E-Signatures
○ Operational controls
• Support the creation of URS statements related to Part 11/Annex requirements
• Provide input to the validation planning phase and risk assessment
31
System Risk Assessment
• System level risk assessment to determine whether the system is high, medium or low risk
• Simple risk assessment tool
• Should assess the risk impact to the following:○ Product Quality
○ Patient Safety
○ Compliance
○ Safety
○ Business Process
○ Complexity
Provides input to the validation strategy
32
System Risk Assessment
• The outcome of the system level risk assessment should
support a risk based validation approach
• High risk systems should require a more stringent validation
effort
• Medium risk systems should require a less stringent approach
than high risk systems
• Low risk systems should require a lesser stringent approach
than medium risk systems
33
What does this mean to us?
• Procedures should clearly describe the required validation
deliverables for each risk level
• High risk systems should have more testing and validation
deliverables
• Medium or low risk systems should require less testing of non-
critical low risk requirements
• Medium or low risk systems procedures should allow
combining documents and integrating validation activities
34
Requirement Risk Assessment
• Performed to determine whether each requirement high, medium or low risk
• The requirements risk assessment should assess the risk impact to the following:○ Product Quality
○ Patient Safety
○ Compliance
○ Safety
• The outcome of these risk assessments is one key component for risk based computer validation
36
How to apply your results? – Strategy Example
• High risk systems should require testing all high and medium risk requirements
• High risk systems should require testing a sampling of low risk requirements
• Medium risk systems should require testing all high risk requirements with a sampling of critical medium and low risk requirements
• Low risk systems should require testing only critical high risk requirements and sampling medium requirements
Strategy should be clearly defined in either Plan or Procedure
40
Problem Statement
• How much is the cost of Validation?
○ Cost of validation – typically ~25% of total Capital
• How much TIME?
○ Inadequate cycle times
○ Inability to support timelines based on business needs
41
Challenges Impacting a Risk Based Approach
• Conservative interpretation of regulatory requirementsRegulation: “The system must be validated”
Interpretation: We must validate the system in-house and have the vendor also performing full validation!
Regulation: “The need for a vendor audit should be based on a risk assessment”
Interpretation: We must audit all software vendors regardless of their risk level
Regulation: “Systems should have a periodic review process that includes data integrity”
Interpretation: We must perform data integrity periodic reviews for all systems monthly!
42
Challenges Impacting a Risk Based Approach
• Overly conservative Quality unit
• Poor or very vague understanding of what is really high risk
• Lack of understanding of the risk of the status quo vs implementing new system/process
• Conservative and subjective assumptions about regulatory expectations during an audit:○ “I am sure that Health Authorities will be looking for….”
○ “We will be audited for xyz regulatory requirements”
43
Challenges Impacting a Risk Based Approach
• Validation ○ Lack of understanding of the Intended Use of the system
• Engineering○ Lack of understanding of the Regulations
• Quality unit○ Vague regulatory understanding related to computer system
validation
○ Non-technical resources supporting technical implementations
○ Assigning unqualified QA resources to support complex technical projects
44
Effect
• Delays in implementing critical systems that improve efficiency and COMPLIANCE
• Increased implementation cost
• Significant and unnecessary resource commitment
• Significant duplication of effort
• Moving target expectations!
46
What can be done?
• Ensure an accurate and clear understanding of high risk factors
• Build a Team:○ Resources with the appropriate experience and background Technical
skills, including the Quality Unit▪ Don’t have them? Train them!
○ Accurate and pragmatic understanding of regulatory requirements by all Team Members
○ Computer validation background and skills
• Clear understanding of the risk of the status quo vs changing the process
48
Validation Planning
• Do you have a Computerized System Master Validation Plan?
• How many Validation Plans do you create for an implementation of a computerized system?
• How often do you create Validation Plans?
• Are you departing from your Master Plan?
More documents are not equal to higher quality!
49
Validation Planning
• A risk based approach for validation planning should consider the size and complexity of the system/implementation○ For large complex implementations or changes one Validation Plan
document should be adequate
○ For small or simple systems implementations the validation plan can be integrated into one of the protocols or change control records
50
Requirements Definitions
• Discussions and interviews○ Include key stakeholders
○ Ask specific questions
• Observation○ Observe business function
51
Requirements Definitions
• Workflow analysis
○ Map your process!
○ Assessment of workflows
○ Development of use cases
• Workshops
○ Classify “must have” and “nice to have”
52
Requirements Definitions Pitfalls
• Multiple requirements within a single requirement statement
• No formal change management process
• Poor scope management
• Not classifying requirements
• Ambiguous requirements
• Delivering a large document instead of clear and concise
requirements
53
Requirements Definition
• Requirements need to be clearly understood by the technical
team
• Unambiguous
• Should articulate the needs of the system owner and business
• Need to translate applicable regulatory expectations into clear
requirements
54
Requirement Definition Risk Based
• Integrate software and equipment requirements into one
document
• Keep document approvers to only those that are key
stakeholders
• Electronic record instead of paper
55
Validation Protocol
• Integrate and combine documents○ IOQ instead of individual IQ,OQ protocols
○ Benefit: One document instead of two reduces cycle times
• Keep document approvers to only those that are key stakeholders
• Electronic record instead of paper, consider paperless validation systems
56
Summary Reports
• For large projects a summary report should be created
• Protocols for routine validation activities can have a summary
page integrated into the protocol instead of a report
○ Summary page should summarize the protocol execution, closed
deviations and be approved with the protocol
○ Summary page should provide an statement on whether the protocol
met all the acceptance criteria
57
Risk Based Summary Reports
• Status Quo (The Problem)▪ A Final Report will be written following execution of the qualification exercise in Test
and Production environment. The report will provide a summary of the test results
generated during execution of the protocol and indicate if the acceptance criteria
listed were met. The report will be approved by the signature approvers for the
protocol.
▪ The report will include the following information:
▫ Execution copy of the protocols with all raw data and associated documents!
▫ A summary of all discrepancies logged during execution of the protocols.
▫ The summary will detail the nature of each discrepancy, the corrective action taken, and the
resolution!
58
Risk Based Summary Report
• Risk Based Approach (The Solution)▪ A summary report will be written following execution of the qualification
exercise in Test and Production environment. The report will provide a
summary of the test results generated during execution of the protocol and
indicate if the acceptance criteria listed were met. The summary report will
be included as the last activity in the executed protocol and will be
reviewed and approved by the signature approvers of the protocol
▪ The summary report will be:▫ included as the last section of the Execution copy of the protocol
• reviewed and approved by the signature chain of the protocol
▫ summarize all test results and whether acceptance criteria were met
▫ summarize all discrepancies and their disposition logged during execution of the protocol
61
Document Approvers
• Typical approval cycle is more than five validation document approvers
• Lean approach for document approvers should be two○ System Owner or SME
○ Quality
62
Benefits – Less Approvers
• Reduced cycle times
• Faster turnaround
• Cost efficient
• If you have an EDM,
○ Reduce the numbers of EDM users (higher privileges)
○ Lower license cost for document approvers
63
Test Script vs Protocol
• Not all validation activities require a large complex protocol
• The decision between a test script or protocol should be based on the complexity of the system or change
• Routine validation activities in support of change control can be done using test scripts
• Protocol should be used for large system implementations and complex changes
• Test Script Examples
○ Parameter changes
○ Simple phase logic changes (open/close valve)
○ Security & authentication
64
Pre- Approved Verification Forms
• Implementation of verification forms instead of protocols
• Driven by SOP
• Forms are pre-approved with SOP
• Installation & Functional Verification forms
• Forms can be created by leveraging existing protocol
65
Pre-Approved Verification Forms
• Forms can be created from requirements and design documents
• Forms can be used for the validation of changes to existing systems
• Examples of verification forms
• Security verification
• Recipe verification
• Audit trail verification
• Parameter verification
• P&ID verification
• Loop check verification
66
Verification Forms(cont.)
System Number: CC # (s) / WO #(s):
System(s) Description: Requirements document:
User Rights Verification
Step Action Expected Response Actual Response
Meets Acceptance
Criteria
(Pass / Fail)
Verified By Initial / Date
1 Login with Administrator account into the application.
Successfully logged into the Administrator account
Does Actual response meet the Expected response?
Yes No
Verification Criteria:
Visual
Screenshot
N/A
Screen shot Attached:
Yes No ,
Attachment #: ______
2
Document the user group to be modified.
___________________
___________________
Appropriate user groups are identified
Privileges are documented
Does Actual response meet the Expected response?
Yes No
Verification Criteria:
Visual
Screenshot
N/A
Screen shot Attached:
Yes No ,
Attachment #: ______
3 List the privileges to be
Added /Modified Deleted
___________________________________________________________________________________________________________________________________________________________________________
Step Action Expected Response Actual Response
Meets Acceptance
Criteria
(Pass / Fail)
Verified By Initial / Date
67
Step Action Expected Response Actual Response
Meets Acceptance
Criteria
(Pass / Fail)
Verified By Initial /
Date
Make Copies As Needed
Privilege: _________________
Test Procedure:
___________________________
___________________________
___________________________
___________________________
___________________________
___________________________
___________________________
_
Privileges specified in step# 3
are successfully verified
Does Actual response meet the Expected
response?
Yes No
Verification Criteria:
Visual
Screenshot
N/A
Screen shot Attached:
Yes No ,
Attachment #: ______
7 Log off from the User account. Successfully logged off from
the User account
Does Actual response meet the Expected
response?
Yes No
Verification Criteria:
Visual
Screenshot
N/A
Screen shot Attached:
Yes No ,
Attachment #: ______
Approved By Quality (Print/Sign):_____________________________ Date:______________
Completed by: Date:
68
Integrated equipment and automation verification
• Installation and functional verification forms○ Pump verification
○ Alarm/interlock verification
○ Valve and miscellaneous equipment verification
○ Agitator verification
• Integrating equipment verification forms into the automation protocols
• Leveraging calibration data
69
Benefits of integrating equipment and automation verification
• Reduced number of verification documents
• Cycle time reduction
• Faster equipment turn around time
• Cost reduction
70
Paperless Validation
• Eliminates paper validation documentation and system specifications
• Integrates electronic deviations to protocols
• Integrates the creation of traceability matrix with electronic protocols
• Enables electronic execution of protocol
• Electronic review and approval of protocols and system specifications
71
Paperless Validation
• Automates and manages the validation life cycle
• Provides real time validation status of any system and metrics
• Provides a holistic view of project status and validation
deliverables for internal and external auditors, with real time
status
72
Paper Validation Benefits
• Significant cycle time reduction
• Significant error reduction
• Enables faster release of equipment to support GMP operations
• Return of investment in less than 12 months
73
Summary
• We covered a lot of ideas!
○ What is risk based computer validation?
○ Key components of a risk based computer validation program
○ Strategies for aligning current practices into a risk based computer
validation program
○ Risk based Computer Validation strategies