1 hiv drug resistance training module 7: hiv genotyping assay validation
TRANSCRIPT
![Page 1: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/1.jpg)
1
HIV Drug Resistance Training
Module 7:
HIV Genotyping Assay Validation
![Page 2: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/2.jpg)
2
Topics
Getting Ready to Validate Validation Concepts Validation Procedures
![Page 3: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/3.jpg)
3
Objectives
Describe principles behind validation procedures.
Identify the procedures needed to gain confidence that the results are accurate and reproducible.
Given specific laboratory, customize procedures.
![Page 4: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/4.jpg)
4
getting ready to validate
Why is it important to validate assays?What should you do to prepare for validation?When do you need to validate?
![Page 5: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/5.jpg)
5
Validation of Genotyping Assay Performance
Helps ensure quality results Easier to compare results from labs using
different methods To establish performance characteristics
used to compare assays Required prior to testing any samples for
WHO Validation is assay-specific
– For instance, plasma vs. DBS sensitivity could be different, and extractions steps are not shared, so these methods should be validated separately.
![Page 6: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/6.jpg)
6
Minimum Requirements
Minimal requirements for an assay validation to establish/confirm analytical performance characteristics:– Sensitivity– Specificity– Accuracy– Precision– Reproducibility
![Page 7: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/7.jpg)
7
Preparation: Procedures
Standard Operating Procedures (SOPs) – For what?– How?
Quality control (QC) SOPs– For what?
Personnel?
![Page 8: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/8.jpg)
8
Preparation: Validation Protocol
Written validation protocol finalized before results are analyzed– No changes to protocol allowed once validation
experiments started Determine acceptance criteria for each
section in advance Validation performed under the same
conditions (facility, equipment, reagents, personnel) that will be used for real samples
![Page 9: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/9.jpg)
9
When to Repeat Validation Procedures
If there is failure to pass any criteria:– Document corrective action and re-validate– Do NOT simply repeat experiments with same
assay If procedure changes:
– Minor changes equivalency testing– Major changes re-validation for in-house
assays:• Primer modifications and replacement• Change RT and/or PCR enzymes
– See module 11, QA/QC
![Page 10: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/10.jpg)
10
Discussion
Why is it important to validate assays? What should you do to prepare for
validation? When do you need to validate?
![Page 11: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/11.jpg)
11
validation concepts
What criteria are we measuring against?
![Page 12: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/12.jpg)
12
Components of Genotyping Assay Validation
Accuracy Precision/Reproducibility Sensitivity for RT-PCR amplification Sensitivity for detection of minority variants Specificity
![Page 13: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/13.jpg)
13
Accuracy Definition
Are “known” mutations detected?
![Page 14: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/14.jpg)
14
Accurate Inaccurate
Accuracy: How close do we get to the expected (target) value or result.
![Page 15: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/15.jpg)
15
Accuracy Discussion
Best tested using: – clonal virus or lab constructs (e.g. site-directed
mutants)– very well-characterized, non-clonal sample (e.g.
artificial 50/50 mixtures of clones) Test for:
– “Resistance-associated” mutations– Likely to be frequently encountered
Pro: Usually the easy to measure Con: Difficult to define criteria ahead of
time for very new assays
![Page 16: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/16.jpg)
16
Precision Definition
Ability to generate the same result on multiple aliquots of the same sample within a test run (intra-assay variability) – Fewer samples (e.g. 3 to 5), more replicates
(e.g. 5 to 10)
![Page 17: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/17.jpg)
17
Precise(although inaccurate!)
Imprecise(although sometimes accurate)
Precision: How close are all the results to each other?
![Page 18: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/18.jpg)
18
Accuracy and Precision
The degree of fluctuation in the measurements is indicative of the “precision” of the assay.
The closeness of measurements to the true value is indicative of the “accuracy” of the assay.
Quality Control is used to monitor both the precision and the accuracy of the assay in order to provide reliable results.
![Page 19: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/19.jpg)
19
Precise: NoAccurate: No
Accuracy and Precision
Precise: YesAccurate: No
Precise: YesAccurate: Yes
![Page 20: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/20.jpg)
20
Reproducibility
Ability to generate the same result on multiple aliquots of the same sample in different test runs (inter-assay variability)– More samples (e.g. up to 40), fewer replicates (2
or 3)– Over time (e.g. 2 weeks)– Between operators– Among lots of critical reagents
![Page 21: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/21.jpg)
21
Reproducibility: When conditions change, how does that affect the
results?
![Page 22: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/22.jpg)
22
Reproducibility and Precision Criteria
Based on nucleotide sequence identity in:– Pairwise comparisons or– Comparison to a consensus sequence
Mismatches may be considered a difference (if “compatible” e.g. A and R, or R and D, but not A and Y, or M and K)– Acceptance criteria may depend on complexity
of samples tested Based on amino acid sequence identity
– “resistance-associated” mutations can be analyzed separately
![Page 23: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/23.jpg)
23
Critical Reagent Lot-to-lot Variability is a Component of Assay Reproducibility
Reagent vendors test products (e.g. PCR enzymes) using procedures different from HIVDR genotyping
Cannot assume that performance is the same, between lots
Incorporate at least 2 lots of critical reagents in validation experiments
Perform lot-release testing as part of routine QC (see module 11)
![Page 24: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/24.jpg)
24
Sensitivity: Amplification
Definition: What is the minimum viral load required to generate an accurate result?
Considerations:– Importance placed on multiple subtypes varies
with intended application– Overlaps with linearity if samples also
genotyped and sequences compared – May or may not involve probing multiple variant
detection (mixtures) at low VL
![Page 25: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/25.jpg)
25
Sensitivity: Detection of Minority Variants
In samples with mixtures What % of the pool must minor variant be
in order to be reliably detected?– Can separate out post-amplification steps (e.g.
by mixing plasmids and sequencing vs. mixing viruses and amplifying then sequencing)
– In reality, affected by viral load, though this is not often highlighted
![Page 26: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/26.jpg)
26
Dependence of Sensitivity of Detection of Minor Variants and Input Viral Load
Assume that 200 µl plasma used for RNA extraction, 25% used for RT-PCR; RT successful for 20% of RNA molecules; minority variant present at 10% of total
Viral load (copies/ml)
RNA copies in RT rxn
Amplifiable genomes
Copy no. (minor variant)
100,000 5,000 1000 100
10,000 500 100 10
1000 50 10 1
![Page 27: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/27.jpg)
27
Discussion
What criteria are we measuring against?
![Page 28: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/28.jpg)
28
validation procedures
How can we set up validation procedures for our lab?
![Page 29: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/29.jpg)
29
Validation Criteria (Example)
Accuracy– 100% of known mutations must be detected
Precision/Reproducibility– ≥ 90% of pairwise comparisons must be at least
98% identical, mixtures counted as difference Sensitivity for amplification
– ≥ 95% of samples with viral loads between 500 and 1000 copies/ml must be amplified and successfully genotyped (n ≥ 10)
Sensitivity for detection of minority variants– defined as the lowest % at which a mutation is
detected in > 50% of replicate tests
![Page 30: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/30.jpg)
30
Specimen Selection
Specimens used in the validation experiments should be chosen to test the specific aspect of assay performance (accuracy, reproducibility, etc.)
Especially for precision and reproducibility, chose specimens that are similar to those that will be tested routinely (i.e. specimen type, genetic subtype, viral load range)
Specimens should be well-characterized in pre-validation experiments before starting the actual validation
![Page 31: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/31.jpg)
31
Validation Procedure Customization
Details of the variables to be considered are likely to be different in each lab– Physical set-up– Personnel (number of qualified analysts)– Degree of automation
Validation protocols can be reviewed in consultation with other accredited labs or with WHO staff
![Page 32: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/32.jpg)
32
Discussion
How can we set up validation procedures for our lab?
![Page 33: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/33.jpg)
33
Reflection
What do we need to do to make sure we are correctly validating our assays?
![Page 34: 1 HIV Drug Resistance Training Module 7: HIV Genotyping Assay Validation](https://reader034.vdocuments.mx/reader034/viewer/2022042423/55153ad25503465e608b5df3/html5/thumbnails/34.jpg)
34
Summary
Getting Ready to Validate Validation Concepts Validation Procedures