center for biofilm engineering al parker, phd, biostatistician center for biofilm engineering...
TRANSCRIPT
Center for Biofilm Engineering
Al Parker, PhD, BiostatisticianCenter for Biofilm EngineeringMontana State University
Statistics and Biofilms
June 29, 2012
Standardized Biofilm Methods Laboratory
Darla GoeresAl Parker
Marty Hamilton
Diane Walker
Lindsey Lorenz
Paul Sturman
Kelli Buckingham-Meyer
What is statistical thinking?
Data
Experimental Design
Uncertainty and variability assessment
What is statistical thinking?
Data (pixel intensity in an image? log(cfu) from viable plate counts?)
Experimental Design - controls - randomization- replication (How many coupons?
experiments? technicians? labs?)
Uncertainty and variability assessment
Why statistical thinking?
Anticipate criticism (design method and experiments accordingly)
Provide convincing results (establish statistical properties)
Increase efficiency (conduct the least number of experiments)
Improve communication
Why statistical thinking?
Standardized Methods
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
A standard laboratory method is said to be relevant to a real-world scenario if, given the same inputs, the laboratory outcome is equivalent to the real-world outcome.
Relevance
Elbow Prosthesis - in vivo study
Urinary catheter in vivo study
Urinary Catheter Biofilm
CV Catheter in vivo study
Biofilm in the Catheter Tip
1,000 X magnification Sheep (control)
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
A standard laboratory method is said to be reasonable if the method can be performed inexpensively using typical microbiological techniques and equipment.
Reasonableness
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
Resemblance of Controls
Independent runs of the method produce nearly the same control data, as indicated by a small
standard deviation.
Statistical tool:
nested analysis of variance (ANOVA)
• 86 mm x 128 mm plastic plate with 96 wells• Lid has 96 pegs
Resemblance Example: MBEC
1 2 3 4 5 6 7 8 9 10 11 12
A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
MBEC Challenge Plate
disinfectant neutralizer test control
Resemblance Example: MBEC
Mean LD= 5.55
Control Data: log10(cfu/mm2) from viable plate counts
row cfu/mm2 log(cfu/mm2)A 5.15 x 105 5.71B 9.01 x 105 5.95C 6.00 x 105 5.78D 3.00 x 105 5.48E 3.86 x 105 5.59F 2.14 x 105 5.33G 8.58 x 104 4.93H 4.29 x 105 5.63
Exp RowControl
LDMean
LD SD1 A 5.71
5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63
2 A 5.41
5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41
Resemblance Example: MBEC
Resemblance from experiment to experiment
Mean LD = 5.48
Sr = 0.26
the typical distance between a control well LD from an experiment and the true mean LD
Resemblance from experiment to experiment
The variance Sr2
can be partitioned:
98% due to among experiment sources
2% due to within experiment sources
S
nc • m
c2
+
Formula for the SE of the mean control LD, averaged over experiments
Sc = within-experiment variance of control LDs
SE = among-experiment variance of control LDs
nc = number of control replicates per experiment
m = number of experiments
2
2
S
m
E2
SE of mean control LD =
CI for the true mean control LD = mean LD ± tm-1 x SE
8 • 2
Formula for the SE of the mean control LD, averaged over experiments
Sc = 0.02 x (0.26)2 = 0.00124
SE = 0.98 x (0.26)2 = 0.06408
nc = 8
m = 2
2
2
2SE of mean control LD =
0.00124+
0.06408= 0.1792
95% CI for the true mean control LD = 5.48 ± 12.7 x 0.1792
= (3.20, 7.76)
Resemblance from technician to technician
Mean LD = 5.44
Sr = 0.36
the typical distance between a control well LD and the true mean LD
The variance Sr2
can be partitioned:
0% due to technician sources
24% due to between experiment sources
76% due to within experiment sources
Resemblance from technician to technician
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
Repeatability
Independent runs of the method in the same laboratory produce nearly the same outcome, as indicated by a small
repeatability standard deviation.
Statistical tool: nested ANOVA
Repeatability Example
Data: log reduction (LR)
LR = mean(control LDs) – mean(disinfected LDs)
Exp RowControl
LDMean
LD SD1 A 5.71
5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63
2 A 5.41
5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41
Repeatability Example: MBEC
1 2 3 4 5 6 7 8 9 10 11 12A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
Repeatability Example: MBEC
Mean LR = 1.63
Exp RowControl
LDControl
Mean LD ColDisinfected 6.25% LD
Disinfected Mean LD LR
1 A 5.71
5.55 4.51 1.04
1 B 5.95 1 4.671 C 5.78 2 4.411 D 5.48 3 4.331 E 5.59 4 4.591 F 5.33 5 4.541 G 4.931 H 5.63
2 A 5.41
5.41 3.20 2.21
2 B 5.71 1 4.782 C 5.54 2 2.712 D 5.33 3 3.482 E 5.11 4 3.232 F 5.48 5 1.822 G 5.332 H 5.41
Repeatability Example
Mean LR = 1.63
Sr = 0.83
the typical distance between a LR for an experiment and the true mean LR
S
nc • m
c2
+
Formula for the SE of the mean LR, averaged over experiments
Sc = within-experiment variance of control LDs
Sd = within-experiment variance of disinfected LDs
SE = among-experiment variance of LRs
nc = number of control replicates per experiment
nd = number of disinfected replicates per experiment
m = number of experiments
2
2
2
S
nd • m
d2
+S
m
E2
SE of mean LR =
Formula for the SE of the mean LR, averaged over experiments
Sc = within-experiment variance of control LDs
Sd = within-experiment variance of disinfected LDs
SE = among-experiment variance of LRs
nc = number of control replicates per experiment
nd = number of disinfected replicates per experiment
m = number of experiments
2
2
2
CI for the true mean LR = mean LR ± tm-1 x SE
Formula for the SE of the mean LR, averaged over experiments
Sc2 = 0.00124
Sd2 = 0.47950
SE2 = 0.59285
nc = 8, nd = 5, m = 2
SE of mean LR =
8 • 2 2
0.00124+
0.59285
5 • 2
0.47950+ = 0.5868
95% CI for the true mean LR = 1.63 ± 12.7 x 0.5868
= 1.63 ± 7.46
= (0.00, 9.09)
How many coupons? experiments?
nc • m m
0.00124+
0.59285
nd • m
0.47950+margin of error= tm-1 x
no. control coupons (nc): 2 3 5 8 12no. disinfected coupons (nd): 2 3 5 5 12
no. experiments (m) 2 8.20 7.80 7.46 7.46 7.163 2.27 2.15 2.06 2.06 1.974 1.45 1.38 1.32 1.32 1.276 0.96 0.91 0.87 0.87 0.84
10 0.65 0.62 0.59 0.59 0.57100 0.18 0.17 0.16 0.16 0.16
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
A method should be sensitive enough that it can detect important changes in parameters of interest.
Statistical tool: regression and t-tests
Responsiveness
disinfectant neutralizer test control
Responsiveness Example: MBEC
A: High Efficacy
H: Low Efficacy
1 2 3 4 5 6 7 8 9 10 11 12
A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
Responsiveness Example: MBEC
This response curve indicates responsiveness to decreasing efficacy between rowsC, D, E and F
Responsiveness Example: MBEC
Responsiveness can be quantified with a regression line:
LR = 6.08 - 0.97row
For each step in the decrease of disinfectant efficacy, the LR decreases on average by 0.97.
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
A standard laboratory method is said to be rugged if the outcome is unaffected by slight departures from the protocol.
Ruggedness
Parameters in the protocol:
Sonication Power: 130, 250, 480 watts
Sonication Duration: 25, 30, 35 minutes
Treatment Temperature: 20, 22, 24 oC
Incubation Time: 16, 17, 18 hours
Ruggedness Testing of the MBEC
Ruggedness Test Design
Run Incubation Time
TreatmentTemperature
Sonication Power
Sonication Duration
1 17 hrs 22°C 250 watts 30
2 18 hrs 20°C 130 watts 25
3 16 hrs 24°C 480 watts 35
4 18 hrs 24°C 480 watts 25
5 18 hrs 24°C 130 watts 35
6 18 hrs 20°C 480 watts 35
7 16 hrs 20°C 480 watts 25
8 17 hrs 22°C 250 watts 30
9 16 hrs 20°C 130 watts 35
10 16 hrs 24°C 130 watts 25
1 2 3 4 5 6 7 8 9 10 11 12
A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
MBEC Challenge Plate
disinfectant neutralizer test control
Ruggedness Testing of the Controls
TempSonDurPower
Inc
2422203525303525
480130480130250480130480130161818161718161618
5.75
5.50
5.25
5.00
4.75
4.50
4.25
4.00
me
an lo
g d
en
sity
Ruggedness Testing of the Controls
LD(controls) = 5.027 + 0.1111(IncubationTime – 17) - 0.0042(SonicationDuration -30) - 0.1178(TreatmentTemperature – 22) + 0.0004(SonicationPower – 250) + 0.3893(BiofilmGrowth – 5.87) All terms are small and not of practical importance inside the range of values tested
None of the model terms were statistically significant
This model allows one to quantifiably predict how deviations from the protocol affect the experimental outcome
1 2 3 4 5 6 7 8 9 10 11 12
A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
MBEC Challenge Plate
disinfectant neutralizer test control
Ruggedness Testing of the LRs
plate
row
6
5
4
3
2
1
0
log
re
du
ctio
n
Ruggedness Testing of the LRs
LR(H) = 0.2157 – 0.3738(IncubationTime – 17)* + 0.0015(SonicationDuration -30) – 0.1001(TreatmentTemperature – 22) + 0.0003(SonicationPower – 250)
Only IncubationTime was statistically significant*
Except for IncubationTime, the terms are small and not of practical importance inside the range of values tested
Ruggedness Testing of the LRs
Only IncubationTime was statistically significant*
Except for TreatmentTemperature, the terms are small and not of practical importance inside the range of values tested
LR(A) = 5.7219 + 0.1254(IncubationTime – 17)* + 0.0015(SonicationDuration -30) – 0.2831(TreatmentTemperature – 22) + 0.0003(SonicationPower – 250)
Results of the ASTM ILS for the MBEC
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
Collaboration
ASTM Interlaboratory Study (ILS) Process
Register test method
Conduct ruggedness testing
Minimum of 6 participating labs
Technical contact• Instructions• Supplies• Data template
Research report
Precision & Bias statement
ASTM ILS #25570
Eight labs
Three experimental test days at each lab
Three disinfectants tested/experiment day• non-chlorine oxidizer• phenol• quaternary ammonium compound
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
Control Data
Control Data
Untreated Control Variability
LabNo Exp
Mean LD
Within plate
%
Among plate
%
Among exp day
%
Among lab
%
Repeatability SD
Reproducibility SD
1 3 7.50 40% 34% 25% 0.1369 2 3 7.58 20% 27% 53% 0.4206 3 3 6.27 39% 12% 49% 0.1696 4 3 7.92 17% 0% 83% 0.2315 5 3 7.80 64% 0% 36% 0.1624 6 3 7.72 8% 7% 85% 0.5301 7 3 8.13 76% 24% 0% 0.1438 8 3 8.16 51% 0% 49% 0.2706
All 24 7.48 4% 11% 9% 76% 0.3252 0.6669
Attributes of a Standard Method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Responsiveness
Ruggedness
Reproducibility (inter-laboratory)
Independent runs of the method by different researchers in different laboratories produce nearly the same outcome (e.g. LR).
This assessment requires a collaborative (multi-lab) study.
Reproducibility
Treated Data: LR (Non-chlorine oxidizer)
Treated Data: LR (Phenol)
Treated Data: LR (Quat)
Oxidizer Results
Disinfectant Row Mean LRWithin
Among lab %
Repeatability SD
Reproducibility SDlab %
Oxidizer
A 5.50 75% 25% 1.0557 1.2205B 4.41 96% 4% 1.4918 1.5196C 3.03 92% 8% 1.6093 1.6771D 1.72 85% 15% 1.5658 1.6986E 0.60 50% 50% 0.8844 1.2488F -0.08 34% 66% 0.3776 0.6453G -0.19 100% 0% 0.4687 0.4687H -0.18 100% 0% 0.5223 0.5223
Phenol Results
Disinfectant Row Mean LRWithin
Among lab %
Repeatability SD
Reproducibility SDlab %
Phenol
A 5.64 100% 0% 1.2578 1.2578B 4.76 100% 0% 1.2747 1.2747C 2.59 80% 20% 1.2467 1.3979D 1.15 57% 43% 0.8984 1.1905E 0.34 29% 71% 0.326 0.6059F -0.02 52% 48% 0.2521 0.3509G -0.11 56% 44% 0.2015 0.2683H -0.15 100% 0% 0.3009 0.3009
Quat Results
Disinfectant Row Mean LRWithin
Among lab %
Repeatability SD
Reproducibility SDlab %
Quat
A 3.64 36% 64% 0.9036 1.512B 2.26 35% 65% 0.862 1.4522C 1.34 46% 54% 0.8372 1.2406D 0.95 27% 73% 0.606 1.1715E 0.58 26% 74% 0.5302 1.0394F 0.18 50% 50% 0.3901 0.5501G -0.01 78% 22% 0.3944 0.4472H -0.11 100% 0% 0.3598 0.3598
Repeatability at a glance …
6543210
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Mean LR
Re
pe
atab
ility
SD
OxidizerPhenol.Quat.
Dis.
Repeatability SD vs lab means
Reproducibility at a glance …
6543210
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
Mean LR
Re
pro
du
cib
ility
SD
OxidizerPhenol.Quat.
Dis.
Reproducbility SD versus lab means
ASTM Precision and Bias Statement
Untreated Control Data Variance Assessment
# of Labs
# of Exps
Mean LDa
Sources of Variability
Repeatability SDb
Reproducibility SDb
Within plate
%
Amongplate
%
Among exp day
%
Among lab
%8 24 7.48 4% 11% 9% 76% 0.3252 0.6669
6543210
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Mean LR
Re
pe
atab
ility
SD
OxidizerPhenol.Quat.
Dis.
Repeatability SD vs lab means
6543210
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
Mean LR
Re
pro
du
cib
ility
SD
OxidizerPhenol.Quat.
Dis.
Reproducbility SD versus lab means
12.0 Precision and Bias 12.1 Precision:12.1.1 An interlaboratory study (ASTM ILS #650) of this test method was conducted at eight laboratories testing three disinfectants (non-chlorine oxidizer, phenol and quaternary ammonium compound) at 8 concentrations (depicted in Fig. 2). An ANOVA model was fit with random effects to determine the resemblance of the untreated control data and the repeatability and reproducibility of the treated data. 12.1.2 The reproducibility standard deviation was 0.67 for the mean log densities of the control biofilm bacteria for this protocol, based on averaging across eight wells on each plate. The sources of variability for the untreated control data are provided in Table 1.
Table 1. Untreated control data variance assessment.
12.1.3 The repeatability (Fig. 5) and reproducibility (Fig. 6) of each disinfectant at each concentration is summarized.
12.1.4 For each of the three disinfectant types considered, the protocol was statistically significantly responsive to the increasing efficacy levels. The log reduction of the non-chlorine oxidizer increased by 0.87 for each increase in efficacy level. The log reduction of the phenol disinfectant increased by 0.87 for each increase in efficacy level. The log reduction of the quat increased by 0.5 for each increase in efficacy level.
12.2 Bias:12.2.1 Randomization is used whenever possible to reduce the potential for systematic bias.
Summary
Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria
Good experiments use control data and randomization.
Invest effort in more experiments versus more replicates (coupons or wells) within an experiment.
Assess uncertainty by SEs and CIs.
For additional statistical resources for biofilm methods, check out: http://www.biofilm.montana.edu/category/documents/ksa-sm
Center for Biofilm Engineering
A National Science Foundation Engineering Research Center established in 1990
www.biofilm.montana.edu
Questions?