response to mark powell's comments
TRANSCRIPT
Risk Analysis, Vol. 33, No. 3, 2013 DOI: 10.1111/risa.12021
Response
Response to Mark Powell’s Comments
Peyton M. Ferrier and Jean C. Buzby
In his recent comments on “The Economic Ef-ficiency of Sampling Size: The Case of Beef Trim,”Mark Powell (2013) expresses concerns over the va-lidity of our findings stemming mainly from the spec-ification of our benefit function.(1) Specifically, Dr.Powell believes, first that we incorrectly specified theprevalence rate; second that this error causes ourbenefit function to overstate the damages associatedwith any given prevalence rate; and, third, that thereare other errors with our model. We disagree and ex-plain our reasoning in this response.
Before addressing Dr. Powell’s specific concerns,we reiterate that our article’s purpose was to addressthe choice of sampling size in the testing of food forhealth and other risks. Within official statements de-scribing the USDA’s Food Safety and Inspection Ser-vice (FSIS) sampling protocol, it is asserted that thesampling size of food safety tests represents a specificstatistical significance level for a given set of condi-tions about the world. Specifically, FSIS guidelinesfor beef trim sampling state: “60 selected portionsare needed to have 95% confidence that contami-nation will be detected when the percentage of po-tential portions (that could have been selected) thatare contaminated is equal to 5%.”(2) These guide-lines do not justify why the 95% significance levelis the appropriate statistical criterion for the test’ssignificance (β) nor why 5% is deemed an accept-able conjectured prevalence rate (p) around whichto design a test. As various interest groups and pol-icymakers have noted, specifying lower levels of β
and p necessitates a larger sample size n. While im-proving risk outcomes, increasing sample sizes alsoincreases the costs of testing. Our article applies cost-benefit analysis to the selection of a test’s β-levelor, equivalently, its sample size, to account for thistradeoff.
First, in addressing the comments, please notethat our definition of the prevalence rate (p) differs
from Dr. Powell’s presentation of it. We define theprevalence rate as “the rate at which samples in acontaminated lot contain Escherichia coli O157:H7.”We specifically chose this definition to parallel thenotion of p representing the “percentage of poten-tial portions (that could have been selected)” defini-tion used in the FSIS draft guidelines. In contrast, Dr.Powell’s comments characterize our prevalence rateas: “p = prevalence rate of E. coli O157:H7 withincontaminated lots.” Whereas our definition specifi-cally refers to the presence of E. coli O157:H7 onpotential samples, Dr. Powell’s focus is on the distri-bution of the pathogen throughout the entire surfacearea of the carcass.
Dr. Powell then notes that the likelihood that asampled piece of beef trim contains E. coli O157:H7(given some distribution of the pathogen on car-casses) depends on the surface area of the sampledtrim. We do not deny this but feel that it is besidethe point. We have tacitly assumed that the samplingprocess, including the size and surface areas of “po-tential portions,” are roughly equivalent across tests.If one allows for significant deviation from that as-sumption, one introduces an entirely new line of in-quiry into the appropriate design of tests. We feelthat consideration of the prevalence rate under ourdefinition is appropriate given the particular policyquestion we aim to address (i.e., the choice of sam-pling size) and given the existing FSIS guidelines jus-tifying the sample size. In reference to Dr. Powell’sassertion that our specification of the prevalence rateconstitutes a fundamental error, we disagree and feelhis argument is primarily semantic. He redefines ourprevalence rate and then asserts that we are incorrectowing to the disagreement.
Second, Dr. Powell believes that our definitionof the prevalence rate leads to our overestimatingthe damages of the pathogen within our benefit func-tion. This function is the averted health damage
353 0272-4332/13/0100-0353$22.00/1Published 2013. This article is a U.S. government work and is in the public domain for the USA.
354 Ferrier and Buzby
associated with a sample size (n), given the lot size(L), the inoculum level (x), and the distribution ofthe prevalence rate across samples (as given by theBeta distribution parameters, μp and σ p). It is equalto the monetized value of the illnesses avoided byexposure to E. coli O157:H7 through beef servingsgenerated from contaminated lots. Our damage func-tion incorporates two features of the prevalence rate(using our definition). First, a higher prevalence rate(that goes undetected) generates more contaminatedservings and is likely to cause more damage (i.e.,illness). Second, a higher prevalence rate means itis more likely the E. coli O157:H7 pathogen that ispresent will be detected for any given sample size.Our benefit function (with some added description)is:
B (n|L, x, μp, σp)
= −r × D ×∫ 1
0p × S(L|x)︸ ︷︷ ︸
servingscontaminatedfrom a lot of
size L andinnoculum level x
× (1 − p)n︸ ︷︷ ︸probabiliy ofnot detectingcontaminationin a tested lot
× g (p|μp, σp) dp.
The S(L|x) function relates the prevalencerate to the number of contaminated servings. AsDr. Powell notes, this deterministic function is theproduct of lot size (L), the number of 100 g per serv-ing per MT (F), and a function J. Dr. Powell describesJ as “the number of contaminated ground beef serv-ings resulting from each initially contaminated beeftrim servings and is conditional on a deterministic ‘in-oculum’ (contamination) value (x).”
Again, Dr. Powell’s definition of J differs fromthat in our article, which states that J is: “the propor-tion of ground beef that is contaminated when 100%of the lot of beef trim is contaminated for a given in-oculum level of x.” We constructed this function froma set of experiments by Flores(3,4) in which he esti-mates the spread of E. coli O157:H7 when a pieceof beef trim is contaminated with predetermined lev-els of E. coli O157:H7 inoculum, ground with otheruncontaminated beef trim, and then tested for thepresence of the pathogen in the final volume. In theFlores experiments, the final volume of contaminatedproduct depends on the level of contamination on asingle piece of beef. These figures can be used to finda percentage of the total product contaminated basedon a percentage of the contaminated samples.
In the Flores experiments, the ratio of theweights of individual contaminated pieces to thewhole volume of beef trim to be ground is 0.62%.To reflect the fact the prevalence rate may be higheror lower than 0.62%, we scaled the J function up by1/.0062 to reflect a prevalence rate of 100% and thenwe immediately multiplied it by p (or 0.01 if p equals1%) to scale it down to reflect the prevalence ratethat we were simulating. This specification allows usto consider the change in the number of servings con-taminated for a given change in the prevalence rate.
Dr. Powell asserts that “the first time p entersEquation (1), it clearly represents the prevalence of100 g servings.” It does not. As we have just de-scribed, our method of simulating the percentage of100 g servings that are contaminated depends on thepercentage of beef trim samples contaminated (p, aswe define it) at a given inoculum (x). In our sim-ulations, we allow for different distributions of theprevalence rate and inoculum levels. We also incor-porate sensitivity analysis with regard to the lot size,the cost of sampling, the size of health damages, andthe contamination rate. Different modeling assump-tions regarding the conversion of J would undoubt-edly impact our results. For example, we scale thespread of contamination by the prevalence rate, es-sentially assuming a linear relationship between thetwo variables, but there are other possible assump-tions one might consider. We do not think, however,such changes would have a substantial impact on theresults. A major challenge to the use of simulationmethods is setting an end point for how many alter-native modeling assumptions should be considered.We do not present other modeling relationships inour article because, after some point, it distracts fromthe main message of the article.
Finally, Dr. Powell asserts that we have madeother errors but does not detail his concerns for thesake of brevity. This is frustrating because it casts as-persions on our work without affording us an oppor-tunity to defend what we have done. It is our feelingthat these other concerns stem mainly from disagree-ment about our modeling assumptions rather thanspecific methodological problems with our analysis.
DISCLAIMER
The opinions expressed herein are the views ofthe authors and do not necessarily reflect the officialpolicy or position of their institutions.
Response to Mark Powell’s Comments 355
REFERENCES
1. Powell M. Regarding The economic efficiency of sampling size:The case of beef trim. Risk Analysis, 2013; 33(4).
2. Food Safety and Inspection Service. Compliance Guidelinesfor Sampling Beef Trimmings for Escherichia coli O157:H7(Draft for Stakeholder Comment) Washington, DC: USDA,2008.
3. Flores RA, Tewart TES. Empirical distribution models forEscherichia coli 57:H7 in ground beef produced by a mid-size commercial grinder. Journal of Food Science, 2004;69(5):M121–M126.
4. Flores RA. Modeling the behavior and fate on microbialpathogens in beef processing particle reduction operations. InJuneja V, Cherry J, Tunick M (eds). Advances in MicrobialFoods Safety. Oxford, UK: Oxford University Press, 2006.