response to mark powell's comments

3
Risk Analysis, Vol. 33, No. 3, 2013 DOI: 10.1111/risa.12021 Response Response to Mark Powell’s Comments Peyton M. Ferrier and Jean C. Buzby In his recent comments on “The Economic Ef- ficiency of Sampling Size: The Case of Beef Trim,” Mark Powell (2013) expresses concerns over the va- lidity of our findings stemming mainly from the spec- ification of our benefit function. (1) Specifically, Dr. Powell believes, first that we incorrectly specified the prevalence rate; second that this error causes our benefit function to overstate the damages associated with any given prevalence rate; and, third, that there are other errors with our model. We disagree and ex- plain our reasoning in this response. Before addressing Dr. Powell’s specific concerns, we reiterate that our article’s purpose was to address the choice of sampling size in the testing of food for health and other risks. Within official statements de- scribing the USDA’s Food Safety and Inspection Ser- vice (FSIS) sampling protocol, it is asserted that the sampling size of food safety tests represents a specific statistical significance level for a given set of condi- tions about the world. Specifically, FSIS guidelines for beef trim sampling state: “60 selected portions are needed to have 95% confidence that contami- nation will be detected when the percentage of po- tential portions (that could have been selected) that are contaminated is equal to 5%.” (2) These guide- lines do not justify why the 95% significance level is the appropriate statistical criterion for the test’s significance (β ) nor why 5% is deemed an accept- able conjectured prevalence rate (p) around which to design a test. As various interest groups and pol- icymakers have noted, specifying lower levels of β and p necessitates a larger sample size n. While im- proving risk outcomes, increasing sample sizes also increases the costs of testing. Our article applies cost- benefit analysis to the selection of a test’s β -level or, equivalently, its sample size, to account for this tradeoff. First, in addressing the comments, please note that our definition of the prevalence rate (p) differs from Dr. Powell’s presentation of it. We define the prevalence rate as “the rate at which samples in a contaminated lot contain Escherichia coli O157:H7.” We specifically chose this definition to parallel the notion of p representing the “percentage of poten- tial portions (that could have been selected)” defini- tion used in the FSIS draft guidelines. In contrast, Dr. Powell’s comments characterize our prevalence rate as: “p = prevalence rate of E. coli O157:H7 within contaminated lots.” Whereas our definition specifi- cally refers to the presence of E. coli O157:H7 on potential samples, Dr. Powell’s focus is on the distri- bution of the pathogen throughout the entire surface area of the carcass. Dr. Powell then notes that the likelihood that a sampled piece of beef trim contains E. coli O157:H7 (given some distribution of the pathogen on car- casses) depends on the surface area of the sampled trim. We do not deny this but feel that it is beside the point. We have tacitly assumed that the sampling process, including the size and surface areas of “po- tential portions,” are roughly equivalent across tests. If one allows for significant deviation from that as- sumption, one introduces an entirely new line of in- quiry into the appropriate design of tests. We feel that consideration of the prevalence rate under our definition is appropriate given the particular policy question we aim to address (i.e., the choice of sam- pling size) and given the existing FSIS guidelines jus- tifying the sample size. In reference to Dr. Powell’s assertion that our specification of the prevalence rate constitutes a fundamental error, we disagree and feel his argument is primarily semantic. He redefines our prevalence rate and then asserts that we are incorrect owing to the disagreement. Second, Dr. Powell believes that our definition of the prevalence rate leads to our overestimating the damages of the pathogen within our benefit func- tion. This function is the averted health damage 353 0272-4332/13/0100-0353$22.00/1 Published 2013. This article is a U.S. government work and is in the public domain for the USA.

Upload: jean-c

Post on 09-Apr-2017

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Response to Mark Powell's Comments

Risk Analysis, Vol. 33, No. 3, 2013 DOI: 10.1111/risa.12021

Response

Response to Mark Powell’s Comments

Peyton M. Ferrier and Jean C. Buzby

In his recent comments on “The Economic Ef-ficiency of Sampling Size: The Case of Beef Trim,”Mark Powell (2013) expresses concerns over the va-lidity of our findings stemming mainly from the spec-ification of our benefit function.(1) Specifically, Dr.Powell believes, first that we incorrectly specified theprevalence rate; second that this error causes ourbenefit function to overstate the damages associatedwith any given prevalence rate; and, third, that thereare other errors with our model. We disagree and ex-plain our reasoning in this response.

Before addressing Dr. Powell’s specific concerns,we reiterate that our article’s purpose was to addressthe choice of sampling size in the testing of food forhealth and other risks. Within official statements de-scribing the USDA’s Food Safety and Inspection Ser-vice (FSIS) sampling protocol, it is asserted that thesampling size of food safety tests represents a specificstatistical significance level for a given set of condi-tions about the world. Specifically, FSIS guidelinesfor beef trim sampling state: “60 selected portionsare needed to have 95% confidence that contami-nation will be detected when the percentage of po-tential portions (that could have been selected) thatare contaminated is equal to 5%.”(2) These guide-lines do not justify why the 95% significance levelis the appropriate statistical criterion for the test’ssignificance (β) nor why 5% is deemed an accept-able conjectured prevalence rate (p) around whichto design a test. As various interest groups and pol-icymakers have noted, specifying lower levels of β

and p necessitates a larger sample size n. While im-proving risk outcomes, increasing sample sizes alsoincreases the costs of testing. Our article applies cost-benefit analysis to the selection of a test’s β-levelor, equivalently, its sample size, to account for thistradeoff.

First, in addressing the comments, please notethat our definition of the prevalence rate (p) differs

from Dr. Powell’s presentation of it. We define theprevalence rate as “the rate at which samples in acontaminated lot contain Escherichia coli O157:H7.”We specifically chose this definition to parallel thenotion of p representing the “percentage of poten-tial portions (that could have been selected)” defini-tion used in the FSIS draft guidelines. In contrast, Dr.Powell’s comments characterize our prevalence rateas: “p = prevalence rate of E. coli O157:H7 withincontaminated lots.” Whereas our definition specifi-cally refers to the presence of E. coli O157:H7 onpotential samples, Dr. Powell’s focus is on the distri-bution of the pathogen throughout the entire surfacearea of the carcass.

Dr. Powell then notes that the likelihood that asampled piece of beef trim contains E. coli O157:H7(given some distribution of the pathogen on car-casses) depends on the surface area of the sampledtrim. We do not deny this but feel that it is besidethe point. We have tacitly assumed that the samplingprocess, including the size and surface areas of “po-tential portions,” are roughly equivalent across tests.If one allows for significant deviation from that as-sumption, one introduces an entirely new line of in-quiry into the appropriate design of tests. We feelthat consideration of the prevalence rate under ourdefinition is appropriate given the particular policyquestion we aim to address (i.e., the choice of sam-pling size) and given the existing FSIS guidelines jus-tifying the sample size. In reference to Dr. Powell’sassertion that our specification of the prevalence rateconstitutes a fundamental error, we disagree and feelhis argument is primarily semantic. He redefines ourprevalence rate and then asserts that we are incorrectowing to the disagreement.

Second, Dr. Powell believes that our definitionof the prevalence rate leads to our overestimatingthe damages of the pathogen within our benefit func-tion. This function is the averted health damage

353 0272-4332/13/0100-0353$22.00/1Published 2013. This article is a U.S. government work and is in the public domain for the USA.

Page 2: Response to Mark Powell's Comments

354 Ferrier and Buzby

associated with a sample size (n), given the lot size(L), the inoculum level (x), and the distribution ofthe prevalence rate across samples (as given by theBeta distribution parameters, μp and σ p). It is equalto the monetized value of the illnesses avoided byexposure to E. coli O157:H7 through beef servingsgenerated from contaminated lots. Our damage func-tion incorporates two features of the prevalence rate(using our definition). First, a higher prevalence rate(that goes undetected) generates more contaminatedservings and is likely to cause more damage (i.e.,illness). Second, a higher prevalence rate means itis more likely the E. coli O157:H7 pathogen that ispresent will be detected for any given sample size.Our benefit function (with some added description)is:

B (n|L, x, μp, σp)

= −r × D ×∫ 1

0p × S(L|x)︸ ︷︷ ︸

servingscontaminatedfrom a lot of

size L andinnoculum level x

× (1 − p)n︸ ︷︷ ︸probabiliy ofnot detectingcontaminationin a tested lot

× g (p|μp, σp) dp.

The S(L|x) function relates the prevalencerate to the number of contaminated servings. AsDr. Powell notes, this deterministic function is theproduct of lot size (L), the number of 100 g per serv-ing per MT (F), and a function J. Dr. Powell describesJ as “the number of contaminated ground beef serv-ings resulting from each initially contaminated beeftrim servings and is conditional on a deterministic ‘in-oculum’ (contamination) value (x).”

Again, Dr. Powell’s definition of J differs fromthat in our article, which states that J is: “the propor-tion of ground beef that is contaminated when 100%of the lot of beef trim is contaminated for a given in-oculum level of x.” We constructed this function froma set of experiments by Flores(3,4) in which he esti-mates the spread of E. coli O157:H7 when a pieceof beef trim is contaminated with predetermined lev-els of E. coli O157:H7 inoculum, ground with otheruncontaminated beef trim, and then tested for thepresence of the pathogen in the final volume. In theFlores experiments, the final volume of contaminatedproduct depends on the level of contamination on asingle piece of beef. These figures can be used to finda percentage of the total product contaminated basedon a percentage of the contaminated samples.

In the Flores experiments, the ratio of theweights of individual contaminated pieces to thewhole volume of beef trim to be ground is 0.62%.To reflect the fact the prevalence rate may be higheror lower than 0.62%, we scaled the J function up by1/.0062 to reflect a prevalence rate of 100% and thenwe immediately multiplied it by p (or 0.01 if p equals1%) to scale it down to reflect the prevalence ratethat we were simulating. This specification allows usto consider the change in the number of servings con-taminated for a given change in the prevalence rate.

Dr. Powell asserts that “the first time p entersEquation (1), it clearly represents the prevalence of100 g servings.” It does not. As we have just de-scribed, our method of simulating the percentage of100 g servings that are contaminated depends on thepercentage of beef trim samples contaminated (p, aswe define it) at a given inoculum (x). In our sim-ulations, we allow for different distributions of theprevalence rate and inoculum levels. We also incor-porate sensitivity analysis with regard to the lot size,the cost of sampling, the size of health damages, andthe contamination rate. Different modeling assump-tions regarding the conversion of J would undoubt-edly impact our results. For example, we scale thespread of contamination by the prevalence rate, es-sentially assuming a linear relationship between thetwo variables, but there are other possible assump-tions one might consider. We do not think, however,such changes would have a substantial impact on theresults. A major challenge to the use of simulationmethods is setting an end point for how many alter-native modeling assumptions should be considered.We do not present other modeling relationships inour article because, after some point, it distracts fromthe main message of the article.

Finally, Dr. Powell asserts that we have madeother errors but does not detail his concerns for thesake of brevity. This is frustrating because it casts as-persions on our work without affording us an oppor-tunity to defend what we have done. It is our feelingthat these other concerns stem mainly from disagree-ment about our modeling assumptions rather thanspecific methodological problems with our analysis.

DISCLAIMER

The opinions expressed herein are the views ofthe authors and do not necessarily reflect the officialpolicy or position of their institutions.

Page 3: Response to Mark Powell's Comments

Response to Mark Powell’s Comments 355

REFERENCES

1. Powell M. Regarding The economic efficiency of sampling size:The case of beef trim. Risk Analysis, 2013; 33(4).

2. Food Safety and Inspection Service. Compliance Guidelinesfor Sampling Beef Trimmings for Escherichia coli O157:H7(Draft for Stakeholder Comment) Washington, DC: USDA,2008.

3. Flores RA, Tewart TES. Empirical distribution models forEscherichia coli 57:H7 in ground beef produced by a mid-size commercial grinder. Journal of Food Science, 2004;69(5):M121–M126.

4. Flores RA. Modeling the behavior and fate on microbialpathogens in beef processing particle reduction operations. InJuneja V, Cherry J, Tunick M (eds). Advances in MicrobialFoods Safety. Oxford, UK: Oxford University Press, 2006.