a predictive modeling and decision-making tool to facilitate

Proceedings of the Institute of Food Technologists’ First Annual Food Protection and Defense Conference

[Session: Modeling and Risk Assessment]

A Predictive Modeling and Decision-Making Toolto Facilitate Government and Industry Responseto an Intentional Contamination of the Food SupplyDR. ANDREW JAINE

BT SAFETY, LLC

Good afternoon. It is fortuitous that I am the final presenter atthis session, as the presentations started off with Col. Hoff-man describing a method for evaluates the risk associated with

a food contamination incident as the product of three components: thethreat to the food target, the vulnerability of the food, and the consequenc-es of any ensuing incident. The previous speakers addressed various as-pects of the threat and vulnerability of the target, and in this presentationI will describe a system for evaluating and managing the last of these threecomponents: the consequences of food contamination incidents.

This system is a predictive model designed to help government andindustry improve their ability to respond to food contamination events. Westarted developing it about 4 years ago, soon after 9/11, when one of mypartners in BTSafety became concerned about how a bioterrorism attackon the food supply could affect the several produce companies that heowns. After looking into the available approaches to reduce the vulner-ability of these companies to this type of risk, he decided that no effectiveapproaches were available and so we started designing an approach. Fairlyearly in that process we realized that we needed some detailed informa-tion about the agents that are the most likely to be involved in a foodevent, so we approached some areas of the federal government for thisinformation. Those initial conversations led to them becoming interestedin what we were doing, and which eventually led us to start working withthem to develop a system based on those designs.

Over the last 4 y this development has grown considerably; nowthere are a wide range of people from different backgrounds involved:from the scientific community, the Food and Drug Administration,Homeland Security (through the “National Center for Food Protectionand Defense”), and also other agencies like the Centers for DiseaseControl, the USDA, from industry, and many others. And much of thework is now funded in part by the FDA and in part by Homeland Se-curity though the National Center for Food Protection and Defense,although some is still privately funded.

As I said, the intent of the system is to enable those involved in re-sponding to food contamination events to understand the potentialconsequences of these types of incidents and then to manage andimprove the speed and effectiveness of their responses. The main thingthat differentiates our system from other models is that it puts all thefactors that influence the consequences of a food event together intoa single model that models the entire evolution of a food event, startingat the point of food production (for example the farm) and following itthrough to the conclusion and an evaluation of the ultimate consequences

– for example, the impact that it has on all of the various affected stake-holders, like consumers, the Public Health System, industry, and so on.

When we started to study the consequences of food contaminationincidents we realized fairly quickly that the magnitude and type of con-sequences of an incident may vary greatly depending on many of thespecific incident characteristics, such as the type of food product, thenature of the contaminating agent, how much product is contaminated,and so on. For example, the impact of an E. coli O157:H7 contaminationof lettuce may well be very different from the impact of an E. coliO157:H7 contamination of ground beef. So to get the required accuracyin each different event scenario we cannot develop a single model thataccommodates all foods and all agents, but instead have to model eachcombination of agent and food in which we are interested separately.

So our model must be very focused and specific. We are not tryingto provide a model for every possible food event that may break out; weare trying to answer specific, well defined questions. But there are manysuch questions, because there are many foods of interest – produce,seafood, meat products, etc., and also many agents of interest, E. coliO157:H7, Clostridium botulinum toxin, Salmonella, Enterobactersakazakii, and so on. So there are an enormous number of possiblecombinations. And to develop unique models for even a small subset ofthese would be prohibitively time-consuming and expensive.

To overcome this we are developing a generalized system that willaccommodate all types of event involving any types of agent and food.We do this by decomposing each event into the entire set of distinctphases that have a significant affect on the event’s consequences,and then decompose each phase into its distinct characteristics.

It is probably easiest to explain this by drawing a parallel with buildinga house. There are, of course, an enormous number of different types ofhouses, with different appearances, contents, prices, etc. However, almostall houses share the same basic parts. They all have bathrooms, bed-rooms, closets, living rooms, and so on. And, while many of these roomsmay look quite different, the things that go into each room are largely thesame. For example: all rooms have floors, walls, and so on. There aresome things that are specific to any given type of room, for example, bath-rooms have faucets, baths, mirrors, etc., whereas dining rooms don’t. Buteven then most of the items that are unique to a specific type of room arethe same for all rooms of that type. So, if you were a house builder, howwould these similarities affect the way you would build a new house?

First, and most importantly, you must decide on the objectives forthe house. Do you want to build single-family houses which would be

A predictive modeling and decision-making tool . . .


appropriate for many different types of family, or specific, customhomes, for example a home for a person who loves open spaces andwants the whole house to be one big room? The way that you ap-proach your project will be quite different depending on these choices.

And you have very similar considerations when you start to builda computerized modeling system; you have to accurately define yourobjectives. So when we started building this predictive modeling sys-tem, which we call the “Consequence Management System” (CMS),the first thing that we did was to establish a very specific set of objec-tives for it. The most important of these objectives is that the CMS mustbe able to accurately predict the ultimate consequences of any typeof food contamination incident, including the effects of any interven-tions, and to present those predictions in a form that is easily under-stood by the average user. So our objective, to use our home con-struction analogy, was to build a model that is the equivalent of the“multi-purpose, single family home”; not the equivalent of a customhome that would only meet the needs of one buyer.

And, continuing with this analogy, when you have established theobjectives for the home, then the next step is to develop a detailedarchitectural plan for the construction of the home that will ensure thatthe development will meet your defined objectives and that all of thecomponents will integrate seamlessly into the whole. By developing theplans first you can get an accurate estimate of the tools and buildingmaterials you will need, and also, if you are planning to build severalhouses, or a subdivision, then you can get additional benefits and savea lot of money by using this planning process to find the ways in whichthe homes that you want to build are similar, and then capitalizing onthese similarities by mass-purchasing or mass-producing those mod-ules. This will enable you to get economies of scale while still making surethat each home is unique in the ways that are most important to thebuyer: color, number of rooms, size of the dining room, décor, or whatever.

Similarly, after defining our objectives for the CMS we then designedan overall architectural plan for how to develop it to meet those needs.And because we wanted to be able to model all types of event, we cre-ated an architecture that enables us to get all the possible economiesof scale by taking advantage of the similarities between events. We dothis by decomposing all different types of food event into a set of phases,where each phase represents a step in the evolution of the event thathas a significant impact on the consequences; and then integrating allthese phases together into a single model of the entire event. We havecurrently identified fourteen such phases, for example: food sourcing,food distribution; agent characteristics; agent-food interaction; agentdose response, and so on, however this list is still increasing.

Back to our home construction analogy: when you have finishedyour plans and you know exactly the design for the home or homesthat you want to build, then the next thing that you have to do is collecttogether the resources that you need for the construction. To build ahome the resources are a crew and the right set of tools. Some ofthese resources are for specific purposes, for example if you are goingto have tile floors then you need a tiler and a set of tiling tools, to in-stall the plumbing you need a plumber and a set of plumbing tools; etc.And then you need general resources, like laborers and general usetools like hammers and screwdrivers. However, while you may needmany different types of resource, the resources required for eachroom of the house, and for each different house, generally remain thesame. For example, due to aesthetic or other differences you may usea different type of tile in different rooms, but the tiling resources willapply all the tile, regardless of type. Similarly the plumbing resourcesare used for all the plumbing in the house; and the general use resourc-es are used throughout the house. So you do not need to put togetherdifferent sets of resources for different rooms, or even for differenthouses. Instead, if you get the right resources to start with you canuse them over and over for different rooms in the same house and for

other, houses. So if you are going to build a subdivision you get reallygood resources, as they will last.

In our case, as I said earlier, we decompose food events into phas-es, which in our analogy loosely correspond to the various rooms ofa house. And in the same way that the various rooms have differentsets of components, most of which can be built with similar resources,so, similarly, the various phases of an event can have different setsof characteristics, but most of these characteristics can be modeledwith similar resources.

So, as a part of our planning for CMS we identified all the differenttypes of model construction resources that we need to build the variousevent phases, and, like with our house construction, sometimes uniqueresources are required to model a specific characteristic of a phase,but in most cases the resources are applicable across all types ofphases. We then built these resources into CMS. So, the CMS is likea collection of resources that enable you to model the various phasesof food events, some of these enable you to model how a food issourced, others enable you to model how it is distributed; others helpyou model agents, how an agent interacts with a human to make themill, how the Public Health System will respond when an ill person entersthe system, and so on.

But I expect that by now you are thinking that this sounds a verycomplicated system, and this is true of some of the internal workingsof CMS. It has to be, because the internal workings of some phases ofthe evolution of a food outbreak are pretty complicated. However, youmay recall that I said that one of our fundamental objectives is to be ableto present the results in a form that are easily understood by the averageuser. So, while we work very hard to make sure that that the internalworkings of the system are a science-based, accurate reflection of thereal life evolution, we try equally hard to take all of that complexity andimplement it in an interface that shows the effects of the evolving sim-ulation to the user in an impactful, real-time, and visual way.

Those of you who were at Dr. Offutt’s presentation this morning mayrecall that she said “you can put up lots and lots of data, but if you givethem a map to look at, they like it and understand it”. We agree with thatcompletely. So, one of the ways in which we show the results of our sim-ulations is as a map of the United States that changes in real time to showhow the simulated event progresses geographically over time. And toshow the temporal progress of various stages of the event, the CMSoutput also has a set of bar charts that depict the number of cases thatare present at each stage of the event at each point in the simulation.

To implement all of the required complexity in a system that runs inreal—time requires a huge amount of flexibility from our modeling en-vironment, and unfortunately none of the existing off-the-shelf modelingtoolkits provides that amount of flexibility, so we are developing CMS inoriginal program code. In this way we can ensure that can support alltypes of event, from small, unintentional events like the illness that resultswhen a bird flies over a field and deposits some E-coli O157:H7 onseveral heads of lettuce, up to large, intentional events, like one thatwould occur if bioterrorists crop dusted a field with some nasty agent.

And CMS can model any number of different phases for each event,and any of those phases can be modeled using any of a wide range ofpossible characteristics, each of which is available to any phase. Andbecause CMS is written in original program code, when we run into aunique characteristic that we have not seen before we can build it intothe CMS ourselves. And we expected this to happen frequently, in partbecause we are constantly moving on to new foods and agents that wemake new demands, but also because the current knowledge of manyexisting bioterrorism agents is still changing rapidly, which often requiresnew capabilities. Because we expected this, we have structured CMSspecifically to make sure that it is easy to incorporate these types ofchange. So, with this design any new scenario that does not involve atotally different type of characteristic can be modeled by simply adding



the new data to the system, and any that requires new characteristicscan be modeled by adding the required new characteristics to CMS.

Finally, to go back to our home construction analogy one last time,when you have set the objectives, developed the plans, put together therequired development resources and you are ready to start actuallybuilding the house, then you need the building materials. And in mostcases the purchaser has very specific likes and dislikes about the color,size, texture, etc. that they want, so many of the materials must beselected specifically for each house. But again, if you are building asubdivision, you can save a lot of cost and time by giving the purchasersome alternatives to select from, and then buying the materials in bulk.

Similarly, our “building materials” are the data that we collect abouteach agent, product, etc. And we find that, to meet our objective of gettingthe sort of accuracy that we need in the predictions of the ultimateconsequences of each incident, we must model each agent/food com-bination individually. So, while we decompose all events into a set ofsimilar phases, we then have to collect the data and models to informeach of these phases separately for each combination of agent andproduct. But in many cases we can get economies of scale, becausethe same food manufacturer that makes food X also makes food Y, sowhen we go in to talk to them, we can collect at the same time the in-formation for all the products that they produce that are of interest to us.

But accurately modeling a food incident often requires a lot of data.For example, for each selected food we must collect detailed data tobuild a model of the entire movement of the food through its distributionchain, both geographically and temporally, from production to con-sumption, that is, where does the food go and how long does it taketo get there? These data must be very specific. So take, for example,lettuce sourcing. Lettuce is, of course, a seasonal product. So if youbuy your lettuce in summer it will probably come from Salinas, Cali-fornia. If you buy it in the fall it could still come from Salinas, but alsocould come from Huron, California. However, if you buy it in the winterit will probably come from Yuma, Arizona because Yuma has the largestwinter production. So we must collect data from all those sources.

And a similar level of data is required about each of the stages of thedistribution system, such as the movement of the product from the fieldto the retail store, the characteristics of how it is received and handledin the back room of the store before it comes onto the shelf, how longit takes the consumer to go in and buy it and take it home, and so on.

Next we need consumption information, and this includes itemssuch as how long it takes the consumer to eat the product after theytake it home, how much of the product they eat over what period oftime, how that consumption varies with demographic profile, and soon. Next, a similar level of data collection is required for each agent thatwe need to support, such as how consumers in each demographicsegment would respond to consumption of various levels of the agent,what levels of contamination of the agent in the food could reasonablybe encountered, and so on. Then we need to collect data to informmodels of how many of the consumers of the contaminated productwould get ill, how ill they would get, and what they would do about thatillness, for example, would they seek medical attention?

Then we need to collect data to inform models of how the variouspatients would interact with the Public Health system, by modeling therange of likely actions on the part of the medical system (which ofcourse depends on the type of contaminating agent, the extent of thecontamination, and so on). We call this phase “Public Health Re-sponse” and it is one of the areas in which we are investing a consid-erable amount of time. Once an event happens and people around thecountry start getting ill; it describes how long we would expect it to takefor our Public Health System to receive reports of the various illness-es, then to pick out the related illnesses from the “noise” of all the otherillnesses that are occurring at the same time, so that they can relatethese various illnesses together and identify the outbreak, then to get

the information and the authority they need to decide on and initiateappropriate response. In this area we are working very closely witha group from the “National Center for Food Protection and Defense”led by Dr. Don Schaffner to build a model of that entire process to beintegrated with the CMS.

And so we collect data and build models to follow the event throughall of its phases until we reach its ultimate conclusion, and at that pointwe model the impact of the event on consumers, the Public Healthsystem, and the economic impact on industry and the country, etc.However, hopefully, before that ultimate conclusion, somewhere alongthis evolution the information about the unfolding event will stimulatesomeone to initiate some mitigating intervention. For example, the PublicHealth System receives enough reports to enable them to recognizethat an outbreak is actually happening and they decide to intervene.Or affected consumers call an 800-number hotline to the company thatproduced the product and the company recognizes the outbreak anddecides to intervene, and so on. So, we also model this aspect of theevent by collecting data to build models of the processes that lead upto the recognition of the outbreak, the various intervention alternativesthat are available at each point in the simulation, and the probable ef-fectiveness of each type of intervention.

And, when we have collected together all these data and models,we use the resources built into the CMS to build the data into a com-prehensive model of a wide range of various scenarios for potentialevents, for example events with the same agent and product but withdifferent points of contamination in the food distribution chain, or withvarious levels of contamination, different quantities of contaminatedfood, different demographic subgroups, etc. So our development pro-gram has a very intensive set of data collection activities, and in thoseactivities we try very hard to collect real hard data to inform the modelwherever possible.

Some phases of the event, for some foods and agents, are veryrich in data. For example, we have data that we have sourced fromlettuce industry on every shipment of lettuce made during the last fouryears by several lettuce companies, so the food distribution for thatproduct can be modeled to a high degree of accuracy using these data.However, other phases, such as the characteristics of the responseof the Public Health system, are less rich in data, so the existing datamust be supplemented with statistical models. In other cases, like thedose/response curve for several potential bioterrorism agents, thedata pathway is very sparse, so these parts of the CMS must be in-formed almost entirely by computerized models. So our data collectionprocess is sometimes going out to various data sources and obtainingthe available data, in other cases it means going out and collectingexisting statistical models, and in yet other cases, where neither datanor models exist, we must build new statistical models.

Occasionally during this data collection process we find that there areseveral different data sources for a particular characteristic of a phase,and in some cases these different data are in conflict, with some peoplebelieving one data set, and others believing other data sets. In thesesituations our job, we believe, is not to select between those data andmodels, but to provide a system that will accommodate all of thosevariations and let the user select which variant they prefer, and tochange between variants, and even, if they prefer, to put in their ownmodel. To achieve that, the CMS supports different, user selectablealternative models for most phases, and comes with a data entry systemand a complete statistical modeling subsystem that enables experiencedusers to even write their own formulae for complex statistical distributions.

So, there are large amounts of data and models to collect and build.But we are not trying to do this collection alone; we are working coop-eratively through relationships with a large number of external entities.We are receiving considerable help from a number of groups at the“National Center for Food Protection and Defense” and from FDA, CDC



(who are holding expert elicitations at our request on some of theagents that we’re working on), USDA, ERS, a large number of industrypartners, and the scientific community. Plus, we are of course fillingin the gaps with our own teams of experts in the various fields. And allof this is leading to CMS becoming a very data rich environment.

So, you may wonder what a user sees in an actual simulation runwith the CMS. Well, to start a simulation using the CMS, the user mustfirst select the specific characteristics of the outbreak to be simulated,such as the food and agent involved, the point at which the productis contaminated, the quantity of product contaminated, the level ofcontamination, and so on. CMS helps in this selection by suggestingreasonable values. For example, if you select the point of contaminationas the field, then it will suggest the quantity of product based on theaverage field size, but the user can then change that selection at will.As the simulation runs it clicks off the hours since the simulated con-tamination happened on an elapsed time counter, and at each of thosesimulated hours it shows the output in three distinct displays:

The first display is a map of the United States that shows the geo-graphic location of the product as it moves through the distributionchannel. As the event moves through the different phases at a specificgeographic location it shows the progress by displaying a symbol at thelocation that is colored to indicate the current state of the event there.So, for example, when the product first arrives at a retail store at the newlocation a colored symbol appears on the map at that location. As thefood at that location then moves to the home the color changes, and thecolor continues to change to show when it first gets consumed, thenwhen the first consumer at that location gets ill, and so on.

The second display is a bar chart that shows the quantity of prod-uct at each stage of distribution at the specific point in time in the sim-ulation. So, for example, bars on the chart represent the quantity ofproduct that remains in the field, on the truck, at retail, and so on. Fol-lowing consumption, the bars indicate the quantity of product con-sumed, the number of resulting illnesses, and so on. Then subsequentbars show the number of people who have received medical attentionthrough the Public Health System, and mortality if that occurs.

The third display mechanism is a set of textual displays that showthe current state of epidemiologic knowledge surrounding the eventand the current evaluation of the event impact.

The first of these text displays shows each significant change in theknown epidemiologic information about the event, such as when the firstillness occurs, when the first report is made to the PHS, when the PHSwould identify the agent, and so on. The second text display shows thesystem’s evaluation of the total impact of the event at the current time usingthe user selected impact metric. The system will support a range of dif-ferent impact metrics, including “Quality Adjusted Life Years” (the impacton the quality of life of each of the patients with distinct forms of diseasepresentation), the economic impact on various sectors of the economy,and so on, and the user can select which impact metric they prefer to see.

But part of the power of the system is that, in addition to showing theconsequences of the event itself, it also demonstrates the effects ofimplementing mitigating interventions, and the effects changes in the timingof those interventions. At any time during the simulation the user can clicka button to indicate that they want to invoke an intervention, and in re-sponse CMS will display a list of the interventions that are feasible at thatpoint in time. The feasible interventions are selected by CMS based onwhat epidemiologic information is known at the time the user requests theintervention. So if, for example, the user initiates an intervention prior tothe time when the epidemiologic models indicate that the outbreak wouldbe recognized by the Public Health System, then the system indicatesthat the only interventions that are available are those that would beappropriate if you have received some advance warning of a pending orongoing outbreak (for example a phone-in threat). If that is not the casethen the user can click a button to allow the simulation to proceed.

If they initiate an intervention at a later time, for example when theepidemiologic models indicate that the outbreak would be recognized,and the agent identified, but not the product, then it will show a list of theinterventions that are appropriate to that specific level of epidemiologicknowledge, and so on throughout the entire life-cycle of the outbreak.And so, throughout the entire event evolution, every hour the CMS showsgraphically how many locations it estimates are involved; how widelythey are geographically dispersed; the quantities of product or peopleat each stage of distribution, consumption or morbidity; and a summaryof the total impact of the event to that point, in the user’s chosen impactmetric. And based on what they see, the user can select to invoke anyinterventions that may be available, and if they do then the evolution ofthe event is changed, to reflect the changes that would result from thatintervention, and the impact would also change accordingly.

So, CMS allows the user to assess the likely consequences of a foodcontamination event, and also to assess how changing the parametersof various phases of the event, for example by increasing the level ofcontamination of the agent, or by intervening in the event by making apublic announcement, would impact the entire consequence of the event.

So, how can this information be used? Well, I think Col. Hoffmanarticulated the answer to that far better than I could when he said thatin order to evaluate the risk associated with a specific food event youmust be able to evaluate the consequences of the event, and that iswhat CMS helps you do. So, the consequence evaluations that it pro-vides are useful in advance of any event for planning, to evaluate whatproduct/agent combinations are of greatest potential impact, and howwe would expect to see those events evolve. But the CMS can alsobe useful during an event, because they give an understanding of howfast products move, how fast events evolve, and how quickly we haveto intervene for the intervention to be effective, and so on. And, anotherobvious use of the CMS is in training in food safety and food defense,for table-top exercises and other similar activities.

In conclusion, I have described the process that we are using tobuild the CMS, and that the process requires extensive cooperationfrom many sectors of government and industry. And I have describeda little of the excellent cooperation that we are receiving from manyareas of government – including the FDA, the Dept. of HomelandSecurity, CDC, USDA, CDC the Public Health System, and manyothers. But probably one of the most important areas of CMS is the waythat it characterizes the distribution system for a food, and often to dothis we require the collection of confidential data from industry. Toensure that we can maintain a very high level of confidentiality of suchdata we have spent the last year and a half with our lawyers workingwith the lawyers for FDA, for the National Center for Food Protectionand Defense and with lawyers representing various sectors of privateindustry to work out a contractual arrangement under which we can,as a private company, collect data from other private companies whilefully protecting their interests.

Under this agreement, when we gather confidential data it is totallyanonymized, and all such data is incorporated into the CMS solely inaggregation with data from other sources through statistical averagesthat prevent the individual data source from being identifiable. Then,when we have completed our processing of the data the original dataare destroyed. Further, since BTSafety is a private entity, we are notsubject to discovery through the Freedom of Information act. So thesecontractual arrangements give us the ability to collect all the data thatwe need, while providing the companies that provide the data with theassurances that the confidentiality of their data is totally protected.

Finally, our thanks to the many agencies, universities, companiesand individuals who are collaborating with us on this project, and aspecial thanks to the Food and Drug Administration, to the Dept. ofHomeland Security and to the National Center for Food Protectionand Defense for providing funding for this work.

Andy Jaine, BTSafety, LLCAndy Jaine, BTSafety, LLC

Consequence Management SystemConsequence Management System

Consequence Management System (CMS)

Consequence Management Consequence Management System (CMS)System (CMS)

A predictive modeling tool to help A predictive modeling tool to help enhance government and industry enhance government and industry response to contaminations of the response to contaminations of the

food systemfood system

What does CMS do?What does CMS do?What does CMS do?Models the entire evolution of food Models the entire evolution of food eventseventsQuantify the timing and Quantify the timing and consequencesconsequencesof the eventof the event

Consumer exposure and outcome Consumer exposure and outcome Impact on public health infrastructureImpact on public health infrastructureImpact on the economy, business, public, etc.Impact on the economy, business, public, etc.Effect of different interventions Effect of different interventions Containment and remediationContainment and remediation

Integrates and builds on the results of Integrates and builds on the results of stage modelsstage models

What can CMS be used for ? . . . What can CMS be used for ? . . . What can CMS be used for ? . . .

““WhatWhat--If” scenario planningIf” scenario planningQuantify morbidity, mortality, economic impactQuantify morbidity, mortality, economic impactIdentify impacts on affected constituenciesIdentify impacts on affected constituencies

(consumers, general public, health care, food industry, governme(consumers, general public, health care, food industry, government) nt)

Assist in decision making and priority settingAssist in decision making and priority settingFacilitate allocation of resourcesFacilitate allocation of resources

Consequence assessmentConsequence assessmentHelps weigh the cost/benefit of various policy and intervention Helps weigh the cost/benefit of various policy and intervention decisions decisions Illustrates the time frames that would maximize the Illustrates the time frames that would maximize the effectiveness of policies and actionseffectiveness of policies and actions

TrainingTrainingFood system, agents, crisis managementFood system, agents, crisis managementTable top exercises Table top exercises

Structure of CMSStructure of CMSStructure of CMSA tool that enables modeling of the evolution of all A tool that enables modeling of the evolution of all types of food event temporally and geographicallytypes of food event temporally and geographically

DataData--centriccentric -- reflects real data and real prior incidentsreflects real data and real prior incidentsVisualVisual -- easily visualize and assess the impact of easily visualize and assess the impact of decisions decisions FlexibleFlexible -- accommodates all reasonable scenariosaccommodates all reasonable scenariosPracticalPractical –– operates when some attributes are operates when some attributes are unknown or impreciseunknown or impreciseExtensibleExtensible –– facilitates easy enhancement to include facilitates easy enhancement to include improved data and models as they become availableimproved data and models as they become available

CMS System CMS System CMS System

U.S. Patent 6874000U.S. Patent 6874000

““The Consequence Management System The Consequence Management System has resulted in a collaborative program has resulted in a collaborative program involving FDA, DHS, USDA, CDC and involving FDA, DHS, USDA, CDC and EPA. Government recognizes a EPA. Government recognizes a public/private partnership is critical for public/private partnership is critical for defense of the U.S. food supply. I applaud defense of the U.S. food supply. I applaud and encourage continued collaboration and encourage continued collaboration by the private sector in the development by the private sector in the development of the CMS.”of the CMS.”

Lester Crawford Lester Crawford Former Commissioner Former Commissioner of FDA July 11, 2005of FDA July 11, 2005

NCFPD Industry WorkgroupNCFPD Industry WorkgroupJuly 12, 2005July 12, 2005

Protection of Confidential DataProtection of Confidential DataProtection of Confidential DataSecure server installed at BT Safety used Secure server installed at BT Safety used exclusively for data storage (no internet exclusively for data storage (no internet connection)connection)Password protected and limited accessPassword protected and limited accessCompanies have the option of customized Companies have the option of customized confidentiality agreementsconfidentiality agreementsData sanitized of all company referencesData sanitized of all company referencesCompany participation acknowledged only if Company participation acknowledged only if permission is granted by the companypermission is granted by the company

Data ConfidentialityData ConfidentialityData Confidentiality

Original data destroyed or returnedOriginal data destroyed or returnedWhenever possible, aggregate data Whenever possible, aggregate data used in the modelused in the modelAs a private entity, data collected by As a private entity, data collected by BTSafety is not discoverable through BTSafety is not discoverable through Freedom of Information Freedom of Information

The project is a collaboration of a number The project is a collaboration of a number of agencies, universities and companiesof agencies, universities and companies

National Center for Food Protection and Defense Industry Workgroup

National Center for Food Protection and National Center for Food Protection and Defense Industry WorkgroupDefense Industry Workgroup

We gratefully acknowledge funding by the following

organizations:

We gratefully acknowledge We gratefully acknowledge funding by the following funding by the following

organizations:organizations:

a predictive modeling and decision-making tool to facilitate

Documents