Environews Spheres of Influence

An Agency Self-Exam

Risk Assessment

at theEPAT he U.S. Environmental Protection Agency (EPA) has for the

first time conducted an internal investigation of its own

approach to risk assessment. The investigation’s results are contained in

a 193-page staff paper titled An Examination of EPA Risk Assessment

Principles and Practices, released in final form 25 March 2004. The

staff paper is not a guidance document—rather, it is a snapshot of how

risk assessments are currently performed at the EPA. The paper also

provides recommendations for how the agency can strengthen and

improve its risk assessments.

“We’re not going to change this document,” says Kerry Dearfield, a

senior scientist in the EPA Office of the Science Advisor, who played a

key role in coordinating the effort. “We want people to look at it and

determine if they agree with its conclusions or not. The intent is to cre-

ate a dialogue about how we can move EPA risk assessment forward.”

Environmental Health Perspectives • VOLUME 112 | NUMBER 8 | June 2004 A 483

Spheres of Influence | Risk Assessment at the EPA

Get

ty Im

ages

; Mat

t Ra

y/EH

P

The inspiration for this effort arose froman unprecedented February 2003 request bythe White House Office of Managementand Budget (OMB) for public comment onrisk assessment procedures across the federalgovernment. The several hundred com-ments received, many of which focusedspecifically on methods used at the EPA,were eventually incorporated into theOMB’s 2003 annual report to Congress.The OMB’s focus on risk assessment galva-nized EPA officials, who, at the urging ofPaul Gilman, science advisor for the EPAand assistant administrator of the EPAOffice of Research and Development, con-vened a task force to review how risk assess-ments were being performed at the agency.

“This review morphed into the idea ofthe staff paper,” Dearfield recalls. “We want-ed to look at the overarching issues that thecomments had raised and look for opportu-nities to refine or otherwise revise practices.”

Hundreds of staffers throughout theEPA contributed to the effort. Dearfieldsays the task force deliberately sought theinput of staff-level risk assessors. “Thesenior managers did not contribute asmuch,” he explains. “What we were reallylooking for was the ground-level view—wewanted to know what the risk assessors areactually doing. We wanted the real truth,warts and all, and I think we succeeded ingetting that.”

Detailed Content

The staff paper itself reflects the EPA’s del-icate position as a public health agencypositioned between industry and environ-mental concerns. Much of the documentprovides detailed descriptions of methodsand discussions about how conservatism,uncertainty, and variability influence therisk assessment process. In terms of struc-ture, the staff paper is arranged around aseries of themes, including uncertainty andvariability, the use of default parametersand extrapolation assumptions, site- andchemical-specific assessments, and ecologi-cal assessment.

The paper presents many of the EPA’scurrent practices while describing ongoingefforts to make the risk assessment process

more data-intensive and robust. Opportu-nities to enhance risk assessment are alsodescribed. For instance, the reportacknowledges that accumulation of moretoxicity and exposure data will lessenreliance on defaults. The report highlightsthe need to increase the transparency andclarity of risk assessment, for the benefit ofboth risk managers, who make decisionsbased on the outcomes, and the public,who must understand the assumptionsupon which those decisions are based.

“The EPA should be commended ondeveloping and sharing this report,” saysChristopher Portier, chief of the NIEHSEnvironmental Toxicology Program. “Otheragencies, including our own, can read thisdocument and better understand the limita-tions of risk assessment as it is currentlypracticed. This will help us to do the sciencenecessary to aid the EPA in developing astronger scientific basis for risk assessments.”

To a large extent, the staff paper wasintended as a way to address concernsposed during the OMB review, particularlythose articulated by the American Chem-istry Council (ACC), which provided thebulk of comments about EPA risk assess-ment to the OMB. EPA officials, includingDearfield and Gilman, met with represen-tatives of this leading industry group atthe start of the process to discuss a rangeof issues, particularly agency approaches to

defaults and other methods to addressuncertainty that the ACC claims are overlyconservative and protective.

Peter Preuss, director of the EPANational Center for Environmental Assess-ment, says he’s not surprised at the level ofthe ACC’s involvement. “They are probablythe group that has organized itself best todeal with these issues, many of which theyhave raised previously,” he says.

With few exceptions, environmentalgroups were not engaged in dialogue withthe EPA to the same degree as industry. Themain environmental concerns were raised bythe Natural Resources Defense Council(NRDC), which proposed that conservativedefaults and other precautionary measuresare justified because populations are typical-ly exposed to contaminant mixtures whose

cumulative health impacts are largelyunknown. Jennifer Sass, a senior scientist inthe NRDC’s public health group, describesthe staff paper as a “private conversationbetween industry and the EPA.” Nonethe-less, Sass points out that, with the staffpaper, “EPA makes a scientifically credibledefense that its risk assessments are not over-ly cautious.”

The Nature of ConservatismA degree of caution is deliberately built intoEPA risk assessment to protect against theuncertainty generated by data gaps, particu-larly those relating to exposure and chemicaleffects in humans. Agency scientists usegeneric data, obtained from EPA guidancedocuments such as the Exposure FactorsHandbook, to fill in data gaps so that riskassessments can proceed. Among these para-meters are age- and population-specificinhalation rates, food and water consump-tion rates, residential exposure durations,and others.

Conservatism is also achieved withnumerical safety factors (or uncertainty fac-tors, as the EPA calls them) that account forthe uncertainty of extrapolating from animaldata to human effects. Safety factors areincorporated into the EPA’s acceptable life-time human exposure levels for pollutants.These levels typically have their basis in ani-mal data, usually the no-observed-adverse-effect level (NOAEL) from a bioassay usinga limited number of test animals. But thanksto statistical uncertainty, a NOAEL dosecould conceivably cause effects in 10–20%of a study population, says George Lucier,former associate director of the NIEHSNational Toxicology Program and now anadjunct senior scientist for EnvironmentalDefense. Safety factors are therefore appliedto account for unknowns such as inter-species differences in response and thepotential for heightened sensitivity amongsome human populations, such as children.

When combined, defaults and safetyfactors can have the effect of magnifyingcalculated risk levels to a degree that manyin industry believe is unreasonable. Forinstance, when doing a screening assess-ment, it’s not unusual for the EPA toassume that a hypothetical resident threat-ened by contaminants at a Superfund sitemight be exposed to these agents for 30years. If one were to also assume the resi-dent is maximally exposed to these contam-inants via consumption, inhalation, anddermal pathways, then the toxicity threatcan appear more extreme than it likely isamong most individuals.

In its comments to the OMB and in dis-cussions with the EPA, ACC members sug-gested the agency’s selections of defaults and

A 484 VOLUME 112 | NUMBER 8 | June 2004 • Environmental Health Perspectives

Spheres of Influence | Risk Assessment at the EPA

hat we were really looking for was the ground-level view— we

wanted to know what the risk assessors are actually doing. We wanted

the real truth, warts and all, and I think we succeeded in getting that.

–Kerry DearfieldEPA Office of the Science Advisor

safety factors constitutes a set of “policydecisions” made with insufficient considera-tion of the data from which the values werederived. Moreover, the ACC claims the var-ious judgments that go into selecting thesevariables are not well described in EPA riskassessments, thus undermining the trans-parency and clarity of the process.

“The EPA needs to quantify the impactof [its] choices,” says Leslie Hushka, a toxi-cologist with ACC member companyExxonMobil. “It’s essential to evaluate moreoptions, to be more objective, and to be pol-icy-neutral in developing risk assessments.”For example, she says, even though theEPA’s risk assessment guidelines recom-mend an examination of risk ranges andaverages (or central tendencies) in hazardand exposure assessments, risk assessorschoose numbers on the high end. As a result,she says, “The agency and the public simplydo not know what the consequences of thesechoices are.”

Examining AlternativesIn response to these comments, the EPAstaff paper acknowledges that clarity andtransparency comprise “aspect[s] of EPA’spractices that need strengthening.” Defaultparameters, which the agency has no inten-tion of abandoning, are described in thepaper as “appropriate . . . within the range ofplausible outcomes . . . and based on pub-lished studies, empirical observations,extrapolation from related observations,and/or scientific theory.”

The staff paper also highlights ongoingefforts to ensure that “nothing appears hid-den or buried in an assessment—so thatnothing keeps one from understanding theimpact of the elements that go into estimat-ing and characterizing risk.” These effortsinclude, among others, the EPA’s new can-cer guidelines, expected to be finalized in thefall of 2004, which emphasize a full exami-nation of data before invoking defaults; thedevelopment of new models to support riskassessment; and an upgrading of the agency’sIntegrated Risk Information System, whichis the principle source of toxicity values usedto describe chemical hazards.

But the paper also concedes a need for“better communication of the data, assump-tions, and choices used in risk assessment,”noting that “close attention to our guidancedocuments will ensure transparency and clar-ity.” An opportunity to better characterizeuncertainty, the paper suggests, is providedby probabilistic modeling, a statistics-basedmethod for risk assessment long championedby industry.

Probabilistic models substitute distrib-utions of values for “point estimates,”which are single values that describe given

variables in the risk equation, such asexposure frequency or duration. In a prob-abilistic model, computers continuallyselect values from a range of data points foreach parameter, running through the calcu-lations thousands of times until measures ofcentral tendency are achieved. Industry haslong argued that probabilistic model out-puts closely mirror real-world exposure sce-narios, in contrast to point estimate meth-ods, which are easily biased high or low,depending on chosen values.

Preuss acknowledges that the agency hasbeen cautious in its use of probabilisticmethods thus far. “It’s not a method thatyou apply willy-nilly or in all cases,” he says.“You need a fundamental set of data to useit. But we’re in favor of it and will apply itwhere we feel it brings some added value.”

Consistent with this view, the staff papernotes that the EPA “should encouragegreater use and reliance on probabilisticmodeling when appropriate.”

According to Preuss, the Risk Assess-ment Forum at EPA, coordinated by hisoffice and comprising scientists from acrossthe agency, is currently looking for oppor-tunities to apply probabilistic methods.This forum studies scientific issues withinthe agency and advises policy on the basisof their findings. “We’re also looking at aspectrum of other issues—for instance,how choices are made in deriving referencedoses and concentrations, and how to do abetter job of assessing cumulative expo-sure,” he says.

The issue of cumulative exposure tochemical mixtures is described in the paperin some detail. The paper references a stake-holder comment submitted to the OMBthat accuses the EPA of “assuming the toxi-city of a chemical mixture is equal to thesum of the toxicity of each individual chem-ical, regardless of the toxicity type [or] com-petition or antagonism among chemicals.”The EPA addresses this charge by notingthat current guidance directs risk assessorsto consider chemical interaction data“whenever possible.” Even so, the staff pa-per recommends that agency scientistscontinue to aggressively “flesh out ap-proaches to cumulative risk . . . to produce

the most scientifically rigorous evalua-tions that the state-of-the-science canaccommodate.”

Preuss says a point that became abun-dantly clear as the internal examinationunfolded is that stakeholders are oftenunaware of technical progress at the agency.“Many of the comments allude to risk assess-ment practices that people believe we usethat in fact we do not use,” he says. “Thesediscussions have gone on for many years,and meanwhile the methods and approacheshere have become vastly more sophisticated.It’s important that people who make thesecomments are aware of these advances.Frankly, I’m hopeful that with the work thatcomes out of this staff paper, a lot of theseissues can be put to bed, and we can moveaway from these old discussions.”

A Springboard for ChangeLooking ahead, EPA officials are opti-mistic the staff paper will serve as a vehicleto open dialogue among staff, managers,and stakeholders. Dearfield highlightsplans to hold workshops with the EPA’sScience Advisory Board, in addition toindustry and environmental groups, pro-fessional societies, and other outside par-ties. A request for comments on issues forfurther discussion, made in the FederalRegister the day the staff paper wasannounced, is currently in place, with adeadline for submission of 23 June 2004.Meanwhile, risk assessment will go on asone of the EPA’s most important func-tions—a paradigm through which all thatis known about toxicity and humanresponse can be funneled directly into reg-ulatory decision making.

“This document could not have comeat a better time,” says Portier. “The paceof science has increased over the last tenyears, with molecular biology and elec-tronics serving as the key catalysts. As theEPA advances through the next decade, itneeds to assess how these technologies willbe used to improve risk assessments.Knowing fairly and honestly what is donetoday will play a critical role in formingwhat can and will be done in the future.”

Charles W. Schmidt

Environmental Health Perspectives • VOLUME 112 | NUMBER 8 | June 2004 A 485

Spheres of Influence | Risk Assessment at the EPA

he EPA needs to quantify the impact of [its] choices. It’s essential to

evaluate more options, to be more objective, and to be policy-neutral in

developing risk assessments.

–Leslie HushkaExxonMobil