mcclay 1995 beyond before and after

8/6/2019 McClay 1995 Beyond Before and After

1/6

Alberta Environmental Centre, Bag 4000, Vegreville, Alberta T9C IT4, CanadaA.S. McClay

Proceedings of the Eighth International Symposium on Biological Control of Weeds2-7 February 1992, Lincoln University, Canterbury, New Zealand.Delfosse, E.S. and R.R. Scott (eds). DSIR/CSIRO, Melbourne, pp. 213-9 (1995).

projects. I will also argue that the experimentalmethod is the preferred approach for evaluatingthe effectiveness of biological control, and thatwe need to give more attention to experimentaldesign, replication, and statistical analysis thanhas been done until now.Elements Needed in Evaluation of BiologicalControl

1. Assess the Weed, Not the Agent

The requirements for evaluation of a weedbiological control project have been consideredby many authors (e.g., Crawley 1990, Harris1981, McEvoy et at. 1991, McLaren and Amor1979, Wapshere et al. 1989). I consider themost important elements to be:

Even the most thriving biological control agentwill not control its target weed if it does notcause damage of a type to which the weedpopulation is vulnerable. Post-release studiesfocused only on the performance of thebiological control agent-its establishment,

Beyond IIBefore-and-After: II Experimental Design and Evaluationin Classical Weed Biological Control

Classical biological control is often described asan ecological field experiment on the largestscale (Myers 1978). The introduction of weedbiological control agents into new regionsprovides opportunities to study many intriguingquestions in population dynamics, genetics,environmental adaptation, and plant-insectinteractions. In our fascination with thesequestions affecting the mechanism of biologicalcontrol, however, we should not lose sight of theoutcome of the process. In this paper I will,therefore, focus on a single, specific question,which is central to the business of appliedbiological control. This is: how do we show thata biological control agent is effective in reducingthe severity of a weed problem? I do notconsider here the separate question of whatdegree of control is regarded as "sufficient", butonly the methods used in assessing the degreeof control. I will consider the elements requiredfor evaluating the effectiveness of a weedbiological control project, review the methodsdescribed in the published literature, andsuggest some guidelines for evaluation of future

Introduction

To evaluate the effectiveness of weed biological control agents, it is necessary to determinetheir effects on weed populations in the field. The experimental approach, whose keyelements are replication of experimental units and randomization of treatments, is the bestway to separate these effects unambiguously from other sources of variation. I surveyed 57published evaluations of classical weed biological control projects. Only 16 studies usedexperimental methods, and among these only 6 used measures of weed population.Randomization, formal experimental design, and thorough statistical analysis were rarelyapplied. Possible reasons for the infrequent use of experiments include the mobility of manybiological control agents, the logistic difficulty and long duration of the required experiments,the shortage of agents in the early stages of projects, and the spectacular nature of somehistoric biological control successes, which appears to make quantitative evaluationunnecessary. A greater emphasis on sound experimental design is necessary to preservethe credibility of classical weed biological control.


2/6

214 Proceedings of the Eighth International Symposium on Biological Control of Weeds

Literature Survey

experimental route. These approaches differ,not in the technical complexity of their methodsor of their statistical analyses, but in the level ofintervention by the investigator. In observationalstudies the choice of sites or plants attacked bythe biological control agent is left to the agentitself, or to factors external to the study, while inexperimental studies the investigator controlsthe allocation of attack. These 2 categorieswere described by Hurlbert (1984) respectivelyas "mensurative experiments" and "manipulativeexperiments. "The observational method depends ondocumenting a correlation in space or timebetween attack by the biological control agentand a reduction in weed populations. Althoughthis may be fairly convincing evidence in c1ear-cut cases, this approach can never conclusivelyseparate the effects of biological control agentsfrom those of other environmental factors. If weobserve that at sites where the agent isabundant, the weed population is lower, thismay be because the agent is controll ing theweed. However, it may also be because theagent prefers, for example, dry sites, and theweed grows better in moist sites. Correlations intime are subject to similar problems. Forexample, it has been observed at a number ofsites that release of Ceutorhynchus litura (F.)(Coleoptera: Curculionidae) has been followedby a decline in Canada thistle popUlations(Peschken and Wilkinson 1981, Rees 1990).However, it is not yet clear whether the insect isresponsible for these declines. The advantageof the experimental approach is that byallocating biological control treatments toexperimental units according to an appropriatedesign, their effects can be separated fromthose of all other environmental factors affectingthe weed population.

With these principles in mind, I surveyed 57published papers describing the approachesused in evaluation of specific weed biologicalcontrol projects against 33 target weeds in 10countries. Papers were selected from previousInternational Symposia on Biological Control ofWeeds, from a computerized literature search,from my own reprint collection, and by following

4. Prove Responsibility of the Agent

The outcome of classical biological controldepends on many environmental factors actingover a prolonged time on both the weed and thebiological control agent. These factors may bedifficult or impossible to reproduce ingreenhouse or artificial field plot conditions. Thebest test of the ability of an agent to providecontrol is therefore to study it under natural fieldconditions.

population dynamics, dispersal, etc.-cantherefore not answer the question posed above.

Weeds cause problems because they occur aspopulations. Any evaiuation of weed controlmust, therefore, be based on some measure ofweed population, expressed on a per unit areabasis. Many frequently-used measures, such asdimensions or biomass of individual plants, areexpressed on a per plant basis and thus carryno information on the weed population. Others,such as percent infestation, are really measuresof biological control agent population rather thanof weed response. Such measures may bevaluable in studies of the mechanism ofbiological control, but they cannot themselvestell us if biological control has been successful.

2. Assess Weed Populations, Not Individuals

3. Assess in the Field

The method of evaluation used must be able toensure that effects are properly attributed totheir causes; that is, to separate reductions inweed population caused by the biological controlagent from spatial and temporal variation due tofactors such as weather, soil conditions,competition, allelopathy, management, andother phytophages and diseases.The essential problem is, therefore, to takesome measure of a weed population in the fieldand demonstrate, against a background ofnatural variability, that the introduction of abiological control ageni causes a reduction inthat measure: in other words, to detect atreatment effect and separate it from othersources of variability. We can approach thisproblem by either the observational or the


3/6

Beyond "Before-and-After" in Weed Biological Control

Types of Measures Used

215.S. McClay

Forty-two papers (74%) used systematicobservational methods of evaluation; the mostpopular approach (29 papers) was a longitudinalstudy, in which one or several sites weresampled repeatedly over a period ranging frommonths to years. If the study period begins at orbefore the time of agent release, this is thetraditional "before-and-after" study. Only 9 ofthe longitudinal studies, however, reported datarecorded at or before the time of release, andmany began several years after agentestablishm,ent. Ten studies reported data fromcontrol or unattacked sites or plants, and 13gave neither pre-release nor control site data.The other frequently-used observationalapproach (15 papers) was to compare datataken simultaneously from sites or plantsattacked by the biological control agent and fromsites or plants to which the biological controlagent has not spread. These differed from theexperimental studies referred to above in thatthe selection of release and control sites wasnot made by the investigator as part of thestudy.Six papers (11%) used only an informal ordescriptive approach to evaluation; thesepapers may have included some quantitativedata, but did not describe a systematic samplingscheme or experimental design. A further 27papers (47%) contained an informal ordescriptive account as well as more systematicobservational or experimental data. The use ofsequential photographs is often recommended(Goeden and Ricker 1981); such photographswere used in 7 (12%) of the papers.

Forty-five papers (79% of the total) reportedmeasures taken at the level of individual plants.The most commonly used (34 papers) weremeasures of intensity of attack, such aspercentage of stems mined or number of larvaeper seed head. Numbers of structuresproduced (leaves, flowers, stems etc.) werereported in 24 papers, plant dimensions (height,diameter etc.) in 20 papers, reproductive outputin 19, mortality, longevity, etc., in 13, andbiomass in 10 papers.Thirty-one papers (54% of the total) reportedpopulation-level measures on the weed. The

up citations in other papers. The criteria I usedin selecting papers were: classical biological control of a weed with anarthropod or plant pathogen; post-establishment field study in region ofintroduction; original research report, containing fielddata and information on methods; stated or implied intent to evaluateeffectiveness of biological control agent(s) incontrolling the weed in the field; and published in an accessible source in 1971 orlater.I excluded papers describing the establishment,spread, or population dynamics of biologicalcontrol agents, and containing no, or only casualor anecdotal, data on target weed impact,unless their stated intentwas to evaluate theeffectiveness of the agent. I also excludedreview or historical papers, unless theycontained details of methods and original fielddata. Although not a random sample, I believethese papers are typical of the methods used forevaluating the effectiveness of classical weedbiological control over the last 20 yrs.Aspects of the papers analysed included:the approach used for evaluation(experimental, observational, or

informal/descriptive); the measures used to estimate target weedabundance or performance; the use of replication, randomization andexperimental design; and approaches to data analysis andpresentation.A full list of the papers selected and of thecategories used in the analysis is available onrequest.

Approaches to EvaluationOf the 57 papers analysed, only 16 (28%) usedexperimental methods as defined above.Among these the most popular technique wasthe insecticidal (or fungicidal) check (10 papers).In 4 studies, release sites were compared withcontrol sites. Only 2 studies used an exclusionor confinement cage check.

Results


4/6


most commonly used was number of plants perunit area (23 papers). The only other frequentlyused population-level measures were biomassper unit area and percentage cover, bothreported in 11 papers. Reproductive output perunit area and seed densities in the soil werereported in only 2 and 1 paper, respectively.Population-level measures were used lessfrequently in experimental papers (35%) than inobservational papers (55%).Design, Analysis and Data PresentationThese were difficult to categorize, as plotlayouts, experimental designs, and samplingschemes were often described in vague terms.In the observational studies, it was notpracticable to draw a sharp distinction betweenreplication and multiple sampling, so all studieswhere quantitative measures were calculatedfrom multiple samples at any level wereconsidered to be replicated. Experimentalstudies were considered to be replicated if theyinvolved the systematic application of the sametreatments and assessment methods on morethan one experimental unit (plant, plot, releasesite etc.) Studies were identified as being"pseudoreplicated" (Hurlbert 1984) whenmultiple subsamples taken from unreplicatedtreated and control plots or sites were explicitlyor implicitly used to test for treatment effects.Of the 42 observational studies, 27 werereplicated at some level, 11 were unreplicated,and 5 were pseudoreplicated. Fourteen madereference to some randomization procedure insampling.Among the 16 experimental studies, 12 werereplicated, 1 was unreplicated, and 4 werepseudoreplicated (1 paper described a factorialexperiment which was properly replicated withregard to one factor but pseudoreplicated withregard to another). Only 6 papers referred torandomization of treatments, and 5 mentioned aspecific experimental design (e.g. randomizedcomplete block, Latin square). In 1 of thesecases a figure showed clearly that the plotlayout did not in fact correspond to the designstated in the methods section, and in someothers vague descriptions of layout left this pointin doubt.

In the whole sample of papers, most resultswere presented in the form of simple calculationof means from samples, presented graphicallyor in tables, without statistical analysis (26papers, 46% of total). In 8 papers (14%) meanswere compared by t-tests or similar elementarysignificance tests, and 11 papers (19%) statedthat analysis of variance was used. In only 1 ofthese papers, however, was a full analysis ofvariance table presented, showing sources ofvariation, degrees of freedom, mean squaresand F-values. A variety of other statistical andmathematical analyses, including regressionand life-tables, were used in 11 papers (19%).DiscussionExisting PracticeThis survey has shown that the experimentalapproach has been infrequently used inevaluating the effectiveness of weed biologicalcontrol agents. In those experimental studieswhich have been done, a majority focused oneffects at the level of individual plants ratherthan on the weed population. Data on weedpopulations were reported more frequently inobservational studies than in experimentalstudies. Almost half of the observationalstudies, however, reported neither pre-releasenor control site data, and therefore tell us littleabout the effectiveness of the biological controlagent.Although most experimental studies werereplicated in some way, few mentioned thattreatments were randomized and few specifiedany formal experimental design.Pseudoreplication was common in bothobservational and experimental studies: forexample, in insecticidal check experiments,multiple samples from each of a pair of sprayedand unsprayed plots may be treated as if theywere replicates. Clearly, very few of the 57papers surveyed would be capable of providinga statistically valid, experimental demonstrationof the effectiveness of a biological control agent.By contrast, a comparison of these results withany issue of Weed Science or Weed Researchwill show that in the evaluation of chemical weedcontrol, the use of replicated, randomized, field


5/6


to allow various hypotheses and combinations oftreatments to be tested (e.g., Anderson andMcLean 1974, Box et al. 1978, Gomez andGomez 1984). It is important to select anappropriate design at the beginning of a study toensure that a valid analysis will be possible.The appropriate experimental methods anddesign to evaluate a particular agent, orcombination of agents, against a particulartarget weed depend on many factors. One ofthe most widely varying of these is the agent'sdispersal rate. Aphthona nigriscutis Foudras(Coleoptera: Chrysomelidae) released againstleafy spurge in Canada spreads at a rate of 5-10m y(' for the first few years (A.S. McClay and P.Harris, unpublished data); in contrast, Epiblemastrenuana (Walker) (Lepidoptera: Tortricidae)released against parthenium weed inQueensland spread at a rate of up to 1.6 x 105 my(' (McFadyen 1985).When dispersal is slow, the use of replicatedpairs of release and control plots, separated bysufficient distance that the agents will not reachthe control plots for 2-3 yrs, may be a suitableexperimental method. The release plot shouldbe selected randomly from within each pair ofplots-it is not valid to select a release site andthen choose a nearby "control" site after thefact. To reduce between-plot error the releaseand control plots within each pair should bematched as closely as possible with regard totheir soil type, topography, vegetation etc. Sucherror can be further controlled by usingmeasurements of environmental factors at thesites as covariates. This is a randomizedcomplete block design with 2 treatments. Datacan be analysed by analysis of variance if theappropriate assumptions are met. A nonparametric method, the Wilcoxon signed-rankstest, is also available if these assumptions areviolated (Sokal and Rohlf 1981). This type ofexperiment could be incorporated at very littleadditional cost into a standard redistribution andmonitoring program.When dispersal is rapid, experimentalevaluation requires some type of exclusion,whether by cages, insecticides, mechanicalremoval or a combination. A good example ofthe combined use of these techniques isdescribed by McEvoy et al. (1991). Cages areexpensive and vulnerable to damage; they also

produce alterations in microclimate which mustbe controlled for by sham cages. A number ofdifferent insecticides have given satisfactoryresults in insecticidal-check trials (e.g. Brown etal. 1987, Gordon etal. 1985, Louda 1984).Plots used in exclusion experiments can befairly close together, tending to reduce betweenplot error as well as travel times and expense.Blocking and covariates can also be used toreduce between-plot error in this type ofexperiment.It is obviously not possible here, nor am Iqualified, to make recommendations on thedetails of experimental design for every possibleevaluation program. My purpose is to argue thatbiological control researchers should beconscious of the experimental nature of theirwork when evaluating the success of biologicalcontrol, that field experiments should be plannedwith sufficient replication and an appropriatedesign to test a specific hypothesis, and thatexperimental design and analysis should bedescribed in explicit detail in publications. Mostresearchers have access to statisticians orbiometricians who can advise on experimentaldesign and analysis; their help should be soughtwhen an evaluation is being planned, and notafter the data have been collected.Field experiments can certainly be expensiveand time-consuming, and funding for weedbiological control is always scarce. However, Ibelieve the onus is on weed biological controlresearchers, when preparing work plans andproposals, to provide for appropriateexperimental tests to determine whetherbiological control agents actually do the job forwhich they were released. We need to makeclear to all project clients that such experimentsare not an optional extra, but an essential part ofthe program.AcknowledgementsI thank my colleagues Lloyd Dosdall, ZackFlorence, Peter Harris, John O'Donovan, andPaul Sharma for valuable comments anddiscussions.ReferencesAnderson, V.L. and RA McLean. 1974. Design ofExperiments: A Realistic Approach. Dekker. New

York, 418 p.


6/6

USA Freeman, T.E. (ed.). University of Florida,Gainesville, pp. 181-8.

Peschken, D.P. and AT . Wilkinson. 1981. BiologicalControl of Canada thistle (Cirsium arvense): releasesand effectiveness of Ceutorhynchus litura (Coleoptera:Curculionidae) in Canada. Canadian Entomologist113:777-85.

Rees, N.E. 1990. Establishment, dispersal and influenceof Ceutorhynchus litura on Canada thistle (Cirsiumarvense) in the Gallatin Valley of Montana. WeedScience 38:198-200.

Sakal, R.R. and F.J. Rohlf. 1981. Biometry: ThePrinciples and Practice of Statistics in BiologicalResearch. Second edition. W.H. Freeman, New York,859 p.Tietjen, G.L. 1986. A Topical Dictionaryof Statistics.Chapman and Hall, New York, 171 p.Wapshere, AJ., E.S. Delfosse and J.M. Cullen. 1989.Recent developments in biological control of weeds.Crop Protection 8:227-50.

Beyond "Before-and-After" in Weed Biological Control

Box, G.E.P, W.G. Hunter, and J.S. Hunter. 1978. Statisticsfor Experimenters: An Introduction to Design, DataAnalysis and Model Building. Wiley, New York, 653 p.

Brown, V.K., AV . Gange, I.M. Evans, and AL. Storr. 1987.The effect of insect herbivory on the grow1h andreproduction of two annual Vicia species at differentstages in plant succession. Journal of Ecology75:1173-89.

Crawley, M.J. 1990. Plant life-history and the success ofweed biological control projects. Proceedings of the VIIInternational Symposium on Biological Control ofWeeds, 6-11 March 1988, Rome, Italy. Delfosse, E.S.(ed.). Ministero dell' Agricoltura e delle Foreste,Rome/CSIRO, Melbourne, pp. 17-26.Goeden, R.D. and D.W. Ricker. 1981. Santa Cruz Island revisited. Sequential photography records thecausation, rates of progress, and lasting benefits ofsuccessful biological weed control. Proceedings of theV International Symposium on Biological Control ofWeeds, 22-27 July 1980, Brisbane, Australia. Delfosse,E.S. (ed.). CSIRO, Melbourne. pp. 355-65.

Gomez, K.A and AA Gomez. 1984. StatisticalProcedures for Agricultural Research. Second edition.Wiley, New York, 680 p.

Gordon, A.J., R.L. Kluge and S. Neser. 1985. Effect of thegall midge, Zeuxidiplosis giardi (Diptera:Cecidomyiidae) on seedlings of SI. John's wort,Hypericum perforatum (Clusiaceae). Proceedings ofthe VI International Symposium on Biological Control ofWeeds, 19-25 August 1984, Vancouver, Canada.Delfosse, E.S. (ed.). Agriculture Canada, Ottawa, pp.743-8.

Harris, P. 1981. Evaluating biocontrol of weeds projects.Proceedings of the V International Symposium onBiological Control of Weeds, 22-27 July 1980, Brisbane,Australia. Delfosse, E.S. (ed.). CSIRO, Melbourne.pp.345-53.

Hurlbert, S.H. 1984. Pseudoreplication and the design ofecological field experiments. Ecological Monographs54:187-211.Louda, S.M. 1984. Herbivore effect on stature, fruiting andleaf dynamics of a native crucifer. Ecology 65:1379-86.

McEvoy, P., C. Cox and E. Coombs. 1991. Successfulbiological control of ragwort, Senecio jacobaea, byintroduced insects in Oregon. Ecological Applications1:430-42.

McFadyen, R.E. 1985. The biological control programmeagainst Parthenium hysterophorus in Queensland.Proceedings of the VI International Symposium onBiological Control of Weeds, 19-25 August 1984,Vancouver, Canada. Delfosse, E.S. (ed.). AgricultureCanada, Ottawa, pp. 789-96.

McLaren, I.W. and R.L. Amor. 1979. Assessment ofeffectiveness. Australian Applied EntomologicalResearch Conference, Queensland AgriculturalCollege, June 1979, pp. 127-40.Myers, J.H. 1978. Biological control introductions asgrandiose field experiments: adaptations of thecinnabar moth to new surroundings. Proceedings of theIV International Symposium on Biological Control ofWeeds, 30 August-2 September 1976, Gainesville, FL,

A.S. McClay 219

mcclay 1995 beyond before and after

Documents