adaptive management: making it happen through participatory systems analysis

& ResearchPaper

Adaptive Management: Making itHappen Through Participatory SystemsAnalysis

Carl Smith1,3*, L. Felderhof2,3 and O. J. H. Bosch1,3

1School of Natural and Rural Systems Management, The University of Queensland, Australia2Queensland Parks and Wildlife Service, Atherton, Queensland, Australia3Cooperative Research Centre for Tropical Savannas Management, Australia

Participatory systems analysis was used to construct system models of the operatingenvironment for fire management in conservation reserves in north Queensland, Australia.The aim of the study was to identify stumbling blocks to the adaptive management of fireand to test whether this could be done using participatory methods and a systemsmodelling tool called Bayesian Belief Networks (BBN). Results from the case studyindicate that the participatory system analysis approach provides a co-learning environ-ment that captures the collective (corporate) knowledge of the factors influencing plan-ning, implementing, monitoring and reviewing outcomes, thus allowing critical successfactors (CSFs) influencing the success of adaptive management to be identified. BBNprovided the scaffolding for piecing together this knowledge, allowing managers tostructure complex problems and conduct dynamic sensitivity and scenario analysis toidentify where intervention or investment can significantly improve the practice ofadaptive management within a natural resource management (NRM) agency. Copyright# 2007 John Wiley & Sons, Ltd.

Keywords systems thinking; Bayesian Belief Networks; fire management; natural resourcemanagement

INTRODUCTION

Adaptive management has been advocated formany years as a way of managing naturalresources in the face of uncertainty and varia-

bility, especially for systems where the outcomesof management decisions are difficult to predict(Holling, 1978; Bosch et al., 2003). The term refersto a systematic process for improving manage-ment policy and practice by learning from theoutcomes of previous operational policies andpractices (British Columbia Forest Service, 2000).The process is cyclic (Figure 1), with plans forachieving natural resource management (NRM)

SystemsResearch andBehavioral ScienceSyst. Res.24, 567^587 (2007)Published online inWiley InterScience(www.interscience.wiley.com)DOI:10.1002/sres.835

*Correspondence to: Carl Smith, School of Natural and Rural SystemsManagement, The University of Queensland, Gatton 4343, Queens-land, Australia.E-mail: [email protected]

Copyright # 2007 John Wiley & Sons, Ltd.

objectives developed using current knowledge,and monitoring undertaken to track the successof implemented management actions. Reviewingmanagement outcomes contributes new knowl-edge, which can then be used to refine manage-ment plans for future implementation.

The benefits of adaptive management aretwofold. First, it allows for decisions and actionto be based on experience and second, itengenders a culture of continuous improvementby consciously reflecting on previous manage-ment outcomes. Adaptive management makessense intuitively, however, it is not automaticallyundertaken, either by individual land managersor organizations responsible for NRM. This isbecause the activities of individuals and organ-izations are influenced by their priorities and theresources (time, skills, equipment, informationand personnel), policies and procedures thatmake up their operating environment. Hence theoperating environment of an individual ororganization is paramount to the success ofadaptive management.

The factors critical to achieving each step in theplanning, implementing, monitoring and review-ing cycle need to be in place for adaptivemanagement to progress from an idealisticframework to practical application. However,to determine these is often a difficult task for anindividual or an organization, due to the

multitude of interdependent factors that combineto determine the success of each step. Individualand organizational idiosyncrasies also mean thatthese success factors and their relationships willdepend on the unique operating environment inwhich adaptive management must take place.

This paper demonstrates the potential forusing participatory systems analysis to assistorganizations with implementing adaptive man-agement. We illustrate how systems analysisinvolving stakeholders can be applied to identi-fying adaptive management stumbling blocksand target management interventions. A casestudy conducted with the Queensland Parks andWildlife Service (QPWS), Australia, is used as anexample.

PARTICIPATORY SYSTEMS ANALYSIS

Systems designed by human beings are ‘purpo-seful’ systems (Jackson, 2003). In other words,they are designed or managed for a purpose, toachieve particular outcomes. Organizations andother human-managed systems, such as con-servation reserves, are good examples of purpo-seful systems. However, in NRM, there are manyindividuals with an interest in such systems(stakeholders) and eachwill have amental modelof the system and its purpose depending on theirindividual understanding, experience, educationand values. This means that among stakeholdersthere can be a multitude of views about thepurpose of NRM systems and the factors thataffect these purposes.

In managing purposeful systems, it is import-ant to accommodate the different world views ofthe stakeholders involved so that any proposedmanagement interventions are informed by abreadth of available experience, and acceptableto those who will need to implement changes orlive with the consequences of their implementa-tion. By combining a broad range of tools andtechniques developed in the field of systemsthinking, with participatory methods thatinvolve stakeholders, participatory systemsanalysis aims to provide a way of analyzingmanagement problems within purposeful sys-tems.

Figure 1. Generalized adaptive management cycle (afterBosch et al., 2003)

Copyright � 2007 JohnWiley & Sons,Ltd. Syst. Res.24, 567^587 (2007)DOI:10.1002/sres

568 Carl Smith et al.

RESEARCHPAPER Syst. Res.

The term participatory refers to a bottom-upapproach in which stakeholders participate insolving their management problems instead ofbringing in outside experts to solve them. Systemsanalysis refers to the application of systemsthinking to identify the root causes of manage-ment problems, and the potential implications ofmanagement decisions, by identifying the indi-vidual factors that may affect an outcome and thecausal relationships between them. In participa-tory systems analysis, the involvement of stake-holders allows the multitude of factors that mayinfluence outcomes or objectives to be identified,whilst systems thinking provides a mechanismthrough which these stakeholders can interactand discuss their understanding of the manage-ment system and the dependency relationshipsbetween these factors.

The participatory systems analysis processinvolves setting management objectives, abstractmodelling to explore the effect of decisions orscenarios on management objectives, developingplans for implementing preferred decisions ormanagement interventions, and monitoring thesystem to track management successes and adaptmanagement interventions where necessary(Lynam, 2001). The focus of the paper is on theinitial steps of the participatory systems analysisprocess, that is, setting objectives and abstractmodelling. Examples are given of how modelscan be used to manage the operating environ-ment for the success of adaptive management.

APPLYING PARTICIPATORY SYSTEMSANALYSIS TO ADAPTIVE MANAGEMENT

The success of adaptivemanagement depends onindividuals and organizations being able todevelop and implement plans, monitor outcomesand then review success within purposefulsystems. For adaptive management to work,factors critical to the success of each step need tobe put into place, and the stumbling blocks toeach step need to be removed (Figure 2). Theidentification of these critical success factors(CSFs) and stumbling blocks is not straightfor-ward because they will often be unique fordifferent individuals and organizations. There-

fore, identifying them means unravelling theunique operating environment of an individualor organization, which is where the process ofparticipatory systems analysis can help.

Each step in the adaptive management cycle isa point at which participatory systems analysiscan be applied. Planning, implementation,monitoring and reviewing objectives can be setwith stakeholders, and abstract models builtaround these to describe success factors and theirrelationships (Figure 3). These conceptualmodels can then be used to assess systemperformance and identify where managementintervention could lead to improved perform-ance. The remainder of this paper uses the resultsof a case study to demonstrate how the initialsteps in participatory system analysis can beapplied to examining adaptive managementwithin an organization responsible for firemanagement.

A CASE STUDY USING FIREMANAGEMENT

In Australia, fire management on conservationreserves is a pertinent issue because the targeteduse of fire is recognized as essential for species

Figure 2. Adaptive management nested within an operat-ing environment


AdaptiveManagement 569

Syst. Res. RESEARCHPAPER

conservation (Gill et al., 1981, 1999; Bradstocket al., 2002). However, the complexity of ecologi-cal processes, the inherent variability within andbetween fires, and the numerous permutationsbetween fire, terrain, climate, weather and biotameans that is it very difficult for landmanagers tomake accurate predictions about ecologicalresponses to fire (Whelan et al., 2002). Thus‘learning by doing’ using adaptive managementis promoted as a rational approach for ecologicalfire management.

As a first step towards implementing adaptivemanagement, a comprehensive fire managementsystem was introduced into the QPWS, Australia(Melzer and Clarke, 2003). The system providedreserve managers with set procedures for firemanagement planning and reporting. It wasintroduced in combination with a customizedgeographic information system for the storage ofdata used in planning and monitoring. Veg-etation monitoring plots also existed in someareas, but plots were not comprehensivelydistributed throughout the reserve system.

Despite having the ingredients for an adaptivefire management system available, uptake byreserve managers was variable. Resources allo-cated to fire management were competingheavily with other management priorities (such

as facilities maintenance) and managers wereexperiencing difficulty adapting to the number ofnew systems that were being promoted. In aneffort to identify the CSFs and stumbling blocksto adaptive fire management, QPWS initiated acase study focussed on conservation reserves innorth Queensland.

The goal of the case study was to unravel theoperational issues across the agencies’ jurisdic-tions to determine whether there were commonelements that could be addressed, or if issueswere disparate and required individualsolutions. It was recognized that substantialgainsmight be achieved from investing in actionsapparently tangential to fire management (e.g. ITskills). However, such resource allocationoptions were difficult to evaluate. The organiz-ation also sought a method to compare perform-ance between work units and a system forongoing monitoring and charting of perform-ance. Links were subsequently made betweenresearchers and fire management practitioners,which enabled this project to proceed.

BUILDING ABSTRACT MODELS OFADAPTIVE MANAGEMENT

The purpose of abstract modelling was todescribe QPWS fire management operations inthe context of the adaptive management cycle.The model building process consisted of twomain steps: (a) building influence (cause andeffect) diagrams to identify the CSFs related toadaptive management objectives and (b) con-verting these influence diagrams into systemsimulation models for analyzing the adaptivemanagement operating environment (Figure 4).

Building Influence Diagrams

Influence diagrams were constructed using theadaptive management cycle as a template fororganizing objectives and CSFs. For QPWS firemanagers, three sources of prior informationallowed an initial set of objectives and CSFs to beidentified (Table 1). These were:

Figure 3. Participatory systems analysis applied to adap-tive management




1. Fire Management Performance Indicators pre-viously developed in consultation with parkrangers and field staff. They covered the gen-eral categories of Planning, Operations,Reporting, Technology, Information, Monitor-ing, Communication and Training.

2. Fire Management System Documentation outlin-ing QPWS planning and reporting proceduresfor fire management (Melzer and Clarke,2003).

3. Best Practice Fire Management Workshop out-comes, which were diagrams constructed bypark rangers and field staff in previous work-shops identifying the issues they felt werebarriers to best fire management practiceand the broad cause and effect relationshipsbetween them.

Based on the prior information, a draftinfluence diagram was constructed for eachadaptive management step. Two workshopswere then held with QPWS staff (both rangersand managers) to review the draft influence

diagrams. A review process similar to thatoutlined by Cain (2001) was used. This involvedstarting at the objective and systematicallystepping backwards through the influence dia-grams to list those factors influencing each factor(Figure 5). This step-wise process allowed work-shop participants to add, change or removefactors included in the diagrams or modifydependency relationships.

After the model review process, the influencediagrams for ‘monitoring’ and ‘reviewing’ weremerged into one because the workshop partici-pants found it difficult to separate them into twodistinct systems models. The result was threeinfluence diagrams—fire management planning,implementation and monitoring-reviewing.

Building System Models

To build system models, the revised influencediagrams were converted into Bayesian BeliefNetworks (BBN) (Jensen, 2001) using NeticaTM

software (Norsys Software Corporation, 1998).Figure 6 is an unpopulated section of the‘implementation’ model, illustrating the depen-dency of ‘Burn Condition Information Deficien-cies’ on the availability of ‘skills & equipment toassess fuel’ characteristics and ‘weather data toassess conditions’. The probabilities that ‘BurnCondition Information Deficiencies’ will be in thestateRarely, Sometimes or Frequently are stored in acondition probability table. The empty templatefor this table is given in Table 2. The rows inTable 2 are scenarios constructed from differentcombinations of the input variables. Hence, thefirst row stores the probabilities that ‘BurnCondition Information Deficiencies’ will be inthe state Rarely, Sometimes or Frequently given‘Have Skills & Equipment to Assess Fuel’ is in thestate Yes and ‘Obtain Weather Data to AssessConditions’ is in the state Yes.

To capture the data needed to populate theprobability tables for the fire managementplanning, implementation and monitorin-g-reviewing BBN, three questionnaires wereprepared (one for each BBN). The questionnairescontained a multiple-choice question for eachnode in the models. The choices available in each

Figure 4. Processes used to build system models of theadaptive management operating environment (AM, adap-tive management; CSFs, critical success factors; BBN,

Bayesian Belief Networks)




Table 1. Initial set of fire management objectives and critical success factors identified from prior information

Objectives for conservation reserves Critical success factors

Planning From fire management performance indicators(a) Fire strategies completed � Satellite imagery(b) Statements of fire management intent completed � IT skills(c) Planned burn programmes completed � GPS and computer mapping(d) Wildfire response procedures completed � Vegetation fire response information

� Fauna fire response information� Cultural resource information� Internal communication� Communication with local community(i.e. neighbours and traditional owners)

� Communication with general public� Communication with fire managementprofessionals

� Understanding of QPWS fire managementsystem and fire management policy

From best practice fire management workshops� Mapping availability� Time� Community consultation� Support from other agencies� Support from QPWS managers� Knowledge sharing� Basic data sets

Implementation Existing fire management performance indicators(a) Planned burn objectives achieved � Aerial incendiary program

� Burning operations equipment� Community fire equipment� Staff availability and experience

Best practice fire management workshops� Staff availability� Work hours� Staff sharing� Weather forecast training� Weather information� Fire training� Fire equipment availability� Equipment sharing� Time� Travel allowance� Overtime� Plan flexibility

Monitoring Existing fire management performance indicators(a) Vegetation fire management outcomes known � Hot spot imagery(b) Fauna fire management outcomes known � Information and data storage(c) Cultural fire management outcomes known � Vegetation monitoring sites

� Fauna monitoring sites

(Continues)




Table 1. (Continued)

Objectives for conservation reserves Critical success factors

� Cultural resource monitoring sitesBest practice fire management workshops� Time

Reviewing Existing fire management performance indicators(a) Fire reports submitted � GPS and computer mapping

Best practice fire management workshops� Time

Planned Burn Objs AchievedTime Allocated to Burn Programme Fire Bans Interrupt Burn Programme

Personnel Unavailability Burn Condition Info Deficiencies

Able to Reschedule Missed Burns

Other Personnel Available

QPWS Staff Avail Cause Problems Have Skills & Equip to Assess Fuel

Obtain Weather Data to Assess Conditions

Equip & Service Deficiencies

Aerial Incendiary AdequateLocation & Comm Equip Adequate Fire Line Equip Deficiencies

a)

Planned Burn Objs AchievedTime Allocated to Burn Programme Fire Bans Interrupt Burn Programme

Personnel Unavailability Burn Condition Info Deficiencies

Able to Reschedule Missed Burns

Other Personnel Available

QPWS Staff Avail Cause Problems Have Skills & Equip to Assess Fuel

Obtain Weather Data to Assess Conditions

Equip & Service Deficiencies

Aerial Incendiary AdequateLocation & Comm Equip Adequate Fire Line Equip Deficiencies

b)

Figure 5. Processes used to review draft influence diagrams, (a) starting at the objective and (b) stepping backwards througheach factor




question matched the states given to the nodes.For example, Table 3 is a section of thequestionnaire developed for the implementationmodel. The questions relate to BBN nodes inFigure 6. For some questions, options wereprovided to capture the reasons for negativeresponses. For instance, the Fuel Characteristicsquestion in Table 3 lists a series of reasons for aNot Really response.

The questionnaires were pilot tested andreviewed by a group of QPWS staff before beinggiven to 28 park rangers responsible for mana-ging conservation reserves in north Queensland.Table 4 is a sample of the data collected for thequestions listed in Table 3. Table 5 illustrates howthese data were used to populate the conditionalprobability table for ‘Burn Condition InformationDeficiencies’ (Table 2). Figure 7 shows thepopulated BBN where Yes has been selectedfor both skills and equipment, and weather data.

Appendices 1–3 show the completed planning,implementation and monitoring-reviewing sys-temsmodels for QPWS firemanagement. In thesethree models, the states of each node have beenordered from best (top) to worst (bottom). Theprobability distributions in the nodes show thepercentage distribution of responses provided bythe park rangers surveyed.

USING ABSTRACT MODELS TO ANALYZETHE ADAPTIVE MANAGEMENTOPERATING ENVIRONMENT

Once constructed, system models can be used inseveral ways to analyze the adaptive manage-ment operating environment. First, they can beused to assess performance with respect to CSFsand objectives, and to identify commondeficiencies that may be stumbling blocks toadaptive management. Second, they can be usedto assess the potential improvements in objectiveachievement that might be expected if stumblingblocks were reduced or removed throughmanagement intervention. Third, they can beused to adaptively manage the operatingenvironment through the on-going assessmentof the success of management interventions.Examples of these three system model functionsare given in the following sections.

Assessing Performance

The probability distributions for nodes in thesystem models show the percentage of obser-vations (or cases) falling into each state. If thenode states are ordered from best (most positive)

Burn Condition Info DeficienciesRarelySometimesFrequently

33.333.333.3

Have Skills & Equip to Assess FuelYesNot Really

50.050.0

Obtain Weather Data to Assess ConditionsYesNot Really

50.050.0

Figure 6. Unpopulated Bayesian Belief Network with three nodes

Table 2. Template of the conditional probability table for ‘Burn Condition Information Deficiencies’ in Figure 6

Have Skills &Equip toAssess Fuel?

Obtain WeatherData to AssessConditions?

Burn Condition Info Deficiencies?

Rarely Sometimes Frequently

Yes YesYes Not ReallyNot Really YesNot Really Not Really




to worst (most negative), then their probabilitydistributions can serve as performanceindicators. Good performance is indicted whenthe probability distribution of a node is skewedtowards its best state. Poor performance, or acommon deficiency, is indicated when theprobability distribution of a node is skewedtowards its worst state. Figure 8 is an examplefrom the QPWS firemonitoring-reviewing model(Appendix 3). It shows good performance withrespect to all CSFs related to monitoringvegetation on sites accept for time (see ‘TimeAllocated to Veg Monitoring’), where most (57%)of park rangers reported ‘Not Really’ allocatingtime to vegetation monitoring.

Overall, the fire monitoring-reviewing model(Appendix 3) shows better performance in thefactors related to the vegetation monitoringobjective (see ‘Know Veg Fire Outcomes’) thanthose related to the fauna and cultural monitor-

ing objectives (see ‘Know Fauna Fire Outcomes’and ‘Know Fire Cultural Outcomes’). Thispattern is similar in the fire planning model(Appendix 1), which shows that the factorsrelated to the setting of vegetation fire manage-ment objectives (see ‘Able to Set Veg Fire MgmtObjs’) performed better than those related to thesetting of fauna and cultural fire managementobjectives (see ‘Able to Set Fauna FireMgmtObjs’and ‘Able to Set Cultural Fire Mgmt Objs’). In thefire implementation model (Appendix 2) theability to reschedule missed burns (see ‘Able toReschedule Missed Burns’), aerial incendiaryservices (see ‘Aerial Incendiary Adequate’),community fire line equipment (see ‘CommunityFire Line Equip Adequate’) and fire line equip-ment available for hire (see ‘Hire Fire Line EquipAdequate’) all had low performance.

Table 6 below summarizes the probabilitydistributions of the objective nodes for all QPWS

Table 3. Section of the questionnaire for implementation BBN

1.1. Burn condition informationFuel characteristicsDo you have adequate skills and equipment to assess fuel characteristics on your park(such as fuel curing % and fuel load)?1. Yes2. Not Really: Please circle reasons why(a) personnel with skills needed to assess fuel characteristics are lacking or difficult to access(b) equipment needed to assess fuel characteristics is lacking or difficult to access(c) information on assessing fuel characteristics is difficult to find or access(d) available information on assessing fuel characteristics is difficult to use or interpret(e) other reason(s) (if other please specify on the line below)

Weather dataCan you obtain the weather data needed to adequately assess burning conditions on yourpark (such as windspeed, wind direction, temperature and relative humidity)?1. Yes2. Not Really: Please circle reasons why(a) weather data for my area is difficult to find or access(b) weather data for may area lacks the required information(c) weather data for my area is difficult to use or interpret(d) other reason(s) (if other please specify on the line below)

1.2. OverallHow often does a lack of burn condition information interrupt planned burning operationson your park?1. Frequently—more than half the time2. Sometimes—less than half the time but occasionally3. Rarely—hardly ever




fire management models. The distributionsindicate poor performance in the areas of firestrategy documentation, and knowledge of faunaand cultural fire management outcomes.

Assessing Interventions

Areas of poor performance highlighted in thesystem models represent areas where interven-

Table 4. Data collected from respondents to questions in Table 3 (�, missing data)

Conservationreserve

Have Skills &Equip to Assess Fuel?

Obtain WeatherData to Assess Conditions?

Burn ConditionInfo Deficiencies?

1 Not Really Not Really Sometimes2 Yes Yes Frequently3 Yes Not Really Sometimes4 Yes Yes �

5 Yes Yes Sometimes6 Yes Not Really Frequently7 Yes Yes Rarely8 Yes Yes Rarely9 Yes Yes Frequently10 Yes Yes Sometimes11 Yes Yes Rarely12 Yes Yes Rarely13 Yes Yes Sometimes14 Yes Not Really Rarely15 Not Really Yes Rarely16 Yes Yes Sometimes17 Not Really Yes Rarely18 Yes Yes Rarely19 Yes Yes Rarely20 Not Really Yes Rarely21 Not Really Yes Rarely22 Yes Yes Rarely23 Yes Yes Frequently24 Not Really Not Really Frequently25 Yes Yes Rarely26 Yes Yes Sometimes27 Yes Yes Rarely28 Yes Yes Rarely

Table 5. Populated conditional probability table for ‘Burn Condition Information Deficiencies’ derived from data in Table 4

Have Skills &Equip toAssess Fuel?

Obtain WeatherData to AssessConditions?

Burn Condition Info Deficiencies?

Rarely Sometimes Frequently

Yes Yes 55.5% (10 in 18 cases) 27.7% (5 in 18 cases) 16.6% (3 in 18 cases)Yes Not Really 33.3% (1 in 3 cases) 33.3% (1 in 3 cases) 33.3% (1 in 3 cases)Not Really Yes 100% (4 in 4 cases) 0% (0 in 4 cases) 0% (0 in 4 cases)Not Really Not Really 0% (0 in 2 cases) 50% (1 in 2 cases) 50% (1 in 2 cases)

Note: Although there are 19 cases in Table 4 where ‘Have Skills & Equip to Assess Fuel’¼Yes and ‘ObtainWeather Data to AssessConditions’¼Yes, the data for ‘Burn Condition Info Deficiencies’ is missing for one of these (Case 4), thus only 18 cases are used.





55.627.816.7


100 0


100 0

Figure 7. Populated Bayesian Belief Network with Yes selected for both skills and equipment, and weather data

Veg Indicator Species KnownYesNot Really

82.117.9

Time Allocated to Veg MonitoringYesNot Really

42.957.1

Veg Sites MonitoredRegularlyAd hocNot Really

39.240.320.5

Veg Reference Sites EstablishedYesNot Really

71.428.6

Veg Fire Mgmt Objectives SetYesNot Really

82.117.9

Have Veg Monitoring Skills & EquipYesNot Really

77.822.2

Figure 8. Section of QPWS fire monitoring-reviewing model

Table 6. Probability distribution for fire management objectives

Model Objective Probability distribution

Planning (a) Documented fire strategy 19% Yes, 29% Draft, 52% No(b) Statement of fire management intent 79% Yes, 7% Draft, 14% No(c) Approved planned burn program 68% Yes, 32% No(d) Wildfire response procedure 43% Yes, 26% Draft, 31% No

Implementation (a) Planned burn objectives achieved 42% Mostly, 29% Some,30% Not Really

Monitoring-reviewing (a) Vegetation fire management outcomes known 40% Mostly, 32% Sometimes,28% Not Really

(b) Fauna fire management outcomes known 24% Mostly, 40% Sometimes,36% Not Really

(c) Cultural fire management outcomes known 23% Mostly, 17% Sometimes,33% Not Really, 27% NotApplicable (NA)

(d) Fire reports submitted 44% Mostly, 28% Sometimes,27% Rarely




tion or investment may be needed to lift thechance of adaptive management success. How-ever, managers also want to know whatimprovements might be expected if these stum-bling blocks are reduced or removed. Animportant feature of system models (and BBN,in particular) is that they can be used to conductsensitivity and scenario analysis. These analysescan be used to assess both the relative importanceof stumbling blocks (through sensitivity analysis)and where removing stumbling blocks mightlead to the greatest improvements (throughscenario analysis).

Table 7 is an example of a sensitivity analysisconducted on the same section of the QPWS firemonitoring-reviewing model shown in Figure 8.The table was constructed by recording thedifferences made to state probabilities of ‘VegSites Monitored’ when parent nodes werechanged from their best to worst state. Thismethod of sensitivity analysis is described byCain (2001). The parent nodes in Table 7 areranked in order from most influential (one) toleast influential (five). Note, ‘Veg Sites Mon-itored’ is most sensitive to a state change in ‘TimeAllocated to VegMonitoring’ and ‘Veg ReferenceSites Established’, which indicates that these

factors are of high relative importance to theoccurrence of vegetation site monitoring. Theresult produced by a state change in vegetationmonitoring skills and equipment (see ‘Have VegMonitoring Skills & Equip’) is one inconsistencyin Table 7, indicating an increased probability inregular vegetation site monitoring when in theworst state. This may have been a result ofconservation reserve managers involved inregular site monitoring feeling as though theirskills and equipment for the task wereinadequate.

The outcomes of performance assessment(Figure 8) and sensitivity analysis (Table 7) showtime allocation to be an area of both poorperformance and relatively high influence onvegetation site monitoring. Therefore, timeallocation (or lack of) represents a significantstumbling block to vegetation site monitoring.Figure 9 is the scenario where conservationreserve managers have allocated time to veg-etationmonitoring. It shows a 21% increase in thepercentage of conservation reserves reportingregular vegetation site monitoring (increase from39% in Figure 8 to 61% in Figure 9). Thisdemonstrates that if performance in time allo-cation could be improved through intervention, a

Table 7. Sensitivity analysis for ‘Veg Sites Monitored’ in the QPWS fire monitoring-reviewing model

Rank Parent node Veg Sites Monitored

Regularly Ad hocþNot Really

1 Time Allocated to Veg Monitoring¼Yes 61 39Time Allocated to Veg Monitoring¼Not Really 23 77Difference �38 þ38

2 Veg Reference Sites Established¼Yes 50 50Veg Reference Sites Established¼Not Really 12 88Difference �38 þ38

3 Have Veg Monitoring Skills & Equip¼Yes 33 67Have Veg Monitoring Skills & Equip¼Not Really 60 40Difference þ27 �27

4 Veg Indicator Species Known¼Yes 43 57Veg Indicator Species Known¼Not Really 21 79Difference �22 þ22

5 Veg Fire Mgmt Objectives Set¼Yes 40 60Veg Fire Mgmt Objectives Set¼Not Really 37 63Difference �3 þ3




relatively large improvement in vegetationmonitoring could be expected.

Scenario analysis can also be used to assess theimpact of multiple interventions and to examineboth the expected local and flow-on impact ofthese interventions. Comparing Figure 10a to bindicates the expected flow-on impact of inter-vention in both time allocation to vegetationmonitoring and to establishing vegetation refer-ence sites. Note the improvement in vegetationsite monitoring (see ‘Veg Sites Monitored’),general vegetation assessment (see ‘GeneralVeg Assessment Undertaken’) and the flow-onimprovement in knowledge of vegetation fireoutcomes (see ‘Know Veg Fire Outcomes’).

Adaptively Managing the OperatingEnvironment

The use of system models has the potential toguide decisions on interventions to improveoperations and outcomes. The decision makingprocess is not a ‘once-off’ activity, however, assystemmodels only provide an assessment of thesystem at the time of their construction. Oneadvantage of using BBN in system modelling isthat their underlying probabilities can beupdated over time using subsequent surveyresults. This allows for periodic assessment of

the operating environment and also for evalu-ation of the success of any implemented inter-ventions. Figure 11 is a hypothetical case wheresubsequent survey results illustrate the impact ofinterventions designed to improve time allo-cation to vegetation monitoring. In this case, thepercentage of conservation reserves allocatingtime to vegetation monitoring increased to 90%with a subsequent improvement in vegetationsite monitoring, general vegetation assessmentand knowledge of vegetation fire outcomes.

The process of using system models for theongoing assessment and management of theoperating environment provides a mechanismfor managers to evaluate the impact of theirinterventions (Figure 12). Regularly updatingsystem models also provides an opportunity fornew knowledge to be incorporated into them. Ifnew CSFs become apparent, new nodes can beadded to the models, or nodes can be removed asold stumbling blocks are overcome.

CREATING A CO-LEARNINGENVIRONMENT WITH PARTICIPATORYSYSTEMS ANALYSIS

The combined understanding and experience ofseveral conservation reserve managers, plus


82.117.9


100 0


61.214.224.5


71.428.6


82.117.9

Have Veg Monitoring Skills & EquipYesNot Really

77.822.2

Figure 9. Scenario where time has been allocated to vegetation monitoring




information obtained from existing documents,were used to develop the QPWS fire manage-ment system models. No one person or docu-ment explained the range of CSFs believed toinfluence planning, implementation, monitoringand reviewing objectives, nor the dependency

relationships between them. This was obtainedthrough the gradual development of the systemmodels, adding and combining the knowledgeand experience of the different stakeholdersparticipating. The system models providedscaffolding for integrating an organization’s

Know Veg Fire OutcomesMostlySometimesNot Really

40.031.728.4

Veg Interp Skills/Tools AvailableYesNot Really

60.739.3

Veg DMS AvailableYesNot Really

53.646.4


82.117.9

Veg Monitoring Skills & EquipYesNot Really

77.822.2

Veg Sites Monitored

RegularlyAd hocNot Really

39.240.320.5


71.428.6


42.957.1

General Veg Assessment Undertaken


16.349.134.6


82.117.9

a)Know Veg Fire Outcomes

MostlySometimesNot Really

52.228.219.6


60.739.3


53.646.4


82.117.9


77.822.2

Veg Sites Monitored


79.413.67.04


100 0


100 0

General Veg Assessment Undertaken


24.562.612.9


82.117.9

b)

Figure 10. Section of QPWS fire monitoring-reviewing model showing (a) the current situations and (b) the expected impactof intervention in time allocation and vegetation reference site establishment

Know Veg Fire OutcomesMostlySometimesNot Really

45.830.124.1


60.739.3


53.646.4


82.117.9


77.822.2


57.418.823.8


71.428.6


90.010.0

General Veg Assessment UndertakenRegularlyAd hocNot Really

23.160.216.7


82.117.9

Impact Impact

Figure 11. Hypothetical example of an updated model showing the impact of intervention targeted at improving timeallocated to vegetation monitoring




knowledge and then combining this with dataobtained from survey. Hence, the process ofmodel development provided a conduit forco-learning, knowledge sharing and developinga common understanding of the factors drivingthe success of adaptive management.

Participation was, therefore, an importantaspect of the model development process.Besides creating a co-learning environment, theparticipatory nature of the process helped todevelop a sense of ownership of the models byQPWS personnel. This was reflected in thecommitment of these personnel to developcapacity within their organization to use themodels into the future. Thus, the system modeldevelopment process helped to institutionalizeadaptive management into the organization.

LIMITATIONS OF THE SYSTEMMODELLING APPROACH

The main limitations to the system modellingapproach described in this paper relate to the useof BBN. Nodes in a BBN with several parentshave large conditional probability tables contain-ing many scenarios. The number of scenarios in anode’s probability table is equal to the number ofstates of each of its parent nodes multipliedtogether. Hence, a node with three parents, eachhaving three states, will have a conditionalprobability table containing 27 scenarios

(3� 3� 3). Adding just one more state to oneparent node will produce 36 scenarios (3� 3� 4)or linking just one more parent node with threestates will produce 81 scenarios (3� 3� 3� 3).Often there is insufficient data available topopulate such large probability tables. Theconditional probability table for ‘Planned BurnObjs Achieved’ in the QPWS fire implementationmodel (Appendix 2) is an example, whichcontains 216 scenarios. Of these only 15 couldbe populated with probabilities obtained fromthe survey of park rangers. For the remaining 201scenarios the probabilities are unknown, and anequal probability (33.3) was assigned to each ofthe three states of ‘Planned Burn Objs Achieved’.

The number of nodes contained in a BBN posesanother difficulty. With many nodes, the influ-ence that any one node may have on an outcomeis diluted, making it difficult to identify sensi-tivities in the model. Generally, nodes that arecloser to an outcome or objective node (i.e. havefewer nodes between them and the outcomenode) have greater influence over the outcome.Because of this affect, overly complex pathwaysin BBN dilute the influence of outer nodes.

The difficulties in populating large probabilitytables and the dilution effects in complexnetworks suggest that when using BBN to modelsystems, it is necessary to summarize the manyfactors that may influence outcomes into as fewnodes and states as possible. This means thatBBN cannot be used to integrate all possiblefactors that may contribute to the success ofadaptive management, only a few CSFs and keydependency relationships between them. This isa particular limitation of using quantitative (orhard) system models to analyze purposefulsystems.

DIFFICULTIES EXPERIENCED INDEVELOPING THE QPWS FIREMANAGEMENT MODELS

At the time the QPWS fire management modelswere built, quantitative benchmarks for firemanagement performance had not been set bythe organization. Qualitative performance indic-tors meant that it was difficult to define many of

Figure 12. Use of the system models for on-going manage-ment of the operating environment




the states in the system models. Due to thequalitative states applied to many nodes,answers to some of the associated surveyquestions were inconsistent. This inconsistencywas reflected in the behaviour of some sections ofthe models. For example, Figure 13 shows asection of the QPWS fire implementation modelwith (a) no scenario inserted and (b) ‘Have Skills& Equip to Assess Fuel’ set to Not Really. Onewould expect the probability for the Rarely stateof ‘Burn Condition Info Deficiencies’ to be less inFigure 13b; however, it has increased. Thisindicates that states, and their definitions, areone aspect of the QPWS models that need futureconsideration. Better definition of states, or usingquantitative benchmarks for states, wouldimprove consistency in the answering of surveyquestions and would make it easier to interpretmodel results.

CONCLUDING REMARKS

The case study demonstrated the use of the initialsteps of participatory systems analysis to identifyoperational stumbling blocks to adaptive man-

agement. We contend that such analysis, even ifonly applied conceptually, is key to facilitatingthe practical application of adaptive manage-ment. Together with stakeholder involvement, asystems approach to analyzing the adaptivemanagement operating environment has theparticular advantage of assisting the integrationof corporate knowledge. The process of systemmapping provides a framework in which partici-pants can share their understanding of CSFs andtheir dependency relationships. This creates aco-learning environment that facilitates com-munication among managers and identifies adiverse range of CSFs that affect planning,implementing, monitoring and reviewing keyNRM objectives.

The abstract models developed within theparticipatory systems analysis process can beused to assess current operational performancewith respect to planning, implementing,monitoring and reviewing objectives and theirCSFs. They also provide a tool for conductingsensitivity and scenario analyses that allowsignificant stumbling blocks in the operatingenvironment to be highlighted, and the expectedlocal and flow-on impacts of management


58.124.517.3


78.621.4


82.117.9

a)


82.18.938.93


0 100


82.117.9

b)

Figure 13. Section of the QPWS fire implementation model with (a) no scenario inserted and (b) Not Really selected for‘Have Skills & Equip to Assess Fuel’




interventions to be tested. This is a significantadvantage over stand-alone performanceindicators that do not allow for dynamic scenarioanalysis.

We used BBN to build abstract models. Thecase study highlights a number of benefits ofusing BBN in system modelling. First, theyprovide a way to diagrammatically capture theCSFs influencing objectives, and their depen-dency relationships, described by managers.Second, they provide a way to integrate a diverserange of CSFs (such as equipment, skills, data,information, human resources, time, policies,etc.) and quantifying their relationships. Third,they allow variability and uncertainty in theserelationships to be accommodated using con-ditional probabilities. Fourth, they can be used asa tool to identify poor performance areas, assessthe relative influence of poor performance areason objectives through sensitivity analysis, andtest the expected local and flow-on impacts ofimproving performance in particular areas.Another advantage of BBN modelling is thaton-going monitoring and survey results can beused to update models over time. This allows fora periodic assessment of performance and anevaluation of the efficacy of interventions tar-geted at removing operational stumbling blocksto adaptive management.

The problems associated with complex BBN(large probability tables and dilution of theinfluence) meant that the range of CSFs believedto influence objectives had to be summarized intoas few nodes, with as few states, as possible. Thismeant that all factors mentioned by stakeholderscould not be captured in the models.

A final conclusion that can be drawn from thispaper is that the process used to build models ofthe adaptive management operating environ-ment is just as important as the modelsthemselves. If the process used is inclusive ofmanagers and their knowledge, and the tools arecomprehensible, then the chance that the out-comes of the systems analysis will be adopted isimproved. This was the experience in the QPWScase study where staff participating in thesystems analysis have begun to develop theirown capacity to use and update the systemmodels on an on-going basis. In the long term this

capacity will significantly help the organizationto operationalize adaptive management.

ACKNOWLEDGEMENTS

The authors would like to gratefully acknowl-edge the co-operation of QPWS staff and con-servation reserve managers for their significantinput into the case study. The authors would alsolike to thank Dr John Ludwig (CSIRO, Sustain-able Ecosystems) for his comments on the manu-script and the CRC for Tropical SavannasManagement for their financial support.

REFERENCES

Bosch OJH, Ross AH, Beeton RJS. 2003. Integratingscience and management through collaborativelearning and better information management. Sys-tems Research and Behavioural Science 20: 107–118.

Bradstock RA, Williams JE, Gill AM. 2002. FlammableAustralia: The Fire Regimes and Biodiverstiy of a Con-tinent. Cambridge University Press: Cambridge, UK.

British Columbia Forest Service. 2000. Definitions ofAdaptive Management. http://www.for.gov.bc.ca/hfp/amhome/Amdefs.htm [25 October 2005].

Cain J. 2001. Planning improvements in natural resourcesmanagement: guidelines for using Bayesian networks tosupport the planning and management of developmentprograms in the water sector and beyond. Centre forEcology and Hydrology: Wallingford, UK.

Gill AM, Groves RH, Noble IR. 1981. Fire and theAustralian Biota. Australian Academy of Science:Canberra, Australia.

Gill AM, Woinarski JC, York J. 1999. Australia’s Biodi-versity: Response to Fire. Biodiversity Technical PaperNo. 1. Environment Australia: Canberra, Australia.

Holling CS. 1978. Adaptive Environmental Assessmentand Management. Wiley: New York.

Jackson MC. 2003. Systems Thinking: Creative Holism forManagers. John Wiley and Sons: UK.

Jensen FV. 2001. Bayesian Networks and Decision Graphs:Statistics for Engineering and Information Science.Springer-Verlag: New York.

Lynam T. 2001. Participatory Systems Analysis: An Intro-ductory Guide. Institute of Environmental StudiesSpecial Report No. 22. University of Zimbabwe:Harare, Zimbabwe.

Melzer R, Clarke J. 2003. Fire Management SystemVolume 1: Planning and Reporting. Queensland Parksand Wildlife Service, Environmental ProtectionAgency: Brisbane, Australia.




Norsys Software Corporation. 1998. Netica ApplicationUser’s Guide. Norsys Software Corp.: Vancouver, BC,Canada. Available from http://www.norsys.com

Whelan RJ, Rodgerson L, Dickman CR, Sutherland EF.2002. Critical life cycles of plants and animals: devel-

oping a process-based understanding of populationchanges in fire-prone landscapes. In Flammable Aus-tralia: The Fire Regimes and Biodiverstiy of a Continent,Bradstock RA, Williams JE, Gill AM (eds). Cam-bridge University Press: Cambridge, UK.




APPENDIX

1:QPWSFIREPLANNIN

GMODEL(SHADED

NODESAREOBJECTIV

ES)




APPENDIX

2:QPWSFIREIM

PLEMENTATIO

NMODEL(SHADED

NODESAREOBJECTIV

ES)




APPENDIX

3:QPWSFIREMONITORIN

G-REVIEWIN

GMODEL(SHADED

NODESAREOBJECTIV

ES)




adaptive management: making it happen through participatory systems analysis

Documents