data analysis or simulation model: a critical evaluation of some methods

ELSEVIER Ecological Modelling 78 (1995) 51-60

E(OLOOI(IIL mODELLIn6

Data analysis or simulation model: a critical evaluation of some methods

Onno F.R. van Tongeren Netherlands Institute of Ecology, Centre for Limnology Rijksstraatweg 6, 3631 AC Nieuwersluis, Netherlands

Received 10 February 1993; accepted 15 June 1993

Abstract

The use of regression, ordination and dynamic ecosystem modelling in limnology is discussed by evaluating some of the vices and virtues of these techniques. Both general characteristics of the approaches and a few examples are used to stress the importance of analysis of the residuals for evaluation of the models. Not completely unexpected, a simple ordination model did not perform worse than a complicated ecosystem model, both applied to the same lake system. Both models are shown to fail for prediction purposes, mainly because the error variance in the data used for parameter estimation is large compared to the variance that can be explained.

Integration of regression, multivariate analysis and dynamic ecosystem modelling, all followed by analysis of the residuals, is advised. Nested models are proposed as a solution for the problem of changing parameters and for the problem of parameters being different among lakes.

Keywords: Freshwater ecosystems; Limnology; Model evaluation

1. Introduction

Traditional limnology studies physical, chemical and biotic parameters and processes of more or less closed freshwater systems: "Limnology is the study of the functional relationships and pro- ductivity of freshwater communities as they are affected by their physical, chemical and biotic environment" (Wetzel, 1983). Although this defi- nition includes river and brook ecosystems, most of the work of limnologists has been devoted

Present and correspondence address: Data Analyse Ecolo- gie, Waemelslant 27, 6931 HS Westervoort, Netherlands.

towards lakes. In these lakes, ecosystem parameters are measured and process rates estimated for functional groups of organisms. Descriptive limnology relates the systems state variables at a high integration level to each other and to variables like nutrient loading or total nutrient con- tent (e.g. Vollenweider, 1976, 1987; Peters, 1986). Until recently modellers in limnology followed the tradition of lumping species into functional groups in order to build dynamic models that were used to predict the changes in systems fol- lowing a management measure, e.g. reduction of phosphorus loading.

This is in contrast to vegetation ecology, which has a long tradition of describing plant communi-

0304-3800/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSDI 0304-3800(94)00117-Z

52 O.F,R. ~an Tongeren /Ecological Modelling 78 (1995) 51-60

ties using classification and ordination techniques (e.g. Westhoff and Van der Maarel, 1980; Whit- taker, 1980a,b) and terrestrial animal ecology, which is traditionally oriented more towards population dynamics and predator-prey relationships (e.g. Anderson et al., 1979). Nowadays more limnologists are modelling processes at the population or individual level (e.g. DeAngelis et al., 1984; DeAngelis, 1988). Most of the dynamic models have been severely criticized in the recent past, as well as limnology (Peters, 1990, 1991) and ecology as a whole (Simberloff, 1983; Peters, 1991). In this paper I will describe some of the vices and virtues of several approaches, with em- phasis on the many pitfalls that exist. I will use a few examples from the Water Quality Loosdrecht (WQL) project, a multidisciplinary project, coor- dinated at the Limnological Institute (Nether- lands Institute of Ecology, Centre for Limnology, Nieuwersluis), which are illustrative for many other studies. My conclusion is that everyone should try to learn from the approaches used by others and that a combination of several approaches most probably will lead to both a better understanding of the ecosystems and to more accurate predictions.

2. Simple regression models

Simple regression models have proven to be useful tools for description of relationships and for prediction. The major reason for preferring regression models over dynamic models is that they include an estimate of the confidence limits for prediction. Moreover, there are simple techniques for the analysis of the residuals (observed values - expected values), one of the main as- sumptions in (multiple) linear regression being that the residuals have zero mean and a homoge- neous variance and that they are uncorrelated (e.g. Snedecor and Cochran, 1980; Ter Braak, 1987a; Krambeck, 1995). Using these techniques, problems existing in the traditional regression analysis with spatial (e.g. Burrough, 1987) or tem- poral autocorrelation (e.g. Box and Jenkins, 1976) are readily solved. Major criticism on the use of regression models is that they do not provide

insight in the underlying mechanisms or functional relationships (e.g. Peters, 1991) because they are static. This main criticism is valid only when regression models are used to describe or to predict state (in most cases biotic) variables from a set of independent (abiotic) variables. However, especially in experimental research, also when dynamic relationships (e.g. growth rates, feeding rates, etc.) are studied, regression analysis and Analysis of Variance (ANOVA) have proven to be the most applied and most successful methods. The major problems with regression analysis, when applied in observational studies on ecosystems, are related to causality and multicollinearity, many of the predictor variables being correlated. However, additional knowledge of the functional relationships may lead to a proper choice of regression model and predictor variables. Nowadays, generalized linear models (Mc- Cullagh and Nelder, 1983), allowing other distri- butions for the error than the normal distribution, are replacing variance-stabilizing transformations more and more. Widespread in ecology are misapplications of regression techniques, because results are selected from a series of regressions using different transformations on the basis of the percentage of the variance explained by the model without checking the distribution of the data and the residuals. Krambeck (1995) gives some very nice examples.

3. Multivariate statistics

Although the use of multivariate techniques in limnology is not widespread, especially ordination techniques deserve more attention in my opinion. In general, these techniques are applied to sam- ples by species abundance matrices, but it is also feasible to use them for the analysis of process rates instead of species abundances. The major reason for my appreciation of these techniques is that the unconstrained analyses, also referred to as indirect gradient analysis, like Principal Com- ponent Analysis (PCA, e.g. Gauch, 1982) and Correspondence Analysis (CA, Ter Braak, 1987b,c), a synonym for Reciprocal Averaging (RA, Hill, 1973), are powerful tools to detect

O.ER. van Tongeren / Ecological Modelling 78 (1995) 51-60 53

patterns in and to reduce the dimensionality of complex multivariate data. The canonical variants of these techniques, also referred to as direct gradient analysis, Redundancy Analysis (RDA, Rao, 1964, 1973; Ter Braak, 1987b) or Reduced Rank Regression and Canonical Correspondence Analysis (CCA, Ter Braak, 198719, 1989), perform even better when studying explicitly the relationship between the environment of the system and the systems components.

A major drawback of these techniques is the assumption that all variables have the same type of relationship with the latent variables or the independent variables. In case of biotic variables this relationship is the model of their response to the environmental variables. Linear relationships, or monotonous relationships that may he trans- formed into (almost) linear relationships by a properly chosen transformation of the variables, are analysed using PCA, respectively RDA or

0 . 6 . . . . . . . . . . . . a '?

0 . 4

( ,9

"~ 0.2

0 o 0

03 - 0 . 2 (1)

r r

-0.4 I

I -0.6 i

v

v

83 85

v

"2

r"

89

V

91

0.8

b

0.6 C,J

0 0 .2 ' ~: (.3 (,'3

7 q

"~ -0.2 Or

-0.4

-0,6 ~ 83

?

T -: 7 ~, g v .3c v

,S v ~! 7

85 87 89 91

Fig. 1. Changes in the residuals of the RDA scores of the phytoplankton analysis (Van Tongeren et al., 1992) with time. (a) Axis 1. Average residuals for different seasons are indicated by lines: thick line represents spring, dashed thick line autumn. Average residuals of winter, and early and late summer are indicated by thin lines. (b) Axis 2. Average residuals for different seasons are indicated by lines: thick line represents spring, dashed thick line early summer.

54 O.F.R. van Tongeren /Ecological Modelling 78 (1995) 51-60

related techniques. Unimodal response curves and data representing proportions are analysed using CA or Gaussian ordination, respectively CCA or related techniques. However, more complicated response curves like bimodal curves occur in na- ture as well as mixtures of different response models. Nevertheless, in general the ordination techniques describe the relationships fairly well.

As follows from the preceding, the major drawback of these ordination techniques is that all species (or functional groups) are assumed to follow the same type of response model. Al- though this may not be true in general, this largely facilitates the interpretation of the results. However, after completion of the analysis the results should be checked against the observations. As with regression, a more formal analysis of the residuals may be of great help in doing this, although this implies an enormous amount of work if done for all variables separately.

Redundancy Analysis was applied to analyse the zooplankton and phytoplankton counts of the WQL project (Van Tongeren et al., 1992). The most important conclusions of this analysis have been published and still are correct, but only little effort was made to check whether the chosen linear model was the best model. The phytoplankton analysis resulted in a linear model of phytoplankton scores on temperature and time. An informal analysis of the residuals of the scores on the first axis of the phytoplankton analysis (Fig. 1) reveals a more complex pattern than the simple linear relationship with temperature and time that underlies the model. The residuals of the first axis (negatively correlated with temperature, Fig. la) reveal the expected delay in the response to temperature, spring residuals being positive and autumn residuals being negative. Re- placing temperature in the model by a lagged temperature or by an integrated temperature would have increased the explained variance to a large extent. The residuals of the second axis (negatively correlated with time, Fig. lb) show a more complex pattern, being positive in 1984, 1985 and from summer 1989 onwards and negative in the years 1986 through 1988. This indicates a non-linearity in the change of the system with time, which is most obvious in spring and

early summer. However, there seems to be a relationship between the residuals and the estimated internal (Keizer and Sinke, 1992) and external (Engelen et al., 1992) phosphorus loads. Care should be taken in drawing such conclusions, because they are based on correlations between a few points only. A major reduction in the external load from 1984 onwards may be related to the decrease of the summer and autumn residuals in the period 1984-1986, whereas the lower internal load from 1987 onwards may be related to the sudden change in the trend of the residuals of the spring observations.

Some artefacts of the techniques should be mentioned, because they may hamper a correct interpretation. In PCA and related techniques the so-called horse-shoe effect often indicates that unimodal relationships with the underlying gradient are present. In that case CA or related techniques may better demonstrate the relationships with the environment. However, when only one strong gradient is present in the data, CA shows the arch-effect, an artefact that is largely removed by detrending (Hill and Gauch, 1980), an informal procedure that nevertheless leads to a more straightforward interpretation in many cases. The analysis of the residuals gives no indi- cation that the horse-shoe effect plays a role in this analysis.

4. Ecosystem models

Dynamic ecosystem models seem to be an ele- gant solution to gain more insight in the processes and forcing functions and to make reliable predictions of the consequences of changing management. However, there are several weaknesses in the modelling approach both when prediction is the major aim and when models used to gain more insight into the functioning of the ecosystem are getting complex. On the other hand the modelling approach is a useful tool to investigate the complex interactions between systems components in relatively simple systems, which hardly can be studied using an experimental approach. The most important limitation of the modelling approach, when used as the only tool, is that

O.F.t~ van Tongeren / Ecological Modelling 78 (1995) 51-60 55

causal relationships are implicit in the process formulations. This is no problem for theoretical models in which we explicitly try to reveal the pattern of the complex interactions, but for management models, which are developed to predict, the causal relationships should be proven, in other words the structure of the model should be vali- dated (identified, cf. identification of the model structure; Beck, 1983).

In order to reduce the complexity of ecosystem models, the ecosystem is arbitrarily divided into a relatively low number of compartments or state variables. Processes are formulated as differential equations, relating the compartments to each other. Lumping of species into heterogeneous functional groups or model compartments or lumping of developmental stages of species that change their feeding habits when growing may lead to undesired central tendency effects (e.g. Van Tongeren et al., 1992): the decrease of one component, e.g. species, and concurrent increase of another component of the same compartment will offset each other, thus masking the drastic changes that occur. Dynamic relationships between model compartments, expressed in the model parameters, may change as a consequence, but the rigid structure of the models does not allow for this. When such models are used for prediction purposes, the probability of a good prediction is very low in general, but sharply decreasing when one or more boundary conditions are changed in order to predict future changes in a different management scenario or after reduction of major loads.

An ecosystem model for the Loosdrecht lakes, PC-Loos was developed by Janse and Aldenberg (1990a,b, 1991; Janse et al., 1992). An example of the result of an, in my opinion undesired, central tendency effect is the low variance in the modelled zooplankton concentration, which is an order of magnitude smaller than the variance of the observations until 1985 and about a factor 4 lower from 1986 onwards (not shown). An alternative explanation for this phenomenon is that growth and death rates of zooplankton are underesti- mated.

A solution for this may be to increase the number of state variables, in other words to model

more strictly defined ecological groups or even the populations of all species instead of very broadly defined functional groups. In the original PC-Loos model the algae were combined into one state variable. Discussion in the WQL-group, mainly based on the results of a preliminary multivariate analysis, led to the three now defined state variables, being diatoms, green algae and filamentous cyanobacteria. However, the increase in the number of parameters to be estimated and the number of interactions between state variables are almost proportional to the square of the number of state variables. Maybe this will eventu- ally allow for a more accurate prediction (however, see Peters, 1990, 1991), but, since the number of state variables and interactions is too high, a complete analysis of the properties of the model (global sensitivity analysis, isocline analysis) is impossible. Moreover, the high number of parameters to be estimated requires an even higher number of independent observations (cf. autocorrelation problems in time series analysis) or a huge number of experiments for a correct estimation, by calibration or by regression respectively, of the parameters.

Alternatively, we may change state variables into forcing (independent) variables that are measured in the system and 'fed' into the model, thereby reducing the problem to a population dynamic model with few populations. Such population dynamic models, using essentially the Lotka-Volterra equations or modifications there- of, and individual based models (Cellular au- tomata, e.g. Hogeweg and Richter, 1982; IBMs, e.g. DeAngelis and Gross, 1991) are fairly simple, Because of the fact that the number of boundary conditions is relatively high and the number of parameters is fairly low, they are very successful in making predictions at the population level. However, combining several of such models into an ecosystem model will inevitably lead to the same problems that occur in complex ecosystem models. Nevertheless, the problems with parameter estimation will be largely reduced, because parameters are estimated excluding the effects of the uncertainty in the prediction of the state variables that have been changed into forcing functions. However, measurement errors in the

56 O.F.R. van Tongeren / Ecological Modelling 78 (1995) 51-60

forcing functions still may impose several con- straints on the accuracy of the parameter estimates.

Ideally all parameters for a model should be estimated independently, but in practice most parameters cannot be estimated with sufficient accuracy or, even worse, cannot be estimated at all. The problem mainly stems from the uncertainty in the data used to calibrate models (e.g. Warwick and Cale, 1988). Calibration techniques are often used to adjust the model parameters in order to obtain a better fit. In analogy with multiple regression analysis, where correlated independent variables hamper a correct estimation of the regression parameters resulting in high standard errors for the estimated regression coeffi- cients, parameter sets obtained by calibration techniques are likely to be unreliable, but formal tests are not available. The main problem here is that outliers in the observations cause deviations in the estimates for certain model parameters that are compensated by changes in highly correlated parameters (cf. the multicollinearity problem, i.e. the effect of highly correlated variables that may replace each other in descriptive multiple regression models).

Recently some more attention has been paid to this problem. The full parameter space of the

model is reduced to a parameter space that gives model outputs that are sufficiently accurate, allowing different combinations of parameters. Techniques range from exploring the parameter space using a grid (e.g. Prentice et al., 1987; Aldenberg et al., 1995) to sophisticated algorithms that, starting from a random set of points in the parameter space, converge to a configura- tion of points comprising the parameter space related to the desired output range (e.g. Klepper, 1989; Klepper and Rouse, 1991). However, the main problem in this case is to choose the output variables and their allowable output ranges, which till now is a matter of best professional judgement (see also the next paragraph).

Verification and validation procedures are more or less informal. The judgement whether the model describes or predicts reasonably well is often largely based on a graphic representation of the measured values of the state variables and the modelled values. However, often the residuals are not analysed, although they might give more insight into the model structure. Two examples from Janse and Aldenberg (1992) are pre- sented in Figs. 2 and 3. The first example is derived from the modelled and observed chlorophyll-a concentrations. The modelled dynamics of chlorophyll-a are more or less consistent with the

15o + ] + + + A u t u m n 1 9 8 3

+ 4-

4 - - "~ ' "~ ~ 4- " .4- + ..... ÷ • + ,..:I= S u m m e r 1 9 8 3 ~,: 9 o-~-~+*---~ "'~-....~.!, '".-7' '/++

~= "-.j \ ................. ~ . . . . \ +

100 " ~ ~ +

+

-i so ~o 16o Igo 260 2~o aoo E x p e c t e d c h l o r o p h y l l - a ( m g / m ^ 3)

Fig. 2. Residuals of chlorophyll-a concentration against expected chlorophyll-a concentration. Note the negative correlation between expected value and residuals. The residuals were measured from the graphs in Janse et al. (1992). Lines demonstrate that the model is responding faster to changes in the environment than the real data.

O.F.t~ van Tongeren /Ecological Modelling 78 (1995) 51-60 57

data for the years 1984 through 1986. However, in general, the residuals are negatively correlated with the expected values (Fig. 2). More important, however, is that this complex dynamic model shows the same type of deviation from reality as the simple ordination model, residuals in spring being more negative than residuals in autumn, indicating that the modelled response to temperature is too fast, which is most probably caused by overestimated growth and death rates. The

most important problem with the PC-Loos model is that the residuals of total seston are changing with time (Fig. 3), indicating that the model tends to overestimate the seston concentrations before restoration measures that were taken in 1984 and to underestimate the seston concentration some time after the restoration measures. As a consequence, prediction of future changes in the tur- bidity will be unreliable.

Overall measures of fit, like residual sum of

10

4- E

2 -

® o~ "5 0 -

rr"

- 2 -

- 4 -

. 6 ~

-8 1'0 1'1 1~2 1'3 1'4 1 5

Expected seston (mg C.I ^ -1)

5 ,

3 ̧ "7 (

2-

O3

o 69 "5 -1

-2

- 3 -

- 4 -

-5 1983

A -,-A A A A_ - + ++ +

4- 4- 4- 4- . ~ + + + +

4-4- + 4- + +4- ÷ 4- + 4-

+- 4- + +

i T - - - - 4 - - - i ÷

4- 4- + +. ,+ -~+ ÷

+ 4- + + 4- 4-

4- + 4- 4-

4-4-

19'84 19'85 19'86 19'87 19'88 19'89 19'90 1991 T i m e

Fig. 3. Residuals of seston concentration. (a) Against expected seston concentration, lines as in Fig. 2. (b) Against time, lines (connecting the yearly average residuals) demonstrate increasing residuals with time. Dashed lines: observations with extremely high seston concentration (possibly due to wind resuspension) are excluded.


squares, summed over all observations over all state variables often merely express that a few state variables with high standard deviations are modelled well. Percentage variance explained by the model averaged over all state variables and the posterior density (Box, 1971; Janse and Aldenberg, 1992) are rather sensitive to unex- plained variation in some variables of minor importance. Anyhow, if not all state variables are included, the chosen measure of goodness of fit merely reflects the a priori ideas of the modeller on the importance of the state variables. If a state variable is considered to be sufficiently important to be included in the model, it should contribute to the goodness of fit measure, if not, the model should be simplified.

Finally, but certainly not always the least important there is the problem of the propagation of errors: although there are very sophisticated algorithms for integration, minor deviations stem- ming from simple rounding errors may grow into unrealistic proportions as a consequence of the repeated calculations, especially when there are few feedbacks in the model.

5. Integration of the methods into one framework: discussion

The results of indirect gradient analysis, pro- vided that the interpretation is straightforward, can be used to display relationships (correlations or similarities) between species. These displays provide us with an important tool to distinguish ecological groups (guilds), which may be lumped into composite variables to be used as state variables in the dynamic models. An additional analysis of the relationship between sample scores in the ordination and environmental variables or, preferably, a direct gradient analysis on process rates may guide us in the choice of forcing functions. Ordination analysis of several lakes with different environmental conditions or management regimes provides a tool for a more formal approach towards "educated speculation" (Van Straten, 1992), especially when instead of state variables rates of processes are analysed. In the

latter case also differences in process rates among lakes are quite easily detected.

Some recent models, based on concepts of thermodynamics (J~rgensen, 1990, 1995; Ripl, 1995), allow a less rigid structure, with changing parameters. However, the theoretical basis for these so-called "structural models" (J~rgensen, 1986, 1988, 1995), although promising, is still un- der discussion.

A more straightforward approach may be to combine the virtues of regression analysis and those of dynamic modelling in "nested models". In such models the local parameter estimates are obtained from a (descriptive) regression of the (calibrated or measured) parameters of the same model, applied to a number of lakes with different properties, on the major characteristics of the lakes (e.g. morphometry, soil type) and the forcing functions (e.g. climate and major loads). A nested model thus consists of regression models embedded into a dynamic ecosystem model. The range of lakes used for these regressions defines the domain of the model. The full parameter space of the model can easily be derived from all independent sets of parameter estimates. The uncertainties in the model parameters follow straightforward from the regression, and can be used for the construction of confidence limits in prediction. These confidence limits will become narrower when certain combinations of parameter estimates are not allowed, because they do not occur in the parameter space, due to relationships between the parameter estimates; in other words: if the relationships between parameter estimates (e.g. covariances) are explicitly taken into account.

Within one lake, essentially the same approach could be used to assess changes in parameters that appear to be related to the state variables. Options are to calibrate the model for separate years or for separate seasons in order to detect long-term changes or seasonal fluctuations in the parameters. At first sight, it seems that by doing this, a structural error in a model is repaired by some arbitrary mathematical trick: an essentially non-linear relationship, specified as a linear relationship in the model, can for example turn into a non-linear one by doing this. However, some-

O.ER. van Tongeren /Ecological Modelling 78 (1995) 51-60 59

times we have insight into processes derived from experiments. If, for example, we a priori know that a process is essentially linear (quantitatively), the deviations from linearity expressed as a changing parameter indicate qualitative changes in the state variables instead of non-linearity in the process. On the other hand, an essentially non-linear process modelled as a linear process with changing parameter may well be identified by experiments, designed a posteriori to test those relationships that show a changing parameter.

6. Conclusions

Both the ordination model and the dynamic ecosystem model used in the WQL study showed important deviations from reality. For the ordination model seasonal autocorrelations could be expected, but the dynamic ecosystem model should not show such deviations. More important, long-term trends in the residuals that are observed show that both models fail in prediction. This is mainly due to the large error variance in the data used to estimate the parameters for both models. Analysis of the residuals showed these shortcomings of both models and should be applied, whenever possible.

Nested models seem to be both an attractive alternative for structural models and a guide for educated speculation.

Acknowledgements

The author thanks Ramesh Gulati, Wolf Mooij, Cajo ter Braak and Koos Vijverberg for their critical comments.

References

Aldenberg, T., Janse, J.H. and Kramer, P.R.G., 1995. Fitting the dynamic model PCLake to a multi-lake survey through Bayesian statistics. Ecol. Model., 78: 83-99.

Anderson, R.M., Turner, B.D. and Taylor, L.R. (Editors), 1979. Population Dynamics. Blackwell, Oxford.

Beck, M.B., 1983. Uncertainty, system identification and the prediction of water quality. In: M.B. Beck and G. van

Straten (Editors), Uncertainty and Forecasting of Water Quality. Springer Verlag, Berlin, pp. 3-15.

Box, M.J., 1971. A parameter estimation criterion for multire- sponse models applicable when some observations are missing. Appl. Stat., 20: 1-7.

Box, G.E.P. and Jenkins, G.M., 1976. Time Series Analysis: Forecasting and Control. Holden-Day, San Francisco, CA.

Burrough, P.A., 1987. Spatial aspects of ecological data. In: R.H.G. Jongman, C.J.F. ter Braak and O.F.R. van Ton- geren (Editors), Data Analysis in Community and Land- scape Ecology. PUDOC, Wageningen, pp. 213-251.

DeAngelis, D.L., 1988. Strategies and difficulties of applying models to aquatic populations and food webs. Ecol. Model., 43: 57-73.

DeAngelis, D.L. and Gross, L.D. (Editors), 1991. Individual Based Models and Approaches in Ecology: Populations, Communities and Ecosystems. Chapman and Hall, Lon- don.

DeAngelis, D.L., Adams, S.M., Breck, J.E. and Gross, L.J., 1984. A stochastic predation model: application to large- mouth bass observations. Ecol. Model., 24: 25-41.

Engelen, G.B., Kal, B.F.M., Buyse, J.J. and Vanpruissen, F.G.M., 1992. The hydrology of the Loosdrecht lakes area. Hydrobiologia, 233: 21-38.

Gauch, H.G., 1982. Multivariate Analysis in Community Ecol- ogy. Cambridge University Press, Cambridge, 298 pp.

Hill, M.O., 1973. Reciprocal Averaging, an eigenvector method of Ordination. J. Ecol., 61: 237-249.

Hill, M.O. and Gauch, H.G., 1980. Detrended Correspon- dence Analysis, an improved ordination technique. Vege- tatio, 42: 47-58.

Hogeweg, P. and Richter, A.F., 1982. INSTAR, a discrete event model for simulating zooplankton population dynamics. Hydrobiologia, 95: 275-285.

Janse, J.H. and Aldenberg, T., 1990a. PCLOOS, a eutrophication model of the Loosdrecht lakes. WQL report no. 1990-1. Report no. 714502001, Natl. Inst. of Public Health and Environ. Prot., Bilthoven, 92 pp.

Janse, J.H. and Aldenberg, T., 1990b. Modelling phosphorus fluxes in the hypertrophic Loosdrecht lakes. Hydrobiol. Bull., 24: 69-89.

Janse, J.H. and Aldenberg, T., 1991. Modelling the eutrophication of the shallow Loosdrecht lakes. Verh. Int. Ver. Limnol., 24: 751-757.

Janse, J.H., Aldenberg, T. and Kramer, P.R.G., 1992. A Mathematical model of the phosphorus cycle in Lake Loosdrecht and simulation of additional measures. Hydro- biologia, 233: 119-136.

Jcrgensen, S.E., 1986. Structural dynamic model. Ecol. Model., 31: 1-9.

J0rgensen, S.E., 1988. Use of models as experimental tool to show that structural changes are accompanied by increased exergy. Ecol. Model., 41: 117-126.

J0rgensen, S.E., 1990. Ecosystem theory, ecological buffer capacity, uncertainty and complexity. Ecol Model., 52: 125-133.

J0rgensen, S.E., 1995. State of the art of ecological modelling in limnology. Ecol. Model., 78: 101-115.


Keizer, P. and Sinke, A.J.C., 1992. Phosphorus in the sedi- ment of the Loosdrecht lakes and its implications for lake restoration perspectives. Hydrobiologia, 233: 39-50.

Klepper, O., 1989. A model of carbon flows in relation to macrobenthic food supply in the Oosterschelde estuary (S,W. Netherlands). Thesis, Agricultural University Wa- geningen, 270 pp.

Klepper, O. and Rouse, D.I., 1991. A procedure to reduce parameter uncertainty for complex models by comparison with real system output illustrated on a potato growth model. Agric. Syst., 36: 375-395.

Krambeck, H.-J., 1995. Application and abuse of statistical methods in mathematical modelling in limnology. Ecol. Model., 78: 7-15.

McCullagh, P. and Nelder, J.A., 1983. Generalized Linear Models. Chapman and Hall, London, 261 pp.

Peters, R.H., 1983. The Ecological Implications of Body Size. Cambridge Studies in Ecology, 329 pp.

Peters, R.H., 1986. The role of prediction in limnology. Lim- nol. Oceanogr., 31: 1143-1159.

Peters, R.H., 1990. Pathologies in Limnology. Mere. Ist. Ital. Idrobiol., 47: 181-217.

Peters, R.H., 1991. A Critique for Ecology. Cambridge Uni- versity Press.

Prentice, I.C., Van Tongeren, O. and de Smidt, J.T., 1987. Simulation of heathland vegetation dynamics. J. Ecol., 75: 203-219.

Rao, C.R., 1964. The use and interpretation of principal component analysis in applied research. Sankhya A, 26: 329-358.

Rao, C.R., 1973. Linear Statistical Inference and Its Applica- tions, 2rid edn. Wiley, New York, NY, 625 pp.

Ripl, W., 1995. Management of water cycle and energy flow for ecosystem control: the energy-transport-reaction (ETR) model. Ecol. Model., 78: 61-76.

Simberloff, D., 1983. Competition theory, hypothesis-testing, and other community ecological buzzwords. Am. Nat., 122: 626-635.

Snedecor, G.W. and Cochran, W.G., 1980. Statistical Meth- ods, 7th edition. The Iowa State University Press, Ames, IA, 507 pp.

Ter Braak, C.J.F., 1987a. Regression. In: R.H.G. Jongman, C.J.F. ter Braak and O.F.R. van Tongeren (Editors), Data Analysis in Community and Landscape Ecology. PUDOC, Wageningen, pp. 29-77.

Ter Braak, C.J.F., 1987b. Ordination. In: R.H.G. Jongman,

C.J.F. ter Braak and O.F.R. van Tongeren (Editors), Data Analysis in Community and Landscape Ecology. PUDOC, Wageningen, pp. 91-173.

Ter Braak, C.J.F., 1987c. CANOCO - A FORTRAN pro- gram for canonical community ordination by [partial] [detrended] [canonical] correspondence analysis, principal component analysis and redundancy analysis (version 2.1). TNO Institute of Applied Computer Science, Wagenin- gen, the Netherlands.

Ter Braak, C.J.F., 1989. CANOCO - an extension of DECO- RANA to analyze species-environment relationships. Hy- drobiologia, 184: 169-170.

Van Liere, L., van Tongeren, O.F.R., Breebaart, L. and Kats, W., 1990. Trends in chlorophyll and total phosphorus in the Loosdrecht lakes, The Netherlands. Ver. Int. Ver. Limnol., 24: 707-710.

Van Straten, G., 1992. Predicting lake ecosystem behaviour: Uncertainty and educated speculation. In: E. Rott (Editor), Abstracts of Int. Conference Mathematical Modelling in Limnology, November 4-7, 1992. Institut fiir Botanik, Innsbruck, 26/27.

Van Tongeren, O.F.R., van Liere, L., Gulati, R.D., Postema, G. and Boesewinkel-de Bruyn, P.J., 1992. Multivariate analysis of the plankton communities in the Loosdrecht lakes - Relationship with the chemical and physical environment. Hydrobiologia, 233: 105-117.

Vollenweider, R.A., 1976. Advances in defining critical loading levels for phosphorus in the lake eutrophication. Mem. Ist. Ital. Idrobiol., 33: 53-83.

Vollenweider, R.A., 1987. Scientific concepts and methodolo- gies pertinent to lake research and lake restoration. Schweiz. Z. Hydrol., 2: 129-147.

Warwick, J.J. and Cale, W.G., 1988. Estimating model relia- bility using data with uncertainty. Ecol. Model., 41: 169- 181.

Westhoff, V. and Van der Maarel, E., 1980. The Braun- Blanquet approach. In: R.H. Whittaker (Editor), Classifi- cation of Plant Communities. Junk, the Hague, 408 pp.

Wetzel, R.G., 1983. Limnology. Saunders College Publishing, Philadelphia, 767 pp.

Whittaker, R.H., 1980a. Approaches to classifying vegetation. In: R.H. Whittaker (Editor), Classification of Plant Com- munities. Junk, the Hague, 408 pp.

Whittaker, R.H. (Editor), 1980b. Ordination of Plant Commu- nities. Junk, the Hague, 388 pp.

data analysis or simulation model: a critical evaluation of some methods

Documents