Environmental Impact Assessments: A Statistical Encounter

Dave SavilleSaville Statistical Consulting Ltd

P O Box 69192Lincoln 7640New Zealand

Outline of talk

• Background• Previous statistical approaches• My alternative approach• Discussion

2

Background

3

• Noise and/or visual impacts of existing or proposed public (or private) amenities are of concern to planning bodies.

• Social scientists may be called upon to survey attitudes to noise or visual effects when planning hearings are due to take place.

• The results of such social surveys are used to help decide the size of an appropriate buffer zone around existing or proposed amenities (within which housing development is not permitted), or to decide whether it is appropriate to develop new amenities in a particular area. 4

Examples of such amenities are:• Rifle range• Speedway• Wind farm• Stadium• Airport• Motorway

(Fictitious) Elephant park (remember Cashin + Burma?)

5

How did I get involved?Through an approach for statistical advice by a social scientist who conducted social impact surveys prior to planning hearings for two such projects.

6

Previous statistical approaches

7

The elephant park

8

The elephant park…. add noise contours…..

9

10

60 dBA

55 dBA

50 dBA

11

60 dBA

55 dBA

50 dBA

Add houses

12

60 dBA

55 dBA

50 dBA

Add houses

Survey the occupants on noise impact

13

60 dBA

55 dBA

50 dBA

Approach #1: Choose a cutoff like 52 dBA. Then do a statistical analysis comparing responses from houses within the contour with those from houses outside the contour. Repeat for all such contours (by 1dBA).

14

Sample sizes by dBAdBA Total respondents44 18145 3546 3647 4248 5049 1250 2351 1552 10

53-54 1655-58 1759-66 17

15

Approach #1: Choose a cutoff like 52 dBA. Then do a statistical analysis comparing responses from houses within the contour with those from houses outside the contour. Repeat for all such contours (by 1dBA increments).

For each such analysis, count the number of variables which differ significantly between houses inside and outside the “cutoff” contour.

For example, there was a significant difference between those living inside and outside the “cutoff” contour for only 1 response variable when the cutoff was 51, 52, 53 or higher dBA,BUT for 3 response variables when the cutoff was 50 dBA or less.

On this basis, it was argued that there was an “important separation between those living in areas of 50 dBA and under, and those in areas of 51 dBA and over.” (Hence 50 dBA should be the threshold dBA level, with no development closer….)

16

dBA Sample size Propn(p)<45 179 0.145

45-49 174 0.16750-54 63 0.22255+ 34 0.294

p is proportion 'very much' orextremely annoyed by trumpeting.

My Thoughts: There are fewer and fewer houses as you get closer to the elephant park, so the imbalance in the two sample sizes increased with increasing dBA, decreasing the chance of achieving statistical significance. (Hence this is not a good basis for decision making.)

The sample sizesand one variable:

17

Approach #2: Divide land into 4 zones on the basis of dBA, and argue as follows:(a) Level of noise is acceptable in zone B;(b) No statistically significant difference between zones B and C;(c) Therefore level of noise is acceptable in zone C (hence developers owning land in zone C can go ahead…).

Level of noiseZone dBA Sample size Propn(p) not in dispute

A <45 179 0.145 AcceptableB 45-49 174 0.167 AcceptableC 50-54 63 0.222D 55+ 34 0.294 Unacceptable

18

Approach #2: Divide land into 4 zones on the basis of dBA, and argue as follows:(a) Level of noise is acceptable in zone B;(b) No statistically significant difference between zones B and C;(c) Therefore level of noise is acceptable in zone C (hence developers owning land in zone C can go ahead…).

Zone dBA Sample size Propn(p) Level of noiseA <45 179 0.145 AcceptableB 45-49 174 0.167 AcceptableC 50-54 63 0.222 AcceptableD 55+ 34 0.294 Unacceptable

19

My Thoughts: You could equally well argue as follows:(a) Level of noise is unacceptable in zone D;(b) No statistically significant difference between zones C and D;(c) Therefore level of noise is unacceptable in zone C (hence developers owning land in zone C cannot go ahead…).

Level of noiseZone dBA Sample size Propn(p) not in dispute

A <45 179 0.145 AcceptableB 45-49 174 0.167 AcceptableC 50-54 63 0.222D 55+ 34 0.294 Unacceptable

20

My Thoughts: You could equally well argue as follows:(a) Level of noise is unacceptable in zone D;(b) No statistically significant difference between zones C and D;(c) Therefore level of noise is unacceptable in zone C (hence developers owning land in zone C cannot go ahead…).

Zone dBA Sample size Propn(p) Level of noiseA <45 179 0.145 AcceptableB 45-49 174 0.167 AcceptableC 50-54 63 0.222 UnacceptableD 55+ 34 0.294 Unacceptable

21

Result: Using this line of reasoning, you can prove that the elephant noise is acceptable in zone C (the 50-54 dBA zone that is in dispute), AND that it is unacceptable in zone C. (A contradiction has been arrived at!!)

22

Result: Using this line of reasoning, you can prove that the elephant noise is acceptable in zone C (the 50-54 dBA zone that is in dispute), AND that it is unacceptable in zone C. (A contradiction has been arrived at!!)

Interesting aside: The hearing commissioner was apparently quite switched-on statistically, pointing out the fallacy in the argument without my client having to say a word (luckily he still paid my fee….).

My alternative approach

23

24

Fit a trend line or curve to model the proportion of households “very much” or “extremely” annoyed as a function of the dBA level of the households.

25

Details of curve fitting

• Generalized linear model.• Response variable is binomial:

“0” if the annoyance level is 2 or less (0=not at all annoyed; 1=slightly; 2=moderately);“1” if the annoyance level is 3 or 4 (3=very much; 4=extremely annoyed).

• The 454 individual responses are used in the modeling.

• Logit (log odds) link function.• Explanatory variable is the dBA value for the individual

household.

26

The resulting fit (back-transformed from logits).

0.000

0.200

0.400

0.600

0.800

1.000

40 45 50 55 60 65Prop

orti

on "

very

muc

h" o

r "e

xtre

mel

y"

anno

yed

dBA

Propn(p)

Fitted(p)

27

Details of sample sizes and points on curve

(Equation is: Fitted proportion = elin / (1 + elin) where lin = -4.55 + 0.0633 x dBA)

dBA No. of 3 & 4 Total respondents Propn of 3 and 4(p) Fitted proportion (curve)44 26 181 0.144 0.14645 5 35 0.143 0.15446 7 36 0.194 0.16347 9 42 0.214 0.17248 7 50 0.140 0.18149 1 12 0.083 0.19050 6 23 0.261 0.20051 4 15 0.267 0.21152 1 10 0.100 0.221

53-54 4 16 0.250 0.23855-58 3 17 0.176 0.27459-66 7 17 0.412 0.320

28

Suggested usage of fitted curve• I suggested that the client groups would need to decide upon a

proportion of the residents that they could tolerate being “very much” or “extremely” annoyed with the elephant trumpeting noise (considering just this one response variable at this stage). For example, they might decide upon 0.20 (or 20%) as a maximum proportion.

• The solution from the curve is 50 dBA (as can also be seen in the above table, where the figure of 0.200 corresponds to a dBA level of 50 dBA). That is, if the noise control boundary is taken to be 50 dBA, 20% of the residents living on this boundary would be predicted to be “very much” or “extremely” annoyed with the elephant trumpeting noise (based upon this one survey at one point in time).

• This solution puts the onus for decision-making back on the community (where it belongs), not on the statistician.

Discussion

29

30

This talk illustrates the fact that analyses of data on environmental impact can suffer from the same sorts of problems that biometricians in agriculture and related fields encounter on a daily basis – in particular, the carrying out of pair-wise tests of significance between groups when a trend analysis is clearly more appropriate (c.f., paper by Maindonald and Cox; paper by Tom Little “If Galileo published in HortScience”).

(I should comment that my client commented “after the event” that these trend analyses are commonplace in this field.)

The End

31