natural resources defense council’s & powder river basin resource council… ·...

NATURAL RESOURCES DEFENSE COUNCIL’S & POWDER RIVER BASIN

RESOURCE COUNCIL’S PETITION FOR REVIEW

EXHIBIT 14

5 6 4 4

CHARACTERIZATION OF BACKGROUND WATER QUALITY

FOR STREAMS AND GROUNDWATER

FERNALD ENVIRONMENTAL MANAGEMENT PROJECT FERNALD, OHIO

REMEDIAL INVESTIGATION and FEASIBILITY STUDY

I-

May 1994 .

U.S. DEPARTMENT OF ENERGY FERNALD FIELD OFFICE

DRAFT’”FINAL 4 . , . . ,

J

I 5644 Background Study

May 1994

TABLE OF CONTENTS

List of Tables List of Figures List of Acronyms 1.0 Introduction -

1.1 Brief History of the Site 1.2 Purpose 1.3 Geologic Setting 1.4 Hydrologic Setting

1.4.1 Great Miami River 1.4.2 Paddys Run 1.4.3 Great Miami Aquifer 1.4.4 Glacial Overburden 1.4.5 Monitoring Wells

1.5 EPA Guidance on Background Characterization 1.6 Summary of Revisions Made to the Draft Report (May 1993)

2.0 Previous Studies

3.0

2.1 Environmental Monitoring Program 2.2 U.S. Geological Survey Surface Water Monitoring 2.3 U.S. Geological Survey Groundwater Study 2.4 IT Corporation Final Interim Report 2.5 Argonne National Laboratory Environmental Survey 2.6 Ohio Department of Health 2.7 Ohio Environmental Protection Agency Study of the Great Miami River 2.8 Previous RI/FS Background Studies 2.9 RCRA Groundwater Monitoring Development of the RI/FS Background Data Set 3.1 Sampling Locations

3.1.1 Surface Water, 3.1.2 Groundwater

3.1.2.1 Identification of Potential Background Monitoring Wells 3.1.2.2 Screening of Background Locations 3.1.2.3 General Water Chemistry and Charge Balance 3.1.2.4 Summary

3.2 Sample Collection 3.3 Analytical Procedures

Eiw? iv

vii viii 1-1 1-1 1-3 1-4 1-7 1-7

1-10 1-11 1-13 1-13 1-15 1-16 2-1 2-1 2-1 2-3 2-3 2-3 2-3 2-6 2-6 2-7 3-1 3-1 3-1 3-2 3-2 3-4

3-11 3-15 3-19 3-20

0USIG:BACKGRDISECS. 1-9.TOC.IOS-W i

Background Study May 1994

TABLE OF CONTENTS (continued)

Page

3.4 Data Validation Procedures 3.5 Validated and Deleted Data Data Set Modifications and Statistical Analysis Procedures 4.1 Overview 4.2 Modifications of the Background Data Set

4.2.1 Treatment of Rejected/Nonvalidated Data 4.2.2 Treatment of Nondetect Data 4.2.3 Identification and Treatment of Outliers and Other "Suspect" Data 4.2.4 Data Averaging

4.3.1 Testing of Data Distribution 4.3.2 Parametric Descriptive Statistics 4.3.3 Nonparametric Descriptive Statistics 4.3.4 Comparison of Populations

4.4 Summary of Revisions to Chapter 4 of the Draft Report

4.0

4.3 Statistical Analysis

5.0 Glacial Overburden a 5.1 Radiological Constituents 5.2 Inorganic Constituents 5.3 Organic Constituents

6.1 Radiological Constituents 6.2 Inorganic Constituents 6.3 Organic Constituents



References

6.0 Great Miami Aquifer

7.0 Great Miami River

8.0 Paddys Run

9.0 Conclusions

3-20 3-21 4-1 4-1 4-1 4-2 4-2 4-3, 4-5 4-5 4-5 4-7 4-7 4-8 4-8 5-1 5-1 5-2 5-3 6-1 6-1 6-2 6-3 7-1 7-1 7-1 7-2 8-1 8-1 8-1 8-2 9-1 R-1.

0USIG:BACKGRDISECS. 1-9.,W./OS-W ii

5644 . - Background Study

May 1994

TABLE OF CONTENTS (continued)

Appendix A - Data from Previous Studies Appendix B - Drilling Logs and Well Construction Information Appendix C - Radiological Data Appendix D - Inorganic Chemical Data Appendix E - Organic Chemical Data Appendix F - Statistical Procedures, Equations, and Results Appendix G - Summary of Revisions to the "Characterization of

Background Water Quality for Streams and Groundwater" Draft Report (May 1993)

Monitoring Wells in the Tributary Sections of the Great Miami Aquifer

Appendix H - Summary Statistics of Inorganic Constituents for Background

000004 OU5IG:BACKGRDISECS. 1-9.TOc./OS-W iii

Table E-19 Rejected/Nonvalidated Organic Data for Background Surface

Water in the Great Miami River

Well No. W-1 W-1 W-1

Sample lab Validated . QA Date ID qualifier Constituent Result Qualifier type

05/20/93 120064-2 U 4-Nitroaniline 25 R N 05/20/93 120068-1 U 4-Nitroaniline 25 R D 05/20/93 120072-2 U 4-Nitroaniline 25 R T

, .. . . . .

E-210

c'

Well Sample lab Validated QA No. Date ID qualifier Constituent Result Qualifier type W-5 03/25/93 113493 U 2,4-Dinitrophenol 50 R N

I W-5 03/25/93 113493 U 4,6-Dinitro-2-methylphenol 25 R N

Table E-20 Rejected/Nonvalidated Organic Data for Background Surface

Water in Paddys Run

E-21 1

d.

APPENDIX F

STATISTICAL PROCEDURES, EQUATIONS, AND RESULTS

TABLE OF CONTENTS

List of Tables Shapiro-W& Test for Normality Shapiro-Francia Test for Normality Rosner's Test for Many Outliers Data Averaging Sample Arithmetic Mean - Normal Distriiution Sample Arithmetic Standard deviation - Normal Distribution Estimated Coefficient of Variation - Normal Distribution Estimated Mean of a Lognormal Distribution Estimated Standard Deviation of a Lognormal Distribution Sample Median - Nonparametric Technique Upper One-Sided 95% Confidence Limit - Normal Distribution Upper One-sided 95% Confidence Limit - Lognormal Distribution Upper One-sided 95% Confidence Limit - Nonparametric Technique 95'h Percentile - Normal Distribution 9Sth Percentile - Lognormal Distribution 95'h Percentile - Nonparametric Technique F-Test T-Test The Wilcoxin Rank Sum Test Kruskal-Wallis Test

F-ii F-5

F-14 F-23 F-30 F-33 F-34 F-35 F-36 F-38 F-39 F-40 F-43 F-47 F-49 F-50 F-51 F-52 F-57 F-60 F-64

F-i

000924

LIST OF TABLES

F- 1

F-2

F-3

F-4

F-5

F-6

F-7

F-8

F-9

F-10

F-11

F-12

F-13

F-14

F-15

F-16

F-17

F-18

Formulas for Summary Statistics

Coefficients 3 for the Shapiro-Wilk W Test for Normality

F- 1

F-7

Quantiles of the Shapiro-Wilk W Test for Normality (Values of W Such that 100 p% of the Distribution of W Is Less Than WJ

Example Data Set Number 1

Standard Normal Curve for a Z Distribution

F-9

F-11

F-16

Percentage Points of the W' Test for n > 50


F-18

F-2 1

Approximate Critical Values Lamda (it 1 ) for Rosner's Generalized ESD Many-Outlier Procedure for alpha = 0.05



F-25

F-27

F-3 1

Quantiles of the t Distribution (Values o f t Such that 100 p%

Values of H (1-alpha) for Computing One-sided (Upper) 95% Confidence

of the Distribution Is Less Than 5) F-4 1

Limits on a Lognormal Mean F-44

Percentage Points of the F Distribution (Fo.025,dm,dn)


F-53

F-55

Example Data Set Number 6 F-62

Quantities of the Chi-square Distribution With v Degrees of Freedom

Ranking of Example Data Set Number 7

Ranking of Example Data Set Number 7 By Group

F-67

F-68

F-69

F-ii

000925

FEMP Background Study . May 1

Table F-1 Formulas for Summary Statistics

statistic

ShapiiW& Test (Gilbert 1987, Equations 12.3 and 12.4)

ShaphFrancia Test (Shapiro-Francia, 19tz)

_ _ ~

Rosner's Test for Many Outliers (Gilbert 1987, Equations 15.1 to 15.3)

Sample Arithmetic Mean (Gilbert 1987, Equation 43)

Formula

2

where: n n 2

d E C xi2 -- i = l f,P i = l 4

n

- n - 1

k =; i fn i sevcn

= i fn i sodd a, = Shapiro-Wilk coefficient x, = ith data value in the ordered data set

n = numberofdatapoints W = Shapim-Wilk test statistic

= square of the ith data value in the ordered data set

where:

mi = normal qwntile a-1 = inverse of standard normal distribution x, = ith data value in the ordered data set s2 = sample arithmetic variance W' = Shapiro-Francia test statistic

See page F-23.

.'--cy I n

i.1

where: n = number of data points x, = i* data value in the ordered data set x = arithmetic mean -

F- 1

64% TABLE F-1 (Continued)

FEMP Background Study May 1994

Statistic

Sample Arithmetic Standard Deviation (Gilbert 1987, Equation 4.4)

Estimated Coefficient of Variation (Gilbert 1987, Page 34)

Estimated Mean of a Lognormal Distribution (Gilbert 1987, Equation 13.7)

Estimated Standard Deviation of a Lognormal Distribution (Gilbert 1987, Equation 13.8)

Formula

whcm: n = number of data points

x = arithmetic mean s2 = arithmetic variance s = arithmetic standard deviation

3 = data Set value

CV - s f i

where: - x = sample arithmetic mean s = sample arithmetic standard deviation CV = estimated coefficient of variation

ji - exp [. + $1 where:

6y = arithmetic standard deviation of the In transformed data

p = estimated mean of a lognormal distribution

- y = arithmetic mean of the In transformed data

A

b - /b' [ exp s; - 11

where: A = estimated mean of a lognormal distribution P 6y = arithmetic standard deviation of the In transformed data

= estimated standard deviation of the lognormal distribution

000927

F-2 * . n

r.'.., -... 3 . . . ' 2

statistic

Simple Median - Nonparametric Technique (Gilbert 1987, Equation 13.15 and 13.16)

~ ~~~

Upper 95% Confidence Limit on the Mean - Normal Distribution (Gilbert 1987, Equation 11.6)

Upper 95% Confidence Limit on the Arithmetic Mean for Lognormal Distribution (Gilbert 1987, Equation 13.13)

TABLE F-1 (Continued)

Formula

I f n i s o d d

sample median = xI(" .

If n is cvcn:

w h C X

xm = i" data value in the ordered data set

n = number of data points

whem - x = arithmetic mean t o.95,n-, = student t distribution value n = number of data points s = arithmetic standard deviation 95% UCL, = one-sided upper 95% confidence limit for a normal

distribution

where: y = arithmetic mean of the In transformed data sy2 = arithmetic variance of the In transformed data sy = arithmetic standard deviation of the In transformed data

b.95 = value used to compute one-sided 95% confidence limit on a lognormal mean

n = number of data points 95% U q = one-sided upper 95% confidence limit for a lognormal

-

distribution

000928 . .

F-3

PeMP Background Study May 1994

TABLE F-1 (Continued)

statistic

Upper 95% Confidence Limit on the Median - Nonparametric Technique (Gilbert 1987, Equation 1322)

statistic

Upper 95% Confidence Limit on the Median - Nonparametric Technique (Gilbert 1987, Equation 1322)

0-95 Quantitle - Nonnal Distribution (Gilbert 1987. Equation 11.1)

0.95 Quantile - Lognormal Distribution (Gilbert 1987, Equation 13.24)

0.95 Quantilc - Nonparametric Technique

P t a t

T-tat

Wilcoxon Rank Sum t a t

hka l -Wal l i s test

Formula

wbelC: n = number of data points %.95 = upper 95% limit from a standard normal curve for a Z distribution U = rank in an ascending order data set that corresponds to the one-sided

f(U) = U rounded up to an integer (e.&, 242 -. 25) 95% UCL,.,, = data point in the ascending ordered data set at rank f(u)

upper 95% confidence limit for nonparametric distribution *.

where: - x = arithmetic mean s = arithmetic standard deviation &,g5 = 0.95 limit from a standard normal curve for a Z distribution 95" PercentileN = 9Sth percentile for normal distribution

where: 7 = arithmetic mean of the In transformed data 5 = arithmetic standard deviation of the In transformed data

Z& = upper 95% limit from a standard normal curve for a distribution 9Sth PercentileL = 9Sth percentile for lognormal distribution

Q = 0.95 n

sth PercentileNp - x ~ ~ ( ~ ) ~

where: n = number of data points Q ' = rank in an ascending order data set that corresponds to the 95"

percentile based on a nonparametric technique f(Q) = Q rounded up to an integer (e.& 14.1 4 1s) 95" PercentileNp = data point in the ascending order data bet at rank f(Q)

See WEC F-52.

See pa@ F-57.

Set page F-60.

See page F-64.

F-4 (900929

FEW Background Study May 1994

Shapiro-Wilk Test for Normality

The W test developed by Shapiro and Wilk (Gilbert 1987, Equations 12.3 and 12.4) was used to determine whether or not a data set has been drawn from a population which is normally distributed for sample size of 50 or less. By conducting this test on the natural logarithm of each data value, the W test was used to determine whether or not the sample was drawn from an underlying lognormal distribution. The null hypothesis to be tested is:

The population has a normal (lognormal when the data is transformed) distribution.

versus

H A : The population does not have a normal (lognormal when the data is transformed) distribution.

If H, is rejected, then HA is accepted.

The following presents a step-by-step procedure for conducting the W test. The equation for calculating W is:

l 2 k

1. Compute the denominator (d) of the W test statistic

2 n n

i=l

F-5 000930


where:

n c x, = x, + x2 + .... + xn

i= l

n

i=l 2 2 c 4 = x i 2 +x2 + ... + d

2. Order the n data points in ascending order (smallest to largest) such that x, - x2 5 x3 I...< q

3. Compute k, where:

n 2

k = - i f n is even

n - 1 2

k = - i f n is odd

4. Find the coefficients a,, +, a,, ..., ak for the sample size n from Table F-2.

5. Compute W

1 w = - d

6. Reject H,, at the a significance level if W is less than the quantile given in Table F-3.

To test the null hypothesis that the population has a lognormal distribution, transform the observed data to y,, y2, ..., yn where yi = In 3. Repeat steps 1 through 6 as described above.

F-6

000931

I i

3 4 5 6 7 8 9 0.7071 0.6872 0.6646 0.6431 0.6233 0.6052 0.5888 O.OOO0 0.1677 0.2413 0.2806 0.3031 0.3164 0.3244

0.0000 0.0875 0.1401 0.1743 0.1976 O.oo00 0.0561 0.0947

0.0000

- May 1994

10 0.5739 0.3291 0.2141 0.1224 0.0399

4 -

19 0.4808 0.3232 0.2561 0.2059 0.1641 0.1271 0.0932 0.061 2 0.0303 0.0000

E 1 2 3 4

' 5 6 7

20 0.4734 0.321 1 0.2565 0.2085 0.1686 0.1334 0.1013 0.071 1 0.0422 0.0140

E 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -

15 0.51 50 0.3306 0.2495 0.1878 0.1353 0.0880 0.0433 0.0000

11 0.5601 0.331 5 0.2260 0.1429 0.0695 0.0000

I

16 0.5056 0.3290 0.2521 0.1939 0.1447 0.1005 0.0593 0.0196

13 0.5359 0.3325 0.2412 0.1707 0.1099 0.0539 0.0000

12 0.5475 0.3325 0.2347 0.1586

0.0303 1 0.0922

14 0.5251 0.3318 0.2460 0.1802 0.1240 0.0727 0.0240

~

28 0.4328 0.2992 0.2510 0.2151 0.1857 0.1601 0.1372 0.1162 0.0965 0.0778 0.0598 0.0424 0.0253 0.0084

~

29 30 0.4291 0.4254 0.2968 0.2944 0.2499 0.2487 0.2150 0.2148 0.1864 0.1870 0.1616 0.1630 0.1395 0.1415 0.1192 0.1219 0.1002 0.1036 0.0822 0.0862 0.0650 0.0697 0.0483 0.0537 0.0320 0.0381 0.0159 0.0227 0.0000 0.0076

24 0.4493 0.3098 0.2554 0.2145 0.1807 0.1512 0.1245 0.0997 0.0764 0.0539 0.0321 0.0107

25 0.4450 0.3069 0.2543 0.2148 0.1822 0.1539 0.1283 0.1046 0.0823 0.0610 0.0403 0.0200 0.0000

26 0.4407 0.3043 0.2533 0.21 51 0.1836 0.1563 0.1316 0.1089 0.0876 0.0672 0.0476 0.0284 0.0094

0.3185 0.2578 0.21 19 0.1736 0.1399 0.1092 0.0804 0.0530 0.0263 0.0000

27 0.4366 0.3018 0.2522 0.21 52 0.1848 0.1584 0.1346 0.1 128 0.0923 0.0728 0.0540 0.0358 0.01 78 0.0000

0.3156 0.3126 0.2571 0.2563 0.2131 0.2139 0.1764 0.1787 0.1443 0.1480 0.1150 0.1201 0.0878 0.0941 0.0618 0.0696 0.0368 0.0459 0.0122 0.0228

0.0000

000932 F-7

E 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 - Ei

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 -

31 0.4220 0.2921 0.2475 0.2145 0.1 874 0.1641 0.1433 0.1243 0.1066 0.0899 0.0739 0.0585 0.0435 0.0289 0.0144 0.0000

41 0.3940 0.271 9 0.2357 0.2091 0.1876 0.1693 0.1531 0.1384 0.1249 0.1 123 0.1004 0.0891 0.0782 0.0677 0.0575 0.0476 0.0379 0.0283 0.0188 0.0094 0.0000

32 0.41 88 0.2898 0.2462 0.2141 0.1878 0.1651 0.1449 0.1265 0.1093 0.0931 0.0777 0.0629 0.0485 0.0344 0.0206 0.0068

42 0.391 7 0.2701 0.2345 0.2085 0.1874 0.1694 0.1535 0.1392 0.1259 0.1 136 0.1020 0.0909 0.0804 0.0701 0.0602 0.0506 0.041 1 0.031 8 0.0227 0.0136 0.0045

Table F-2 (Continued) Coefficients ai for the ShapireWilk W Test for Normality

33 0.4156 0.2876 0.2451 0.21 37 0.1880 0.1660 0.1463 0.1284 0.1 116 0.0961 0.081 2 0.0669 0.0530 0.0395 0.0262 0.01 31 0.0000

43 0.3894 0.2684 0.2334 0.2078 0.1871 0.1695 0.1539 0.1398 0.1269 0.1149 0.1035 0.0927 0.0824 0.0724 0.0628 0.0534 0.0442 0.0352 0.0263 0.01 75 0.0087 0.0000

source: Table A-6, Gilbert 1987.

. . * / , I. ., , Ui' u ' '

CIN/OU5RI /419195/TABF2.XLS/5-94

34 0.4127 0.2854 0.2439 0.2132 0.1882 0.1667 0.1475 0.1301 0.1140 0.0988 0.0844 0.0706 0.0572 0.0441 0.0314 0.0187 0.0062

-

- 44

0.3872 0.2667 0.2323 0.2072 0.1868 0.1695 0.1542 0.1405 0.1278 0.1 160 0.1049 0.0943 0.0842 0.0745 0.0651 0.0560 0.0471 0.0383 0.0296 0.021 1 0.01 26 0.0042

35 0.4096 0.2834 0.2427 0.21 27 0.1883 0.1673 0.1487 0.1317 0.1 160 0.1013 0.0873 0.0739 0.061 0 0.0484 0.0361 0.0239 0.01 19 0.0000

45 0.3850 0.2651 0.2313 0.2065 0.1865 0.1695 0.1545 0.1410 0.1286 0.1170 0.1062 0.0959 0.0860 0.0765 0.0673 0.0584 0.0497 0.041 2 0.0328 0.0245 0.0163 0.0081 0.0000

F-8

36 0.4068 0.2813 0.2415 0.2121 0.1883 0.1678 0.1496 0.1331 0.1179 0.1036 0.0900 0.0770 0.0645 0.0523 0.0404 0.0287 0.01 72 0.0057

46 0.3830 0.2635 0.2302 0.2058 0.1862 0.1695 0.1548 0.1415 0.1293 0.1 180 0.1073 0.0972 0.0876 0.0783 0.0694 0.0607 0.0522 0.0439 0.0357 0.0277 0.0197 0.01 18 0.0039

37 0.4040 0.2794 0.2403 0.21 16 0.1883 0.1683 0.1505 0.1344 0.1 196 0.1056 0.0924 0.0798 0.0677 0.0559 0.0444 0.0331 0.0220 0.01 10 0.0000

47 0.3808 0.2620 0.2291 0.2052 0.1859 0.1695 0.1550 0.1420 0.1300 0.1 189 0.1085 0.0986 0.0892 0.0801 0.071 3 0.0628 0.0546 0.0465 0.0385 0.0307 0.0229 0.0153 0.0076 0.0000


38 0.401 5 0.2774 0.2391 0.21 10 0.1881 0.1686 0.1513 0.1356 0.121 1 0.1075 0.0947 0.0824 0.0706 0.0592 0.0481 0.0372 0.0264 0.0158 0.0053

48 0.3789 0.2604 0.2281 0.2045 0.1855 0.1693 0.1551 0.1423 0.1306 0.1 197 0.1095 0.0998 0.0906 0.081 7 0.0731 0.0648 0.0568 0.0489 0.041 1 0.0335 0.0259 0.0185 0.01 11 0.0037

-

39 0.3989 0.2755 0.2380 0.21 04 0.1880 0.1689 0.1520 0.1366 0.1225 0.1092 0.0967 0.0848 0.0733 0.0622 0.051 5 0.4090 0.0305 0.0203 0.0101 0.0000

49 0.3770 0.2589 0.2271 0.2038 0.1851 0.1692 0.1553 0.1427 0.1312 0.1205 0.1105 0.1010 0.091 9 0.0832 0.0748 0.0667 0.0588 0.051 1 0.0436 0.0361 0.0288 0.0215 0.0143 0.0071 0.0000

40 0.3964 0.2737 0.2368 0.2098 0.1878 0.1691 0.1526 0.1376 0.1237 0.1 108 0.0986 0.0870 0.0759 0.0651 0.0546 0.0444 0.0343 0.0244 0.0146 0.0049

50 0.3751 0.2574 0.2260 0.2032 0.1847 0.1691 0.1554 0.1430 0.1317 0.1212 0.1 113 0.1020 0.0932 0.0846 0.0764 0.0685 0.0608 0.0532 0.0459 0.0386 0.031 4 0.0244 0.01 74 0.01 04 0.0035

000935

f 5644

FEMp Background Study May 1994

Table F-3 Quantiles of the Shapiro-Wdk W Test for Normality

(Values of W Such That 100 p% of the Distribution of W Is Less Than W,,)

n

3 4 5 6 7 8 9 10 11 12 13 14 1s 16 17 18 19 20 21 22 23 24 25 24 27 28 29 30 31 32 33 34 35 36 37 38 3? 40 41 42 43 44 45 46 47 48 49 50

wo 01

0.753 0.687

' -. 0.686

0.m 0.749 0.764 0.781 0.792 0.805 0.814 0.825 0.835 0.844 O S 1 0.858 0.863 0.868 0.873 0.878 0.881 0.884 0.886 0.891 0.894 0.8% 0.898 0.900 0.902 0.904 0.906 0.908 0.910 0.912 0.914 0.916 0.917 0.919 0.920 0.922 0.923 0.924 0.926 0.927 0.928 0.929 0.929 0.930

0.713

wo 0.2

0.7% 0.707 0.715 0.743 0.760 0.78 0.791 0.806 0.817 0.828 0.837 0.846 O S 5 0.863 0.869 0.874 0.879 0.884 0.888 0.892 0.8% 0.898 0.901 0.904 0.906 0.908 0.910 0.912 0.914 0.91s 0.917 0.919 0.920 0.922 0.924 0.925 0.927 0.928 0.929 0.930 0.932 0.63 0.934 0.935 0.936 0.937 0.937 0.938

Wo 05

0.767 0.748 0.762 0.788 0.803 0.818 0.829 0.842 0.850 0.859 0.866 0.874 0.881 0.887 0.892 0.897 0.901

, 0.905 0.908 0.911 0.914 0.916 0.918 0.920 0.923 0.924 0.926 0.927 0.929 0.930 0.931 0.933 0.934 0.935 0.936 0.938 0.939 0.940 0.941 0.942 0.943 0.944 0.945 0.945 0.946 0.947 0.947 0.947

wo 0.789 0.792 0.806 0.826 0.838 0.851 0.859 0.869 0.876 0.883 0.889 0.895 0.901 0.906 0.910 0.914 0.917 0.920 0.923 0.926 0.928 0.930 0.931 0.933 0.935 0.936 0.937 0.939 0.940 0.941 0.942 0.943 0.944 0.945 -

0.946 0.947 0.948 0.949 0.950 0.951 0.951 0.952 0.953 0.953 0.954 0.954 0.955 0.955

woso 0.959 0.935 0.927 0.927 0.928 0.932 0.935 0.938 0.940 0.943 0.945 0.947 0.950 0.952 0.954 0.956 0.957 0.959 0.960 0.961 0.%2 0.963 0.964 0.965 0.965 0.966 0.966 0.967 0.967 0.968 0.968 0.969 0.969 0.970 0.970 0.971 0.971 0.971 0.972 0.972 0.973 0.973 0.973 0.974 0.974 0.974 0.974 0.974

Source: Table A-7, Gilbcrt 1987

F-9 000934

. , '.. i. ', 1


Example:

To illustrate the application of the Shapiro-Wilk test, the data in Table F-4 is used.

1. Compute d, the denominator of the W test.

2 n n

i= l

1 36

= 0.0808 - - (1.634)*

= 0.0066

2. Order data points from low to high.

3. Compute k for n = 36

4. Find coefficients a,, a,, ..., aU for n = 36 from Table F-2.

a, = 0.4068 a, = 0.1678 a,, = 0.0900 a16 = 0.0287 a, = 0.2813 a, = 0.1496 a,, = 0.0776' a,, = 0.0172 a, = 0.2415 a, = 0.1331 a13 = 0.0645 a18 = 0.0057 a, = 0.2121 a, = 0.1179 al, = 0.0523 a, = 0.1883 a,, = 0.1036 aU = 0.0404

000935

. 5644

Chemical Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Barium Total


Table F-4 Example Data Set Number 1

Validated result,

0.034 0.049 0.05 0.05 0.05 0.05 0.064 0.035 0.035 0.039 0.04 0.04 0.04 0.04 0.043 0.044 0.044 0.045 0.045 0.047 0.048 0.049 0.05 0.051 4 0.052 0.054 0.055 0.057 0.06 0.06 0.061 0.061 0.062 0.064 0.066 0.073

mg/L

Concentration used Midation qualifier UJ U U U U U U

J J

J

J

J J J

J

J

in stat1 Normal 0.01 7 0.0245 0.025 0.025 0.025 0.025 0.032 0.035 0.035 0.039 0.04 0.04 0.04 0.04 0.043 0.044 0.044 0.045 0.045 0.047 0.048 0.049 0.05 0.051 4 0.052 0.054 0.055 0.057 0.06 0.06 0.061 0.061 0.062 0.064 0.066 0.073 1.634

1 Lognormal -4.075 -3.709 -3.609 -3.689 -3.609 -3.689 -3.442 -3.352 -3.352 -3.244 -3.219 -3.21 9 -3.219 -3.219 -3.147 -3.124 -3.124 -3.101 -3.101 -3.058 -3.037 -3.01 6 -2.996 -2.968 -2.957 -2.91 9 -2.900 -2.865 -2.81 3 -2.81 3 -2.797 -2.797 -2.781 -2.749 -2.71 8 -2.61 7

-1 13.201

000936 F-1 1


5. Compute W.

1 d

= - [0.4068 (0.073 - 0.017) + 0.2813 (0.066 - 0.0245) + 0.2415 (0.064 - 0.025)

+ 0.2121 (0.062 - 0.025) + 0.1883 (0.061 - 0.025) + 0.1678 (0.061 - 0.025)

+ 0.1496 (0.060 - 0.032) + 0.1331 (0.06 - 0.035) + 0.1179 (0.057 - 0.035)

+ 0.1036 (0.055 - 0.039) + 0.0900 (0.054 - 0.04) + 0.0770 (0.052 - 0.04)

+ 0.0645 (0.0514 - 0.040) + 0.0523 (0.050 - 0.040) + 0.0404 (0.049 - 0.043)

+ 0.0287 (0.048 - 0.044) + 0.0172 (0.047 - 0.044) + 0.0057 (0.045 - 0.045)12

- (0.080)2

= 0.971

- 0.0066

6. Fail to reject H, at the 0.05 significance level because W,, = 0.971 is greater than Wcritical of 0.935, the quantile given in Table F-3, and conclude that the data were drawn from a population with an underlying normal distribution.

The W test was repeated after the transformation of yi = In resulted in W,,. = 0.928. Because the calculated value is less than the critical value (WrritiFl = 0.935), there is sufficient evidence to reject the null hypothesis. The conclusion is that the data set was not drawn from a population having an underlying lognormal distribution.

When the W test fails to reject the null hypotheses for both the normal and lognormal distributions, the W,,, value which exceeds the Wrritid value the most is selected as the distribution for the data set. When the .W test rejects the null hypotheses for both the

5644 FEMP Background Study

May 1994

normal and lognormal distribution but one is close (W,,, > 0.95 Wcritia,) to the test value, then this distribution (normal or lognormal) is selected. If both the normal and lognormal W,,, values are not close to Wcritia,, then the distribution is considered to be undefined and nonparametric statistical analyses will be used to described the data set.

F-13 000938

'. .

. '. , .. : .

a FEW Background Study May 1994

Shapiro-Francia Test for Normality

The W test developec by Shapiro and Wilk was used to determine u,ether or not a data set has been drawn from a population which is normally distributed for sample size of 50 or less while the Shapiro-Francia test (Shapiro and Francia, 1972) was used when the sample size was greater than 50.

Like the Shapiro-Wilk test, the Shapiro-Francia test statistic (Wl) can be calculated using the natural logarithm of each data value. This is used to determine whether or not the sample was drawn from an underlying lognormal distribution. The null hypothesis to be tested is:

H,: The population has normal (lognormal when data is transformed) distribution.

versus

HA: The population does not have a normal (lognormal when data is transformed) distribution.

If H, is rejected, then HA is accepted.

To calculate the test statistic one can use the following formula:

where represents the ith ordered value of the sample and where m, denotes the approximate expected value of the ith ordered normal quantile. The value for m, can be approximately computed as:

F-14

5644


where @-' denotes the inverse of the standard normal distribution with zero mean and unit variance. These values can be computed by hand using a normal probability table (Table F-5) or via simple commands in many statistical computer packages.

1. Order the n data points in assending order (smallest to largest) such that x1 <x2 < x3 < ... e%. a, - - - -

2. Computer [i/(n+ l)] when n is the total number of samples and i is the ith data point.

3. Compute the normal quantile (mi).

4. Compute the square of the normal quantile.

5. Compute the multiplication of the normal quantile times the corresponding data value.

6. Compute the arithmetic standard deviation.

n

i=l c (3 - 3 2

n-1 s =

where:

7. Compute the denominator of Shapiro-Francia Test.

8. Compute the numerator of the Shapiro-Francia Test.

9. Compute W'.

10. Reject H, at the Q significance level is W I is less than the quantile given in Table F-6.

F-15


z -3.4 -3.3 -3.2 -3.1 -3.0

-2.9 -2.8 -2.7 -2.6 -2.5

-2.4 -2.3 -2.2 -2.1 -2.0

-1.9 -1.8 -1.7 -1.6 -1.5

-1.4 -1.3 -1.2 -1.1 -1.0

-0.9 -0.8 -0.7 -0.6 -0.5

-0.4 -0.3 -0.2 -0.1 -0.0

0.0 0.1 0.2 0.3 0.4

0.5 0.6 0.7 0.8 0.9

p

Table F-5 Standard Normal Curve for a 2 Distribution

0

0.0003 0.0005 0.0007 0.0010 0.0013

0.0019 0.0026 0.0035 0.0047 0.0062

0.0082 0.0107 0.0139 0.0179 0.0228

0.0287 0.0359 0.0446 0.0548 0.0668

0.0808 0.0968 0.1151 0.1357 0.1587

0.1841 0.2119 0.2420 0.2743 0.3085

0.3446 0.3821 0.4207 0.4602 0.5000

0.5000 0.5398 0.5793 0.6179 0.6554

0.6915 0.7257 0.7580 0.7881 0.8159

.p;”* < .

0.01

0.0003 0.0005 0.0007 0.0009 0.001 3

0.001 8 0.0025 0.0034 0.0045 0.0060

0.0080 0.01 04 0.01 36 0.01 74 0.0222

0.0281 0.0351 0.0436 0.0537 0.0655

0.0793 0.0951 0.1 131 0.1335 0.1562

0.1814 0.2090 0.2389 0.2709 0.3050

0.3409 0.3783 0.4168 0.4562 0.4960

0.5040 0.5438 0.5832 0.621 7 0.6591

0.6950 0.7291 0.761 1 0.7910 0.81 86 ,

CI N/OU5RI /TABF5.XLS/5-94

0.02

0.0003 0.0005 0.0006 0.0009 0.0013

0.001 8 0.0024 0.0033 0.0044 0.0059

0.0078 0.01 02 0.0132 0.01 70 0.021 7

0.0274 0.0344 0.0427 0.0526 0.0643

0.0778 0.0934 0.1112 0.1314 0.1539

0.1788 0.2061 0.2358 0.2676 0.3015

0.3372 0.3745 0.41 29 0.4522 0.4920

0.5080 0.5478 0.5871 0.6255 0.6628

0.6985 0.7324 0.7642 0.7939 0.821 2

0.03

0.0003 0.0004 0.0006 0.0009 0.001 2

0.001 7 0.0023 0.0032 0.0043 0.0057

0.0075 0.0099 0.01 29 0.01 66 0.021 2

0.0268 0.0336 0.041 8 0.0516 0.0630

0.0764 0.091 8 0.1093 0.1292 0.1515

0.1 762 0.2033 0.2327 0.2643 0.2981

0.3336 0.3707 0.4090 0.4483 0.4880 0.51 20 0.551 7 0.591 0 0.6293 0.6664

0.7019 0.7357 0.7673 0.7967 0.8238

0.04

0.0003 O.OOO4 0.0006 0.0008 0.0012

0.001 6 0.0023 0.0031 0.0041 0.0055

0.0073 0.0096 0.01 25 0.01 62 0.0207

0.0262 0.0329 0.0409 0.0505 0.061 8

0.0749 0.0901 0.1075 0.1271 0.1492

0.1736 0.2005 0.2296 0.261 1 0.2946

0.3300 0.3669 0.4052 0.4443 0.4840 0.5160 0.5557 0.5948 0.6331 0.6700

0.7054 0.7389 0.7704 0.7995 0.8264

F-16

0.05

0.0003 0.0004 O.OOO6 0.0008 0.001 1

0.001 6 0.0022 0.0030 0.0040 0.0054

0.0071 0.0094 0.01 22 0.01 58 0.0202

0.0256 0.0322 0.0401 0.0495 0.0606

0.0735 0.0885 0.1056 0.1251 0.1469

0.1711 0.1977 0.2266 0.2578 0.291 2

0.3264 0.3632 0.401 3 0.4404 0.4801

0.5199 0.5596 0.5987 0.6368 0.6736

0.7088 0.7422 0.7734 0.8023 0.8289

0.06

0.0003 0.0004 0.0006 0.0008 0.001 1

0.001 5 0.0021 0.0029 0.0039 0.0052

0.0069 0.0091 0.01 19 0.0154 0.0197

0.0250 0.031 4 0.0392 0.0485 0.0594

0.0721 0.0869 0.1038 0.1230 0.1446

0.1685 0.1949 0.2236 0.2546 0.2877

0.3228 0.3594 0.3974 0.4364 0.4761

0.5239 0.5636 0.6026 0.6406 0.6772

0.7123 0.7454 0.7764 0.8051 0.8315

0.07

0.0003 0.0004 0.0005 0.0008 0.001 1

0.0015 0.0021 0.0028 0.0038 0.0051

0.0068 0.0089 0.01 16 0.0150 0.01 92

0.0244 0.0307 0.0384 0.0475 0.0582

0.0708 0.0853 0.1020 0.1210 0.1423

0.1660 0.1922 0.2206 0.251 4 0.2843

0.31 92 0.3557 0.3936 0.4325 0.4721

0.5279 0.5675 0.6064 0.6443 0.6808

0.7157 0.7486 0.7794 0.8078 0.8340

0.08

0.0003 0.0004 0.0005 0.0007 0.0010

0.0014 0.0020 0.0027 0.0037 0.0049

0.0066 0.0087 0.01 13 0.0146 0.0188

0.0239 0.0301 0.0375 0.0465 0.0571

0.0694 0.0838 0.1003 0.1 190 0.1401

0.1635 0.1894 0.21 77 0.2483 0.281 0

0.31 56 0.3520 0.3897 0.4286 0.4681

0.5319 0.5714 0.61 03 0.6480 0.6844

0.7190 0.751 7 0.7823 0.81 06 0.8365

000941

0.09

0.0002 0.0003 0.0005 0.0007 0.0010

0.001 4 0.001 9 0.0026 0.0036 0.0048

0.0064 0.0084 0.01 10 0.0143 0.01 83

0.0233 0.0294 0.0367 0.0455 0.0559

0.0681 0.0823 0.0985 0.1170 0.1379

0.161 1 0.1867 0.2148 0.2451 0.2776

0.3121 0.3483 0.3859 0.4247 0.4641

05359 0.5753 0.61 41 0.651 7 0.6879

0.7224 0.7549 0.7852 0.81 33 0.8389

- Z 1 .o 1.1 1.2 1.3 1.4

1.5 1.6 1.7 1.8 1.9

2.0 2.1 2.2 2.3 2.4

2.5 2.6 2.7 2.8 2.9

3.0 3.1 3.2 3.3 3.4

-

-

-

-

-

-

0.04

0.8508 0.8729 0.8925 0.9099 0.9251

0.9382 0.9495 0.9591 0.9671 0.9738

0.9793 0.9838 0.9875 0.9904 0.9927

0.9945 0.9959 0.9969 0.9977 0.9984

0.9988 0.9992 0.9994 0.9996 0.9997

0 0.8413 0.8643 0.8849 0.9032 0.91 92

0.9332 0.9452 0.9554 0.9641 0.971 3

0.9772 0.9821 0.9861 0.9893 0.991 8

0.9938 0.9953 0.9965 0.9974 0.9981

0.9987 0.9990 0.9993 0.9995 0.9997

0.05

0.8531 0.8749 0.8944 0.9115 0.9265

0.9394 0.9505 0.9599 0.9678 0.9744

0.9798 0.9842 0.9878 0.9906 0.9929

0.9946 0.9960 0.9970 0.9978 0.9984

0.9989 0.9992 0.9994 0.9996 0.9997

pr 0.8438 0.8665 0.8869 0.9049

~ 0.9207

0.9345 0.9463 0.9564

0.9778 0.9826

0.9896 0.9920

0.9955 0.9966 0.9975 0.9982

0.9987 0.9991 0.9993 0.9995 0.9997

0.06

0.8554 0.8770 0.8962 0.91 31 0.9279

0.9406 0.9515 0.9608 0.9686 0.9750

0.9803 0.9846 0.9881 0.9909 0.9931

0.9948 0.9961 0.9971 0.9979 0.9985

0.9989 0.9992 0.9994 0.9996 0.9997


0.07

0.8577 0.8790 0.8980 0.91 47 0.9292

0.9418 0.9525 0.9616 0.9693 0.9756

0.9808 0.9850 0.9884 0.9911 0.9932

0.9949 0.9962 0.9972 0.9979 0.9985

0.9989 0.9992 0.9995 0.9996 0.9997

Stands

0.02

0.8461 0.8686 0.8888 0.9066 0.9222

0.9357 0.9474 0.9573 0.9656 0.9726

0.9783 0.9830 0.9868 0.9898 0.9922

0.9941 0.9956 0.9967 0.9976 0.9982

0.9987 0.9991 0.9994 0.9995 0.9997

Tablc rd Nom

0.03

0.8485 0.8708 0.8907 0.9082 0.9236

0.9370 0.9484 0.9582 0.9664 0.9732

0.9788 0.9834 0.9871 0.9901 0.9925

0.9943 0.9957 0.9968 0.9977 0.9983

0.9988 0.9991 0.- 0.9996 0.9997

- -

-

-

-

-

-

F-5 (Continued) I Curve for a 2 Distribution

0.08

0.8599 0.881 0 0.0997 0.9162 0.9306

0.9429 0.9535 0.9625 0.9699 0.9761

0.981 2 0.9854 0.9887 0.9913 0.9934

0.9951 0.9963 0.9973 0.9980 0.9986

0.9990 0.9993 0.9995 0.9996 0.9997

0.09

0.8621 0.8830 0.901 5 0.9177 0.931 9

0.9441 0.9545 0.9633 0.9706 0.9767

0.981 7 0.9857 0.9890 0.991 6 0.9936

0.9952 0.9964 0.9974 0.9981 0.9986

0.9990 0.9993 0.9995 0.9997 0.9998

Source: Devore, J.L., 1982. "Probability & Statistics for Engineering and the Sciences", Table A-3, Brooks/Cole Publishing Company, Monterey, CA.

. _

ClN/OU5Rl/rABFSlX~S/5-94

000942

F-17

FEMP Background Study May1994

TABLE F-6

PERCENTAGE POINTS OF THE WI TEST FOR n > 50

~ ~~

n 0.01 0.05

51 53 55 57 59 61 63 65 67 69 71 73 75 77 79

81 83 85 87 89 91 93 95 97 99

0.935 0.938 0.940 0.944 0.945 0.947 0.947 0.948 0.950 0.95 1

0.953 0.956 0.956 0.957 0.957

0.958 0.960 0.96 1 0.96 1 0.961 0.962 0.963 0.965 0.965 0.967

0.954 0.957 0.958 0.96 1 0.962

0.963 0.964 0.965 0.966 0.966 0.967 0.968 0.969 0.969 0.970

0.970 0.97 1 0.972 0.972 0.972 0.973 0.973 0.974 0.975 0.976

~~~ ~~

Source: Table A-3, U.S. EPA 1992.

CIN/OUSRI/WP/419195/TableF-7/5-94

. . ‘ I . . - . ”* I

F-18

000943


To test the null hypothesis that the population has a lognormal distribution, transform the observed data to yi, y2...yn where yi = In xi. Repeat steps 1 through 10 as described above.

Example:

To illustrate the application of the Shapiro-Francia test, the data in Table F-7 is used.

1. Order the n data points in ascending order (smallest to largest) as in Column 2.

2. Compute (i/(n+ 1)) where n is the total number of samples (n = 83) which is presented in Column 3.

3. Compute the normal quantile (q) corresponding to Column 3 probabilities. The normal quantiles are presented in Column 4.

4. Compute the square of the normal quantiles which are presented in Column 5.

5. Compute the multiplication of the normal quantiles (Column 4) times the corresponding data point (Column 2). These results are presented in Column 6.

6. Compute the arithmetic standard deviation.

s = 0.019 (See arithmetic standard deviation example for the procedure for calculation of this value; note that the data sets are not the same.)

7. Compute the denominator of the Shapiro-Francia Test.

n

i = l (n-1) s2 rnf = 82 (0.019)* (75.611)

= 2.238

F-19 000944

FEMP Background Study May 1994 0

8. Compute the numerator of the Shapiro-Francia Test.

9. Compute W*. 0.55 1 w' = - 2.238

= 0.246

10. Reject H, at the 0.05 significance level because W I = 0.246 is less than Weritifal of 0.985, the quantile given in Table F-6, and concluded that the data were not drawn from a population with an underlying normal distribution.

The Shapiro-Francia Test was repeated after the transformation of yi = In W I test 7 0.659. Because the calculated value is less than the critical value (Wcritica, = 0.985), it is concluded that the data set was not drawn from a population having an underlying lognormal distribution.

resulted in

When the W I test fails to reject the null hypotheses for both the normal and lognormal distributions, the W test value which exceeds the W I critifal value the most is selected as the distribution for the data set. When the W test rejects the null hypotheses for both the normal and lognormal distribution but one is close (W I > 0.95 W Icritical) to the test value, then this distribution is (normal or lognormal) selected. If both the normal and lognormal W test are not close to W I critical, then the distribution is considered to be undefined and nonparametric statistical analyses will be used to describe the data set.

F-20

Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

-


- xi

0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001

-

CI N/OU5R1/419195/rABF6.X~S/5-~

(i/(n+ 1)) 0.01 2 0.024 0.036 0.048 0.060 0.071 0.083 0.095 0.107 0.119 0.131 0.143 0.155 0.167 0.179 0.190 0.202 0.214 0.226 0.238 0.250 0.262 0.274 0.286 0.298 0.31 0 0.321 0.333 0.345 0.357 0.369 0.381 0.393 0.405 0.41 7 0.429 0.440 0.452 0.464 0.476 0.488 0.500 0.512 0.524 0.536 0.548 0.560 0.571 0.583 0.595

Normal quantile, mi

-2.260 -1 981 -1.803 -1.668 -1.559 -1.465 -1 -383 -1.309 -1.242 -1.180 -1.122 -1 .om -1.016 -0.967 -0.921 -0.876 -0.833 -0.792 -0.751 -0.71 2 -0.674 -0.637 -0.601 -0.566 -0.531 -0.497 -0.464 -0.431 -0.398 -0.366 -0.334 -0.303 -0.272 -0.241 -0.210 -0.180 -0.150 -0.120 -0.090 -0.060 -0.030 0.000 0.030 0.060 0.090 0.120 0.150 0.180 0.21 0 0.241

F-21

mi-2 5.108 3.923 3.250 2.784 2.430 2.147 1.913 1.714 1.542 1.392 1.259 1.140 1.033 0.936 0.848 0.768 0.694 0.627 0.565 0.508 0.455 0.406 0.362 0.320 0.282 0.247 0.215 0.186 0.159 0.134 0.1 12 0.092 0.074 0.058 0.044 0.032 0.022 0.014 0.008 0.004 0.001 0.000 0.001 0.004 0.008 0.014 0.022 0.032 0.044 0.058


mi*xi -0.002 -0.002 -0.002 -0.002 -0.002 -0.001 -0.001 -0.001 -0.001 -0.001 4.001 -0.001 4.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 -0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

000946


- Rank 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83

Total

-

- -

xi 0.001 0.001 0.001 0.001 0.001 4 0.0015 0.001 5 0.0016 0.002 0.002 0.0025 0.0025 0.0025 0.0025 0.0026 0.003 0.004 0.004 0.004 0.0044 0.005 0.005 0.005 0.005 0.006 0.006 0.009 0.01 1 0.028 0.029 0.06 0.08 0.14

-

-

Table F-7 (Continued) Example Data Set Number 2

(i/(n+ 1)) 0.607 0.61 9 0.631 0.643 0.655 0.667 0.679 0.690 0.702 0.71 4 0.726 0.738 0.750 0.762 0.774 0.786 0.798 0.810 0.821 0.833 0.845 0.857 0.869 0.881 0.893 0.905 0.91 7 0.929 0.940 0.952 0.964 0.976 0.988

Normal quantile, mi

0.272 0.303 0.334 0.366 0.398 0.431 0.464 0.497 0.531 0.566 0.601 0.637 0.674 0.712 0.751 0.792 0.833 0.876 0.921 0.967 1.016 1.068 1.122 1.180 1.242 1.309 1.383 1.465 1.559 1.668 1 .a03 1.981 2.260

F-22

____

mi-2 0.074 0.092 0.112 0.134 0.159 0.186 0.215 0.247 0.282 0.320 0.362 0.406 0.455 0.508 0.565 0.627 0.694 0.768 0.848 0.936 1.033 1.140 1.259 1.392 1.542 1.714 1.913 2.147 2.430 2.784 3.250 3.923 5.108 75.61 1

ml*xi 0.000 0.000 0.000 0.000 0.001 0.001 0.001 0.001 0.001 0.001 0.002 0.002 0.002 0.002 0.002 0.002 0.003 0.004 0.004 0.004 0.005 0.005 0.006 0.006 0.007 0.008 0.012 0.016 0.044 0.048 0.108 0.158 0.31 6 0.742

000947

i


Rosner's Test for Many Outliers

To use Rosner's Test (Gilbert 1987, Equations 15.1 to 15.3) it is necessary to specify an upper limit of the number of potential outliers present. This analysis was performed for up to ten outliers. Rosner's Test requires the calculation of a test statistic Ri+l using the following equation:

where:

Ri+l = test statistic for deciding whether the i + 1 most extreme values in the complete data set are statistical outliers

i = 0 for the first suspected outlier

i = 1 for the second suspected outlier

i = 9 for the tenth suspected outlier

= sample arithmetic mean of the remaining data set after the i most extreme observations have been deleted

F-23

000948


s(') = sample standard deviation of the remaining data set after the i most extreme observations have been deleted

and 5 = jth observation in the data set

n = total number of observations in the data set

A suspected extreme value is determined to be an outlier if the calculated value of Ri+l exceeds the critical value Ai+l for a sample of size n (Table F-8).

In applying Rosner's Test when there is only one suspected outlier, i = 0 and

and x(O) = suspected outlier

2') = sample arithmetic mean of the n observations including the suspected outlier

s(O) = sample arithmetic standard deviation of the n observations including the suspected outlier

F-24

564% FEMP Background Study

May 1994

Table F-6 Approximate Critical Values Lamda (i+ 1) for Rosnets Generalized

ESD Many-Outlier Procedure for alpha = 0.05

n 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 60 70 80 90 100 150 200 250 300 350 400 450 500 750 1000 2000 3000 4000 5000

1 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.1 1 3.12 3.13 3.20 3.26 3.31 3.35 3.38 3.52 3.61 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41

2 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.1 1 3.12 3.19 3.25 3.30 3.34 3.38 3.51 3.60 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41

Source: Table A-1 6, Gilbert 1987.

3 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.1 1 3.19 3.25 3.30 3.34 3.38 3.51 3.60 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41

4 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.10 3.18 3.24 3.29 3.34 3.37 3.51 3.60 3.67 3.72 3.77 3.80 3.84 3.86 3.95 4.02 4.20 4.29 4.36 4.41

5 2.73 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.09 3.17 3.24 3.29 3.33 3.37 3.5i 3.60 3.67 3.72 3.76 3.80 3.83 3.86 3.95 4.02 4.20 4.29 4.36 4.41

10 2.59 2.62 2.65

, 2.68 2.71 2.73 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.89 2.91 2.92 2.94 2.95 2.97 2.98 2.99 3.00 3.01 3.03 3.04 3.05 3.14 3.21 3.26 3.31 3.35 3.50 3.59 3.66 3.71 3.76 3.80 3.83 3.86 3.95 4.02 4.20 4.29 4.36 4.41

CIN/OU5RI/419195/rABF8.XLS/5-94

1 *.<,. I . - .

F-25


When Rosner's Test is applied in a situation where there are two suspectd outliers, i= 1 and

and

x(') = the second suspected outlier

3') = the sample arithmetic mean after the first suspected outlier x(O) has been deleted from the data set

dl) = the sample arithmetic standard deviation after the first suspected outlier x(O) has been deleted from the data set

Example:

Table F-9 presents example data set number 3 which is used for this example. The highest and lowest posted concentrations of 4850 pg/L and 189 pg/L, respectively, are considered to be potential outliers. To begin with, Rosner's Test is applied to determine whether or not the value of 4850 pg/L is a statistical outlier. In this case, i = 0 and

The da I are best described by a lognormal distribution and as such

where:

Y'O) = x(o)

= In (4850)

= 8.487

F-26

. . . J

000959

FEMP B a c k g r o R g t

Chemical Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese Manganese


Validated result, ug/L 189 301 351 370 386 422 437 451 456 481 488 521 534 535 543 581 61 5 61 9 747 766 785 840 941 1050 1070 1090 1150 1460 1500 4850

Validation qualifier

J

J J

J

J

Concentration used in stat

Normal 189 30 1 35 1 370 386 422 437 451 456 481 488 521 534 535 543 581 61 5 61 9 747 766 785 840 941 1050 1070 1090 1150 1460 1500 4850

.its (a) Lognormal

5.242 5.707 5.861 5.914 5.956 6.045 6.080 6.1 11 6.122 6.176 6.190 6.256 6.280 6.282 6.297 6.365 6.422 6.428 6.61 6 6.641 6.666 6.733 6.847 6.957 6.975 6.994 7.048 7.286 7.31 3 8.487

(a) When the validation qualifier contains a 'U", then one-half of the concentration is used in the statistical calculations.

F-27


To) = sample arithmetic mean of In x using all 30 observations in the data set

= 6.477

s(O) = sample arithmetic standard deviation of In x using all 30 observations in the data set

= 0.6098

Thus

18.487 - 6.477) 0.6098

R, =

= 3.296

Because R, = 3.296 is greater than the critical value of Ai+, = 2.91 (Table F-8) for a significance level of 0.05, Rosner's Test indicates that the observed value of 4850 pg/L is a statistical outlier.

Next Rosner's Test is used to determine whether or not there is sufficient evidence to conclude that the minimum value of 189 pg/L is a statistical outlier.

After deleting the maximum value of 4850 pg/L from the data set, calculate

where:

y") = In (189)

= 5.242

yl) = 6.407

s(l) = 0.4889

i 5-64.4

FEMF' Background Study May 1994

Then

15.242 - 6.4071 R, = 0.489

= 2.38

Because R, = 2.38 is less than the critical value of 1, = 2.89 (Table F-8) for a significance level of 0.05, Rosner's Test indicates that there is insufficient evidence to reject the observed value of 189 pg/L. The value is not a statistical outlier. This test is repeated on the high and low ends until there is insufficient evidence to reject the null hypothesis or 10 outliers are identified. This test identifies outliers but professional judgment was used to determine whether the value should be removed from the data set.

F-29 .

000954

EMF' Background Study May 1994

Data Averaging

Data averaging was conducted when two or more samples were collected at the same sampling location on the same day (Le., duplicates, triplicates, etc.). Multiple samples collected on a particular day and location were averaged to obtain one sample per day at the sampling location. Data averaging was conducted to avoid statistical bias which would have resulted from favoring one day's multiple sampling over one sample on a particular day. The data was separated into data groups by the following: media (Le., Glacial Overburden, Great Miami Aquifer, Great Miami River, and Paddys Run), constituent type (Le., radiological, inorganic, and organic constituents), and filter type (Le., filtered and unfiltered). The following steps were repeated for each of the above data groups:

1. Nondetect values were assigned a value of one-half the detection limit.

2. The data were sorted by three key parameters: 1) chemical name, 2) sample location, and 3) sampling date.

3. A record-by-record comparison was performed to identify records for which all three sort parameters were the same. This step identified multiple samples collected on a particular day for a specific constituent.

4. A sample arithmetic mean was calculated on the records with matching key parameters.

5. If all of the values being averaged were nondetects, the average was also a nondetect. If one or more of the values being averaged were detect values, then the resulting average was also considered to be a detect value.

6. A new line was added to the database for the averages of the matching records.

7. The individual records that were combined to create the average line were removed from the statistical data set.

Example:

The example data set is presented in Table F-10. To simplify this example, only one constituent was selected. Seven ammonia samples were collected from 1988 to 1993. The following steps were performed to calculate the averaged values that were used in the statistical analysis:

1. Nondetect values were assigned a value of one-half the detection limit. This occurred for five of the seven values (i.e., c 1 equals 0.05).

CrN/OUS~/wp/4191%/AppENDIxF/s-W F-30 i A ' , > L .. , 000955


05/20/93 06/23/93 06/23/93 08/29/88 04/03/89 05/20/93 05/20/93 05/20/93 06/23/93 06/23/93 05/20/93 05/20/93 06/23/93


120072-2 12041 6 12041 4 1092 1178 120064-2 120068-1 120072-2 12041 6 120414 120068-1 120072-2 12041 6

Well No. w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1 w-1

0.1 1 0.1 . 0.1 0.1 0.1

Sample

N UJ N UJ D UJ T U N

05/20/93 120072-2 06/23/93 I 12041 6

0.002 0.0039 0.0032 0.0025

U N D T

J N

04/03/89 05/20/93 05/20/93 06/23/93

08/29/88 04/03/89 05/20/93 05/20/93 06/23/93

1178 120068-1 120072-2 12041 6

1092 1178 120068-1 120072-2 12041 6

0.1 0.0493 0.0884 0.0893 0.0906

05/20/93 120072-2 06/23/93 I 12041 6

N N D T N

0.002 0.0098 0.005 0.005 0.005 0.005

77 70.1 61.2 76.5 77.1 72.3 68.4

Lab qualifier

U N N

U D U T U N U D

N N N D T N

- . D

U U U U U U U

uw uw U U B B

BW U

B B B B U U U U

U

U U U U

08/29/88 04/03/89 05/20/93 05/20/93 06/23/93

Constituent Alkalinitv as CaC03

1092 1178 120068-1 120072-2 12041 6

Alkalinitjl as Alkalinity as CaC03 Alkalinity as CaC03 Aluminum Aluminum Aluminum Aluminum Ammonia Ammonia Ammonia Ammonia Ammonia Ammonia Ammonia Antimony Antimony Antimony Antimony Arsenic Arsenic Arsenic Arsenic Arsenic Arsenic Barium Barium Barium Barium Barium Barium Barium Beryllium Beryllium Beryllium Beryllium Cadmium Cadmium Cadmium Cadmium Cadmium Cadmium Cadmium Calcium Calcium Calcium Calcium Calcium Calcium Calcium

F-31

233 I:, 230 I I D 1.27 I I D

2.14 1-33 I 1 I:, 1.64 I I D 0.1 I J I N

0.1 I U I D 0.005 I U I D

0.005 I:, 0.005 1 UJ I D 0.005 I U I N

0.002 I U I D 0.089 I I N

0.0847 1 I D 0.002 I U I D

0.002 O*Oo2 I u I:, 0.002 I U I D 0.006 I I N

- 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3

-

-

-

-

-

-

-

-

-

000956

2.

3.

4.

5.

6.

7 .


The data were sorted by three key parameters: 1) chemical name (ammonia), 2) sample location (W-1), and 3) sampling date. There only was one sampling location for W-1 and W-5; however, multiple sampling locations (wells) were sampled for the glacial overburden and Great Miami Aquifer (5 and 24, respectively).

A record-by-record comparison identified the following matches: three samples collected on May 20, 1993 and two samples collected on June 23, 1993. Individual records were identified on August 29, 1988 and April 3, 1989.

All samples collected on May 20 and June 23, 1993, had nondetect values of ~ 0 . 1 (0.05 for statistics); therefore, the averaged value for these two days was a nondetect value of e 0.1.

The averaged values are both nondetects, since all of the data used to calculate the average value were nondetects.

New lines were added to the database for these two averaged records from the matching key parameters.

The five lines that were combined to create the averaged records were removed from the statistical database.

F-32

00095'7

F" Background Study May 1994

Sample Arithmetic Mean - Normal Distribution

The sample arithmetic mean for a normal distribution is given by:

Example:

Using example data set number 1 presented in Table F-4, the sample arithmetic mean is calculated as follows:

- X =

= 0.045

C0.017 + 0.0245 + .... + 0.066 + 0.073]/36

000958


Sample Arithmetic Standard Deviation - Normal Distribution

The sample arithmetic standard deviation for a population based on a sample is given by:

Example:

n

i=l c (3 - 3 2

- - n-1 4

2 n 4 2 - [ E 31 In

i=l i=l n-1

Using example data set number 1 presented in Table F-4, the sample arithmetic standard deviation for a normal distribution is calculated as follows where n = 36 and:

n

i=l xi2 = (0.017)2 + (0.0245)2 + ....+ (0.066)2 + (0.073)2

= 0.081

[ $ 3[ = [0.017 +0.0245 + .... + 0.066 + 0.07312 i=l

= 2.670

= 1 0.081 - 2.670136 36 - 1

= 0.014


Estimated Coefficient of Variation - Normal Distribution

The estimated coefficient of variation for a normal distribution is calculated by the following:

cv = sb

where: - x = arithmetic mean s = sample standard deviation CV = estimated coefficient of variation

Example:

Using example data set number 1 presented in Table F-4, the estimated coefficient of variation for a normal distribution is calculated as follows: e - x = 0.045

s = 0.014

CV = 0.014/0.045 = 0.31


Estimated Mean of a Lognormal Distribution

The estimated mean of a lognormal distribution (Gilbert 1987, Equation 13.7) can be calculated by using the sample mean (y) and the sample standard deviation (s,.) of the log-transformed data. The formula is as follows:

Example:

Using the data in Table F-4, the estimated mean of a lognormal distribution is calculated as follows where n = 36 and:

n

[ln(0.017) + In(0.0245) +.....+ ln(0.066) WX,) - i = l

Y = 36 n

- -113.201 - 36

= -3.144

+ =

n

n - 1 n - 1


F-37

n C p(%)p = h1(0.017)~ + ln(O.O245)* +....+ 1n(0.066)2 + ln(0.073)2

i = l

= 360.063

n

i = l p(+)] = ln(0.017) + ln(0.0245) +....+ ln(0.066) + ln(0.073)

= -113.201

360.063 - (-113.201)2/36 36-1

= 0.342

1 ( 0.342)2 2

= 0.046


Estimated Standard Deviation of a Lognormal Distribution

The estimated standard deviation of a lognormal distribution (Gilbert 1987, Equation 13.8) can be calculated by using the estimate for the mean of the lognormal distribution (ii) and the sample standard deviation (3) of the log-transformed data. The formula is as follows:

Example:

Data in Table F-4 were used to calculate the estimated standard deviation of a lognormal distribution. The parameters used in this example were presented in the example for estimating the mean of a lognormal distribution where 0 = 0.046 and 3 = 0.342. Therefore, the estimated standard deviation of a lognormal distribution is calculated as follows:

6 = (k0.046)2 [exp (0.342)2 - 11

= 0.0162

- .

963


May 1994

Sample Median - Nonparametric Technique

The true median of an underlying distribution (Gilbert 1989, Equations 13.15 and 13.16) is that value above which and below which half of the distribution lies. The median of any distriubtion, no matter what it's shape, can be estimated by the following:

Sample median = x [ ( ~ + ~ ) / ~ ~ if n is odd

where:

xpl = i* data value in the ordered data set

Example:

Using example data set number 1 presented in Table F-4, the sample median is calculated as follows where n = 36 and:

- - '18 + '19

2

= (0.045 + 0.045)/2

= 0.045

If n were 35 rather than 36, then the median would be at XI8 = 0.045.

000964 F-39


Upper One-sided 95% Confidence Limit - Normal Distribution

The mean el) for a sample of size n is referred to as a point estimate of the true but unknown population mean (p) . If a second sample of size n is drawn from the sample population, the sample mean &) will most likely not be equal toil. In fact if the sampling process is replicated many times, the sample means themselves will have a distribution. Further, the distribution of means of samples of size n will tend toward a normal distribution, if n is sufficiently large.

The 100 (1-a) Upper Confidence Limit (Gilbert 1987, Equation 11.6) of the population mean ( p ) can also be calculated. When a = 0.05, the upper one-sided 95 percent confidence limit is:

S 95% UCL, = x' + to*ss,-, - J;;

where:

toass a-l = value from the "t" distribution in Table F-11.

It should be noted that the 95 percent confidence limit for a second sample of size n drawn from the same population will most likely not be the same as that for the first sample. In theory if a limit estimate is calculated for the means of a very large set of samples of size n, the true population mean will be less than the limit 95 percent of these limits. If the number of degrees of freedom is not listed in Table F-11, then linear interpolation was performed to obtain the t-value.

F-40 000965

FEMP Backgro

Degrees of freedom

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 1 20

Infinite

Table F-11 Quantiles of the t Distribution (Values oft Such that loop%

of the Distribution Is Less Than tp)

t at 0.60 0.325 0.289 0.277 0.271 0.267 0.265 0.263 0.262 0.261 0.260 0.260 0.259 0.259 0.258 0.258 0.258 0.257 0.257 0.257 0.257 0.257 0.256 0.256 0.256 0.256 0.256 0.256 0.256 0.256 0.256 0.255 0.254 0.254 0.253

t at 0.70 0.727 0.61 7 0.584 0.569 0.559 0.553 0.549 0.546 0.543 0.542 0.540 0.539 0.538 0.537 0.536 0.535 0.534 0.534 0.533 0.533 0.532 0.532 0.532 0.531 0.531 0.531 0.531 0.530 0.530 0.530 0.529 0.527 0.526 0.524

jource: Table A-2, Gilbert 1987.

. , , : , > ” , ., . , -. ‘.I

ClN/OU5RI/WP/419195/rABFll .XLS/5-94

t at 0.80 1.376 1.061 0.978 0.941 0.920 0.906 0.896 0.889 0.883 0.879 0.876 0.873 0.870 0.868 0.866 0.865 0.863 0.862 0.861 0.860 0.859 0.858 0.858 0.857 0.856 0.856 0.855 0.855 0.854 0.854 0.851 0.848 0.845 0.842

t at 0.90 3.078 1.886 1.638 1.533 1.476 1.440 1.415 1.397 1.383 1.372 1.363 1.356 1.350 1.345 1.341 1.337 1.333 1.330 1.328 1.325 1.323 1.321 1.319 1.318 1.316 1.315 1.314 1.313 1.31 1 1.310 1.303 1.296 1.289 1.282

t at 0.95 6.31 4 2.920 2.353 2.132 2.015 1.943 1.895 1.860 1.833 1.812 1.796 1.782 1.771 1.761 1.753 1.746 1.740 1.734 1.729 1.725 1.721 1.71 7 1.714 1.71 1 1.708 1.706 1.703 1.701 1.699 1.697 1.684 1.671 1.658 1.645

t at 0.975 12.706 4.303 3.182 2.776 2.571 2.447 2.365 2.306 2.262 2.228 2.201 2.179 2.160 2.145 2.131 2.120 2.1 10 2.101 2.093 2.086 2.080 2.074 2.069 2.064 2.060 2.056 2.052 2.048 2.045 2.042 2.021 2.000 1.980 1.960

t at 0.990 31.821 6.965 4.541 3.747 3.365 3.143 2.998 2.896 2.821 2.764 2.71 8 2.681 2.650 2.624 2.602 2.583 2.567 2.552 2.539 2.528 2.51 8 2.508 2.500 2.492 2.485 2.479 2.473 2.467 2.462 2.457 2.423 2.390 2.358 2.326

t at 0.995 63.656 9.925 5.841 4.604 4.032 3.707 3.499 3.355 3.250 3.169 3.106 3.055 3.01 2 2.977 2.947 2.921 2.898 2.87% 2.861 2.845 2.831 2.819 2.807 2.797 2.787 2.779 2.771 2.763 2.756 2.750 2.704 2.660 2.61 7 2.576

000966 F-41


Example:

Using example data set number 1 in Table F-4, the upper one-sided 95 percent confidence limit for a normal distribution was calculated as follows:

n = 36

X = 0.045 S = 0.014

-

h.Es = 1.6905

0.014 95% UCL, = 0.045 + (1.6905) - m = 0.049


Upper One-sided 95% Confidence Limit - hgnormal Distribution

The procedure for calculating the upper 95% confidence limit for the lognormal distribution (Gilbert 1987, Equation 13.13) is given by:

2 sy H0.W + 0.5 sy + -

4 x - 95% UCL, = exp

where:

y = l n x - y = arithmetic mean of y

sY = standard deviation of y


H,,, = value from Table F-12 for sample of size n

Example:

The data presented in Table F-4 were used for this example. The mean of the log transformed data (y) equals -3.144 and the standard deviation of the log transformed data equals 0.342. These were previously calculated in the section on calculating the estimated mean of a lognormal distribution. Tables for determining H values are presented individually by number of data points for various significance levels. A summary of these tables for a 0.05 significance level (Ho.95) is presented in Table F-12. When the number of data points was not listed in this Table F-12 then linear interpolation between columns was performed. In addition, if sy did not match the sy in the first column of Table F-12, then linear interpolation between column values was performed. The upper 95% one-sided confidence limit for the lognormal distribution is calculated as follows:

where: n = 36 y = -3.144 (see estimated mean of lognormal distribution example) s = 0.342

= 1.807 (linear interpolation between sy values)

F-43

FEMP Background Study May 1994 i

I ! - I I I I I

000969 CIN~OlJ5dl/'WP/419195/TABF12.XLS/5-94 F-44

FEMP Backgr &I%!# May 1994

a

6 c 0 P E

n Q c P - 9 .- Y

I 0 r

F45

FEW Background Study May 1994 .

(0.342) (1.807) 95% UCL, = exp -3.144 + 0.5(0.342)* + I J%=r = exp (-2.981)

= 0.051

Thus the upper one-sided 95% confidence limit for the lognormal distribution is 0.051.

, cIN/OU5~/wp/419195/AppENDIxp/s-94 F-46 ,.. . . *

000971

5.644.


Upper One-sided 95% Confidence Limit - Nonparametric Technique

The upper 95% one-sided confidence limit for an undefined distribution is based on a noparametric technique (Gilbert 1987, Equation 13.2). It is simply the upper 95% confidence limit on the median of the data set. The following equations are used to calculate the upper one-sided 95% confidence limit for an undefined distribution:

where:


&.% = upper 95% limit from a standard normal curve for a Z distribution [Table F-5 at 0.95 (&.,) = 1.6451

a U = rank in an ascending order data set that corresponds to the one-sided

95% confidence limit on the median

f(U) = U rounded up to an integer (e.g., 24.2 + 25)

Example:

Using the data presented in Table F-4, the upper one-sided 95% confidence limit on the median is calculated by the following steps.

1. Order the data in ascending order

2. Detennine the number of data points (36)

3. Obtain the &.9s from Table F-5 (1.645)

F-47


4. Calculate U

36+1 + 1.645 U =

= 23.435

5. Round up to an integer (24)

6. Determine the "Uth value in the ascending order data set value at the 24th rank = (0.514)

The 95% UCL, for this data set is 0.514.

F-48

5644


95* Percentile - Normal Distribution

The 95* Percentile (or Quantile) for a normal distribution are used to determine maximum background concentrations from data sets that are normally distributed. The 95* percentile for a normal distribution (Gilber 1987, Equation 11.1) is calculated based on the following equation:

where: - X = sample arithmetic mean

S = sample arithmetic standard deviation

= upper 95% limit from a standard normal curve for a 2 distribution [Table F-5 at 0.95 (Gags) = 1.6451

Example:

Using example data set number 1 in Table F-4, the 9Sh percentile for a normal distribution was calculated by the following:

- X = 0.045

S = 0.014

9Sth Percentile, = 0.045 + (1.645) (0.014)

= 0.068

F-49 000974


95* Percentile - Lognormal Distribution

The 9Sth Percentile (or Quantile) for a lognormal distribution (Gilbert 1987, Equation 1

13.24) is calculated based on the following equation.

where: - y = sample arithmetic mean of y

= sample arithmetic standard deviation of y sY

Z,,, = upper 95% limit from a standard normal curve for a Z distribution [Table F-5 at 0.95 (Z,,,) = 1.6451

Example:

Using example data set number 1 in Table F-4, the 9Sh Percentile for a lognormal distribution was calculated by the following:

- y = -3.144

= 0.342 sY

&*, = 1.645

9Sh PercentiZq = exp [-3.144 + (1.645) (0.342)]

= exp (-2.581)

= 0.0757

. .,. ' '

cIN/OUS~/wp/4l9195/AppENDDLF/s-W F-50


9S” Percentile - Nonparametric Technique

The 95* for an undefined distribution is based on a nonparametric technique. It is calculated by the following equations:

Q = 0.95 n

9Sth Percentile, = x[f(o)l

where:


Q = rank in an ascending order data set that corresponds to the one-sided 0.95 quantile based on a nonparametric technique

f(Q) = Q rounded up to an integer (e.g., 14.1 + 15)

Example:

Using the example data set presented in Table F-4, the 9Sth percentile based on a nonparametric technique is calculated by the following steps:

1. Order the data in ascending order

2. Determine the number of data point (36)

3. Calculate Q

Q = (0.95) (36)

= 34.2

4. Round Q up to an integer (35)

5. Determine the “Qth value in the ascending order data set (value at 3Sth rank = 0.066)

The 9Sth Percentile, for this data set is 0.066.

F-5 1

.._ :. . . . .5644 . I.


F-Test

The F-test is conducted to test whether there is no difference between two population variances from a combined data set that is normally distributed. By conducting this test on the natural logarithms of each data value, the F-test was used to determine whether there is no difference between population variances from a combined data set that is lognormally distributed. An alpha of 0.05 was selected for this test. The null hypothesis to be tested is:

H,: The populations have equivalent variances (0: = 0:)

versus

The alternative hypothesis Rejection Region for a Level 0.05 Test

The following is the procedure to determine whether the two groups have variances which are statistically significantly different.

1. Calculate the sample variances of the two groups (si and si).

2. Calculate the test statistic.

3. Determine the two Fcritica, values. These critical values are FOeOu, n1 1, n2 1, which is obtained from Table F-13, while FOaWs, n1 1, n2 is obtained from 1/Fo.a, n2 - 1, "1- 1.

4. Compare Ftat versus the Fcritica, values.

Based on the rejection table listed above, when Fta, is between the two Fcritica, values, there is insufficient information to conclude that the sample variances are from two different populations. Some statistical packages provide p-values (significance levels) based on the F-test statistic (Ftat). When using these statistical packages, a p-value (significance level) of 0.05 was used. When the p-value was greater than or equal to 0.05, then the null hypothesis (H,) was not rejected. When the p-value was less than 0.05, then the null hypothesis was rejected or the alternative hypothesis was accepted.

cIN/OUS~/wp/sl91%/AppENDIX.F/s-94 F-52 1 ~ :'. .:

5644 FEMP Background Study.

May 1994

.. . CIN/OU5RI/WP/419195/rABFl3.XLS/5-94 F-53


May 1994

Example:

The log transformed data column from Table F-14 was used to conduct the F-test. The procedure for determining whether the two populations have the same sample variances was calculated as follows:

1. Calculate the sample variances.

s2 = - 1 c (3 -a2 n-1 i=l

s: = 1.415

si = 1.853

2. Calculate the test statistic. 2 2

Ftcst = S l b 2

= 1.41511.853 = 0.764

3. Determine the two Fcritica, values.

F0.Ms,9.14 = 3'21

F0.975,9,14 = 1 / F , . ~ , ~ 4 , 9

= 113.80 = 0.263

No. of data

, points 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Averaae

FEk

Normal 0.05 0.05 0.05 0.05 0.05 0.275 0.275 0.476 0.735 0.756 0.851

1 1 1 1.5 0.541


Log- normal -2.996 -2.996 -2.996 -2.996 -2.996 -1.291 -1.291 -0.742 -0.308 -0.280 -0.161 0.000 0.000 0.000 0.405 -1.243

Total Uranium,

ug/L 0.1 0.5 0.7 1 1.1

1.2325 1.791 2 2

3.852

0.1 0.1 0.1 0.1 0.55 0.55 0.476 0.735 0.756 0.851

1 1 1 1.5

Val idat ion qualifier

UJ J J

J

UJ UJ UJ U U* U* J J J J J J

1

Conc. used in stat

Normal 0.05 0.5 0.7 1

1.1 1.2325 1.791 2 2

3.852

1.423

tics (a) Log-

normal -2.996 -0.693 -0.357 O.OO0 0.095 0.209 0.583 0.693 0.693 1.349

-0.042

PriW

:- 5644 FEMP Background Study

May 1994

(a) When the validation qualifier contains a "u", then one-half of the concentration is used in the statistical calculations.

F-55

a FEW Background Study May 1994

4. Compare Ftat versus the Fcritical values.

F0.975, 9, 14 ' Ftest and F0.025, 9, 14 > F t a

0.263 c 0.764 and 3.21 > 0.764

Since Fta is between the two Fcritical values, the null hypothesis (H,) is not rejected. If F, were not between the Fcritica, values, the null hypothesis would be rejected. Based on these results, there is insufficient evidence to conclude that the sample variances are from two different populations (lognormally distributed).

a


T-Test

The T-test is conducted to test whether there is no difference between two population means with equal variances from a combined data set that is normally distributed. By conducting this test on the natural logarithms at each data value, the T-test was used to determine whether there is no difference between two population means with equal variances from a combined data set that is lognormally distributed. An alpha of 0.05 was selected for this test. The null hypothesis to be tested is:

Ho: The populations have equal means

versus

The alternative hypothesis Rejection Region for a Level 0.05 Test

The following is the procedure to determine whether the two groups have means which are statistically significantly different.

1. Calculate the sample means of the two groups (yl and y2).

2. Calculate the sample variances of the two groups (s: and sl>.

3. Calculate the estimated pooled standard deviation. r 105


1

nl n2

,+- I

5. Determine the critical values (Tcritica,) from Table F-11.

6. Compare Ttat versus the Tcritid values.


Based on the rejection table listed above, when the T,, statistic is between the two critical values, there is insufficient information to conclude that the means are from two different populations. Some statistical packages provide p-values (significance levels) based on the Ttat statistics. When using these statistical packages, a p-value (significance level) of less than 0.05 was used. When the p-value was greater than or equal to 0.05, then the null hypothesis (H,) was not rejected. When the p-value was less than 0.05, then the null hypothesis was rejected or the alternative hypothesis was accepted.

Example:

The log transformed data column from Table F-14 was used to conduct the T-test. The procedure for determining whether the two populations have the same means is as follows:

1. Calculate the sample means. - x1 = -0.042 (from Table F-14) - x2 = -1.243 (from Table F-14)

2. Calculate the sample variances.

sf = 1.415 (from F-test example)

= 1.853 (from F-test example)

3. Calculate the estimated pooled standard deviation.

(10 - 1)(1.415) + (15 - s = [ 10 + 15 - 2

= 1.297

F-58 000983



- (( -0.042) - (-1.243)) I Tta -

1 10 15

1.297 J - + -

= 2.268

5. Determine tk. 3tical values from Table F-11.

6. Compare Tt, versus the critical values.

Ttcst L To.ws,u and Ttest I -To.ws,u

and 2.268 > -2.069 - 2.268 > 2.069 - Since T,,, is greater than To.97s,u, the null hypothesis is rejected. Based on these results, there is a statistically significant difference between these two means. Furthermore, these data sets are not from the same lognormal distribution.

0

0 '- , .. F-59


The Wilcoxon Rank Sum Test

The Wilcoxon Rank Sum test is a procedure which can be used to determine whether two sample groups have equivalent means. This test assumes that the distributions of the two populations are identical in shape (variance), but the distributions need not be symmetric. The Wilcoxon Rank Sum test was used when comparing two populations (Le., FEMP vs. private wells in the glacial overburden), while the Kruskal-Wallis test was used when comparing three or more populations (Le., Dry Fork, Ross, and Shandon tributaries of the Great Miami Aquifer). In general, the Wilcoxon Rank Sum test should be employed whenever the proportion of nondetects is greater than 15 percent but less than 90 percent. However, in order to provide valid results, the Wilcoxon Rank Sum test should not be used unless both data sets contain at least four samples. The following equations present a step-by-step procedure for conducting the Wilcoxon Rank Sum test.

1.

2.

3.

Combine the Group 1 with the Group 2 data and rank the ordered values from 1 to N. Assume there are n Group 1 samples and m Group 2 samples so that N = m + n.

Compute the Wilcoxon statistic W:

1 2

n

i= l W = Ei - - n (n + 1)

where Ei are the ranks of the Group 1 samples. (Large values of the statistic W give evidence that the groups are not from the same populations.)

Compute an approximate Ztat. To find the critical value of W, a normal approxi- mation to its distribution is used. The expected value and standard deviation of W under the null hypothesis (Le., the groups are from the same population) are given by the formulas

1 2

E(W) = - mn; .

An approximate Z,,, for the Wilcoxon Rank Sum test may be calculated by the following equations:

000985


1 W - E ( W ) - -

SD’ (W) where SD’ = [TN % =

The factor of 34 in the numerator serves as a continuity correction since the discrete distribution of the statistic W is being approximated by the continuous normal distribution. If n,m > 10 and ties are present, an adjustment to the approximate Z,,, must be made:

g f& - 1) j =i

+ 1 - N(N - 1)

and g is the number of tied groups and tj is the number of tied data in the jth group.

4. For a one-tailed 0.05 significance level test for Ho versus the HA (Le., the measurements from population l tend to exceed those from population 2), reject H, and accept HA if 2, 2 ZO4). For a one-tailed a significance level test for H, versus the HA that the measurements from population 2 tend to exceed those from population 1, reject H, and accept HA if 2, - < Z&).

Example:

The data for this example are presented in Table F-15.

1. Combine the FEMP and private Total Uranium data and rank the ordered values from 1 to 25 as shown in Table F-15.

2. Compute the Wilcoxon statistic W

1 1 2 2

n

i=l W = Ei - -n(n + 1) = 178 - - (10) (11) = 123

3. Compute an approximate Z,,,. The expected value and standard deviation of W under the null hypothesis are given by the formulas

E(W) = - 1 (15) (10) = 75; 2

F-6 1

Overall ranks

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Sum



Total Uranium values used

in calculations (a) 0.05 0.05 0.05 0.05 0.05 0.05 0.275 0.275 0.476 0.5 0.7 0.735 0.756 0.851

1 1 1 1 1.1

1.2325 1.5 1.791 2 2

3.852

Type of

well Private Private Private Private Private FEMP Private Private Private FEMP FEMP Private Private Private Private Private Private FEMP FEMP FEMP Private FEMP FEMP FEMP FEMP

RZ Private

1 2 3 4 5

7 8 9

N/A

N/A N/A 12 13 14 15 16 17

N/A N/A N/A

N/A N/A N/A N/A

21

147 (a) These values are from Table F-14.

CIN/OU5R1/419195/TABF15.XLS/5-94 F 4 2 , I

000987

5644


An approximate Z,,, for the Wilcoxon Rank-Sum test then follows as: -,v.- 1

4. Compare the approximate && to the upper 95'h percentile of the standard normal distribution = 1.645. Since the approximate Z,,, is greater than 1.645, the null hypothesis may be rejected at the 5 percent significance level, suggesting that there is statistically significant evidence that the populations have different means. The FEMP and private wells for Total Uranium in the Glacial Overburden are not from the same populations and not not be combined.

F-63


1

x11

x 1 2

1 Xln

Kruskal-Wallis Test for Comparing Populations

2 3 ... k

X2l '31 ... 'k1

x22 '32 ... xk2

...

xhk ...

3 x3n

2 X2"

The Kruskal-Wallis test (Gilbert 1987) for comparing populations does not require that data sets be drawn from underlying distributions that are normal or even symmetric, but the K distributions are assumed to be identical in shape. The null hypothesis is

Ho: The populations from which the data sets have been drawn have the same means.

The alternative hypothesis is

HA: At least one population has a mean larger or smaller than at least one other population.

The data can be illustrated as follows:

The total number of data points is m = n1 + n2 + ... + n,; the q need not be equal.

Theses steps must be followed:

1. Rank the m data points from smallest to largest in which the smallest value has rank 1 and the largest value has rank m. If data points are equal (Yies") assign the midrank (e.g., data points 10 and 11 are the same; therefore, 10.5 is used for the rank of both points). If less-than-minimum detectable values occur, treat them as tied values that are less than the smallest detected value that is greater than the minimum detection limit. Suppose the m ranked values are as follows:

F-64 OQ0989

6644

0.05 0.05 0.05 0.08 0.09 0.12 0.12 0.20 0.21 0.21 0.21

Data Value I E? 1 2 3 4 5 6 7 8 9 10 11

Assigned Rank

2 2 2 4 5 6.5 6.5 8 10 10 10


2. Compute Rj the sum of the ranks for each data set.

3. If there are no tied or less-than-minimum detectable values, compute.

where:

m Rj 9 k

= total number of data values over all data sets = sum of ranks of the jth data set = number of values in the jth data set = number of data sets

4. If there are ties or less-than-detectable values, compute:

F-65 000990


where:

g ‘i

= number of groups with ties = the number of tied data in the jth group

5. Reject H, at a level and accept Ha if K, (g) is equal to or greater than xl*pl using a probability level of l-a and k-1 degress of freedom taken from Table F-16.

Example:

To illustrate the application of the Kruskal-Wallis & test consider the example data set presented in Table F-16. For these data, m = 30 + 30 + 21 = 81.

1. Rank all 81 observations (Table F-17).

2. The sum of the ranks for each group are as follows (Table F-18).

R, = 1454 R, = 1209.5 R, = 657.5

3. Compute:

(1454)* (1209.5)* + (657.5), ] ] - (82) l2 [ 30 21 +

81 (82) 30

= 0.001807 (70,471 + 48,763 + 20,586) - 246

= 252.61 - 246

= 6.61

5644 .: FEMP Background Study

May 1994

value 0.990 0.995 6.63 7.88 9.21 10.60 11.34 12.84 13.28 14.86 15.09 16.75 16.81 18.55 18.48 20.28 20.09 21.95 21.67 23.59 23.21 25.19

26.76 28.30 29.82 31.32 32.80 34.27 35.72 37.16 38.58 40.00

41.40 42.80 44.18

~ 48.29 49.65

1 50.99 52.34 53.67

Degrees of

heedom V 1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17 18 19 20

21 22 23 24 25 26 27 28 29 30

40 50 60 70 80 90 100 X

,

31.26 32.91 34.53 36.12 37.70 39.25 40.79 42.31 43.82 45.31

46.80 48.27 49.73

"5:: 54.05 55.48 56.89 58.30 59.70

Table F-16 Quantiles of the Chi-square Distribution with v Degrees o Freedom

- 0.006 0.00 0.01 0.07 0.21 0.41 0.68 0.99 1.34 1.73 2.16

2.60 3.07 3.57 4.07 4.60 5.14 5.70 6.26 6.84 7.43

8.03 8.64 9.26 9.89 10.52 11.16 11.81 12.46 13.12 13.79

20.71 27.99 35.53

51.17 59.20 67.33

-

43.28

-2.58 -

- 0.010 0.00 0.02 0.1 1 0.30 0.55 0.87 1.24 1.65 2.09 2.56

3.05 3.57 4.1 1 4.66 5.23 5.81 6.41 7.01 7.63 8.26

8.90 9.54 10.20 10.86 11.52 12.20 12.88 13.56 14.26 14.95

22.16 29.71 37.48 45.44 53.54 61.75 70.06

-

-2.33 -

Prob 0.025 0.00 0.05 0.22 0.48 0.83 1.24 1.69 2.1 8 2.70 3.25

3.82 4.40 5.01 5.63 6.26 6.91 7.56 8.23 8.91 9.59

10.28 10.98 11.69 12.40 13.12 13.84 14.57 15.31 16.05 16.79

24.43 32.36 40.48 48.76 57.15 65.65 74.22 -1.96

- -

-

3il.W a 0.05 0.00 0.10 0.35 0.71 1.15 1.64 2.17 2.73 3.33 3.94

4.57 5.23 5.89 6.57 7.26 7.96 8.67 9.39 10.12 10.85

1 1.59 12.34 13.09 13.85 14.61 15.38 16.15 16.93 17.71 18.49

26.51 34.76 43.19 51.74 60.39 69.13 77.93 -1.65

-

-

obtain 0.100 0.02 0.21 0.58 1.06 1.61 2.20 2.83 3.49 4.17 4.87

5.58 6.30 7.04 7.79 8.55 9.31 10.09 10.86 11.65 12.44

13.24 14.04 14.85 15.66 16.47 17.29 18.11 18.94 19.77 20.60

29.05 37.69 46.46 55.33 64.28 73.29 82.36

-

-1.28 -

g a va 0.250 0.10 0.58 1.21 1.92 2.67 3.45 4.25 5.07 5.90 6.74

7.58 8.44 9.30 10.17 11.04 11.91 12.79 13.68 14.56 15.45

16.34 17.24 18.14 19.04 19.94 20.84 21.75 22.66 23.57 24.48

33.66 42.94 52.29 61.70 71.14 80.62 90.13 -0.67

-

-

le of c 0.500 0.45 1.39 2.37 3.36 4.35 5.35 6.35 7.34 8.34 9.34

10.34 11.34 12.34 13.34 14.34 15.34 16.34 17.34 18.34 19.34

20.34 21.34 22.34 23.34 24.34 25.34 26.34 27.34 28.34 29.34

49.33 59.33 69.33 79.33 89.33 99.33

0

-

39.34

-

-squre 0.750 1.32 2.77 4.1 1 5.39 6.63 7.84 9.04 10.22 1 1.39 12.55

13.70 14.85 15.98 17.12 18.25 19.37 20.49 21.60 22.72 23.83

24.93 26.04 27.14 28.24 29.34 30.43 31.53 32.62 33.71 34.80

45.62 56.33 66.98 77.58 88.13 98.65 109.14 0.674

-

-

small 0.900 2.71 4.61 6.25 7.78 9.24 10.64 12.02 13.36 14.68 15.99

17.28 18.55 19.81 21.06 22.31 23.54 24.77 25.99 27.20 28.41

29.62 30.81 32.01 33.20 34.38 35.56 36.74 37.92 39.09 40.26

51.81 63.17 74.40 85.53 96.58 107.57 I 18.5C 1.282

- -

-

'than 0.950 3.84 5.99 7.81 9.49 1 1.07 12.59 14.07 15.51 16.92 18.31

19.68 21.03 22.36 23.68 25.00 26.30 27.59 28.87 30.14 31.41

32.67 33.92 35.17 36.42 37.65 38.89 40.1 1 41.34 42.56 43.77

55.76 67.50 79.08 90.53 101.s 113.15 124.34 1.645

- -

-

te tab1 0.975 5.02 7.38 9.35 11.14 12.83 14.45 16.01 17.53 19.02 20.48

21.92 23.34 24.74 26.12 27.49 28.85 30.19 31.53 32.85 34.1 7

35.48 36.78 38.08 39.36 40.65 41.92 43.19 44.46 45.72 46.98

59.34 71.42 83.30 95.02 106.62 118.14 129.56 1.96

-

-

24.73 26.22 27.69 29.14 30.58 32.00 33.41 34.81 36.19 37.57

38.93 40.29 41.64 42.98 44.31 45.64 46.96 48.28 49.59 50.89 63.69 76.15 88.38 100.4 1 12.31 124.12 135.81 2.326 -

- 0.W 10.83 13.82 16.27 18.47 20.51 22.a 24.32 26.12 27.88 29.59

-

2.576

73.40

99.61 ~ 112.3: 124.W 137.2'

~ 149.4! 3.090

186.66

- For degrees of freedom (v) > 100, chi-squared=v[l-2/9v+X *(2/9v)*0.5]^3 or chi-~quared=0.5*[X+(2v-l)~0.5]~2

Source: Table A-19, Gilbert 1987.

if less accuracy is needed, where X is given in the last row of the table.

a ;., 000992

F-67


Table F-17. Ranking of Example Data Set Number 7 (Concentrations in pg/L)

Rank Group ChemicalA Rank Group ChemicalA

!I 1 2 3 4 5

6 7 8 9 10 11 12 l3 14 15 16 175 175) 19 20 21 22 23 24 25 265

28 29 30 31 32 33 34 35 36 37 38 39 405

42

45)

263

405)

1 2 3 1 2 2 3 3 3 2 1 3 3 3 2 2 2 2 1 2 3 3 1 2 2 2 3 3 1 1 1 1 3 3 3 3 1 2 3 2 2 1

189 251 288 301 301 303 304 338 345 348 351 352 356 358 360 364 366 366 370 371 372 378 386 387 405 406 406 420 422 437 451 456 459 464 467 472 481 482 483 486 486 488

43 445

46 47 48 49 50 51 52 53 54 555

57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81

445)

555)

1 1 2 1 1 3 2 2 1 2 1 1 2 2 2 2 3 2 1 3 1 1 2 2 1 3 1 2 2 1 1 1 1 2 2 1 1 3 1

521 534 534 535 543 560 561 566 581 583 615 619 621 621 654 667 681 745 747 759 766 785 812 829 840 931 941 1020 1040 1050 1070 1090 1150 uoo 1410 1460 1500 1750 4850

Group 1 = Dry Fork Group 2 = Ross Group3 = Shandon

) = ties \

FEM

P Ba

ckgr

ound

Study

May

1994

Tab

le F

-18.

Ran

king

of E

xam

ple

Dat

a Se

t Num

ber

7 by G

roup

(C

once

ntra

tions

in p

g/L

)

Dry

For

k R

oss

Shan

don

1 2

3 4

5

45

7

11

6 8

19

10

9 23

15

12

29

16

13

30

17.5

14

31

17

.5

21

32

20

22

37

24

265

42

25

28

43

265

33

445

38

34

46

405

35

47

405

36

51

445

39

53

49

48

54

50

59

61

52

62

63

555

68

64

555

80

67

57

69

58

72

60

73

65

74

66

75

70

78

71

79

76

81

n SUm

1454

12

095

6575

cEN

/OU

SRI/

wp/

4 I9

1%/A

F'PE

ND

IXF/

SW

-/

.

/

F-69

O

Q0

994

EM

F' Ba

ckgr

ound

Stu

dy

May

199

4

4.

Bec

ause

ther

e ar

e tie

s, ca

lcul

ate

a m

odifi

ed c

:

--

I

1-

1

E 'j (

f -

1)

M

(m2

- 1)

j=

1

whe

re: g tl t2 t3 t4 tS t6

= 6

= n

umbe

r of

grou

ps w

ith t

ies

= 2

= n

umbe

r of

data

poi

nts

in th

e fir

st tl

zd g

roup

(R

ank -3

) =

2 =

num

ber o

f da

ta p

oint

s in

the

seco

nd ti

ed g

roup

(R

ank

17.5

) =

2 =

num

ber o

f da

ta p

oint

s in

the

third

tie

d gr

oup

(Ran

k 26

.5)

= 2

= n

umbe

r of

data

poi

nts in

the

four

th ti

ed g

roup

(R

ank

46.5

) =

2 =

num

ber

of d

ata

poin

ts in

the

fifth

tied

gro

up (R

ank

44.5

) =

2 =

num

ber

of d

ata

poin

ts in

the

sixth

tied

gro

up (R

ank

55.5

) ,

6.61

1-

1

[ 2(3

) +

2(3

) +

2(

3) +

2(

3) +

2(

3) +

2

~1

81

[(8

1)2 -

1)

- 6.

61

- 0.

9999

3

= 6.

61

5.

Bec

ause

=

6.6

1 is

grea

ter t

han

= 5

.99,

the

null

hypo

thes

is H

,, is

reje

cted

at

the a

= 0

.05 le

vel.

The

con

clus

ion

is th

at a

t lea

st o

ne p

opul

atio

n be

ing

com

pare

d ha

s a

mea

n di

ffer

ent f

rom

the

othe

r pop

ulat

ions

.

F-70

natural resources defense council’s & powder river basin resource council… ·...

Documents