Watershed Hydrology Modeling: What is Considered Calibrated?

Watershed Hydrology Modeling: What is Considered Calibrated?

Presented by:

Jeremy Wyss, HIT

Tetra Tech

27th Annual Alabama Water Resources Conference

Orange Beach, Alabama

27th Annual Alabama Water Resources Conference

Orange Beach, Alabama

Watershed ModelsWatershed Models

Play an important role in linking sources of pollutants to receiving waterbodies as point and nonpoint source loads.

Are driven by precipitation, land use, impervious area, slope, soil type and drainage area.

Are simplified mathematical representations of complex real world systems which make use of known interrelationships to predict change in response to perturbation of independent variable or forcing function from its current state.

To be considered credible, the ability of the model to represent real world conditions should be demonstrated through calibration/corroboration.

LSPC, HSPF, SWAT, WAM, SWMM, WARMF, etc…

Model Calibration/CorroborationModel Calibration/Corroboration

Calibration – parameter adjustment to achieve the best fit between model prediction and field observations.

Explicit focus on available stream gages at multiple locations in the watershed (each has it’s own uniqueness to capture).

Calibration is for a specific period of time.

Corroboration – utilizes the calibrated model for a different period of time at multiple locations in the watershed.

Investigates whether a models predictive capability is similar to that achieved over the calibration period.

Statistical and Graphical comparison of observed and simulated.

Hydrology Graphical ComparisonHydrology Graphical Comparison

Daily Average Flow (normal scale)Daily Average Flow (log scale)Daily Average Water BalanceMonthly Average Regression Monthly Average Flow BalanceMonthly Average Flow

y = 0.7394x + 97.379

R2 = 0.8785

0

200

400

600

800

0 200 400 600 800

Average Observed Flow (cfs)

Ave

rag

e M

od

ele

d F

low

(cf

s)

Avg Flow (1/1/1998 to 12/31/2009)Line of Equal ValueBest-Fit Line

J F M A M J J A S O N D

0

200

400

600

800

1 2 3 4 5 6 7 8 9 10 11 12

Month

Flo

w (c

fs)

0

1

2

3

4

5

6

Mon

thly

Rai

nfal

l (in

.)

Avg Monthly Rainfall (in.)Avg Observed Flow (1/1/1998 to 12/31/2009)Avg Modeled Flow (Same Period)

Average by Month Regression Average by Month Distribution

J F M A M J J A S O N D

0

100

200

300

400

500

600

700

1 2 3 4 5 6 7 8 9 10 11 12

Month

Flo

w (

cfs

)

0

1

2

3

4

5

6

Mo

nth

ly R

ain

fall (

in.)

Average Monthly Rainfall (in) Observed (25th, 75th)Median Observed Flow (1/1/1998 to 12/31/2009) Modeled (Median, 25th, 75th)

Monthly Distribution Box Plot10

100

1000

10000

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Percent of Time that Flow is Equaled or Exceeded

Da

ily A

ve

rag

e F

low

(cfs

)

Observed Flow Duration (1/1/1998 to 12/31/2009 )

Modeled Flow Duration (1/1/1998 to 12/31/2009 )

Daily Average Flow Duration

0%

20%

40%

60%

80%

100%

120%

Jan-98 Jul-99 Jan-01 Jul-02 Jan-04 Jul-05 Jan-07 Jul-08

No

rma

lize

d F

low

Vo

lum

e (

Ob

se

rve

d a

s 1

00

%)

Observed Flow Volume (1/1/1998 to 12/31/2009 )

Modeled Flow Volume (1/1/1998 to 12/31/2009 )

Daily Average Flow Accumulation

0

500

1000

1500

J-98 J-99 J-01 J-02 J-04 J-05 J-07 J-08

Month

Flo

w (

cfs

)

0

2

4

6

8

10

12

14

Month

ly R

ain

fall

(in.)

Avg Monthly Rainfall (in.)Avg Observed Flow (1/1/1998 to 12/31/2009 )

Avg Modeled Flow (Same Period)

Hydrology Statistical ComparisonHydrology Statistical ComparisonE = Nash-Sutcliffe Coefficient of Efficiency

E’ = Garrick Baseline Adjusted Coefficient of Efficiency

Used to assess the predictive power of hydrological models

Range from -∞ to 1

-infinity < E < 0: The observed mean is a better predictor than the model

Equal to 0: The long-term average of the observed data is just as good a predictor as the model

Between 0 and 1: The closer the model efficiency is to 1, the more accurate the model is

Obtaining a value of 0.7 or better is a reasonable indicator of adequate fit.

Hydrology Calibration PitfallsHydrology Calibration Pitfalls

10

100

1000

10000

100000

1/1/2001 7/1/2001 1/1/2002 7/1/2002 1/1/2003 7/1/2003 1/1/2004 7/1/2004 1/1/2005 7/1/2005 1/1/2006 7/1/2006 1/1/2007 7/1/2007

Date

Flow

(cfs)

0

1

2

3

4

5

6

7

8

9

10

Daily

Rain

fall (i

n.)

Avg Daily Rainfall (in.) Avg Observed Flow (1/1/2001 to 12/31/2007 ) Avg Modeled Flow (Same Period)

10

100

1000

10000

100000

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Percent of Time that Flow is Equaled or Exceeded

Da

ily A

ve

rag

e F

low

(cfs

)

Observed Flow Duration (1/1/2001 to 12/31/2007 )

Modeled Flow Duration (1/1/2001 to 12/31/2007 )

0.1

1

10

100

1000

10000

1/1/2002 10/1/2002 7/1/2003 4/1/2004 1/1/2005 10/1/2005 7/1/2006 4/1/2007 1/1/2008 10/1/2008 7/1/2009 4/1/2010 1/1/2011 10/1/2011 7/1/2012

Date

Flow

(cfs)

0

1

2

3

4

5

6

7

8

9

10

Daily

Rain

fall (i

n.)

Avg Daily Rainfall (in.) Avg Observed Flow (1/1/2002 to 9/30/2012 ) Avg Modeled Flow (Same Period)

Hydrology Acceptance CriteriaHydrology Acceptance CriteriaModel Component Very Good Good Fair Poor

Error in total volume ≤ 5% 5 - 10% 10 - 15% > 15%

Error in 50% lowest flow volumes

≤ 10% 10 - 15% 15 - 25% > 25%

Error in 10% highest flow volumes

≤ 10% 10 - 15% 15 - 25% > 25%

Error in storm volume ≤ 10% 10 - 15% 15 - 25% > 25%

Winter volume error ≤ 15% 15 - 30% 30 - 50% > 50%

Spring volume error ≤ 15% 15 - 30% 30 - 50% > 50%

Summer volume error ≤ 15% 15 - 30% 30 - 50% > 50%

Fall volume error ≤ 15% 15 - 30% 30 - 50% > 50%

R2 daily values > 0.80 > 0.70 > 0.60 ≤ 0.60

R2 monthly values > 0.85 > 0.75 > 0.65 ≤ 0.65

Nash-Sutcliffe > 0.75 > 0.65 > 0.50 ≤ 0.50

• How accurate is the model?

• Is the model good enough for this evaluation?

Our “Rating” ApproachOur “Rating” Approach• Assign weight to each statistical measure

• Assign value to “Very Good”, “Good”, “Fair” and “Poor” ranges

• Multiply weight by value and sum results to obtain quantitative score

• Compare quantitative score to range to produce qualitative grade

• Produce a summary of statistical bias (all +, all -, or mixed)

• Process completed for Calibration/Validation on Period of Record and for each individual year

Floyds Fork Watershed: Hydrology Calibration Maps produced by M.Akasapu, 02-04-2013

NAD_1983_UTM_Zone_17N0 3 61.5

Miles

0 3 61.5Kilometers

Jefferson

Bullitt

Spencer

Shelby

Henry

Oldham

Salt River

Floy

ds F

ork

Ced

ar C

reek

Long

Run

USGS ID: 03297900

USGS ID: 03298000

USGS ID: 03298135

USGS ID: 03298150

USGS ID: 03298200

Floyds Fork

Che

now

eth

Run

Pen

nsyl

vani

a R

un

Cur

rys

Fo

rk

USGS ID: 03298300

USGS ID: 03298250

Legend

Flow Calibration

VG (80-75)

G (74-55)

F (54-35)

P (34-20)

Waterways

Watershed Boundary

County

Next StepsNext Steps

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1/1/1998 10/1/1998 7/1/1999 4/1/2000 1/1/2001 10/1/2001 7/1/2002 4/1/2003 1/1/2004 10/1/2004 7/1/2005 4/1/2006 1/1/2007 10/1/2007 7/1/2008 4/1/2009

Date

Wate

r Bala

nce (

Obs +

Mod

)

Avg Modeled Flow (1/1/1998 to 12/31/2009 ) Avg Observed Flow (1/1/1998 to 12/31/2009 ) Line of Equal Value

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1/1/1998 10/1/1998 7/1/1999 4/1/2000 1/1/2001 10/1/2001 7/1/2002 4/1/2003 1/1/2004 10/1/2004 7/1/2005 4/1/2006 1/1/2007 10/1/2007 7/1/2008 4/1/2009

Date

Wate

r Bala

nce (

Obs +

Mod

)

Avg Modeled Flow (1/1/1998 to 12/31/2009 ) Avg Observed Flow (1/1/1998 to 12/31/2009 ) Line of Equal Value

• Add/Incorporate/Create and Standard Operation Procedure for visual component

• Determine how to incorporate Modeling Efficiencies into Rating

Comments/Questions?Comments/Questions?

Watershed Hydrology Modeling: What is Considered Calibrated?

Watershed Hydrology Modeling: What is Considered Calibrated?