1

STM Module Testing and Verification

CWEMF Annual MeetingFeb. 28, 2011

Eli Ateljevich, Ph.D., P.E., DWRKaveh Zamani, UCD

Fabian Bombardelli, Ph.D., UCD Jamie Anderson, Ph.D., P.E., DWR

Modeling Support BranchBay-Delta OfficeDepartment of Water Resources

Civil and Environmental EngineeringUniversity of California-Davis

This Talk

• Short intro to STM• Types of tests and verification• STM accuracy goals and test suite• Some results• Future work

33

DSM2 Sediment Transport Module

PTMParticle Tracking

QUALWater Quality

STMSediment and

Transport

HYDROHydrodynamics

Why Transport?

• Inline with Hydro or offline with tidefile:– Inline: Baroclinic term for hydro– Inline: Operating rules based on water quality– Offline: Testable using synthesized flows– Offline: Tidefile issues

• Flexible description of dispersion• Rigorous verification and testing

Testing Concepts

6

7

8

9

The unit testing concept

calculate_gradient

test_calculate_gradient

Unit Tests Cover the Possibilities

Step 1: Gradient

Step 2: Extrapolate

Step 3: Update

Fortran Unit Test Framework(FRUIT)

• FRUIT– open source– written in Fortran 95 – has assertion (true/false pass/fail), fixture,

setup, teardown, report, spec generation, driver generation

– as is often the case, used for more than just unit tests

– the only real issues are culture and will

12

Convergence tests are the main tool for verification

• Why they are important– They ensure the underlying physics– They are what models promise– They are based on a hard criterion

• Why they are pesky– They can fail when accuracy is fantastic– May or may not use the models in the

“convergent zone”– They take some work for nonlinear, spatially

varying problems (Richardson Extrapolation or manufactured solutions)

0

0.1

0.2

0.3

0.4

0.5

0.6

16,000 18,000 20,000 22,000 24,000 26,000

Con

cent

ratio

n (V

/V)

Channel Length (m)

Advection Diffusion, Uniform Flow

C ncell = 256

C ncell = 128

C ncell = 64

C exact

0.53

0.54

0.55

0.56

0.57

0.58

0.59

24,200 24,400 24,600 24,800 25,000 25,200

Con

cent

ratio

n (V

/V)

Channel Length (m)

Advection Diffusion, Uniform Flow

C ncell = 256

C ncell = 128

C ncell = 64

C exact

The algorithm verification goal of STM is to be second order O(2) on the domain interior and accurate/O(1) on problems involving significant boundary interaction and non-linear sources.

½ *∆t, ½* ∆x ¼ * error

-The model should also be robust- And attend the “real” requirements of users

16

21

Testing Report Card 1

22

Test Name Test ResultAdvection (plug flow)Uniform unidirectional flow, Gaussian mass distribution, zero concentration BC Passed

Uniform bidirectional flow, Gaussian mass distribution, zero concentration remote BC Passed

Uniform bidirectional flow, Gaussian mass distribution, specified concentration BC PassedTidal flow, Gaussian distribution of mass, zero concentration remote BC PassedTidal flow, Sinusoidal distribution of mass, zero concentration remote BC PassedDiffusion (mixing) Smooth distribution of mass, concentration value BC (Fletcher, 1991) PassedSmooth distribution of mass, flux value BC (Fletcher, 19912) PassedGaussian distribution of mass, zero concentration value BC PassedGaussian distribution of mass, flux value BC PassedReaction (growth/decay)Linear decay, 2nd order Heun ordinary differential equation solver PassedLinear decay, 3rd order Runge‐Kutta ordinary differential equation solver Passed

23

Test Name Test ResultAdvection & Diffusion

Uniform flow, Gaussian mass, concentration remote BC, constant dispersion coefficient Passed

Uniform flow, Gaussian mass, value concentration BC, constant dispersion coefficient accept

Spatially varying flow and dispersion coefficient (Zoppou & Knight, 1998) reviewTemporally varying flow and dispersion coefficient (Crank, 1975) ~ In progressAdvection & ReactionUniform flow, Gaussian mass distribution, linear decay Passed

Tidal flow, Gaussian distribution of mass, zero concentration remote BC, linear decay Passed

Tidal flow, sinusoidal distribution of mass, zero concentration remote BC, linear decay Passed

Advection, Diffusion & Reaction

Uniform flow, Gaussian mass, specified conc. BC, const. dispersion coeff., linear decay accept

Uniform flow, Gaussian mass, remote conc. BC, const. dispersion coeff., linear decay reviewNon‐uniform flow, time and spatially varying disp. coefficient, non‐linear decay ~ In progress

Testing Report Card 2

0.0

0.2

0.4

0.6

0.8

1.0

5000 10000 15000 20000 25000 30000 35000

Con

cent

ratio

n (V

/V)

Channel Length (m)

Advection Uniform FlowΔx=100 m, Δt=2.5 min, number of cells = 256

C (t=0)

C (t=T/4)

C (t=T/2)

C (t=3T/4)

C (t=T)

C exact

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

200 5,200 10,200 15,200 20,200 25,200

Con

cent

ratio

n (V

/V)

Channel Length (m)

Advection Diffusion, Uniform FlowΔx=400 m, Δt = 10 min, number of cells 64

C (t=0) C (t=T/4)

C (t=T/2) C (t=3T/4)

C (t=T) C exact

0.0

0.5

1.0

1.5

2.0

2.5

150,000 250,000 350,000

Con

cent

ratio

n (V

/V)

Channel Length (m)

Tidal Flow, Remote BCΔx=400 m, Δt = 5.81 min, number of cells 256

C (t=0)

C (t=T/4)

C (t=T/2)

C (t=3T/4)

C (t=T)

Tidal Flow, Linear Decay, Remote BC

27

0.0

0.4

0.8

1.2

1.6

2.0

40,000 45,000 50,000 55,000 60,000 65,000 70,000

Con

cent

ratio

n (V

/V)

Channel Length (m)

C(t=0)C(t=T/4)C(t=T/2)C(t=3T/4)C(t=T)Exact

Are convergence tests enough?• No. They are brutal. They expose bugs.

But they leave behind some bugs as well.• There are reasons to look at accuracy

Grid Size

Err

or Machine precisionexceeds round off

error

Computerprecision

1Order of Converge

IIIIII

Bottom line• Verification is one part of “getting the right

answer for the right reason”• We have a really complete test suite

– May help solve the “grey literature” issue• STM is second order or O(1)/accurate • We know where the algorithmic weakness

are (and they are minor)

Next steps

• Sediment (UCD)• Add it to the DSM2 suite (DWR)

– But with enough input to support testability– Channel network treatment will influence the

accuracy of the model

30