Reinaldo Garcia, PhD

A proposal for testing two-dimensional models to use in the National Flood Insurance Program

Presentation Outline

MotivationStatus quo

Why we need improvementsWhat are others doing

Suggested components for new model review guidelines

Model performance and predictive capability

Depends on many factors, including:• Model mathematical formulation• Numerical method• Appropriate use of boundary conditions• Adequate configuration of grid or mesh• Choice of roughness coefficients and time steps• Data quality• Expertise of the modeler to perform the simulation.

To ensure adequately testing the predictive capabilities of a 2D flood model, testing guidelines should be based on standardized

benchmarks and specifications

Status Quo

FEMA requires that any hydrologic or hydraulic computer model used in support of flood insurance studies must be reviewed, tested and accepted by a government agency responsible for the implementation of programs for flood control and/or regulation of floodplains National Flood Insurance Program (NFIP) regulations. Subparagraph 65.6. FEMA relies on other agencies to review and test a model and determine whether it is scientifically correct and technically sound.FEMA List: Numerical Models Meeting the Minimum Requirement of NFIP Current Nationally Accepted Hydraulic Models

Why we need improvements

FEMA does not establish a standard testing procedure to determine if a model is scientifically or computationally correctNo specifications or metrics to verify that the model can provide adequate or accurate information to support NFIP studies. Inconsistent testing methods depending on the agency or modeler doing the evaluation. Federal agencies often have no budget or time test a model that is not on the FEMA list. Some government agencies also develop their own models…

Status Quo

Is not openIs not fully reliable

Hampers innovationDoes not serve well FEMA’s mission

What are others doing?

Components of sound model review guidelines

Model verificationModel validation

Standardized benchmarksMetricsOpen

Model vs. Real World

CalibrationAdjusting numerical or physical parameters in the program to improve comparison with data

VerificationDetermining that the program accurately represent the conceptual model (program solves the equations right)

QualificationFidelity of the equations to reality

ValidationDetermining the degree to which the model is an accurate representation of the real world.

Verification

Example of model verificationComparison with Lynch & Gray 2-D analytical solutions

Example of model verification

Validation

“No one believes model results except the one who performed the calculation,

...everyone believes observed or measured data except the one who did the measurements”

P. J. Roache

Validation

UK Benchmarks

Provide a dataset against which 2D hydraulic modeling packages can be evaluated

Provide evidence to ensure that such models used for flood and coastal risk management are capable of adequately predicting the variables upon which flood risk management decisions are based.

UK BenchmarksFlooding of a disconnected water body Assess basic capability to simulate flooding of disconnected water bodies on

floodplains or coastal areas and wetting and drying of floodplains

Filling of flood plain depressions Tests capability to predict inundation extent and final flood depth flow low momentum flow over complex topographies. Also tests flooding of disconnected water bodies and wetting and drying of floodplains. Predicts inundation extent, with an emphasis on final distribution of water rather than peak levels.

Momentum conservation over a small (0.25m high) obstruction

Tests capability to simulate flow at relatively low depths over an obstruction with an adverse slope.

Speed of flood propagation over an extended flood plain

Tests simulation of speed of propagation of flood wave and the prediction of velocities at the leading edge of the advancing flood.

Valley flooding Tests simulation of major flood inundation at the valley scale. Dam break Tests simulation of shocks and wake zones close to a failing dam. This could also

be applied to other examples like rapid opening of sluice gates, failures of embankments, etc.

River to floodplain linking Evaluates capability to simulate flood volume transfer between rivers and floodplains using 1D and 2D model linking.

Rainfall and sewer surcharge Tests capability to simulate shallow flows in urban areas with inputs from rainfall and sewer surcharge.

Test relevance for specific applications

Application Test Large scale flood risk mapping 1 & 2 Catchment Flood Management Plan 1, 2 & 7 Flood Risk Assessment and detailed flood mapping

1, 2, 3 & 7

Strategic Flood Risk Assessment 1, 2, 3, 4, 7 & 8 Flood hazard mapping 1, 2, 3, 4, 7 & 8 Contingency planning for real time flood management

1, 2, 3, 4, 5, 7 & 8

Reservoir flooding mapping 1, 2, 3, 4, 5 & 6

Benchmark Examples

Flooding a disconnected water body

UK Benchmark Examples

Assess model’s ability to conserve momentum over an obstruction in the topography.Important when simulating flooding in urbanized floodplains.

UK Benchmarks

Should be complemented with verification

Americas Society of Civil Engineers

ASCE: Verification and Validation of 3D Free-Surface Flow Models, 2008Three main components:– Analytical solutions– Lab/flume experiments– Field scale real-world applications

Provides a data set for 3D models only

Suggestions for Model Review Guidelines

Verification should precede validationValidation based on set of benchmarks Acceptable error tolerances (metrics)Benchmarks based on intended use: not all models are necessarily adequate for all projectsStandardized test and benchmark data sets should be public and freely available

Implementing these model review guidelines would better serve FEMA’s mission.

Thank you / Questionsrey@hydronia.com