United States Coast Guard 1985

Evaluation of a Multi-Model Storm Surge Ensemble for the New York Metropolitan

Region

Brian A. Colle

Tom Di Liberto

Stony Brook University

ADCIRC Water-level and Flooding12-km MM5 Forecast 1200 UTC 11 December 1992

meters

Colle, B. A., F. Buonaiuto, M. J. Bowman, R. E. Wilson, et al., 2008: Simulations of past cyclone events to explore New York City’s vulnerability to coastal flooding and storm surge model capabilities , Bull. Amer. Meteor. Soc.

ADCIRC Surge Forecast

Current Real-Time Systems

Stony Brook Storm Surge Model

Stevens Institute of Technology’s Storm Surge model (NYHOPS)

NOAA Extratropical Storm Surge model

http://hudson.dl.stevens-tech.edu/maritimeforecast/

http://www.nws.noaa.gov/mdl/etsurge

http://stormy.msrc.sunysb.edu/

Real-Time Modeling Systems• All three models use different ocean models and

atmospheric forcing. Storm Surge Forecasting Systems

Institution Atmospheric Forcing Ocean Model Start TimeStony Brook 5 MM5 / 3 WRF members ADCIRC 0000 UTC

Stevens Institute of Technology NCEP - NAM model Princeton Ocean Model 12:00 AM

NOAA NCEP - GFS model NOAA Extratropical Storm Surge model 0000 UTC

• Stony Brook Storm Surge Model (SBSS) uses 5 MM5 and 3 WRF members

Stony Brook Storm Surge Model Atmospheric Ensemble MembersMembers Model Microphysics PBL Scheme Radiation Initial Condition Cumulus 9a MM5 Simple Ice MRF Cloud Radiation WRF-NMM GrellBMMY MM5 Simple Ice MY CCM2 GFS Betts Miller GRBLK MM5 Simple Ice Blackadar CCM2 NOGAPS GrellK2MRF MM5 Reisner MRF Cloud Radiation GFS Kain FritschK2MY MM5 Simple Ice MY CCM2 Canadian Model Kain Fritsch221 WRF Ferrier YSU RRTM WRF-NMM Kain FritschGFS WRF Ferrier YSU RRTM GFS model GrellNOG WRF WSM3 YSU RRTM NOGAPS Betts Miller

Real-time Surge Model Grids

Blumberg et al. 1999

SIT Grid

SBSS Grid

NOAA ET Grid

Motivation for Storm Surge Ensembles

MM5 (GRMRF)-NAM WRF(GRYSU)-GFS

0000 UTC April 16th, 2007 – SLP (contour), Temp (shaded) and wind

WRF-GFS

MM5-NAM

OBS

Data and Methods• Data: Nov. 2007 – March 2008 and Oct. 2008 – Dec. 2008 (75

in total)• Deterministic: Mean Error, Root Mean Square Error • Probabilistic: Rank (Talagrand) Histograms, Brier Score, Brier

Skill Score and Reliability Diagrams

• Bias correction was applied after the first month (Nov. 2007).– Use a regression approach of

storm surge observations (> 0 and < 0 m) versus the storm surge mean error.

• Use daily NCEP-NCAR reanalysis to look at the composite flow patterns associated with some of the errors.

Surge Mean Errors

Bias Corr-ALL

NOAA-ET

ALLBias Corr-ALL

NOAA-ET

SBSS

SIT

SIT

Surface Wind Speed Biases

NCEP-NAM

Top 10 Largest Negative Error Days

- 24

• Days determined from calculating the largest 24-48 h negative mean error from the SBSS ensemble member 9a

• Largest negative error day is 11/04/2007 when an extratropical hurricane Noel impacted the region

- 48 h - 24h

Northeast winds occur 24 hours prior to large negative error

Trough moves east/deepens 48 hours prior to large negative errors

0 h

Potential Wave Impacts

Daily Averaged Significant Wave Height, m

Da

ily M

ea

n E

rro

r, m

Significant Wave Height at buoy 44017 vs. 24-48 h Mean Error at Montauk

Top 10 Largest Positive Error Days

• Days determined from calculating the largest 24-48 h positive mean error from the SBSS ensemble member 9a

• Largest negative error period was 12/23 – 12/26/2008

- 24h- 48 h

Pressure gradient strengthens 24 hours prior to large negative error

0 h

RMSE vs Forecast Hour

Bias Corr -ALL

NOAA-ET

SBSS

ALL

SIT

Percentage Best and WorstBEST WORST

NOAA-ET

NOAA-ET

SIT

SIT

SBSS SBSS

Rank Histogram

ALL

ALL-BC

Brier Scores vs Threshold

ALL-BC

ALL

ENS3-BC

ENS3

SBSS

Brier Skill Score (vs SBU CTL)

ALL

ENS3

ENS3-BCALL-BC

SBSS

Reliability Diagrams

SBSSALL

> 0.3 m Surge

> 0.4 m Surge

SBSS

ALL

ALL

SBSS

Reliability Diagram

SBSSALLENS-3

> 0.3 m Surge

> 0.4 m Surge

ALL

ALL

SBSS

SBSS

ENS3

ENS3

Conclusions• All surge models have a slight negative bias overall, which is largest

in the NOAA-ET model. One can not use the last 7-14 days for bias correction, since bias depends on the sign of the surge.

• Stevens Institute (SIT) has greater deterministic accuracy (lower RMSEs) than all the SBSS MM5 and WRF ADCIRC members, which highlights the importance of adding different (and good) ocean models to the ensemble.

• The largest SBSS mean surge errors are dependent on the synoptic flow patterns. Positive surges with nor-easters are underpredicted on average, while offshore flow with an anticyclone to the west favors positive errors (underpredicted “blow-out” conditions).

• Most of the ensemble probabilistic skill and reliability originates from the three different ocean models on average, not from using one ocean model and multi-model atmospheric forcing (MM5 and WRF).

• Recommendation: In addition to coupling NOAA-ET to the SREF (NWS plans to do this soon), added skill can be obtained by also using different ocean (surge) models (ADCIRC, POM, ROMS, FVCOM, etc…).