advances in the unstructured wavewatch iii within earth

Building a Weather-Ready Nation // 1NATIONAL WEATHER SERVICE

NATIONALWEATHERSERVICE

Advances in the Unstructured WAVEWATCH III Within Earth System Modeling Framework

ALI ABDOLALI

NOAA/NCEP/EMC, College Park, MD, USA*[email protected]

mailto:*[email protected]

Building a Weather-Ready Nation // 2NATIONAL WEATHER SERVICENATIONAL WEATHER SERVICE Building a Weather-Ready Nation // 2

vWAVEWATCH III Developmentq Performance Enhancement & New Features (Aron Roland, Tyler Hesser & Jane M. Smith)

ü An Efficient Parallelization Algorithm

ü Implicit Numerical Solver

ü Neumann Boundary Condition

q New Physics (Aron Roland, Tyler Hesser & Jane M. Smith)

ü Vegetation Source Term (VEG1)ü Adapted Depth Breaking Source Terms (Battjes and Janssen-DB1 / Thornton and Guza (1983)-DB2)ü Adapted Triad Interaction Source Term (TR1)q Validationü Lab Cases

ü Real Case Applications (Arslaan Khalid & Celso Ferreira)q Ensemble Modelingü Error Propagation from Atmospheric Modelsü Uncertainty Evaluation

q Wave-Surge Coupling (Andre Van der Westhuysen & Saeed Moghimi, Zaizhong Ma, Avichal Mehra)

q Conclusion & Outlook

Outline


https://github.com/NOAA-emc/ww3

Open Source Software Development Paradigm

Users Mailing List:https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.wwatch3.users

NOAA-EMC/WW3:Ali Abdolali and Jessica MeixnerIfremer: Mickael AccensiERDC/USACE: Tyler HesserUK MetOffice: Chris Bunney Wave Summer School, every July, UMD

https://github.com/NOAA-emc/ww3

https://github.com/NOAA-EMC/WW3

mailto:[email protected]

https://github.com/umr-lops/WW3


https://github.com/erdc/WW3


https://github.com/ukmo-waves/WW3



Performance Enhancement & New Features

Abdolali A., Roland, A., Van Der Westhuysen, A., Meixner, J., Chawla, A., Hesser, T., Smith, J.M. and M. Dutour Sikiric (2020), Large-scale Hurricane Modeling Using Domain Decomposition Parallelization and Implicit Scheme Implemented in WAVEWATCH III Wave Model, Coastal Engineering, 157, 103656, https://doi.org/10.1016/j.coastaleng.2020.103656

The WAVEWATCH III® Development Group (WW3DG), 2019: User manual and system documentation of WAVEWATCH III® version 6.07. Tech. Note 333, NOAA/NWS/NCEP/MMAB, College Park, MD, USA, 326 pp. + Appendices

https://doi.org/10.1016/j.coastaleng.2020.103656


Domain Decomposition (DD) vs. Conventional

Decomposition method in WW3 (Card Deck- CD)

Chawla & Tolman, April 2007 WISE 2007 11/34Chawla etal, Nov 12, 2007 10th Intl. Wave Hindcasting and Forecasting Workshop 11/20

NMWW3

Resolution in minutes of the 8 grids making up the multi-grid model

3• WW3 was initially developed for

structured grids.• The multi grid capabilities were

suitable for multi scale applications with structured grids.

Multi_1 setup


• Extending the capabilities of the model to have unstructured grids with CD decomposition did not impose considerable difficulties for coarse grids with less than ~2 M elements.

• Card Deck is not efficient for high resolution grid (street level) with a large number of elements.

• Coupling with Surge Model in nearshore region requires more efficient decomposition.

Parallelization

Abdolali et al. 2020


No.Cores /(No.Dir # No.Freq)0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 3

Per

form

ance

[Rea

lTim

e/

Com

puta

tion

alTim

e]

0.1

0.3

0.5

0.7

0.91

3

5

7

910

30

50

Regular 1/12°, Explicit CDRegular 1/60°, Explicit CD"xmin : 200 m, Explicit CD"xmin : 200 m, Explicit DD"xmin : 200 m, Implicit DD"xmin : 10 m, Explicit CD"xmin : 10 m, Explicit DD"xmin : 10 m, Implicit DD

Performance

Abdolali et al 2020

SL18 mesh (9 M nodes)

Hurricane Ike (12 days)

CPU: 2000

Computational time: 7 hrs

WW3 limit prior

to recent

developments


New Physics


Vegetation Source Term in WW3• Wave-vegetation interaction based on Mendez and Losada (2004)

!d= drag coefficient, bv= stem diameter, "= relative stem length, #=vegetation density

• Implementation called with VEG1 switch

• Operates in serial or parallel, implicit or explicit, structured or

• unstructured grids

• Called after depth limited breaking but before bottom interactions

• Spatially and temporally variable vegetation coefficients read with

ww3_prnc, or homogeneous variables in ww3_shel

manuscript submitted to Enter journal name here

2 Formulation

Without vegetation, wave energy flux remains constant if no energy is lost or gained.In the presence of vegetation, the wave energy flux, following Dalrymple et al. [1984],Kobayashi et al. [1993] and Mendez and Losada [2004] becomes:

@F

@x= �✏⌫ ! @

@x

hE.cg

i= �✏⌫ (1)

✏⌫ =1

2p⇡⇢CDb⌫N

⇣kg

2�

⌘3 sinh3(k↵h) + 3 sinh(k↵h)

3k cosh3(kh)H

3rms

(2)

where wave energy is defined as

E =1

8⇢gH

2 (3)

Combining Eqs. 1-3:

@H2rms

@x=

�✏⌫

18⇢gcg

=2

3p⇡CDb⌫Nk

sinh3(k↵h) + 3 sinh(k↵h)

[sinh(2kh) + 2kh] sinh(kh)H

3rms

(4)

The Drag Coe�cient is a function of wave parameters and vegetation species. Keule-gen and Carpenter number is a dimensionless number which describes the relative im-portance of the drag force over inertia for vertical obstacle in an oscillating flow. Therelation between CD and KC number should be evaluated based on experiments.

KC =umaxTp

b⌫(5)

where

umax =�a

tanh(kh)=

⇣⇡HsTp

⌘

tanh( 2⇡hL

)(6)

–4–


3 Implementation in WW3

The mean rate of energy dissipation per unit horizontal area due to wave damp-ing by vegetation is:

✏⌫ =1

2p⇡⇢CDb⌫N

⇣kg

2�


3k cosh3(kh)H

3rms

(7)

A spectral version to be implemented in WW3 is divided by �⇢g and written inspectral/directional form:

Sveg(�, ✓) =Dtot

Etot

E(�, ✓) (8)

Dtot = � 1

2gp⇡C̄Db⌫N

⇣k̄g

2�̄

⌘3 sinh3(k̄↵h) + 3 sinh(k̄↵h)

3k̄ cosh3(k̄h)H

3rms

(9)

where H2rms

= 8Etot, the mean frequency �̄,mean wave number k̄ and mean en-ergy are given by:

�̄ =⇣ 1

Etot

Z 2⇡

0

Z 1

0

1

�E(�, ✓)d�d✓

⌘�1(10)

k̄ =⇣ 1

Etot

Z 2⇡

0

Z 1

0

1pkE(�, ✓)d�d✓

⌘�2(11)

Etot =

Z 2⇡

0

Z 1

0E(�, ✓)d�d✓ (12)

Finally,

Sd,veg = �r

2

⇡g2C̄Db⌫N

⇣k̄

�̄


3k̄ cosh3(k̄h)

pEtotE(�, ✓) (13)

4 To do list in WW3

• Add the formulation (Dalrymple et al. 1994)• Add switch VEG1• Include VEG in ww3 prnc and ww3 prep to read Vegetation parameters.• Include VEG1 in ww3 shel and ww3 multi

–5–


3 Implementation in WW3

The mean rate of energy dissipation per unit horizontal area due to wave damp-ing by vegetation is:

✏⌫ =1

2p⇡⇢CDb⌫N

⇣kg

2�


3k cosh3(kh)H

3rms

(7)

A spectral version to be implemented in WW3 is divided by �⇢g and written inspectral/directional form:

Sveg(�, ✓) =Dtot

Etot

E(�, ✓) (8)

Dtot = � 1

2gp⇡C̄Db⌫N

⇣k̄g

2�̄


3k̄ cosh3(k̄h)H

3rms

(9)

where H2rms

= 8Etot, the mean frequency �̄,mean wave number k̄ and mean en-ergy are given by:

�̄ =⇣ 1

Etot

Z 2⇡

0

Z 1

0

1

�E(�, ✓)d�d✓

⌘�1(10)

k̄ =⇣ 1

Etot

Z 2⇡

0

Z 1

0

1pkE(�, ✓)d�d✓

⌘�2(11)

Etot =

Z 2⇡

0

Z 1

0E(�, ✓)d�d✓ (12)

Finally,

Sd,veg = �r

2

⇡g2C̄Db⌫N

⇣k̄

�̄


3k̄ cosh3(k̄h)

pEtotE(�, ✓) (13)

4 To do list in WW3

• Add the formulation (Dalrymple et al. 1994)• Add switch VEG1• Include VEG in ww3 prnc and ww3 prep to read Vegetation parameters.• Include VEG1 in ww3 shel and ww3 multi

–5–

Dalrymple et al. (1984)Mendez and Losada 2004

Building a Weather-Ready Nation // 10NATIONAL WEATHER SERVICE US Army Corps of Engineers

UNCLASSIFIED

UNCLASSIFIED

Laboratory Test� 1.5 m-wide wave flume

• 64.1 m long, 1.5 m deep� Wave and Water Levels

• Depths: 30.5 cm, 45.7 cm, 53.3 cm• ls/h ratios of 1.0 (emergent), 0.91, 0.78• Irregular waves

► Tp ~ 1.25 s to 2.25 s► Hm0 ~ ranging from 5.0 cm to 19.2 cm

� Polyolefin tubing• 6.4 mm diameter• 41.5 cm stem length• densities of 100, 200, and 400 stems/m2

►correspond to element spacing of 10.0 cm, 7.1 cm, and 5.0 cm

Anderson & Smith 2014

In collaboration with USACETyler Hesser; Jane M. Smith; Mary B. Anderson Lab Case


Author's personal copy

toward the shoreline). WGs 4–13 were installed along the center of thechannel at x = 26.0, 26.9, 27.4, 27.9, 28.5, 29.5, 31.0, 32.7, 34.4, and36.2 m, in that order (Fig. 1). WGs were calibrated daily to minimizecalibration error and to ensure that wave heights were bounded bythe calibrated range.

3. Wave height attenuation analysis

Reflection coefficients for the flume were estimated using a three-gauge separation technique based on the method of Goda and Suzuki(1976). The wave reflection analysis was performed using WGs 1–3,and reflection coefficient Kr varied from 5% to 12%. There was little

change in Kr between the setup with and without vegetation. Themean reflection coefficients were 8.2%, 8.3%, and 8.7% for N=0, 200,and 400 stems/m2, respectively. As Kr remained relatively small, theeffect of wave reflection on the results was considered negligible. Wavereflection from the absorber-covered back slope was not calculated.

The effects of stem density, submergence, incident wave height,and peak wave period on wave propagation were evaluated usingmeasurements of wave height. The wave spectral density S(f) at eachgauge was extracted from the demeaned water surface elevation timeseries with a fast Fourier transformation. The data were broken intosegments of 2048 points, and the spectra were smoothed by averagingfive neighboring frequency bands. The resulting resolution bandwidthwas 0.061 Hz, and spectral estimates had 60° of freedom. The localzero-moment wave height H was estimated from the wave spectra

Fig. 2. Schematic of idealized vegetation stem configurations. The circles areN= 200 stems/m2. The addition of a center stem, indicated by the square, increases thedensity to N=400 stems/m2.

Fig. 3. Installed idealized vegetation bed (N= 400 stems/m2).

Table 3Incident single- and double-peaked irregular wave conditions measured at the beginningof the vegetation (WG 5).

Wavetype

Depthh (cm)

WaveheightH0 (cm)

PeakperiodTp (s)

PeakwavelengthLp (m)

ls/h(−)

H0/h(−)

h/Lp(−)

Single-peaked

53.3 11.1 ± 0.07 1.5 2.89 0.78 0.21 0.1853.3 11.0 ± 0.10 1.75 3.53 0.78 0.21 0.1553.3 11.2 ± 0.06 2.0 4.16 0.78 0.21 0.1345.7 8.1 ± 0.03 1.5 2.74 0.91 0.18 0.1745.7 10.9 ± 0.05 1.5 2.74 0.91 0.24 0.1745.7 13.9 ± 0.07 1.5 2.74 0.91 0.30 0.1745.7 5.0 ± 0.03 2.0 3.91 0.91 0.11 0.1245.7 10.7 ± 0.04 2.0 3.91 0.91 0.23 0.1245.7 15.3 ± 0.10 2.0 3.91 0.91 0.33 0.1245.7 19.2 ± 0.14 2.0 3.91 0.91 0.42 0.1230.5 11.3 ± 0.09 1.25 1.88 1.36 0.37 0.1630.5 11.0 ± 0.11 1.5 2.36 1.36 0.36 0.1330.5 11.2 ± 0.10 1.75 2.82 1.36 0.37 0.1130.5 11.1 ± 0.16 2.0 3.28 1.36 0.36 0.0930.5 11.2 ± 0.13 2.25 3.73 1.36 0.37 0.08

Double-peaked

53.3 13.7 ± 0.04 1.25/2.0 – 0.78 0.26 –

53.3 10.9 ± 0.03 1.25/2.0 – 0.78 0.20 –

45.7 13.6 ± 0.04 1.25/2.0 – 0.91 0.30 –

45.7 10.7 ± 0.05 1.25/2.0 – 0.91 0.23 –

30.5 13.0 ± 0.18 1.25/2.0 – 1.36 0.43 –

30.5 10.7 ± 0.14 1.25/2.0 – 1.36 0.35 –

85M.E. Anderson, J.M. Smith / Coastal Engineering 83 (2014) 82–92

26 28 30 32 34 36Distance from Wavemaker [m]

0.08

0.09

0.1

0.11

0.12

0.13

Hs [m

]

N=0 N=200 N=400 Obs. (Solid); Model (dashed)

Vegetationh = 53.3 cm

27 28 29 30 31 32 33 34 35 36Distance from Wavemaker [m]

1

1.5

2

2.5

T p [s]

N = 200

bv = 6.4 mm

ls = 41.5 cm

ls/h = 0.77861

Cd = 0.36347

KC = 51.7581

N = 400

bv = 6.4 mm

ls = 41.5 cm

ls/h = 0.77861

Cd = 0.33961

KC = 51.8769


0.1

0.12

0.14

0.16

0.18

Hs [m

]




1

1.5

2

2.5

T p [s]

N = 200

bv = 6.4 mm

ls = 41.5 cm

ls/h = 0.9081

Cd = 0.34303

KC = 85.7424

N = 400

bv = 6.4 mm

ls = 41.5 cm

ls/h = 0.9081

Cd = 0.35392

KC = 85.78


0.06

0.08

0.1

0.12

Hs [m

]




1

1.5

2

2.5

T p [s]

N = 200

bv = 6.4 mm

ls = 41.5 cm

ls/h = 1.3607

Cd = 0.36115

KC = NaN

N = 400

bv = 6.4 mm

ls = 41.5 cm

ls/h = 1.3607

Cd = 0.43131

KC = NaN


Magothy Bay

Building a Weather-Ready Nation // 15NATIONAL WEATHER SERVICEFigure: Eastern Shore, Virginia

Figure: Magothy Bay, Virginia

For Official Use Only - Pre-decisional information


Validation (transect 3)



x [m]0 5 10 15 20 25 30

h [m

]

-0.8

-0.6

-0.4

-0.2

0

0.2Obs

f [Hz]0 0.5 1 1.5 2 2.5 3

[m2/

Hz]

#10-3

0

2

4

6

8SET 1ASET 1BSET 1C

Depth breaking/triad interactionLaboratory case (boers 1996) ww3_tp2.19

A new Neumann boundary condition is added to WW3


Large Scale ApplicationIsabel 2003

Ike 2008Sandy 2012Irma 2017

Florence 2018


HURRICANE Irma Sep. 2017

HSOFS Mesh~1.8 M nodes~3.5 M elementslowest resolution.: ~200m



HWRF

WW3WW3 v6.07Parallelization: Domain DecompositionScheme: ImplicitAbdolali et al 2019

Three moving nested gridsData AssimilationOcean Coupling40 Ensemble Members

DeterministicRun



Validating HWRF and WW3 ensembles



U10: Ensemble Mean and StDev


Z. Ma, B. Liu, A. Mehra, A. Abdolali, A. van derWesthuysen, S. Moghimi, S. Vinogradov, Z. Zhang, L. Zhu, K. Wu, R. Shrestha, A. Kumar, V. Tallapragada, N. Kurkowski(2020), Investigating the impact of High-resolution Land-sea Masks on Hurricane Forecasts in HWRF, Atmosphere, 2020, (in press)


Hs: Ensemble Mean and StDev


Ali Abdolali, Andre van der Westhuysen, Zaizhong Ma, Avichal Mehra, Aron Roland and Saeed Moghimi (2020) Evaluating the Accuracy andUncertainty of Atmospheric and Wave Model Hindcasts During Severe Events Using Model Ensembles, Ocean Dynamics (Under Review).


Validation data(waves/wind)


USGS Rapid Deployment

HURRICANE Florence Sep. 2018


WAVEWATCH IIIHs

HWRFU10



Results – NDBC



Results – USGS

Photo of wave height sensor at Stone Chimney Rd at Lockwoods Folly Inlet, Brunswick County, NC, 09/12/2018. Photograph by Anthony Gotvald, USGS GA.



Wave-Surge CouplingCOASTAL Act

Alaska Coastal Ocean Forecast System (ALCOFS)

https://gm-ling.github.io/ALCOFS-R/


COASTAL Act: Named Storm Event Model• An ADCIRC-WW3 application

(“App”) in NUOPC/NEMS

• Based on NEMS interfaces or

“Caps” and ESMF framework

• One-way atmospheric forcing

from gridded data file (HWRF

model)

• Two-way exchange between

ADCIRC and WW3 models

• To include river discharge from

NWM

Moghimi et al. 2020

Bakhtyar et al. 2020

Atmospheric Data:

HWRF/HRRR

Storm Surge

Model: ADCIRC

Wave Model:

WAVEWATCH III

Wind speed

Water levels, currents

Hydrologic Model:

National Water

Model

Radiation stress

Pressure at MSL and surface, Downward Short/Long-Wave Radiation Flux, Precipitation Rate, Humidity, Temperature, Wind speed

DischargeWater levels, currents Wind speed

pressure at MSL

Named Storm Event Model (NSEM)


Sign Wave Height (Hs)Coupled

∆Hs: Coupled - Stand Alone WW3

Galveston Bay detail


Max ∆Hs

Max ∆Hs

Hurricane Ike (Sept 3-14, 2008)ADCIRC-WW3: Wave height validation (Galveston)

Moghimi, S.; Van der Westhuysen, A.; Abdolali, A.; Myers, E.; Vinogradov, S.; Ma, Z.; Liu, F.; Mehra, A.; Kurkowski, N. Development of an ESMF Based Flexible Coupling Application of ADCIRC and WAVEWATCH III for High Fidelity Coastal Inundation Studies. J. Mar. Sci. Eng. 2020, 8, 308.


Remaining issues

§ The memory management in WW3, mostly relying on Global Arrays, are now localized throughout the source code to be efficiently utilized within the domain decomposition parallelization concept. Localized in the sense the each CPU just knows about his domain.

§ This provides us the needed flexibility on Different HPC architectures and provides the possibility to run 2-way coupled within ESMF & ADCRIC multi-million grid points unstructured grids.

§ Near shore physics Implemented (Vegetation Source Term, Adapted Triad Interaction and Depth Breaking Source terms).

§ Validation for Laboratory Cases and Large Scale Application for Hurricane Irma 2017 (40 ensembles), Hurricane Florence 2018 are conducted.

§ Fully Coupled Wave-Surge (WW3-ADCIRC) is developed, one way coupled to NWM (Fully coupled Wave-Surge-Riverine models under development).


References• Abdolali A., Roland, A., Van Der Westhuysen, A., Meixner, J., Chawla, A.,

Hesser, T., Smith, J.M. and M. Dutour Sikiric (2020), Large-scale HurricaneModeling Using Domain Decomposition Parallelization and Implicit SchemeImplemented in WAVEWATCH III Wave Model, Coastal Engineering, 157,103656, https://doi.org/10.1016/j.coastaleng.2020.103656

• The WAVEWATCH III® Development Group (WW3DG), 2019: User manual andsystem documentation of WAVEWATCH III® version 6.07. Tech. Note 333,NOAA/NWS/NCEP/MMAB, College Park, MD, USA, 326 pp. + Appendices

• Ali Abdolali, Andre van der Westhuysen, Zaizhong Ma, Avichal Mehra, AronRoland and Saeed Moghimi (2020) Evaluating the Accuracy and Uncertainty ofAtmospheric and Wave Model Hindcasts During Severe Events Using ModelEnsembles, Ocean Dynamics (Under Review).

• Z. Ma, B. Liu, A. Mehra, A. Abdolali, A. van derWesthuysen, S. Moghimi, S.Vinogradov, Z. Zhang, L. Zhu, K. Wu, R. Shrestha, A. Kumar, V. Tallapragada,N. Kurkowski (2020), Investigating the impact of High-resolution Land-seaMasks on Hurricane Forecasts in HWRF, Atmosphere, 2020, (in press)

• Bakhtyar, R., Maitaria, K., Velissariou, P., Trimble, B., Mashriqui, H., Moghimi,S., Abdolali, A., Van der Westhuysen, A.J., Ma, Z., Clark, E.P. and Flowers, T.(2020). A new 1D/2D Coupled Modeling Approach for a Riverine-EstuarineSystem under Storm Events: Application to Delaware River Basin. Journal ofGeophysical Research: Oceans, doi.org/10.1029/2019JC015822

• Moghimi, S.; Van der Westhuysen, A.; Abdolali, A.; Myers, E.; Vinogradov, S.;Ma, Z.; Liu, F.; Mehra, A.; Kurkowski, N. Development of an ESMF BasedFlexible Coupling Application of ADCIRC and WAVEWATCH III for High FidelityCoastal Inundation Studies. J. Mar. Sci. Eng. 2020, 8, 3

08. https://doi.org/10.3390/jmse8050308• Moghimi, S., Vinogradov, S., Myers, E. P., Funakoshi, Y., Van der Westhuysen,

A. J., Abdolali, A., Ma, Z., & Liu, F. (2019). Development of a flexible couplinginterface for ADCIRC model for coastal inundation studies. NOAA technicalmemorandum NOS CS; 41, doi.org/10.25923/akzc-kc14.

Global Unstructured Grid 5M node, 6 km minimum resolution (W/ W. Pringle & J. Westerink - ND)

https://doi.org/10.1016/j.coastaleng.2020.103656

https://doi.org/10.1029/2019JC015822

https://doi.org/10.3390/jmse8050308

http://doi.org/10.25923/akzc-kc14

advances in the unstructured wavewatch iii within earth

Documents