nmmb tutorial running nmmb ratko vasic ncwcp, april 1 st 2015

NMMB tutorialRunning NMMB

Ratko VasicNCWCP, April 1st 2015

Retrieving model source code (SVN)

• https://svnemc.ncep.noaa.gov/projects/nems/trunk

• Usage:#>svn co https://svnemc.ncep.noaa.gov/projects/nems/trunk

• Directory structure:

trunk:exe/job_notused/READMEREADME.GFSREADME.NMMREADME.TESTSsrc/tests/

https://svnemc.ncep.noaa.gov/projects/nems/trunk

Directory structure contd.

src:atmos/chem/conf/configure*ENS_Cpl/ESMFVersionDefine_ESMF_3.hESMFVersionDefine_ESMF_630rAPI.hESMFVersionDefine_ESMF_NUOPC.hESMFVersionLogic.hMAIN_NEMS.F90makefilemodule_EARTH_GENERIC_COMP.F90module_EARTH_GRID_COMP.F90module_EARTH_INTERNAL_STATE.F90module_MEDIATOR.F90module_NEMS_GRID_COMP.F90module_NEMS_INTERNAL_STATE.F90

atmos:fim/gen/gsm/io/makefilemodule_ATM_GRID_COMP.F90module_ATM_INTERNAL_STATE.F90nmm/phys/post/share/

Compilation on known machines:#> cd ~/trunk/src#> ./configure

Run ./configure with one argument:

'configure 3_wcoss' : ESMF 3.1.0rp2 library on wcoss 'configure 3_zeus' : ESMF 3.1.0r series library (i.e. ESMF 3.1.0rp2, 3.1.0rp5) on zeus 'configure 3_gaea' : ESMF 3.1.0rp2 library on gaea 'configure 3_yellowstone' : ESMF 3.1.0rp2 library on

yellowstone........................................#> ./configure 6_nmm_zeus#> make nmm#> ls –g ../exetotal 30548-rwxr-xr-x 1 ptmp 11801 Mar 28 00:05 mkDepends.pl-rwxr-xr-x 1 ptmp 31248845 Mar 29 07:02 NEMS.x

Configure model• Configure directory#> cd ~/trunk/src/conf#> lsconfigure.nems.Gaea.intelconfigure.nems.Gaea.pgiconfigure.nems.Jet.ifortconfigure.nems.Linux.g95.......modules.nems.Gaea_ESMF_NUOPC*modules.nems.wcoss_ESMF_3*modules.nems.wcoss_ESMF_630rAPI_gsm*modules.nems.wcoss_ESMF_630rAPI_nmm*.......

#> cat modules.nems.Zeus_ESMF_630rAPI_nmm# This script is responsible for loading modules that are# compatible with the ESMF 6.3.0r API.

module purgemodule load mpt intel esmf/6.3.0rp1 netcdf

Configure model (contd.)• Configure file (Snippet form configure.nems.Zeus.intel_12)SHELL = /bin/shinclude $(ESMFMKFILE)ESMF_INC = $(ESMF_F90COMPILEPATHS)ESMF_LIB = $(ESMF_F90LINKPATHS) $(ESMF_F90LINKRPATHS) $(ESMF_F90ESMFLINKLIBS)

NETCDF_INC = -I$(NETCDF)/includeNETCDF_LIB = -L$(NETCDF)/lib -lnetcdf

LIBDIR=/contrib/nceplibs/nwprod/libNEMSIO_INC = -I${LIBDIR}/incmod/nemsioBACIO_LIB = -L${LIBDIR} -lbacio_4EXTLIBS = $(NEMSIO_LIB) \ $(BACIO_LIB) \FC = ifort -lmpi -openmp -convert big_endian -assume byterecl -mkl=sequentialFPP = -fppF77 = ifort -lmpi -openmp -convert big_endian -assume byterecl -mkl=sequentialFREE = -freeFIXED = -fixedR8 = -r8

FINCS = $(ESMF_INC) $(NEMSIO_INC) $(NETCDF_INC)TRAPS =#TRAPS = -g -fno-inline -no-ip -traceback -ftrapuv -fpe0 -ftz -check all -check noarg_temp_created -fp-stack-check

FFLAGS = $(TRAPS) $(FINCS) -fp-model strictOPTS_NMM = -O3

Procedure summary:

• checkout model: svn co https://svnemc.ncep.noaa.gov/projects/nems/trunk

• configure: cd ~/trunk/src ; ./configure 6_nmm_zeus

• make:make nmm

• executable:cd ~trunk/exe ; ls NEMS.x

Regression tests

• More groups work on the same project• Possibility of breaking other’s code

#> cd ~trunk/tests#> ./rt.shUsage: ./rt.sh -c <model> | -f | -s | -l <file> | -m | -h

-c create new baseline results for <model> -f run full suite of regression tests -s run standard suite of regression tests -l runs test specified in <file> -m compare against new baseline results -h display this help

Running model (nmm_run)RUNDIR=_RUND_SRCDIR=_SRCD_mkdir -p $RUNDIRcd $RUNDIR

cp _RTPWD_/NMMB_reg/input_nmmb_regional.d01_nemsio input_domain_01_nemsiocp _RTPWD_/NMMB_reg/nmmb_rst_01_nio_0024h_00m_00.00s restart_file_01_nemsiocp _RTPWD_/NMMB_reg/boco.* .

cp _RTPWD_/data/* .

cp $SRCDIR/tests/solver_state.txt .

elif [ $SCHEDULER = 'pbs' ]; then cp $SRCDIR/tests/nmm_qsub . qsub nmm_qsub

Running model (nmm_qsub)#!/bin/sh#PBS -o out#PBS -e err#PBS -N rt_31806_1#PBS -A nems#PBS -l procs=48#PBS -l walltime=00:15:00

export MPI_GROUP_MAX=96export ESMF_RUNTIME_COMPLIANCECHECK=ON:depth=4

source .../modules.nemsmodule list

cd $PBS_O_WORKDIR

echo "Model started: " `date`mpiexec_mpt -prefix " %g:" -np $PBS_NP ./NEMS.xecho "Model ended: " `date`

Running model (configure_file)##### Grid Decomposition #####inpes: 05 # Number of compute tasks in the I directionjnpes: 09 # Number of compute tasks in the J direction

##### Fundamental Timestep (seconds) #####dt_int: 180 # Integer secondsdt_num: 0 # Numerator of fractional seconddt_den: 1 # Denominator of fractional second

nhours_fcst: 48 # Length of the forecast (hours)minutes_history: 180 # Frequency of history output (minutes)

restart: false # True--> A restarted runminutes_restart: 720 # Frequency of restart output (minutes)

Running model (configure_file)(contd.)

global: true # True--> Global ; False--> Regionalhydro: false # True--> Hydrostatic ; False--> Nonhydrostaticadiabatic: false # True--> Adiabatic ; False--> Diabatic

shortwave: rrtm # Radiation schemeslongwave: rrtmconvection: bmj # Convection schememicrophysics: fer_hires # Microphysics schemeturbulence: myj # Turbulence schemessfc_layer: myj # Surface layer schemesland_surface: noah # Land surface schemes

nemsio_input: falsewrite_hst_bin: true # True--> Write history files in binary formatwrite_hst_nemsio: true # True--> Write history files in NEMSIO formatwrite_rst_bin: true # True--> Write restart files in binary formatwrite_rst_nemsio: true # True--> Write restart files in NEMSIO format

Adding new variable

• Adding local variable to internal state:Add it to a file: ~trunk/src/atmos/nmm/module_SOLVER_INTERNAL_STATE.F90

follow existing example i.e. “MAVAIL”REAL(kind=KFPT),DIMENSION(:,:),ALLOCATABLE :: MAVAIL ALLOCATE(int_state%MAVAIL(IMS:IME,JMS:JME))

REAL(kind=KFPT),DIMENSION(:,:),ALLOCATABLE :: XXX ALLOCATE(int_state%XXX(IMS:IME,JMS:JME))

• Adding variable for history/restart:1) Add it to a file: ~trunk/src/atmos/nmm/module_SOLVER_INTERNAL_STATE.F90

follow existing example i.e. “ACPREC”REAL(kind=KFPT),DIMENSION(:,:),ALLOCATABLE :: ACPREC CALL SET_VAR_PTR(int_state%VARS,NV,'ACPREC' ,int_state%ACPREC ,(/ IMS,JMS /),(/ IME,JME /) )

2) Add in output namelist:~trunk/tests/solver_state.txtfollow same example (“ACPREC”)'ACPREC' H R O - - T 'Accumulated precip (m)'

Restarting model

• While running original run, turn on restart file writing and choose interval in configure_file:

minutes_restart: 120 # Frequency of restart output (minutes)write_rst_nemsio: true # True--> Write restart files in NEMSIO format

• Running restarted run:mv nmmb_rst_01_nio_0016h_00m_00.00s restart_file_01_nemsioIn configure file:restart: true # True--> A restarted run

• Submit model as usual, it will find restart file, read in data and continue from that point

Writing time series

• copy file ~trunk/tests/ts_locations.nml to run directory

• choose variables in solver_state.txt :'PBLH' H R O - - T 'PBL height (m)'

• ts_locations.nml example:&TS_LOCATIONS NPOINTS=2, POINTS_LON=-106.0, -110.0, POINTS_LAT=54.0, 50.0/

nmmb tutorial running nmmb ratko vasic ncwcp, april 1 st 2015

Documents