-
1
DevelopmentalTestbedCenterReportAOP2016Activities
1April2016–31March2017
1 Introduction
TheDevelopmentalTestbedCenter(DTC)isadistributedfacilitywithcomponentsattheNationalCenterforAtmosphericResearch(NCAR)andtheNationalOceanicandAtmosphericAdministration(NOAA)EarthSystemResearchLaboratory(ESRL)GlobalSystemsDivision(GSD).ThepurposeoftheDTCistoprovidealinkbetweentheresearchandoperationalcommunitiessoNumericalWeatherPrediction(NWP)researchcanbeefficientlytransferredtooperations.Inaddition,theDTCprovidestheresearchcommunityaccesstothelatestoperationalNWPcodepackagesforresearchapplications.TheDTCmeetsitsgoalsby:maintainingandsupportingcommunitycodepackagesthatrepresentthelatestNWPtechnology,performingextensivetestingandevaluation(T&E)ofnewNWPtechnology,developingandmaintainingastate-of-the-artverificationpackage,andconnectingtheNWPresearchandoperationalcommunitiesthroughworkshopsanditsvisitorprogram.Overthepastyear,DTCactivitieswereorganizedintofivefocusareas:Verification,DataAssimilation(DA),Hurricanes,RegionalEnsemblesandGlobalModelTestBed(GMTB).
FundingfortheDTCisprovidedbyNOAA’sNationalWeatherService(NWS)andOfficeofOceanicandAtmosphericResearch(OAR),theAirForce(AF),NCAR,andtheNationalScienceFoundation(NSF).ThisreportprovidesadescriptionoftheactivitiesundertakenbytheDTCbetween1April2016and31March2017.TheseactivitiesincludethosedescribedintheDTC2016AnnualOperatingPlan(AOP),aswellasafewcarry-overactivitiesfromtheDTCAOP2015.TheperformanceperiodfortheGMTBis1Julythrough30June.ThisreportalsoprovidesastatusupdateontheGMTBactivitiesthrough31March2017.
1.1 DTCManagement
TheexternalmanagementstructureoftheDTCincludesanExecutiveCommittee(EC),aManagementBoard(MB),andaScienceAdvisoryBoard(SAB).Currentmembershipsarelistedbelow.TheMBandECareresponsibleforapprovingtheDTCAnnualOperatingPlan(AOP),whichdefinestheworktobeundertakenbytheDTCinagivenyear,whereastheSABischargedwithprovidingtheDTCDirectorwithadviceonfuturedirectionsoftheDTCandreviewingproposalssubmittedtotheDTCVisitorProgram.
TheDTChosteditsannualSABmeetingatNCAR’sFoothillsCampusinBoulder,CO,on14-15September2016.ThepurposeofthismeetingwastodiscussstrategicfuturedirectionsfortheDTC.Participationinthisannualmeetingwasstrongerthanusual,with16SABmembersparticipatingin-personandthe17thmemberparticipatingremotely.Day1ofthemeetingconsistedofbriefingsfromtheDTC’soperationalpartners,anoverviewpresentation,highlightsbytaskareaandabreakoutgroupdiscussiononbuildingcommunity.Day2startedoffwithapresentationonDTC’scommunityinteractions,followedbybreakoutgroupdiscussionsbytaskarea.ThemeetingwrappedupwithabriefinganddiscussionofSABrecommendations.GeneralandtaskspecificrecommendationsstemmingfromthismeetingarepostedontheDTCwebsite(http://www.dtcenter.org/SAB/SAB-recommendations-Sept2016.pdf).
InMarch2016,theNationalCentersforEnvironmentalPrediction(NCEP)director,BillLapenta,chargedtheUCARCommunityAdvisoryCommitteeforNCEP(UCACN)withconductingareviewoffour
-
2
representativetestbedsmanagedorco-managedbyNCEPinassociationwithoneoftheNCEPCenters.TheDTCwasoneoftheselectedtestbeds.ThereviewcommitteefortheDTCconsistedof:ShuyiChen(chair–UniversityofMiami),PeterNeilley(TheWeatherCompany),LanceBosart(StateUniversityofNewYork[SUNY]-Albany)andAndyBrown(UKMetOffice).Aspartofitsreview,theDTCmanagementhostedaDTCOverviewwebinaron30Septembertoprovidebackgroundinformationtotheexternalreviewpanel,followedbyanonsitevisitbythereviewcommitteeon6-7October2016.TheonsitevisitconsistedofpresentationsbyDTCmanagementandtaskleadsandone-on-onediscussionsbetweenthereviewcommitteeandacross-sectionofDTCstaff.
DTCExternalManagementCommittees:
ExecutiveCommittee ManagementBoard JimHurrell NCAR JoshHacker NCAR MikeFarrar NOAA/NWSBillLapenta NOAA/NWS JoeKlemp NCAR FredToepfer NOAA/NWSRalphStoffler AirForce MichaelGremillionAirForce StanBenjamin NOAA/OAR/ESRLKevinKelleher NOAA/OAR JeffCetola AirForce Jian-WenBao NOAA/OAR/ESRL
ScienceAdvisoryBoardAdamClark NationalSevereStormsLaboratory(NSSL)RobertFovell SUNY–AlbanyKristenCorbosiero SUNY–AlbanySharanyaMajumdar UniversityofMiamiKathyGilbert NationalCentersforEnvironmentalPrediction(NCEP)/WeatherPredictionCenterGeoffDiMego NCEP/EnvironmentalModelingCenter(EMC)JenniEvans PennsylvaniaStateUniversityDavidGochis NCARS.R.Gopalakrishnan NOAA/AtlanticOceanographicandMeteorologicalLaboratory(AOML)DavidVollmer UnitedStatesAirForce(USAF)AcademyTomAuligne JointCenterforSatelliteDataAssimilation(JCSDA)TimWhitcomb NavalResearchLaboratory(NRL)BradColman ClimateCorporationZhuoWang UniversityofIllinoisKellyMahoney CooperativeInstituteforResearchinEnvironmentalSciencesRussSchumacher ColoradoStateUniversityKayoIde UniversityofMaryland
Overthepastyear,theDTChostedtwoMBmeetings:atwo-hourconferencecallon31October2016anditsannualin-personMBmeetingon18-19January2017atNCAR’sFoothillsCampusinBoulder,CO.ThefocusoftheOctoberconferencecallwastoreportonrecommendationsfromtheSABanddiscussinitialguidanceonprioritiesforAOP2017.ThepurposeoftheJanuarymeetingwastodiscussandrefinetheDTC’sproposalforAOP2017anddiscussnominationsforSABmemberstoreplacecurrentmemberswhosetermexpiresinJune2017.
DTCmanagementparticipatedintwoDTCECconferencecalls(17May2016,29September2016)andtheannualin-personECmeetingatNWSHeadquartersinSilverSpring,MD,on3March2017.RecentDTCaccomplishments,recommendationsfromtheSAB,proposedactivitiesforAOP2017,andthefuturedirectionoftheDTCwerediscussedatthein-personmeeting.TheECalsoapprovedtheDTCDirector’sproposaltorotateoffsixSABmemberswhosetermsexpireinJune2017(RobertFovell,KristenCorbosiero,SharanyaMajumdar,GeoffDiMego,JenniEvans,andKellyMahoney)andaddsixnewSABmembers(three-yeartermbegins1July2017).ThesixnewSABmembersare:VincentLarson
-
3
(UniversityofWisconsin-Milwaukee),XuguangWang(UniversityofOklahoma),TomGalarneau(UniversityofArizona),PhilPegion(ESRL/PhysicalSciencesDivision),RustyBenson(GeophysicalFluidDynamicsLaboratory[GFDL])andVijayTallapragada(NCEP/EMC).QuarterlyreportsontheprogresstodatewerealsopreparedforeachactivityanddistributedtotheECandMBmembers.
1.2 CommunityInteractions
MaintainingstrongtiestoboththeresearchandoperationalNWPcommunitiesiscriticaltotheDTC’sabilitytosuccessfullymeetitsmission.Overthepastyear,strongtieswiththeoperationalcommunityweremaintainedthroughtheDTC’sinteractionswithourpartnersattheoperationalcenters(i.e.,EMCandAirForce)bothatthemanagementlevelandthroughourteamleadinteractionswiththeappropriateteamleadsand/orfocalpointsattheoperationalcenters.TheDTCalsoworkedtowardstrengtheningitstiestothebroaderresearchcommunitythroughworkshops,tutorialsandtheDTCVisitorProgram.InformationonDTC-sponsoredtutorialsisprovidedinSection2.3.TheDTCalsoengagesthecommunitythroughthedistributionofitsnewsletter“Transitions”thatservesasaforumfortheresearchandoperationalcommunitiestoshareinformation.Overthepastyear,theDTCdistributedthreeissuesofTransitions.AllissuesofTransitionscanbeaccessedat:http://www.dtcenter.org/newsletter/.
1.2.1 CommunityOutreachEvents
InJune2016,theDTCco-hostedwithNCAR’sMesoscaleandMicroscaleMeteorology(MMM)Laboratorythe17thWeatherResearchandForecasting(WRF)Users’WorkshopatNCAR’sCenterGreenCampusinBoulder,CO.ThefirstdayconsistedoflecturesonWRFsoftwareandbestcomputingpractices,followedbya3-dayworkshopconsistingof67talksandnearly80posters.Thelastdayconsistedoffourmini-tutorialsontheMesoscaleModelEvaluationTestbed(MMET),VisualizationandAnalysisPlatformforOcean,AtmosphereandSolarResearchers(VAPOR),ensemblepredictionandNCARCommandLanguage(NCL).TheMMETinstructionalsessionwasorganizedandconductedbyDTCstaff.About190peoplefrom19countriesattendedtheworkshop(http://www2.mmm.ucar.edu/wrf/users/workshops/WS2016/WorkshopPapers.php).
AlsoinJune2016,theDTCco-hostedwithNCEP/EMCthe7thEnsembleUsers’WorkshopattheNOAACenterforWeatherandClimatePrediction(NCWCP)inCollegePark,MD.Thisworkshopattractedmorethan150participantsrepresentingabroadcross-sectionofexpertiserangingfromensembledeveloperstotheendusersofensembleproducts.Theover-archinggoaloftheworkshopwasdetermininghowtosupporttheNWSasitmovestowardaseamlessoperationalensembleforecastsystematstorm-toglobal-scales,fromshort-termtoseasonaltimescales,usingatmosphere-onlytoocean-waveandcoupledensemblepredictionsystems.Theworkshopconsistedoforalandposterpresentations,aswellasopendiscussionovera3-dayperiod.Workshoppresentationsandalistofattendeesarepostedathttp://www.dtcenter.org/events/workshops16/ensembles/.AreportontheworkshopisalsoavailableontheDTCwebsite(http://www.dtcenter.org/eval/ensembles/).
1.2.2 DTCVisitorProgram
TheDTCVisitorProgramsupportsvisitorstoworkwiththeDTCtotestnewforecastingandverificationtechniques,modelsandmodelcomponentsforNWP.Thegoalistoprovidetheoperationalweatherpredictioncenters(e.g.,NCEPandAirForce)withoptionsfornear-termadvancesinoperationalweatherforecastingandtoprovideresearcherswithNWPcodesthatrepresentthelatestadvancesintechnology.Italsooffersanopportunityforvisitorstointroducenewtechniquesthatwouldbeof
-
4
particularinteresttotheresearchcommunityintothepublicly-releasedsoftwaresystemssupportedbytheDTC.
Overthepastyear,theDTCprovidedsupportforonevisitorprojectselectedin2014(seeTable1.2.2-1),fourprojectsselectedin2015(seeTable1.2.2-2)andfourprojectsselectedforfundingin2016(seeTable1.2.2.-3).Duringthistimeperiod,Dr.Roebbercompletedhisprojectandsubmittedhisprojectreport,soallprojectsawardedin2014arenowcomplete.Mr.Otkinandcolleaguesalsocompletedtheirprojectandsubmittedtheirprojectreport.Theremaining2015projectsarenearingcompletion,withonlytheprojectreportsremaining.Forthe2016projects,Mr.IaconoandMr.HendersonhavecompletedtheirprojectandsubmittedtheirprojectreportandonlythereportremainsforDr.Niyogi’sgraduatestudentprojectbySubashiniSubramanian.Theotherprojectsawardedin2016arewellunderwayandtheprojectsawardedin2017areeitherunderwayoranticipatedtogetunderwayinthecomingmonths.Allvisitorprojectreportsreceivedoverthepastyearareavailableonthe“VisitorProgram”portionoftheDTCwebsite(http://www.dtcenter.org/visitors/).Inadditiontoprojectreportsandrelevantcodedeliverables,theDTCstartedschedulingvisitorseminarsduringtheirfinalDTCvisitthatareopentolocalareascientistsaswellasremoteparticipants.FeedbackfromboththevisitorsandlocalareascientistsaboutthisincreasedexposureoftheDTCvisitorprojectshasbeenoverwhelminglypositive.Oneadditionalprojectisintheprocessofbeingawarded.
Table1.2.2-1.2014VisitorProjectsPI Institution ProjectTitle
PaulRoebberUniversityofWisconsin-Milwaukee
Demonstrationproject:Developmentofalargememberensembleforecastsystemforheavyrainfallusingevolutionaryprogramming
Table1.2.2-2.2015VisitorProjectsPI Institution ProjectTitle
JasonOtkinUniversityofWisconsin-Madison
ObjectbasedverificationfortheHRRRmodelusingsimulatedandobservedGOESinfraredbrightnesstemperatures
GretchenMullendore(MariuszStarzec)
UniversityofNorthDakota
Mesoscalemodelintercomparisonatconvection-allowingresolutionusingMODE
DevNiyogi(XingLiu) PurdueUniversity
ImprovingWRFweatherforecastthroughenhancedrepresentationofcropland-atmosphereinteractions
JoelBedardUniversityofQuebec-Montreal
Implementationandvalidationofageo-statisticalobservationoperatorfortheassimilationofnear-surfacewindsinGSI
Table1.2.2-3.2016VisitorProjectsPI Institution ProjectTitle
MichaelIacono/JohnHenderson
AtmosphericandEnvironmental
Research
TestingrevisionstoRRTMGcloudradiativetransferandperformanceinHWRF
RobertFovell SUNY-Albany ImpactofplanetaryboundarylayerassumptionsonHWRFDevNiyogi(Subashini
Subramanian)PurdueUniversity DevelopingcapabilityinidealizedHWRFforassessingtheimpactoflandsurfaceontropicalcycloneevolution
ShaowuBao CoastalCarolinaUniversityEvaluationofthemicrophysicsschemeinHWRF2016versionwithremote-sensingdata
-
5
Table1.2.2-4.2017VisitorProjectsPI Institution ProjectTitle
PatrickSkinner UniversityofOklahoma
QuantifyingthevalueofradardataassimilationintheCommunityLeveragedUnifiedEnsembleusingobject-basedverificationmethods
KarinaApodaca ColoradoStateUniversityR2OtransitionoftheGOES-RGLMlightningassimilationcapabilityinGSIforuseintheNCEPGDAS
BillGallus IowaStateUniversityUseoftheCLUEtoexamineimportanceofmixedphysicsinensembles
PaulRoebberUniversityofWisconsin-Milwaukee
AnAdaptiveBayesianModelCombination(BMC)PostProcessorfortheHRRR-TLEForecastSyste
JiangZhu UniversityofAlaska-FairbanksAdvancedDataAssimilationTechniquesAppliedtoaRegionalHighResolutionRapidRefreshModelinAlaska(HRRR-Alaska)
Ting-ChiWu ColoradoStateUniversity
EvaluationoftheNewlyDevelopedObservationOperatorsforAssimilatingSatelliteCloudandPrecipitationObservationsinGSIwithintheHWRFsystem
2 SoftwareSystems
Toserveasabridgebetweenoperationsandresearch,theDTCprovidesaframeworkforthetwocommunitiestocollaborateinordertoacceleratethetransitionofnewscientifictechniquesintooperationalweatherforecasting.Thisframeworkisbasedonsoftwaresystemsthatareasharedresourcewithdistributeddevelopment.Thecurrentoperationalsystemsareasubsetofthecapabilitiescontainedinthesesoftwaresystems.Ongoingdevelopmentofthesesystemsismaintainedunderversioncontrolwithmutuallyagreeduponsoftwaremanagementplans.TheDTCcurrentlyworkswiththefollowingsoftwaresystems:
• WeatherResearchandForecasting(WRF)–NWPmodel+pre-andpost-processors• HurricaneWRF(HWRF)-setoftoolsfortropicalstormforecasting,includingacoupled
atmosphereandoceansystem• UnifiedPost-Processor(UPP)• GridpointStatisticalInterpolation(GSI)dataassimilation(DA)system• EnsembleKalmanFilter(EnKF)DASystem• Modularend-to-endensemblesystem• ModelEvaluationTools(MET)–Verificationpackage
TheDTCdoesnotgenerallycontributetothedevelopmentofnewscientifictechniquesforthesesoftwarepackages.ThetwoexceptionsareMETdevelopmentandsomelimitedphysicspackagedevelopmentforWRFtoaddressshort-comingsbroughttolightbyDTCT&E.TheDTCcontributestothesoftwaremanagementofallofthesesystemsandusersupportforthepublicly-releasedsystems(WRF,HWRF,UPP,GFDLvortextracker,GSI,EnKFandMET).Allsoftwaremanagementandusersupportactivitiesarecollaborativeeffortswiththedevelopers,wheretheexactroleoftheDTCdependsonthesoftwarepackage.ThemaindevelopersofthesepackagesareaffiliatedwithEMC,ESRL,NCAR,GlobalModelingandAssimilationOffice(GMAO)oftheNationalAeronauticsandSpaceAdministration(NASA),NationalEnvironmentalSatellite,DataandInformationService(NESDIS),JCSDA,GFDL,UniversityofRhodeIsland(URI)andtheHurricaneResearchDivision(HRD)ofNOAA’sAOML.
TheDTCisworkingwithEMCtounifytheverificationsystemsbetweenthetwoorganizationsthroughMETandMETViewer,MET’saccompanyingdatabaseanddisplaysystem.DTCstaffvisitedEMCforaweekduringearlyMay.Thisvisitconsistedof18meetingswithapproximately50EMCstafftodiscuss
-
6
theircurrentverificationpracticesandimmediateneeds.InformationgatheredduringthesemeetingsissummarizedinarequirementsdocumentthatwasreleasedtoallparticipantaswellastheEMCDirectoron15September2016.TheDTCisusingthisinformationtodevelopaunifiedverificationsystem,calledMET+,throughNextGenerationGlobalPredictionSystem(NGGPS)fundingoutsideoftheDTC.Briefly,MET+isasetofpythonwrapperstosimplifysetting-upandrunningMETtoallowresearcherstoleveragetheirownuniquealgorithms,andsystematicallyplotthefieldsandresults.
FortheGMTB,theDTChasbeenworkingwithEMCandthephysics-developmentcommunity(ESRL,NCAR,NRL,anduniversities)toestablishaCommonCommunityPhysicsPackage(CCPP)thatwillserveasaframeworkforefficientlydevelopingandtransitioningcurrentandnext-generationphysicsparameterizationsintooperationstomeettheneedsofNGGPS.AnotherimportantcomponentofthisworkisestablishinganInteroperablePhysicsDriver(IPD)thatprovidesaframeworkforphysicalparameterizationsuiteswithintheCCPPtointerfacewithdifferentdynamiccores.Overthepastyear,therequirementsfortheIPDandCCPPhaveundergoneextensivereviewandrefinement,andinformedasoftwaredesign.TheinitialdesignwaspresentedtoEMCandtheNationalUnifiedOperationalPredictionCapability(NUOPC)PhysicsInteroperabilitygroup,aswellasseveralsubsequentrevisionsthatincorporatedfeedbackfromthesegroups.Thisdesigndocument,aswellasthephysicsaliasinglayer(alsoknownasIPDv4)developmentperformedbyGFDLandmadeavailableforFV3inMarch2017,servedasthefoundationfortheinitialCCPPandIPD.GMTB’scontributiontotheIPDeffortwillenableconstructingphysicssuitesatruntimebyparsingauser-friendlyconfigurationfile,allowingforrunningtheparameterizationswithintheCCPPinaveryflexiblemanner.
TheGMTBintendstofacilitateanenvironmentfortheCCPP,definedasasetofpracticesinwhichdesign,development,anddeploymenthappensimultaneouslyandrapidlyinthesameecosystem.Tosupportthisbasicprincipleofmodernsoftwaredesign,theGMTBalsocompleteddocumentsthatdescribetheconceptandmanagementoftheCCPPdevelopmentandsoftware,aswellasaproposedgovernancestructuretomanagetheevolutionoftheCCPP.TheconceptanddesignoftheCCPPdescribesanecosystemfordevelopmentandtransitionofphysics,whereanEMCdevelopercaneasilydeveloponhisownorengageexternaldevelopers.Codeiselevatedtosupportedoroperationalstatusfollowingasuiteofscientifictests.Detailsofthesetestsaretobedeterminedbyagovernancestructureyettobefinalized.AproposalforthisgovernancestructureisincludedintheGMTB’sCCPPRoadmap.TheGMTBtestharnesswillplayakeyroleinmakingthesetestsaccessibletoalldevelopers.Acodemanagementplanwascreatedtosupporttheecosystemandmeetthegovernanceneeds.
Duringthefinaltwomonthsofthereportingperiod(Feb-March2017),theCCPPwascreatedwithplaceholderphysics–askeletontoguidetheconnectiontorealphysics.Fortestingpurposes,theGMTBSingleColumnModel(SCM)wasmodifiedtocallasuitecomposedoftheCCPPplaceholderphysicsthroughtheIPD.Thisimplementationcanbeconsideredthesimplest“dycorecap”fortheIPD–theSCMreplacesactualdynamicswithadvectiveforcing,butitenablesthetranslationofSCMstateanddiagnosticvariablestothosewithinthephysicssuite.ThefirstCCPPreleaseisplannedforearly2018.NCARandGSDphysicsdevelopershavebeenengaged,andwillbeconnectingnewphysicsunderseparatefunding
Buildingondocumentationeffortsfromyear1,comprehensivetechnicalwebdocumentsarenowservedfromtheDTCwebsite,describingboththebackgroundandcurrentfunctionoftheIPDv2usedduringtheNGGPSdynamicalcoretestandtheinitialmemberoftheCCPP,the2016operationalGlobalForecastSystem(GFS)physicssuite.ThecontentofbothdocumentswasgeneratedroughlyequallybytheNCARandGSDmembers,drawingonexpertisefromallcontributorsasappropriate.AreviewbyphysicsexpertsatNOAAEMCwassolicitedandobtained,withfeedbackintegratedintothefinaldocuments.ThedocumentationfortheIPDandCCPPcanbeaccessedatthefollowingURLs,
-
7
respectively:http://www.dtcenter.org/GMTB/gmtb_ipd_doc/andhttp://www.dtcenter.org/GMTB/gfs_phys_doc/.
Theprocessforupdatingthedocumentationgoingforwardisinfluxatthistime.AttherequestofEMC,sourcefilescontainingtheDoxygen-formattedcommentsusedtogeneratethedocumentationwebpagesweremergedintothetrunkofEMC’scoderepository.Inaddition,allexternalfiguresandfilesnecessarytogeneratethedocumentationhavebeenplacedinsidea‘docs’directorynexttothephysicssourcefiles.Asaresult,documentationmaybeversioncontrolledanditispossibleforbothDTCstaffandthoseatNOAAEMCtogeneratethedocumentationoutput.AlthoughaplanforfullyintegrateddocumentationforallcomponentsofNGGPSisnotcomplete,theDTChasprovidedinputtodecision-makerswithinNOAAregardingtheuseofDoxygenastheappropriatetoolandfordefiningtheprocessfordeveloperstoupdatedocumentationasdevelopmentprogresses.
2.1 SoftwareManagement
Whilespecificsoftwaremanagementplansdifferbetweenthevarioussoftwarepackages,theyallcontainthefollowingelements:
• Coderepositoriesmaintainedunderversioncontrolsoftware.• Protocolsforproposingmodificationstothesoftware,whetherthemodificationsaresimply
updatestocurrentfeatures,bugfixesortheadditionofnewfeatures.• Testingstandardsproposedsoftwaremodificationsmustpasspriortobeingcommittedtothe
coderepository.• Additionaltestingstandardsusedtomorethoroughlychecktheintegrityoftheevolvingcode
base.
Givenallthesesoftwarepackagescontinuetoevolveovertime,alltestingstandardsmustbeupdatedperiodicallyinordertomeetthemaintenancerequirementsofthecodebase.Overthepastyear,theDTCcontinuedtocollaboratewiththevariousdevelopergroupsontheseongoingsoftwaremanagementactivities.TheDTCalsocontinuedtoprovideapathwayfortheresearchcommunitytocontributetothedevelopmentofthesesoftwaresystems.Noteworthyeventsfromthisworkoverthepastyearare:
• WRF–Overthepastyear,worktowardsaddinganoptiontoruntheAdvancedResearchWRF(ARW)dynamiccorewithanewsmoothedterrain-followinghybrid-verticalcoordinatewascompleted.ThiscodewillbeincludedinthenextreleaseofWRFandwillbebackwardscompatibleifthenewoptionisnotselected.All2016operationalHWRFforecastsystemcapabilitieswerecommittedtotheWRFtrunk,andweremadeavailableforthenextcommunityreleaseofWRF.ThesecapabilitiesincludetheGFShybridEddy-DiffusivityMass-Flux(EDMF)PlanetaryBoundaryLayer(PBL)scheme(availableforbothARWandNonhydrostaticMesoscaleModelontheEgrid-NMME),updatestothesurfacefluxexchangeswiththecoupledocean(NMMEonly),updatedscale-awareSimplifiedArakawa-Schubert(SAS)cumulusparameterization(ARWandNMME),alandfalloptionfortheidealizedtropicalcyclone(TC)capability(NMMEonly),aswellasmiscellaneousbugfixesandtuningparameters.
• UPP–TheDTCcontinuedtoworkcloselywithEMCtomanagetheUPPcodebasethroughregularbi-monthlymeetings.TostreamlinetheeffortstokeepthecommunityUPPinsyncwithEMC’soperationalUPP,thecommunityUPPsourcecodewasmigratedtoabranchoftheEMCrepositoryforeasiercodesharingandsyncing.ThroughcollaborationswithoperationaldevelopersandsupportedDTCvisitors,themostrecentcommunityreleaseofUPPincludedfull
-
8
GRIB2outputcapability,alongwithnewmicrophysics-specificreflectivityoutputandsyntheticsatellitefields.
• HWRF–TheDTCcontinuedtosupportHWRFdevelopersinusingandaddinginnovationstothecoderepository.TheDTCcompleteddevelopmenttoenablethemultistormconfigurationtorunusingthesamesetupastheoperationalconfiguration,withtheexceptionofoceancoupling,andaddedthiscapabilitytotheHWRFrepository.FixeswerecommittedtotherepositorytoenablebackwardscompatibilityforthepreviousoperationalHWRFconfiguration.Additionally,theDTCdevelopedanewcapabilitytostartaHWRFrunfromtheWRFcomponentwhenusingwrapperstoaidHWRFcommunitydeveloperswhowanttomodifyandruntheatmosphericforecastcomponentwithoutgainingexpertiseonthefullend-to-endsystem.Theabilitytosimulatelandfallwithintheidealizedtropicalcyclonecapability,aninnovationdevelopedbyaDTCvisitor,wastransitionedtotheHWRFcoderepository.TheHWRFrepositorywasadaptedtoaccommodatethetransitionoftheWRFandWRFPre-processingSystem(WPS)repositoriesfromSubversion(SVN)toGitandthecommunityGSIrepositorytransitiontoVLab.UpdatedproceduresandinstructionswerepublishedtotheHWRFdevelopers’webpageandassistancewasprovidedviatheHWRFhelpdesk.TheDTCprovidedcoordinationofdevelopmentactivitiesbychairingtheHWRFdevelopers’committeebi-weeklymeetings.Additionally,DTCprovidedenhancedsupportfordeveloperscontributingtotheHWRFsystem,includingHurricaneForecastImprovementProject(HFIP)-fundedprincipalinvestigators.Tofacilitateinter-developercollaboration,theDTCcontinuedtohostanhwrf-contribrepositoryforpeer-to-peersharingofcode.TheNCARandGSDstaffconductedthisworkjointly.TheHWRFv3.7aandv3.8aUsers’GuideswerepublishedasGSDtechnicalnotes:
Biswas,M.K.,L.Carson,C.Holt,L.Bernardet,2016:CommunityHWRFUsersGuideV3.7a.NOAATechnicalMemorandumOARGSD-46,doi:10.7289/V5SJ1HMD,144pp.
Biswas,M.K.,L.Carson,K.Newman,L.Bernardet,C.Holt,2017:CommunityHWRFUsers’GuideV3.8a,NOAATechnicalMemorandumOARGSD-47,doi:10.7289/V5/TM-OAR-GSD-47,149pp.
• GSIandEnKF–Overthepastyear,theDTCmadeafewcriticalupgradestocurrentcodemanagementandsupporteffortsfortheDAsystems.TheDTCtransitionedthehelpdesktoRequestTracker(RT)forbettertrackingofusers’requestsandquestions.TheDTCbuiltanewcommunityrepository(svn)onNOAA’sVlabserverandsuccessfullytransitionedallcommunityrepositorydeveloperstothisnewrepository.TheDTCalsoinitiatedeffortstoworkwithEMCtoimprovetheefficiencyofcodemanagementfrombothsidesasfollows:startedeffortstounifythecodebuildtool(usingcmake)forGSI,EnKF,andNCEPI/Olibraries;convertedtheuser’sguidestoLaTeXtosharewithalldevelopersthroughthecoderepository;andtransitionedcommunityutilities(e.g.,formatconversion,diagnosticplottingscripts)totheEMCrepository.TheseeffortswillcontributesignificantlytotheunificationoftheDTC-EMCcoderepositoriesoverthecomingyear.On-goingeffortsincluded:supportGSI/EnKFusersthroughthenewcoderepositoryandhelpdesk,performcodereviewsforeachproposedcodeupdateandsynchronizetheDTCcommunitycoderepositorywiththetrunkofEMC’soperationalrepository,coordinatetheGSI/EnKFdevelopmentamongdistributeddevelopersbychairingtheDAReviewCommitteeandhostingthreereviewcommitteemeetings.AnotableoutcomeofthereviewcommitteemeetingswastheadditionofJCSDAasthe10thmemberofthecommittee.
ForAOP2016,theDTCscaledbackitsworkwiththeNOAAEnvironmentalModelingSystem(NEMS)/NonhydrostaticMultiscaleModelontheBgrid(NMMB)duetothelackofanyT&Eactivitiesutilizingthis
-
9
softwarepackage.WhiletheDTCisnolongeractivelyengagedincodemanagementorusersupportforthiscodebase,itdidcontinuetotesttheportabilityoftheNEMSsoftwarepackageandassociatedlibrariesthroughregressiontestingforregionalapplicationsontheNCARsupercomputer,Yellowstone.Overthepast6months,significantinfrastructureandcodearchitecturechangesoverthepastsixmonthsledtofailedportabilitytestsandretrofittingtoanewplatformisconsideredbeyondthescopeoftheDTCactivitiesatthistime.RelevantfeedbackwasprovidedtoEMCNEMSdevelopers.
2.2 VerificationToolDevelopment
Overthepastyear,theDTCverificationteamcompletedtwoMETreleases.METv5.2wasreleasedtothecommunityon15August2016andMETv6.0on3April2017.Approximately20majorenhancementsand50smallerbugfixesweresupportedtothecommunityinthereleases,includingtheupgradetoreading/writingNetworkCommonDataForm(NETCDF)4,a25%speed-upoftheGridpointStatisticalInterpolation(GSI)diagnostictool,andconfigurationfilechangestomakeMETeasiertoset-up.
Tosupportglobalverification,thegrid-to-gridverificationtoolswereupdatedtoincludeaconfigurationoptiontoapplygridboxareaorcosine-latitudeweightingtothecomputationofcontinuousstatistics,suchasAnomalyCorrelation,RootMeanSquareErrorandBias.AlltoolsareprovidedwithGRIB1/2tablesupportfornon-NCEPtablessuchasUKMetOfficeandEuropeanCenterforMediumRangeForecasting(ECMWF).TheASCII2NCtool,whichcreatesNetCDFfilesfrompointobservationsinASCIIformat,wasenhancedtoincludeconfigurationoptionstospecifytheexpectedfrequencyofobservationsandomitoutputwhennotenoughvaliddataarepresent.ThisfeaturewasrequestedbytheNEMSGlobalAerosolComponent(NGAC)group.A“DESC“column(shortfordescription)wasaddedtoMETtoallowuserstoaddadescriptorformoreeffectivestratificationofstatisticsinbothMETandMETViewer.TosupportNOAA’stropicalcyclone(TC)verification,theTC-pairstool,whichmatchesTCforecastsandobservations,wasupdatedtohandleinterpolatedmodelswhosemodelidendsin'3',readprobabilisticforecastsfromthe“E-deck”fileformat,andincludemoreflexiblewaysofpassingthetooldifferentfilenamesforthebestandoperationaltracks.
SeveralmeasuresandmethodswereaddedtoMETtosupportverificationoftotalcloudfractionontheglobalscale.NewinterpolationmethodswereaddedtoallowinterpolationofmodeloutputtobehandledinthemannersimilartotheWorldWideMergedCloudAnalysis(WWMCA)cloudfractionmapping.ThetoolforregriddingWWMCAwasenhancedtodrawadditionalfields,suchasSatelliteID(SatID)andpixelage,fromthebinarycloudanalysisfiles.Also,theGen-VX-masktool,whichcreatesabitmappedmaskedarea,wasupdatedtocomputethesolarazimuthandanglevaluesbasedonlocationandtimeofdaytoallowforthederivationofday/nightmask.
TheMETViewerdatabaseanddisplaysystemisanotherverificationtoolunderdevelopmentduringtheAOP.Itwasmodifiedtosupportthenewfileformats,statisticsand“line”typesintroducedviatheMETv5.2andMETv6.0releases.TheDTCmodifiedtheloadinglogictohandlespecialcasesintroducedbydifferentversionsoftheNOAAmesoscaleandensembleverificationstatisticdatabasefileformat(VSDB).TheinterfacewasalsoenhancedtocomputethemeanandaratioofseveralcurvestosupporttheRegionalEnsemblesteam.Basicdatabasepurgingscriptswerealsogenerated.METViewerv1.9-1.12werereleased,includingtheadditionofaTaylorDiagramtemplate,enhancementstotheeventequalizationlogicandtheMethodforObject-basedDiagnosticEvaluation(MODE)attributecomputationforEMC’sMesoscaleandGlobalbranches,respectively.METViewerdatabasedesignwasinterrogated,severalchangesweremadetospeedupdataloadingandquerying.Thesechangesincludedpartitioningthedatabase,notattemptingtoloademptyfiles,andprovidingthecapabilitytoremoveasingleormultiplerecords,ifneeded.Newdatabasetechnologies(e.g.Couchbase)werealso
-
10
investigatedandfoundpromising.TheDTCalsoinvestigatedoptionstomodifytheMETViewerinterfacetoallowuserloginsandsaveuserpreferences.
TeleconswithNOAAMETandMETVieweruserswereheldeverymonth,startinginOctober2016.Inviteesinclude25engagedNOAAstafffromEMC,WeatherPredictionCenter(WPC)andGSD.ThediscussionsweredrivenbyquestionsarisingfromincreaseduseofMETandMETViewer.Inresponsetothesemeetings,theDTCstaffansweredmanyquestions,fixedseveralbugsintroducedwithloadingVSDBdataintoMETViewer,improvedthespeedofloadingMETViewerandaddednewfeaturestobothMETandMETViewer.METViewercodewasmovedintoGitHubtoallowbothDTCnodesandexternalcollaboratorsaccess.Additionally,planninganddevelopmentoftheinitialpythonwrappersaroundthecomponents,calledMET+,wasdistributedacrossbothDTCnodes.TheGSDnodecontributionestablishedagreaterunderstandingoftheMET+components(METandMETViewer)andallowedmoremeaningfulcollaborationbetweenthenodes.
2.3 Publicly-ReleasedSystems
TheDTCcurrentlycollaborateswithdevelopersonsevensoftwaresystemsthatundergoapublicreleaseprocess:WRF,UPP,HWRF,GFDLvortextracker,GSI,EnKFandMET.Assistancecontinuedtobeofferedthroughemailhelpdesksforallpackages.Informationregardingthetimingandversionofthemostrecentrelease,alongwiththecurrentnumberofregisteredusersandaveragehelpdeskticketspermonthforeachpackagearelistedinTable2.3-1.Table2.3-2containsalistofthewebaddressesforeachsoftwarepackage’susers’page.
Table2.3-1:Codereleases,numberofregisteredusersandnumberofhelpdeskticketspermonthforthepublicly-releasedsoftwarepackagessupportedbytheDTCoverthepastyear.
SoftwarePackagePublicRelease
Version Timing RegisteredUsers Helpdeskticketspermonth
WRFV3.8 April2016
~32,700 ~400V3.8.1 August2016
UPP V3.1 September2016 ~740 ~10HWRF V3.8a November2016 1399
~30GFDLVortexTracker V3.5b September2013 617
GSI V3.5 August20161,687
~20EnKF V1.1 August2016 ~2-5MET V5.2 August2016 3180 ~20-25
Table2.3-2:Userspagewebsitesforpublicly-releasedsoftwarepackages.SoftwarePackage UsersWebsites
WRF http://www.mmm.ucar.edu/wrf/users/
UPP http://www.dtcenter.org/upp/users/
HWRF http://www.dtcenter.org/HurrWRF/users/
GSI http://www.dtcenter.org/com-GSI/users/
EnKF http://www.dtcenter.org/EnKF/users/
MET http://www.dtcenter.org/met/users/
-
11
InadditiontogeneralMETusersupport,theDTCverificationteamactivelyrespondedtorequestsfromNOAAusersregardingtheuseofMETandMETViewer.Accomplishmentsoverthepastyearinclude:1)increasednumberofMETandMetViewerusersfromNOAA(from74to100);2)workedwithNCEPEnvironmentalModelingCenter(EMC),WeatherPredictionCenter(WPC),andClimatePredictionCenter(CPC)toidentifyrequirementsnecessarytounifyverificationbetweenDTCandEMC;3)whitepaperelucidatingtherequirementssubmittedtopointsofcontactatEMCandEMCdirector;4)startedexploringdatabasedesignstohandlelargedatasets;and5)participatedintheNGGPSVerificationandValidationTeamdiscussions.
TheDTCrevampedtheonlinetutorialsforbothGSIandEnKF.Theonlinetutorialswereredesignedtoprovidemoreuser-friendlyinstructionsalongwiththelatestGSIandEnKFcapabilities.Testcaseswerecarefullyselectedwithnewtestingperiodsanddomains,coveringupdatedandadditionalconfigurationswithalternativedataassimilationtechniques[3DVar,3Dhybrid,4DhybridEnsembleVariational(EnVar)],datatypes(conventionalandsatelliteradiancedata),andforecastmodels(ARW,HWRF,NMMB,WRF-chem,GFS).
2.4 DTC-supportedsoftwarecontainers
Manytimesthebiggesthurdlewhenrunninganewsoftwaresystemisgettingitsetupandcompiledontheintendedcomputerplatform.Buildingcomplexsystemsthatrequireanumberofexternallibrariescanbealargeissueforuserstoovercome.Inordertorelievesomeofthisdifficulty,anewtechnologyreferredtoasa“container”hasbeendevelopedthatallowsforcompletesoftwaresystemstobebundledandshippedtousers.Thecontainersincludeeverythingthatisneededtorunthesoftwarecomponent,includingtheoperatingsystem(toolsandlibraries)andcode-thusallowingfortheusertoquicklyproduceoutputwithoutbeingdelayedbytechnicalissues.
ContainershavebeenestablishedoutsideoftheDTCforportionsofanend-to-endNWPsystem,includingWPS,WRF,andNCL.DuringAOP2016,DTCstaffestablishedcontainersfortheUPPandMETsoftwaresystems.Inaddition,datasetsthatmakeuptwoMesoscaleModelEvaluationTestbed(MMET)caseswerebundledinacontainer.ContainersareavailableviaGitHubtorunMET(https://github.com/NCAR/container-dtc-met)andtoruntheend-to-endsystem(includingMETandMMETdataset(https://github.com/NCAR/container-dtc-nwp).Byestablishingtheseadditionalcontainers,theDTCisassistingtheusercommunity(especiallystudents)withefficientlyrunningNWPcomponentsinanefforttofosterconnectionswithfuturecollaborators.
3 TestingandEvaluationT&EactivitiesundertakenbythedevelopersofnewNWPtechniquesfromtheresearchcommunityaregenerallyfocusedoncasestudies.However,inordertoadequatelyassessthesenewtechnologies,extensiveT&Emustbeperformedtoensuretheyareindeedreadyforoperationalconsideration.DTCT&Egenerallyfocusesonextendedretrospectivetimeperiods.Thecasesselectedincorporateabroadrangeofweatherregimesrangingfromnull,toweakandstrongevents.Theexactperiodschosenvarybasedonthephenomenonoffocusforthetest.ThetechniquetobetestedmustbepartofthecoderepositoriessupportedbytheDTCtoensurethatthecodehasreachedacertainlevelofmaturity.TheDTC’sevaluationoftheseretrospectiveforecastsincludesstandardverificationtechniques,aswellasnewverificationtechniqueswhenappropriate.Allverificationstatisticsundergoastatisticalsignificance(SS)assessmentwhenappropriate.Byconductingcarefullycontrolled,rigoroustesting,includingthegenerationofobjectiveverificationstatistics,theDTCisabletoprovidetheoperationalcommunitywithguidanceforselectingnewNWPtechnologieswithpotentialvalueforoperationalimplementation.DTCtestingalsoprovidestheresearchcommunitywithbaselinesagainstwhichtheimpactsofnew
-
12
techniquescanbeevaluated.Thestatisticalresultsmayalsoaidresearchersinselectingmodelconfigurationstousefortheirprojects.
3.1 RegionalEnsembles
MesoscaleNWPsystemsareutilizedinbothresearchandoperationalforecastingapplicationsandcanbeconfiguredtosuitabroadspectrumofweatherregimes.DuetothenumberofapproachesdevelopedandofferedbyNWPsystems,itisnecessarytorigorouslytestselectconfigurationsandevaluatetheirperformanceforspecificapplications.
OnepaperassociatedwithapastRegionalEnsemblesactivitywaspublishedinMonthlyWeatherReview:
Jankov, I., J. Berner, J. Beck, H. Jiang, J.B. Olson, G. Grell, T. G. Smirnova, S. G. Benjamin, J. M. Brown, 2017: A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev., 145, 1161-79.
3.1.1 MesoscaleModelEvaluationTestbed(MMET)
TheMesoscaleModelEvaluationTestbed(MMET;http://www.dtcenter.org/eval/meso_mod/mmet)providestheopportunityfortheresearchcommunitytoconducttheirownT&Eofanewtechnique.DatasetsforanumberofcasesdeemedtobeofhighinterestbyEMCaredistributedviaRAMADDA,aRepositoryforArchiving,ManagingandAccessingDiverseDAta(http://ramadda.org/).MMETdatasetsincludeavarietyofinitializationandobservationdatasets,aswellasbaselinesforselectoperationalconfigurations.Casesofinterestand/orpersistentoperationalmodelissueswereidentifiedthroughouttheyearbyleveragingadirectlinktoEMC’sModelEvaluationGroup(MEG)throughDTCstaffparticipationinMEGweeklytelecons.Alistofoperationalcasesofinterestand/orpersistentmodelweaknesseswascompiledbasedontheseweeklydiscussionsandpublicizedontheMMETwebpage(http://www.dtcenter.org/eval/meso_mod/mmet/additional_cases.php).
Inthepast,asnewversionsoftheWRFandNEMScodewerereleased,MMETcaseswerereruntoprovidecurrent,baselineresultsfortheusercommunity.Whilethefullend-to-endsystemisnolongerupdatedonanannualbasis,previousversionswillremainavailablethroughtheRAMADDAdataserver.Rather,thisyear,operationalmodeloutputforseveralNWPsystems(bothdeterministicandprobabilistic)wereevaluatedandtheobjectiveverificationscoresprovidedtotheresearchcommunitythroughMMET.TheoperationalforecastsystemsincludedtheNorthAmericanMesoscale(NAM),RapidRefresh(RAP),High-ResolutionRapidRefresh(HRRR),andHurricaneWRF(HWRF).ThesebaselinesareprovidedforallnewandexistingMMETcasesforeachavailableoperationalmodel.Twonewcaseswereestablishedthisyear:1)acaseoverAlaska(20150826:TyphoonAtsaniremnantsaffectingAlaska)and2)ahurricanecase(20160928-29:HurricaneMatthew).AnothernewadditionthispastyearistheevaluationoftheStormScaleEnsembleofOpportunity(SSEO)datacollectedfromthe2016HazardousWeatherTestbed(HWT)SpringExperiment.ThisevaluationincludeddeterministicperformanceresultsforeachindividualSSEOmemberalongwithprobabilisticresultsfromtheensembleasawholeforselectvariables;theplotsforselectvariables,thresholds,andmetrics,aswellastheMETViewerXMLsusedtogeneratetheplots,areavailableviaRAMADDA.
Inaddition,communityoutreacheventscontinuetobeprovidedbytheDTCtopromoteenhancedconnectionswithfuturecommunitycollaboratorsandpromotetheuseofMMETdatasets.A1.5hourinstructionalsessionwasofferedduringthe17thWRFUsers’WorkshopinJune2016toraiseawarenessaboutandhighlightthetoolsavailabletothecommunity-at-largethroughMMET.AposterwaspresentedonMMETattheAmericanMeteorologicalSociety(AMS)AnnualMeetinginJanuary2017;
-
13
Harrold,M.,J.K.Wolff,andT.Hertneky,2017:MesoscaleModelEvaluationTestbed(MMET):HelpingConnecttheResearchandOperationalCommunities.28thConferenceonWeatherandForecasting/24thConferenceonNumericalWeatherPrediction,Seattle,WA,January24-27,2017.
andaproposaltoholdashortcourseonusingDTC-supportedcontainers(includingMMETcases)atthe2018AnnualAMSmeetingisbeingwrittenforsubmissionattheendofApril.Finally,amanuscriptwaspreparedandacceptedforpublicationintheNovembereditionoftheBulletinoftheAmericanMeteorologicalSociety(BAMS).
Wolff,J.K.,M.Harrold,T.Hertneky,E.Aligo,J.Carley,B.Ferrier,G.DiMego,L.Nance,Y.-H.Kuo,2016:MesoscaleModelEvaluationTestbed(MMET):AresourcefortransitioningNWPinnovationsfromresearchtooperations(R2O).Bull.Am.Met.Soc.DOI:http://dx.doi.org/10.1175/BAMS-D-15-00001.1
3.1.2 HRRREnhancements
TheHigh-ResolutionRapidRefresh(HRRR)modelbecameoperationalatNCEPon30September2014,andhasfoundwideacceptancebyforecastersinsideandoutsideoftheNationalWeatherServiceasguidanceforavarietyofweatherphenomenaacrossallseasons.TheHRRRusestheARWdynamicalcore,aphysicspackagethathasprovedeffectiveatcapitalizingonthecloud-permittingresolutionofthemodel,anduniqueinitializationproceduresusingradarandsatellitedata,aswellasconventionalin-situobservations,togetherwithaRapidRefresh(RAP)forecast.Inviewoftheimportanceofaccurateshort-termforecastsforvulnerablecoastalareas,particularlyalongtheGulfandAtlanticcoasts,theDTCinvestigatedthevalueofexpandingtheHRRRdomaintowardtheeastandsouth.Atthetimethisworkwasoriginallyproposed,theoperationalNCEPHRRRforecastsonlyextendedto15h,butESRLisnowrunningtheHRRRexperimentallyto36h.ThelateralboundariesoftheHRRRdomainareoftenwithinthecirculationofnortheastcoastalsnowstormsandland-fallingtropicalcycloneswhenthesesystemsarewithina24-36-hstrikingdistanceoftheUSmainland.ExpandingtheboundariesofthecurrentHRRRdomainthusbecameincreasinglyimportantfortheselongerforecastlengths.
ThreetropicalcyclonecasestudieswereconductedwithHRRRversion2(HRRRv2),whichistheversionimplementedoperationallyatNCEPinAugust2016.Themodelinitializationtimesforthethreestormswerechosensuchthatthecenterofeachstormwasnearorslightlyoutsidethestandarddomainbutwellinsidetheextendeddomain.Thestormsandinitializationtimesofinterestwere:Bonnie(12UTC27May2016),Colin(00UTC6Jun2016)andHermine(12UTC31Aug2016).ExtendeddomainsimulationswerealsoconductedforHurricaneHermine,withandwithoutlightningdataassimilation,toaforecastlengthof36h.Herminewaschosenbasedonthelargenumberoflightningstrikesoccurringinanareaofrelativelylowsimulatedreflectivityvaluesatmodelinitializationtime.
Projectdeliverablesweretwo-fold:(1)upgradedproceduresforinitializationofconvection-permittingmodelsoveroceanareaspronetodeepconvection,mesoscaleconvectivesystemsandtropicalcyclones,and(2)arecommendationfordomainconfigurationandphysicssuiteforanexpandedHRRR.Theresultsofourexperiments,particularlythoseforHurricaneHermine,suggestedforecastvalueassociatedwithexpandingtheHRRRdomaintothesouthandeast.ResultsofsensitivityteststolightningdataassimilationpromptedinclusionofalightningreflectivityproxyalgorithmwithintheexperimentalHRRRv3,foreventualimplementationatNCEPin2018.Amorecomprehensivesummaryofthisworkisavailableonlineat:http://www.dtcenter.org/eval/ensembles/dtc_expanded_domain_report.docx
-
14
3.1.3 Testingandevaluationofsmoothedterrain-followingcoordinateinWRF
Oncethenewsmoothed,terrain-followingcoordinatewascommittedtotheRAP/HRRRrepositoryinlate2016,theGSDnodeofDTCbeganimmediatetestingtoanalyzetheimpactsofthisnewverticalcoordinate.ThisnewfeaturewasemployedintheexperimentalRAPandHRRRmodelsrunatGSD,andanumberofcoldstartRAPrunswereconductedascontrolledtestsofthenewverticalcoordinate.Itquicklybecameclearthataproblemexisted,asnumerousmodelcrashesfollowedandspurious/excessivejet-levelwindcharacteristicswerefounduponanalysisofmodelresults.ThesefindingsweresharedwiththeWRFdevelopers.Abugrelatedtothemapscalefactorwasfoundandcorrectedinearly2017.
FollowinganewversionofthecodebeingcommittedtotheGSDrepository,testingbeganagaininMarch2017toassessthehybridverticalcoordinate.Initialqualitativetestsusingcold-startRAPforecastsproducedstableresults.Differenceswerefoundtobenegligibleatinitializationtime,butbysixhoursintotheforecasts,250-hPawindspeedvalueswithinthecoreoftheupper-leveljetforthehybridcoordinatewerefoundtoexceedvaluesinforecastsusingtheoriginalverticalcoordinate.Outsideofthemajorupper-leveljet,anumberofareaswerefoundwherewindswereslowerforthehybridcoordinate.Mostofthedifferenceswerefoundtooccurover,ordownwindofmajormountainranges,specificallyinthewesternareasofNorthAmerica.
Furthertestingwasundertakenwithaweek-long,fully-cycledRAPexperiment,conductedfortheperiodbetween7-13March2017andanotherfive-dayexperimentfor3-7September2016.Resultsfromthesetworetrospectiverunsindicatedthehybridcoordinateresultsweresensitivetothestrengthoftheupper-leveljet.FortheSeptember2016experiment,ContiguousUnitedStates(CONUS)-wide,upper-levelrelativehumidityandwindspeedRMSEandbiasimprovedbyastatisticallysignificantamountwhenthehybridcoordinatewasused.FortheMarch2017timeperiod(Fig.3.1.3-1),similarstatistically-significantreductionsinRMSEandbiaswerefoundfortheCONUSasawhole;however,windspeedbiasfortheWesternCONUSshowedastatistically-significantdegradation.Theupper-leveljetwasstrongerintheMarch2017retro,andthereforeitishypothesizedthatthesmoothed,terrain-followingcoordinateisresultinginwindsthatareslightlytoostrongovermountainousterrainduringsynopticallyactiveperiods.Itshouldbenotedthatverificationbelowabout500hPashowednostatisticallysignificantdifferencesbetweenthetworetrospectivesimulations,highlightingtheimpactofthehybridverticalcoordinateonjet-levelwinds,particularlyinmountainousregions.
Afinalweek-longsimulationwasconductedwiththeHRRRfrom3-10September2016toassesstheimpactofthesmoothed,terrain-followingcoordinateatconvective-resolvingresolution.Resultsfromthisretrospectivesimulationshowedavastlyreducedimpactofthehybridcoordinate.Nostatisticallysignificantdifferenceswerefoundbetweenthetwoverticalcoordinatesouttothefull12hoursofeachforecast,andonlyminor,non-statisticallysignificantdifferenceswerefoundat12hours,limitedtoregionsabove200hPa.Onepossibleexplanationforthesefindingsisthatthehybridcoordinateissensitivetoresolution,withtheconvective-allowingmodelbeingabletobetterresolvemountainousterrain,therebyminimizinganydifferencesfoundbetweentheretrospectiveruns.Givenanypotentialdecreasedsensitivityathigherresolutions,itisalsopossiblethattheretroperiodofSeptember2016didnotcontainupper-levelwindspeedvaluesnecessarytoillustrateadifferenceinverticalcoordinatesat3-kmresolution.
Overall,thehybridverticalcoordinateappearstoproducethelargestimpactatupperlevels,particularlyforwindspeed.Inaddition,thesedifferencesareamplifiedovermountainousterrain,wherethelargestdisplacementsofthehybridverticalcoordinatefromthetraditionalverticalcoordinatearefound.
-
15
Resultsalsoshowedthatasresolutionincreases,theexpecteddifferencesbetweenthetwoverticalcoordinatesdecrease,likelyrelatedtoimprovedterraininhigher-resolutionmodels.
Giventhesefindingsandapparentsensitivitytoseasonandresolution,theNCARnodeoftheDTCwillinvestigatethehybridverticalcoordinatethroughanumberofMMETcasestudiesoveravarietyofdifferentsynopticconditions.TheseresultsareforthcomingandafinalreportforthistestingactivitywillbeavailableontheDTCwebsitebytheendofJune2017.Amanuscripthighlightingfindingsforthisactivitywillbesubmittedbeforetheendof2017.
Figure3.1.3-1.Hybridcoordinate(red)andterrain-followingcoordinate(blue)resultsfromtheRAPretrospectiveforecastsfor7-13March2017.CONUSRMSEisshownfortemperature(upperleft),relativehumidity(uppercenter),andwindspeed(upperright),andbiasforwindspeed(lowerleft).WindspeedbiasforthewesternCONUSisshowninthelowercenterplot,andatimeseriesofaveragedwindspeedbiasforthewesternCONUSfrom300-150hPaisshowninthelowerright.
3.1.4 AddressinguncertaintythroughstochasticparameterperturbationswithintheHRRRensemble
Inmostexistingregionalensemblesystems,model-relateduncertaintyisaddressedbyusingmultipledynamiccores,multiplephysicssuites,oracombinationofthesetwoapproaches.Whiletheseapproacheshavedemonstratedpotential,itistime-consumingandcostlytomaintainsuchsystems,especiallyinoperations.Inordertomovetowardamoresustainableandunifiedsystem,stochasticparameterperturbationswithintheHRRRphysicssuitewereinvestigatedwithafocusonplanetaryboundarylayer(PBL)andLandSurfaceModel(LSM)processes.
ForAOP2016,theRegionalEnsembleteamestablishedatestharnessusingtheRocotoWorkflowManagementSystemtoconductfunctionallysimilarend-to-endtestingoftheHRRRmodelinbothadeterministicandensemblemode.ThistestharnessincludesMETverificationtaskstoevaluatethedeterministicandprobabilisticforecastoutput.TheinclusionofMETintheworkflowprovidesthe
-
16
opportunitytonotonlyverifythefinalproducts,buttoalsoiterativelyadjusttheensembledesignwhileexamininghowprobabilisticstatisticschangewhendifferentapproachesareutilized.
Duetothehighlevelofcomplexityofrunningafrequentlyupdating(hourly),highspatialresolution(3km),largedomain(ContiguousUnitedStates-CONUS)ensemblesystem,extensivehighperformancecomputing(HPC)resourceswereneededtomeetthisobjective.AproposalwaswrittenandsupercomputingresourceswereprovidedthroughtheNCARStrategicCapability(NSC)projectsupport.ThisHPCallocationallowedforamoreextensivesetoftestsleadingtomorerobustresultsthanwouldhaveotherwisebeenpossible.
Asafirststeptowarddesigningthetest,numeroussensitivitytestsofstochasticparameterperturbations(SPP)appliedtovariousparameterswithinthePBLschemeandincombinationwithmorecommonlyusedstochasticapproachesStochasticKineticEnergyBackscatter(SKEB)andStochasticPerturbationofPhysicsTendencies(SPPT),werecarriedout.AvarietyoftestswerealsoperformedinvolvingtheLSMscheme.Preliminaryresultshighlightinginitialtestingofselectspatialandtemporalde-correlationlengthscalesofsoilmoistureperturbationswerepresentedaspostersatthe2016FallAGUmeeting
Wolff,J.K.,I.Jankov,J.Beck,L.Carson,J.Frimel,M.Harrold,H.Jiang:M.Xu,2016:AddressingmodeluncertaintythroughstochasticparameterperturbationswithintheHighResolutionRapidRefresh(HRRR)ensemble.Presentedat2016FallMeeting,AGU,SanFrancisco,CA,December12-16,2016.andthe2017AnnualAMSmeeting.
Andthe2017AnnualAMSmeeting
Beck,J.,I.Jankov,H.Jiang,J.K.Wolff,M.Harrold,J.Frimel,andL.Carson,2017:AnevaluationofstochasticphysicswithintheHighResolutionRapidRefreshEnsemble(HRRRE)andtheimpactsofHighPerformanceComputing(HPC).3rdSymposiumonHighPerformanceComputingforWeather,Water,andClimate,Seattle,WA,January24-27,2017.
Basedontheoutcomeofthesensitivitytests,aretrospectiveexperimentwasdesigned.Inadditiontothestochasticensembleconfiguration,theplanincludedacontrolensembledesignedtoincludeavarietyofPBLandLSMschemestorepresentthecurrentstateofregionalensembleconfigurations.TheSSEOobtainedfromNSSLandSPCcolleaguesforalimitedvariabledatasetservedasasecondbaselineforthistest.AdeterministicHRRRrunwithoutperturbationswasalsoperformedtoprovideabaselinetomakesuretheSPPperturbationsdidnotintroduceanunrealisticbias.Severalextendedretrospectiverunshavebeencompletedandtheteamisintheprocessofanalyzingtheresults.AreportonthefindingswillbeavailableontheDTCwebsitebytheendofJune2017.TheRegionalEnsembleteamwillalsobepreparingamanuscriptforsubmissiontoMonthlyWeatherReview,asafollowuptorecentlypublishedresultsfortestingofSPPintheRAPframework.
3.1.5 WRFtestingandevaluationactivity
InresponsetoWRFconfigurationrecommendationsfromNCAR’sMMMdivision,theAFrequestedtheDTCconductaWRFconfigurationtestwithWRFv3.8.1toprovidecriticalinformationforapossibleoperationalimplementation.InadditiontotransitioningtoanewversionofWRF,theseconfigurationupdatesincludedmigrationtotheThompsonaerosol-awaremicrophysicsschemeandtheupdatedRapidRadiativeTransferModelforGlobalClimateModels(RRTMG)radiationscheme.Toaddressthisrequest,DTCstaffconductedanend-to-endtestandevaluationactivitytoassessthesensitivityofreplacingtheAirForce’scurrentWRFv3.5.1operationalconfiguration,whichwaspreviouslytestedbytheDTC,withaproposedconfigurationforv3.8.1.Forthistest,theDTCgeneratedretrospectiveWRFv3.8.1forecastsforthesamecasesusedforitspriorWRFv3.5.1testingactivity(1July-30September
-
17
2011and1January-31March2012)andcomparedthesenewretrospectiveforecaststothearchivedv3.5.1forecasts.DetailsoftheWRFv3.5.1andv3.8.1configurationsaresummarizedinTable3.1.5-1.Aprojectwebpageisbeingfinalizedandwillincludepertinentinformationregardingthetestsetupandthefullsuiteofresults,alongwithacomprehensivefinalreport(http://www.dtcenter.org/eval/meso_mod/afwa_test/wrf_v3.8.1/index.php).
Table3.1.5-1:Physicssuiteconfigurationsettingsforversion3.5.1and3.8.1,alongwithothernamelistdifferencesbetweenversions.
PhysicsSuite v3.5.1 v3.8.1 Othernamelistchanges(v3.5.1→v3.8.1)
Microphysics WSM5 Thompsonaerosol-aware timestep:90s→60s eta_levels rh2qv_method:1→2 icloud:1→3 aer_opt:1→3 swint_opt:turnedon ysu_topdown_pblmix:turnedon use_aero_icbc:true diff_opt:1→2 dampcoef:0.05→0.2 epssm:0.1→0.5 scalar_adv_opt:0→1
Radiation(LW/SW) RRTM/Dudhia RRTMG/RRTMG
SurfaceLayer Monin-ObukhovsimilaritytheoryRevisedMM5Monin-Obukhov
LandSurface Noah NoahPBL YSU YSUConvection Kain-Fritsch Kain-Fritsch
Thetestingmethodologyallowedforpair-wisedifferencestobecomputedbetweenv3.5.1andv3.8.1,includinganassessmentofbothstatisticallysignificant(SS)andpracticallysignificant(PS)pair-wisedifferences.Consistentwiththesignificantchangesassociatedwiththev3.8.1configuration,alargenumberofSSandPSpair-wisedifferenceswereobservedforboththesurfaceandupperairmetrics.Briefly,intermsofBCRMSE,anumberofPSdifferenceswereseenfor2-mtemperatureanddewpointtemperature,generallyfavoringAFv3.8.1;veryfewofthedifferencesfor10-mwindspeedwerePS(notshown).Intermsofbias(seeFig.3.1.5-1),acoldtemperaturebiasat2mwasgenerallyobservedforbothconfigurations;however,AFv3.8.1wasgenerallythepreferredconfiguration,withafewexceptions.For2-mdewpointtemperature,PSdifferencesforbiasgenerallyfavoredAFv3.8.1.WhileveryfewPSdifferenceswerenotedfor10-mwindspeed,theSSdifferencesforbiasfavoredAFv3.5.1.Intermsofupperairverificationresults,upper-airtemperaturebiasshowedAFv3.8.1asthepreferredconfigurationovertheWestduringthesummer;resultsweremoremixedovertheWestduringthewinterandintheEastforbothseasons.Upper-airdewpointtemperaturebiasshowedAFv3.5.1wasfavoredintheWest,whileintheEast,AFv3.8.1wasfavoredduringthesummerwithmoremixedresultsinthewinter.Upper-airwindbiasSSdifferencesgenerallyfavoredAFv3.5.1inthesummerwithsomePSdifferences,whilewinterwasmixedwithonlyoneinstanceofPSdifferences.
WhenexaminingtheGOIndex(Fig.3.1.5-2),askillscoredevelopedbytheAF,AFv3.5.1wasshownasthebetterperformerforboththe00and12UTCinitializationsduringthesummerandwinterseasons.Basedontheoverallresultsfortheindividualmetrics,aninvestigationintothecauseoftheAFv3.8.1degradationintermsoftheGOIndexwasconductedandshowedthatremovalofRMSEfor400hPaheightfromthecalculationresultedinareversalofperformance,whereAFv3.8.1waspreferredforallbutthe00UTCwinteraggregation.
-
18
Figure3.1.5-1.Timeseriesplotof(a)2-mAGLtemperature(°C)(b)2-mAGLdewpointtemperature(c)10-mAGLwindspeedmedianmeanerror(bias)forthe00UTCinitializationsaggregatedacrossthesummercasesfortheEast(solid)andWest(dashed)verificationdomains.AFv3.5.1isinblueandAFv3.8.1inred.Theverticalbarsattachedtothemedianrepresentthe99%CIs.
Figure3.1.5-2.BoxplotsofGOIndexvaluesaggregatedacrossthesummerandwinterseason,stratifiedbyinitializationtime,where00UTCisinredand12UTCisinbluefor(a)thestandardGOIndexcalculationand(b)theGOIndexcalculationwithout400hPaheight.Themedianvalueisthethickblacklinelocatedatthevertexofthenotches,thenotchesaroundthemedianareanapproximationofthe95%confidenceaboutthemedian,thewhiskers,denotedbytheblack,dashedlines,denotethelargestvaluesthatarenotoutliers,andthecirclesrepresenttheoutliers.
b)
c)
a)
-
19
3.2 Hurricanes
3.2.1 ImpactofThompsonmicrophysicsinHWRF
2013T&EactivitiesrevealedthatThompsonmicrophysicsinthe2013versionofHWRFproducedimprovementsintrackfortheAtlantic(AL)basin,butdegradedthetrackforecastsfortheEasternNorthPacific(EP)basin.GiventhesignificantupgradestotheoperationalHWRFsystemafterthe2013version,performancewhenusingtheThompsonmicrophysicsschemewithinthe2015HWRFwasre-evaluated.ThisT&EactivitywasdesignedinclosecollaborationwiththeEMCHWRFteamtoinform2016pre-implementationtesting,wheretheThompsonandadvectedFerrier-AligomicrophysicsschemeswerebothcandidatesforreplacementoftheoperationalFerrier-Aligomicrophysicsscheme.ThefocusoftheDTC’sevaluationwastodeterminetheimpactofreplacingtheoperationalFerrier-AligomicrophysicsschemewiththeThompsonmicrophysicsscheme.TheoperationalFerrier-Aligoschemeadvectstotalcondensateonly,whereastheThompsonschemeadvectsindividualspecies.TheexperimentsincludedfivestormsfromtheALbasinandelevenstormsintheEPbasinthatoccurredduringthe2014and2015seasons.ParticularemphasiswasplacedonEPbasinstormsinresponsetothe2013T&Eresults.Priortoconductingtheretrospectivetest,boththeThompsonschemeandthepartialcloudiness(PC)schemewithintheRapidRadiativeTransferModelforGlobalClimateModels(RRTMG)parameterizationweremodifiedinanefforttounderstandandaddressthecauseoftheincreasedtrackerrorintheEPbasin.ThesemodificationsincludedfallspeedchangeswithintheThompsonmicrophysicsschemeandalterationstotheRRTMGpartialcloudinessschemetoimplementabugfixandchangethelowerlimitofthesnowandiceparticlesize.ThemajorityoftheworkonthisT&EactivitywascompletedpriortothisreportingperiodandwasdescribedfullyintheAOP2015report.ResultsrevealedthattheexperimentalconfigurationproducedimprovedtrackandintensityforecastsintheALbasin.However,intheEPbasin,theexperimentalconfigurationimprovedthespatialdistributionofclouds,buttheseimprovementsdidnottranslateintoimprovementsintrackandintensityforecasts.
ThefullreportforthisThompsonmicrophysicsevaluationisnowavailableontheDTCwebpage:http://www.dtcenter.org/eval/hwrf_thomp2016/.
TheDTCpresentedresultsfromthisworkduringthe32ndAMSHurricaneandTropicalMeteorologyConference(April2016)andthe17thAnnualWRFUsers’Workshop(June2016):
Holt,C.,M.Biswas,Z.Zhang,S.Trahan,L.Bernardet,G.Thompson,K.Newman:Anevaluationofalternativespecies-advectingmicrophysicsschemesinHurricaneWRF,32ndConferenceonHurricanesandTropicalMeteorology,18-22April2016,SanJuan,PR.
Thompson,G.,L.Bernardet,K.Newman,M.Biswas,andC.Holt:Towardsimprovingexplicitlyresolvedandsub-grid-scalecloudsinHurricaneWRF.32ndConferenceonHurricanesandTropicalMeteorology,18-22April2016,SanJuan,PR.
Holt,C.M.K.Biswas,Z.Zhang,S.Trahan,L.R.Bernardet,G.Thompson,K.M.Newman:Anevaluationofalternativespecies-advectingmicrophysicsschemesinHurricaneWRF.WRFUsers’Workshop,27-30June2016,Boulder,CO.
3.2.2 HWRFphysicsadvancementForAOP2016,theDTCpartneredwithDTCVisitorProgramPrincipleInvestigatorsandsubjectareaexpertstohelpcoordinateandtesttheperformanceofalternatephysicsschemesandinnovationsrelativetothecurrentparameterizationswithintheHWRFphysicssuite.Physicsadvancementsconsideredfortestingcoveredradiation,planetaryboundarylayer(PBL),microphysicsandcumulus
-
20
parameterizations,summarizedinTable3.2.2-1.InadditiontocoordinationandsupportforHWRFphysicsdevelopers,theDTCevaluatedthecodereadinessofcandidatephysicsadvancementsandconsultedwiththeEMChurricaneteamontopprioritiesforHWRF2017pre-implementationtesting.
Table3.2.2-1.Candidatephysicsadvancementsfortestingandevaluation.Innovationsselectedfortestingareindicatedinbold.
PhysicsDeveloper Institution Scheme DescriptionM.Iacono,J.Henderson AER–DTCVisitor
ProgramPIRRTMGradiation Cloudoverlapmethodology
G.Thompson NCAR/RALandDTC RRTMGradiation Modifiedpartialcloudinessscheme
G.Thompson NCAR/RALandDTC Thompsonmicrophysics
EnhancedschemebasedonAOP2015T&Eresults
S.Bao CCU–DTCVisitorProgramPI
advectedFerrier-Aligomicrophysics
InvestigationofadvectedmicrophysicswithinHWRF
R.Fovell U.Albany–DTCVisitorProgramPI
YonseiUniversityPBL HWRFsensitivitytoalternativePBL/surfacelayerschemes
G.Grell,E.Grell NOAA/ESRL–NGGPSPI
Grell-Freitascumulus Replacementschemeforscale-awareSimplifiedArakawa-Schubertscheme
RetrospectivecaseswererunforfourstormsintheALbasin(Edouard,Gonzalo,Matthew,Fiona)andtwostormsintheEPbasin(Patricia,Dolores)thatoccurredduringthe2014-2016hurricaneseasons.Alimitednumberofstormswererunduetocomputationalconstraints.Inanefforttoincreasestormdiversitywithlimitedresources,126hourforecastswererunevery18hours,with12hourforecastsforallinitializationsforcyclingpurposes.Thisrestrictionlimitsthesamplesizeforforecastleadtimesbeyond12hours,decreasingthelikelihoodofobtainingstatisticallysignificantdifferences.Fourparallelexperimentswereruntotestthesensitivityofthethreeexperimentalphysicsconfigurations.Thecontrol(CL)wasrunusingthe2016operationalHWRFdefaultsettings.TwoRRTMGcloud-radiationexperimentswereconductedtotestthesensitivityofanalternatecloudoverlap(CO)methodologyandtheimpactofamodifiedpartialcloudinessscheme.Additionally,acumulusparameterizationreplacementtest(GF)wasruntoinvestigatetheimpactoftheGrell-Freitas(GF)schemecomparedtotheoperationalscale-awaresimplifiedArakawa-Schubert(SAS).
AnewcloudoverlaptechniquefortheRRTMGradiationparameterization,exponential-random(ER),wastestedasareplacementforthedefaultmaximum-randomassumption.TheERtechniquealterstheoverlapofcontinuouscloudlayerstoallowforanexponentialtransitionfrommaximumtorandom.Otherapplicationshaveshownthismethodtobemorerealisticrelativetoradarmeasurementswithinverticallydeepclouds,addingmotivationtotestwithinHWRF.ThetrackandintensityresultsintheALbasinsuggestedmodestreductionsintrackerror,particularlybeyond2days(Figure3.2.2-1).However,thesedifferencesarenotstatisticallysignificant(SS).Absoluteintensityerrorsindicatesmallernon-SSmeanerrorsoutto30hours,withmixedimpactthroughouttheintermediateandlongerleadtimes(Figure3.2.2-1).Boththeexperimentalandcontrolconfigurationsexhibitedanegativeintensitybias(notshown),withreducednon-SSmeanbiasesforCObeyond3days.DuetolimitedcasesintheEPbasin,resultsarependingtheadditionofstormstothesample.
ApartialcloudinessschemewasimplementedwithintheRRTMGradiationschemeforthe2015operationalHWRFsystemtoaddressexcessiveshort-waveradiationreachingthesurfaceduetotransparencyofSAScloudstoRRTMGandalackofstratusrepresentation.Adjustmentsweremadetotherelativehumiditythresholdmethodologytofurtheraddresssolarradiationbiases.PriortestsofthesemodificationsusingWRF-ARWoverCONUSresultedinreducedsolarradiationbiases.TheseupdatesweretestedtoassesswhetherschemeimprovementswouldtranslatetotheHWRFsystem.SimilartotheCOconfiguration,trackerrorsintheALbasinhadatendencyforslightlyreducedmeanerrorsforthePCconfigurationrelativetoCLatthelongestleadtimes.Again,thesedifferenceswere
-
21
notSS(Figure3.2.2-1).AbsoluteintensityerrorsweresmallerforthePCconfigurationbeyond48hours,withSSdifferencesfavoringPCatthe84-,90-,and108-hourleadtimes.ThePCconfigurationexhibitedanegativeintensitybiasof-5to-10kts(notshown),similartotheCLconfiguration.Meanintensitydifferenceswithrespecttoleadtimeweremixedandnon-SS.AswiththeCOconfiguration,resultsfromtheEPbasinarependingincreasedsamplesize.
Figure3.2.2-1.Meantrackerrors(left)andabsoluteintensityerrors(right)intheALbasinwithrespecttoleadtime.TheCL(operational)isinblack,PCingreen,andCOinblue.Pair-wisedifferences(experimentminuscontrol)areshowninlightshadeswith95%confidenceintervals.
TheGFschemeemploysanensembleapproachtorepresentconvection,usingacollectionofparametersandalgorithmstorepresentconvectivetriggers,verticalmassflux,andclosures.Additionally,theschemeisscale-aware,makingitsuitableforHWRF’snestedgridconfiguration.Trackerrorsindicatestatisticallysignificant(SS)differencesbetweentheGFandCLforsomeoftheearlyleadtimes(18-himprovementand48-h,54-hdegradation),whereasmeandifferencesshownon-SSsmallertrackerrorsforGFbeyond84-hours(Figure3.2.2-2).ThemeanintensitybiasforGFappearstobesmallerthanthatfortheCL,butthedifferencesarenotSS(Figure3.2.2-2).TheoperationalHWRFisknowntounder-predictintensityforstrongALstorms,whichisheavilyrepresentedwiththissample.TheGFconfigurationtendstoreducethisdominatetendency;however,errorbarssuggesttheGFconfigurationmaytendtowardover-prediction.IntheEPbasin,intensitytracessuggestatendencyfortheGFconfigurationtobetterrepresentrapidintensification(RI)forspecificinitializationtimes.AdditionalRIcasesintheEPbasinareunderinvestigation.
AdditionalevaluationisunderwaytoverifyHWRF-simulatedbrightnesstemperatures(BT)againstGeostationaryOperationalEnvironmentalSatellite(GOES-13,channel4)BTs.Currently,HurricaneMatthewhasbeenverifiedontheparentdomain(d01)andinnermostnest(d03).Inadditiontothefouraforementionedconfigurations,asupplementaltest(PC+CO)wasruntoincorporatethecombinedimpactofthePCandCOinnovations.Figure3.2.2-3demonstratesthedifferentattributesofeachconfigurationanddomain,shownbyprobabilitydensityfunctions(PDFs)ofthebrightnesstemperature.Fortheparentdomain,theobservedBTfrequencyincreasessteadilyupto280K,followedbyasharpincreasepeakingat290K.TheGFandCOconfigurationspeakatcoolertemperatures,whereasthePCandPC+COexperimentsdemonstrateanimprovementastheyshiftthePDFtowardswarmerBTs.Fortheinnernest,theobservationaldistributionisapproximatelyuniform.Conversely,themodel-simulatedBTPDFsareclearlybi-modal.Notably,onbothdomainsthemodel-simulatedBTsallunderestimatetheobservedfrequencyofBTfrom235-275K.
-
22
Figure3.2.2-2.Meantrackerrors(left)andmeanintensityerrors(right)intheALbasinwithrespecttoleadtime.CL(operational)isinblack,GFinred,pair-wisedifferences(GF-CL)areshowningreywith95%confidenceintervals.
Fig.3.2.2-3.Probabilitydensityfunctions(PDFs)oftheobserved(yellowline)andmodel-simulatedbrightnesstemperaturesfortheGF(green),PC+CO(purple),CO(blue),PC(red),andCL(black)simulationsatforecasthour24ond01(left)andd03(right).
Fractionsskillscore(FSS)wasalsocomputedtoshowtheskillofeachconfiguration(notshown).ResultsindicatetheGFconfigurationperformsthebestforallBTthresholdsexceptthewarmest(greaterthan290K)ontheparentdomain,whereasthePCconfigurationdemonstratedthebestskillontheinner-nestforBTthresholdsgreaterthan250K.Notably,noneoftheconfigurationsworsenedHWRF’sabilitytoreproducetheobservedBTPDForsubstantiallydegradedtheFSSrelativetotheCL.
Theresultsofthepre-implementationtestingweretoadoptthePCinnovationsforthe2017operationalHWRFconfiguration.TheGFconfigurationcontinuedtodemonstratepromisewithlargerpre-implementationtestsperformedbyEMC,howeverreproducibilityissueswhenrunningwithadifferentnumberofprocessorscauseddelays,resultinginadeferreddecisionontheGFconfigurationforthe2018HWRFimplementation.Finally,inclusionoftheCOinnovationintoHWRFwastableduntil
-
23
developersimplementanamelistoptionforthenewoverlapmethod.Furtheranalysisofallthreeconfigurationsareunderwayforinclusionintheprojectfinalreport,whichwillbepostedontheDTCwebsite(http://www.dtcenter.org/eval/hwrf_GF_PC_CO/).Additionalcasesarebeingaddedtoincreasethesample,particularlyintheEPbasin.Furthermore,casestudiesfocusingonparticularstormsareongoingtobetterunderstandthebehaviorofeachphysicsinnovation.
3.3 DataAssimilation
OnepaperassociatedwithpastDAactivitieswaspublishedinBAMS:
H.Shao,J.Derber,X.-Y.Huang,M.Hu,K.Newman,D.Stark,M.Lueken,C.Zhou,L.Nance,Y.-H.Kuo,andB.Brown,2016:BridgingResearchtoOperationsTransitions:StatusandPlansofCommunityGSI.Bull.Amer.Meteor.Soc.,97,1427–1440,doi:10.1175/BAMS-D-13-00245.1.
3.3.1 RegionalEnsembleBasedDAT&E
UnderAOP2015funding,theDTCbuiltanexperimental4DhybridEnVardataassimilationsysteminthecontextofRAP,evaluatedthecodereadinessforregionalapplications,andprovidedfeedbacktodevelopersonitsinitialassessmentofforecastimpactsof4DhybridEnVardataassimilation.Throughouttheprocessofsettingupandtestingthiscapability,theDTCreportedbugfixesandmissingcapabilitiestotheGSIdevelopers.Replacingthe3DhybridEnVardataassimilationstepintheRAPworkflow,4DhybridEnVardataassimilationgeneratedminimalimpactsontheanalysesandforecasts.Inadditiontoreportingontheoutcomeofitstest,theDTCidentifiedareasthatneedmoreworktoimprovethecurrent4DEnVarcapabilities.Thefinalreport,whichisavailableontheDTCwebpageathttp://www.dtcenter.org/eval/data_assim/4denvar/rap_15km/,providesadetaileddescriptionoftheexperimentsanddiscussionoftheresults.
TheDTCalsopresentedtheresultsattwoworkshops:
H.Shao,M.Hu,C.Zhou,K.Newman,X.Zhang,andC.Holt,2016:Testingandevaluationoffour-dimensionalensemblevariationaldataassimilationforregionalweatherforecasts.The7thNOAATestbedandProvidingGroundsWorkshop,CollegePark,Maryland.
C.Zhou,M.Hu,K.Newman,H.Shao,andX.Zhang,2016:InitialassessmentoftheGSI-based4Dhybridensemble-variationaldataassimilationanditsapplicationforregionalforecasts.The17thAnnualWRFUsers'Workshop,Boulder,Colorado.
K.Newman,M.Hu,C.Zhou,andH.Shao,2016:InvestigatingthecapabilityofGSIfour-dimensionalensemblevariationaldataassimilationforWRF-ARWapplications.The17thAnnualWRFUsers'Workshop,Boulder,Colorado.
3.3.2 HighResolution(3km)EnVarTestingandEvaluation
TheDAT&EactivityforAOP2016focusedon4DhybridEnVarcapabilitiesforhigh-resolutionregionaldataassimilationincontextoftheHRRRsystem.HRRRcurrentlyusesaGSI-based3DhybridEnVarDAsystem,whichusestheglobalensemble(~30km)fortheensemblebackgrounderrorcalculationandone-hourARW(3-km)forecastsinitializedwithRAP(13km)analysesfromthepreviouscycle(socalled“pre-forecast”)astheDAbackground.ThisT&Eactivitywasdividedintotwofocusareas:1)demonstrationof4DhybridEnVarsystemforHRRR,and2)feasibilityandimpactassessmentforfastcyclingof4DhybridEnVar.
-
24
3.3.2.1 Demonstrationof4DhybridEnVarsystemforHRRR
TheDTCsetuptwoworkflows:1)aworkflowbasedonoperationalHRRRsystemand2)aworkflowthatreplacedthe3DhybridEnVarDAsystemwithanexperimental4DhybridEnVarsystem.Similartothe3DhybridEnVar,the4DhybridEnVarwasconfiguredtouseensembleinputfromtheglobalensembletocomputetheflow-dependenterrorcovariance.Notethateach4Danalysisrequiresmultiple-timelevels(3timelevelsforthistest)ofensembleandbackgroundinputfiles,incontrasttotheoneensembleandonebackgroundfilerequiredforeach3Danalysis).TheDTCusedtheseworkflowstoconducthourlyupdateexperiments.Duetocomputingconstraints,theDTCselectedareducedHRRRdomainforthetest.Thistestfocusedonthetimeperiod3-10September2016,whichincludedafast-evolvingconvectivescaleevent.Figure3.3.2.1-1showsthetestdomainwiththeanalyzedreflectivityfromthe4DhybridEnVarexperimentat1200UTCon8September2016(leftpanel)andtheobservedradarreflectivityat1155UTCon8September2016(rightpanel).
Figure3.3.2.1-1.Thetestdomainand4DhybridEnVaranalyzedreflectivityat1200UTCon8September2016(leftpanel)andtheobservedradarreflectivityat1155UTCon8September2016(rightpanel).
Figure3.3.2.1-2showsthedomain-averagedRMSEforthewindbackgroundandanalysesfrom3Dand4Druns.Theresultsindicatethe4DhybridEnVartechniqueimprovesthefitofbothbackgroundandanalysestoobservations.Similarresultswerealsofoundforhumidityandtemperatureatmostofverticallevels.Fortheforecasts,theimpactsofthe4Dtechniquebecamelargerforlongerforecastrangeandresultedinremarkabledifferencesfromthe3Dresults.Figure3.3.2.1-3showsthedifferencebetweenthesimulatedreflectivityforecastsfromthe4Dand3Drunsatdifferentleadtimes.Thepatterninthedifferencefieldspointstotheimpactsofthe4Dtechniqueontherainbandlocationsandmagnitudeatconvectivescales.Atforecasthour6,themagnitudeofthedifferenceislargersuchthatthereflectivitydifferencesareofthesameorderofmagnitudeastheobservedreflectivity.TheDTCiscurrentlyperformingmorein-depthdiagnosticsandanalysisofverificationresultsfromthistestingactivity.AdetailedreportisexpectedtoavailablebetheendofMayandwillbepostedontheDTCwebpage.
Inaddition,theDTCexaminedtheimpactsofreplacingtheGFSensemblewithahigh-resolutionARWensembleinthehybridruns.Acasestudyshowsthata3DhybridrunusingtheARWensemble(notshownhere)canresultinlargedifferencesin6-hforecastscomparedwith3DhybridrunsusingtheGFSensemble.Thisoutcomeindicatestheensemblerepresentationatconvectivescalescanleadtoimpactssimilartoreplacing3Dwith4Ddataassimilation.
-
25
Figure3.3.2.1-2.Thetimeseries(leftpanel)andverticalprofilesofdomainaveraged(rightpanel)RMSEsforthewindbackgroundandanalysesgeneratedfromthe3D(redlinesforbackgroundandgreenlineforanalyses)and4D(blacklinesforbackgroundandbluelinesforanalyses)hybridEnVarexperiments.
Figure3.3.2.1-3.Differencebetween1-hforecasts(leftpanel)and6-hforecasts(rightpanel)ofmodelsimulatedreflectivityproducedbythe4Dand3DhybridEnVarexperiments.
3.3.2.2Feasibilityandimpactassessmentforfastcyclingof4DhybridEnVar
Basedontheoutcomefromthefirstfocusarea,theDTCmodifiedtheworkflowtoincreasetheanalysisupdatefrequencyandcycledthehybriddataassimilationtoevery15minutesthroughoutthepre-forecasthour.TheDTCalsoaddedthecapabilitytoGSItoperformsub-hourly(inminutes)analysisupdates.Thenewworkflowsforthe3Dand4Dsub-hourlycyclingdataassimilationsystemforHRRRareshowninFigure3.3.2.2-1.Duetothesubstantialincreaseincomputationalresourcesassociatewithgoingtosub-hourly,theDTConlyperformedtestsforasubsetoftestingperiod(8-10September2016)usedforthefirstfocusarea.Theresultsshowedthatboththe3Dand4Dsub-hourlycyclinganalysisproducedabetterfittoobservationsthanthatofthe3Dhourlycycling,followedbyneutraltoslightlynegativeimpactsatthefollowinghours(figuresnotshown).Thesub-hourly3Dand4DEnVarconfigurationsmayneedtobetunedintermsofobservationtimewindowsforassimilation,orobservationerrormayneedtobeadjustedforhigh-frequencycycling.Itisalsoimportanttokeepinmindthatthesesub-hourlyexperimentsusedaGFSensemble,whichmayleadtorepresentationissuesforbackgrounderrorsatconvectivescales.TheDTCisperformingfurtherdiagnosticstocomparethe3Dand4Dresults.AdetailedreportisexpectedtoavailablebytheendofMayandwillbepostedtotheDTCwebpage.
3Dbackground4Dbackground3Danalysis4Danalysis
-
26
Figure3.3.2.2-1.Sub-hourly3D(leftpanel)and4D(rightpanel)hybridEnVarworkflowdiagramsshowingtheevolutionofthe15-minuteassimilationcyclespriortothe24-hourforecastissuedonthehour.NotethatmultipleGFSensemblesand30-minHRRRforecastsateachcycleareneededfor4DhybridEnVarassimilation.
3.4 GlobalModelTestBed(GMTB)
3.4.1 PhysicsTestbed
TofacilitatethedevelopmentofanadvancedphysicssuiteforNWS’sNGGPS,theDTCisdevelopingauniform‘testharness’toenablein-depthinvestigationofvariousphysicalparameterizations.ThistestharnessiscurrentlybeingusedbytheGMTBforitsT&Eactivities,andhasbeenmadeavailabletocommunityscientistsworkingwiththeGMTB.Asanexample,developersoftheGrell-FreitascumulusparameterizationranpreliminarytestsusingtheGMTBtestharnesstopreparecodeforamorecomprehensivetest,whichwillbeconductedbytheGMTBstaff.Thetestharnessmimicsthelogicalprogressionfortestingnewlydevelopedparameterizationsthattypicallytakesplacewithinthescientificcommunity.Componentsaregraduallyaddedasonemovesthroughthehierarchyuntilthefullforecastmodelcomplexityisreached.ItisdesignedtocomplementboththeexistingtestingprotocolatEMCandindependenttestingtypicallyperformedbyparameterizationdevelopers.Figure3.4.1-1illustratesthehierarchicaltiersofthetestharness,representshowtheDTCenvisionsthedivisionofeffort(GMTB’slikelyroledenotedbyblue)andshowshowtheharnessfitswithinEMC’sexistingtestingframework.
3.4.1.1 SingleColumnModel
AspartoftheGMTBphysicstestharness,aSingleColumnModel(SCM)thatmakesuseoftheIPDhasbeendevelopedandlightlytested.TheSCMisdrivenbyspecifyinganinitialprofilerepresentingthethermodynamicstateandaccompanyinghorizontalwinds.Changestotheprofilecausedbylarge-scaleadvectionareappliedthrough“forcing”terms,whicheffectivelyreplacethedynamicsofathree-dimensionalmodel.Aphysicssuitecalculatessub-gridscaleprocessesandchangestheprofileinconcertwiththeappliedforcing.
DesignoftheSCMfocusedoncommunity-friendlinessbyminimizingexternaldependenciesandusingcommunity-sanctionedcodingpractices.UsingtheSCMonlyrequiresthecmakeutilityforbuildingandtheFortrannetCDFlibrary(I/O)tobeinstalledandaccessible.Python-basedscriptsforplottingandanalysis,easilysetupwithaneditableconfigurationfile,areincludedinthedistribution.Asofnow,theGFSphysicssourcecodeisbundledwiththeSCMsourcecodeasaseparaterepository.Fortestingpurposes,thiscodeisupdatedoccasionallytoworkwiththetop-of-trunkGFSphysicscode,althoughit
-
27
willlikelyonlysupportspecific,taggedversionsofthiscodeinthefuture.ThecodehasbeentestedonalatemodelMacintosh,aswellasonNOAA’sResearchandDevelopment(R&D)machine(Theia)andNCAR’sYellowstone.Inaddition,aGMTBSCMUser’sGuideandtechnicaldocumentationwasdevelopedwithDoxygenandisavailableontheDTC’swebsite:http://www.dtcenter.org/GMTB/gmtb_scm_doc/.
Figure3.4.1-1.DiagramillustratingthetestinghierarchyplantosupportphysicsdevelopmentforNGGPS.LRindicateslowresolution,MRmediumresolution,andHRhighresolution.Colorshadingindicateswherethedifferentgroupsareanticipatedtofocustheirefforts(red–physicsdevelopers,blue–GMTBtaskwithintheDTC,andgreen–EMC).PPstandsforphysicsparameterization.
Sofar,theSCMissetuptorunindividualcaseslikethosesuppliedbytheGlobalEnergyandWatercycleEXchanges(GEWEX)GlobalAtmosphericSystemStudies(GASS)program.Thesecasesoftenderiveinitialconditionsandadvectiveforcingfromfieldcampaigns,andareintendedtostudyspecificphysicalphenomenaandhowtheyarerepresentedbyphysicssuites.The“catalog”ofcasestousewiththeSCMisawork-in-progress,withoneshallowconvectivecasebasedonthetransitionfromstratocumulus-to-cumulusasobservedduringtheAtlanticStratocumulustocumulusTransitionEXperiment(ASTEX)fieldcampaignandonedeepconvectivecaseasobservedduringtheTropicalWarmPool–InternationalCloudExperiment(TWP-ICE)fieldcampaign.Bothcasesareinitializedandforcedbasedonobservationsmadeduringtheirrespectivefieldcampaigns.Althoughbothcasesusehorizontaladvectivetendencieswithprescribedverticalmotion,itispossibletoconfiguretheSCMtousetotaladvectivetendenciesandrelaxationforcingasdescribedinRandallandCripe(1999).Goingforward,theGMTBwilladdmorecasestothecatalog,includingcasesthatwillrequirechangestotheunderlyingGFSphysicscode(e.g.,abilitytoturnoffspecifiedphysicsschemeswithinthesuite).ThecasesaddedbytheGMTBcanalso
-
28
serveasanexampleforcommunitymemberstoaddcasesofinterest.Inaddition,theSCMissetuptoeasilyrunusingforcingensemblesthatcanbeusedtounderstandaphysicssuite’sresponsetouncertaintyintheforcing.
3.4.1.2 WorkflowforLow/MediumResolutionGlobalForecastTests
Buildingonpreviousprogress,theGMTBsuccessfullyestablishedanend-to-endworkflowsystemforrunningNEMS/GlobalSpectralModel(GSM)andUPP(stronglyleveragingEMCcapabilities),aswellasrunningDTC-contributedcomponents.Theend-to-end-workflowsystemreachedamaturestatethatallowedforrunningatestoftheGrell-Freitas(GF)convectiveparameterization(seedescriptioninsection3.4.2).TheDTC-contributedworkflowcomponentsforcreatingPython-basedforecastplots(e.g.temperature,moisture,convectivevs.non-convectiveprecipitation)andverificationresults(e.g.,near-surface,upper-air,andprecipitationverification)continuedtobeupgradedtoincludeadditionalfeaturesandflexibility.AscripttoplottropicalcyclonetracksforeachmodelinitializationwascreatedandaddedtotheautomatedworkflowanumberofconfigurationfilesforMETViewer(auserinterfaceforplottingMEToutput)weremodifiedandupgradedtogenerateimprovedverificationplots.Thisworkincludedaddingthe‘scorecard’capabilitytotheverificationarsenal;the‘scorecard’isawaytosummarizepatternsintheperformancedifferencesbetweentwoconfigurations,includinglevelofsignificance,forspecifiedmetrics,variables,levels,regions,andtimes.The‘scorecard’wasdevelopedbytheNCARVerificationteamwithNGGPSfundingandwasmadeavailabletoGMTBforbeta-testingaheadofitsreleasetoEMCandothers.
Workisalsounderwaytoexpandthetestbedcapabilitiestoequipphysicsdeveloperswithawiderangeoftoolstoassessstrengthsanddeficienciesofphysics.ThecapabilitytoproducebiasinformationfromGSIdiagnosticfiles,whichprovideO-B(observation–background)informationwillsoonbeavailable.Inaddition,theGMTBiscollaboratingwithNGGPSPIJasonOtkintoincludesyntheticsatelliteoutputfromUPPtohelpwithevaluatingthemodel’sabilitytoaccuratelysimulatecloudsandmoisture.TheGMTBhasbeeniteratingwithJ.OtkinandhisteamtouserawmodeloutputfromtheGFtesttorunthroughUPPinordertotestupdatestotheradiativetransfermodelemployedbyUPP.Tropicalcyclogenesisverificationisintheprocessofbeingimplemented.Inaddition,thecapabilitytoperform6-hglobalprecipitationverificationisunderdevelopment. TheGMTBrevitalizeditscollaborativedialoguewithEMC’sglobalteamwithrespecttomigratingfromtheircurrentscriptingarchitecturetoaunifiedglobalworkflow,whichincludesattendingbi-weeklymeetingshostedbyEMC.TheGMTBisactivelytestingandrunningthenewRocoto-basedworkflow(v3.0.0)inpreparationforthenexttestingeffort,whichwillincludecycledDA.
Inaddition,theteamhascontinuedtomanagethescripts,andconfigurationfilesusedintheGMTBworkflowthroughaGitrepositoryonVLab.AsubstantialefforthasalsobeenputforthtodocumenttheGSM/UPPworkflow,aswellasthediagnosticandverificationworkflow.AstheGMTBtransitionstoEMC’sunifiedglobalworkflow,thedocumentationwillcontinuetobeupdatedaccordingly.SimilartootherGMTBdocumentation,theworkflowdocumentationusesDoxygen.
3.4.2 Grell-Freitasconvectiveparameterizationtest
TheGMTBconductedatestoftheGrell-Freitas(GrellandFreitas2014)convectiveparameterizationtoprovideinputontheestablishmentanddevelopmentofanadvancedphysicssuiteforNOAA’sGFS.Thisparameterizationwasselectedfortesting,throughconsultationwithEMCandtheNGGPSProgramOfficeandPhysicsTeam,becauseofitspotentialforimprovingforecasts.Itisastate-of-the-artschemethatincludesascale-awarefeature,whichmakestheschemesuitableforuseacrossawiderangeof
-
29
modelresolutions.Itincorporatesanensembleapproachinitsrepresentationofconvection,whichallowsperturbationbystochasticfieldsfordeterministicforecasting,aswellasensembledataassimilation.Flux-formverticaltracertransport,wetscavenging,andaerosolawarenessarealsooptionsinthisscheme.Thescheme’smaturity,itshistoryofoperationaluseatNCEPintheRAP,andthefactthatitsdevelopmentisfundedbyNGGPSalsoledtochoosingtheGFschemefortesting.
ThistestwasconductedusingGMTB’shierarchicaltestbed,whichcurrentlyconsistsofaSCMandaworkflowforrunningtheGFS.Inbothcases,acontrolusingtheGFSoperationalSimplifiedArakawaSchubert(SAS)convectiveschemewascreated(GFS-SAS)andcomparedagainsttheexperimentalconfiguration(GFS-GF)whosephysicssuitewasthesameasGFS-SASexcepttheGFdeepandshallowconvectionschemeswereusedinplaceofSAS(Table3.4.2-1).Theentiretestresults,aswellasthefinalreport,arepostedontheDTCwebsite.Selectresultsarepresentedbelowtohighlightthekeyfindings.
Table3.4.2-1.TablesummarizingthebasicelementsofGrell-Freitasconvectiveparameterizationtest.
FortheSCM,asinglecasebasedonadeepconvection-focusedfieldcampaignwasusedtoprovideinsightintohowGFS-SASperformscomparedtoGFS-GF.ThetestingparadigmfollowstheonedescribedinRandalletal.(2003)andZhangetal.(2016),namelyinitialconditionsandcolumnforcingarederivedfromobservationsobtainedduringIntenseObservationPeriods.TheatmosphericphysicssuitethatmakesuptheSCMisallowedtorespondtotheforcingbygeneratingparameterizedcloudsandprecipitation,radiativeheating,verticalmixing,etc.Givenidenticalforcing,theGFS-GFsuiteproducedsmallerconvectivetendenciesandamuchlowerconvectiveprecipitationratiothantheGFS-SASsuite(Fig.3.4.2-1).GFS-GFreducedthedrybiasintheboundarylayerandgenerallyproducedahighercloudfractionduringthedeepconvectiveperiodcomparedtoGFS-SAS.
Theglobalforecastswererunatarelativelylowresolution(~34km),infree-forecastmode(nodataassimilationorcycling)andwithouttuningofthephysicssuite.TheoperationalGFSanalyseswereusedtoinitializetheretrospectivecoldstartforecastscoveringthetimeperiodJune–August2016(JJA).TheglobalmodelforecastsdisplayedbehaviorsimilartotheSCM,inthesensethatGFS-GFhadlowerconvectiveprecipitation(Fig.3.4.2-2).DifferencesbetweenprecipitationcharacteristicsofGFS-SASandGFS-GFwerenoticeableovertheCONUSdomain.Whilethe6-hprecipitationfrequencybiasesshowedaprominentdiurnalsignalforbothconfigurationsatthe0.01”threshold(Fig.3.4.2-3),aswellas0.1”and0.25”thresholds(notshown),GFS-GFproducedalargerdiurnalsignalinthefrequencybias,withsimilarmagnitudestoGFS-SASat18-UTC,butlowerbiasat00UTC.
-
30
Figure3.4.2-1:Meanprofilesoftemperaturetendencies(Kday-1)fortheactivephaseoftheTWP-ICEcase.Colorsdenoteforcing(red),PBLscheme(green),convectiveschemes(deep+shallow,blue),andmicrophysicsscheme(purple).Linetypesdenotethephysicssuite:GFS-SAS(solid)andGFS-GF(dashed).Tendenciesduetolongwaveandshortwaveradiationareinorangeandbrown,respectively.
Figure3.4.2-2:Average6-haccumulatedconvectiveprecipitation(mm)overthethree-monthtestperiod(JJA2016)atthe120-hforecastleadtimeforGFS-GF-GFS-SAS.
Theglobalforecastswererunatarelativelylowresolution(~34km),infree-forecastmode(nodataassimilationorcycling)andwithouttuningofthephysicssuite.TheoperationalGFSanalyseswereusedtoinitializetheretrospectivecoldstartforecastscoveringthetimeperiodJune–August2016(JJA).TheglobalmodelforecastsdisplayedbehaviorsimilartotheSCM,inthesensethatGFS-GFhadlowerconvectiveprecipitation(Fig.3.4.2-2).DifferencesbetweenprecipitationcharacteristicsofGFS-SASandGFS-GFwerenoticeableovertheCONUSdomain.Whilethe6-hprecipitationfrequencybiasesshowedaprominentdiurnalsignalforbothconfigurationsatthe0.01”threshold(Fig.3.4.2-3),aswellas0.1”and
-
31
0.25”thresholds(notshown),GFS-GFproducedalargerdiurnalsignalinthefrequencybias,withsimilarmagnitudestoGFS-SASat18-UTC,butlowerbiasat00UTC.
Figure3.4.2-3:Frequencybiasof6-haccumulatedprecipitation(in)forGFS-SAS(red)andGFS-GF(green)aggregatedovertheCONUSdomainforthe0.01”thresholdasafunctionofforecastleadtime(h)forJJA2016.Theverticalbarssurroundingtheaggregatevaluerepresentthe95%CIs.
Withthecopiousamountofverificationresultsproducedforthistest,a“scorecard”wasastraightforwardwaytoidentifypatternsintheperformancedifferencesbetweenthetwoconfigurations,includinglevelofsignificance,forspecifiedmetrics,variables,levels,regions,andtimes.Thescorecardforglobalsub-regionshelpedidentifythatupper-airwindspeedhadthefeweststatisticallysignificant(SS)differencescomparedtoothervariables,asshowninFig.3.4.2-4forNorthernHemisphere(NH).TheNHclearlysignaledGFS-SASasperformingbetterfortheearlierpartoftheforecastperiod.However,GFS-GFwasthebetterperformerfortemperaturebiaslaterintheforecastperiod.Follow-updiagnosticsindicatedtheimprovedperformanceofGFS-GFforthismetricwasrelatedtotheGFS-SASwarmingupprogressivelyovertheNHthroughouttheforecastperiod.
Tropicalcyclonetrackerrorsaveragedoverallthebasinsweresimilarforbothmodelconfigurations(Fig.3.4.2-5a).WhileaccuracyinTCintensityforecasts(Fig.3.4.2-5b)isnotexpectedofamodelrunatthiscoarseresolution,itisinterestingtonotethatstormsproducedbyGFS-SASaremoreintense(notshown)andhavelessabsoluteintensityerrorthanthoseproducedbyGFS-GF.
TheGMTB’stestingandevaluationoftheGFcumulusparameterizationillustratedthecomplexity--yetscientificusefulness--ofconnectinganewschemetotheGSM.ThesuccessofthistestwasheavilydependentoninteractionsamongandinvestmentbytheGMTB,thephysicsdeveloper,andEMC’sGlobalTeam.TheclosecollaborationanditerationwiththedeveloperhelpedensuretheGMTBproperlyconnectedtheGFparameterizationwithintheGSMcode.Inaddition,thecollaborationwiththeEMCGlobalTeamwasessentialtotheGFtest.TestingwiththeSCMresultedinseveralkeyfindings,withonefindingaligningwithresultsfromthe3-Dglobalforecasts.WhilenotallresultsfromtheSCMcouldbetranslatedtothefullglobalforecasts,thistesthighlightedtheutilityandprocessofthehierarchicaltesting.Movingforward,itwillbenecessarytofurtherengagewithEMCandcontinuetogetfeedbackregardingdesiredverificationmethodsanddisplays,whichwouldthenbeprioritizedforfutureimplementationinthetestbed.
-
32
Figure3.4.2-4:ScorecarddocumentingtherelativeperformanceofGFS-SASandGFS-GFovertheNHformeanbiasandRMSEfortemperature,relativehumidity,andwindspeedbyforecastleadtimeandverticallevelforJJA2016.Green(red)shadingindicatesGFS-GF(GFS-SAS)wasbetterthanGFS-SAS(GFS-GF)atthe95%significancelevel.Smallgreen(red)arrowsindicateGFS-GF(GFS-SAS)wasbetterthanGFS-SAS(GFS-GF)atthe99%significancelevel.Largegreen(red)arrowsindicateGFS-GF(GFS-SAS)wasbetterthanGFS-SAS(GFS-GF)atthe99.9%significancelevel.GreyshadingindicatesnostatisticallysignificantdifferencesbetweenGFS-SASandGFS-GF.
Figure3.4-6:(a)Meantrackerrors(nm),and(b)meanabsoluteintensityerrors(kt)with95%confidenceintervalswithrespecttoleadtime(h)forGFS-SAS(red),GFS-GF(green)andtheirpairwisedifferences(black)inallbasinsforJJA2016.
-
33
3.4.3 CICEtest
Overthepastyear,theGMTBcompletedatestofTheLosAlamosseaicemodel(CICE),whichstemmedfromtheNGGPSSeaIceModelingWorkshoporganizedbyGMTBinFebruary2016.Asdescribedintheworkshop’sfinalreport(link),participantsrecommendedthetentativeadoptionofCICE,pendingfollow-uptesting,andaddressingconcernsraisedregardingmodelgovernanceanddifferencesinstaggeringbetweenthegridsusedinNCEP’sUnifiedGlobalCoupledSystem(UGCS)oceanmodelsandCICE.
Thetestplan,devisedjointlybyGMTB,EMC,theworkshopcommitteeandinterestedworkshopparticipants,includedexperimentsoveraone-yearperiod,withCICEbeingruninastandaloneframework,forcedbyatmosphericandoceanicfieldsfromtheNCEPoperationalClimateForecastSystemversion2(CFSv2).GMTBconductedthe30-dayforecastruns,andevaluationwasperformedjointlybyGMTB,EMC,andESRL.
Theexperimenthadthreephases,asdescribedinTable3.4.3-1,withvaryingmodelresolutionandapproachestoinitializingtheatmosphereandoceanfields,aswellasconstrainingtheSeaSurfaceTemperature(SST).
Table3.4.3-1.CICEmodelresolutionatthepole(km),datasetforatmosphericinitializationandforcing,datasetforoceaninitialization,andmethod/datasetforoceanforcing.
CICE Atmos Init and Forcing Ocean Init Ocean Forcing
Phas
e 1 30 km CFSv2 1.00 CFSv2 1.00 CFSv2 10 6-hourly forcing
2 15 km CFSv2 0.20 CFSv2 0.50 CFSv2 0.50 6-hourly forcing
3 15 km CFSv2 0.20 CFSv2 0.50 Freely evolving
Generallyspeaking,thePhase-1andPhase-2forecastsattheendofthemonth-longintegrationsareingoodagreementwiththeCFSv2initialconditionsatthebeginningofthenextmonth,indicatingaverygoodforecastfortheendofthemonth.Themajorexceptionsareinthesummerseasonsofbothhemispheres,whereexcessivemeltingoccurs(Fig.3.4.3-1).Follow-updiagnosticsindicatedthatbasalmelting,causedbywarmSSTs,wasthecauseoftheexcessivemelting.
Figure3.4.3-1.Iceextent(1012m2)duringmonth-longintegrationsforPhase1(dashed)and2(solid)intheNorthern(top)andSouthernhemisphere(bottom).ThecirclesindicatetheinitialconditionsfromCFSv2atthebeginningofeachmonth.
-
34
WhentheSSTfromCFSv2forcingwasreplacedbyfreely-evolvingSSTfromtheCICEinternalmixed-layeroceanmodel,basalmeltingwasreducedandpredictionoficeextentattheendofsummerwassubstantiallyimproved(Fig.3.4.3-2).Comprehensiveresultsfromthistestcanbefoundinthereport(link)andonthetestwebsite(link).
Figure3.4.3-2.SameasFig.3.4.3-1,exceptforPhase2(solid)andPhase3(dashed).
3.5 CloudVerification
AttherequestoftheAF,theDTCinvestigatedapproachesforevaluatingcloudpredictionsfromNWPmodelsandstatisticalpredictionsofcloudproperties.ThisevaluationincludednumerousNWPmodelforecastsutilizedbytheAF.RawmodeloutputfromtheARWlimitedareamodel,overtheNorthernHemisphere,andtheAF