ec-earth f2f esgf...• certificates and pids landing pages primavera specific ec-earth specific bsc...
TRANSCRIPT
![Page 1: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/1.jpg)
EC-Earth andESGF
KimSerradellPierre-Antoine Bretonniere
(Computational Earth Sciences,BSC)Prashanth Dwarakanath
(Linkoping University/NSC)
03/12/2017 7thAnnual ESGFF2F2017
AR
![Page 2: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/2.jpg)
Outline
• BarcelonaSupercomputingCenter• EC-Earthclimatemodel• BSCExperience• LIU• EC-EarthandESGF
![Page 3: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/3.jpg)
BarcelonaSupercomputing CenterCentroNacionaldeSupercomputación
SpanishGovernment 60%
CatalonianGovernment 30%
Univ.Politècnica deCatalunya (UPC)10%
BSC-CNSisaconsortiumthatincludes
BSC-CNSobjectives
SupercomputingservicestoSpanishandEUresearchers
R&DinComputer,Life,Earthand
EngineeringSciences
PhDprogram,technologytransfer,publicengagement
![Page 4: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/4.jpg)
EarthSciencesDepartmentWhat
Environmental forecasting
HowDevelop a capability to model air quality processes from urban to global and the impacts on weather, health and ecosystems
Implement climate prediction system for subseasonal-to-decadal climate prediction
Develop user-oriented services that favour both technology transfer and adaptation
Use cutting-edge HPC and Big Data technologies for the efficiency and user-friendliness of Earth system models
Why
Our strength …… research …… operations …… services …… high resolution …
Earth system services
Atmospheric composition
Computational Earth sciences
Climate prediction
![Page 5: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/5.jpg)
EC-Earth
![Page 6: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/6.jpg)
EC-EarthModelEC-EARTH coupled model
• TheIntegratedForecastingSystem(IFS)asatmospheremodel
• TheNucleusforEuropeanModellingoftheOcean(NEMO)asoceanmodel
• TheOASIS3-MCT coupler
• TheLouvain-la-Neuve sea-IceModel3(LIM3)asseaicemodel
![Page 7: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/7.jpg)
Communitydrivendevelopment
Schematic outline ofthe EC-Earth consortium andits members asofMay 2014.
![Page 8: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/8.jpg)
PRIMAVERAH2020
• a EuropeanUnionHorizon2020 project,• anewgenerationofadvancedandwell-evaluatedhigh-resolutionglobalclimatemodels,
• simulationsandpredictionsofregionalclimatewithunprecedentedfidelity,
• forthebenefitofgovernments,businessandsocietyingeneral.
Oceansurfacecurrentsfrom
HadGEM3-basedglobalcoupled(atmosphere-ocean/sea-ice)modelsatthree
differentresolutions- (left)25km-1/12degree,(middle)60km-1/4degree,(right)130km-1degree(courtesyof
MalcolmRoberts)
![Page 9: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/9.jpg)
BSCExperience
![Page 10: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/10.jpg)
BSCDataNode
• ESGFTier2“data”node:publishingBSCandAEMETPRIMAVERAandCMIP6data
• IndexnodeatSTFC(partnerinvolvedinPRIMAVERA)
• PRIMAVERAStream1datastilluploadedtoJasminbutpublishedfromBSCnode
• PRIMAVERAStream2datawillbeonlystoredandpublishedfromBSC
• Estimateof~600TBofPRIMAVERAdata
![Page 11: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/11.jpg)
BSCDataNode
Indexnode Datanode
![Page 12: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/12.jpg)
BSCDataNode
![Page 13: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/13.jpg)
BSCcmorization
• Part ofPRIMAVERAproject
• ece2cmor3 is trying tostandardize the cmorization within EC-Earth
• Developed by asetofpartners (KNMI/e-Science CenterandBSC)
• But each partner cmorizes its own data(lots ofcoordination needed)• online(on HPCasthe simulation runs toonly storecmorized data)• offline(adding timetopublish)
![Page 14: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/14.jpg)
BSCDataNode(installer)
• March2017:
• startedinstallingversion2.3topublishCMIP5/SPECSdataastest
=>impossibleduetodependenciespointingtoupdatedversionsofpackagesbreakingcompatibilityoftheoverallinstallation
• August2017:
• Successfullyranversion2.5oftheinstaller,evenifstillunderdevelopment
![Page 15: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/15.jpg)
BSCDataNode(publisher)
Oncewehadthenodeupandrunning,updating:
• updatingtheesg.[primavera-cmip6].ini
• esgcet_models_table.txt
• certificatesandPIDslandingpages
PRIMAVERAspecific
EC-Earthspecific
BSCspecific
![Page 16: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/16.jpg)
BSCrecommendations
• Freshstartinstallationtesting
• Semanticversioningof• dependencies(gitsubmodules?)• releases
• Allowuserstoinstalldependenciesbyhandwithintheinstallerrun
• Identifywho(personorinstitution)isresponsibleforwhat(PIDs,installation,publication,...)
• Wiki/documentationfor“beginners”• Howto’s?• Howtodeployaproductionandatestingserver
![Page 17: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/17.jpg)
LIU
![Page 18: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/18.jpg)
LiU on ESGFandCDNOT
• LiU operatesaTier-1ESGFnode,offeringallESGFservices,includingindexpeeringtootherdatanodes,fundedbySMHIandIS-ENESprojects.
• HostsCORDEX,CMIP5,SPECS,andCLIPCprojects,fromSMHI.• ManagesattributeservicesforCORDEXdataaccess.• [email protected],forusersregisteredtoaccessCORDEXdataonESGF.
• LiU ManagesoneoftheESGFFederationCAsandhelpeddevelopCApolicyforESGF.
• Prashanth Dwarakanath (LiU)co-leadstheESGFInstallationWorkingTeam,andisarepresentativeforEC-EARTHinCDNOT.
• KaiLu(LiU)isanactivecontributortousersupportonESGFUsersandIWTmailinglists.
![Page 19: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/19.jpg)
Ongoing andfuture activities on CDNOT
• LiU hascontributed toachieving better documentation byauthoring manuals for datanode administrators,andlookstotake an activeroleindocumenting procedures for CMIP6datapublication andnode operations.
• Hasconducted trainingworkshopsandcode sprints andisworking on developing asuiteoftools toreduceuncertaintieswith infrastructure setups for workshopsandcollaborativedevelopment events.
• Actively working on setting upofan Incident-responseteam,andtodisseminate traininganddocumentation toensureadoption ofsecurity best-practices.
![Page 20: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/20.jpg)
EC-EarthandESGF
![Page 21: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/21.jpg)
Previousexperiences
• CMIP5andSPECS• CentralizedatBADC
• Onlyareducedsetofpartnerswasinvolvedinthepublic• DataNodeinLIU• Partnerswithout datanodehadto:
• Upload lotsofdata• Needingassistance
![Page 22: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/22.jpg)
Futureexperiences
• CMIP6(PRIMAVERA)• Moredata
• spatialresolutionincreasing• spatialdisseminationincreasing(morepartners)
• Morepartners• moreESGF/EC-Earthdatanodes• more“ESGFbeginners”
• Howtodealwithsuchaproject?• attheEC-Earthlevel• attheESGFlevel
![Page 23: EC-Earth F2F ESGF...• certificates and PIDs landing pages PRIMAVERA specific EC-Earth specific BSC specific BSC recommendations • Fresh start installation testing • Semantic](https://reader033.vdocuments.mx/reader033/viewer/2022043011/5fa6fec30ccc3f4042682eda/html5/thumbnails/23.jpg)
EC-Earth’scontributionstoCMIP6Institute Contact Contributions HPCplatform ESGFnode
1.AEMET J.A.ParodiPerdomo
DECK(EC-Earth3-CC) CrayXC30@ECMWF+BULL@AEMET
?(BSC?)
2.BSC/IC3 F. Doblas-Reyes DECK,DCPP,HighResMIP (?) Marenostrum@BSC BSC
3.CNR J.vonHardenberg DECK (EC-Earth-CC,EC-Earth-CC-LR) CNR
4.DMI S. Yang DECK,DCPP,GeoMIP,HighResMIP, ISMIP6,LS3MIP,SenarioMIP
CrayXC30@DMI/IMO DMI
5.ENER A.Alessandri LS3MIP CrayXC30@ECMWF
6.FMI H.Korhonen AeroChemMIP CrayXC30@FMI FMI/UHEL
7.IPMA P.Viterbo ??
8. KIT A.Arneth C4MIP,LUMIP
9.KNMI R.Bintanja DECK,AeroChemMIP,CFMIP(?),HighResMIP,SenarioMIP
CrayXC30@ECMWF+BULL@KNMI
KNMI
10.MetEirre/ICHEC
R.McGrath DECK,HighResMIP,SenarioMIP CrayXC30@ECMWF+CrayXC40@PRACE
ICHEC
11.LU P. Miller C4MIP,LUMIP
12.MISU L.Brodeau DECK
13.NGSU Q.Zhang PMIP NSC
14.SMHI K.Wyser DECK,C4MIP,HighResMIP,C4MIP,SenarioMIP
LIU
15.UHEL R.Makkonen AeroChemMIP CrayXC30@CSC FMI/UHEL