Lesson3cdms2,masks,cdscan,cdutil,genutil

cdms2:selectingdataimport cdms2 DATAPATH = /CAS_OBS/mo/sst/HadISST/ f = cdms2.open(DATAPATH + sst_HadISST_Climatology_1961-1990.nc) x = f(sst) # retrieves the whole dataset - a slab # Selecting a specific area x = f(sst, latitude=(0., 35.), longitude= (20., 100.)) print x.shape # y # z Selecting specific times = f(sst, time=(0-6-1, 0-6-31)) You can also select the specific time from a slab = x(time=(0-6-1, 0-6-31))

# You can squeeze out singleton dimensions y = f(sst, time=(0-6-1, 0-6-31), squeeze=1) # You can also change the order of returned dimensions z = x(time=(0-6-1, 0-6-31), order=xyt)

Queryingtheslab# You can query the order of returned dimensions! print 'Order of dimensions for x is:', x.getOrder() # You can change the order of returned dimensions! x2 = f('sst', latitude=(0., 35.), longitude=(20., 100.), order='xty') # If I just want to ensure that time as my first axis x3 = f('sst', order=t') # You can check the axis and its properties lat_axis = x2.getLatitude() print lat_axis.info() lat_bounds = lat_axis.getBounds() # Axes can be retrieved by index position my_ax = x2.getAxis(1) print my_ax.info()

Bounds!

lat_bounds_2

(lat,lon) lat_bounds_1 lon_bounds_1 lon_bounds_2

Specifyingpreciseregionstoextract# If I need a specific point - say lat, lon = 32.1, 100.3 x3 = f('sst', latitude=(32.1, 32.1, 'cob'), longitude=(100.3, 100.3, 'cob')) print x3.shape # The first 2 positions are either closed or open # The 3rd position is b or e or ..

cdutil overview Setofclimatedataspecificutilities. Thecdutil Packagecontainsacollectionof subpackagesusefultodealwithClimate Data Subcomponentsare: region times:toolstodealwiththetimedimension. vertical averager

cdutil:region Thecdutil.region moduleallowstheuserto extractaregionexactly.i.e.resettingthe latitudeandlongitudeboundstomatchthe areaexactly,thereforecomputinganexact averagewhenpassedtotheaverager function. Predefinedregionsare: AntarcticZone,AAZ(Southoflatitude66.6S) ArcticZone,AZ(Northoflatitude66.6N) NorthernHemisphere,NH#usefulfordatasetwith latitudecrossingtheequator SouthernHemisphere,SH Tropics(latitudesband:23.4S,23.4N)

Cdutil:region cdutil.region cdms2selectortoextractexactregion(i.e reset boundscorrectlysoaveragingaccountforonly actualareaaveragednotthefullcell.S=f(var, cdutil.region.NorthernHemisphere)

CreatingyourownregionsNINO34 = cdms2.selectors.Selector(cdutil.region.domain(l atitude=(-5., 5.), longitude=(190., 240.)))

Interpolation(regridding)# Suppose we have a slab ds1 = f('sst', latitude=(0., 35.), longitude=(20., 100.), \ time=('0-6-1', '0-6-31'), order='xty) print 'ds1.shape = ', ds1.shape, 'Axis order=', ds1.getOrder()

# Let us now extract another dataset.f2 = cdms2.open(/CAS_OBS/climatology/NCEP_NCAR_Climatology_ltm/slp. day.ltm.nc') ds2 = f2('slp', latitude=(0., 35.), longitude=(20., 100.), \ time=('1-6-1', '1-6-31')) # I want to transform ds2 into the grid in ds1 ds3 = ds2.regrid(ds1.getGrid()) print 'ds3.shape = ', ds3.shape, 'Axis order=', ds3.getOrder() # Alternate regridder from regrid import Regridder ingrid = ds2.getGrid() outgrid = ds1.getGrid() regridFunc = Regridder(ingrid, outgrid) new_ds2 = regridFunc(ds2)

Verticalregridding Youcanregrid pressurelevelcoordinatesinthevertical axisusingthepressureRegrid()method. Youneedtodefine,oruseanexisting,verticalaxis. ThenusethepressureRegrid methodonthevariable youwishtoregrid,passingitthenewlevelasthe argument: Ifvar isthevariabletoregrid andthenewlevs is theverticalaxistoregrid to:>>> var_on_new_levels = var.pressureRegrid(levout)

cdms2:Usingmaskssst_mask = cdms2.MV2.getmask(ds1(order='tyx')) print 'sst_mask.shape after reorder = ', sst_mask.shape print sst_mask.__class__ # So we resize the mask sst_mask.resize(ds3.shape) # ds4 = cdms2.createVariable(ds3[:], mask=sst_mask, id='masked_psl', fill_value=1.e+20)

cdscan Autilitythathelpsyoumanagefilesbetter. Whenyouhavemany.nc (or.ctl)filesyoucan usethisutilitytogenerateasinglexmlfile thatmakeslifesimple. Trythefollowing:cdscan -x some_filename.xml DATA_PATH1/*.nc

Youcanalsochangethetimeaxiswhileyouareat it!cdscan x some_filename.xml i 1 rmonths since 1-1-1 DATA_PATH2/*.nc

cdutil:times cdutil.times fortimeaxes,gearedtowardclimatedata Climatology,Departures,AnomaliesToolsworksonBOUNDS, NOTontimevalues,designedformonthlyseasons,butone cancreateanengineforotherkindofdata(daily,yearly, etc).ac=cdutil.times.ANNUALCYCLE.climatology(s)

Inordertosetboundsyoucanuse:cdutil.setTimeBoundsMonthly(Obj) cdutil.setTimeBoundsYearly(Obj) cdutil.setTimeBoundsDaily(Obj, frequency=1)(Obj canbeslabortimeaxis)

Createyourownseasons:MONSOON = cdutil.times.Seasons(JJAS)

cdutil importseverythinginthetimes modulesoyoucanjust calle.g.:cdutil.setTimeBoundsMonthly(slab/axis)

Theimportanceofbounds

CDATusedtosetboundsautomatically.E.g.:longitude = [0, 90, 180, 270]

bounds = [[-45, 45], [45, 135], [135, 225], [225, 315]]

Seemsreasonable,butimagineamonthlymeantimeserieswherethetimes arerecordedon1st dayofeachmonth: timeax=[1999-1-1, 1999-2-1, , 2100-12-1] CDATassumesthateachmonthrepresentstheperiodof15th lastmonthto 15th thismonth. Sincecdutil toolsuseboundstheywillbemisinterpretingthedata.Needto settheboundssensibly: >>> cdutil.setTimeBoundsMonthly(timeax)

Predefinedtimerelatedmeans DJF,MAM,JJA,SON(seasons)>>> djf_mean=cdutil.DJF(my_var)

SEASONALCYCLE(meansforthe4predefinedseasons [DJF,MAM,JJA,SON]) arrayofabove.>>> seas_mns=cdutil.SEASONALCYCLE(my_var)

YEAR(annualmeans) ANNUALCYCLE(monthlymeansforeachmonthofthe year) EXERCISE:Trycalculatingtheclimatological annualcyclefor theNCEPReanalysisdatayouhavereadin.

Climatologies anddeparturesSeasonextractorshave2functionsavailable: climatology:whichcomputestheaverageofall seasonspassed. ANNUALCYCLE.climatology(),willreturn the12monthannualcyclefortheslab:>>> ann=cdutil.ANNUALCYCLE.climatology(v)

departures:whichgivenanoptionalclimatologywill computeseasonaldeparturesfromit.>>> d=cdutil.ANNUALCYCLE.departures(v, cli60_99)

#Notethatthesecondargumentisoptionalbutcanbeaprecomputed climatologysuchasherecli60_99 isa19601999climatologybutthevariablev is definedfrom19002000.Ifnotgiventhentheoverallclimatologyforv isused.

Simpleuserdefinedaveraging Youcancreateyourownsimpleaveragesusing arrays,slabsorvariablesintheusualway: Averagingover4timesteps:>>> t.shape (4, 181, 360) >>> av=(t[0]+t[1]+t[2]+t[3])/4

Drawbacks: Doesntretainyourmetadata. Cannotaveragesimplyacrossaxeswithinavariable.

MV2Averaging TheMV2modulehasanaveragingfunction:MV2.average(x, axis=0, weights=None, returned=0)

computestheaveragevalueofthenonmaskedelementsofx alongtheselectedaxis.Ifweightsisgiven,itmustmatchthe sizeandshapeofx,andthevaluereturnedis:

elementscorrespondingtothosemaskedinx orweightsare ignored.Ifreturned,a2tupleconsistingoftheaverageand thesumoftheweightsisreturned.

MV2Averaging:example Tocalculateasetofzonalmeans:import cdms2, MV2 f=cdms2.open(/CAS_OBS/mo/sst/HadISST/ sst_HadISST_Climatology_1961-1990.nc) data=f(sst) print data.shape zm=MV2.average(data, axis=2) print zm.shape print zm.info()

Thecdutil averagerfunction Thecdutil.averager()functionisthekeyto spatialandtemporalaveraginginCDAT. Masksaredealtwithimplicitly. Powerfulareaaveragingfunction. Providescontrolovertheorderofoperations (i.e.whichdimensionsareaveragedoverfirst). Allowsweightingsforthedifferentaxes: passyourownarrayofweightsforeachdimension, usethedefault(grid)weightsorspecifyequal weighting.

Usageofcdutil.averagerresult = averager( V, axis=axisoptions, weights=weightoptions, action=actionoptions, returned=returnedoptions, combinewts=combinewtsoptions)axisoptions hastobeastring.Youcanpassaxis='tyx',or'123',or'x (plev). weightoptions isoneof'generate|weighted|'equal'|unweighted| array|MaskedVariable actionoptions is'average'|'sum[Default='average]. Youcaneitherreturntheweightedaverageortheweightedsumofthe data.

Example:RegionAveraging,and Climatologyimport cdutil # define the your custom regions. NINO3 = cdms2.selectors.Selector(cdutil.region.domain(latitude=(-5., 5., 'ccb'), longitude=(210., 270., 'ccb'))) fsst = cdms2.open(INDIR + 'sst_HadISST_1870-1_2011-1.nc) nino3_data = fsst('sst', NINO3) print nino3_data.shape, nino3_data.getOrder() # Compute the Spatial average nino3_average = cdutil.averager(nino3_data, axis='xy') # Anomaly from climatology computed over 1961-1990 nino3_slice = nino3_average(time=('1961-1-1', '1990-12-31')) nino3_clim = cdutil.ANNUALCYCLE.climatology(nino3_slice) print nino3_clim.shape # Now departures nino3_anomaly = cdutil.ANNUALCYCLE.departures(nino3_average, nino3_clim) print nino3_anomaly.shape

EXERCISE ExtracttheSSTdataandcomputeglobal anomaliesfromthe19611990climatologyfor thewholelengthofdataset. Averagetheanomalydata overx andy axes usingequalweightsforbothaxesand compareagainstareaweightedaverage.

genutil:generalutilities genutil.statistics:setofbasicstatistical functions correlation,covariance,geometricmean, laggedcorrelation,laggedcovariance, linearregression,meanabsdiff,median, array_indexing,percentiles,arrayindexing, rank,autocorrelation,rms,autocovariance, std,variance

StatisticsExample c1=genutil.statistics.correlation(a,b,axis=t) c1.shape c2=genutil.statistics.correlation(a,b, axis=xy) c2.shape

Supportforothergridtypes

RectGrid Associatedlatitudeandlongitudeare 1Daxes,withstrictlymonotonicvalues. CurveGrid Latitudeandlongitudeare2D coordinateaxes(Axis2D). GenericGrid Latitudeandlongitudeare1D auxiliarycoordinateaxes(AuxAxis1D)

CurvilinearandGenericGrids

Acknowledgements DeanWilliams,CharlesDoutriaux (PCMDI, LLNL) Dr.JohnnyLin