1 3

DOI 10.1007/s00382-016-2994-xClim Dyn

Evaluating global reanalysis datasets for provision of boundary conditions in regional climate modelling

Ditiro B. Moalafhi1 · Jason P. Evans2 · Ashish Sharma1

Received: 24 August 2015 / Accepted: 15 January 2016 © Springer-Verlag Berlin Heidelberg 2016

climate variability, ERA-I is the best followed by MERRA. Overall, MERRA is preferred for generating lateral bound-ary conditions for this domain, followed by ERA-I. While a “better” LBC specification is not the sole precursor to an improved downscaling outcome, any reduction in uncer-tainty associated with the specification of LBCs is a step in the right direction.

Keywords Reanalyses · AIRS · Lateral boundary conditions · RCM · Southern Africa

1 Introduction

With the inherently complex interplay of atmospheric processes, the need for studying the possible atmospheric mechanisms underlying both regional and global climate has necessitated a need for consistent data sets pertaining to atmospheric circulation. The most convenient data sets in this regard are in the form of global reanalyses. In most cases, the products of reanalyses are freely available and have gained wide usage (Bromwich and Fogt 2004). Rea-nalysis is described as a climate or weather model simula-tion of the past that includes data assimilation of historical observations (Bengtsson et al. 2007). The use of reanaly-sis products has become widespread in the field of climate research due, in part, to the lack of globally and tempo-rally complete direct observations (Qian et al. 2006; Zhang et al. 2013). For this reason reanalysis products have been used to drive land surface models, study the climate sys-tem and provide lateral boundary forcing for regional cli-mate models (Decker et al. 2012). We present here a 4-D framework for assessing the relative accuracy of alternate reanalysis products in reference to their use for specifying

Abstract Regional climate modelling studies often begin by downscaling a reanalysis dataset in order to simulate the observed climate, allowing the investigation of regional cli-mate processes and quantification of the errors associated with the regional model. To date choice of reanalysis to perform such downscaling has been made based either on convenience or on performance of the reanalyses within the regional domain for relevant variables such as near-surface air temperature and precipitation. However, the only infor-mation passed from the reanalysis to the regional model are the atmospheric temperature, moisture and winds at the location of the boundaries of the regional domain. Here we present a methodology to evaluate reanalyses derived lateral boundary conditions for an example domain over southern Africa using satellite data. This study focusses on atmospheric temperature and moisture which are easily available. Five commonly used global reanalyses (NCEP1, NCEP2, ERA-I, 20CRv2, and MERRA) are evaluated against the Atmospheric Infrared Sounder satellite tempera-ture and relative humidity over boundaries of two domains centred on southern Africa for the years 2003–2012 inclu-sive. The study reveals that MERRA is the most suitable for climate mean with NCEP1 the next most suitable. For

Electronic supplementary material The online version of this article (doi:10.1007/s00382-016-2994-x) contains supplementary material, which is available to authorized users.

* Ashish Sharma a.sharma@unsw.edu.au

1 School of Civil and Environmental Engineering, The University of New South Wales, Sydney, NSW 2052, Australia

2 Climate Change Research Centre, The University of New South Wales, Sydney, NSW 2052, Australia

D. B. Moalafhi et al.

1 3

atmospheric lateral boundary conditions (LBCs) over a domain prior to use in regional climate modelling.

Although reanalyses use some common station data, the products differ in various ways. They use different verti-cal and horizontal resolutions, data assimilation methods, physical parameterizations and sea surface temperature prescriptions for boundary conditions. While reanalyses continue to contribute significantly to a more detailed and comprehensive understanding of the dynamics of the earth’s atmosphere, the various reanalysis products show deficiencies and inconsistencies especially at regional scales. Bromwich et al. (2007) compared several reanaly-sis datasets including National Centre for Environmental Prediction/National Centre for Atmospheric Research Rea-nalysis (NCEP1), NCEP2, European Centre for Medium-Range Forecasts Reanalysis (ERA-40), ERA-15 and Japanese 25-year Reanalysis (JRA-25) over high latitude regions and concluded that unreliability of reanalyses dur-ing non-summer months over Antarctica was due to limited quantities of satellite data for assimilation prior to the mod-ern satellite era. From this, it was inferred that performance would be worse for the data scarce southern hemisphere. Over the Amazon basin, European Centre for Medium-Range Forecasts Reanalysis-Interim (ERA-I) is cold-biased compared to in situ station data (Betts et al. 2009). The cold bias of ERA-I was also found by Brands et al. (2013) in evaluating reanalyses uncertainty over tropical African regions. In yet another study mostly over tropical Africa and also covering large portions of the Atlantic ocean and the western Indian ocean by Druyan and Fulakeza (2013), it was revealed that though ERA-I has an edge in forcing a regional atmospheric model (RM3) for precipitation simu-lation, NCEP2 lateral boundary conditions produced better simulations of some features. As more reanalyses became available to the climate research community, evaluation of reanalysis products has increased over time (Bosilovich et al. 2008) but it is still worth noting that these evalua-tions have different scientific focus, precluding a common conclusion about which reanalyses is superior (Zhang et al. 2013). It should also be noted that attempts have been made to modify lateral boundaries for biased GCM simulations (Rocheta et al. 2014) resulting in dynamical inconsisten-cies based on the bias correction adopted. Approaches that attempt to correct bias across the entire lateral boundary variable field (instead of one at a time) are considered more appropriate for this purpose (Mehrotra and Sharma 2015).

One of the very few studies in which reanalyses were evaluated at high temporal resolution (i.e., daily), focusing on surface variables, was by Pitman and Perkins (2009). In that study, evaluations were made on ERA-40, NCEP2 and JRA-25 over five regions including Africa. Bosilovich et al. (2008) also evaluated surface products of ERA-40, JRA-25, NCEP1, and NCEP2 at global and regional scales

and concluded that there are inconsistencies in quality of reanalysis products and improvements to a reanalysis data-set may be limited to the availability (in both space and time) and quality of the assimilated data. Decker et al. (2012) evaluated the performance of Climate Forecast Sys-tem Reanalysis (CFSR), ERA-40, ERA-I, and Modern-Era Retrospective Analysis for Research and Applications (MERRA) products against flux tower observations of tem-perature, wind speed, precipitation, downward shortwave radiation, net surface radiation, and latent and sensible heat fluxes at 33 different locations in the Northern Hemisphere. The evaluations, which were made at 6-hourly and monthly time-scales, gave mixed results and thus no reanalyses was identified as the best across all the variables. Over south-ern Africa, Zhang et al. (2013) evaluated ERA-40, ERA-interim, JRA-25, MERRA, CFSR, NCEP-R1, NCEP-R2 and 20CRv2 based on seasonal cycle in precipitation and concluded that 20CRv2 and CFSR performed better than the rest.

It is worth noting that even when reanalyses are evalu-ated for provision of LBCs for regional climate modelling, such evaluations have only been done at the surface. When running regional climate models (RCMs), lateral bound-ary conditions (LBCs) that extend vertically through the atmosphere need to be created. Other than the initial con-ditions which are discarded along with several months of spin-up time for climate simulations, these LBCs are the dominant input to RCM simulations and as such they are assumed to be accurate and reliable though to date this has rarely been tested. If one wants to downscale the climate of the recent past with a RCM then LBCs are created from an available reanalysis. The choice of which reanalyses to use in this situation may be informed by studies such as those above that evaluated reanalyses performance based on surface variables. However, variables such as near sur-face air temperature and precipitation within the regional domain are dependent on both parameterizations in the rea-nalysis system (e.g. for the land surface or cumulus con-vection) that differ from the regional model, and on data that is assimilated within the regional domain. Thus the information in these fields only partially reflects informa-tion present in the LBCs for RCMs and the results of such evaluations may be only partially relevant. For the first time, this study evaluates 4-D variables directly relevant to RCM LBCs, atmospheric temperature and humidity, at the boundaries of an example domain to quantify errors associ-ated with these fields and the potential for using such an evaluation to determine which reanalysis can provide the most robust LBCs for long term climate simulations over Southern Africa. Ideally such an evaluation would address all atmospheric LBCs (temperature, moisture and winds) however a spatially continuous wind dataset does not cur-rently exist limiting this evaluation to temperature and

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

moisture. The reanalyses wind fields are compared to each other at the lateral boundary locations to assess how much variability between reanalyses exists. While the methodol-ogy is general, the application is specific to the location of the lateral boundaries and the evaluation results may differ for other regional domains. Here the technique is evaluated for its robustness to small domain variations by considering an additional domain with lateral boundaries offset from the original domain. The technique described herein can be applied to any region, and can be evaluated directly at the location of the intended RCM boundaries. This should assist with the choice of reanalysis to use to provide bound-ary conditions to the RCM so that the most accurate LBCs possible are used for long climate simulations. While good modelling outcomes may occur for a number of reasons, using the best set of inputs (in our case LBCs) is a neces-sary first step in any model application exercise.

The rest of the paper is as follows. Datasets which include Atmospheric Infrared Sounder (AIRS) satellite data and global reanalyses data are described in Sect. 2. Descriptions of statistical measures employed in evalua-tions of reanalyses against AIRS satellite data are presented as methodology in Sect. 3 followed by Sect. 4 where results are presented and discussed. Conclusions follow in Sect. 5.

2 Datasets

2.1 Observations

Atmospheric Infrared Sounder (AIRS) satellite data ver-sion 6 was chosen as it provides 4-D global data for both temperature (24 levels) and relative humidity (12 levels). AIRS was launched on EOS Aqua on 4 May 2002, together with AMSU A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sound-ing system (Susskind et al. 2006). The instrument suite is designed to support climate research and improve weather forecasting. The data, which is at 1° × 1° grid resolution, is available from National Aeronautics and Space Admin-istration (NASA) at http://acdisc.gsfc.nasa.gov/opendap/Aqua_AIRS_Level3/. The data used are level 3 standard products of both descending and ascending overhead passes at 1:30 and 13:30 h local time respectively referenced by 0° longitude every day.

AIRS is the first of a new generation of advanced sat-ellite-based atmospheric sounders that produces daily, high accuracy, high vertical and horizontal resolution pro-files of temperature, water vapour and minor gases, along with cloud and surface properties over most of the Earth’s surface (Aumann et al. 2003; Tobin et al. 2006). The high-accuracy retrieval goals of AIRS of 1 K Root Mean Squared errors in 1 km layers below 100 hPa for air temperature,

10 % Root Mean Squared errors in 2 km layers below 100 hPa for water vapour concentration, combined with the large temporal and spatial variability of the atmosphere and difficulties in making accurate measurements of the atmos-pheric state, necessitated continuous careful and detailed validation using well-characterized ground-based sites (Tobin et al. 2006). In this regard the continuing efforts to validate and give quality assurance to AIRS products, has always been positive (Fetzer 2006; Divakarla et al. 2006). One of the insurance factors in use of AIRS products, for example, is that temperature profile accuracy in the tropo-sphere matches that achieved by radiosondes. Temperature and water vapour retrievals from AIRS are in good agree-ment with global radiosonde measurements over both land and ocean (Divakarla et al. 2006). With the latest version (version 6), significant improvement in the ability to obtain both accurate temperature profiles and surface skin tem-peratures under partial cloud cover conditions has been achieved (Susskind et al. 2014).

2.2 Reanalysis products

The reanalysis products which were used in the study are the European Centre for Medium-Range Forecasts Interim Reanalysis (ERA-I), National Centre for Environmen-tal Prediction/National Centre for Atmospheric Research Reanalysis (NCEP1), National Centre for Environmen-tal Prediction-Department of Energy reanalysis (NCEP2), Modern Era Retrospective-analysis for Research and Applications (MERRA), and Twentieth Century Reanalysis Version 2 (20CRv2). The Climate Forecast System Reanal-ysis (CFSR) could not be considered as it does not cover the entire period of interest up to the end of 2012. NCEP1, NCEP2 and 20CRv2 data were available from Earth Sys-tem Research Laboratory (ESRL) at http://www.esrl.noaa.gov/psd/data/while ERA-I data were sourced from European Centre for Medium-Range Weather Forecasts (ECMWF) at http://apps.ecmwf.int/datasets/. MERRA data were available from National Aeronautics and Space Administration (NASA) at http://disc.sci.gsfc.nasa.gov/daac-bin/DataHoldings.pl. Attributes of these reanalysis are summarized in Table 1.

While all reanalyses assimilate some of the same data there are differences between them. NCEP1 and NCEP2 use the lowest resolution, followed by 20CRv2 and MERRA with ERA-I using the highest resolution. 20CRv2 does not assimilate any observations in the upper atmos-phere, and hence is less constrained there than the other reanalyses. ERA-I and MERRA are forced by the same SST dataset (OISST—Reynolds et al. 2007) and hence are likely to be more similar near the surface over the ocean. Ultimately, differences between the reanalyses are caused

D. B. Moalafhi et al.

1 3

by many interacting processes within these extremely large and complicated model/data systems.

3 Methodology

Evaluations were made at the boundaries of the main domain (North, South, East and West) and stretching inwards by 4°, as shown in Fig. 1. ERA-I was available at 0.75° × 0.75°, NCEP1 and NCEP2 at 2.5° × 2.5°, 20CRv2 at 2.0° × 2.0°, and MERRA at 1.5° × 1.5° grid resolutions. Picking the boundary panels at 4° was mostly deemed accommodative of the grid resolutions of all the reanalysis products and also gives some flexibility in variations of the boundaries of the domain. The resultant boundary panels of the domain were thus demarcated as North (8.5 to 50.5°E, 0.5 to −2.5°S), South (8.5 to 50.5°E, −37.5 to −40.5°S), East (50.5 to 53.5°E, 0.5 to −40.5°S), and West (5.5 to 8.5°E, 0.5 to −40.5°S). Elevation over the domain varies between 0 and 3500 m. The highest areas are mostly the mountainous area of Lesotho at around longitude 29° and

latitude −30° towards the south and some areas northwards around longitude 29°–35° and latitude 0°–5°.

3.1 Processing of AIRS observational data

Level 3 AIRS satellite data version 6 (2003–2012) were available globally at 1° × 1° grid resolution. Both the descending (1:30 am) and ascending (1:30 pm) overpass times are combined to produce a time series of data every 12 h. These were then sub-setted to the four (4) boundary panels but maintaining the original 1° × 1° grid resolution and taking pressure levels of 50 hPa to 1000 hPa inclusive (50, 70, 100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, 1000 hPa) for temperature and 100 hPa to 1000 hPa inclusive (100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, 1000 hPa) for relative humidity. For derivation of summary statistics for relative humidity, data is used for pressure levels of 300 hPa to 1000 hPa since NCEP1’s rela-tive humidity covers only levels of 300 hPa to 1000 hPa.

3.2 Processing of reanalyses data

Reanalyses temperature and relative humidity were availa-ble at 6 hourly intervals and 3 hourly intervals for MERRA. The products were sub-setted to the four different bound-ary panels and re-gridded using linear interpolation to the 1° × 1° grid resolution of AIRS data and the chosen set of pressure levels.

3.3 Evaluation of reanalyses against AIRS observations

Since the AIRS data are available every 12 h, reanalyses data were linearly shifted to the over-head pass times of the AIRS data. The 1-h lag from coordinated universal time (UTC) for every 15° longitude across the domain was also taken into consideration. The time shifted reanalysis were then re-sampled to the same time resolutions of AIRS (i.e., 12 hourly). Mean bias, root mean squared error (RMSE), mean absolute error (MAE) and temporal correlation were derived over the individual boundary panels to compare the time series of reanalyses variables with corresponding time

Table 1 Summary details of the global reanalyses that were used in the study

Source: Zhang et al. (2013)

Name Organization Temporal coverage Horizontal resolution Vertical levels SST & sea-ice forcing Assimilation

ERA-I ECMWF 1979–present T255 (80 km) 60 Daily OISST (from 2002)

4DVAR

NCEP-R1 NCEP/NCAR 1948–present T62 (210 km) 28 GISST 3DVAR

NCEP-R2 NCEP/DOE 1979–present T62 (210 km) 28 SST AMIP-2 3DVAR

MERRA NASA GMAO 1979–present 1.5° by 1.5° (167 km) 72 Weekly OISST 3DVAR

20CRv2 NOAA/ESRL PSD 1871–present T62 (210 km) 28 HadlSST1.1 Ensemble Kalman Filter

Fig. 1 Elevation in metres above mean sea level (mamsl) showing the range over both the main and secondary domain

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

series from AIRS at each of the four lateral boundaries. Mean Bias here is the simple difference of the means given by

where Mj is reanalysis averaged temperature or relative humidity over an jth boundary panel and Oj is averaged AIRS (observations) temperature or relative humidity over the jth panel.

Along with the bias, the RMSE is calculated for each panel to determine which of the reanalyses is closest to the AIRS data.

where n is the number of time steps (i.e., 7306), Oi and Mi are the AIRS observations and reanalysis product values respectively for i = 1,…,n. Here i is an ith time step.

Average magnitude of the errors without considering their direction was computed through MAE.

where M is the reanalysis product, O is the AIRS observa-tion, i is the ith time step and n is the number of time steps.

The temporal correlation coefficient is used to evalu-ate the goodness-of-fit by performing linear regression between reanalyses and AIRS observations.

where n is the number of time steps (7306), oi = Oi − Oi, mi = Mi − M, Oi and Mi are the ith time step AIRS obser-vation and ith time step reanalysis product for i = 1,…,n, and O and M are mean values of AIRS observations and reanalysis products respectively.

The reanalyses performances are also ranked to choose the most suitable reanalysis for downscaling over the domain. For the smallest error statistic under a panel when considering a field, a reanalysis gets a score of 1. Equal error statistics are given equal rank among the reanalyses. For cor-relation, the best performance (i.e., rank 1) is for the highest temporal correlation coefficient. These scores based on all error statistics are added for all the panels for both tempera-ture and relative humidity individually and collectively. The reanalysis that has the smallest overall score is considered the overall best. There are different interests in downscaling reanalyses which usually influence the error statistics used to choose the most suitable reanalysis. The different interests

(1)Mj − Oj

(2)RMSE =

√

√

√

√

1

n

(

n∑

i=1

(Mi − Oi)2

)

(3)MAE =1

n

n∑

i=1

|Mi − Oi|

(4)r =

∑ni=1

oimi√

∑ni=1

o2i

√

∑ni=1

m2

i

can broadly be grouped into climate mean and climate vari-ability (temporal). To give an idea on which reanalysis could be most suitable under the broad categories, reanalyses rank-ing are further made for climate mean (mean bias) and cli-mate variability (temporal correlation) individually.

Although we focused on 12-hourly time steps of the AIRS fields, an attempt was also made to give an idea on whether the comparisons based on the error statistics are valid at weekly, monthly and annual time scales.

4 Results and discussion

Based on the four boundary panels, mean Bias, RMSE, MAE and temporal correlation were determined for the dif-ferent reanalyses based on comparisons with AIRS obser-vations and averaged over all pressure levels. The evalu-ations are made over the whole study period (2003–2012 inclusive), the driest year (2005) and the wettest year (2012) at 12-hourly time intervals based on the timescale of AIRS satellite data. Results for all reanalyses are dis-played in tables. Vertical profiles and cross-sections are also examined. Spatial patterns of the temperature and rela-tive humidity fields are not directly assessed in this study. This assessment is used to arrive at an overall performance associated with each of the datasets, across all the variables analysed. Results from these assessments are presented in the following sub-sections, as well as the supplementary material associated with the article.

4.1 Mean bias

4.1.1 Temperature

Based on Table 2, all reanalyses overestimate tempera-ture over all the boundary panels with the lowest bias over the northern boundary panel (except for 20CRv2 and MERRA). Among the reanalyses, ERA-I has the lowest bias over all the boundary panels. The reanalyses perfor-mances are ranked (Table 5) and ERA-I has the best rank-ing for temperature mean bias over all panels.

Over the vertical levels of the atmosphere (Fig. 2), rea-nalyses have some variations in mean bias. Generally rea-nalyses under-estimate temperature at 700 hPa over all the boundary panels. Other under-estimations are near the top of the atmosphere (i.e., at 50 hPa). From the 700 hPa pressure level across all the boundary panels, tempera-ture is over-estimated towards the lower atmosphere (i.e., 1000 hPa). All reanalyses are within 1.5 K of each other through most of the atmosphere except at 100 hPa, 70 hPa and 50 hPa where 20CRv2’s over-estimation of tempera-ture is clear deviation from the rest of reanalysis.

D. B. Moalafhi et al.

1 3

Based on Fig. 3 (see also Supplemental figure S1), temperature is under-estimated at 700 hPa across all the boundary panels over latitude. This under-estimation of temperature at 700 hPa was also revealed by Fig. 2. ERA-I also underestimates temperature just above the tropo-pause while for the rest of reanalyses, temperature is over-estimated at that level. Temperature over-estimations for NCEP1, NCEP2, ERA-I and MERRA at the lower levels of the atmosphere are more pronounced at high latitudes.

4.1.2 Relative humidity

It is evident that all reanalyses overestimate relative humid-ity over all the boundary panels except NCEP2 for the northern boundary panel (Table 2). Generally ERA-I has the largest positive bias over the boundary panels. Among the reanalyses, NCEP1 has the smallest bias for the north-ern and western boundary panels. For the southern bound-ary panel, MERRA has the lowest bias while NCEP2 has the lowest bias for the eastern boundary panel. Ranking the reanalyses reveals that NCEP1 is the best in relative humid-ity mean bias over the panels (Table 5).

Plots of relative humidity over levels of the atmosphere (Fig. 2) reveal that all reanalyses are generally within 20 % of each other throughout the atmosphere with bias rang-ing between −40 % (NCEP2 at 100 hPa over the north-ern panel) and +30 % (ERA-I at 150 hPa over the north-ern panel). NCEP1 and MERRA under-estimate relative humidity at 850 hPa over the northern (also 20CRv2) and southern boundary panels. Reanalyses have largest over-estimations of relative humidity at the tropopause level (i.e., at 200–150 hPa) over all the boundary panels.

Based on Fig. 4, ERA-I, MERRA and 20CRv2 have the largest tropopause error near the equator while NCEP2 has it at high latitudes. 20CRv2 has the largest error at both the equator and high latitudes. Apart from these, ERA-I and MERRA have biases fairly evenly spread over latitude.

4.2 Root mean squared error (RMSE)

4.2.1 Temperature

As from Table 2, all reanalyses generally struggle at higher latitudes (i.e., at the southern boundary panel) with best performances over the northern boundary panel. MERRA has the lowest RMSE’s over the western and eastern boundary panels while NCEP1 and NCEP2 do best for the southern and northern boundary panels respectively. Rank-ing of the reanalyses based on their performances reveals that NCEP1 is generally the best over all four boundary panels for temperature RMSE (Table 5).

Based on RMSE plots over levels of the atmosphere (Fig. 5) the following are revealed.Ta

ble

2 E

rror

sta

tistic

s fo

r th

e in

divi

dual

bou

ndar

y pa

nels

Bol

d va

lues

= b

est

NC

EP1

NC

EP2

ER

A-I

20C

Rv2

ME

RR

A

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Tem

pera

ture

(K

)

S0.

552.

211.

630.

880.

562.

231.

640.

880.

132.

251.

590.

860.

202.

822.

140.

770.

142.

331.

410.

84

N0.

371.

421.

150.

640.

401.

391.

110.

620.

001.

531.

220.

580.

852.

011.

670.

490.

121.

491.

140.

58

W0.

411.

741.

380.

770.

411.

751.

390.

770.

091.

771.

360.

750.

522.

281.

860.

610.

041.

701.

160.

75

E0.

451.

601.

300.

790.

471.

621.

310.

790.

091.

671.

300.

750.

462.

111.

720.

650.

041.

591.

120.

75

Rel

ativ

e hu

mid

ity

(%)

S4.

5021

.116

.30.

488.

2719

.615

.20.

579.

0018

.313

.90.

6210

.321

.916

.40.

503.

9717

.113

.40.

64

N0.

7218

.814

.30.

32-5

.43

24.2

15.4

0.27

12.4

21.0

13.0

0.53

6.43

21.4

14.0

0.38

7.42

23.2

14.1

0.45

W1.

4817

.513

.50.

451.

5419

.213

.70.

458.

9416

.911

.60.

665.

9018

.313

.40.

485.

0417

.110

.90.

63

E3.

1619

.615

.20.

412.

1620

.514

.20.

438.

6517

.512

.00.

628.

7920

.715

.00.

474.

3718

.111

.60.

58

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

• All reanalyses are within 0.5 K of each other through most of the atmosphere, though MERRA is consistently worse across the mid troposphere over the southern boundary panel.

• MERRA also performs badly in the lower troposphere over the northern boundary panel.

• 20CRv2 is consistently worse near the top of the atmos-phere (i.e., between pressure levels of 100 hPa and 50 hPa).

• For the southern boundary panel, reanalyses struggle at the tropopause level (i.e., 150 hPa to 200 hPa).

RMSEs vertical contour plots at the respective panels (Fig. 6 and Supplemental figure S3) reveal the following.

• Reanalyses perform worse at high latitudes.• MERRA performs badly at high latitudes (i.e., over the

southern boundary) and also struggles in the lower trop-

osphere near the equator (i.e., over the northern bound-ary panel).

• Reanalyses have over-estimations just above the tropo-pause level. These errors are more pronounced for 20CRv2 near the equator.

• At higher latitudes (over the southern boundary panel), reanalyses perform badly near the tropopause (near 200–150 hPa).

4.2.2 Relative humidity

Based on Table 2, ERA-I is the best over the eastern and western boundary panels while NCEP1 does best over the northern and eastern panels. The ranks of reanalyses per-formances indicate that ERA-I and NCEP1 are joint best performers compared to the rest (Table 5).

Examining relative humidity RMSE plots over the levels of the atmosphere (Fig. 5), the following are key findings.

Fig. 2 Mean Bias for Temperature (K) and Relative humidity (%) over the levels of the atmosphere for the boundary panels

D. B. Moalafhi et al.

1 3

Fig. 3 Vertical contour plots of Temperature mean bias (K) for the eastern and western boundary panels

Fig. 4 Vertical contour plots of relative humidity mean bias (%) for the eastern and western boundary panels

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

• ERA-I and MERRA consistently do better through mid-troposphere for the western, eastern and southern boundary panels.

• All reanalyses perform similar and are closer to each other for the southern and northern boundary pan-els than for the eastern and western boundary panels through mid-troposphere.

• Generally, reanalyses perform better near the surface (i.e., at 1000 hPa) over the eastern, western and northern boundary panels while for the southern boundary panel, best performances are also near the top of the atmos-phere (i.e., at 100 hPa).

Based on Fig. 7 (also see Supplemental figure S4) on relative humidity RMSE vertical contour plots over latitude (and longitude), the following are the key findings that sup-port findings arrived at from Fig. 5.

• ERA-I and MERRA consistently do better through mid-troposphere for the western, eastern and southern boundary panels which agrees with Fig. 5.

• ERA-I and MERRA also do better through mid-tropo-sphere for the northern panel at high longitudes.

• ERA-I and MERRA have the largest over-estimations of relative humidity near the tropopause. Best perfor-mances are generally at the lower levels of the tropo-sphere which reinforces findings through Fig. 5.

4.3 Temporal correlation

4.3.1 Temperature

Reanalyses are better correlated over time with AIRS at high latitudes (Table 2). NCEP1 is the best when consider-ing all boundary panels (Table 5).

Fig. 5 Root mean squared error (RMSE) for temperature (K) and relative humidity (%) over the atmosphere for the boundary panels

D. B. Moalafhi et al.

1 3

Fig. 6 Vertical contour plots of temperature (K), root mean squared error (RMSE) for the eastern and western boundary panels

Fig. 7 Vertical contour plots of relative humidity (%), root mean squared error (RMSE) for the eastern and western boundary panels

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

Based on Fig. 8 of temporal correlations over the tropo-sphere, 20CRv2 is consistently the least correlated across all the boundary panels. It is noticed that 20CRv2 has the lowest time correlations particularly at 50 hPa across all the boundary panels. Except for 20CRv2, the rest of the reanaly-ses perform very similar to each other throughout the tropo-sphere with highest temporal correlations notably at 100 hPa.

Vertical contour plots reveal that reanalyses tempo-ral correlations with AIRS are highest at high latitudes (Fig. 9). NCEP1 and NCEP2 have notable high temporal correlations at the top of the tropopause over the eastern, western and northern boundary panels (Fig. 9, see also Supplemental figure S10). Worst performances are gener-ally noticeable around 600 hPa to 700 hPa except for the southern boundary panel.

4.3.2 Relative humidity

ERA-I has the highest temporal correlations over the north-ern, western and eastern boundary panels while MERRA

is the best for the southern boundary panel (Table 2). For all the reanalyses across all the panels, worst performances are generally at the lower layers of the troposphere (Fig. 8). Based on the vertical contour plots, there is an inclination of reanalyses to be better correlated over time with AIRS at high latitudes (Fig. 10, also see Supplemental figure S11).

Since 20CRv2 uses Ensemble Kalman filter system data assimilation (Table 1), which only assimilates surface-pres-sure observations, this could partly explain its particularly poor performance especially around tropopause level. This reanalysis also has relatively coarser vertical and horizon-tal resolutions. MERRA and ERA-I have the finest verti-cal and horizontal resolutions compared to the rest of the reanalyses considered in this study (Table 1). This could contribute to their relatively better performances due to fewer interpolation errors. Although MERRA uses 3D-var-iational data assimilation while ERA-I uses the 4D-varia-tional approach, the two reanalyses performances are simi-lar, which might also be as a result of their common use of Optimum Interpolation Sea Surface Temperature (OISST)

Fig. 8 Temporal correlation coefficient for temperature and relative humidity (%) over the levels of the atmosphere for the boundary panels

D. B. Moalafhi et al.

1 3

Fig. 9 Vertical contour plots of Temperature temporal correlation coefficient for the eastern and western boundary panels

Fig. 10 Vertical contour plots of relative humidity temporal correla-tion coefficient for the eastern and western boundary panels

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

forcing (Reynolds et al. 2007). This SST forcing is contrib-uting to the superiority of the MERRA and ERA-I products for the East, West and South boundaries.

4.3.3 Performance in extreme wet and dry years

Further, evaluation was made for the driest and wettest years of the study period. The driest and wettest years were ascertained based on Global Precipitation Climatol-ogy Project (GPCP) rainfall data and these are identified as 2005 and 2012 respectively. For the driest year (i.e., 2005), and considering both temperature and relative humidity, MERRA gives the best performance across all the panels as shown in Table 3 and subsequent ranks (Table 6). The rank score for MERRA is 42 followed by ERA-I with 53. Even for the wettest year (i.e., 2012), MERRA has the best rank score of 41 followed by ERA-I with rank score of 57 across all the panels as shown in Table 4 (also see Table 7). ERA-I’s over-estimation of relative humidity is more evident for the driest year. It is also revealed that ERA-I under-estimates temperature across all boundary panels for both years.

Since MAE results are similar to those of RMSE, the former was left out in computing the rankings of reanalyses to avoid giving more weight to one type of error statistics. Considering all the ranks for mean bias, RMSE and tempo-ral correlation for both temperature and relative humidity over the study period (2003–2012) and for the driest (2005) and wettest (2012) years, MERRA has the overall best rank of 140 followed by ERA-I with rank score of 170 (Tables 5, 6, 7) over all the boundary panels. Although ERA-I is the second best reanalysis for the domain, it over-estimates rel-ative humidity and thus using it to provide boundary condi-tions for the domain could likely result in too much precipi-tation over the region. The performance of ERA-I in mean bias are consistent with the findings of Zhang et al. (2013) and Kalognomou et al. (2013) in which ERA-I manifests a very strong moisture convergence over the eastern equato-rial Atlantic and consequently resulting in strong precipita-tion over the southern African region. Ratna et al. (2013) also revealed that forcing WRF with ERA-I over southern Africa results in positive rainfall biases over the region irrespective of convective parameterization schemes used. The cold-bias of ERA-I over all the boundary panels is also echoed by a study over the Amazon basin (Betts et al. 2009) in which ERA-I is cold-biased compared to in situ station data. This cold bias of ERA-I were also found by Brands et al. (2013) in evaluating reanalyses uncertainty over tropical African regions. The summary of discussions on the reanalyses performances considering the whole study period (2003–2012), the driest (2005) and wettest (20,120 years is also given in Table 8.

Tabl

e 3

Err

or s

tatis

tics

over

the

indi

vidu

al b

ound

ary

pane

ls f

or th

e dr

iest

yea

r (2

005)

of

the

stud

y pe

riod

Bol

d va

lues

= b

est

NC

EP1

NC

EP2

ER

A-I

20C

Rv2

ME

RR

A

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Tem

pera

ture

(K

)

S0.

551.

721.

390.

940.

551.

741.

420.

94−

0.27

1.33

1.03

0.95

0.02

2.49

1.97

0.80

0.14

1.30

1.00

0.95

N0.

251.

341.

100.

680.

251.

331.

070.

65−

0.45

1.09

0.88

0.75

0.92

2.14

1.83

0.57

0.13

1.28

1.03

0.70

W0.

411.

511.

250.

820.

391.

401.

260.

81−

0.33

1.10

0.89

0.88

0.59

2.18

1.82

0.68

0.07

1.13

0.90

0.86

E0.

431.

521.

170.

850.

431.

511.

180.

85−

0.35

1.09

0.89

0.89

0.52

2.09

1.75

0.70

0.06

1.12

0.90

0.86

Rel

ativ

e hu

mid

ity

(%)

S4.

5719

.215

.10.

568.

0217

.914

.20.

649.

7317

.714

.00.

6911

.121

.517

.30.

544.

4015

.111

.30.

74

N−

0.72

18.3

13.9

0.37

−6.

5424

.319

.10.

3212

.52

20.7

16.7

0.56

4.54

21.0

16.9

0.40

6.92

22.3

17.2

0.51

W1.

4716

.712

.90.

490.

9118

.614

.50.

499.

0716

.312

.90.

705.

8917

.814

.40.

495.

0415

.011

.30.

71

E3.

3918

.314

.10.

461.

9219

.414

.90.

488.

7916

.613

.10.

678.

2019

.816

.00.

504.

2315

.711

.80.

67

D. B. Moalafhi et al.

1 3

Reanalyses performances are ranked vis a vis climate mean and climate variability to suggest which are most suitable for downscaling in each category (Table 9). The ranking reveals that MERRA reanalysis is the most suit-able (rank score 49) when consideration is based on climate mean. Alternatively, NCEP1 is the next most suitable (rank score 58). If choice of reanalysis is sought for climate vari-ability downscaling undertakings, ERA-I (rank score 41) is most suited followed by MERRA (rank score 44). It is also noted that MERRA and ERA-I are the most suitable through RMSE which combines both magnitude and variability.

To test the reanalyses performances over different time scales, RMSEs were computed at weekly and monthly time scales for the whole study period (2003–2012), the driest year (2005) and the wettest year (2012) for the respective boundary panels. The results (not presented here) in terms of RMSEs magnitudes and ranking of the reanalysis are consistent with those from the daily results presented, suggesting that persis-tence and related biases are of the same order across all the reanalyses datasets considered. Even results of evaluations for time correlation at monthly time scale (for the study period and the rainfall extreme years) and annual time scale (for the study period) are also consistent with those at daily time scale.

4.3.4 Wind field comparison between reanalyses

Due to the lack of reliable observations of wind that span the atmosphere in space and time, it is not possible to perform the same evaluation at the RCM lateral bounda-ries that has been performed for temperature and humid-ity. Here the differences in wind fields between the rea-nalyses at the RCM lateral boundaries are investigated to provide an indication of the variance in this aspect of the RCM LBCs. We arbitrarily choose ERA-I as the reference dataset, and calculate the bias, RMSE, MAE and temporal correlation for each remaining reanalyses against this refer-ence, over each of the boundary panels for the whole study period (2003–2012), driest (2005) and wettest (2012) years (see supplemental tables S1 to S3, and figures S12 to S15).

The statistics over the entire period examined suggests that the MERRA is most similar to ERA-I with the low-est MAEs, highest temporal correlations and amongst the lowest biases. It is also notable that NCEP1 consistently produces the lowest RMSEs. While the meridional wind biases are generally quite small for MERRA, the zonal wind biases are consistently higher. These differences are, however, very small. Also, while MERRA frequently has the highest temporal correlation, this correlation can be as low as 0.64 for meridional wind on the northern boundary. Thus, while the wind fields of MERRA are more similar to ERA-I than any of the other reanalyses, the temporal sequence of winds can differ between them and hence is only weakly constrained by the assimilated observations.Ta

ble

4 E

rror

sta

tistic

s ov

er th

e in

divi

dual

bou

ndar

y pa

nels

for

the

wet

test

yea

r (2

012)

of

the

stud

y pe

riod

Bol

d va

lues

= b

est

NC

EP1

NC

EP2

ER

A-I

20C

Rv2

ME

RR

A

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Bia

sR

MSE

MA

Er

Tem

pera

ture

(K

)

S0.

501.

761.

420.

930.

551.

791.

450.

93−

0.22

1.43

1.12

0.94

0.20

2.50

1.97

0.81

0.13

1.34

1.04

0.95

N0.

271.

321.

080.

630.

261.

291.

050.

61−

0.36

1.34

1.10

0.59

0.90

2.20

1.86

0.45

0.12

1.31

1.05

0.63

W0.

341.

521.

250.

800.

381.

521.

260.

79−

0.30

1.32

1.07

0.81

0.63

2.25

1.86

0.59

−0.

011.

130.

910.

84

E0.

431.

471.

240.

790.

471.

501.

270.

79−

0.27

1.28

1.05

0.80

0.52

2.10

1.75

0.60

0.02

1.15

0.93

0.82

Rel

ativ

e hu

mid

ity

(%)

S4.

3120

.515

.90.

528.

0918

.714

.90.

616.

3215

.211

.40.

729.

9521

.016

.70.

563.

8216

.111

.70.

71

N−

0.22

18.4

13.9

0.36

−5.

9025

.418

.30.

289.

8020

.314

.70.

596.

3824

.118

.10.

426.

7824

.317

.40.

51

W1.

2416

.412

.70.

471.

4119

.514

.60.

466.

4014

.710

.70.

735.

3419

.114

.50.

474.

8715

.911

.30.

70

E3.

0019

.214

.80.

441.

7221

.115

.70.

436.

1216

.011

.70.

678.

5221

.016

.30.

504.

1117

.512

.50.

64

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

Examination of the wind field statistics associated with extreme dry and wet years reveals MERRA to be much more similar to ERA-I than any other reanalysis. In fact MERRA wind fields are more similar to ERA-I wind fields during these extreme years than during the record as a whole. 20CRv2 wind fields are least similar to ERA-I during these extreme years with the lowest temporal cor-relations, highest RMSEs and MAEs, and biases that reach over 1 m/s for zonal wind on the southern boundary.

While these results do not evaluate the performance of any particular reanalysis, they do show that ERA-I and MERRA have the most similar wind fields at the lat-eral boundary locations. Hence the wind field differences

between these reanalyses are expected to produce small differences inside the RCM domain compared to the other reanalyses. That is, the relative ranking of ERA-I and MERRA based on temperature and relative humidity at the lateral boundaries as an indicator of preferred RCM LBCs is more robust than that of 20CRv2 whose large wind field differences may dominate the RCMs internal climate.

4.3.5 Application over a different domain

The robustness of the approach was evaluated through doing the same assessment with a different (smaller) sec-ondary domain (Fig. 1) stretching between longitude 9.5°

Table 5 Reanalyses ranking based on error statistics of Table 2 (whole period: 2003–2012)

NCEP1 NCEP2 ERA-I 20CRv2 MERRA

Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r

Temperature (K)

S 4 1 1 5 2 1 1 3 3 3 5 5 2 4 4

N 3 2 1 4 1 2 1 4 3 5 5 5 2 3 3

W 3 2 1 3 3 1 1 4 3 5 5 5 2 1 3

E 3 2 1 5 3 1 1 4 3 4 5 5 2 1 3

Total 13 7 4 17 9 5 4 15 12 17 20 20 8 9 13

Relative humidity (%)

S 2 4 5 3 3 3 4 2 2 5 5 4 1 1 1

N 1 1 4 2 5 5 5 2 1 3 3 3 4 4 2

W 1 3 4 2 5 4 5 1 1 4 4 3 3 2 2

E 2 3 5 1 4 4 4 1 1 5 5 3 3 2 2

Total 6 11 18 8 17 16 18 6 5 17 17 13 11 9 7

19 18 22 25 26 21 22 21 17 34 37 33 19 18 20

59 72 60 104 57

Table 6 Reanalyses ranking based on error statistics of Table 3 (driest year: 2005)

NCEP1 NCEP2 ERA-I 20CRv2 MERRA

Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r

Temperature (K)

S 4 3 3 4 4 3 3 2 1 1 5 5 2 1 1

N 2 4 3 2 3 4 4 1 1 5 5 5 1 2 2

W 4 4 3 3 3 4 2 1 1 5 5 5 1 2 2

E 3 4 3 3 3 3 2 1 1 5 5 5 1 2 2

Total 13 15 12 12 13 14 11 5 4 16 20 20 5 7 7

Relative humidity (%)

S 2 4 4 3 3 3 4 2 2 5 5 5 1 1 1

N 1 1 4 3 5 5 5 2 1 2 3 3 4 4 2

W 2 3 3 1 5 3 5 2 2 4 4 3 3 1 1

E 2 3 5 1 4 4 5 2 1 4 5 3 3 1 1

Total 7 11 16 8 17 15 19 8 6 15 17 14 11 7 5

20 26 28 20 30 29 30 13 10 31 37 34 16 14 12

74 79 53 102 42

D. B. Moalafhi et al.

1 3

Table 7 Reanalyses ranking based on error statistics of Table 4 (wettest year: 2012)

NCEP1 NCEP2 ERA-I 20CRv2 MERRA

Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r

Temperature (K)

S 4 3 3 5 4 3 3 2 2 2 5 5 1 1 1

N 3 3 1 2 1 3 4 4 4 5 5 5 1 2 1

W 3 3 3 4 3 4 2 2 2 5 5 5 1 1 1

E 3 3 3 3 4 3 2 2 2 5 5 5 1 1 1

Total 13 12 10 14 12 13 11 10 10 17 20 20 4 5 4

Relative humidity (%)

S 2 4 5 4 3 3 3 1 1 5 5 4 1 2 2

N 1 1 4 2 5 5 5 2 1 4 3 3 3 4 2

W 1 3 3 2 5 5 5 1 1 4 4 3 3 2 2

E 2 3 3 1 5 4 4 1 1 5 4 5 3 2 2

Total 6 11 15 9 18 17 17 5 4 18 16 15 10 10 8

19 23 25 23 30 30 28 15 14 35 36 35 14 15 12

67 83 57 106 41

Table 8 Summary reanalyses performances for temperature and relative humidity

Whole period: 2003–2012 Driest year: 2005 Wettest year: 2012

Temperature (K) Best: NCEP1 Best: MERRA Best: MERRA

Second best: MERRA Second best: ERA-I Second best: ERA-I

Worst: 20CRv2 Worst: 20CRv2 Worst: 20CRv2

Relative humidity (%) Best: MERRA Best: MERRA Best: ERA-I

Second best: ERA-I Second best: ERA-I Second best: MERRA

Worst: 20CRv2 Worst: 20CRv2 Worst: 20CRv2

Overall Best: MERRA Best: MERRA Best: MERRA

Second best: NCEP1 Second best: ERA-I Second best: ERA-I

Worst: 20CRv2 Worst: 20CRv2 Worst: 20CRv2

Conclusion MERRA best followed by ERA-I

Table 9 Summary of reanalyses performances under fitness of magnitude (mean bias) and variability (temporal correlation)

Rankings are shown in brackets

NCEP1 NCEP2 ERA-I 20CRv2 MERRA

Magnitude Variability Magnitude Variability Magnitude Variability Magnitude Variability Magnitude Variability

Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r

Whole period (2003–2012)

19 18 22 25 26 21 22 21 17 34 37 33 19 18 20

Driest year (2005)

20 26 28 20 30 29 30 13 10 31 37 34 16 14 12

Wettest year (2012)

19 23 25 23 30 30 28 15 14 35 36 35 14 15 12

Totals

58(2) 67 (3) 75 (3) 68 (3) 86 (4) 80 (4) 80 (4) 49 (2) 41 (1) 100 (5) 110 (5) 102 (5) 49 (1) 47 (1) 44 (2)

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

to 38.5° and latitude −17.5° to −37.5°. Evaluations based on mean bias, RMSE, MAE and temporal correlation were made over four (4) 4° boundaries of the domain.

The outcomes from this assessment are similar to those for the main domain. In terms of overall rankings, MERRA comes out to be the best and followed by ERA-I. In fact all the five (5) reanalyses still rank the same as with the main domain. The results of such are summarized in supplemen-tal tables S4 to S9 with contour plots also shown through supplemental figures S16 through S31.

5 Conclusions

Four dimensional (4-D) atmospheric fields (temperature and relative humidity) used in regional climate model LBCs from five global reanalyses that have been used widely and that have given mixed performances both globally and regionally, have been evaluated against AIRS observations over southern Africa. This analysis is aimed at identifying the reanalysis that can provide the most accurate LBCs for climate simulations over the region. This study is the first of its kind to undertake 4-D evaluation of relevant LBC var-iables from global reanalyses at the intended boundaries of a domain of interest. By considering the vertical extent of the boundaries, from the surface through the troposphere, we ensure that the choice of reanalysis to provide LBCs to the RCM is as objective as possible and subsequently the most accurate LBCs possible are used for climate simula-tions. AIRS products at a grid resolution of 1° × 1° and at selected pressure levels were sub-setted to four (4) panels. These panels are located at the boundaries of the domain and extend by 4° inwards. Reanalyses were subsequently similarly sub-setted and interpolated as per the AIRS’s cho-sen pressure levels and also temporally resampled to match the AIRS satellite data.

The study reveals that reanalyses generally over-esti-mate mean temperature over all the boundary panels, espe-cially from 700 hPa level towards the surface. Although 20CRv2’s over-estimation of mean temperature at the top is a clear deviation from the rest, all reanalyses perform similarly to each other throughout most of the troposphere. Considering all the boundary panels together, the best performance for temperature mean bias goes to ERA-I. Mean relative humidity is also over-estimated by reanaly-ses, except for NCEP2 over the northern boundary. ERA-I generally performs worst with largest positive bias over all boundary panels. Notable mean relative humidity over-estimations by all reanalyses are at the tropopause level over all boundary panels. When all the boundary panels are considered, NCEP1 has the best performance in relative humidity mean bias.

Analysis through RMSE reveals that reanalyses perfor-mances are worse at higher latitudes as revealed through the southern boundary panel especially at the tropopause level. 20CRv2 performs consistently worse near the top of the atmosphere over all boundary panels. When look-ing at all the boundary panels, NCEP1 has the best per-formance in temperature RMSE. ERA-I and NCEP1 have joint best performances in relative humidity RMSE over all the boundary panels. Generally reanalyses perform best for relative humidity near the surface.

Generally reanalyses are more correlated in time with AIRS at high latitudes and NCEP1 is the best in this regard when considering all the boundary panels. 20CRv2 has consistently the lowest temporal correlations particularly around the tropopause. Inclinations towards high tempo-ral correlations for high latitudes are also evident for rela-tive humidity though not that well pronounced. ERA-I and MERRA are best performers in the mid-troposphere.

Based on mean bias, RMSE, and temporal correlation evaluations for both temperature and relative humidity over all the boundary panels, MERRA is the best followed by NCEP1 and ERA-I. Further analysis using the driest (2005) and wettest (2012) years reveals that MERRA still ranks the best followed by ERA-I for both temperature and rela-tive humidity mean bias, RMSE and temporal correlation over all the boundary panels. It is noted that ERA-I’s over-estimation of relative humidity is more evident for the dri-est year and it also under-estimates temperature across all the boundary panels for both the driest and wettest years. Based on all the results of the whole study period (2003–2012), the driest year (2005) and the wettest year (2012), MERRA and ERA-I appear to be the best overall reanal-yses for provision of temperature and humidity LBCs for the study domain from the African equator down to the southern-most coast of South Africa including Madagascar. ERA-I has the best performance in terms of capturing the time variability of the atmospheric temperature and humid-ity fields as shown by the RMSE. When considering tem-poral correlation, ERA-I also has an edge over MERRA. However, MERRA has the best overall performance being slightly worse than ERA-I at capturing the variabil-ity but considerably better in terms of the biases present. When considering choice of reanalysis for climate mean, MERRA is the most suited over the domain. Alternatively if the choice is for climate temporal variability studies, ERA-I appears to be the most suitable. It is revealed that RMSEs (at weekly and monthly time scales) and temporal correlation coefficient (at monthly and annual time scales) are consistent with those from the daily results presented. The robustness of the approach was demonstrated over a different domain in the same region in which the reanalyses rankings came out the same with MERRA the overall best

D. B. Moalafhi et al.

1 3

followed by ERA-I. It should be noted that RCM LBCs also include wind fields that, due to the lack of a suitable observation dataset, have not been evaluated here.

It is interesting to contrast the findings here with the study of Zhang et al. (2013) that assessed a number of rea-nalyses datasets over a similar study region, but using the seasonal cycle in precipitation observations as the basis for comparison. The outcome from that study concluded that the two reanalysis products to be recommended for further use over this region were the CFSR and the 20CRv2. While the present study does not assess CFSR, it finds that the 20CRv2 is less preferable for use in downscaling as com-pared to MERRA and ERA-I when 4-D fields of tempera-ture and humidity at the lateral boundaries are considered. While the 4-D wind fields are not evaluated here, those of 20CRv2 are shown to differ substantially from ERA-I (com-pared to MERRA) and these circulation differences may dominate the climate differences within the RCM domain. This is an illustration of the differences that can result when reanalyses datasets are assessed using ground observations versus 4-D atmospheric data, and further emphasises the importance of the type of assessment reported here if the end aim is to use the reanalysis dataset to form LBCs for regional climate modelling over the study domain.

This 4-D high resolution treatment of domain bounda-ries in evaluating the global reanalyses fields for provision of LBCs for regional climate modelling is the first of its kind and adds another dimension for improved objectivity and accuracy in the evaluation of reanalyses for provision of LBCs for regional models. This approach can be applied to any region, and reanalyses can be evaluated at the loca-tions of the intended RCM boundaries. Although the results of this study are domain-specific, the robustness of this approach has also been demonstrated over another domain in the same region. However, the approach presented here addresses a necessary criterion for improving RCM simula-tions, which is the specification of accurate LBCs.

Acknowledgments We acknowledge funding support from the University of Botswana and the Australian Research Council (FT110100576 and FT100100197) that helped carry out this research.

References

Aumann HH, Chahine MT, Gautier C, Goldberg M, Kalnay E, McMillin L, Revercomb H, Rosenkranz PW, Smith WL, Staelin DH, Strow L, Susskind J (2003) AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products and processing systems. IEEE Trans Geosci Remote Sens 41:253–264

Bengtsson L et al (2007) The need for a dynamical climate reanalysis. Bull Am Meteorol Soc 88(4):495–501

Betts AK, Koehler M, Zhang Y (2009) Comparison of river basin hydrometeorology in ERA-Interim and ERA-40 reanalyses with observations. J Geophys Res. doi:10.1029/2008JD010761

Bosilovich MG, Chen J, Robertson FR, Adler RF (2008) Evaluation of global precipitation in reanalyses. J Appl Meteorol Climatol 47(9):2279–2299

Brands S, Herrera S, Fernández J, Gutiérrez JM (2013) How well do CMIP5 earth system models simulate present climate conditions in Europe and Africa? Clim Dyn 41(3–4):803–817. doi:10.1007/s00382-013-1742-8

Bromwich DH, Fogt RL (2004) Strong trends in the skill of the ERA-40 and NCEP-NCAR reanalyses in the high and midlati-tudes of the southern hemisphere, 1958–2001. Am Meteorol Soc 17:4603–4619

Bromwich DH, Forgt RL, Hodges KI, Walsh JE (2007) A tropo-spheric assessment of the ERA-40, NCEP, and JRA-25 global reanalyses in the polar regions. J Geophys Res. doi:10.1029/2006JD007859

Decker M, Brunke MA, Wang Z, Sakaguchi K, Zeng X, Bosilovich MG (2012) Evaluation of the reanalysis products from GSFC, NCEP, and ECMWF using flux tower observations. J Clim 25(6):1916–1944

Divakarla MG, Barnet CD, Goldberg MD, McMillin LM, Maddy E, Wolf W, Zhou L, Liu X (2006) Validation of Atmospheric Infra-red Sounder temperature and water vapor retrievals with matched radiosonde measurements and forecasts. J Geophys Res. doi:10.1029/2005JD006116

Druyan LM, Fulakeza M (2013) Downscaling reanalysis over conti-nental Africa with a regional model: nCEP versus ERA Interim forcing. Clim Res 56(3):181–196. doi:10.3354/cr01152

Fetzer EJ (2006) Preface to special section: validation of atmospheric infrared sounder observations. J Geophys Res. doi:10.1029/2005JD007020

Kalognomou EA et al (2013) A diagnostic evaluation of precipitation in CORDEX models over southern Africa. J Clim 26:9477–9506. doi:10.1175/JCLI-D-12-00703.1

Mehrotra R, Sharma A (2015) Correcting for systematic biases in multiple raw GCM variables across a range of timescales. J Hydrol 520:214–223. doi:10.1016/j.jhydrol.2014.11.037

Pitman AJ, Perkins SE (2009) Global and regional comparison of daily 2-m and 1000-hPa maximum and minimum temperatures in three global reanalyses. Am Meteorol Soc 22:4667–4681

Qian T, Dai A, Trenberth KE, Oleson KW (2006) Simulation of global land surface conditions from 1948 to 2004. Part I: forcing data and evaluations. J Hydrometeorol 7:953–975

Ratna SB, Ratnam JV, Behera SK, deW Rautenbach CD, Ndarana T, Takahashi K, Yamagata T (2013) Performance assessment of three convective parameterization schemes in WRF for down-scaling summer rainfall over South Africa. Clim Dyn 42(11–12):2931–2953. doi:10.1007/s00382-013-1918-2

Reynolds RW, Smith TM, Liu C, Chelton DB, Casey KS, Schlax MG (2007) Daily high-resolution-blended analyses for sea sur-face temperature. J Clim 20(22):5473–5496. doi:10.1175/2007jcli1824.1

Rocheta E, Evans JP, Sharma A (2014) Assessing atmos-pheric bias correction for dynamical consistency using potential vorticity. Environ Res Lett 9(12):124010. doi:10.1088/1748-9326/9/12/124010

Susskind J, Barnet C, Blaisdell J, Iredell L, Keita F, Kouvaris L, Mol-nar G, Moustafa C (2006) Accuracy of geophysical parameters derived from Atmospheric Infrared Sounder/Advanced Micro-wave Sounding Unit as a function of fractional cloud cover. J Geophys Res. doi:10.1029/2005jd006272

Susskind J, Blaisdell JM, Iredell L (2014) Improved methodology for surface and atmospheric soundings, error estimates, and qual-ity control procedures: the atmospheric infrared sounder sci-ence team version-6 retrieval algorithm. J Appl Remote Sens 8(1):084994. doi:10.1117/1.jrs.8.084994

Evaluating global reanalysis datasets for provision of boundary conditions in regional…

1 3

Tobin DC, Revercomb HE, Knuteson RO, Lesht BM, Strow LL, Han-non SE, Feltz WF, Moy LA, Fetzer EJ, Cress TS (2006) Atmos-pheric Radiation Measurement site atmospheric state best esti-mates for Atmospheric Infrared Sounder temperature and water vapor retrieval validation. J Geophys Res. doi:10.1029/2005JD006103

Zhang Q, Körnich H, Holmgren K (2013) How well do reanaly-ses represent the southern African precipitation? Clim Dyn 40(3–4):951–962