Seasonal Prediction Based on EOF Analyses of GCM Ensemble Means

Ruping Mo and David. M. Straus

Center for Ocean-Land-Atmosphere Studies Institute of Global Environment and Society, Inc.

4041 Powder Mill Road, Suite 302 Calverton, MD 20705

e-mail: straus@cola.iges.org

COLA Technical Report 75 November 1999

Abstract

In this study, we construct a regression prediction scheme for seasonal-averaged anomalies based on the EOF analysis of GCM ensemble means. The predictors of the regression equation are the principal components associated with the EOFs of the ensemble mean. This scheme, when applied to a 17-year ensemble generated by the GCM of the Center for Ocean-Land-Atmosphere Studies (COLA), achieves significant seasonal-averaged forecast skill over the tropics and some extratropical regions during Northern Hemisphere winter. This skill, either deterministic or probabilistic, is generally comparable to the corresponding skill achieved from direct application of the ensemble. In some regions, such as the extratropical western Pacific and East Asia, the EOF-based scheme is capable of realizing some implicit skill associated with the GCM ensemble, leading to significant improvement of seasonal forecast skill. Hidden by the climate noise, this implicit skill cannot be utilized by direct application of the ensembles. It emerges after the noise is reduced or totally removed from the data through the EOF and regression analyses. Generally speaking, implicit skill may be associated with different principal components in different locations. Therefore it is possible to carefully select the principal components as the predictors in the regression equation to achieve the best regional forecast. Our application also suggests that the COLA GCM can identify an ENSO-independent signal over the extratropical North Pacific, which may be related to the local air-sea interaction.

Applications of the EOF-based regression prediction scheme to ensembles

generated by other GCMs also lead to some interesting results. The scheme is generally applicable when the dynamical signal and the climate noise contained in the ensemble means are clearly separable through the EOF analysis. The possibility of generalizing this single-model scheme to a multi-model version is discussed at the end of the paper.

1. Introduction

The predictability of seasonal climate variations has been studied a great deal

using the ensemble forecast approach, in which several deterministic forecasts are created

using a state-of-the-art general circulation model (GCM). This approach can be used to

forecast probabilities of a number of probable future states instead of a definite single

state. Some recent studies (e.g., Shukla 1998; Shukla et al. 2000a, 2000b; Krishnamurti et

al. 1999) have revealed that ensemble forecasting with current GCMs leads to significant

skill in predicting seasonal anomalies in the tropics and some extratropical regions,

especially over the Pacific/North American (PNA) sector. Although the existence of such

seasonal predictability has long been suggested by statistical and theoretical studies on the

world-wide impact of the El Niño/Southern Oscillation (ENSO) phenomenon (Bjerknes

1966, 1969; Hoskins and Karoly 1981; Horel and Wallace 1981; Wallace and Gutzler

1981; Sardeshmukh and Hoskins 1988; Chen and van den Dool 1997; Mo et al. 1998),

successfully predicting the ENSO-related large-scale anomalies with a GCM on seasonal

timescales is indeed a major breakthrough in climate research (Barnston et al. 1999;

Mason et al. 1999).

The simplest method of utilizing an ensemble forecast is to issue a forecast using

the ensemble mean. This should be considered as a pseudo-deterministic forecast, given

that the single value of the ensemble mean is always subject to a random error associated

with the ensemble. It has been argued that a probabilistic approach is more appropriate

than a definite approach for the seasonal forecast for the chaotic atmosphere. A probability

forecast tries to predict the possible future states and estimate their chances to occur. In

the context of ensemble forecasting, this usually involves dividing a meteorological

1

2

variable into several categories and calculating the percentage of ensemble members in

each category. For example, one may consider a cold event in a winter as the occurrence

of seasonal average surface temperature being one standard deviation less than normal.

The probability forecast at each gridpoint for this happening in the GCM is simply given

by the fraction of ensemble integrations meeting the event criterion. Alternatively, one

may consider the ensemble mean as the best estimate of the future state, and estimate its

chance to occur based on the dispersion of the ensemble (Déqué et al. 1994; Mo and

Straus 1999).

Pan and van den Dool (1998) pointed out that the prediction of the future state

using exclusively the corresponding ensemble could be too confident because the

ensemble spread is too small to catch future reality. They suggested that a combination of

the current forecast and some historical forecasts could help to increase the reliability of

forecast. This idea has inspired Mo and Straus (1999) to derive a forecast scheme based on

the regression of the historical observations on the corresponding ensemble means of

historical forecasts. Application of this scheme to the ensemble hindcasts generated by the

GCM of the Center for Ocean-Land-Atmosphere Studies (COLA) showed some

noticeable improvements of the probability forecast over the regions where the boundary-

forcing signal is relatively weak as compared to the internal variability (noise) of the

model atmosphere.

The regression forecast discussed in Mo and Straus (1999) was constructed based

on the observation and forecast data at each gridpoint. While treating each gridpoint

independently is simple and straightforward, it may not be the best way to exploit the

GCM-generated ensembles. It has been recognized that large-scale characteristic patterns

associated with both boundary forcing and with internal variability (unrelated to boundary

conditions) are simulated well in some GCMs (Straus and Shukla 2000). Thus it is

reasonable to expect our current GCM to be capable of capturing some of the information

of the dominant patterns in the atmosphere. Under such circumstances, a regression

scheme treating each gridpoint independently obviously can not optimally benefit from the

captured teleconnection information. To overcome this drawback and improve the

performance of the regression forecast, we try in this study to construct predictors based

on the empirical orthogonal functions (EOFs) of the ensemble mean. These EOFs are

essentially a different representation of the GCM covariance matrix derived from the

historical ensemble mean. If they also represent the best knowledge of the observed

teleconnection patterns, then a regression forecast based on their fluctuations could lead to

significant improvement of forecast skill in regions where the EOFs explain significant

variance.

We will apply this EOF-based regression scheme to ensembles generated by a

number of different GCMs (see Section 2 for detailed description). Note that our scheme

represents a statistical-dynamical approach to the seasonal forecast. It can be considered as

a generalization of the EOF-based prediction algorithm recently developed by Kim and

North (1998, 1999) from a purely statistical approach. The details of our scheme are

outlined in Section 3. The scheme is evaluated using the COLA ensemble in Sections 4

and 5. It is applied to ensembles generated by other GCMs in Section 6. Further discussion

and conclusions are presented in Section 7. Some verification techniques are described in

the appendix.

3

2. Data description

The principal ensemble used to evaluate the forecast schemes developed in this

study were generated from the COLA GCM, which has a moderately high resolution

(rhomboidal 40) with 18 discrete levels in the vertical (Kinter et al. 1997). As part of a

project of seasonal dynamical prediction (Shukla et al. 2000a), nine integrations were

carried out each year, starting from different observed initial conditions of the atmosphere,

12 hours apart, centered in mid-December for the winters 1981/82-1997/98. The observed

sea surface temperature (SST) and sea-ice were prescribed in the boundary conditions.

Therefore, any skill derived from these integrations should be considered as only potential

forecast skill of the COLA GCM in response to SST anomalies. The observational data

used to verify the model performance are obtained from the NCEP/NCAR reanalysis (see

Kalnay et al. 1996).

To test the general applicability of the EOF-based regression forecast scheme, we

shall also make use of four other DSP ensembles generated using the GCMs of the

National Centers for Environmental Prediction (NCEP, 5 members, 1982/83-1995/96), the

National Center for Atmospheric Research (NCAR, 10 members, 1981/82-1996/97), the

Geophysical Fluid Dynamics Laboratory (GFDL, 10 members, 1979/80-1994/95), and the

Data Assimilation Office (DAO, and its model is denoted as GSFC, 9 members, 1980/81-

1995/96). The initial conditions for all these integrations were derived from observations

in mid-December. We also use an ensemble generated from the GCM of the European

Centre for Medium-Range Weather Forecasts (ECMWF, 9 members, 1979/80-1992/93),

in which the initial conditions were obtained from observations in mid-November. Further

details of these models and their ensembles are available in Shukla et al. (2000a).

4

We shall apply the forecast scheme only to the 500-hPa geopotential height

interpolated to a global grid and averaged over the period between January 1

and March 31 of each year. Therefore, the numbers of years covered by the data are 17,

14, 16, 17, 15, and 14 for the ensembles of the COLA, NCEP, NCAR, GFDL, GSFC, and

ECMWF, respectively.

5.25.2

3. The EOF analysis and the EOF-based regression scheme

EOF analysis, also known as principal component (PC) analysis in other academic

communities, has become a widely used tool in analyzing both observations and numerical

simulation of the global circulation (Bretherton et al. 1992). To illustrate this

methodology, let denote the anomaly field of a variable measured at M

locations for N times

) ,( nm txQ

), M,1(m ),,1( Nn . The eigenvectors of its covariance

matrix, say form an orthogonal set of basis vectors or EOFs. When the data are

projected onto each EOF, the resulting time series are the PCs given as

),( mj xE

(1) ),,1( ,)(),()(1

M

mmjnmnj LjxEtxQtT

where L is the rank (i.e., the number of positive eigenvalues) of the covariance matrix. It

cannot be greater than neither M nor N, i.e.,

).,(min NML

jT shows how the jth EOF evolves in time. It is conventional to order the EOFs and the

associated PCs so that the first EOF explains the largest fraction of the variance of the

data, and the last EOF explains the smallest.

We can reconstruct the data from and , i.e., jE jT

5

(2) .)()() ,(1

L

jnjmjnm tTxEtxQ

Figure 1 shows the correlation patterns associated with the first six PCs derived

from the observed 500-hPa heights of the 17 NH winters (1982-1998). In order to ensure

equal areas get equal weights in the areal integration (sum over grid points in Eq.(1)), we

have multiplied the height at each gridpoint by the square root of the cosine of latitude of

that gridpoint before doing the EOF analysis. The fraction of total variance explained by

each mode is indicated in the title of the corresponding map. The variance distribution of

each mode is displayed using shaded contouring. We see that the first EOF, which

explains about 31% of the total variance, appears to be a combination of the well-known

PNA pattern and the North Atlantic oscillation (NAO) discussed in Wallace and Gutzler

(1981). The fact that they are summarily represented by the leading EOF suggests that

they may not always be totally independent of each other. The second EOF is also a

combination of two patterns. One is located in the PNA sector, but is distinctly different

from the PNA pattern seen in the first EOF (see Straus and Shukla 2000). The other

pattern is similar to the eastern Atlantic pattern defined in Wallace and Gutzler (1981).

The third, fourth, and fifth EOFs represent respectively the western Atlantic pattern, the

Eurasian pattern, and the western Pacific pattern defined in Wallace and Gutzler (1981).

The sixth EOF, which explains only 4.9% of the total variance, represents a stationary

planetary wave in middle latitudes.

The correlation patterns associated with the first six PCs of the ensemble means of

the COLA GCM for the same period are shown in Fig.2. Here the ENSO-related pattern

over the PNA sector emerges as the leading mode, which explains nearly 45% of the total

variance. This pattern is much stronger and more well-defined than its observed

6

counterpart in Fig.1b, because part of the internal variability is averaged out in taking the

ensemble mean. The pattern representing the seasonal mean internal variability over the

PNA sector is combined with the eastern Atlantic pattern in the second EOF. The third

EOF of the ensemble mean represents a combination of the western Atlantic pattern and

the western Pacific pattern (again, see Wallace and Gutzler 1981). The fourth, fifth, and

sixth EOFs could be considered as other planetary waves simulated by the COLA GCM.

We now proceed to outline the procedure of the EOF-based regression prediction.

Let and denote the anomalies of the ensemble mean and observation, respectively.

The jth EOF and PC of the ensemble mean derived from N historical records are denoted

as and . Then can be regressed on as follows,

eQ

ejE

oQ

ejT oQ e

jT

J

j

enmn

ejmjnm

o LJtxtTxbtxQ1

(3) )( ),()()(),(

where are the regression coefficients, jb is a normally distributed variable, and is

the rank of the covariance matrix of the ensemble mean. Because the PCs of the ensemble

mean are uncorrelated with each other, the least squares estimates of are easily

obtained as

eL

jb

(4) )()(),()(ˆ1

2

1

N

nn

ej

N

nn

ejnm

omj tTtTtxQxb

When the ensemble mean of the future state, , is available, the

predicted future state is given by

),( 1Nme txQ

(5) )( )()(ˆ),(ˆ1

11e

J

jN

ejmjNm

o LJtTxbtxQ

where

7

8

(6) .)(),()(1

11

M

mm

ejNm

eN

ej xEtxQtT

Alternatively, we can also predict the PCs of the observations based on their regressions

on the PCs of the ensemble means, and then reconstruct the predicted anomalies as

(7) )(ˆ)(),(ˆ1 1

11

oL

i

J

jN

ejijm

oiNm

o tTxEtxQ

where are the corresponding regression coefficients, is the ith EOF derived from N

historical observations, and is the rank of the of the covariance matrix of the

observations. Apart from some small computational errors, Eqs.(5) and (7) should lead to

the same result, and we confirmed that they do.

ij oiE

oL

In the following sections, the regression forecast will evaluated using a cross-

validation approach. For a data set of size 1N , the cross-validation is carried out by

withholding an observation each time, constructing the regression model with the

remaining developmental data sets of size N, and then using the model to predict the

withheld observation. The resulting 1N regression forecasts are then verified by the

corresponding observations. 1N

4. Deterministic forecasts based on the COLA ensembles

In this study, a deterministic forecast is defined as a single-value prediction

carrying no probability information. For example, the ensemble mean at a gridpoint of

several GCM runs, regardless of the ensemble spread, can be used to issue a deterministic

forecast for that gridpoint. An application of the EOF-based regression method described

in the previous section can also lead to a single-value prediction for each gridpoint. The

skill of these two forecast schemes is evaluated in this section within a deterministic

framework using the mean-square-error skill score (MSS), and will be further evaluated

within a probabilistic framework in next section using the ranked probability skill score

(RPSS). The definitions of these skill scores are given in the appendix. Suffice it to say

that both of them have a range of to 1, with positive value indicating a forecast better

than a zero skill climatological forecast based on the distribution.

For convenience, the scheme that uses directly the ensemble mean of each

gridpoint in a winter to issue the forecast for that gridpoint in the same winter will be

referred to as a purely-dynamical forecast (PDF), and the scheme with application of the

EOF-based regression will be referred to as a statistical-dynamical forecast (SDF). Figure

3 shows the MSS of the 17-year COLA ensembles for the PDF and the SDFs with various

PCs of the ensemble mean used as predictors. We see that almost all positive scores are

statistically significant at the 95% confidence level. The simple PDF (Fig.3a) achieves

remarkably high skill over the tropics and some extratropical regions. In particular, the

impressive scores over the PNA sector indicate that the COLA GCM is well capable of

capturing the ENSO response (Shukla 1998). This skill will be referred to as the explicit

skill of the GCM, since it exists in the original GCM ensembles. On the other hand,

unacceptably large negative scores of PDF can be seen over Alaska, Central Europe, and

near the Himalayas. These poor scores, as implied by Eq.(A1), result mainly from the

large variances of the ensemble mean relative to their observational counterparts in the

regions with weak or negative correlation coefficients. In other words, the model

atmosphere must be much noisier than the real atmosphere in these regions. In the SDF

with the first PC of the ensemble mean as the only predictor (Fig.3b), however, no large

9

negative scores are apparent. This is a consequence of the regression analysis, which is

insensitive to the error in variance. Theoretically, there should be no negative score at all

in the SDF. If all predictors in the regression model are pure noise, the corresponding

regression coefficients should all be zero, and the SDF reduces to the climatological

forecast. Under such circumstances, the MSS is zero. Therefore those negative scores in

Fig.3b-d should be understood as random errors of the correlation between the noise and

observation. Note that the magnitudes of these errors increase as the number of predictor

increases. The reason for this problem is that the addition of a variable to a regression

equation almost always increases (and never decreases) the variance of a predicted

response. Addition of a useless predictor then will only contribute to increase the error of

prediction. Note that the large errors seen in Fig.3d are not directly relevant to the original

model noise seen in Fig.3a.

The first PC of the ensemble mean in fact represents the major ENSO response

identified by the COLA model (Straus and Shukla 2000). The skill scores achieved by the

SDF with this leading PC as the only predictor are statistically significant over the tropics

and the extratropical PNA sector (Fig.3b). As compared with the straightforward PDF

(Fig.3a), this simple SDF is less skillful over the tropical and extratropical Pacific, equally

skillful over Eastern Canada and around the Gulf of Mexico, and more skillful over the

equatorial Indian Ocean, the eastern equatorial Atlantic and part of the extratropical North

Atlantic. The disadvantage of this SDF over the Pacific region indicates either that the first

PC of the ensemble mean is incapable of representing all ENSO effects, or that there is an

ENSO-independent signal over this region. Such a signal could be related to the strong

coupling of the extratropical atmosphere with the SST anomalies over the North Pacific

10

(e.g., Namias 1969; Zhang et al. 1996; Mo et al. 1998). On the other hand, the advantage

of the SDF over the PDF in some regions suggests that the GCM ensembles contain some

implicit skill that cannot be utilized by direct application of the ensembles. This skill is

hidden by the climate noise, and emerges only after the noise is reduced or totally

removed from the data.

Fig.3c shows that including the second PC into the regression model has little

effect on the forecast skill over the PNA sector. However, the contribution of this mode is

very significant over the extratropical western North Pacific, East Asia, the Arctic, and

West Europe. In particular, the significant positive scores over Japan and the surrounding

area are in sharp contrast to those negative scores either in Fig.3a or Fig.3b. This

improvement implies that, while inclusion of the ENSO-independent PNA pattern seen in

Fig.2b apparently does not improve the forecast skill, the eastern Atlantic pattern in the

second GCM EOF (Fig.2b) is useful in some regions. The COLA GCM has no explicit

skill over Japan, as seen in Fig.3a. However, the EOF analysis identifies implicit skill over

this region. In the EOF analysis, the correct fluctuation signals of various patterns are

identified from the ensemble mean, and the regression analysis then combines these

signals with the spatial structure of the observed patterns.

The loss of skill by the SDF over the tropical and extratropical eastern Pacific in

Fig.3b,c is reversed when the first six PCs are taken into account (Fig.3d). It can be shown

that in this regard the sixth PC makes the most significant contribution. However,

including up to six PCs in the regression equation introduces a large amount of error in

other regions. This kind of regression error is also responsible for the degradation of

forecast skill over East Canada. This implies that some globally insignificant modes are

11

locally important. They could be taken into account, but only at the expense of some other

regions. In practice, we could select different PCs as predictors for different regions to get

the best regional forecast.

5. Probabilistic forecasts based on the COLA ensembles

Because of the chaotic nature of the atmosphere, a seasonal forecast should be

looked upon as probabilistic rather than deterministic. For a single-model ensemble, a

simple probability forecast can be constructed by counting the ensemble members

associated with some pre-defined events. Here we consider a three-category forecast, in

which the standardized anomaly of the seasonal-mean 500-hPa height is classified as

above normal (larger than 1), normal (between -1 and 1), and below normal (less than -1).

The occurrence of each event in a specified year at a gridpoint is given by the fraction of

ensemble members of the same year within the corresponding category. Following Mo

and Straus (1999), we shall refer to this scheme as counting probability forecast.

In order to take the advantage of the EOF and regression analyses shown in the

preceding section, we may also construct a probability forecast based on the prediction of

the EOF-based regression model. As before, the predictors are the principal components

of the ensemble mean. We then assume that the predicted value of a variable is normally

distributed, with the variance of the distribution estimated from the mean-square error of

the regression model (Mo and Straus 1999). With the notation defined in Section 3, the

estimated variance is given by

(8) )()(ˆ),(1

),(ˆVar1

2

11

N

nn

ej

J

jmjnm

oNm

o tTxbtxQJN

txQ

12

13

where is the degrees of freedom of the variance, reduced from N in the presence

of J regression coefficients (Montgomery and Peck 1982). This scheme will be

called regressive probability forecast. It is different, however, from the gridpoint-based

regressive probability forecast discussed in Mo and Straus (1999). In this section, we shall

only consider the regression model based on the PCs of the ensemble mean discussed in

Section 4.

)( JN

Jbb ˆ,,1

Both the counting and regressive probability forecasts are evaluated using the

RPSS (defined in the appendix) for the above-mentioned three-category forecast (Fig.4).

In order to ensure statistical stability, the RPSS for a gridpoint is computed using gridpoint

data collected over the surrounding region within 600 km, with each forecast at each

gridpoint over the region considered as a separate, independent forecast. As shown in

Fig.4a, the counting probability forecast is significantly high over the tropics and the

extratropical PNA sector. Again we see that the basic response to the ENSO signal is

successfully captured by the simplest regression scheme (Fig.4b). With the first PC of the

ensemble mean as the only predictor, the regression scheme cannot fully recover the

significant skill of the counting scheme over the tropical and extratropical Pacific. But it is

more skillful over the tropical Atlantic and the tropical Indian Ocean. Including the second

PC into the regression equation leads to significant skill over the extratropical western

Pacific and East Asia (Fig.4c), where the skill scores of the counting probability forecast

are negative. The forecast skill over Canada is also slightly improved by considering the

second PC. As the first six PCs are used as predictors, the skill of the regression scheme

increases noticeably over the tropical Pacific, the western equatorial Atlantic, and North

Africa. Again, we see the degradation of skill over East Canada in Fig.4d due to the fact

that too many useless predictors are included in the regression equation. These results are,

in general, consistent with those derived from the deterministic framework in the

preceding section.

6. Application to ensembles of othe GCMs

In this section, the applicability of the EOF-based regression forecast is tested by

replacing the COLA ensemble with the ensembles from each of the other five GCMs

mentioned in Section 2. The result of each model is evaluated using the MSS. Note that,

since different models cover different periods, the MSS from one model is not strictly

comparable to the MSS from another model.

Figure 5 shows the results derived from the NCEP ensemble and can be compared

to Figure 3. As in the COLA model, the first EOF of the NCEP ensemble mean represents

the typical ENSO signal (not shown). Therefore when the first PC of the ensemble mean is

used as the only predictor in the SDF, the ENSO response is successfully predicted

(Fig.5b). This simple SDF appears to be less skillful over the tropics than the PDF

(Fig.5a), but it is more skillful over the extratropical North Pacific and Central Canada.

Including the second PC has very little effect on the forecast skill. However, when the first

seven PCs are included, the skill scores over Europe and the Far East are significantly

improved, while the skill over Canada is notably degraded. These features are very similar

to those seen in the COLA ensemble.

The results derived from the NCAR ensemble are shown in Fig.6. Unlike the

COLA and NCEP GCMs, the ENSO signal in the NCAR model is represented by the

second, instead of the first, PC of the ensemble mean. This is confirmed in Fig.7, which

14

shows the EOFs of the NCAR GCM (and can be compared to Figures 1 and 2). The first

EOF of this model (Fig.7a) is the typical PNA pattern (similar to Wallace and Gutzler

1981) that represents mainly the internal variability of the model atmosphere. Therefore it

is not surprising that no significant skill can be achieved over the extratropical eastern

Pacific and North America by the SDF with the first PC as its only predictor (Fig.6b).

When the second PC is included, the ENSO response is evident (Fig.6c). When the first

six PCs are included, significant skill can be seen over the Far East. Note that this

significant skill also occurs in the corresponding PDF (Fig.6a). Therefore it can be

considered an explicit skill of the NCAR GCM. In the COLA and NCEP GCMs, however,

the skill over the Far East is not explicit.

Results from the GFDL and GSFC ensembles are presented in Figures 8 and 9. We

see that in neither case is there an obvious advantage of the SDF over the PDF. In

particular, the ENSO response is too weak when the first PC is used as the only predictor.

The reason for this problem is that the first EOF of the ensemble mean is a combination of

the ENSO response and the PNA pattern of the internal variability (not shown). In other

words, the EOF analyses fail to separate the ENSO signal from the model noise. Under

such circumstances, the skill of the SDF could be seriously undermined. Nevertheless,

when the first six PCs are considered, we can still see some impressive performances of

the SDF over Canada and the North Pacific as well (Fig.9d); the significant pattern almost

looks like the ENSO-related pattern, but shifted westward.

Figure 10 shows the result of the ECMWF ensemble. In this model, the SDF with

the first PC as the only predictor (Fig.10b) is more skillful over the eastern equatorial

Pacific and Canada than the PDF (Fig.10a). The explicit skill over the Far East achieved

15

by the model (Fig.10a) is recovered in the SDF that uses the first six PCs of the ensemble

mean as predictors (Fig.10d). Significant implicit skills over Canada are also evident in

Fig.10d.

In summary, the EOF-based regression forecast is generally applicable when the

dynamical signal and model noise contained in the ensemble means are clearly separable

through the EOF analysis. A successful application of this scheme can lead to significant

improvement of the forecast skill over certain regions.

7. Discussions and conclusions

Successful prediction of seasonal anomalies in the atmosphere depends on our

understanding of the dynamics of large-scale, low-frequency teleconnection patterns. It is

generally believed that some important teleconnection patterns are forced by the SST

anomalies, which in turn arise from coherent air-sea interaction. Dynamical seasonal

predictions are carried out on the premise that the atmospheric responses to the low-

frequency boundary fluctuations are predictable using a state-of-the-art GCM. However,

there is also evidence that substantial low-frequency variability can also arise as a result of

internal nonlinear atmospheric dynamics. To a forecaster, this internal variability

represents the climate noise and is basically unpredictable. If this noise is too strong

compared to the boundary-forcing signal, it will seriously undermine the predictability of

seasonal anomalies. In this study we show that, although there is no guarantee, the internal

variability contained in many GCM ensembles can be separated from the boundary-

forcing signal through application of the EOF analysis, and at least part of its impact on

the seasonal prediction can be removed through further application of regression analysis.

16

When the EOF-based regression forecast scheme is applied to the ensemble of

seasonal integrations of the COLA GCM, significant skill is seen over the tropics and the

extratropical PNA sector. This skill is considered as an explicit skill of the GCM, because

it is also achievable from direct application of the original ensemble mean. The major

advantage of using the EOF-based regression scheme is that it can realize some implicit

skill of the GCM ensembles, leading to significant improvement of the forecast in some

regions, especially over the extratropical western Pacific and East Asia for the COLA

GCM. This implicit skill may be associated with different principal components in

different locations. Therefore it is possible to carefully select the principal components as

the predictors in the regression model to achieve the best regional forecast.

Application of the EOF-based regression forecast may also improve our

understanding of certain atmospheric anomalies and their GCM simulations. We have

shown that the predictability in the PNA sector derived from the COLA GCM can be

partitioned into an ENSO-related component and an ENSO-independent component. The

ENSO-related component is the atmospheric response to the SST anomalies over the

tropical Pacific. This component, usually associated with the first EOF of the ensemble

mean, contributes to significant skill of the GCM over the tropical Pacific, extratropical

North Pacific, Central and East Canada, and the Gulf of Mexico. The ENSO-independent

component, which is likely related to the ENSO-independent air-sea interaction in the

extratropical North Pacific, has a noticeable effect on the forecast skill over the

extratropical North Pacific and the tropical Pacific as well.

The performance of the EOF-based regression forecast depends on the "quality" of

the ensemble-mean EOFs. We tested the scheme using six ensembles generated from

17

different GCMs and found that it is generally applicable and usually leads to some local

improvement of the seasonal prediction. The scheme could fail, however, to offer any

useful information if the EOF analysis cannot lead to a clear separation between the

boundary-forcing signal and the internal noise in the ensemble. In particular,

distinguishing between the SST-forced variability and internal variability in the

extratropical North Pacific (Straus and Shukla 2000) is essential for a satisfactory

application. However, there is no guarantee that such a separation will always happen for

any ensemble through a regular EOF analysis. It might be possible to obtain a better

separation by performing a rotated EOF analysis (e.g., Mo et al. 1998). Either way, we

suggest that the EOF-based regression forecast scheme should be used to complement,

rather than complete with, other commonly used schemes, such as the purely-dynamical

forecast mentioned in this study.

Finally, we mention the possibility of generalizing the single-model EOF-based

regression forecast to a multi-model version. Some recent studies (e.g, Krishnamurti et al.

1999; Palmer et al. 2000) have shown certain improvement of the ensemble forecast

obtained by blending forecast skills of different GCMs. In principle, a multi-model

regression forecast can be constructed by using some or all principal components derived

from the ensemble of each model to build a super-multiple regression equation. In

practice, however, two potential problems may render this approach useless. One is the

overfitting problem. The final regression equation may contain too many predictors, which

will lead to large variance of the predicted response. Another potential problem is called

the multicollinearity, which occurs when some of the predictors are highly intercorrelated.

In the presence of multicollinearity, the least-squares estimates of the regression

18

19

coefficients are unstable and have large variances. The hazard associated with these

problems could be prevented if the data records for all participating GCMs are long

enough. We can also manually chose only the useful principal components of each model

to be the predictors of the multi-model regression equation.

Acknowledgements

The authors wish to thank J. Shukla, A. Schlosser, B. Kirtman and Y. Fan for their

helpful suggestions, L. Marx and D.A. Paolino for assistance in the collection of data, and

B. Doty and C. Steinmetz for technical support.

Appendix: Some skill scores for forecast verifications

Verification of an ensemble forecast can be assessed within either a deterministic

or a probabilistic framework. For a deterministic forecast, the correlation coefficient

between the forecasts and the observations, r say, may be used as a skill score to evaluate

the forecast scheme relative to a climatological forecast. Here the climatological forecast

is simply a forecast of the long-term average of the variable, and then has no correlation

with the observed variable. Therefore, a negative r implies that the climate mean is a

better predictor than the forecast value of the model, or the converse if r is positive.

Although the correlation coefficient can be used to identify some potential

information contained in the forecast, it is not considered as a good measure of the actual

forecast skill due to its insensitivity to the error magnitude. Murphy (1988) introduced a

more accurate measure called the mean-square-error skill score (MSS), which, when

applied directly to the anomaly fields of forecast and observation, can be written as (see

Murphy 1988; Livezey 1995)

(A1) 2MSS

o

f

o

f

S

Sr

S

S

where r is, as before, the correlation coefficient between the forecasts and the

observations, and are the standard deviations of the forecasts and the observations,

respectively. The MSS can be considered as the mean-square error of the forecast scaled

by the mean-square error of the climatological forecast. Eq.(A1) shows that a perfect

forecast ( ) corresponds to For

fS

MSS

oS

1 of SSr and 1 . 10 r , the possible

maximum value of MSS is 2r , which can be achieved when . The forecast is

worse than the climatological forecast when

of rSS

0MSS , i.e., . Note that even for orS2fS

20

21

a forecast with perfect correlation (i.e., 1r ), the MSS can still be negative when

. of SS 2

When a forecast is assessed within a probabilistic framework, the mean squared

error between the predicted and observed probabilities can be used as a measure of

forecast skill. Such a measure was introduced by Brier (1950) for a simple two-category

probability forecast. The so-called Brier score was further generalized by Epstein (1969)

into a ranked probability score (RPS) for general multiple-category forecasts. More

specifically, the RPS measures the mean squared distance between the cumulative

probabilities of the forecast and the observation, and then is capable of taking into account

the fact that an error in predicting a category close to the observed one is less important

than an error in predicting a category far away from the observed one (for detailed

definition, see Mo and Straus 1999). Scaling the RPS of the forecast by the RPS of the

climatological forecast leads to a ranked probability skill score (RPSS), i.e.,

(A2) gy)(climatolo RPS

RPS (forecast)1RPSS

In this study, we define the climatological probability of an event at a gridpoint for

a winter season as the frequency of occurrence of the event calculated from the

observations of all other winter seasons at the same gridpoint, and a climatological

forecast of the event as the forecast in which the climatological probability is predicted.

According to the above definition, a positive RPSS corresponds to a forecast better

than the climatological forecast. It could be possible, however, that such a positive value is

not significantly different from zero, and could be achieved simply by chance. In this

study, this statistical significance problem is addressed using a Monte Carlo approach. To

outline the method, we label a multi-year ensemble as "Data I" and the corresponding

observations as "Data II". The original RPSS is calculated from Eq.(A2), with Data I

verified by Data II in chronological order. We then create a randomized data by replacing

each field of Data I with a field randomly chosen from the other years (note that this is

different from shuffling Data I in the time domain). The corresponding RPSS is computed

again from Eq.(A2), with Data I replaced by the randomized data. The same procedure is

repeated 1000 times, each time storing the score values. The original score is considered

statistically significant at the 95% confidence level if it is not exceeded by more than 50

values of the corresponding scores obtained using the randomized data. The same method

is also applied to test the significance of MSS.

22

References

Barnston, A.G., A. Leetmaa, V.E. Kousky, R.E. Livezey, E. O'Lenic, H. van den Dool, A.J.

Wagner, and D.A. Unger, 1999: NCEP forecasts of the El Niño of 1997—98 and its U.S.

impacts. Bull. Amer. Meteor. Soc., 80, 1829–1852.

Bjerknes, J., 1966: A possible response of the atmospheric Hadley circulation to equatorial

anomalies of ocean temperature. Tellus, 18, 820-829.

Bjerknes, J., 1969: Atmospheric teleconnections from the equatorial Pacific. Mon. Wea. Rev.,

97, 163-172.

Bretherton, C.S., C. Smith, and J.M. Wallace, 1992: An intercomparison of methods for

finding coupled patterns in climate data. J. Climate, 5, 541-560.

Brier, G.W., 1950: Verification of forecasts expressed in terms of probabilities. Mon. Wea.

Rev., 78, 1-3.

Chen, W.Y. and H.M. van den Dool, 1997: Asymmetric impact of tropical SST anomalies on

atmospheric internal variability over the North Pacific. J. Atmos. Sci., 54, 725-740.

Déqué, M, J.F. Royer, and R. Stroe, 1994: Formulation of gaussian probability forecasts based

on model extended-range integrations. Tellus, 46A, 52-65.

Epstein, E.S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl.

Meteor., 8, 985-987.

Horel, J.D. and J.W. Wallace, 1981: Planetary-scale atmospheric phenomena associated with

the interannual variability of sea surface temperature in the equatorial Pacific. Mon. Wea.

Rev., 109, 813-829.

Hoskins, B.J. and D.J. Karoly, 1981: The steady linear response of a spherical atmosphere to

thermal and orographic forcing. J. Atmos. Sci., 38, 1179-1196.

Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-year reanalysis project. Bull. Amer.

Meteor. Soc., 77, 437-471.

23

24

Kim, K.-Y. and G.R. North, 1998: On EOF-based linear prediction algorithm: Theory. J.

Climate, 11, 3046-3056.

Kim, K.-Y. and G.R. North, 1999: EOF-based linear prediction algorithm: Examples. J.

Climate, 12, 2076-2092.

Kinter III, J.L., D.G. DeWitt, P.A. Dirmeyer, M.J. Fennessy, B.P. Kirtman, L. Marx, K.E.

Schneider, J. Shukla, and D.M. Straus, 1997: The COLA atmosphere-biosphere general

circulation model. Volume 1: formulation. COLA Tech. Report, 51, 46pp.

Krishnamurti, T.N., C.M. Kishtawal, T.E. LaRow, D.R. Bachiochi, Z. Zhang, C.E. Williford,

S. Gadgil, and S. Surendran, 1999: Improved weather and seasonal climate forecasts from

multimodel superensemble. Science, 285, 1548-1550.

Livezey, R.E., 1995: The evaluation of forecasts. Analysis of Climate Variability, H. von

Storch and A. Navarra, Eds., Springer, 177-196.

Mason, S.J., L. Goddard, N.E. Graham, E. Yulaeva, L. Sun, and P.A. Arkin, 1999: The IRI

seasonal climate prediction system and the 1997/98 El Niño event. Bull. Amer. Meteor.

Soc., 80, 1853–1874.

Mo, R., J. Fyfe, and J. Derome, 1998: Phase-locked and asymmetric correlations of the

wintertime atmospheric patterns with the ENSO. Atmosphere-Ocean, 36, 213-239.

Mo, R. and D.M. Straus, 1999: Probability forecasts of seasonal average anomalies based on

GCM ensemble means. Technical Report 74, Center for Ocean-Land-Atmosphere Studies,

25 pp.

Montgomery, D.C. and E.A. Peck, 1982: Introduction to Linear Regression Analysis. John

Wiley & Sons, New York, pp504.

Murphy, A.H., 1988: Skill scores based on the mean square error and their relationships to the

correlation coefficient. Mon. Wea. Rev., 116, 2417-2424.

Namias, J., 1969: Seasonal interactions between the North Pacific Ocean and the atmosphere

during the 1960s. Mon. Wea. Rev., 97, 173-192.

Palmer, T.N., C. Brankovic, and D.S. Richardson, 2000: A probability and decision-model

analysis of PROVOST seasonal multi-model ensemble integrations. Quart. J. Roy. Meteor.

Soc. (submitted).

Pan, J. and H. van den Dool, 1998: Extended-range probability forecasts based on dynamical

model output. Weather Forecasting, 13, 983-996.

Sardeshmukh, P.D. and B.J. Hoskins, 1988: The generation of global rotational flow by steady

idealized tropical divergence. J. Atmos. Sci., 45, 1228-1251.

Shukla, J., 1998: Predictability in the midst of chaos: A scientific basis for climate forecasting.

Science, 282, 728-731.

Shukla, J., J. Anderson, D. Baumhefner, C. Brankovic, Y. Chang, E. Kalnay, L. Marx, T.

Palmer, D. Paolino, J. Ploshay, S. Schubert, D.M. Straus, M. Suarez, and J. Tribbia, 2000a:

Dynamical seasonal prediction. Bull. Amer. Meteor. Soc. (submitted).

Shukla, J., D.A. Paolino, D.M. Straus, D. DeWitt, M. Fennessy, J.L. Kinter, L. Marx, and R.

Mo, 2000b: Dynamical seasonal predictions with the COLA atmospheric model. Quart. J.

Roy. Meteor. Soc. (accepted for publication).

Straus, D.M. and J. Shukla, 2000: Distinguishing between the SST-forced variability and

internal variability in mid-latitudes: Analysis of observations and GCM simulations. Quart.

J. Roy. Meteor. Soc. (accepted for publication).

Wallace, J.M. and D.S. Gutzler, 1981: Teleconnections in the geopotential height field during

the Northern Hemisphere winter. Mon. Wea. Rev., 109, 784-812.

Zhang, Y., J.M. Wallace, and N. Iwasaka, 1996: Is climate variability over the North Pacific a

linear response to ENSO? J. Climate, 9, 1468-1478.

25

26

26