forecasting technology costs via the experience curve — myth or magic?

32
Forecasting technology costs via the experience curve Myth or magic? Stephan Alberth The Judge Business School, University of Cambridge, Trumpington Street, Cambridge CB2 1AG, UK IIASA, Laxenburg, Austria Received 13 July 2007; received in revised form 10 August 2007; accepted 14 September 2007 Abstract To further understand the effectiveness of experience curves to forecast technology costs, a statistical analysis using historical data is carried out. Three hypotheses are tested using available datasets that together shed light on the historical ability of experience curves to forecast technology costs. The results indicate that the Single Factor Experience Curve is a useful forecasting model when errors are viewed in their log format. Practitioners should note that due to the convexity of the log curve a mean overestimation of potential cost reductions can arise as values are converted into monetary units. Time is also tested as an explanatory variable, however forecasts made with endogenous learning based on cumulative capacity as used in traditional experience curves are shown to be vastly superior. Furthermore the effectiveness of increasing weights for more recent data is tested using Weighted Least Squares with exponentially increasing weights. This results in forecasts that are less biased, though have increased spread when compared to Ordinary Least Squares. © 2007 Elsevier Inc. All rights reserved. Keywords: Forecasting; Experience curves; Renewable energy 1. Introduction In the absence of easy to use and reliable models or methods to make cost projections for new technologies, experience curves have been used extensively in the literature. Their role has principally been to provide indications of potentialcost reduction as experience is gained or potentiallearning investments required to reach a situation of break-even or cost-competitivity. The method has also been criticised for a number of its inherent weaknesses that will be discussed in detail later in this paper. Available online at www.sciencedirect.com Technological Forecasting & Social Change 75 (2008) 952 983 E-mail address: [email protected]. 0040-1625/$ - see front matter © 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.techfore.2007.09.003

Upload: stephan-alberth

Post on 12-Sep-2016

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Forecasting technology costs via the experience curve — Myth or magic?

Available online at www.sciencedirect.com

Technological Forecasting & Social Change 75 (2008) 952–983

Forecasting technology costs via the experiencecurve — Myth or magic?

Stephan Alberth

The Judge Business School, University of Cambridge, Trumpington Street, Cambridge CB2 1AG, UKIIASA, Laxenburg, Austria

Received 13 July 2007; received in revised form 10 August 2007; accepted 14 September 2007

Abstract

To further understand the effectiveness of experience curves to forecast technology costs, a statistical analysis usinghistorical data is carried out. Three hypotheses are tested using available datasets that together shed light on the historicalability of experience curves to forecast technology costs. The results indicate that the Single Factor Experience Curve is auseful forecastingmodel when errors are viewed in their log format. Practitioners should note that due to the convexity ofthe log curve a mean overestimation of potential cost reductions can arise as values are converted into monetary units.Time is also tested as an explanatory variable, however forecasts made with endogenous learning based on cumulativecapacity as used in traditional experience curves are shown to be vastly superior. Furthermore the effectiveness ofincreasing weights for more recent data is tested using Weighted Least Squares with exponentially increasing weights.This results in forecasts that are less biased, though have increased spread when compared to Ordinary Least Squares.© 2007 Elsevier Inc. All rights reserved.

Keywords: Forecasting; Experience curves; Renewable energy

1. Introduction

In the absence of easy to use and reliable models or methods to make cost projections for newtechnologies, experience curves have been used extensively in the literature. Their role has principallybeen to provide indications of ‘potential’ cost reduction as experience is gained or ‘potential’ learninginvestments required to reach a situation of break-even or cost-competitivity. The method has also beencriticised for a number of its inherent weaknesses that will be discussed in detail later in this paper.

E-mail address: [email protected].

0040-1625/$ - see front matter © 2007 Elsevier Inc. All rights reserved.doi:10.1016/j.techfore.2007.09.003

Page 2: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 1. Cumulative learning investment requirements with different value niche markets [9, p18].

Table 1Comparison of engineering versus experience curve estimates to actual costs

Bottom-upstudy

Year ofstudy

Yearsstatistics

Year ofprojection

Bottom-up costprojection ($Wp)

Experience curve costprojection ($Wp)

Actual average sellingprice ($/Wp)

J3L86-31 target 1978 1976–1977 1986 1,63 0,86 11.94JBL86-31 Cz 1985 1976–1984 1988 2,17 6.35 9.121BL86-31 Dendritic 1985 1976–1984 1992 1,02 2.80 7,70EPRI 1986 1986 1976–1985 2000 1.50 0.79 5,05MUSIC FM. 1996 1996 1976–1995 2000 1,00 4,07 4.05

Figure from [8, p8].

953S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

The IEA publication ‘Experience Curves for Energy Technology Policy’ [1] presents a broad overview ofthe work covered up to the end of the 1990's and also presents the findings from the 1999 IEAworkshop onthe subject. Their recommendation was that experience effects should be “explicitly considered in exploringscenarios to reduce CO2 emissions and calculating the cost of reaching emissions targets” [1, p114].Furthermore, Alberth and Hope [2] showed that the value associated with improved information regardingthe experience effect may be higher than was previously thought and can also have an important effect on‘optimised’ abatement paths. Having such an important effect on themodelled abatement costs, it is vital thatthe strengths and weaknesses of the underlying experience curves are properly understood, and to this endboth research and available data relevant to climate change abatement technologies has been limited.

Hence, in order to improve our understanding of the usefulness of experience curves, this paper presents astatistical analysis of a range of technologies and tests the extent to which they were able to forecast futurecost levels with limited historical data. We first test the hypothesis that the simple single factor experiencecurve provides forecasts with error distributions that are not statistically different from zero (at a 10%significance level) and that are unskewed.

The second hypothesis tested refers to the question ‘do experience curves as a forecasting method offuture costs improve as more experience is accumulated?’ This is done by way of an empirical analysiscomparing the forecasts made with fewer data points to forecasts that are made later on with access to agreater number of data points.

Page 3: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 2. Flowchart of the statistical model developed in Matlab.

954 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

The third hypothesis compares the relative merits of cumulative capacity, or experience as used in thestandard model, to time as the explanatory variable. Here we are comparing what is essentially anexogenous (solely time dependant) cost curve to one that is endogenous (dependant on cumulativeproduction). This is highly relevant since other research suggest that the experience effect is driven by atime dependant, and not experience dependant, relationship. In the case of wind and solar, for example,Papineau found “that although single experience indices may be highly statistically significant, whetherestimated for individual technologies and countries or combined in panels, most of these estimatesbecome insignificant when a time trend is included as an explanatory variable” [3, p427].

Since the forecastingmodel represents in our case a simple heuristicmodel of cost reductions as a function ofcumulative experience (or time) and not a theoretical model, the results remain outside the scope of formaleconometric estimation. Instead they represent an assessment of the historical effectiveness of the variousmodels tested to forecast future price levels, such that the extent towhich themodel's historic forecasting abilityis analogous to future renewable energy technologies remains an important question for further discussion.

Other papers encountered by the author have statically tested the experience curves by way ofregression analysis R2 levels and the significance level of the coefficients, which is arguably of limiteduse within the dynamic long term problem of climate change management. The principal aims of thisresearch are thus similar to that of McDonald and Schrattenholzer [4, p255] who analysed the variabilityand evaluated the usefulness of experience curves for application in long term energy models. However,where they focused on the variability of the actual long term learning rates of energy technologies, thispaper directly evaluates the ability for the Single Factor Experience Curve (SFEC) to make forecastsabout future costs using historical data.

Page 4: Forecasting technology costs via the experience curve — Myth or magic?

Table 2Technology details and sources included in the study

Technology type Units Initialyear

Finalyear

Datapoints

Forecasteddoublings

Source

Big plants CCGT Electricity Usc(90)/kWh —TWh

1981 1997 15 3.6 [15] Cleason Colpier 2002

NuclearInstallation

US$(90)/W —GW

1975 1993 19 2.0 [4] Kouvaritakis et al. (2000) inMcDonald & Schrattenholzer (2001)

SCGT Installation US$(90)/W —GW

1956 1981 14 8.9 [19] IISA–WEC (1910), p.50

Modules Solar Production $/Wp —MWp

1975 2003 29 9.6 [20] Maycock (2005)

Sony LaserDiode Production

Yen —1000⁎units

1982 1994 13 13.3 [4] Lipman and Sperling (1999) inMcDonald & Schrattenholzer 2001

Ford Model-TShipments

$(58)/unit —Million units

1909 1923 12 7.3 [4] Abernathy and Wayne (1974) inMcDonald & Schrattenholzer 2001

Average DramM Bit Production

$/Mbit — Mbit 1974 1998 25 20.6 [21] Victor & Ausubel (2002)

ContinuousOperation

EthanolProduction

$/GJ—GJ 1980 2004 25 5.3 [21] Goldemberg et al. (2004)

AcrylonitrileProduction

$(66)/unit —units

1959 1972 14 3.0 [11] Lieberman (1984)

Polyethylene-LDProduction

$(66)/unit —units

1958 1972 15 3.5 [11] Lieberman (1984)

Polyethylene-HDProduction

$(66)/unit —units

1958 1972 15 3.9 [11] Lieberman (1984)

Polyester FibersProduction

$(66)/unit —units

1960 1972 13 4.4 [11] Lieberman (1984)

955S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

Methodologically, this paper is similar to that of Everett and Farghal [5–7] in their use of experiencecurves. The emphasis and methods used, however, remain different. For example, their work focussed on anarea that should in theory be far easier to forecast since the projects were generally over a shorter time period,remainedmore or less identical and did not cross regional boundaries. Furthermore they focused on the use ofsmoothing techniques that reduced the importance of the more recent data and looked at the error of totalcosts required to reach the ‘final’ cumulative output. Since in this paper we are dealingwith technologies thatare changing over 10 or 20 years or longer with a greater likelihood of fundamental shifts in the experiencecurve, we have considered weighing data in such a way that recent data has a stronger influence on forecasts.Also, since there is generally no pre-defined limit to the total output of a renewable energy, this paper testsforecasts made for a set number of doublings of cumulative capacity into the future and the costs to reach aspecific price level. Special effort has also beenmade to capture the uncertainty distributions of the forecasts.

By evaluating the effectiveness of the experience curve and measuring the uncertainties surrounding itsforecasts, the paper is able to respond in part to the acknowledged limitations of the experience curve model.Such limitations include comments by Wene [1] highlighting “the risk that expected benefits will notmaterialise,” and of Grübler & Messner [8, p510] who warned of the dangers of “best guessparameterisation”. With a greater understanding of the uncertainties, researchers should be able to“incorporate stochastic experience curve uncertainty” directly into the model [3, p10], potentially reducingthe inherent weaknesses of the experience model. This research project aims to support the inclusion of

Page 5: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 3. Technology price (solid) and Annual Production Growth (dashed) for CCGT energy production.

956 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

stochastic modelling of learning by providing statistical data on the uncertainties associated with experiencecurves used to forecast future technology costs.

The following section reviews experience curves and other forecasting methods in more detail. Section3 presents the statistical models developed in this paper including the assumptions made and the datasources used which is followed by the results of the model and a discussion on the use of experiencecurves for technology cost modelling.

2. Experience curves and other forecasting methods

This paper mainly considers the use of the most commonly used SFEC as described by Eq. (1). In thisequation the cost per unit ‘Ct’ depends on the cumulative number of units produced ‘Xt’, a constant and acoefficient ‘b’ that can be found by regression analysis. The Progress Ratio (PR) defined in Eq. (2) is a widelyused ratio of final to initial costs associated with a doubling of cumulative output. The Learning Rate (LR)represents the proportional cost savings made for a doubling of cumulative output as presented in Eq. (3).

Ct ¼ const � X�b ð1Þ

PR ¼ 2�b ð2Þ

LR ¼ 1� PR ð3Þ

If a technology was to follow a well defined experience curve, as shown in Fig. 1, the shaded areawould represent the cumulative investments needed to reach the break-even point. What is important to

Page 6: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 4. Log–log representation of experience curve fit to Brazilian Ethanol data using OLS, WLS and RLS.

957S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

note here is that only the area that lies above the baseline alternative is considered a learning investment(in the case of renewable electricity the baseline assumption is generally considered to be traditional fossilfuel power stations, hence making a further assumption that such a value for cost can be forecasted).

In situations where niche markets exist (for example solar PVelectricity for remote areas or hand helddevices), the required learning investment is further reduced as shown by the unshaded step-like area ofthe diagram. Unfortunately, even a small error, for example of plus or minus .02 in the progress ratio, canlead to large errors in the final break-even point and hence the total required learning investment. Thislimits the ability of experience curves to guide policy on individual technologies.

A general literature review of experience curves can be found in [8], however, there are a number ofother, often closely related, forecasting methods that we have chosen not to use for this research. Forinstance studies based on multi-factor experience curves that use technical factors to explain changes inthe dependant variable (usually price or cost) have been shown to offer highly informative results, such asin the case of the flying fortress [10], the chemical industry [11] and wind power [12]. Neverthelessmodels based on technical factors suffer a limitation that single indices experience curves do not; they relyon intimate knowledge of the mechanisms leading to cost reductions for each individual technology.Although this makes perfect sense in terms of explaining past cost (or price) trends, it may not be asvaluable when making long term cost forecasts where new challenges may require unforeseenmechanisms that cannot be endogenised into a technical factor model (as also mentioned by Coulomb &Neuhoff [12]). Furthermore, such models would be difficult if not impossible to include in many E3(Energy Environment Economics) models due to their complexity and the unavailability of the requireddata, especially for long term forecasting. Hence, for an implementation into a long term IntegratedAssessment Models (IAM) (and in particular a stochastic control model), the most viable solutions remain

Page 7: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 5. Non-linear1 representation of experience curve fit to Brazilian Ethanol data using OLS, Weighted LS and Robust LS.

1 Non-linear refers to the experience curve itself, though here the set of axes itself is linear.2 The role of R&D investments is not widely agreed upon and their effect has been difficult to quantify. If better understood

this element may also prove to be very important.

958 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

the simple SFEC or perhaps an SFEC in combination with an exogenous (time dependent) factor toaccount for cost reductions not directly related to experience.2

One possible limitationwith regards to the experience curve paradigm is the absence of floor-costs that havebeen shown to appear in technologies that reach maturity. One explanation is that when growth of experiencebegins to decline, ‘forgetting by not doing’ becomes an important factor. On the other hand, most technologiesrelevant to climate change are still far from reaching maturity. One fortuitous result for climate abatingtechnology modelling is that experience curves used in this field may not be as affected (at least during theirmost important phase of cost reductions) by this weakness as compared to experience curves used to describecost reductions in more mature technologies. Hence, for the purposes of this paper, technologies well intomaturity are avoided. Efforts to calculate floor-costs with respect to minimum material costs for specifictechnologies have been carried out, for example byZweibel [13] andNeuhoff [14]. Zweibel looked at long termgoals for the solar market and concluded that costs of 1/3 USD/Wp could be reached, thus making it afinancially viable alternative to fossil fuel electricity despite the existence of the floor-costs calculated.According to Schaeffer, however, “engineering studies have always been far too optimistic in assessing futurecosts”. He notes that although some of the predictions with experience curves are “just as bad”, with a longerhistory of statistics, the match of experience curves based projections with actual realisations can be prettygood” [9, p8]. As can be seen from Table 1 neither method used to predict future costs were very accurate andthis was in part due to an assumed continuous growth rate for the experience curve calculation of the forecast

Page 8: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 6. Predicted (post 1980) learning investments at each period. Please note that the reference line represents observed total learninginvestments.

Fig. 7. Forecast error ratio as a function of (log2) cumulative production.

959S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

year's value (sometimes as high as 50%) that did not materialise. Nevertheless, Schaeffer suggested that theexperience curve projectionswere generallymore accurate than the optimistic engineering predictions found inthe literature. On the other hand, calculations based on engineers perceptions of how a technologywill develop,may not be able to take into account important advancements in the corematerials, technologies ormethods thatengineering assessments are based on. This would then lead to engineering floor-cost forecasts being too high.

Another possible limitation is that improvements in quality, such as has been seen in the automotiveindustry, can offset the expected reductions in cost (examples given in McDonald & Schrattenholzer [4]and Colpier & Cornland [15]). Coulomb & Neuhoff [12] suggest that in the case of wind power, turbine

Page 9: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 8. Technology price (solid) and Annual Production Growth (dashed) for CCGT energy production.

3 Please note that prices are used as a proxy for costs throughout.

960 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

size could also have an important effect on learning rates and that wind turbines have suffered from recentdiseconomies of scale, at least on the production side. Once they converge to an optimal size, one couldexpect “faster cost reductions” since simple cost reductions would then be the main focus. Such examplesof structural change can lead to a dynamically shifting learning rate, something that exponentiallyincreasing weights may help to track by focusing on more recent data.

An important role is also played by technology spillover effects. Here the learning mechanism isassociated not just with learning of a single technology but instead an entire cluster of relatedtechnologies. Learning rates that incorporate spillovers within clusters of technologies have also beencalculated and included in energy technology models [16]. To what extent clustering technologiestogether can improve forecasts within the experience curve paradigm remains unclear due to addeduncertainties that could come with the increased complexity of the model.

3. A model for evaluating experience curve forecasts

Regression analysis by way of recursive estimation and comparison of forecasts to historical data is usedto test 3 hypotheses relating to the use of experience curves for forecasting technology costs.3 These are:

H1. Experience curves can be used as a forecasting model of future technology costs with a zero meanand symmetric error distribution.

Hypothesis 1 is tested by considering the shape of the error distribution both in terms of mean deviationand skewness which are tested against a standardised normal distribution.

Page 10: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 9. Experience curves for CCGT using cost of energy production with constant gas prices versus cumulative energy produced

4 This only came into effect for forecasts made with very few data points.

961S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

.

H2. The ability to forecast technology costs improves as more data points are added.

Hypothesis 2 is tested by comparing the distribution of the forecast errors using only the first half of theforecast dataset for each individual technology to that using the second half. The results are then aggregatedacross all 12 technologies to give an overall appraisal of experience curve uncertainty dynamics.

H3. Experience is a more effective explanatory variable than time for forecasting technology costs.

Hypothesis 3 is first tested by simple comparison of theR-squared results of the various technologies usingtime alone, experience and time together and experience alone as the explanatory variables. Similar to themethod used in H1, the errors of forecasts are then compared for 4 to 12 years into the future for either time orexperience. Here it is assumed that the future cumulative capacity is known at each of these future points intime, as may be expected when the quantity of experience gained is a decision variable of government policy.

The steps of the basic model, as executed in Matlab, can be summarised by the flowchart in Fig. 2.

Step 0: The dataset is initialised to the first 2 data points of the series.Step 1: An experince curve is generated from the available dataset at time ‘t’ by regression. Logarithmicbase 2 of both price and cumulative output is used such that each unit increase represents one doubling.Results are restricted to a zero minimum learning rate.4

Step 2: Using the resulting experience curve, forecasts are made for one to six doublings of cumulativelearning by extending out the experience curve. The error of the forecast is calculated both in logarithmicterms and also as a percentage error in monetary units.

Page 11: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 10. Predicted learning investments at each period. The reference line represents the (post 1981) learning investments to reach1997 price levels.

Fig. 11. Forecast error ratio as a function of (log2) cumulative production.

962 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

Step 3: Each of the generated forecasts are compared to the actual historical data from which forecast errorhistograms are drawn.When comparing the forecasts to the historical data, simple linear interpolation is usedbetween future data points. Error is calculated as a deviation from the forecasted value, such that a positivevalue indicates that the forecast is too low and that expected cost reductions are not made, while negativevalues indicate that the forecast is too high, or that the forecast is over-conservative.Step 4: The next point from the time series is added to the current dataset and the analysis from Step 1 isrepeated.

Page 12: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 12. Technology price (solid) and Annual Production Growth (dashed) for Solar Module production.

Fig. 13. Log–log representation of experience curve fit to solar PV module price data using various methods. Note that solar hasnot yet reached large scale competitivity. The price level used as a baseline of 1$/Wp is arbitrarily chosen. Such a price levewould greatly increase the number of competitive applications if not allow PV to become completely cost competitive.

963S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

l

Page 13: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 14. Predicted (post 1975) learning investments at each period. Please note that the reference line represents total investmentsby 2003.

Fig. 15. Forecast error ratio as a function of (log2) cumulative production.

964 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

In this way n−1 experience curves are generated for each type of regression, however the number offorecasts that can be tested are far fewer since the same dataset is used to test the forecasts made usingonly the first portion of the data. Hence many of the forecasts included in this study are made with only alimited amount of information, i.e. 2 to 10 data points due to the limited number of long datasets. This isparticularly true when considering long term (4, 5 or 6 doublings of experience) forecasts.

Page 14: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 16. Histogram of the log of errors over all technologies.

965S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

Once all of the forecasts have been made and errors calculated over all technologies, the distribution oferrors is then considered. The distribution'smean, standard error and skewness are calculated. To increase thereadability of the skewness results, they are directly tested by way of a z-test that calculates the probabilitythat they conform to the expected skew of results randomly drawn from a normal distribution. Although theerrors can not be shown to come from a normal distribution and in actuality almost surely do not, the methoddoes allow for a comparison of results despite varying distributions, methods and population sizes.

When interpreting the error distributions, results that are as close as possible to the realised value aresought after, however in order to be easily applied within IAMs the model's error distribution would alsoideally be ‘well behaved’. In this sense the model should avoid giving apparent systematic biases of

Page 15: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 17. Histogram of the log of errors for 3 doublings of cumulative experience.

966 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

under-prediction or over-prediction and have error distributions with minimum systematic skewness. Thiswould allow for a straight forward implementation within a larger IAM framework as well as a moreintuitive interpretation of the results. Rather than trying to weigh up the relative importance of thesedifferent properties as measured by the variance, bias and skewness of the forecast error distribution, wepresent all the results for each of the different model specifications tested.

It is important to note that within the context of this paper, ‘future’ refers not to time units (except forthe final model where time is an explanatory variable), but to extra units of cumulative experience (output)gained. Hence a prediction is made for a certain cumulative experience without knowing in which year itwill be reached. Furthermore, for the reasons stated by Junginger et al. [17] and others, price data is used

Page 16: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 18. Histogram of the log of errors for 6 doublings of cumulative experience.

967S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

as a proxy for cost data and only the simplest of the experience curve methods, the SFEC, is used in all butthe final model where time is also tested.

4. Data sources

To increase the accuracy of the results as wide a variety as possible of technologies and processes relevantto large scale renewable energies are used. They fall more or less equally into three categories, namely ‘Bigplants’, ‘Modules’ and ‘ContinuousOperation’ as set out byChristiansson [18]. The data came from a varietyof sources including a number of datasets from previous studies that took place at IIASA [4]. In general, rawannual data (in real monetary units) is usedwithout any conversions, filtering or smoothing. One exception is

Page 17: Forecasting technology costs via the experience curve — Myth or magic?

Table 3Summary table of results of forecasts made with 1 to 6 doublings of cumulative capacity over all 12 technologies tested whereresults are considered in their log formats

Log Format Ordinary Least Squares Weighted Least Squares 20%

Doublings Observations Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

1 139 −0.03 0.40 34 11 −0.04 0.38 25 342 100 −0.04 0.60 49 66 −0.04 0.60 52 743 71 −0.13 0.73 14 54 −0.11 0.73 20 474 50 −0.07 0.77 50 43 −0.05 0.78 66 365 38 −0.03 0.92 82 40 0.00 0.97 98 386 31 −0.07 1.03 71 72 −0.01 1.12 97 55Average −0.064 0.743 50 48 −0.040 0.765 60 47

968 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

theCombinedCycleGas Turbine (CCGT)where data provided byColpier&Cornland [15] has already beenconverted from ‘costs per installed capacity’ to ‘costs per electricity produced’. Where possible the focus ison technologies that have remained in their growth stages in order to avoid problems associated with‘forgetting by not doing’ and where data for forecasting at least 3 doublings of technologies is available, withthe exception of nuclear where less data was available, as shown in Table 2.

The result of having such a selecting criteria is that all of the technologies by their very inclusion aretechnologies that have had at least some degree of success. For this reason and a number of other moregeneral data limitations, this set of 12 technologies (which combined allow for up to 130 individual shortterm forecasts in the case of 1 doubling of cumulative experience) should not be assumed to be representativeof all technologies.

5. Results

As well as the aggregated results presented in the final subsection, 3 individual case studies ofparticular relevance to energy and renewable energy technologies is initially be presented in more detail.Each case study comes from one of the three technology groups as set out by Christiansson [18].

5.1. Continuous operation case study — Brazilian Ethanol

Although Brazilian Ethanol production may not be the most general example of a ‘continuous operation’technology, it does provide a valuable case study for evaluating the effectiveness of the experience curve as amechanism for a technology to reach cost competitiveness. It may also be considered as one of the few largescale renewable energy technologies that have been able to reach cost competitiveness (Fig. 3). For eachtechnology the output graphics use the lightest lines to represent the experience curve made with fewer datapoints and the darkest lines with the largest set of data points. The circular markers represent the year inwhich the experience curve was made. For example, in the case of Ethanol in Fig. 4, it can be seen that theslope of the experience curve has mostly increased as experience has been gained. Four different regressionmethods have been tested, namely Ordinary Least Squares (OLS), Weighted Least Squares (WLS) withexponentially increasingweightings by 10% and 20% per year and finally Robust Least Squares (RLS). Dueto its relative success only the OLS (and in some instances the 20% WLS) results are presented.

Page 18: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 19. Histogram of error ratios over all technologies for 1 doubling of cumulative experience.

969S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

The curve for Brazilian Ethanol shows many of the non-linear characteristics as has been demonstratedin the literature such as the “deviations from log-linearity at the beginning and tail of the curve” [23, p7],however this same effect is not systematically found across all technologies tested here. Despite thedeviations it can be seen when looking at the non-linear graphical representation, where the price is nolonger presented in log format as shown in Fig. 5, that even during this earlier period of apparent coststagnation very significant reductions took place.

Brazilian Ethanol also provides an excellent opportunity to consider the relative effectiveness of variousmethods to determine the learning investment required to reach a price level equivalent to an incumbenttechnology. The $10/GJ price level as indicated by the horizontal bar in Fig. 5 was about the averageBrazilian value according to the Goldemberg data for petrol, the incumbent technology (from an updated

Page 19: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 20. Histogram of error ratios over all technologies for 3 doublings of cumulative experience.

970 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

Goldemberg et al. dataset [22]). The ‘learning investments’ required for the technology to reach break-even iscalculated by integrating the extra costs that lie between the horizontal incumbent technology baseline andthe actual data of the price paid to ethanol producers and forecasts thereof. The shape of the learninginvestments required, as depicted in Fig. 5, is the wedge between the two technology costs. To calculate theentire forecasted learning investment required, historical values are used to calculate the investments madeby each dataset up to the date the forecast is made, and then the difference between the experience curveforecast and the baseline is integrated to determine future learning investments needed to reach break-even.

In the case of Brazilian Ethanol, as shown in Fig. 6, it took a number of years and what amounted to theequivalent of about 70% of the total investment before the experience curve forecast was able to provide a

Page 20: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 21. Histogram of error ratios over all technologies for 6 doublings of cumulative experience.

971S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

response out by less than a factor of 10. The figure shows that, in at least this case, the use of aWLS is ableto help track technologies undergoing a gradual shift in learning rates with predictions of similar accuracyoccurring about 1 year earlier with WLS than with OLS.

Finally, as shown in Fig. 7, by comparing the projections at each period for 1 (light) to 6 (dark)doublings of cumulative capacity, one can see whether or not forecasts for individual technologiesimproves as experience is gained. For Brazilian Ethanol it shows that the error increases as forecasts aremade for a greater number of doublings, from 1 through to 5 doublings. This is not surprising since it isgenerally easier to make projections in the shorter term than in the longer term. What is surprising,however, is that the general trend in the absolute (log2) value of the error for each forecast also increases as

Page 21: Forecasting technology costs via the experience curve — Myth or magic?

Table 4Histogram of error ratios over all technologies for 6 doublings of cumulative experience

Ratio format Ordinary Least Squares Weighted Least Squares 20%

Doublings Observations Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

1 139 0.02 0.28 52 0 0.01 0.27 71 02 100 0.06 0.46 20 0 0.06 0.47 19 03 71 0.03 0.49 61 11 0.04 0.50 47 104 50 0.08 0.53 27 35 0.10 0.54 18 435 38 0.17 0.65 12 40 0.22 0.72 6 266 31 0.20 0.80 17 5 0.30 0.91 8 7Average 0.093 0.536 32 15 0.123 0.567 28 14

Fig. 22. Comparing forecasts made with less and more information.

972 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

experience is gained. One must however keep in mind that errors calculated in this way lead to a veryparticular understanding. This is because as prices come down forecast errors need to come downproportionally in order to maintain constant ratio errors. So although the ratio errors have increased, theabsolute errors may still have decreased significantly.

5.2. Big plants case study — CCGT

The dataset for Combined Cycle Gas Turbines (CCGT) originally came from a reduced list of over 200published contract costs in trade journals for new CCGT plants [15]. This data was then converted fromcost per MWof installed capacity to cost per kWh of produced electricity holding gas prices constant. Themain reason for this conversion is that CCGT cost reductions are often traded off against more expensive

Page 22: Forecasting technology costs via the experience curve — Myth or magic?

Table 5Table comparing the values of low versus high information in terms of the mean and standard deviation of the forecast errors inlog terms using OLS

Log format Low information High information

Doublings Mean Standard deviation Mean Standard deviation

1 −0.06 0.36 −0.01 0.442 −0.11 0.54 0.03 0.673 −0.23 0.54 −0.03 0.824 −0.18 0.70 −0.01 0.855 −0.20 0.90 0.11 0.906 −0.17 0.99 0.12 0.98Average −0.159 0.674 0.036 0.777

Table 6Comparison of the R-squared values for OLS regressions with time only, experience and time together and experience only as theexplanatory variable

Comparing the R-squared value of Time only with Time and Experience as well as Experience only for full dataset

Technology type Technology Time Time and Experience Experience

Big plants CCGT Electricity 0.76 0.93 0.80Nuclear Capacity 0.96 0.96 0.92SCGT Capacity 0.71 0.96 0.94

Modules Solar Capacity 0.81 0.98 0.93Laser Diodes 0.93 0.98 0.95Ford-T Production 0.95 0.99 0.96Average Dram Mbit 0.97 0.98 0.98

Continuous Operation Brazilian Ethanol 0.82 0.84 0.81Acrylonitrile 0.91 0.91 0.91Polyethylene-LD 0.96 0.98 0.98Polyethylene-HD 0.97 0.98 0.98Polyester Fibers 0.96 0.99 0.97Mean 0.89 0.96 0.93

The full dataset for each technology is used. The average results over the 12 technologies tested are also shown.

973S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

quality and efficiency improvements. However, in the long-run the reduction of the cost of producingelectricity and not simply the reduction of installation costs becomes more relevant and also represents thelearning of the technology in a more holistic way (Figs. 8 and 9).

In Fig. 10 where we have assumed a target price of the 1997 value of 3.37 USc(1990)/kWh, it is found thatthe forecasted cumulative learning investment is very uncertain when using experience curve analysis and isfound to be off by a factor of 10 or more during certain periods. In this case the worst forecasts are made usingWLS.

Looking at the forecast error in Fig. 11 it can be seen that the 6 forecasts for 2 doublings of cumulativeproduction (in the case of CCGT, about 8 or 9 years into the future) indicated errors in the range of 18%under to 10% over the actual values recorded. The total cost reductions for almost 4 doublings ofcumulative energy production went from 4.3 to 3.37 c/kWh or about 25% of the final price over 16 years.

Page 23: Forecasting technology costs via the experience curve — Myth or magic?

Table 7Forecast results for 4 to 12 years into the future assuming known cumulative capacity for future periods and presenting theforecasted price errors in their log format

Log results Time Experience

Yearsforecast

Observations Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

4 125 0.05 0.73 48 0 −0.10 0.51 3 06 101 0.71 1.08 0 0 −0.12 0.64 7 08 77 0.99 1.40 0 0 −0.12 0.75 18 4310 54 1.23 1.43 0 0 −0.09 0.75 39 8312 38 1.58 1.25 0 94 −0.15 0.85 29 24Average 0.911 1.180 10 19 −0.114 0.699 19 30

Table 8Forecast results for 4 to 12 years into the future assuming known cumulative capacity for future periods and presenting theforecasted price errors in their ratio format

Ratio results Time Experience

Yearsforecast

Observations Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

Mean Standarddeviation

Zero centredtest (%)

Symmetrytest (%)

4 125 0.58 0.97 0 0 −0.01 0.32 69 66 101 1.30 2.53 0 0 0.01 0.41 85 08 77 3.06 8.44 0 0 0.05 0.54 44 010 54 5.58 27.40 14 0 0.07 0.58 37 012 38 3.31 4.59 0 0 0.05 0.54 58 29Average 2.767 8.786 3 0 0.033 0.479 58 7

974 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

5.3. Modules case study — Solar

Solar PV provides a good example of using experience curves to forecast future costs of a renewableenergy technology since solar PV modules are generally accessed by an international market and sinceprices have also been well documented with 2 sets of long time scale datasets (Maycock's World PVMarket Report [20] used in this study and the Strategies Unlimited Datasets) (Fig. 12).

Solar PValso shows the typical random jumps and shifts in learning rate that could be expected from anytechnology having undergone such an increase in cumulative learning. Nevertheless, this technology showsa reasonably smooth experience curve where price reductions have occurred somewhat linearly to increasesin cumulative production when mapped on a log–log scale. There is also what could be described as afundamental shift in the LR occurring towards the end of the 1980's (Fig. 13).

Fig. 14 shows how the forecasted learning investment has increased sharply as it becomes apparent thatthe learning curve was flattening out (or that technology costs were going through a period of stagnation).WLS in particular shows how the total required learning investments points towards much higher thanpreviously expected learning costs (Fig. 15).

Page 24: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 23. Every third year's forecasts are presented in a graphical format comparing the results for time and experience as theexplanatory variable for Brazilian Ethanol. The Markers indicate the year in which the forecast is made base on all data availableup to that year. The green curves with square markers representing the experience curve forecasts while the red curves with circlemarkers represent the forecasts based on time.

975S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

5.4. Aggregated results for experience curve forecasts

In this section the various experience curve formulations and their ability to forecast into the future arecompared by aggregating the forecast errors for each number of doublings into the future within a singlegraphic as shown in Fig. 16.

The results for a single doubling of experience are the most reliable since they use the largest datasetavailable leading to a large forecast error dataset and an error distribution that is reasonably smooth.Unfortunately, a single doubling of experience referred typically to approximately 2 to 6 years dependingon the growth rate of the technology in question and the cumulative experience gained to date. Thissecond factor, the stage that a technology is in, comes into play since the time taken to generate a doublingof experience increases as the stock of cumulative experience increases, and this remains true even if thegrowth rate of a technology remains constant.

These results suggest that the SFEC, when considered in log format, is a very useful first orderapproximation with the distribution of the forecast error being both symmetrical and unbiased with amean value that is statistically not different from zero for both the OLS and WLS methods. It is alsointeresting to note that for this dataset the OLS method offered the best results in terms of mean deviationof the forecast error and as such is the least biased method for forecasting future costs in the short term.The overall error in terms of standard deviation is reduced slightly when using the WLS method.

Page 25: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 24. Similar to Fig. 23, this figure displays the forecasts for Solar using Time and Experience as explanatory variables.

5 For details of the Matlab function used please refer to Trujillo-Ortiz & Hernandez-Walls [26].

976 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

Making forecasts further into the future, Fig. 17 shows that the experience curve continues to providereasonably symmetric and unbiased results even after 3 doublings of cumulative experience, which generallytook between 6 and 12 years. Here the OLS method proves to be the most accurate in terms of variance butworse than the others in terms of bias.

Now looking further ahead to Fig. 18 where there are 6 doublings of cumulative experience it can beseen that the reduced data points available and the reduced number of technologies that contribute to thedataset reduces the quality and reliability of the results. As shown in Table 2, only 4 technologies remainthat contributed data for 6 doublings of experience, SCGT, Solar, DRAMs and Laser Diodes. The resultsare nevertheless indicative of reasonable quality forecasts, since with a progress ratio of, say, 20% thereduction in (log2) costs would be log2 (0.8^6) or approximately −1.93. Hence a mean error of prediction(also log2) of about 0.07 is very low compared to the total reductions that have occurred supporting theuse of experience curves to attain at worst an unbiased and symmetric forecast of future costs. Again wefound that OLS gives the most accurate forecasts in terms of standard deviation of the error, but a highermean deviation than WLS with 10% and 20% weighting factors.

The results of 1 to 6 doublings of technology are also presented in Table 3. Here the mean forecastingerrors and their standard deviation are presented along with the results of a skew-test implemented inMatlab.5 Both the Zero-centred Test and the Symmetry Test represent the probability of obtaining anobserved mean or skewness at least as far away from the zero value associated with the null hypothesis

Page 26: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 25. Similar to Fig. 23, this figure displays every year's forecasts for CCGTusing Time and Experience as explanatory variables

Fig. 26. Error histograms for 4 years into the future using first time and then cumulative experience as the explanatory variableAll results are shown in log format.

977S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

.

from a normal distribution. In all but the case for 1 doubling, the mean error is lower but the variance ofthe results higher with WLS as compared to OLS. At a 90% confidence level indicated by a test result ofless than 10%, the zero centred test indicates that all of the results are unbiased and the Symmetry Testindicates that none of the observed results in their log format are skewed. The mean errors (bias) are small

.

Page 27: Forecasting technology costs via the experience curve — Myth or magic?

Fig. 27. Diagrams show the aggregated (log) forecast errors for the 3 technologies CCGT, Solar and Ethanol for 1 and 3 doublingsof cumulative capacity.

Fig. 28. Histograms show the aggregated (log) forecast errors for the 3 technologies CCGT, Solar and Ethanol forecasting 4 and8 years into the future with cumulative experience.

978 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

and slightly negative when presented in their log format and the absolute value of this error is reducedwhen using aWLSmethod. On the other handWLS leads to a slight increase in standard deviation in longterm forecasts when comparing a 20% weighting factor to standard OLS.

Although it has been shown that experience curves generally give unbiased and reasonably symmetricforecasts for future costs when considered in ‘log’ format, the same can not be said when these results arereturned to their standard format. Because the results are symmetric and unbiased in log format, and due tothe convexity of the logarithmic function, it can be expected that results returned tomonetary values will be

Page 28: Forecasting technology costs via the experience curve — Myth or magic?

979S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

asymmetric and biased. Because of the way the errors are modelled in this paper, with positive errorsrepresenting an over-estimation of cost reductions and negative errors an under-estimation of costreductions, the log to standard transformation shifts the bias towards positive values and increasinglyasymmetric results as the forecasts goes further into the future. This can be seen from Figs. 19, 20 and 21 aswell as from Table 4.

From Table 4, we can see that WLS fares worse than OLS both in terms of Mean and StandardDeviation when errors are viewed as a ratio. At a 90% confidence level the distributions do not show aclear deviation from zero, however for OLS and WLS the mean error appears to be increasing as thenumber of doublings increase, as would be expected if an unskewed log distribution is transformed to anormal (non-log) scale. In 7 out of the 12 cases, 6 for OLS and 6 for WLS, the data is found to be skewed.

5.5. Forecasted error as a function of information

To test the second hypothesis, the error distributions of the forecasts are compared by splitting the validforecasts (those where there exists historical data to test the forecast made) into those made with low andthose made with high levels of information. The low information forecasts are made using only the dataavailable up until these first forecasts are made, while the high information data uses all data availableincluding those points used in the low information forecasts to make forecasts later on. The mean error fromobserved values is then compared to the variance of the error to determine any bias. The results show thathaving more information produced better results in terms of bias for almost all sets of aggregated forecasts.On the other hand, the standard deviation of the error increased by about 50% as can be seenwhen comparingthe results in Fig. 22. These results, as summarised in Table 5, suggest that the experience curvemay becomemore effective as a method for making unbiased forecasts however has difficulty to stand up to the longertime spans between doublings of cumulative capacity. These larger time spans could leave a technologysusceptible to other forms of fluctuations and shocks. Unfortunately these findings are particularly sensitiveto the datasets used and so the results for this hypothesis remain somewhat inconclusive.

5.6. Time versus cumulative capacity as explanatory variable

In this section we compare the relative merits of time versus cumulative capacity (experience) as theexplanatory variable to describe technology cost reductions. Naturally new technologies that are still in theirgrowth phases will show high levels of co-linearity between time and cumulative capacity. This can be seenin Table 6 where the R-squared values for the 12 technologies tested are very high for both time andcumulative capacity when all available data points are used for each technology. Nevertheless, cumulativecapacity is, on average and in almost every case, a better explanatory variable with an R-squared value of93% as compared to 89% for time.

Although many of the results remain quite similar with both descriptive variables, results for certaintechnologies such as Solar and SCGT are greatly improved when using cumulative capacity together withtime, in these cases leading to an increase in R-squared value of 12% and 23% respectively. Nuclearcapacity has an R-squared value that is 4% higher for time than experience.

Table 7 shows the forecast errors for 4 to 12 years forecast into the future. The results for time are generallypositively biased and skewed. Here the experience results are skewed in 2 of the 5 cases and are shown to bebiased in 2 cases. The larger positive mean errors for the time results indicate an overestimation of costreductions, that increases as one looks further into the future. The smaller and rather constant negative

Page 29: Forecasting technology costs via the experience curve — Myth or magic?

980 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

(though not statistically significant at the 10% level) mean errors for experience, if anything, would indicatethat experience curve analysis leads to a slight underestimation of cost reductions and so represents aconservative estimate. Not only is the mean (bias) and standard deviation of the error distribution smallerwhen using experience as the explanatory variable, but the levels of skewness are also far lower (as indicatedby the higher probabilities of the distribution not being skewed).

In Table 8 the results are shown as a ratio error and not as a log error. Here the relative symmetry of theexperience curve results in their log format translates into an overestimation in forecasted cost reductionswhen errors are considered in their ratio form. These results are similar to those found in the previoussection where forecasts are considered for a distinct number of doublings of experience and not a setnumber of years into the future. One difference is that, at least in the medium term of up to 12 years, thebias is not statistically significant for the ratio results. The error distribution does, nevertheless, remainskewed at the 90% confidence level in almost every case (Figs. 23, 24, 25 and 26).

5.7. Singling out the clean energy technologies

Finally, the aggregated OLS forecast errors for the 3 case study technologies alone have been presentedsince they represent three of the most relevant technologies to climate change policy. The results for thesetechnologies are presented both in terms of forecasted doublings in Fig. 27 as well as forecasting a numberof years into the future in Fig. 28.

The results are generally found to be skewed when considered in terms of doublings into the future, aspresented in Fig. 27 for 3 doublings of experience. This suggests that renewable technology forecasts maybe overconfident due to a flattening out of the learning curve as shown for Solar, however any such claimwould need to be verified by a much larger dataset. On the other hand, the learning curve results for up to8 years ahead remains unbiased despite the small number of observations and limited number oftechnologies. In the majority of cases 8 years represent less than 3 doublings of cumulative capacity.

6. Discussion and conclusion

Grübler, Nakicenovic and Nordhaus [24] raise the question of whether “we have sufficient scientificknowledge about the sources and management of innovation to properly inform the policymaking processthat affects technology-dependant domains such as energy” to which they believe the answer is “Not yet”.Perhaps due to this insufficiency and the lack of a clearly superior heuristic or innovation theory forforecasting technology costs, and despite the many shortcomings of the experience curve theory,experience curves continue to be used widely. This current research does not try and improve theunderlying theory of experience curves, but instead tests empirically, using historical data, the validity ofexperience curves for forecasting and provides a first order approximation of the uncertainties that existfor potential growth technologies such as renewable energies. As a result of this current piece of research,evidence supporting the following conclusions has been found:

Hypothesis 1, that experience curves have provided forecasts with amean error not statistically different to0 was not falsified when considering forecast errors in their logarithmic format. In general it was found thatthe bias remained small and relatively constant going into the future at a small negative and generallystatistically insignificant level. The small negative mean shows that the results in their log format led to aslight underestimation of cost reductions. There remains the highly important caveat that due to the convexity

Page 30: Forecasting technology costs via the experience curve — Myth or magic?

981S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

of the logarithmic function and the symmetric unbiased results found when using the log–log format,experience curves viewed in their normal format tend on average to overestimate future cost reductions andincreasingly so as one looks further into the future. Together the results suggest on one hand that the SFECusing price data alone has historically served as a useful estimator of future prices for the given dataset. Onthe other hand, the wide distribution of forecast errors highlights the need for models considering the use ofexperience curves to account for stochastic uncertainty in a manner conforming to observed levels.

Hypothesis 2, that the ability to forecast technology costs “improves” as more data points are addedremains unsure since more data led to an error distribution that was less biased, though had increasedvariance. Perhaps this outcome occurs since, although the actual error of a forecast in monetary units maydecrease significantly, the error in terms of log2 or percentage changes may in fact be increasing. Anotherplausible explanationwould be that uncertainty increases as the time tomake a fixed number of doublings ofexperience increases, as occurred with most technologies since the distance between data values on the x-axis (in log format) generally get closer together as a technology matures. Access to a larger representativedatabase would help to bring more concrete results in particular with respect to this hypothesis.

Hypothesis 3, that time has served as a better explanatory variable than experience (calculated as the log ofcumulative capacity) is found to be FALSE. Here it is found that although both time and experience are highlysignificant as single explanatory variables, experience turns out to be a vastly superior explanatory variablethan time in terms of forecasting error. Forecasts made with experience curves lead to error distributions thatare less biased, have lower variance and are also more symmetrical than those made with time.

Finally it is found that when aggregated over all the technologies tested and viewed in their log format, theuse ofWLSgenerally decreases themean deviation or ‘bias’ of the forecast error distribution but increases itsvariance as compared to the OLS method. In many cases the differences are quite small. This suggests thatthe WLS method could be further looked into.

One of the principal difficulties with informing policy makers on how best to bring about costreductions of renewable energy technologies is to decide how to divide a limited budget. Funds need to beconcentrated enough to bring about desired cost reductions of chosen technologies while being broadenough to offer a range of possible technical solutions. These may be needed in the case that thetechnologies first picked as winners turn out to be undesirable or unsuccessful (one only needs to think ofthe public resistance to onshore wind farms in the UK and elsewhere). As remarked by Wene in his IEApublication, “learning opportunities in the market and learning investments are both scarce resources”suggesting that the concentration of resources is key to generating solutions, whilst on the other hand, the“availability of renewable resources, reliability of the energy system and the risk of technology failurerequire a portfolio of carbon-free technologies” [1] (IIASA approach, see, e.g. [16]).

Here, distributions of forecasted technology price errors are calculated based on historical data. If futureforecasting errors were to follow a similar pattern, research could use this information to help design‘optimal’ energy technology portfolios. Such an approach would almost definitely be problematic becausestrategically accelerating experience growth would almost certainly lead to a distortion of the experiencecurve's performance, especially since single factor experience curves exclude important non-experiencerelated factors. Hence, the possibility of combining an experience element and a time dependant element intoIAM's may offer a way forward. The former element would describe cost reductions closely linked to themodel's decision variables and the second element the multitude of influences, such as improvements inbasic science and general R&D, that remain exogenous to many models. So to answer the question initiallyposed, are experience curve forecasts best described as “Myth or Magic?” the results presented in the papersupport the conclusion that experience curves are not Myth in that they have offered, at least within the

Page 31: Forecasting technology costs via the experience curve — Myth or magic?

982 S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

context tested here, an unbiased forecast of future technology prices. They do not, however, offer up aMagicsolution. In fact, due to the high variance of the forecast errors and the possibility that strategically supportedrenewable technologies fair worse than the basket of technologies tested in this paper, the results would onlysupport their use within a stochastic environment and then only with adequately wide uncertaintydistributions.

A great deal of more work could follow in this area, for example by improving data quality and increasingthe number and scope of technologies tested. Furthermore, the forecasting ability of other formulations suchas the inclusion of technology clusters could also be investigated, as could the circumstances and criterion forthe use of WLS as a preferred method over standard OLS regression. Most importantly however, byunderstanding the relative importance of R&D and experience as drivers of cost reductions (such as resultsfromPapineau [3] and Jamasb [25]) aswell the importance of an exogenous time dependant element in termsof forecasting ability, researchers could develop a more accurate model of technology cost forecasting andimprove their applicability within IAMs.

Acknowledgements

This research is based on work undertaken at the International Institute for Applied Systems Analysis(IIASA) under the supervision of Yuri Ermoliev and at the Judge Business School, Cambridge Universityunder the supervision of Chris Hope. The Author would also like to thank Arnulf Grübler, LeoSchrattenholzer, Marek Makowski, Stephen Pollock, Nebojsa Nakicenovic, Clas Otto Wene, David Victor,Brian O'Neil, Marvin Lieberman, Oswoldo Lucon, Paul Maycock, Ulrika Colpier Cleason and NadejdaVictor for their contributions with data, ideas or words of advice during this research project.

References

[1] IEA, Experience Curves for Energy Technology Policy, IEA Paris, France, 2000.[2] S. Alberth, C. Hope, Climate modelling with endogenous technical change: stochastic learning and optimal greenhouse gas

abatement in the PAGE2002 model, Energy Policy 35 (2007) 1795–1807.[3] M. Papineau, An economic perspective on experience curves and dynamic economies in renewable energy technologies,

Energy Policy 34 (2006) 422–432.[4] A. McDonald, L. Schrattenholzer, Learning rates for energy technologies, Energy Policy 29 (2001) 255–261.[5] J. Everett, S. Farghal, Experience curve predictors for construction field operations, J. Constr. Eng. Manag. 120 (3) (1994)

603–616.[6] J. Everett, S. Farghal, Data representation for predictive performance with experience curves, J. Constr. Eng. Manag.

123 (1) (1997) 46–52.[7] S. Farghal, J. Everett, Experience curves: accuracy in predicting future performance, J. Constr. Eng. Manag. 123 (1) (1997)

41–45.[8] A. Grübler, S. Messner, Technological change and the timing of mitigation measures, Energy Econ. 20 (1998) 495–512.[9] G. Schaeffer (Ed.), Photovoltaic Power Development: Assessment of Strategies Using Experience Curves (acronym

PHOTEX), Synthesis report, Energy Research Centre of the Netherlands (ECN), 2004, accessed via http://www.energytransition.info/photex/docs/synthesisreport.pdf, 3/8/06.

[10] K. Mishina, Learning by new experiences: revisiting the Flying Fortress Experience curve, in: N. Lamoreaux, D.M.G. Raff,P. Temin (Eds.), Learning by Doing in Markets, Firms, and Countries, University of Chicago Press, Chicago, 1999.

[11] M. Lieberman, The experience curve and pricing in the chemical processing industry, Rand J. Econ. 15 (2) (1984) 213–228.[12] L. Coulomb, K. Neuhoff, Experience curves and changing product attributes: the case of wind turbines, Cambridge

University working paper series EPRG 0601, 2005.[13] K. Zweibel, Issues in thin film PV manufacturing cost reduction, Sol. Energy Mater. Sol. Cells 51 (1) (1999) 1–18.

Page 32: Forecasting technology costs via the experience curve — Myth or magic?

983S. Alberth / Technological Forecasting & Social Change 75 (2008) 952–983

[14] K. Neuhoff, Large scale deployment of renewables for electricity generation, Oxf. Rev. Econ. Policy 21 (1) (2005) 88–110.[15] U. Colpier, D. Cornland, The economics of the combined cycle gas turbine—an experience curve analysis, Energy Policy

30 (2002) 309–322.[16] A. Gritsevski, N. Nakicenovic, Modelling uncertainty of induced technical change, Energy Policy 28 (2000) 907–921.[17] M. Junginger, et al., Global experience curves for wind farms, Energy Policy 33 (2005) 133–150.[18] L. Christiansson, Diffusion and experience curves of renewable energy technologies, IIASAWorking Paper SeriesWP-95-126,

1995.[19] IIASA and WEC, Global Energy Perspectives, Cambridge University Press, Cambridge, New York, 1998.[20] P. Maycock, PV REVIEW: World Solar PV Market Continues Explosive Growth, ReFOCUS Sep/Oct, 2005.[21] N. Victor, J. Ausubel, DRAM's as model organisms for study of technological evolution, Technol. Forecast. Soc. Change

69 (3) (2002) 243–262.[22] J. Goldemberg, S. Coelho, P. Nastari, O. Lucon, Ethanol experience curve — the Brazilian experience, Biomass &

Bioenergy 26 (2004) 301–304.[23] M. Antes, S. Yeh, M. Berkenpas, Estimating future trends in the cost of CO2 capture technologies, Final report to IEA GHG

R&D Program, 2005.[24] A. Grübler, N. Nakicenovic, W. Nordhaus (Eds.), Technological Change and the Environment, Resources for the Future,

Washington, DC, 2002.[25] T. Jamasb, Technical change theory and learning curves: patterns of progress in electricity generation technologies, Energy

J. 28 (3) (2007).[26] A. Trujillo-Ortiz, R. Hernandez-Walls, Skekurtest: Hypotheses test concerning skewness and kurtosis. A MATLAB file,

2003 Accessed via http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=3953&objectType=FILE, 11/04/07.

Stephan Alberth has an MPhil in Management from the Judge Business School, University of Cambridge. The above paper hascontributed to his PhD in Management, also at the Judge Business School. Much of the research leading to this paper took placeduring his participation in the Young Scientists Summer Program at IIASA.