what does a negawatt cost

8/7/2019 what does a negawatt cost

1/13

Paul Joskow is Mitsui Professor ofEconomics at the MassachusettsInstitute of Technology and haswritten extensively about electricutility matters.Donald Marron is a Ph.D.candidate in the Department ofEconomics at the MassachusettsInstitute of Technology.

14

What Does a Negawatt ReallyCost? Further Thoughtsand EvidenceKbcost ofenergy conservation programs is considerablyhigher and the energy savings achieved are a good deallower than generally recognized. Utilities need to do abetter job of measuring all relevant costs; they should alsoadopt more credible savings measurement protocols.Paul L. Joskow and Donald B. Marron

eW recently published astudy of energy conserva-tion programs that have been un-dertaken by a sample of U.S. elec-tric utilities. We found that thecosts of saved energy reported byutilities are typically significantlyhigher than well-hewn technicalpotential studies would suggestand that, furthermore, the IV-ported costs are likely to be under-estimated due to various prob-lems in utility cost accounting andenergy savings measurement.The large number of commentswe have Teceivedon our study,coupled with the appearance inthe literature of some related re-

-searchefforts,indicates the desir-ability of additional examinationand discussion of the costs ofsaved energy associated with util-ity conservation activities. To fos-ter this discussion, this paper willrespond to some of the criticismswe have received and will com-ment on three nzcentstudies: anexamination of 40 successfulDSM programs by Flanigan andWehtraub,2 a review of overallconservation program costs at 12utilities by ONeill? and a recentevaluation of the BonnevillePower Administrations (BPA)Residential Weatherization Pro-gramP The latter program is espe-

TheElectricity Journal


2/13

cially interesting to examine inlight of what we feel to be themost significant criticisms of ourearlier work.I. Our Original Study

As a prelude to our discussion,it is useful to review our originalstudy Its objectives were to meas-urethecostsandenergysavingsthat utility conservation programsam actually achieving to comparethese results to frequently citedtechnical potential (TP) studiessuch as those of the Rocky Moun-tain Institute @MI) and the Elec-tric Power Research Institute(EPRI): and to determinewhether utilities am accountingfor and measuring costs and en-ergy savings in ways that are con-sistent with sound economic andregulatory accounting principles.

To perform our study we se-lected a group of electric utilities,many of which are regarded asleaders in conservation initiatives,from which we sought alI rele-vant information about recent,current, and planned conserva-tion programs. The informationwe received primarily coveredprograms in 1990 and 1991, withsome projections for future years.We found conservation costs re-ported by utilities to be signifi-cantly higher than the projectionsembodied in the RMI and EPRIanalyses. Overall program costsreported by the ten utilities in oursample ranged from 1.9 to 6.9cents per kWh (in 1991 dollars),with an average of about 3.4 centsper kWh saved - about 30Shigher than the overall EPRI esti-mate of the cost of technically

available conservation (2.6 cents)and almost 500% higher than theoverall RMI estimate (0.6 cents).For residential programs the aver-age reported costs ranged from3.5 cents to 22.1 cents per kWh,while for commercial/industrialprograms, the range was 1.5 centsto 6.7 cents. Costs reported for in-dividual program components(e.g., lighting, motors, etc.) variedfrom a fraction of a cent to $1.81per kWh saved for residential pm-grams and from a fraction of acent to 18 cents per kWh saved forcommercial/industrial programs.

Ouranalysis suggeststhat utilities oftenunderstate prbgramcosts and overstateprogram energy savings.

Reported costs exceed those ofRMIs and EPRIs TP analyses be-cause utilities report at least someadministrative costs (overhead,program monitoring and evalu-ation, marketing administration,etc.) that am either ignomd @Ml)or understated (EPRI) in TP analy-ses, because some utility pro-grams have measured savingsthat fall significantly below ex anteprojections of the kind that am in-corporated in TP studies, and, inthecaseoftheRMIstudybecauseestimates of costs and perform-ance are excessively optimistic, re-lying on uncertain technologies

withuncertaincostsandlimitedcommemialmarketexperience.The large variance in costs is, inpart, due to large differences incost accounting and energy sav-ings estimation techniques em-ployed by utilities.6

While the costs that we corn-puted exceed TP projections, atleast on average these costs ap-pear, at first blush., to be quite lowcompared to typical long-run mar-ginalsupplycostsinthe5to7cents per kWh range. As a result,some people have interpreted ourresults as demonstrating thatthese programs are cost effective.However, as we emphasize in ourstudy it would be imprudent torush to this judgment without ex-amining further the quality of thecost and energy savings informa-tion reported by utilities. Ouranalysis suggests that utilitiesoften understate program costsand overstateprogramenergysavings; as a result, reported costsof saved energy am often too low.0 n the cost side, many utili-ties fail to track fully theadministrative costs of their pm-

grams. Others fail to measure thecosts incurred by program partici-pants who bear a portion of directcosts as well as a variety of realtransaction costs. On the electric-ity savings side, many utilitiesbase their savings estimates on exante engineering projections ofsavings (like those of the TP stud-ies) rather than expost evaluationsof actual changes in consumption.Utilities and independent anaIyststhat have undertaken camful expt evaluations often find actualenergy savings to be far below

July1993 15


3/13

oliginalprojections. Ehergy sav-ings estimates may also be toohigh because utilities,and IFanalyses,rely upon engineeringlifetimes for equipment ratherthan economic lifetimes; engineer-inglifetimesmaybetoolongbecause economic criteria often leadto the retirement of devices beforethe end of their engineering lives.Finallyenergysavingsmaybetoo high when free riders are ig-nored.

For all these reasons, we foundthat utilities tend, on average, toreport conservation costs that aretoo low and energy savings thataretoohigh. Asaresult,there-ported costs per kWh saved aresystematically lower than the ac-tual costs. While it is impossibleto know exactly how large the un-derstatement is, we would not besurprised if it were a factor ofabout two, on average. In addi-tion, the experience of utilitieswith careful measurement pro-grams indicates that the magni-tude of energy savings achievablethrough utility programs is sub-stantially smaller than indicatedby the TP studies.II . Criticisms of the Study

Almost no one who offeredcomments on our study ques-tioned the validity of the compu-tations that we presented. Manypeople agreed that the cost ac-counting and energy savingsmeasurement issues that we iden-tified axe indeed significant pmb-lems that need to be addressedmore comprehensively Therewas also broad agreement that theactual costs per kWh saved in util-

ity conservation programs are sig-nificantly higher than the most op-timisticTPestimatesthatarefrequently cited in the media, beforestate regulatory agencies, and inother government forums. Withone exception (Amory Lovins),everyone agreed that the Lovins/RMI numbers, in particular, drasti-cally understate the true costs ofenergy conservation programs.

Most of the criticism we re-ceived focused instead on threeissues:

Nearly everyoneagreed that theLovins~ numbersdrastically understatethe true costs of energyconservation programs.

(1) Is it appropriate to comparethe TP studies to the experience ofa sample of utilities?(2) Are the utilities in the sam-ple representative of current util-ity experience?

(3) And, most important, areour results indicative of the futureperformance of utility programs?

A. Is This An AppropriateComparison?Some commentators objected to

the comparison of aggregate TPprojections with the performanceof a sample of utility programs; intheir view, this comparison suf-

fers from the familiar fallacy ofcomparing apples and oranges.Leaving aside the repre-sentativeness of our sample,which we consider below, we be-lieve that the comparison is com-pletely appropriate. The TP stud-ies have been widely displayed inpolicycirclesandinthemediaasguides to the need for and cost ofpolicy intervention on the de-mand side of the electricity mar-ket. Indeed, the National Acad-emy of Sciences developed andrelied on TP studies of this kind inits proposals for policies towardglobal warming.8 The actual per-formance of programs that havebeen developed in response tothese projections is the best basisfor determining whether currentpolicies am delivering the prom-ised benefits.

B. Is the SampleRepresentative?Ithasalsobeenarglledthatour

study is based on an unrepresen-tative sample of immatum con-servation programs. AmoryLovins, for example, charges thatour numbers are based on a small,unscientific sample of utilityprograms that am dubiously repmsentative and often inferior indesign or execution to modern in-dustry norms.@ Lovins offers noexamples of the more rep*sentative utility programs that hehad in mind. The implication ofthis criticism is that if we had ex-amined a more representativesample of utility programs wewould have come up with differ-ent results.

16 TheEt.ectricityJournal


4/13

This criticism is simply unfair.Several of the utilities in our sam-ple have had conservation pro-grams in place, especially for resi-dential customers, for nearly adecade (e.g., Long Island Light-ing). Others are often pointed toas conservation leaders in the util-ity industry (e.g., Central MainePower, Massachusetts Electric,and PC&E). Furthermore, morethanhalfoftheutilitiesinoursample have programs (what wecall subprograms) identified byFlanigan and Weintraub as beingamong the most successful inNorth America. To the extent thatour sample is biased it is probablybiased toward the more matureand highly regarded utilities inthe country As a result, the criti-cism that we have not examinedutilities with good programs isnot supported by any credible evi-dence.

C Will Utility ProgramsPerform Better in the Future?Several commentators have sug-

gested that while our evidencemay be representative of whathappened in the past, it is not rep-resentative of how utility conser-vation programs will perform inthe future. They argue that utili-ties are getting better at the conser-vation business, accounting for allcosts more systematically andmeasuring energy savings andfree riders more accurately Betterinformation, learning by doing,and economies of scale, it is ar-gued, will help to drive down thecosts of utility energy conserva-tion initiatives over time.

We am optimists and am in-clined to believe that things willget a lot better in the future if utili-ties and regulators respond to thelessons learned from experience.Moreover, discussions that wehave had with people associatedwith some utility programs sug-gest that there is now a lot moresensitivity to the issues that wehave raised. Some utilities ampaying more attention to cost-ac-counting issues, introducing andrefining protocols to measure sav-

Tk criticism that wehave not examinedutilities with goodprograms is notsupported by anycredible evidence.ingsactually achieved, and evalu-ating the evolution of the marketto account for free riders How-ever, we have not seen any cred-ible systematic study of utilityprograms to suggest that the costaccounting and energy savingsmeasurement problems we identi-fied have been addressed by thebulk of utilities or that the bottomline on the cost of saved energy isany different from what we foundin our study The best that wecansayisthatitistooearlytotell.As we suggest in Section m,which examines one of the oldestand bestdocumented programs

inthe country,while them may bereasonsto believe it will becomeeasier to achieve cost-effective en-ergysavingsinthefutule,thereareatleastasmanymasonstobelieve savings will come at an evenhigher price than now.III. Related Research

In recent months, a number ofstudies have appeared regardingthe cost and performance of util-ity conservation programs. In par-ticular, two studies published inthis journal have reviewed theperformance of samples of utilityprograms. We believe that it isuseful to comment on these stud-ies in light of the results of ourownresearch.

A. Successful DSM Programs:Flanigan and WeintraubTed Flanigan and June Wein-

traub (F&W) recently reviewed 40successful DSM programs (13of the programs on the list wereselected from utilities in our sam-ple).O Half of these programs arefrom U.S. investor-owned utilities(on which our study focused) andthe other half are from municipal,state, federal, and Canadian utili-ties and agencies (which we didnot examine). Since F&W includemore than one program (what wecall subprograms in our study)for some utilities, programs for 10U.S. investor-owned utilities amincluded in F&Ws study Whiletheir review considered a broadvariety of program attributes, themost important results, at least interms of our own research, arethose regarding the costs of savedenergy and the quality and consis-

J u Z y1993 17


5/13

terry of data reported by utilitiesincalculatingthesefigures.In general terms, F&Ws find-ings appear not very differentfromourown. Intermsofthecost of saved energy F&W findthat figures reported by utilitiesvary widely, from a fraction of acent to nearly 12 cents per kWh,with a simple average of about 2.6cents. Since F&W piescreenedtheir sample for successful pro-grams (which we did not) and ex-cluded customer costs from theircomputations, it is not surprisingthat their reported average islower than that of the programsin our sample, and that the costvariance is significantly smaller,since the unsuccessful programsthat have not been included arelikely to be relatively high cost.=F&W recognize that properaccounting for actual energysavings achieved by programs isvery important, that the care withwhich savings are measured var-ies widely across utilities, and that. . . just when you think youvegot significant levels of savings,you unfortunately may not!nThey commend two utilities fordoing an excellent job in measur-ing actual net savings. One utilitytakes credit for 56% of engineer-ing estimates; the other, 23%.F&W say that caution must betaken to determine whether sav-ings are based on engineering esti-mates . . . or whether process andimpact evaluations have been con-ducted, and whether the savingshave been adjusted for free rider-ship, unusual weather, snapback. . ,, measure persistence and attri-

tion, etc/14 Ourpapersmakees-se&ally the same points.F&W also appear to recognizethat the costs reported by the utili-ties in their sample am not di-rectly comparable because theutilities pay different fractions ofthe total cost of the conservationmeasures included in the programs. Indeed, F&W report otzrythe costs incur& by utilities, con-sciously ignoring customer costs,which they recognize may belarge.

unforhlnately these importantpoints are ignored in F&Wsanalysis of low-cost energy sav-ings. Meaningful measurementof the cost of energy saved by util-ity conservation programs shouldinclude all of the utility and cus-tomer costs incurred in connec-tion with the programs. It shouldalso reflect accurate measurementof the actual energy savingsachieved. The cost of saved en-ergy figures reported by F&W intheir Figum 3 generalty satisfy nei-ther of these criteria and are easilysubject to misinterpretation. In-deed, Peter Miller of the Natural

ResourcesDefensecouncil,an-other criticof our work, haspointed to the F&W study asshowing that these programs areexceptionally cost-effective.~lli

Such a cost-effectiveness conclu-sion cannot be drawn from theF&W figures because, as with thenumbers reported in our study itis likely that they significantly un-derstate true program costs. Al-thoughsomepIogramsintheirsample have employed careful expost measurement of savings,many am still based on ex ante en-gineekg estimates of energy sav-ings which, as we have discussedand F&W themselves appear toconfirm, are likely to be too opti-mistic. Also, F&W include onlyreported utility costs and excludecustomer costs, while a proper ex-amination of cost effectivenessmust include both.

Moreover,F&W accept un-critically the costs re-ported to them by utilities as fullyreflecting all of the costs the utilityitself incurs in connection withthese conservation programs. Asnoted pmviously our analysisfound that many utilities did notaccount fully even for all of theirown costs. All three of these fac-tors imply that the costs reportedby F&W am likely to underesti-mate significantly the true cost ofthese programs.It should also be clear that theway F&W report the cost datamakes it difficult to make norma-tive comparisons across utilities,because neither the costs nor en-ergy savings measurement tech-niques embodied in the utilitynumbers they rely on am compa-

18 TheElectricityJoumal


6/13

rable. For example, a utility thathas used sound cost-accountingand energy savings measummenttechniques while reporting a costof 5 cents per kWh saved mayhave a much better program thanone that reports a cost of 2 centsper kWh saved but failed to ac-count properly for all relevantcosts or to use sound ex pt meas-urement techniques.EvenifF&W had includedonly those utility programsthat fully accounted for all utilityand customer costs and providedaccurate measures of the energysavings the programs achieved,they have not presented enoughinformation to perform a propercost-effectiveness analysis. Utility-specific avoided cost informationis required when making such ananalysis. Aside from several pro-grams which were obviously notcost effective, we eschewed mak-ing cost-effectiveness judgmentsin our study both because of thedeficiencies in information avail-able to make credible calculationsof the cost of energy saved andthe difficulties of integrating suchinformation with avoided cost fig-ures for a large number of utilities.

We must also observe that it isnot a particularly good scientificpractice to examine only thoseprograms that are prescreened forsuccess. This kind of censoredsampling will necessarily lead tobiased results. Moreover, there isa lot to learn from failures as wellas from successes. We were ableto obtain information for all of theprograms conducted by several ofthe utilities having programs in-cluded on F&Ws list. We fail to

see why F&W had to restrict theirstudy to successful programs.

B. The Quality of UtilityPrograms: ONeillMaura ONeill recently dis-cussed the fact that utilities differ

in the quality of their programsand that utilities ought to learnfrom their differencesi To facili-tate this learning she then set outto compare programs at the bot-tom line, in cents per kWh saved.ONeill presents data for first

year program costs divided by an-nual program energy savings for12 utilities in 1992. Unlike F&W,but like us, she examines the en-tire conservation program of theutilities that she selects, not just se-lected sub-programs as do F&W.She indicates that these programshave been screened so that theyinclude all direct and indirect pm-gram costs; she does not explicitlystate whether customer costs areincluded, however, and our suspi-cion is that they are not, or at leastthat they are not reported fully byall of the utilities she selected (oth-erwise, them would be no reason

to believe, as she suggests, thatdifferences in the rebate rateacross utilities would explain asignificant fraction of the ob-served diffemnces).

It is not clear what types of en-ergy measurement protocols havebeen used in ONeills programevaluations. The utilities in thesample are those for which 1992data are supposedly complete,but since her study was preparedbefore May 1993 it is probably toosoon after the end of 1992 to havedetailed ex p& savings measure-ments for these programs in hand.H er major finding is thatfirst year costs vary by afactor of about five. She arguesthat these variations are too largeto be attributable entirely to differ-ences in conservation opportuni-ties, cost conditions, measurelives, and other factors that we ex-pect will lead to cost variationseven for high-quality pmgrams.She interprets the variation in thecosts as Ieflecting opportunities toimprove the efficiency and qualityof these programs, implying thatsome utilities are doing a muchbetter job managing their programs than are others.

We believe that her data requirea much more cautious interpreta-tion. because of the vast differ-ences among utilities in cost ac-counting and energy savingsmeasurement, we are reluctant toassociate observed variation incost per kWh figures with differ-ences in the efficiency and qualityof these programs. There are well-regal&d programs that leportboth high costs and lowcosts, as confirmed by F&W.

July 1993 19


7/13

The problem is that utilities are us-ing very different cost-accountingprotocols and energy savingsmeasurement techniques for whatam often similar types of pro-grams. These considerations mayexplain a great deal of the vari-ance that is observed in the data.Until a common framework forcost accounting is adopted and alIutilities adopt credible ex post en-ergysavingsmeasummenttech-niques, it wiIl be impossible to dis-entangle cost differences resultingfrompmgramqualitydifferencesfrom cost differences associatedwith different cost-accountingrules, different measurement tech-niques, and differences in cost-ef-fective conservation opportunities.Despite these reservations,0Nei.U offers a number of veryuseful suggestions for improvingthe quality of utility pmgrams.We would add to her list: (1) bet-ter accounting for all utility andcustomer costs and (2) adoptionof more credible protocols formeasuring the savings that theseprograms have actually achieved.IV Changes Over Time: TheBPA Experience

Aswediscussedearlier, severalcommentators have suggestedthat our evidence is not indicativeof how utility programs will per-form in the future. They arguethat utilities will do a better job ofcost accounting and that measure-ment of energy savings achievedwill become more accurate asmore experience is gained. Theyalso argue that better information,learning by doing, and economiesof scale will help drive down the

costsof utility energy conserva-tion programs over time. How-ever, these changes could cut bothways.

It is likely that better accountingfor all utility and customer costsand better energy savings pmto-cols wiIl lead to higher, ratherthan lower, measured costs perkWh saved. Furthermore, utilitiesmay have gone after the cheapestconservation opportunities first,so that costs will rise overtime. Arecent evaluation of the residen-

tial weatherization program of theBonneville Power Administration(Bonneville, or BPA) can helpshed light on these dynamic as-pects of utility conservation programs.7

A. TheProgramBonnevillehas beenrunningaresidential weatherization pm-gram for customers with electri-calIy heated homes since 1980.This is the only program of whichweareawarethathasbeeninop-eration for such a long period oftime. The program has appliedhigh standards to cost accounting,measurement of energy savings,

and ovedl program evaluation.BPA accounts for utility costs(both its own and that of a hostutility), as well as customercosts. It also accounts for certainadministrative costs.U dikemanyother utilityprograms, the BPApro-gram has been concerned aboutmeasurement issues from its in-ception Accordingly, each cohortof participants has been matchedwith a control group of non-par-ticipants. The program thenmakes use of ex ante engineeringmodels to predict participant sav-ings for planning purposes aswell as ex pest measurement ofsavings derived from compari-sons between the participants inthe program and the controlgroup to evaluate actual programperformance ex post. These com-parisons between participant andcontrol groups have been madeboth for the first year followingthe instaIlation of weatherizationequipment and for two sub-sequent years as well. This makesit possible to examine whether thesavings measured for the firstyear persist over at least the earlypart of the expected life of thecapital investments made to con-serve electricity

Additional stability and compa-rability in the data result from thefact that measuremen t and evalu-ation have long been the responsi-bility of Oak Ridge National Labo-ratory, probably the mostexperienced group in this area.While we am not entirely comfort-able with the way the controlgroups are selected and utilized,the only component of our list of

20 TheElectricityJournal


8/13

desirable program attributes thatis missing from the BPAprogramis an explicit accounting for freeriders.The BPAprogram provides fi-nancial incentives to residentialelectric space heat customers to in-stall a variety of measures to in-crease energy efficiency. The pro-gramistypicalinthatitreliesonrebates provided by BPA and thehost utility, with only modest cus-tomer contributions required.Bonnevilles efforts in this area began with a pilot program in 1980.All together there are six pro-gram cohorts that have been ana-lyzed. The most recent analysisfocuses on the 1988 and 1989 pm-gr-.The 1991 Northwest Conser-vation and Electric PowerPlan, which addresses the BPArole in the Pacific Northwest re-gion,* lists single-family residen-tial weatherization as a re-sources with an expected cost of3.4 cents per kWhJg This is veryclose to the cost reported by EPRIfor residential space heating con-servation opportunities and is de-rived from engineering analysesthat are not unlike those reliedupon by RMI and EPRI. The OakRidge study indicates that BPAsregional ceiling price for conserva-tion resources is 5.9 cents perkWh saved?0 We take this to beBPAs assessment of the regionalavoided cost of new electricitysupplies against which conserva-tion investments are to be com-pared for cost effectiveness. Thus,if the ex ante engineering assump-tions of program costs used forplanning purposes in the BPAm-

gion are a good approximation tothe actual cost per kWh savedachieved by the program, expen-ditures on residential weatheriza-tion induced by the programshould be very cost effective. Thequestion is: How do the ex anteprojections compare to the ex Postperformance of the program?

B. Results of the EvaluationTable 1 provides some evalu-

ation results regarding the meas-ured cost per kWh saved and therelationship between preretrofitestimated savings based on auditsand engineering models and thepost-retrofit measured savingsbased on comparisons betweenparticipants and controls. IThefirst column reports the estimatedlife-cycle costs for the 1988 and1989 weatherization programsbased on the ex post measured sav-ings and assuming that measuredsavings for the first post-retrofityear persist over the entire ex-pected life of the measures. Thesecond column contains similarfigures that have been adjusted toreflect deterioration in measuredprogram savings over time. Wewill return to this columnpres-ently The third column listsBPAs avoided-cost ceiling. The fi-nal column is the ratio of expost

measuredenergysavingstotheexante engineering estimates.From the last column it is evi-dent that ex pt measured sav-ings are substantially less than theengineeringestimates;measuredsavings are only 30% to 40% of exante projections2 Moreover, incomparing the gross savingsbased on billing records with theengineering predictions for eachparticipant, the study finds thatthe correlation between the two isonly 0.1. Thus, despite manyyears of experience both with exante forecasting and ex pt meas-urement, BPA is still using engi-neering models that performpoorly both in tracking individualusage patterns and in estimatingactual savings. This is consistentwith the conclusions in our pa-pers. Experience and programmaturity do not yet appear tohave solved this problem.The first column in Table 1displays the measured costper kWh saved for the 1988 and1989 programs based on total pro-gram costs (BPA, host utility cus-tomer, and administrative costs)and net measured savings for thefirst post-retrofit year; the netmeasured savings are assumed topersist over the life of the conser-vation measures. The measured

Table 1: BPA ResidentialWeatherizationProgramResults: Most Recent CohortsStudiiMeasuredCost AdjustedCcst AVOi d t d -perkWhSavedp e r k Wh S a v e d kzJlzrs( 1 9 9 1 c e n t s ) ( 1 9 9 l c e n t s ) C o s t C e i l i n g( 1 9 9 l c e n b ) s a v i n g s

5 . 5 6 . 9 5 . 9 0 . 4 29 . 1 1 1 . 4 5 . 9 0 . 3 1

July 1993 2 1


9/13

cost per kWh saved is 5.5 centsper kWh (in 1991 dollars) for the1988 program and 9.1 cents perkWh for the 1989 program. The5.5 cents figure is 62% higher thanthe 3.4 cents value used by theNorthwest Power Planning Coun-cil for planning purposes, whilethe 9.1 cents figure is 168% higher.Both the 1988 and 1989 measuredcosts far exceed the TP values reported by EPRl for residentialheating conservation opportuni-ties. The 1988 program is barelycost effective compared to theBPAs 5.9 cents avoided-cost ceil-ing, while the 1989 program failsthe cost-effectiveness test by a sig-nificant marginC ontrary to the assertions ofsome of our critics, thesehigh measured costs and the largedifferences between measured

costs and engineering estimatescannot be attributed to programimmaturity, startup problems, ora failure to consider economiesthat result from learning by doing. BPAsprogram has been inexistence for a decade and hasbeen studied and refined more ex-tensively that any conservationprogram of which we are aware.Indeed, the experience with theBPA wealherization programsince 1980 makes it clear that onecannot assume that costs will nec-essarily fall over time. This is anempirical issue that can onlybe resolved by analyzing the relevantdata.

Table 2 reports the measuredcost per kWh saved based on exJXI&comparisons between partici-pants and control groups for eachof the six BPA program cohorts

studied since1980.The first col-umn presents information on themeasured cost per kWh saved as-suming that the savings me*ured in the first post-retrofit yearfor each program cohort persistover the life of the measures. Thecheapest savings were clearlyachieved in the earliest years. Pmgram costs in the post-1983 co-horts are significantly higher thanin the earliest cohorts and have in-creased steadily over time. If theBPA is getting better at deliveringconservation services it must begetting harder to find cost-effec-tive conservation opportunitiesover time; the low hanging fruit?has been picked and BPAmustclimb higher up the tree to fmdfruit worth picking. There are avariety of reasons why this mighthappen, including the diffusion ofconservation investments by con-sumers through ordinary marketforces and the increased availabil-ity of information about conserva-tion opportunities.

Unlike many other programs,the BPAtracks energy use by bothparticipants and control groups inthe second and third post-retrofityears. This makes it possible toexamine the persistence of pro

gram savings over time and totest the validity of cost estimatesthat assume that frrst year savingspersist over the entire life of themeasures. Table 3 displays thenet measured savings for eachprogram cohort by year.Focusingonlyon column 1,we see that first year meas-ured savings have declined overtime. In addition, annual pre-ret-refit consumption by both partici-pants and non-participants ismuch higher in the earlier periodthan in the post-1983 period?3The naturfl reduction in aver-age pm-retrofit consumption be-tween the early and later years isslightly larger, in fact, than the av-erage net savings achieved bythese programs during the 1980s..This suggests that the charac-teristics of the remaining tatgetpopulation for these weatheriza-tion retrofit programs havechanged over time. In particular,there are fewer conservation op-portunities in the remaining stockof retrofit opportunities. Thelow hanging fruit? may havebeen picked first and/or custom-ers have made conservation in-vestments on their own over timein response to market forces.

Table 2: BPA Residential Weathetization Program Results: All CohortsPrcgElmcohortPilot (1980-82)Interim (1982-83)1985198619881989

MeasuredCostperkWh Saved (1991 cents )4.432525.35.59.1

AdustedCost perkwhLed(l99lcents)5.54.06.56.66.911.4

22 The Electrkity Jomud


10/13

Table 3: MeasuredAnnualkWhSavingsInPost-RetrofitYearsProWamcohort Ye a r 1 Ye a r 2 Y e a r 3P i l o t ( 1 9 6 0 . 6 2 ) 3 , 8 4 0 3 , 7 9 0 3 , 4 1 0( - 1 1 2 %)l n t e r i m( 1 9 6 2 - 6 3 ) 4 , 2 0 0 3 , 6 0 0 2 , 6 0 0 ( - 4 0 . 5 %)1 9 6 5 2 , 6 1 0 2 , 5 6 5 2 , 6 0 0 ( - 0 . 4 %)1 9 6 6 3 , 0 6 0 2 , 1 1 2 2 , 1 4 0( - 3 0 . 1 %)1 9 6 6 2 , 1 6 0 2 , 0 0 0 ( - 6 . 3 %) N/ A1 9 6 9 1 , 3 3 0 N/ A N/ ANo k Nu mb e t s h p a t wh ma r e c h a n g e ~ f r o my e a r 1 s a v i n g s -s o u r c e :Br a r ma n d Wh i t e ( l 9 9 2 ) , T a # e 7 . 5 , ~ . 7 . 7 .

Table 3 also demonstrates that it program costs could lead theseis dangerous to base cost calcula- programs to fail traditional cost-tions on energy savings achieved effectiveness tests. In the secondin the first year, assuming that column of Table 1 and Table 2 wethese savings wiIl persist tluough- have adjusted the measured costsout the life of the investment. Ta- per kWh saved to reflect the as-ble 3 indicates that, on average, sumption that the present valuethe measured savings within each of energy savings is 20% lowercohort decline (i.e., deteriorate) by than Fvouldbe implied by the as-roughly 10% per year over time. sumption that the savings persist(The Oak Ridge study refers to a at first year levels forever This15% average annual rate of dete- adjustment is consistent with therioration that is computed in a dif- data reported in Table 3. Readersferent way) Data are unavailable can apply any other adjustmentto determine whether the savings that they feel is more reasonable.continue to decline after the third Tables 1 and 2 indicate that ad-year. justmentsinthemeasuredcoststoThe measured cost per kWh reflect the deterioration in meas-saved calculations we dis- ured savings over time rendercussed earlier were based on the none of the post-1983 programsassumption that energy savings cost effective, as the cost per kWhmeasured in the first year persist- saved exceeded BPAs 5.9 centsed over the full life cycle of the avoided-cost ceiling price.conservation measures. Since theBPA studies provide evidence that C. Implicationsthese savings deteriorate over It is not our purpose to pick ontime, the values for the cost per the BPApmgram. The programkWh saved reported in the first has been responsibly imple-column of Tables 1 and 2 are too mented in a region where electric-low. Given the unexpectedly high ity prices have often been too lowcosts of the post-1983 programs, to provide consumers with appm-an adjustment for the deteriora- priate incentives to conserve. Fur-tion of savings over time is impor- thermore, the application of moretant because modest increases in comprehensive benefit-costs tech-

Our point is simply that the re-sults from the BPApmgram, a ma-ture program that has followedthe cost-accounting and savingsmeasurement protocols that wehave suggested, are perfectly con-sistent with our own study EXpost measured savings are 30% to40% of ex ante engineering esti-mates. The measured cost perkWh saved is much higher thanwas indicated in the ex ante engi-neering studies used for planningpurposes. Appropriate cost ac-countingandsavingsmeasure-ment do matter because programsthat look highly cost effective totheplannermayturnouttobewasteful when all relevant costsand behavior are accounted for.V. Conclusion

The factors that we have dis-cussedinthispaper,andinourprevious research, have a numberof implications for the design ofutility conservation programs andfor the interpretation of reportedprogram costs.

A. Should Utilities PromoteEnergy Conservation?Some people have interpreted

our research as arguing that utili-~ u Z y Z 9 9 3 2 3

niques, rather than the simplecost-effectiveness tests that havebeen adopted to justify utility ex-penditures on conservation in thecentral planning world of inte-grated least cost planning, couldlead to a more favorable evalu-ation once the reasons for the sig-nificantdifferencesbetwWnengi-neering estimates of energysavings and ex post measured sav-ings arebetter understood.


11/13

ties have no role in promotingcost-effective energy conserva-tion. This represents a misinter-pretation of our analysis and ourconclusions. Utilities do have auseful role to play in promotingcost-effective energy conserva-tion. Our results indicate, how-ever, that if utilities are going tospend general ratepayer money tofinance energy conservation, theyare going to have to pull theirsocks up. In particular, they mustmake a more serious effort tomeasure and account for cwtomer and utility conservationcosts, to develop and apply cred-ible techniques to measure the en-ergy savings achieved, to accountfor changes in service quality toaccount properly for free riders,and to design their programs sothatmarketbarriersarereducedwith the smallest possible impacton overall rate levels. Further-

more, it is important that utilitiesmake rate design changes that bet-ter align prices and marginal costsso that electricity consumers getthe right price signals and thatutilities identify the most efficientmechanisms for getting their cus-tomers to invest in costeffectiveenergy conservation opportuni-ties.W edo see serious deficien-cies in the way many

utility programs have been con-ceptualized and structumd. Inparticular, the 1124courantframework of integrated least-costplanning in which consumerconservation investments areviewed as a utility resourcen thatutilities must acquire from theircustomers, is the source of a lot ofsloppy thinking (and wasteful ex-penditures) and is unbkely to leadto satisfactory outcomes in thelong run. Furthermore, as compe-

tition evolves in the electricpower industry, this approach-which leads to higher rates andpervasive cross-subsidies - willsimply be unsustainable.

Energy conservation should beconceptual&d as a customerserv-ice and a customer resource, not asa utility resource that is equivalentto a utility supply source. Thecustomer will own the conserva-tion devices, decide how to usethem, and decide when to scrapthem. Nobody has yet invented anegawatt metei to measure en-ergysavingsinawaythatisre-motely equivalent to a meter thatmeasures the kWh that come outof a power plant and into a cus-tomers home or business. Meas-urement problems may be a seri-ous market barrier, but thesebarriers can only be overcome byimproving energy savings meas-urement capabilities, at least in a

Wifh the proper price signals, utilities wuld better lead their customers to energy ejickcy.

24 The Electricity Journal


12/13

statistical sense- not by makingup numbers. The integratedleastcost planning frameworkmakes it too easy to forget that theconsumer is generally in the bestposition to forecast the use ofequipment placed in her home orbusiness and to evaluate the over-all benefits and costs of invest-ments in conservation when PIE-sented with the informationnecessary to make these evalu-ations. Placing the utility in theposition of deciding how millionsof customers should use electric-iv, and then using general rate-payer funds to subsidize consum-ers so that they behave in waysthat some engineering model saysthey should behave, reflects a cen-tral planning mentality that isdoomed to failure.A decentralized customerservice and customer re-source perspective naturally leadsone to focus a lot more on whyconsumers behave as they do andhow utility initiatives of differenttypes an2likely to affect consumerbehavior and ameliorate real rnar-ket imperfections. This framework recognizes that the pay-ments that consumers arerequired to make for energy con-servation investments made ontheir behalf play an important se-ledion role that makes it possibleto exploit the hidden informa-tion that customers have abouttheir energy use patterns and in-vestment plans. Customers whobear the costs of conservation willhave an incentive to agree to payonly for truly cost-effective invest-ments. Thus, the argumentsabout who should pay for utility

conservation programs raise notonly equityissues (cross-subsidies,non-participant burdens, etc.), butvery important t@ieruy issues.An approach to utility conserva-tion programs that requires cu.9tomers to pay the bulk of the costsof conservation investmentsmade on their beha& in one wayor another out of the savings thatthey realize or expect to realize,makes it necessary to convincecustomers that the savings are re-ally there when all relevant fac-

tors are taken into account. Thisapproach will lead to real energysavings rather than just paper sav-ings and will relieve regulators ofthe very difficult task of measur-ing actual savings, imputing etomer costs, dealing with free rid-ers, and changing customerbehavior over time. It will also re-@ utilities to think about theevolution of their conservationprograms into real businesseswhere the bill for conservationservices provided to Mrs. Smith issent to Mrs. Smith for paymentand not divided up and sent to allof her neighbors. In the end wewant least-cost outcomes, not nice

computer printouts produced byintegrated least-cost planning soft-Ware .

B. The Costs of UtilityConservation ProgramsNeither the criticisms that we

have received, nor mane recentstudies that we have reviewed,lead us to change any of the con-clusions that we reached in ouroriginal study The total costs ofenergy efficiency improvementsfacilitated by utility programs are,on average, significantly higherthan implied by well-known TPanalyses. Proper accounting ofprogram costs, accurate measun+ment of energy savings attribut-able to the programs, and properconsideration of the interaction be-tween these programs and nor-mal market processes would re-duce significantly the overallsocietal benefits attributed to util-ity conservation programs. As aresult, some programs that appear cost effective on paperwould be found to be wasteful inreality While we think that thereismorethanafreesnackouttherethat utilities can help to cap-,the free banquet with caviar andchampagne that the public isoften promised is not likely to beachievable with current practices. WE n d n o t e s :1. l? L. Joskow and D. B. Marron,What Doesa Negawatt Really Cost? Evi-akncejivm Utility Conservation Pm-grams, 13 ENERGYJ. v. 4, at 41-74(1992). The results are summarized inI? L. Joskow and D. B. Marron, WhatDoes Utility-Subsidized Energy ED-ciency Really Cost?, SCIENCE260, Apr.16,1993, at 281,370.

July1993 25


13/13

2.T. Flanigan and J. Weintraub, TheMost Successful DSM Programs in NorthAmerica, ALEC.J., May 1993, at 53-65.3. M. ONeill, DSM - Its a ContactSport, ELEC.J., May 1993, at 88-89.4. M. A. BROWNAND D. L. WHITE ,EVALUATIONOF B~NNJ MLLES1988 AND1989 RESIDENTIALWEATHERIZATIONPROGRAM:A NORTHWESTSTUDYOFPROGRAMDYNAMICS(Oak Ridge Nat1Lab, ORNL/CON-323, Dec. 1992).5. A. F! Fickett, C. W. Gell ings, and A.B. Lovins, Eficient Use of Electricity,SCI. AM. 263, Sept. 1990, at 64-74, andreferences therein.6. Obviously, another source of signifi-cant diff erences in program costs isthe diffenznce in the types of conserva-tion measures included in utility pro-grams and differences in theeffectiveness of the programs. Wefound that significant variation acrossutilities persists even for relatively ho-mogeneous sub-programs (e.g. light-ing or motors).7. The analysis of free riders can becomplicated, since proper calculationof the cost of saved energy must ad-just both energy savings and programcosts; see Joskow and Marron,Negawatt supra note 1, and an associ-ated working paper for details.8. NATL ACAD. OF SCL, POLICY IMHJCA-TIONSOF GREENHOUSEWARMING-SYN-THESISPANEL (1991).9. A. 8. Lovins, letter to Christine Gil-bert, Letters Editor, Science,May 2,1993.10. Flanigan and Weintraub, supranote 2.11.Using F&Ws definition of a pro-gram, we examined significantly moreU.S. investorawned utility programs,since, when we selected a utility, weexamined all of its programs or sub-programs for which data we= avail-able.12 . We found that several of the utili-ties with low cost programs in-cluded in F&Ws study also had highcost programs that were not so in-cluded.

13. Flanigan and Weintraub, supranote 2, at 54.14. Id. at 54.15. I? M. Miller, letter to Christine Gi l-bert, Letters Editor, Science, May 12,1993.16. ONeill, supra note 3.17. Brown and White, supra note 4.18. The Northwest Power PlanningCouncil, which prepares the powerand conservation plan under a publicleast-cost planning process requiredby federal law, is a multistate regionalplanning body which Congress agreedcould limit certain resource actions ofBPA. See 16 U.S.C. 9 839b.

19. NORTHWESTPOWERPLANNINGCOUNCIL,1991 NORTHWESTCONSERVA-TIONAND ELECTRI CPOWERPLAN15(vol. 1,1992).Th e number reported in the publica-tion is 3.2 cents per kWh saved, whichappears to be in 1990 dollars. Wehave adjusted this value and others re-ported below to reflect 1991 price lev-els (using the CPI) so that thenumbers are comparable with one an-other and with those reported in ourpaper.20. The number in Brown and White,supra note 4 at 2.11, is 5.6 cents perkWh saved, in 1990 dollars.

21. Assuming measurements are accu-rate, the difference between measuredand projected savings has two compo-nents: actual forecast error and conser-vation by members of the controlgroup. With a well-chosen controlgroup, the naturally occurring conser-vation in the control group should ex-actly offset the naturally occurringconservation that would have oc-curnzd in the treatment group (i.e.,free riders). The measured savingsthus accurately reflect the net aggre-gate effect of the program. To calcu-late costs per kWh, as reported in thetext, measured savings are the appro-priate denominator. The appropriatenumerator, however, is total costs lessthe costs that free riders would haveincurred themselves. Since the ORNLstudy does not estimate free riders, wehave been unable to make that adjust-ment here. These calculations wouldbe complicated if free drivers arealso responsible for conservation inthe control group. In general, we areskeptical of the fxee driver argu-ment, although it is obviously an em-pirical issue.22. Brown and White, supra note 4, at4.2 and 6.2.23. Id. at 7.5, Table 7.4.24. Consistent with note 21, thereader may wonder whether the dete-rioration in savings is due to reducedsavings from installed devices, in-creased conservation by the controlgroup, or other factors. The ORNLstudy does not directly address this is-sue, so we cannot say for sure. For the1988 program, we do have the foll ow-ing data:

Ml z% R KY

PaMpants 1773 1274coniml 3!339_Izzin@3 2177 2003 175Sowx BnrwnandWMe1@2,Table4.5,at4.11Note: Fromyear1 to yaar?, groaawings amongparkpantsdecreased,as didgrosssavingsamongthecontrolgroup.Inthiscasa, at least,themduc-tionin net savings results not from increased con-swation among the control,bulhornwbatantidiylowarsavingsbythepartidpan!s.

what does a negawatt cost

Documents