SOUP TO NUTS ON NON-ENERGY BENEFITS / NEISeedal2017.uci.edu/wp-content/uploads/Wednesday-07-Skumatz-smalle… · SOUP TO NUTS ON NON-ENERGY BENEFITS / NEIS Methods, ... Com’l labor,
Perspectives +-Basic measurementLI Res amp Comrsquol
Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics
Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning
Re-explore BCIntroduction in states growthExpanding literature
Lists
SERA
5
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther
bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther
bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety
bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives
Source (SkumatzSERA1996 on)
Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross
Non-OverlappingConsistent Units
SERA
KEY APPLICATIONS OF NEBS
MARKETING amp ROI ndashSell whatrsquos valuable to customers link to
peers
PROGRAM REFINEMENT ndashPositive amp Negative NEBs for
measures barriers incentives and targeting
TRAIN THE CHAIN ndashAlign Educate Actors on NEB
priorities
POLICY GOALS Quantifies Non-energy goals (eg
Low income jobs etc)
BC TESTS ndashRefined CE for program amp portfolio reduce bias in
investment
6
Source SERA all rights reserved
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
TOPICS Background Measurement of NEBs Risk NEBs in State BC Risk Gaps and Conclusions
SERA
BACKGROUND NEBS Attention growing on NEBs after 20 years BC to assess investments (programs portfolios)
Perspectives +-Basic measurementLI Res amp Comrsquol
Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics
Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning
Re-explore BCIntroduction in states growthExpanding literature
Lists
SERA
5
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther
bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther
bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety
bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives
Source (SkumatzSERA1996 on)
Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross
Non-OverlappingConsistent Units
SERA
KEY APPLICATIONS OF NEBS
MARKETING amp ROI ndashSell whatrsquos valuable to customers link to
peers
PROGRAM REFINEMENT ndashPositive amp Negative NEBs for
measures barriers incentives and targeting
TRAIN THE CHAIN ndashAlign Educate Actors on NEB
priorities
POLICY GOALS Quantifies Non-energy goals (eg
Low income jobs etc)
BC TESTS ndashRefined CE for program amp portfolio reduce bias in
investment
6
Source SERA all rights reserved
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
BACKGROUND NEBS Attention growing on NEBs after 20 years BC to assess investments (programs portfolios)
Perspectives +-Basic measurementLI Res amp Comrsquol
Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics
Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning
Re-explore BCIntroduction in states growthExpanding literature
Lists
SERA
5
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther
bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther
bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety
bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives
Source (SkumatzSERA1996 on)
Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross
Non-OverlappingConsistent Units
SERA
KEY APPLICATIONS OF NEBS
MARKETING amp ROI ndashSell whatrsquos valuable to customers link to
peers
PROGRAM REFINEMENT ndashPositive amp Negative NEBs for
measures barriers incentives and targeting
TRAIN THE CHAIN ndashAlign Educate Actors on NEB
priorities
POLICY GOALS Quantifies Non-energy goals (eg
Low income jobs etc)
BC TESTS ndashRefined CE for program amp portfolio reduce bias in
investment
6
Source SERA all rights reserved
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
20+ YEARS OF NEBS PROGRESShellip
But there still isnrsquot agreement on name - NEB OPI NNEB MB co-benefitshellipSource SERA all rights reserved
Perspectives +-Basic measurementLI Res amp Comrsquol
Explore BC (LIPPT)Expanding RampC Ests Initial mktg applics
Expanding estimates sectors studies methods Wider use in marketingInitial applications in planning
Re-explore BCIntroduction in states growthExpanding literature
Lists
SERA
5
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther
bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther
bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety
bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives
Source (SkumatzSERA1996 on)
Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross
Non-OverlappingConsistent Units
SERA
KEY APPLICATIONS OF NEBS
MARKETING amp ROI ndashSell whatrsquos valuable to customers link to
peers
PROGRAM REFINEMENT ndashPositive amp Negative NEBs for
measures barriers incentives and targeting
TRAIN THE CHAIN ndashAlign Educate Actors on NEB
priorities
POLICY GOALS Quantifies Non-energy goals (eg
Low income jobs etc)
BC TESTS ndashRefined CE for program amp portfolio reduce bias in
investment
6
Source SERA all rights reserved
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
5
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
Utility Society Participant (ResampICI)bullCarrying cost on arrearages bullBad debt written offbullShutoffs ReconnectsbullNotices calls collection costsbullEmergency gas service calls (for gas flex connector and other programs)bullInsurance savingsbullTransmission and distribution savings (usually distribution)bullFewer substations etcbullPower quality reliabilitybullReduced subsidy payments (low income)bullOther
bullEconomic development benefits ndash direct and indirect multipliersbullTax effectsbullEmissions environmental (trading values andor health hazard benefits)bullHealth and safety equipmentbullWater and waste water treatment or supply plantsbullFish wildlife mitigationbullNational securitybullHealth carebullOther
bullWater wastewater bill savingsbullOperating costs (non-energy) bullEquipment maintenancebullEquipment performance (push air better etc)bullEquipment lifetimebullShutoffs ReconnectsbullProperty value benefits sellingbull(Bill-related) calls to utilitybullComfortbullAesthetics appearancebullFires insurance damage (gas)bullLighting quality of light bullNoisebullSafety
bullControl over billbullUnderstanding knowledgebullldquoCarerdquo or ldquohardshiprdquo (low income)bullIndoor air qualitybullHealth lost days at work or schoolbullFewer movesbullDoing good for environmentbullSavings in other fuels or services (as relevant)bullGHG and environmental effectsbullNegatives
Source (SkumatzSERA1996 on)
Net ThreeNet Positive amp negativeNet beyond standard efficiencyNet to gross
Non-OverlappingConsistent Units
SERA
KEY APPLICATIONS OF NEBS
MARKETING amp ROI ndashSell whatrsquos valuable to customers link to
peers
PROGRAM REFINEMENT ndashPositive amp Negative NEBs for
measures barriers incentives and targeting
TRAIN THE CHAIN ndashAlign Educate Actors on NEB
priorities
POLICY GOALS Quantifies Non-energy goals (eg
Low income jobs etc)
BC TESTS ndashRefined CE for program amp portfolio reduce bias in
investment
6
Source SERA all rights reserved
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
KEY APPLICATIONS OF NEBS
MARKETING amp ROI ndashSell whatrsquos valuable to customers link to
peers
PROGRAM REFINEMENT ndashPositive amp Negative NEBs for
measures barriers incentives and targeting
TRAIN THE CHAIN ndashAlign Educate Actors on NEB
priorities
POLICY GOALS Quantifies Non-energy goals (eg
Low income jobs etc)
BC TESTS ndashRefined CE for program amp portfolio reduce bias in
investment
6
Source SERA all rights reserved
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
NegativeNEBs
Solar WH
Appearance -$14 NZ
Maintenance -$9 NZ
Source Skumatz Economic Research Associates research ndash 1998-2005
-005
0
005
01
015
02
Maint
Productiv Perf Life
Op CostTenant S
atComfor
t
Lite Safe SellEnvir
oOther
AampE Owner
Commercial Example
Residential Example
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
NEB MEASUREMENT
8
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
Methods discussion Tradeoffs Multiple methods triangulation Surveys most appropriate for some Balancing precision practical ndash avoid bias stats large ldquoNrdquo Multiple survey approaches ndash story of a ferry Accuracy level neededhellip false comparisonshellip
9
Monetized NEBs
Source Skumatz SERA research
NEB
s
Direct
Secondary
Model
Survey Story of a ferryhellip then it is academic
Corp Records Utility data
Change x valueFinancial Calcs
Third party jobsAnd emissions
Specialized academic Best for some NEBs
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
MEASURING PARTICIPANTNEBs Best practices
WTP not fruitful 7+ better options
Net Three Net positive amp negative Net beyond standard
efficiency Net to Gross
Non-overlapping Consistent units Recommend in ALL
process (impact) surveys - barriers
HTM=Hard to measure HTA=Hard to answer
Direct calc Accurate Small ldquonrdquo
WTPWTA Accepted Volatile HTA
Relative Fast strong HTA
LMSFast
strong clear ETA
Unfamiliar
Logit Conjoint
rankStrong Slow
complex
Regression Defensible Limited data cost
Market value Strong Data
Other Exploring Cost tradeoffs
Source SERA Research
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
11
Monetized NEBs
Direct Secondary Model Survey
Attributable Change (study)
Value or Financial
Calc
X
Total Attrib Stated Relative
Effect
Savings (or translation)(ldquoNormrdquo)
Individual NEB Shares
X
X
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
SERA
PARTICIPANT MEASUREMENT METHODS COMPARISON
005
115
225
335
445
MW
x
NE
Wx
CA
Wx
CA
Wx
SF-N
C
MF-
NC
C-
Lite
C-R
eb
C-R
eb
multLMS VerbalWTP
Source SERA Research
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
Chart1
mult
LMS Verbal
WTP
122
156
42
08
1
2
098
4
06
13
12
075
067
104
115
27
075
055
32
084
091
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
MWx
MWx
MWx
NEWx
NEWx
NEWx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
CA Wx
SF-NC
SF-NC
SF-NC
MF-NC
MF-NC
MF-NC
C- Lite
C- Lite
C- Lite
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
C-Reb
Sheet1
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
MWx
NEWx
CA Wx
CA Wx
SF-NC
MF-NC
C- Lite
C-Reb
C-Reb
mult
122
08
13
075
104
075
084
LMS Verbal
156
1
098
06
12
067
115
055
091
WTP
42
2
4
27
32
Sheet1
mult
LMS Verbal
WTP
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
Sheet2
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
Sheet3
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
SERA
PROS AND CONS OF NEBS MEASUREMENT METHODS
13
Source Skumatz SERA research
Used for Major Advantages Major Disadvantages Direct measurement (specific studies of changes on-site) Comrsquol labor productivity etc
Direct precise attributable Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature) Insurance water health others
Long history easy secondary sources Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case) Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately) Wide variety of Participant NEBs
Large sample sizes amp statistical properties Affordable Multiple estimates leading to similar ranges Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability Recall from survey respondents Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Used for
Major Advantages
Major Disadvantages
Direct measurement (specific studies of changes on-site)
Comrsquol labor productivity etc
Direct precise attributable
Small estimation sample sizes specialized cases poor transferability expensive
Secondary measurement (attributable change in incidence times marginal valuation from secondary literature)
Insurance water health others
Long history easy secondary sources
Credible to reviewers vetted inputs
Not available for all NEBs
Models (third party vetted models of attributable impacts based on local program inputs on base amp test case)
Emissions economics
Third party peer-vetted models available for economics jobs and emissions
Not available for all NEBs
Surveys (multiple academic-based approaches for surveys of participant effects valued appropriately)
Wide variety of Participant NEBs
Large sample sizes amp statistical properties
Affordable
Multiple estimates leading to similar ranges
Direct method of measuring some key NEBs
Concerns about surveys as a source of quantitative values amp reliability
Recall from survey respondents
Proper attribution to programs measures
SERA
PROGRESS TRANSFERABILITY OF NEBS
14
Source Skumatz SERA research
Variability Relevant NEB Categories Program measure invariant (suitable for ldquoadderrdquo)
bull Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
bull Economic ndash societal (depends on measures and local manufacture installation) bull Health and safety health care illnesses ndash societal and participant (measure) bull Water wastewater infrastructure and water bill savings ndash societal and participant bull Participant benefits including equipment operations lifetime OampM comfort noise control
education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent bull Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
bull Payment related ndash utility (arrearages etc stronger for low income targets) bull Health and safety health care illnesses ndash societal and participant (higher with chronically ill
vulnerable populations) bull Participant benefits related to hardship and payments bull Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
Cost savings consistency if transferable vs risk
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Variability
Relevant NEB Categories
Program measure invariant (suitable for ldquoadderrdquo)
middot Environmental emissions ndash links to energy savings (varies with generation mix and local air conditions and time of day but not primarily with measures program)
Program measure dependent
middot Economic ndash societal (depends on measures and local manufacture installation)
middot Health and safety health care illnesses ndash societal and participant (measure)
middot Water wastewater infrastructure and water bill savings ndash societal and participant
middot Participant benefits including equipment operations lifetime OampM comfort noise control education home-improvements Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Climate dependent
middot Participant benefits including comfort but when expressed as percent of energy savings this variability may be mitigated Note if measure bundles are ldquosimilarrdquo participant NEB multipliers are similar in different areas of country
Residential Target dependent (low income or MF vs SF)
middot Payment related ndash utility (arrearages etc stronger for low income targets)
middot Health and safety health care illnesses ndash societal and participant (higher with chronically ill vulnerable populations)
middot Participant benefits related to hardship and payments
middot Initial information indicates non-low-income NEBs for occupant MFs are similar to SF
SERA
RANGES TYPICAL NEB VALUES FOR WxRETROFI
15
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
SERA
PROGRESS AND GAPS Consistency Transferability
Some programs types others less studied Transferability research
Measure-based NEBs Some measure-based estimates For multi-measure programs ndash need to sample for
measures until then Across the board Savings share Regression
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
SERA
PROGRESS AND GAPS Gas vs Electric
Some researchndashusually program-wide not broken down International research found participant NEBs have
similar order of magnitude multipliers Not much research on fuel patterns ndash a gap thin
MF Less-commonly-studied complicated by poor response
and complexity of sector (decision-maker some measures in home some central) separate from low income not common
Study provides some indicative results on occupants vs owners (112 vs 71) some comparisons to SF this is a gap
Commercial gaps in some measures
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
SERA
USING NEB NEI VALUES Improve Cost-effectiveness (CE) of
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Chart3
023
016
014
01
008
006
006
006
005
005
003
0
Chart1
Vendor
NP
Partic
028
02
02
028
013
017
031
022
016
006
008
014
005
006
014
004
009
007
004
003
007
-006
018
006
Sheet1
Sheet1
Sheet2
Sheet3
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Environment
Quality
Quantity
PgmAvail
Flicker
Maintenance
Lifetime
SalesProductivity
Safety
Noise
Control
Illness
Chart1
Vendor
NP
Partic
028
02
02
028
013
017
031
022
016
006
008
014
005
006
014
004
009
007
004
003
007
-006
018
006
Sheet1
Sheet1
Sheet2
Sheet3
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Perform
Perform
Perform
Footprint
Footprint
Footprint
ControlFeatures
ControlFeatures
ControlFeatures
Tenant compl
Tenant compl
Tenant compl
Noise
Noise
Noise
Maint
Maint
Maint
Other
Other
Other
Lifetime
Lifetime
Lifetime
Sheet1
Sheet1
Sheet2
Sheet3
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Boiler
Vendor
NP
Partic
Perform
028
02
02
Footprint
028
013
017
ControlFeatures
031
022
016
Tenant compl
006
008
014
Noise
005
006
014
Maint
004
009
007
Other
004
003
007
Lifetime
-006
018
006
MF New construction energy star
Operating cost
014
Environment
012
Maintenance
01
Lifetime
01
Performance
009
Occupant Satisf
009
Comfort
008
Noise
006
Sellingleasing
006
Aesthetics
005
Light Quality
005
Power QualReliab
004
Safety
002
RIGHTLIGHTS
Environment
023
Quality
016
Quantity
014
PgmAvail
01
Flicker
008
Maintenance
006
Lifetime
006
SalesProductivity
006
Safety
005
Noise
005
Control
003
Illness
0
Sheet1
Sheet2
Sheet3
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Sheet2
Sheet3
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Sheet3
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
NEB VALUES
19
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
ARE NEBS HIGH VALUE Energy savings are often the minority of
benefits from program measuresndash Donrsquot ignore that
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo New construction (incl ES) Lighting Weatherization Audit Home Performance (ampES) Appliances Water heating Insulation Window coverings Eqpt rebate Training outreach Real time pricing Solar renewables
MF Many others
Thousands of surveys results By measures By program types By sectors By stakeholders By geography
Variety of end uses
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
TOP NEBS FOR WEATHERIZATION PROGRAM
Weatherization program multipliergt1
Source SERA Research
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
RESULTS FOR ENERGY STARreg APPLIANCES
0002004006008
01012014016018
Maint
Appear
Perf LifeNoise Satis
Comfort
LiteSafe
ty SellMove
Water
Enviro
Refrig DW CW
Source SERA Research
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
SERA
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
5 increments to 50
Source SERA research
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
Chart5
EnergySmart (insul Retrofit)
Dunedin (insul)
001
024
002
004
002
005
006
005
007
008
008
006
013
0
013
016
049
033
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
Other
Other
Noise
Noise
Budgetfinances
Budgetfinances
Environmental
Environmental
Health
Health
Reduced moves
Reduced moves
Ease of sellingrenting
Ease of sellingrenting
DampnessMold
DampnessMold
Comfort-related
Comfort-related
cx
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Operational NEBs
44
DesignConstruction NEBs
31
Occupant NEBs
25
1
cx
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
nz
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Sheet3
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Share of ZALEH NEBS from various measures
Each presented as multiple of the (overall) energy savings
Super Insulation
037
Double Glazing
026
Solar Design
025
Solar Water Heating
005
EnergySmart (insul Retrofit)
Dunedin (insul)
Other
1
24
Noise
2
4
Budgetfinances
2
5
Environmental
6
5
Health
7
8
Reduced moves
8
6
Ease of sellingrenting
13
0
DampnessMold
13
16
Comfort-related
49
33
101
101
Sheet3
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Super Insulation
Double Glazing
Solar Design
Solar Water Heating
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
EnergySmart (insul Retrofit)
Dunedin (insul)
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
SERA
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
0
002
004
006
008
01
012
Maint
Perf LifeSatis
f
ComfortAesth
NoiseSafe
ty
LightingSelli
ng
Stay in home
Enviro
Sick Days
Calls
ES Homes HP wES Energy savings multiplier=1
Source SERA Research
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
SERA
CampI PROGRAMS ldquoNEBBEDrdquo New construction Lighting Motors Audit Eqpt rebate Commissioning Technical assistance Training outreach PV Retail renewable SPC
DG CHP HVAC Equipment rebate Other
Building codes incentives by cities
Thousands of surveys results By measures By program types By many sectors By stakeholders By geography
Variety of end uses
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
SERA
AUDIT FINANCIAL PROGRAM
0
01
02
03
04
05
06
07
Variety of measureshellip disaggregatedResults by measure type businessType etchellip
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
Besides enviroCosts performance Light comfort Productivity impt ndashVarled by measurehellip
vcv
Source SERA Research
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Chart1
CampI Technical Assistance NEBs
002
003
004
005
005
006
006
007
007
007
009
009
01
011
011
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
m Product losses
l Ease of selling leasing building
n Labor requirements
h Aesthetics appearance
j Noise
b Equipment maintenance
f Tenant satisfaction
d Equipment lifetimes
e Productivity
k Building safety[2]
g Comfort
i Lighting Quality of Light
c Equipment performance[1]
a Operating costs
o Doing good for the Environment[3]
Sheet1
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Percent of NEBs by category per participating building
Percent Ranking this NEB in the Top Three
m Product losses
2
0
l Ease of selling leasing building
3
3
n Labor requirements
4
5
h Aesthetics appearance
5
9
j Noise
5
0
b Equipment maintenance
6
60
f Tenant satisfaction
6
0
d Equipment lifetimes
7
18
e Productivity
7
24
k Building safety[2]
7
3
g Comfort
9
33
i Lighting Quality of Light
9
16
c Equipment performance[1]
10
19
a Operating costs
11
65
o Doing good for the Environment[3]
11
32
Other
0
3
[1] Examples of equipment performance NEBs include improved ability of the equipment to circulate air through the building effectively etc
[2] Examples of ldquosafetyrdquo NEBs include newer equipment having lower potential for safety problems etc
[3] Rather than actual improvements to the environment this represents the value to the respondent of having taken actions they perceive will help the environment
Sheet1
CampI Technical Assistance Program NEBs (Participant)
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Sheet2
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Sheet3
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
EXPRESSING NEBS VALUEndashCxe
Strong value from RetroCxSource SERA Research
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
SOCIETAL NEB EXAMPLES
31
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE Economics Emissions Hardship
32
0
02
04
06
08
1
12
HPWxRetrof Appliance
CA
WI
Natl
(Source Skumatz SERAECEEE 2007 ACEEE 2006)
Jobs Economic
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
EMISSIONS (SOCIETAL NEBS) Simple to complex models (slippery slope) Baseload vs peak Some elements well already accepted Incorporation as adder
33
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
UTILITY NEBSUtility NEBs for Template Program
Debt WriteOff (util)13
Rate Subsidy(util)61
HealthSafety(util)0
Colln Costs (util)0
Gas Calls (util)0
Calls to CSRs(util)2
TampD (util)16
Arrears (util)0
Reconnects (util)0
Notices (util)7
Shutoffs (util)1
Rate subsidy TampD
Payment-related
Source Skumatz Economic Research Associates research
Example Low Income Weatherization
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
ADJUSTED PAYBACKS These NEBs are DOLLAR VALUES so can
be used to adjust payback and ROI Gross payback 56 yrs 25 BC incl all partic NEBs 09 19
Total participant NEB multiplier ranges ~25-300+ depending on measure(s) target(s) climate etc
Value of 100 HALVES Payback doubles ROI
Source Skumatz Economics (SERA)
35
( for NEBs Multiplier of 125)
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
NEBS IN COST-EFFECTIVENESS APPLICATIONS
36
hellip2001 ON
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
37
NEBS IN BC TRC Societal Participant UCT RIMhellip NEBs
For true representation of B amp C elements of NEBs address missing factor bias
Better guide measure program amp portfolio investment Address by 1) incl monetized NEBs or 2) exclude all costs associated with achieving NEBs or 3) use UCT
BC applications were considered early on then pulled back awaiting more quantitative evidence
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
38
Test Benefits Costs States Using Traditionally
Improved treat-ment with NEBs
Utility Cost (or Program Administrator Test) (UCT or PAC)
bull Avoided supply costs for transmission distribution and generation (TDampG)
bull Avoided gas and water supply costs
bull Program administration
bull Participant incentives
bull Increased supply cost
CA CT HI IA IL IN MI MN MO NY OR RI TX VA WA BPA
Use cost only paid by the utility
Ratepayer Impact Measure (RIM) (or No Loserrsquos Test or non-participants test)
Same as above plus bull increased revenue
Same as above plus bull Decreased revenue
AR CO FL GA HI IA IN MI MN NC ND NV SC VA WI
Participant cost bull Utility bill reductions bull Participant incentives
bull Participant direct costs
AR CA FL HI IA IN MI MN NY VA
Participant NEBs
Information readily available on NEB refinements needed to all traditional types of BC tests in use ndashldquoCheck-marksrdquo for which NEBs need to be added To each test has been available since 2002-2005hellip
Source copySkumatz Economic Research Associates (SERA) Rights reserved may be used with permission of author
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
APPLICATION TO BC Initial hesitancy - 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
41
STATE TREATMENT OF NEBS NEBsRegulatory
Screening Application Utilities regions
Program Marketing Fairly widespread use in utilities states across the country
MO
RE AGGRESSIVE ==gt
Test Pgm Screen - adder
IA (10 elec 75 gas 1999) CO (10 adder 25 Low Inc 2008) OR (Carbon $15ton 10 adder 2008) WA (10 adder 2008) VT (15+15 LI) DC (10) NY($15 adder for carbon) NW (15) for low income (LI) or lt1 (CA ID OR WA UT WY NH NY CT)
Test Pgm Screen - readily measured
MA (NEBs must be reliable amp with real economic value utility prop HampS comfort LI eqpt util all costs of complying with foreseeable environmental regulations) CA (low income) VT (maint eqpt replacement LI comfort HampS prop util societal) CO (measureable with current mkt values) NH (as adder LI) BCHydro (maint GHG lifetime product loss productivity floorspace) DC (eqpt comfort HampS prop societal) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) CT (LI) RI (LI quantify util societal HampS eqpt prop comfort) NY (LI eqpt)
Test Hybrid (potential adder amp measured)
CO (measureable with current mkt values) OR (esp CampI carbon value on societal test PV deferred plant extension water sewer savings laundry soap) DC VT
Test Pgm screen - Broad
With quantification MA RI MA order decision - becoming broader - count in res amp ICI demonstratable including survey-based (not yet econ) Broad-based inclusions of all NEBs as an official screen not yet found
Source copySkumatz Economic Research Associates (SERA) may be used with permission of author
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
42
Key
States with NEBs in CE tests for At least one type of program
copy Skumatz SERA Superior CO 2014 all rights reserved may be used with permission of author
STATES WITH NEBS IN CE TESTS
NEB options to datebull Addersbull ldquoReadily Measuredrdquobull Hybridbull ldquoAll-Inrdquo Various stages of deliberations working
groups TRM work etc in states inMidwest mid-Atlantic and elsewhere
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
SELECTED STATE PROGRESS CA
2001 ndash LIPPT model NY
Mid-2000s measurement amp scenarios not included CO
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
SELECTED STATE PROGRESS Dominos ongoing
Midwest ndash Some NEBs in tests intervenor raised TRM process discussions stalled
Mid-Atlantic ndash Considered as part of broader regulatory change ldquoinformational proceedingsrdquo BC expected in next stage
Midwest ndash Regulatory commission decided to conduct revisions of BC rules considered NEB process reversed that section of rules pick up again next year
Sellablerdquo name ndash ldquoPrince Albert in a Canrdquo if thatrsquos what they needhellip
44
Source Skumatz SERA research
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back) Initial hesitancy ndash 2001rsquos LIPPT Lack of literature unfamiliar methods Expansion of research literature values
Early adopters Adders Readily measureable Hybrid ldquoAll inrdquo
Expansion Concern with risk calculation accuracy
45
Source Skumatz SERA research
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
46
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS) Simplified BC Inputs- Lets compare the risksranges
[PV[NTG(Sav+NET NEB)Lifetime]PV(Cost)hellip]
SourceSkumatz SERA Independent research
NTG ndash accuracy measurement incomplete
Savings Impact repeatedly amp expensivelymeasured little variation $100K+
NEBs Lit exists comparability transferability local inexpensive to add to existing studies gaps
EUL Lists 20+ years old Origins () technologies dated varies local values 50 - 2x+ variation
Cost Complicated expensive local changes
RELATIVE Risk from NEBs
Discount rate Not highly complicated purpose use ltWACC risk link regulatory environment
Conclusion ldquoMeasuring (savings) with a micrometer cutting with a chainsawrdquo
NEBs as accurate as many inputs (all could improve)
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
DISCOUNT RATE MATTERS
47
Source Skumatz SERA Research
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10
PV Ratios vs 1 Discount Rate
10 yr PV 20 yr PV
Discount rate - percent
WACCSocialLTT
Source Skumatz Electricity Journal IEPEC paper independent research
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
EUL VARIATIONS-ADOPTED VALUES
Source Skumatz Economic Research Associates SERA independent research
0
50
100
150
200
250
300
350
400
450
0 10 20 30 40 50 60 70
EUL Variation by Measure
67 measures examined66 average40 medianYrs in double digits avg 64 med 5Substantial percentages
Poorly supportedOldTechnology change
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE Tradeoffs ndash How much to improve tests
Depends on costs amp benefits of accuracy improvements (in NEB categories)1 Which NEBs most valuable2 What value range arises from reasonable cost
measurement (eval budget)3 Does inclusion of this RANGE (low vs high value)
change the BC conclusion
On the cusphellip
If NO Yoursquore doneAnd bias addressedsufficiently
IF YESRefine measurement up to value or cost of ldquowrongrdquo decision
Source SERA all rights reserved
SERArsquos lsquoNEB-ItrdquoModel ndash
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
Utility Soc Part Conserv Recrsquom
Rationale
Base Percent X X X Program-invariant
Low Income X X X X Multiple sources
Weatherization X X X Substantial Participantimpacts
Measure Program-specific
X Varies by measure sector
Other Recomrsquos Local Research
50
Source Skumatz SERA
Developing values for multiple states amp utilities
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
Rigor surveys risk put into perspective BC=f[pvNTG(Savings+Net NEB)EULPV(Costhellip)]
Other sources of bias to research (EUL NTG etc) bring high variation
5) States are incorporating NEBs dominos input in deliberations in multiple states
6) and useful besideshellip Source Skumatz SERA researchMay be used with permission of author
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
THANK YOU
Questions
Lisa A Skumatz PhDSkumatz Economic Research Associates (SERA) Phone 303494-1178skumatzserainccom
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
POWER OF NEBS ndashBARRIERS DISCONNECTS
53
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
Negative NEB values cost of barrier
Solar Water Heat NZ$ Euros
Solar Design NZ$ Euros
Appearance (NZ$ Euros) -14 -7 - 3 -2 Maintenance (NZ$ Euros) -9 -5 - 5 -3 Other (NZ$ Euros) - - 3 -2 Total value of Negative NEBs for Measure (and share of energy savings)
-23 -12 (079)
-11 -6 (06)
Implications Negatives barriers Can be very real amp importantCan address with redesign orpresumably rebates Perhaps warrantieshellip
Source Skumatz Economic Research Associates research
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
SERA
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
NEBs Influence on Incorporating HPEE Direction amp Importance by Actor
-5-4-3-2-1012345
Op Co
st
Init C
ost
Maint
Perfo
rm
Prod
uctiv
Tena
nt Sa
t
Comf
ort
Appe
ar
Light
Qual
SellL
ease
Devp Ownr AampE
Note AgtE (~4 vs ~2) for these NEBs
Uses ID ldquowinnersrdquo influencing factors for intervention ldquodisconnectsrdquo program interventions research assess design training target needed actorshellip
Source Skumatz Economic Research Associates Inc 2002
Owners more positive ndashUnderinvestment train
Soup to nuts on non-energy benefits NEIs
TOPICS
BACKGROUND NEBS
20+ YEARS OF NEBS PROGRESShellip
NEB PERSPECTIVES CATEGORIES BEST PRACTICES
KEY APPLICATIONS OF NEBS
USES OF NEBS - NEGATIVES PERCEIVED COST OF BARRIERS
Neb measurement
NEBs MEASUREMENT ndash 4 MAIN MEASUREMENT APPROACHES
MEASURING PARTICIPANTNEBs
HOW THE NEBs ARE MONETIZED (CANrsquoT USE ldquoFEEL GOODrdquo IN A BC or ROI CALCULATION)
PARTICIPANT MEASUREMENT METHODS COMPARISON
PROS AND CONS OF NEBS MEASUREMENT METHODS
PROGRESS TRANSFERABILITY OF NEBS
RANGES TYPICAL NEB VALUES FOR WxRETROFI
PROGRESS AND GAPS
PROGRESS AND GAPS
USING NEB NEI VALUES
Slide Number 19
ARE NEBS HIGH VALUE
RESIDENTIAL PROGRAMS ldquoNEBBEDrdquo
TOP NEBS FOR WEATHERIZATION PROGRAM
RESULTS FOR ENERGY STARreg APPLIANCES
INSULATION RESULTS (DUNEDIN amp ENERGY SMART)
ENERGY STAR HOMES amp HP PROGRAMS - RESIDENTS
CampI PROGRAMS ldquoNEBBEDrdquo
AUDIT FINANCIAL PROGRAM
CampI NEW CONSTRUCTION
CampI TECHNICAL ASSISTANCE
EXPRESSING NEBS VALUEndashCxe
Slide Number 31
JOBS ECONOMICS ndash ALL PROGRAMS ARENrsquoT ALIKE
EMISSIONS (SOCIETAL NEBS)
UTILITY NEBS
ADJUSTED PAYBACKS
NEBS IN COST-EFFECTIVENESS APPLICATIONS
NEBS IN BC
CORRECTING amp IMPROVING CE TESTS ndash 2001 onhellip
APPLICATION TO BC
METHODS TO INCLUDE NEBs IN REGULATORY TESTS
STATE TREATMENT OF NEBS NEBs
Slide Number 42
SELECTED STATE PROGRESS
SELECTED STATE PROGRESS
APPLICATION TO BC AS ldquoNEXT STEPrdquo (circle back)
NEBS IN BC ndash NEB RISK IN CONTEXT (ACCURACY BIAS)
DISCOUNT RATE MATTERS
EUL VARIATIONS- ADOPTED VALUES
KEY QUESTIONS FOR IMPROVING TESTS - BALANCE
IMPLICATIONS FOR JUSTIFIABLE NEBs VALUES ndash SERA FORMAT FOR STATES
TAKEAWAYSON NEBSNEIS
THANK YOUQuestions
Slide Number 53
NEGATIVE NEBS VALUE PERCEIVED COST OF BARRIERS
CampI - NEGATIVE NEBS (BARRIERS) amp ACTOR ndash ldquoDISCONNECTSrdquo
