seem 94 calibration to single family rbsa data analysis and proposed actions
DESCRIPTION
SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions. RTF SEEM Calibration Subcommittee May 7, 2013. Goals for today’s Subcommittee Meeting. Review the following presentation in detail Consensus on next steps Are there any needed changes to the analysis? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/1.jpg)
SEEM 94 Calibration to Single Family RBSA Data
Analysis and proposed actions
RTF SEEM Calibration SubcommitteeMay 7, 2013
![Page 2: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/2.jpg)
2
Goals for today’s Subcommittee Meeting
• Review the following presentation in detail– Consensus on next steps• Are there any needed changes to the analysis?• Is there subcommittee consensus that the RTF should
make a decision stating SEEM94 is calibrated? – Receive suggestions from the subcommittee for
improvements in the presentation• Does it adequately tell the full story?• Is it the appropriate tool present to the RTF (assuming
previously covered sections will be skimmed over)?
![Page 3: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/3.jpg)
SEEM 94 Calibration to Single Family RBSA Data
Analysis and proposed actions
Regional Technical ForumMay 21, 2013
![Page 4: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/4.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 4
![Page 5: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/5.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 5
![Page 6: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/6.jpg)
Purpose of Calibration
6
![Page 7: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/7.jpg)
Background - Purpose 7
Align SEEM with Measured Energy Use• The SEEM model is used to estimate energy savings for
most space-heating-affected residential UES measures using the “calibrated engineering” estimation procedure (see section 2.3.3 of guidelines)– Heat Pumps and Central AC (ASHP, GSHP, DHP)– Weatherization– New Homes– Duct Sealing– Space Conditioning Interaction Factor
• Goal: Ensure SEEM94’s results are grounded in measured space heating energy use of single family homes. Use RBSA as source of measured data.
![Page 8: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/8.jpg)
8
RTF Savings Guidelines2.3.3.2. Model Calibration
In most cases, calibrated engineering procedures will involve at least one stage of modeling in which baseline and efficient case energy consumption are estimated for the measure-affected end use. For example, the heating load for single-family homes is estimated as part of the derivation of UES for ductless heat pump conversion. A simulation model is used to derive the heating end use for typical homes in different climate zones. Ideally, the model would be calibrated to measured heating end use for a sample of homes. If end use data are not available, the model should at least be calibrated to metered total use for the sample. Calibration should also be performed for samples that have adopted the measure, i.e., the efficient case. For measures that affect new buildings the calibration may be limited to the efficient case or to comparable buildings of recent vintage.
Background - Purpose
![Page 9: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/9.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 9
![Page 10: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/10.jpg)
10
RTF Decision History
Background - History
Date RTF Decision Summary Housing Type T-stat Results Data Sources Used in Calibration
Nov-2009 SEEM 92 model is calibrated. Single Family
HP & Gas FAF70°F Day ; 64°F NightElectric FAF and Zonal
66°F Day & Night
1. Res New Const. Billing Analysis (RLW 2007) 2. SGC Metered Data 3. NEEA Heat Pump Study (2005) Note: Very limited representation of Zones 2 & 3
Apr-2011SEEM 93 model is
calibrated. (implicit decision)
Single Family with GSHP 70°F Day ; 64°F Night 1. Missoula GSHP Study (1996)
Dec-2011 Use updated SEEM94 model
Single Family,Manufactured
Homen/a
Ecotope updated SEEM code to model the physics of the house infiltration, rather than rely on a constant stipulated infiltration rate input in previous versions of SEEM.
Dec-2011 SEEM 94 model is calibrated
Manufactured Home
69.4°F Day61.6°F Night
1. NEEM 2006 2. NEEA Heat Pump Study (2005) 3. MAP 1995 4. RCDP (manufactured homes)
Sep-2012 SEEM 94 model is calibrated Multifamily
Walk-up and Corridor68°F Day& Night
Townhouses66°F Day & Night
1. Multifamily MCS (SBW 1994) 2. MF Wx Impact Evaluation for PSE (SBW 2011) 3. New Multifamly Building Analysis (Ecotope 2009) 4. ARRA Verification for King County (Ecotope 2010)
![Page 11: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/11.jpg)
Background - History 11
RTF Decision History (Continued)For “model is calibrated” decisions…Calibration Methodology:1. Use available house and operation characteristics data from
billing/metering studies to develop inputs to SEEM runs;2. Adjust SEEM thermostat setting input to achieve a good
match (on average) between SEEM output (annual heating energy use) and billing/metering study results.
Note: The data sources used were free of (or mostly free of) supplemental fuel usage (wood, propane, oil, etc.) • Collection of reliable electric and gas usage data for space
heat consumption is relatively easy compared to other fuels.
![Page 12: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/12.jpg)
Date Forum Topic Outcome Links
1/23/13 Full RTF
Proposal to adopt calibration:
Send staff back to assess calibration needs related to
climate and measure parameters; and
engage subcommittee.
PresentationMinutes
3/20/13 Sub-committee Status update and check in. Presentation
Minutes
5/7/13 Sub-committee
Review staff’s proposal in detail. Decide whether to recommend RTF adoption.
Today PresentationMinutes
SF Calibration to RBSA - Recent History
Heating System Type
HeatingHigh °F(day)
HeatingLow °F(night)
Electric Zonal64 64
Electric FAF
Gas FAF 68.6 63.9
Heat Pump 69.6 65.4
History - 12
![Page 13: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/13.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 13
![Page 14: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/14.jpg)
Methodology – Overview
14
![Page 15: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/15.jpg)
15
Two Sources of Heating Energy Estimates
RBSA-PRISM. Estimates of annual “space heating use” for each house determine by using PRISM
– PRISM is a “change-point” regression model that uses billing data to estimate temperature-sensitive use
– PRISM analysis based on monthly billing data (at least 2years)
SEEM. Estimated annual space heating energy use for each house based on SEEM engineering model
– RBSA individual home characteristics (e.g., thermal envelop, heating system type, duct tightness) used as model inputs;
– Initial model runs use thermostat set to 68°F day & night• SEEM is a one-zone model, so t-stat setting input represents the average
setting for the entire house• Actual t-stat settings are not well documented (occupant reported settings are
unreliable, especially for “zonal systems”)• Thermostat setting will be used (step 2, below) as the “calibration knob”
Methodology - Overview
![Page 16: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/16.jpg)
16
Step 1 (Regression)
Use regression techniques to identify building characteristics that drive systematic differences between SEEM(68°F) and PRISM space heating energy use estimates.
Methodology - Overview
![Page 17: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/17.jpg)
17
Step 2 (Calibration)Use regression results to determine thermostat set-point that will align (i.e., “calibrate”) SEEM with PRISM annual space heating use– Calibration based on comparing average of all SEEM (68)
annual estimates to average of all PRISM annual estimates– Calibration is based on building characteristics identified in
regression.– SEEM run for each house at varying “day-time” thermostat
settings, with “night-time” thermostat settings based on occupant reported setbacks
Methodology - Overview
![Page 18: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/18.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 18
![Page 19: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/19.jpg)
19
Data Sources
Data Source used in this calibration:Underlying database* for the Single Family Residential Building Stock Assessment (2012)– RBSA study’s database offers recent billing analyses results
and detailed house characteristics on 1404 single family houses in the Region.
– RBSA data allows inputs for SEEM runs to be well defined for individual homes.
Methodology - Data
* Using a pre-release version of the database for this analysis .
![Page 20: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/20.jpg)
20
Key Model Input Parameters
RBSA Data Availability
UA Available for each house.
Weather Zip code (available for each house) linked to nearest TMY3 weather station.
Gas Heating Efficiency Available for some houses; used average for remaining houses.
HP Operation & Efficiency Not readily available. Used ARI control & 7.9 HSPF.
Duct System Leakage and Surface Area
Available for some houses; used average for remaining houses with ducts.
Duct System Insulation and Location
Available for each house.
Infiltration Available for some houses; used a floor area-scaled average (by foundation type) for remaining houses
Mechanical Ventilation Not available. Assumed 2 hours /day at 50 cfm.
Non-Lighting Internal Gains Not available. See next slide for details.
Lighting Internal Gains LPD available for each house; assumed 1.5 hours/day.
T-stat Setting Available based on interviews, but used this as the “calibration knob”.
Methodology - Data
![Page 21: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/21.jpg)
21
Detail: Non-Lighting Internal Gains• Equation:
• Based loosely on Building America Benchmark*– Used the original equation and values (averaged) to determine average internal
gains for RBSA homes.• Original equation also includes Number of Bedroom and Finished Floor Area terms
– Set Number of Bedrooms and Finished Floor Area terms to zero and adjusted Number of People term to achieve same average internal gains for RBSA homes.
• Building America Benchmark based on– “The appliance loads were derived by NREL from EnergyGuide labels, a Navigant
analysis of typical models available on the market that meet current NAECA appliance standards, and several other studies. ”
– “The general relationship between appliance loads, number of bedrooms, and house size, was derived empirically from the 2001 RECS. ”
*Hendron, Robert. "Building America Research Benchmark Definition, Updated December 20, 2007." NREL/USDOE EERE. January 2008. NREL/TP-550-42662
Methodology - Data
![Page 22: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/22.jpg)
Methodology - Data 22
Realistic SEEM Simulations Not Feasible/Possible for All Homes in RBSA
Issue Count
More than one foundation type 331
25% > Ceiling Area to Floor Area > 200%, or Missing Ceiling U-value 36
Footprint Area to Floor Area < 20% 36
30% > Wall Area to Floor Area > 200%, or Missing Wall U-value 24
Missing Floor U-value for Crawlspace Foundation 5
Window Area = 0 3
Window u-value = 0 3
• Resulting House Count: 1011– These issues overlap on some houses, so the sum of
the counts cannot be subtracted from 1404 to get 1011.
![Page 23: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/23.jpg)
23
Data Filters Excluded Some RBSA HomesVariable Filter
Value(s)Notes Count
(filtered out)SEEM Run Valid SEEM run must be valid (> 0 kWh/yr). 4
Billing Energy Use > 1,500 kWh/yr
Intends to screen out partially used or unused houses
38
eRsq and gRsq = 0 or ≥0.45
Screens out houses with poor billing analysis results (0.45 per David Baylon)
398
Non-natural-gas & non-electric Fuel Use
0 Screens out houses with wood, oil, propane, etc. consumption because billing analysis not performed.
352
Primary Heating System
eZonal, eFAF, gFAF, HP
Removes gas boilers, wood stoves, etc. 216
Secondary Heating System Fuel
Electric or Gas
Removes wood stoves, propane heaters, etc. 274
• Gas Billing converted to kWh/year using reported AFUE• Resulting House Count: 293• (The counts for each item overlap here, too)
Methodology - Data
![Page 24: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/24.jpg)
Methodology - Data 24
Additional Data Filter for PRISM Excluded Additional Homes
Exclude any home that had an out-of-range PRISM T-balance for one or more components.
– The PRISM analysis restricted balance point temperatures to be between 48 and 70 ⁰F.• T-balance below 48⁰ is plausible.• T-balance above 70⁰ is not physically plausible. We filter these out since such
values are evidence of a poor PRISM fit.
• Our 293 sites’ T-bal values include…– 10 that defaulted to 70⁰ when PRISM’s initial fit exceeded the max.
(Excluded from analysis.) – 16 that defaulted to 48⁰ when PRISM’s initial fit was below the minimum.
(Kept)– 267 whose PRISM balance points were within the acceptable range.
(Kept)• This leaves 283 sites for the present analysis.
![Page 25: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/25.jpg)
Methodology - Data 25
Final Data SetSEEM values calculated with t-stat = 68°F (constant)
Seem (68) Heating Energy (kWh)
![Page 26: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/26.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 26
![Page 27: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/27.jpg)
27
Methodology - Regression
![Page 28: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/28.jpg)
Methodology – Regression 28
Regression Overview (1)• Analysis
Identify and quantify any systematic patterns (trends) in the differences between SEEM(68°F) and PRISM savings estimates ( ∆ kWh = SEEM(68°F) kWh ‒ PRISM kWh.
• Systematic means “explained by known variables.” (Example: SEEM(68°F) kWh tends to exceed PRISM kWh in cooler climates.)
• Tacit assumption: PRISM estimates roughly unbiased.• Definitions
• “PRISM kWH” = Heating energy use via billing analysis; from RBSA SF dataset.
• “SEEM(68°F) kWh” = Heating energy use via SEEM runs using house-specific characteristics data from the RBSA SF dataset with thermostat set to 68°F
![Page 29: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/29.jpg)
Methodology – Regression 29
Regression Overview (2)• Problem is multivariate… – A single underlying trend (example: ∆ increasing with
heating use) may appear in multiple guises (∆ increasing with HDD, or with U-value, or with building heat loss)
• Approach is multiple regression…– Compare PRISM kWh with SEEM kWh when SEEM is run
with a constant T-stat setting (68°F day, 68°F night.)– Y-variable is the percent difference between SEEM kWh
and PRISM kWh (when SEEM uses T-Stat=68°F).– X-variables are physical characteristics known through
RBSA. (Specifying the x-variables is a large part of the work.)
![Page 30: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/30.jpg)
30
Setting up the Regression (1)Primary interest is in differences between SEEM(68) kWh and PRISM kWh—the Y-variable must capture these differences. – Heteroskedasticity. The SEEM(68) /PRISM differences
generally increase in magnitude in proportion to SEEM(68) kWh (or PRISM kWh). (See earlier graph.)
– Measurement error (random noise). As estimates of heating kWh, SEEM(68) and PRISM both have substantial standard errors.
Methodology – Regression
![Page 31: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/31.jpg)
31
Setting up the Regression (2)
• Note choice of signs: means SEEM > PRISM.• What goes in the denominator? “??” =
“Actual kWh” would be ideal. – Using SEEM kWh or PRISM kWh would skew y-
values. (Next slide.)
– Log-transforms (closely related) not quite right either.
– Instead, divide by midpoint: “??” = (SEEM + PRISM)/2. (Two slides down.)Methodology – Regression
![Page 32: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/32.jpg)
Dividing by PRISM kWh magnifies differences where PRISM’s random error happens to be negative (since these values get artificially small denominators). This biases the percent differences upwards.
32Methodology – Regression
![Page 33: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/33.jpg)
Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.
33Methodology – Regression
![Page 34: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/34.jpg)
Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.
34Methodology – Regression
![Page 35: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/35.jpg)
Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.
35Methodology – Regression
![Page 36: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/36.jpg)
Upward bias (mostly) goes away when we divide by the value halfway between SEEM(68°F) and PRISM.
36Methodology – Regression
![Page 37: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/37.jpg)
Methodology – Regression 37
Building the Regression Model• Goal is to identify variables that lead to systematic
differences between SEEM(68°F) and PRISM.– “Lead to” is only seen in rough trends (think: correlation).
• Looking to capture unknown effects – not a physical model.
• Model development is iterative. – A variable may be weakly correlated with raw y-values
but strongly correlated with y’s that have been adjusted to account for some other variable’s influence.
![Page 38: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/38.jpg)
Important Limitations• Avoiding Colinearity - When a potential x-variable closely
tracks some combination of variables that are already included.– Example – Including both heat loss rate and vintage– This redundancy leads to unstable model fits.– Threshold for “tracks too closely” gets low when the usual suspects
are around:
High noise / faint signal / small sample.
• Pursuing Parsimony. General principle: Don’t over-fit the data (by including too many explanatory variables).
• Incomplete data variables. Some variables (e.g., duct tightness and infiltration) aren’t known for very many houses.
Methodology (Regression) - 38Methodology – Regression
![Page 39: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/39.jpg)
39
Prominent x-variable candidates• Characteristics that likely influence differences
between SEEM(68°F) and PRISM estimates of use– Thermal efficiency drivers (U-values, duct tightness,
infiltration, …)– Heating system type – Climate (i.e., HDDs)
• Following graphs illustrate “influence” of several variables (separately) on percent difference between SEEM(68°F) and PRISM
Methodology – Regression
![Page 40: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/40.jpg)
40Methodology – Regression
SEEM (68 ) - PRISMMidpoint
SEEM (68 ) - PRISMMidpoint
![Page 41: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/41.jpg)
41Methodology – Regression
SEEM (68 ) - PRISMMidpoint
SEEM (68 ) - PRISMMidpoint
![Page 42: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/42.jpg)
42Methodology – Regression
![Page 43: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/43.jpg)
43
Insulation Variables
The big surfaces: Wall, ceiling, and floor. • Express in terms of heat loss (U-values,
weighted by surface area as appropriate)• We separate out Floor U because of different
foundation types. – One variable accounts for ceiling and wall heat
loss.– Another variable accounts for floor heat loss in
crawlspace homes.
Methodology – Regression
![Page 44: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/44.jpg)
44
Variable for Wall/Ceiling U
Applies to all homes (regardless of foundation type).A simple indicator variable:
“Wall/Ceiling Insulation is Poor” ifWall u-value > 0.25, ORCeiling u-value > 0.25, OR Both u-values > 0.25.
This variable captures the main effect of the weighted average. (See next slide)
Methodology – Regression
![Page 45: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/45.jpg)
45Methodology – Regression
![Page 46: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/46.jpg)
46
Variable for Floor U
Particularly interested in crawlspace heat loss since crawlspace insulation is a common measure.
Variable definition: “Yes/No” indicator for uninsulated crawlspace.
Note: Sites with basements, slabs, and insulated crawlspaces all have Uninsulated Crawl = “No”
Methodology – Regression
![Page 47: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/47.jpg)
47
Do these indicator variables really capture the insulation effects?
The next two slides compare various u-values’ relationships with– Unadjusted (raw) percent differences;– Percent differences that have been adjusted for
the two insulation variables included in the regression.
Methodology – Regression
![Page 48: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/48.jpg)
48Methodology – Regression
![Page 49: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/49.jpg)
49Methodology – Regression
![Page 50: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/50.jpg)
50
Heating System Variable
Four distinct heating systems in the sample:Electric zonal Electric FAF Gas FAF Heat pump
After controlling for insulation, heating system effect appears to be captured with just two groups:
“Electric Resistance” = Electric zonal / Electric FAF “Gas/HP” = Gas FAF / Heat Pump
Parsimony: two is better than four!
Methodology – Regression
![Page 51: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/51.jpg)
51Methodology – Regression
![Page 52: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/52.jpg)
52Methodology – Regression
![Page 53: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/53.jpg)
Methodology – Regression 53
Model 1 fit summary: Est. s.e. p-value (Intercept) -0.01 (0.04) 0.80 elec. resistance 0.27 (0.05) 0.00 poor.ins.ceil.wall 0.42 (0.08) 0.00 uninsulated.crawl 0.15 (0.07) 0.04 Adjusted R-square = 0.212
…and with an interaction term for insulation: Est. s.e. p-value Intercept -0.18 (0.04) 0.62 elec. resistance 0.27 (0.05) 0.00 poor.ins. ceil.wall 0.49 (0.09) 0.00 uninsulated.crawl 0.21 (0.08) 0.01 poor.ins.c.w*unins.crawl -0.21 (0.16) 0.19 Adjusted R-square = 0.214
No strong recommendation either way because of low p-value, but proposal is to drop the interaction term.
![Page 54: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/54.jpg)
54
Climate variable
HDD effect is not very pronounced. Next slide shows percent differences (adjusted for effects in the previous regression), versus HDDs• Standard HDDs with constant (65⁰) base.• Plot shows (slight) positive correlation
between HDDs and adjusted y-values. • Group means (x-mean, y-mean) lie very near
the overall trend line.
Methodology – Regression
![Page 55: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/55.jpg)
55
(Black line indicates OLS linear regression fit.)
Methodology – Regression
![Page 56: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/56.jpg)
56
Climate Variable
• A modest linear trend is clear from the plot.
• Could either use indicator variables, or the actual HDDs values (a single continuous variable).– Group means agree with overall linear trend
almost perfectly, so little practical difference.
• We use the continuous variable, x = HDDs.
Methodology – Regression
![Page 57: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/57.jpg)
57
Previous fit: Estimate s.e. p-value (Intercept) -0.01 (0.04) 0.80 elec. resistance 0.27 (0.05) 0.00 poor.ins.ceil.wall 0.42 (0.08) 0.00 uninsulated.crawl 0.15 (0.07) 0.04 Adjusted R-square = 0.212 And now with HDDs: Estimate s.e. p-value Intercept -0.40 (0.15) 0.01elec. resistance 0.27 (0.05) 0.00poor.ins.ceil.wall 0.44 (0.07) 0.00 uninsulated.crawl 0.13 (0.07) 0.07Base-65 HDDs 7.3e-5 (2.7e-5) 0.01 Adjusted R-square = 0.230
Methodology – Regression
![Page 58: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/58.jpg)
Interpreting the HDD Coefficient
Our fitted coefficient for HDDs was . What does this mean in practical terms?
In our sample, the HDDs averages differ by about 1500 HDDS from one climate zone to the next.
Since , the climate zone effect corresponds to about an 11% difference.
58Methodology – Regression
![Page 59: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/59.jpg)
59
So far, so good…• The next two slides compare four variables’
relationships with– Unadjusted (raw) percent differences; and– Percent differences that have been adjusted for all
four variables included in the regression.• HDDs and heat source show zero relationship with
adjusted differences.• Square footage and internal gains relationships
went from weak to weaker (even though they are not included in the model – that’s good!).
Methodology – Regression
![Page 60: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/60.jpg)
60Methodology – Regression
![Page 61: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/61.jpg)
61Methodology – Regression
![Page 62: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/62.jpg)
62
Percent difference versus midpoint has also improved…(And midpoint isn’t in the model either)
Methodology – Regression
![Page 63: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/63.jpg)
63
And the insulation variables’ plots still look good…
Methodology – Regression
![Page 64: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/64.jpg)
64Methodology – Regression
![Page 65: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/65.jpg)
65Methodology – Regression
![Page 66: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/66.jpg)
Methodology – Regression 66
What else should we consider?
Next slide indicates several variables’ correlation with adjusted percent differences.
![Page 67: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/67.jpg)
67
Observations• Duct leakage has the largest apparent
correlation, but this variable is sparsely populated. (We’ll look at it next.)
• PRISM HDDs have moved up – these had almost no correlation with unadjusted differences. (We discuss at the end.)
• RBSA (reported) t-stat values have a slight negative correlation with % differences. (This sign makes sense, but we’d expect more.)
Methodology – Regression
![Page 68: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/68.jpg)
68
Duct Leakage
• Have direct RLF and SLF measurements for 33 homes;
• Also, 87 homes have no ducts (zero leakage);• Another 38 (excluded from analysis) have
ducts entirely inside of conditioned spaces.– Some of these spaces are basements designated
“conditioned” simply because they contain ducts.– 26 of the 38 have “heated basements”
• Not much basis for calibration here…Methodology – Regression
![Page 69: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/69.jpg)
69Methodology – Regression
![Page 70: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/70.jpg)
Methodology – Regression 70
Duct Leakage (continued)• Visually, there’s not much correlation in the
range containing most of the data. • Numerically… – Correlation is 53% when only the 33 measured values
are included;– Drops to 15% when 4 right-most points are omitted;– Values drop to 31.5% and 5.3% when we include
homes without ducts (zero leakage).• Weak (and ambiguous) basis for recommending
adjustment specific to duct-leakage.
![Page 71: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/71.jpg)
71
Infiltration
• Have direct infiltration measurements for 95 homes;
• But even less reason for calibration here…
Methodology – Regression
![Page 72: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/72.jpg)
72Methodology – Regression
![Page 73: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/73.jpg)
Methodology – Regression 73
Infiltration (continued)
• Visually, the relationship is null.• Numerically… – Correlation is -11.8% when all points are included;– Changes to 2.3% when single right-most point is
omitted.• No basis for adjustment for infiltration once
other variables are included.
![Page 74: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/74.jpg)
74
PRISM Balance Point
• A question was raised at the May 20 subcommittee meeting regarding whether the regression should take into account the house balance point determined by PRISM.
Methodology – Regression
![Page 75: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/75.jpg)
75Methodology – Regression
![Page 76: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/76.jpg)
76
PRISM HDDs
• Definitely a trend, but is it unexpected? Does it require action?– We know that unobserved variables drive a portion of SEEM-PRISM
deviations that we treat as noise. – Consider a home where an unobserved variable yields an effective
balance point that is lower than we would expect based only on observed variables. This home will tend to satisfy both:• SEEM(68) kWh > PRISM kWh and • TMY HDD > PRISM HDD
– In other words, the presence of unobserved variables causes a positive correlation between kWh differences and HDD differences.
• Conclusion: The presence of unobserved variables should yield a trend like the one seen on the previous slide.
• So the trend is what we would expect.
Methodology – Regression
![Page 77: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/77.jpg)
77
Proposed Final Model
Variable Estimated Standard p-value coefficient errorIntercept -0.40 (0.15) 0.01elec. resistance 0.27 (0.05) 0.00poor.ins.ceil.wall 0.44 (0.07) 0.00 uninsulated.crawl 0.13 (0.07) 0.07HDDs (Base 65) 7.3e-5 (2.7e-5) 0.01
Adjusted R-square = 0.230
𝑦=𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡+𝛽𝑒𝑙𝑒𝑐 .𝑟𝑒𝑠𝑖𝑠× 𝐼𝑒𝑙𝑒𝑐 .𝑟𝑒𝑠𝑖𝑠+𝛽𝑝𝑜𝑜𝑟 .𝑖𝑛𝑠 .𝑐𝑒𝑖𝑙 .𝑤𝑎𝑙𝑙× 𝐼𝑝𝑜𝑜𝑟 .𝑖𝑛𝑠 .𝑐𝑒𝑖𝑙 .𝑤𝑎𝑙𝑙+𝛽𝑢𝑛𝑖𝑛𝑠 . 𝑐𝑟𝑎𝑤𝑙× 𝐼𝑢𝑛𝑖𝑛𝑠 .𝑐𝑟𝑎𝑤𝑙+𝛽𝐻𝐷𝐷×𝐻𝐷𝐷
Methodology – Regression
![Page 78: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/78.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 78
![Page 79: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/79.jpg)
Methodology – Calibration 79
Final Step: Interpreting Results• From the fitted model, we obtain adjustment
factors that apply to SEEM output to align SEEM with the RBSA-PRISM data.
• A given site’s adjustment factor depends on the values of the explanatory variables for that site.
• Group HDDs by climate zone. Then for each zone, there are 8 possible configurations of the three other variables.
• This yields 24 distinct adjustment factors in all.
![Page 80: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/80.jpg)
80
Calibration FactorsSEEM(68) differs from PRISM by these factors (on average)
0%
20%
40%
60%
80%
100%
No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes
No Yes No Yes No Yes No Yes No Yes No Yes
Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res.
Climate Zone 1 Climate Zone 2 Climate Zone 3
Methodology – Calibration
![Page 81: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/81.jpg)
81
T-Stat Calibration
• Translate percent kWh adjustments into adjustments in daytime t-stat setting (from 68 °F).
• No data limitations here: we can directly observe SEEM’s sensitivity to t-stat settings.
• Method: 1. Run SEEM for each house at multiple temperature settings in 2 degree increments
– Daytime Settings: … 58, 60, 62, … – Nighttime Setback: Daytime setting - setback
» Setback: Use average difference between reported daytime and nighttime t-stat settings in RBSA dataset; by heating system type:
2. Determine relationship of calibration factors to temperature settings for each of the 24 scenarios.3. Interpolate to determine “calibrated” t-stat settings.
(need to add a graph to help explain this)
Heating System Type Avg Setback (°F)
Electric FAF 6.0 Electric Zonal 4.8 Heat Pump 4.3 Gas FAF 4.8
Methodology – Calibration
![Page 82: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/82.jpg)
82
Calibrated Thermostat Settings
50
55
60
65
70
No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes
No Yes No Yes No Yes No Yes No Yes No Yes
Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res. Gas FAF / HP Elec. Res.
Climate Zone 1 Climate Zone 2 Climate Zone 3
Heating System Type Avg Setback (°F)
Electric FAF 6.0 Electric Zonal 4.8 Heat Pump 4.3 Gas FAF 4.8
Methodology – Calibration
![Page 83: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/83.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 83
![Page 84: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/84.jpg)
Discussion 84
Discussion• Are we done?• Decision: SEEM94 is “calibrated”; it will give reliable
heating energy consumption results – for single family houses with the following characteristics:
• Heating System is one or more of the following: Gas FAF, Electric FAF, HP, zonal electric (no other heating system type);
• Occupied/normal houses (PRISM worked); – if the following inputs are used:
• Calibrated Thermostat Settings (see slide above); and• Internal Gains:
![Page 85: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/85.jpg)
Discussion 85
Next Steps• If the RTF agrees it’s calibrated, the RTF will be able to use SEEM94 to help
estimate energy savings for residential single family– Heat Pump
• Conversions• Upgrades• Commissioning, Controls, and Sizing
– Weatherization• Insulation• Windows• Infiltration reduction
– Duct Sealing– New Home Efficiency Upgrades
• “Help” is used here because we will still need to deal with “non-electric benefits” for these measures.– This topic is out of scope for today’s discussion. The goal today is simply to
determine whether SEEM has been calibrated to provide reliable results.
![Page 86: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/86.jpg)
Overview
• Background– Purpose– History
• Methodology– Data– Regression– Calibration
• Discussion• Proposal
Overview - 86
![Page 87: SEEM 94 Calibration to Single Family RBSA Data Analysis and proposed actions](https://reader036.vdocuments.mx/reader036/viewer/2022062521/56816723550346895ddbacbd/html5/thumbnails/87.jpg)
Proposal 87
Proposed Motion
“I _______ move that the RTF consider SEEM94 calibrated for single family houses.”