Model-Based Model-Based Sampling Methodology Sampling Methodology

for the New Form EIA-923for the New Form EIA-923

ASA-EIA Fall MeetingASA-EIA Fall Meeting

October 18, 2007October 18, 2007

Joel DouglasJoel Douglas

““*This is a working document prepared by the Energy Information Administration (EIA) in order *This is a working document prepared by the Energy Information Administration (EIA) in order

to solicit advice and comment on statistical matters from the American Statistical Association to solicit advice and comment on statistical matters from the American Statistical Association Committee on Energy Statistics. This topic will be discussed at EIA's fall 2007 meeting with the Committee on Energy Statistics. This topic will be discussed at EIA's fall 2007 meeting with the

Committee to be held October 18 and 19, 2007.”Committee to be held October 18 and 19, 2007.”

Form EIA-923 IntroductionForm EIA-923 Introduction“Power Plant Operations Report”“Power Plant Operations Report”

Merges Data Elements From 5 Current Merges Data Elements From 5 Current Forms:Forms:• EIA-906EIA-906 “ “Power Plant Report”Power Plant Report”• EIA-920 EIA-920 “Combined Heat and Power Plant “Combined Heat and Power Plant

Report”Report”• EIA-423EIA-423 “ “Monthly Cost and Quality of Fuels for Monthly Cost and Quality of Fuels for

Electric Plants ReportElectric Plants Report””• FERC-423FERC-423 “ “Monthly Report of Cost and Quality of Monthly Report of Cost and Quality of

Fuels for Electric Plants”Fuels for Electric Plants”• EIA-767 EIA-767 “Steam-Electric Plant Operation and “Steam-Electric Plant Operation and

Design Report ”Design Report ”

Form EIA-923 Data SummaryForm EIA-923 Data Summary

Data of Interest for Monthly Sampling:Data of Interest for Monthly Sampling:

• EIA-906EIA-906 - Generation, Consumption, and Stocks - Generation, Consumption, and Stocks

• EIA-920EIA-920 - Generation, Consumption, and Stocks - Generation, Consumption, and Stocks

• EIA-423EIA-423 - Fuel Receipts, Fuel Cost, and Fuel Quality - Fuel Receipts, Fuel Cost, and Fuel Quality

• FERC-423FERC-423 - Fuel Receipts, Fuel Cost, and Fuel Quality - Fuel Receipts, Fuel Cost, and Fuel Quality

Form EIA-923 Data SummaryForm EIA-923 Data Summary

GenerationGeneration ConsumptionConsumption StocksStocks Receipts Receipts (Cost & (Cost & Quality)Quality)

Electricity Only Electricity Only Facilities (EIA-Facilities (EIA-906)906)

Electricity Only Electricity Only Facilities (EIA-Facilities (EIA-906)906)

Electricity Only Electricity Only Facilities (EIA-Facilities (EIA-906)906)

Unregulated Unregulated Facilities (EIA-Facilities (EIA-423)423)

Combined Heat Combined Heat and Power and Power Plants (EIA-920)Plants (EIA-920)

Combined Heat Combined Heat and Power and Power Plants (EIA-920)Plants (EIA-920)

Combined Heat Combined Heat and Power and Power Plants (EIA-Plants (EIA-920)920)

Regulated Regulated Facilities (FERC-Facilities (FERC-423)423)

Sample Selection GoalsSample Selection Goals

Sample Size Reduction Sample Size Reduction • Commercial and Industrial SectorsCommercial and Industrial Sectors• Smaller facilitiesSmaller facilities

Reduce Both Respondent and EIA BurdenReduce Both Respondent and EIA Burden• Respondent Burden = Respondent CountRespondent Burden = Respondent Count• EIA Burden = Facility CountEIA Burden = Facility Count

Reasons for Using Model-Based Cutoff Sampling Reasons for Using Model-Based Cutoff Sampling Instead of Design-Based SamplingInstead of Design-Based Sampling

For Electric Power DataFor Electric Power Data

Difficulty in collecting smallest respondents require that they be left out of the sample. Smallest respondents tend to have disproportionately large nonsampling error

Due to the highly skewed nature of electric power data, the smallest respondents do not affect aggregated totals very much.

Since very good auxiliary / regressor data are available, regression models are useful (i.e. Low variance and model failure is limited to data that should be modeled anyway)

Better use of resources to raise accuracy and lower respondent and Government burden.

EIA has produced good results for nearly two decades utilizing model based cutoff sampling

Sample Selection OutlineSample Selection Outline Sampling groups mirror the generation Sampling groups mirror the generation

groups in the groups in the Electric Power MonthlyElectric Power Monthly publication. publication.

These groups represent the intersection of These groups represent the intersection of the following three levels:the following three levels:

• Facility TypeFacility Type

• Fuel TypeFuel Type

• Geographic RegionGeographic Region

Sample Selection Outline Cont.Sample Selection Outline Cont.

Facility Types (4)Facility Types (4)

• Regulated (REG)Regulated (REG)

• Independent Power Producers (IPP)Independent Power Producers (IPP)

• Commercial (COM)Commercial (COM)

• Industrial (IND)Industrial (IND)

Sample Selection Outline Cont.Sample Selection Outline Cont.

Fuel Types (14)Fuel Types (14)- Coal- Coal

- Geothermal - Geothermal

- Hydroelectric- Hydroelectric

- Natural Gas- Natural Gas

- Nuclear- Nuclear

- Other Gas- Other Gas

- Other Sources- Other Sources

- Petroleum- Petroleum- Petroleum Coke - Petroleum Coke - Pumped Storage- Pumped Storage- Solar- Solar- Waste- Waste- Wind- Wind- Wood- Wood

Sample Selection Outline Cont.Sample Selection Outline Cont.

Geographic Regions (10 Modified Census Geographic Regions (10 Modified Census Divisions)Divisions)

- Middle Atlantic- Middle Atlantic- South Atlantic- South Atlantic- East Central- East Central- West Central- West Central- East North Central- East North Central- New England- New England- West North Central- West North Central- Mountain - Mountain - Pacific Contiguous- Pacific Contiguous- Pacific Non-Contiguous- Pacific Non-Contiguous

Sample Selection Outline Cont.Sample Selection Outline Cont.

Sampling Methodology SummarySampling Methodology Summary

Five StepsFive Steps

• Step #1 – Capacity CutoffsStep #1 – Capacity Cutoffs

• Step #2 – Reported Value CoverageStep #2 – Reported Value Coverage

• Step #3 – Estimation Group CountsStep #3 – Estimation Group Counts

• Step #4 – Acceptable RSEsStep #4 – Acceptable RSEs

• Step #5 – Additional Facilities Step #5 – Additional Facilities

Sample Selection Procedure Step #1Sample Selection Procedure Step #1

Cutoff sampling methodology based on Cutoff sampling methodology based on operating nameplate capacityoperating nameplate capacity

Ensure adequate capacity coverage by Ensure adequate capacity coverage by sector, fuel type, and regionsector, fuel type, and region

• Regulated – 70%Regulated – 70%• Independent Power Producer – 70%Independent Power Producer – 70%• Commercial – 50%Commercial – 50%• Industrial – 50%Industrial – 50%

Sample Selection Procedure Step #2Sample Selection Procedure Step #2

Add additional facilities that rank within a Add additional facilities that rank within a preset percentage of actual generation, preset percentage of actual generation, consumption and stocks and are not consumption and stocks and are not included in the capacity cutoff sample.included in the capacity cutoff sample.

• Regulated – 50%Regulated – 50%• Independent Power Producer – 50%Independent Power Producer – 50%• Commercial – 25%Commercial – 25%• Industrial – 25%Industrial – 25%

Sample Selection Procedure Step #3Sample Selection Procedure Step #3

Ensure adequate estimation group coverageEnsure adequate estimation group coverage

• Estimation groups must have a minimum of 10 Estimation groups must have a minimum of 10 monthly observed valuesmonthly observed values

• Grouping parameters are similar to the sampling Grouping parameters are similar to the sampling parameters, with a few notable exceptions.parameters, with a few notable exceptions.

Sample Selection Procedure Step #4Sample Selection Procedure Step #4

Ensure fuel group Relative Standard Ensure fuel group Relative Standard Errors (RSEs) are within reasonable range Errors (RSEs) are within reasonable range at the US regional level.at the US regional level.

• Regulated – RSE < 5%Regulated – RSE < 5%• Independent Power Producer – RSE < 5%Independent Power Producer – RSE < 5%• Commercial – RSE < 10%Commercial – RSE < 10%• Industrial – RSE < 10%Industrial – RSE < 10%

Sample Selection Procedure Step #5Sample Selection Procedure Step #5

Add additional facilities which do not add Add additional facilities which do not add much to respondent or EIA burdenmuch to respondent or EIA burden

Mainly large coal and natural gas facilities Mainly large coal and natural gas facilities that are in the current EIA-906 / EIA-920 that are in the current EIA-906 / EIA-920 monthly sample.monthly sample.

Approximately 200 additional respondents Approximately 200 additional respondents addedadded

Final Results - CountsFinal Results - Counts

0

1,000

2,000

3,000

4,000

5,000

6,000

Total REG IPP IND COM

PopulationActual SampleProposed Sample

Final Results - PercentFinal Results - Percent

0%

5%

10%

15%

20%

25%

30%

35%

40%

Total REG IPP IND COM

ActualProposed

Final Results – Generation VolumeFinal Results – Generation Volume

0%10%20%30%40%50%60%70%80%90%

100%

Total REG IPP IND COM

ActualProposed

Further InformationFurther Information

The sampling methodology is also The sampling methodology is also described in the OMB Federal described in the OMB Federal Register Notice associated with the Register Notice associated with the Electricity 2008 Project. Electricity 2008 Project.

http://www.eia.doe.gov/cneaf/electrichttp://www.eia.doe.gov/cneaf/electricity/page/fednotice/elect_2008.htmlity/page/fednotice/elect_2008.html