SDDP, SPECTRA and Reality

A comparison of hydro-thermal generation system management

Roger Miller, Electricity Commission3 September 2009

2

Introduction• SDDP (Stochastic Dual Dynamic Programming Model) and

SPECTRA (System, Plant, and Energy Co-ordination using Two Reservoir Approach) are two hydro-thermal generation coordination programs.

• The Electricity Commission uses SDDP in conjunction with GEM (Generation Expansion Model) to model power system operation under possible future generation expansion scenarios.

• SDDP allows quite flexible and detailed modelling of generation and transmission constraints (though the EC doesn’t use most of these features), but takes many hours to solve a typical multi-year optimisation problem.

• SPECTRA, is less flexible and detailed, but can solve an equivalent problem in a matter of minutes.

• In order to assess the usefulness of SPECTRA as a replacement and/or supplement to SDDP, a comparison has been carried out between the outputs of the two models, and with the actual generation patterns observed in the NZ system over recent years.

3

Overview

• Hydro Lake Level Contours• Incremental Water Value Surfaces• Price Duration Curves• Generation Duration Curves• Possible improvements

4

Hydro Lake Level Contours

• Actual - One trajectory per year• Simulations

• One trajectory per inflow sequence• Shows study period up to December 2011

• 5th, 25th, 50th, 75th, 95th percentiles and mean

5

Actual Levels - Lake Pukaki Lake Pukaki - Actual Lake Levels

(Lake Limits: 518 to 532.5 m)

0%

20%

40%

60%

80%

100%

120%

0 92 184 276 368

Day of Year

Per

cen

t F

ull

6

Actual Levels – Lake Tekapo Lake Tekapo - Actual Lake Levels

(Lake Limits: 701.8 to 710.9 m)

-20%

0%

20%

40%

60%

80%

100%

120%

0 92 184 276 368

Day of Year

Pe

rce

nt

Fu

ll

7

Actual Levels - Lake Hawea Lake Hawea - Actual Lake Levels

(Lake Limits: 338 to 346 m)

-20%

0%

20%

40%

60%

80%

100%

120%

0 92 184 276 368

Day of Year

Pe

rce

nt

Fu

ll

8

Observations – Actual levels

Pukaki, Tekapo and Hawea

• all have similar annual cycles• Drawn down through winter reaching a

minimum level around September/October in time for spring snow melt

• Reach Max Level 5 to 25% of the time in first half of year

• Occasionally get very low in spring

9

Actual Levels - Lake Te AnauLake Te Anau - Actual Lake Levels (controlled post 1974)

(Lake Limits: 201.5 to 202.7 m)

-100%

-50%

0%

50%

100%

150%

200%

250%

300%

0 92 184 276 368

Day of Year

Perc

ent F

ull

1974 1975 1976 1977

1978 1979 1980 1981

1982 1983 1984 1985

1986 1987 1988 1989

1990 1991 1992 1993

1994 1995 1996 1997

1998 1999 2000 2001

2002 2003 2004 2005

2006 2007 2008 5%

25% 50% 75% 95%

avg

10

Actual Levels - Lake ManapouriLake Manapouri - Actual Lake Levels

(Lake Limits: 518 to 532.5 m)

-100%

-50%

0%

50%

100%

150%

200%

250%

300%

0 92 184 276 368

Day of Year

Pe

rce

nt

Fu

ll

11

Observations – Actual levels

Manapouri and Te Anau

• Less pronounced annual cycle• Much more variable throughout most of year• Smaller storage relative to their mean inflows• Regularly exceed maximum control level• SDDP/SPECTRA model a hard upper limit at

which forced release occurs (high spill)• In reality levels subside over several weeks

(less spill)• Potential for improved modelling

12

Actual Levels - Lake Taupo Lake Taupo - Actual Lake Levels

(Lake Limits: 355.85 to 357.25 m)

-20%

0%

20%

40%

60%

80%

100%

120%

140%

0 92 184 276 368

Day of Year

Per

cen

t F

ull

13

Actual Levels - Lake Waikaremoana Lake Waikaremoana - Actual Lake Levels

(Lake Limits: 580.29 to 583.29 m)

-40%

-20%

0%

20%

40%

60%

80%

100%

120%

140%

160%

0 92 184 276 368

Day of Year

Pe

rce

nt

Fu

ll

14

Observations – Actual levels

Taupo and Waikaremoana

• Different cycle to South Island lakes• Reach minimum level around May and fill

through the winter• Utilise most of their range but seldom spill or

run out

15

Simulated Lake Levels

16

SPECTRA Lake Levels (GWh) – 1st attempt

52 78 104 130 156 1820

100

200

300

400

500

600

SPECTRA Lake Levels (GWh)

TPO min outflow 90 cumecs; Original IU's; Post Tax NPV $7820.4M

North Island

study week52 78 104 130 156 182

0

500

1000

1500

2000

2500

3000

South Island

study week52 78 104 130 156 182

0

50

100

150

200

250

300

350

400

450

500Taupo & Waikato

study week52 78 104 130 156 182

0

20

40

60

80

100

120

Waikaremoana

study week

52 78 104 130 156 1820

50

100

150

200

250

300Hawea & Clutha

study week52 78 104 130 156 182

0

50

100

150

200

250

300

350

400Manapouri

study week52 78 104 130 156 182

0

200

400

600

800

1000

1200

1400

1600Waitaki Pukaki

study week52 78 104 130 156 182

0

100

200

300

400

500

600

700

Waitaki TEKAPO

study week

1 Jan 20121 July 2009 1 Jan 20121 July 2009 1 Jan 20121 July 2009 1 Jan 20121 July 2009

17

Improvements made:

• Reduced Taupo minimum outflow from 90 to 50 cumecs (resource consent)

• All IU’s set back to neutral except for Manapouri (biased downwards)

• Introduced 20/20 storage grid (NI/SI)• Resulted in 3% saving in fuel costs• Further room for fine tuning

18

SPECTRA Lake Levels (GWh) – “optimised”

52 78 104 130 156 1820

100

200

300

400

500

600

SPECTRA Lake Levels (GWh)

TPO min outflow 50 cumecs; Manapouri IU Bias 0.7; 20/20 grid; Post Tax NPV $7602.3M

North Island

study week52 78 104 130 156 182

0

500

1000

1500

2000

2500

3000

South Island

study week52 78 104 130 156 182

0

50

100

150

200

250

300

350

400

450

500Taupo & Waikato

study week52 78 104 130 156 182

0

20

40

60

80

100

120

Waikaremoana

study week

52 78 104 130 156 1820

50

100

150

200

250

300Hawea & Clutha

study week52 78 104 130 156 182

0

50

100

150

200

250

300

350

400Manapouri

study week52 78 104 130 156 182

0

200

400

600

800

1000

1200

1400

1600Waitaki Pukaki

study week52 78 104 130 156 182

0

100

200

300

400

500

600

700

Waitaki TEKAPO

study week

1 Jan 20121 July 2009 1 Jan 20121 July 2009 1 Jan 20121 July 2009 1 Jan 20121 July 2009

19

SDDP Lake Levels (hm3)

6 12 18 24 30 36 42 480

100

200

300

400

500

600

700

SDDP Lake Levels (hm3)Lake Taupo

study month6 12 18 24 30 36 42 48

0

20

40

60

80

100

120

140

160L Waikaremo

study month6 12 18 24 30 36 42 48

0

100

200

300

400

500

600

Lake Tekapo

study month

6 12 18 24 30 36 42 480

250

500

750

1000

1250

1500

1750

2000

Lake Pukaki

study month6 12 18 24 30 36 42 48

0

100

200

300

400

500

600

700

800

900

1000

1100

Lake Hawea

study month6 12 18 24 30 36 42 48

0

100

200

300

400

500

600

700

800

900

1000

Manapouri

study month

1 Jan 20121 Jan 2008 1 Jan 20121 Jan 2008 1 Jan 20121 Jan 2008

20

SDDP/SPECTRA lake level comparison• SDDP drives lakes up and down more

aggressively! (less conservative)

• Most lakes have a high probability of both running out of water and of spilling

• SDDP trajectories vary significantly from year to year – most apparent in Waikaremoana

• Doesn’t appear to make economic sense

• Possibly due to cut elimination (discussed later)

• SPECTRA settles down to a regular pattern

• SPECTRA more similar to reality (possibly a self-fulfilling prophecy?)

21

Water Value Surfaces

• Represents the expected future value of holding an additional unit of water in storage

• Averaged over historical inflow sequences (in this case 1932 through 2005)

• Gives controlled hydro storage an effective “fuel price” (opportunity cost)

• Function of time of year due to annual inflow and demand patterns

• Function of storage level in all reservoirs

22

Water Value Surfaces (SPECTRA)

• Produced by RESOP (Reservoir Optimisation) module

• 2-reservoir model ( NI and SI lumped model)• Directly calculated for all combinations of

storage (eg. 6x12 or 20x20)• Uses Incremental Utilisation (IU) Curves to

approximately split out into individual reservoirs• Uses heuristic to account for serial inflow

correlation

23

SPECTRA Water Value Surface - SI

1 14 27 40 53 66 79 92

0246810121416

18

20050

100150200250300350400450500550600650700750800850900950

100010501100115012001250

Water Value [$/MWh]

week number

SI Level

(NI level = 50%)

1 July 2008 1 July 2010

24

SPECTRA Water Value Surface - NI

(SI level = 50%)

1 14 27 40 53 66 79 92

02468101214

16

18

200

25

50

75

100

125

150

175

200

225

250

275

300

325

Water Value [$/MWh]

week number

NI Level

1 July 2008 1 July 2010

25

Water Value Surfaces (SDDP)

• Multi-reservoir model• Serial inflow correlation explicitly modelled• Water value implied by slope of Future Cost Function (FCF)• FCF is a multi-dimensional non-linear hyper-surface• Approximated by tangent hyper-planes known as “cuts”

which act as linear constraints in the optimisation• Extra cuts are added at each iteration at the storage and

inflow combinations that occur in the simulation (each time step gets one new cut for every inflow sequence)

• To reduce dimensionality, inactive (non-binding) cuts can be eliminated after a specified number of iterations

• Implied water values tend to be lumpy and not well defined over the whole solution space, especially if cuts are eliminated

26

Obtaining Water Values from SDDP• Tom Halliburton has written a utility to extract

water values from an FCF output text file• Electricity Commission has traditionally

eliminated inactive cuts after 4 iterations• This doesn’t yield meaningful water value

surfaces• Water values are effectively extrapolated from

the cut point over almost the entire storage range of the reservoir

• I suspect there may also be data precision issues in the FCF text file for long studies?

27

SDDP Water Value Surface – Lake Pukaki(inactive cuts eliminated after 4 iterations)

(All lakes equally full, Mean inflow sequence)

Jan-

08

Feb

-08

Mar

-08

Apr

-08

May

-08

Jun-

08

Jul-0

8

Aug

-08

Sep

-08

Oct

-08

Nov

-08

Dec

-08

Jan-

09

Feb

-09

Mar

-09

Apr

-09

May

-09

Jun-

09

Jul-0

9

Aug

-09

Sep

-09

Oct

-09

Nov

-09

25,463

21,442

17,422

13,402

9,381

5,3611,340

0

50

100

150

200

250

300

350

400

450

500

550

Water Value($/MWh)

Storage(CMD)

28

Obtaining Water Values from SDDP (2)

To obtain meaningful Water Value Surfaces:• SDDP was rerun without eliminating any cuts• This significantly increases solution time, so• Study was limited to only 2 years• Risk of end effects

29

Jan-

08

Feb

-08

Mar

-08

Apr

-08

May

-08

Jun-

08

Jul-0

8

Aug

-08

Sep

-08

Oct

-08

Nov

-08

Dec

-08

Jan-

09

Feb

-09

Mar

-09

Apr

-09

May

-09

Jun-

09

Jul-0

9

Aug

-09

Sep

-09

Oct

-09

Nov

-09

25,463

21,442

17,422

13,402

9,381

5,361

1,340

050

100150200250300350400450500550600650700750800850900950

100010501100115012001250130013501400145015001550160016501700175018001850

Water Value($/MWh)

Storage(CMD)

SDDP Water Value Surface – Lake Pukaki(all cuts kept)

(All lakes equally full , Mean inflow sequence)

30

SDDP Water Value Surface – Lake Taupo(all cuts kept)

(All other lakes 50% full, Mean inflow sequence)

Jan-

08

Feb

-08

Mar

-08

Apr

-08

May

-08

Jun-

08

Jul-0

8

Aug

-08

Sep

-08

Oct

-08

Nov

-08

Dec

-08

Jan-

09

Feb

-09

Mar

-09

Apr

-09

May

-09

Jun-

09

Jul-0

9

Aug

-09

Sep

-09

Oct

-09

Nov

-09

8,796

7,407

6,019

4,630

3,241

1,852463

0

50

100

150

200

250

300

Water Value($/MWh)

Storage(CMD)

31

Effect of cut elimination on SDDP simulation

32

6 12 180

100

200

300

400

500

600

700

SDDP Lake Levels (hm3)Lake Taupo

study month6 12 18

0

20

40

60

80

100

120

140

160L Waikaremo

study month6 12 18

0

100

200

300

400

500

600

Lake Tekapo

study month

6 12 180

250

500

750

1000

1250

1500

1750

2000

Lake Pukaki

study month6 12 18

0

100

200

300

400

500

600

700

800

900

1000

1100Lake Hawea

study month6 12 18

0

100

200

300

400

500

600

700

800

900

1000

Manapouri

study month

6 12 180

100

200

300

400

500

600

700

SDDP Lake Levels (hm3)Lake Taupo

study month6 12 18

0

20

40

60

80

100

120

140

160L Waikaremo

study month6 12 18

0

100

200

300

400

500

600

Lake Tekapo

study month

6 12 180

250

500

750

1000

1250

1500

1750

2000

Lake Pukaki

study month6 12 18

0

100

200

300

400

500

600

700

800

900

1000

1100

Lake Hawea

study month6 12 18

0

100

200

300

400

500

600

700

800

900

1000

Manapouri

study month

inactive cuts eliminated after 4 iterations(36 year study)

keep all cuts(2 year study)

33

Price Duration Curves

• For study year 2010• Inflow sequences 1932 through 2005

34

North Island Price Duration Curves

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 110

20

30

40

50

60

708090

100

200

300PDC - North Island, pds=79:130, Avg MC=$62.7/MWh

Ma

rgin

al C

ost

$/M

Wh

Spectra - Avg MC $62.7/MWh

SDDP - Avg MC $58.3/MWh

spill

shortage

35

South Island Price Duration Curves

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 110

20

30

40

50

60

708090

100

200

300PDC - South Island, pds=79:130, Avg MC=$52.6/MWh

Ma

rgin

al C

ost

$/M

Wh

Spectra - Avg MC $52.6/MWh

SDDP - Avg MC $55.1/MWh

shortage

spill

36

• In SI, SPECTRA is $2.50 cheaper.• In NI, SPECTRA is $4.40 more expensive• Since NI is bigger, overall SDDP comes out

cheaper.• This perhaps suggests that on purely economic

grounds SPECTRA’s extra conservatism may not be justified?

• Comes down to appetite for risk and valuation of shortage

• There may be other differences between the models causing this outcome, eg. different demand response/shortage prices

• Not a rigorous comparison

37

A sample ofGeneration Duration Curves

• Actual generation over recent years• Simulated generation over same years with

actual historical inflows

38

SPECTRAAvg 523 MW

SDDPAvg 494 MW

ActualAvg 461 MW

Waikato scheme1998 through 2005

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600

700

800

900

1000

1100Actual GDC for taupo for 01-Jan-1998:31-Dec-2005, Avg MW =461.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600

700

800

900

1000

1100Spectra GDC for delf.TAUP, pds 79-130, flows 1996-2003 Avg MW =523.2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600

700

800

900

1000

1100SDDP GDC - sim year=2010, inflow year=1996 to 2003, stn=TPO, Avg MW =494.3

MW

39

SPECTRAAvg 37 MW

SDDPAvg 40 MW

ActualAvg 48 MW

Waikaremoana scheme1998 through 2005

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120

140Spectra GDC for delf.WAIK, pds 79-130, flows 1996-2003 Avg MW =37.2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120

140SDDP GDC - sim year=2010, inflow year=1996 to 2003, stn=WKA, Avg MW =39.9

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

120

140Actual GDC for GEN.Hydro.Waikaremoana for 01-Jan-1998:31-Dec-2005, Avg MW =48.4

MW

40

SPECTRAAvg 756 MW

SDDPAvg 743 MW

ActualAvg 746 MW

Ohau / Lower Waitaki schemes1998 through 2005

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

500

1000

1500Spectra GDC for delf.WAIT, pds 79-130, flows 1998-2005 Avg MW =756

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

500

1000

1500SDDP GDC - sim year=2010, inflow year=1998 to 2005, stn=WTR, Avg MW =743

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

500

1000

1500Actual GDC for pukaki for 01-Jan-2000:31-Dec-2007, Avg MW =746

MW

41

SPECTRAAvg 528 MW

SDDPAvg 562 MW

ActualAvg 566 MW

Manapouri scheme2003 through 2007

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600

700

800Spectra GDC for delf.MANA, pds 79-130, flows 2001-2005 Avg MW =528

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600

700

800SDDP GDC - sim year=2010, inflow year=2001 to 2005, stn=MANA, Avg MW =562

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

100

200

300

400

500

600

700

800Actual GDC for GEN.Hydro.Manapouri for 01-Jan-2003:31-Dec-2007, Avg MW =566

MW

42

SPECTRAAvg 319 MW

SDDPAvg 311 MW

ActualAvg 344 MW

Huntly E3P 2008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

50

100

150

200

250

300

350

400Spectra GDC for delf2010.E3P, pds 79-130, flows 1932-2005 Avg MW =319

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

50

100

150

200

250

300

350

400SDDP GDC - sim year=2010, inflow year=1932 to 2005, stn=e3p , Avg MW =311

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

50

100

150

200

250

300

350

400

450Actual GDC for GEN.Thermal.Huntly.Gas for 01-Jan-2008:31-Dec-2008, Avg MW =344

MW

43

Huntly E3P

Otahuhu B

TCC

CCGT seasonal temperature effect

2007 2008 20090

50

100

150

200

250

300

350

400

450Actual Time Series for GEN.Thermal.Huntly.Gas for 01-Jan-2007:31-Dec-2008, Avg MW =282

2006 2007 2008 20090

50

100

150

200

250

300

350

400Actual Time Series for GEN.Thermal.Otahuhu.B for 01-Jan-2006:31-Dec-2008, Avg MW =302

2005 2006 2007 2008 20090

50

100

150

200

250

300

350

400Actual Time Series for GEN.Thermal.Stratford.B for 01-Jan-2005:31-Dec-2008, Avg MW =257

MW

44

Possible EC model improvements:• Modify HVDC loss model to include the effect of

DC transfer on AC losses • Update various station capacities• Update HVDC capacity• Update various lake level and outflow constraints• Seasonal variations in lake level limits• Seasonal temperature effect on CCGT capacity• Additional reservoirs (Waipori, Cobb, Coleridge …)• Fewer reservoirs (run Manapouri as uncontrolled?)• Reduce RESOP serial correlation heuristic?

45

Possible program enhancements to SPECTRA / RESOP

• Schedulable thermals in the South Island?• Pumped storage hydros?• More explicit hydro reservoirs in RESOP?

46

Conclusion

• SDDP and SPECTRA currently each have advantages and disadvantages

• Potential to fine tune both EC models• SDDP execution options (cut elimination

strategy, convergence tolerance, max iterations)

• SPECTRA IU curves, trib schedulability, serial correlation

• Model details (ratings, constraints etc)• Possible SPECTRA program enhancements