andy philpott epoc (epoc.nz) joint work with vitor de matos, ziming guan

EPOC Winter Workshop, October 26, 2010 Slide 1 of 31

Andy PhilpottEPOC

(www.epoc.org.nz)

joint work with

Vitor de Matos, Ziming Guan

Advances in DOASA


What is it?

• EPOC version of SDDP with some differences• Version 1.0 (P. and Guan, 2008)

– Written in AMPL/Cplex– Very flexible– Used in NZ dairy production/inventory problems– Takes 8 hours for 200 cuts on NZEM problem

• Version 2.0 (P. and de Matos, 2010) – Written in C++/Cplex– Time-consistent risk aversion– Takes 8 hours for 5000 cuts on NZEM problem

DOASA


Motivation

• Market oversight in the spot market is important to detect and limit exercise of market power.– Limiting market power will improve welfare.– Limiting market power will enable market

instruments (e.g. FTRs) to work as intended.• Oversight needs good counterfactual models.

– Wolak benchmark overlooks uncertainty – We use a rolling horizon stochastic optimization

benchmark requiring many solves of DOASA.• We don’t have access to SDDP. • We seek ways that SDDP can be improved.

DOASA

EPOC Winter Workshop, October 26, 2010 Slide 4 of 31Source: CC Report, p 200

Counterfactual 1The Wolak benchmark


What is counterfactual 1?

– Fix hydro generation (at historical dispatch level).– Simulate market operation over a year with thermal plant

offered at short-run marginal (fuel) cost.– “The Appendix of Borenstein, Bushnell, Wolak (2002)* rigorously

demonstrates that the simplifying assumption that hydro-electric suppliers do not re-allocate water will yield a higher system-load weighted average competitive price than would be the case if this benchmark price was computed from the solution to the optimal hydroelectric generation scheduling problem described above” [Commerce Commission Report, page 190].

(* Borenstein, Bushnell, Wolak, American Economic Review, 92, 2002)

The Wolak benchmark


Counterfactual 1What about uncertain inflows?

wet

dryStochastic program counterfactualThe optimal generation plan burns thermal fuel in stage 1 in case there is a drought in winter. The competitive price is high (marginal thermal fuel cost) in the first stage, but zero in the second (if wet).

Counterfactual 1In the year under investigation, suppose all generators optimistically predicted high inflows and used all their water in summer. They were right, and no thermal fuel was needed at all. Counterfactual prices are zero.

summer winter


Yearly problem represented by this system

S

N

demand

demandWKO

HAW

MAN

H

demand

EPOC Counterfactual


Cost-to-go recursion DOASA


DOASA: Cutting planes define the future cost functionDOASA


SDDP versus DOASADOASA

SDDP (literature) DOASA

Fixed sample of N openings Fixed sample of N openings

Fixed sample of forward pass scenarios (50 or 200)

Resamples forward pass scenarios (1 at a time)

High fidelity physical model Low fidelity physical model

Weak convergence test Stricter convergence criterion

Risk model (Guigues) Risk model (Shapiro)


p11

p13

p12

How DOASA samples the scenario tree


p11

p13

p12



p11

p13

p21

p21

p21



DOASA run times

0

2

4

6

8

10

12

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Number of forward passes through tree

Co

mp

uta

tio

n t

ime

(ho

urs

)


Why do it this way?Lower bounds converge faster


Why do it this way?Upper bound convergence: 5000 forward simulations


Takeaways

• In this case terminating SDDP after 4, or 5, or even 10 iterations (of 200 scenarios each) does NOT guarantee a close to optimal policy.

• Confidence intervals with 200 scenarios are 5 times bigger than with 5000 scenarios.

• Single forward pass is better as it does not duplicate cut evaluation.

• Iterations slow down as cut sets increase. Cut-set reduction needed.

SDDP


Rolling horizon counterfactual

– Set s=0– At t=s+1, solve a DOASA model to compute a

weekly centrally-planned generation policy for t=s+1,…,s+52.

– In the detailed 18-node transmission system and river-valley networks successively optimize weeks t=s+1,…,s+13, using cost-to-go functions from cuts at the end of each week t, and updating reservoir storage levels for each t.

– Set s=s+13.

Application to NZEM


We simulate an optimal policy in this detailed system

MAN

HAW

WKO

Application to NZEM


Gas and diesel industrial price data ($/GJ, MED)Application to NZEM


Heat rates Application to NZEM


Load curtailment costsApplication to NZEM


Market storage and centrally planned storage New Zealand electricity market


New Zealand electricity market

=(NZ)$12.9 million per year (=2.8% of historical fuel cost)

Estimated daily savings from central plan


Savings in annual fuel costTotal fuel cost = (NZ)$400-$500 million per annum (est)

Total wholesale electricity sales = (NZ)$3 billion per annum (est)

New Zealand electricity market


The next steps

How does risk aversion affect prices and efficiency?

How to model this? Use CVaR (Rockafellar and Urysayev, 2000)

Actually, need a time-staged version of this.

(Ruszczynzki, 2010), (Shapiro, 2010)

Application to NZEM


CVaR1-= Conditional value at risk (tail average)

Application to NZEM

0

0.0045

0 100 200 300 400 500 600 700

Annual fuel+shortage cost ($M)

90%

10%

VaR0.9= $420M

CVaR0.9= $460M


Average 2006 storage trajectories minimizing (1-)E[Z]+CVar(Z)

A risk-averse central planner

0

2000000000

4000000000

6000000000

8000000000

10000000000

12000000000

14000000000

16000000000

1 3 5 7 9 111315171921232527293133353739414345474951

Lambda 0

Lambda 0.5

Lambda 0.9


Total cost - residual water value

150 200 250 300 350 400 450 500 550

(NZ)$M

Lambda 0

Lambda 0.5

Lambda 0.9

“Fuel and shortage cost – residual water value” CDF

A risk-averse central planner

0

1


Conclusions

• DOASA is well-tested tool for benchmarking.• We now have a good empirical understanding of

convergence behaviour.• We can model risk aversion effectively.• Next steps

– include 2008-2009 inflow data– simulate central plans with different levels of risk aversion– How much risk can be avoided for $50M fuel cost?– Examine winter 2008 in more detail – especially price

outcomes.• Interested in feedback from participants – is this

worth pursuing? If so how should industry fund it?

andy philpott epoc (epoc.nz) joint work with vitor de matos, ziming guan

Documents