andy philpott electric power optimization ... - epoc.org.nz · 1winter school, kvitfjell, march 24,...

Introduction The system operator Market participant optimization under uncertainty Optimization by the regulator End

Stochastic programming in energy markets1

Andy PhilpottElectric Power Optimization Centre

University of Auckland.www.epoc.org.nz

1Winter School, Kvitfjell, March 24, 2015


The first of two 2.5 hour lectures(four 1.2 hour lectures with 5 minute breaks)

Stochastic programming in energy markets.Introduction to equilibrium in energy markets.Caveat 1: all applications will be confined towholesale electricity markets.Caveat 2: my talks are both about modeling,not algorithms.

why, who and what questions rather than how.examples to illustrate key ideas rather than theorems.


Welfare theorems of partial equilibrium

First welfare theorem: Any perfectly competitiveequilibrium maximizes welfare.Second welfare theorem: An optimal solution thatmaximizes welfare gives prices that support a competitiveequilibrium.


Key assumptions for welfare theorems

competitionagents need to behave as price takers.

convexitydecentralized decision making needs decomposition.

informationall agents must have the same information.

completenessthere must be enough traded instruments to get effi cientequilibrium.

liquiditylow participation in trading can lead to ineffi ciency.


Stochastic programming in energy markets

Stochastic programming about optimization underuncertainty using probability distributions.Who is optimizing?System operator? Try to improve the effi ciency ofeconomic dispatch under uncertainty.Participants in the market? Try to make as much profit aspossible.Regulator? Try to monitor the economic effi ciency ofexisting markets.


Equilibrium in energy markets (tomorrow’s lecture)

For prediction of market outcomes ex-ante.If every participant (agent) optimizes their profit, thenwhat will happen?This gives rise to a non-cooperative game for which weseek a Nash equilibrium.

a set of actions, one for each agent that is optimal for themgiven the actions of other agents.

How much of a second-best solution will an equilibriumgive compared with the best solution?Depends on which of the five assumptions hold.Dealing with uncertainty in this setting is complicated.


Summary

1 Introduction

2 The system operatorIntroductionOptimization and complementarityStochastic dispatch

3 Market participant optimization under uncertaintyPerfect competitionStrategic behaviour of agentsMarket distribution functionsContracts for differences

4 Optimization by the regulatorMarket monitoring of hydroSolving stochastic control problemsModeling risk aversion in hydrothermal models

5 End


Stochastic programming in system operation

System operator optimizes the economic dispatch ofelectricity.Why does an electricity market need a system operator?

incompletenessilliquiditysecurity

In every existing market design, the economic dispatchproblem is deterministic.The form of the dispatch problem depends on theparticular market design.


Example 1: A dispatch problem in a single location

Cost of dispatch is C (x) = x2. The demand is d(p) = 2− 12p.

The optimal supply curve is marginal cost C ′(x) = 2x .


Example 1: Marginal cost of supply=electricity price

Cost of dispatch is C (x) = x2. The demand at price p isd(p) = 2− 1

2p.

The optimal dispatch is when electricity price satisfies p = 2xand markets clear so 2− 1

2p = x . So p = 2 and x = 1.


Example 2: two locations, inelastic demand

C1(x1) = x21 , C2(x2) = 2x22 , d1(p1) = 1, d2(p2) = 4.


Example 2: two locations, inelastic demand

C1(x1) = x21 , C2(x2) = 2x22 , d1(p1) = 1, d2(p2) = 4.

In node 1 we obtain x1 = 1, p1 = 2. In node 2 we obtainx2 = 4, p2 = 16. Total cost of supply is 33.


Example 3: a dispatch problem with a transmission line

C1(x1) = x21 , C2(x2) = 2x22 , d1(p1) = 1, d2(p2) = 4,

transmission line capacity equals 1.



C1(x1) = x21 , C2(x2) = 2x22 , d1(p1) = 1, d2(p2) = 4, transmission

line capacity equals 1.

In the absence of a price for transmission, different choices off ≤ 1 yield different market outcomes. If f = 0.5 then we getx1 = 1.5, p1 = 3. In node 2 we obtain x2 = 3.5, p2 = 14. Totalcost of supply is 26.75.



C1(x1) = x21 , C2(x2) = 2x22 , d1(p1) = 1, d2(p2) = 4, transmission

line capacity equals 1.

In a market for transmission, the transmission owner chooses fto maximize their surplus by charging a price ρ for each MWhtransported that is at least the difference in price p2 − p1.


Economic dispatch problem

The competitive equilibrium with a market for transmissioncorresponds to the social planning economic dispatch problem:

min x21 + 2x22

s.t. −x1 + f ≤ −1 [p1]−x2 − f ≤ −4 [p2]f ≤ 1 [ρ]

x1, x2, f ≥ 0


Form a Lagrangian

L(x1, x2, f , p1, p2, ρ) = x21 + 2x22 + p1(1− x1 + f )

+p2(4− x2 − f ) + ρ(f − 1)

Minimizing gives

min x21 + 2x22 +p1(1− x1 + f ) +p2(4− x2 − f ) +ρ(f − 1)

s.t. x1, x2, f ≥ 0

whence

min x21 − p1x1 +2x22 − p2x2 +(p1 − p2 + ρ)f

s.t. x1 ≥ 0, x2 ≥ 0, f ≥ 0.


Optimality gives a Karush-Kuhn-Tucker (KKT)system

OPT(x1): 0 ≤ 2x1 − p1 ⊥ x1 ≥ 0OPT(x2): 0 ≤ 4x2 − p2 ⊥ x2 ≥ 0OPT(f ): 0 ≤ ρ− p2 + p1 ⊥ f ≥ 0

Market(node 1): 0 ≤ −1+ x1 − f ⊥ p1 ≥ 0Market(node 2): 0 ≤ −4+ x2 + f ⊥ p2 ≥ 0

Market(transmission): 0 ≤ 1− f ⊥ ρ ≥ 0

The symbol "⊥" is pronounced "perp" and a ⊥ b means a>b = 0.


Complementarity problems

Problems involving the simultaneous solution of KKT systems are calledcomplementarity problems. This is because of complementary slacknessconstraints of the form

0 ≤ 2x1 − p1, x1 ≥ 0, x1(2x1 − p1) = 0.

Solving such systems by hand can be very laborious because of all thecases that need checking. There is now very good software for solvingthese in GAMS and AMPL.

The most popular model is PATH due to Michael Ferris available inGAMS. This requires the problems to be cast in the form of a mixedcomplementarity problem.


Mixed complementarity problems

Given vectors l and u in Rn and a mapping F : Rn → Rn, PATHfinds a z in Rn such that for each i = 1, 2, ..., n, exactly one of thefollowing conditions holds:

1 Fi (z) = 0, and li ≤ zi ≤ ui ,2 Fi (z) > 0, and zi = li ,3 Fi (z) < 0, and zi = ui .


Formulating problems in GAMS/PATH

In many GAMS/PATH models we can write the systems so that allvariables and constraints are ≥ 0. This corresponds to li = 0, andui = ∞, and gives the following general formulation:

Given a mapping F : Rn → Rn+, PATH finds a z in Rn such that for

each i = 1, 2, ..., n, exactly one of the following conditions holds:

1 Fi (z) = 0, and zi ≥ 0,2 Fi (z) > 0, and zi = 0.

In other words0 ≤ F (z) ⊥ z ≥ 0


GAMS file for dispatch complementarity problem


GAMS Solution

Solution is x1 = 2, x2 = 3. Cost is 4+ 18 = 22 . Prices aremarginal cost of supply of p1 = 4, and p2 = 12. Transmissionprice ρ = 8.


KKT system defines competitive equilibrium

OPT(x1): 0 ≤ 2x1 − p1 ⊥ x1 ≥ 0OPT(x2): 0 ≤ 4x2 − p2 ⊥ x2 ≥ 0OPT(f ): 0 ≤ ρ− p2 + p1 ⊥ f ≥ 0

Market(node 1): 0 ≤ −1+ x1 − f ⊥ p1 ≥ 0Market(node 2): 0 ≤ −4+ x2 + f ⊥ p2 ≥ 0

Market(transmission): 0 ≤ 1− f ⊥ ρ ≥ 0


Competitive equilibrium

Generator 1 generates x1 = 2 where marginal costC ′1(x1) = 2x1 = 4.Generator 2 generates x2 = 3 where marginal costC ′2(x2) = 4x2 = 12.Transmission owner offers transmission services at $8 perMWh.Demand buys electricity at prices of p1 = 4, and p2 = 12.Demand at node 1 buys 1 MWh at p1 = 4.Demand at node 2 buys 3 MWh from generator 2 atp2 = 12 and 1MWh from generator 1 at node 1 at p1 = 4,but pays ρ = 8 to the transmission owner to transport 1MWh.


What is the takeaway message?

Economic dispatch problem is a surrogate for markettransactions assuming competitive prices for services thatmight not be easily provided in real time by auctionmechanisms (e.g. transmission).The surrogate works because of the welfare theorems ofeconomics but we need assumptions to make these match.In the example, a price for transmission was needed tomake the market effi cient.


Takeaway questions

Question 1: what about nonconvexity e.g. integervariables from unit commitment?Question 2: what about lack of information (uncertainty)?Question 3: what about incompleteness (e.g. lack ofinstruments to trade risk )?Question 4: What about illiquidity (e.g. from verticalintegration of generators and retailers)?Question 5: What about strategic behaviour (e.g.generators trying to influence prices)?


What about integer variables?

Integer variables come from unit commitment. Unitcommitment decisions in general cannot be coordinatedby price.Solution approach 1 (e.g. PJM): arrange these ahead oftime by solving a mixed integer program in a day-aheaddispatch.This gives a least cost dispatch by definition but extraside payments are needed to provide the right incentives(as a market would).Solution approach 2 (e.g. Australia): require generators tobid a "marginal cost" that accounts for start-up costs, andtreat the dispatch problem as convex (a linear program).


What about uncertainty?

Markets are implemented to aim for maximum welfare ofparticipants, so it is natural to seek a market structurethat might achieve this in expectation.Lots of literature on stochastic programming dispatchmodels that minimize expected cost, and robustoptimization models that give best worst-caseperformance over some uncertainty set.Question: do these models make sense in a marketcontext where probability distributions of participants arebound to differ?Status quo: "self-dispatch" approach ignores uncertaintyin dispatch and makes market particpants take care oftheir own uncertain future.Mooted: "central-dispatch" approach uses a publiclyagreed probability distribution to model uncertainty in astochastic program.


Example of stochastic dispatch problem(Pritchard et al, 2010)

This model was discussed at length in the talk by Endre Bjorndahl.

SP:min ∑i(pixi +E[p+i (Xi (ω)− xi )+ − p−i (xi − Xi (ω))+]

)s.t. [πn ] τn(f ) +∑i∈n xi = 0, n ∈ N,

[P(ω)λn(ω)] τn(F )− τn(f ) +∑i∈n(Xi (ω)− xi ) = 0, n ∈ N,ω ∈ Ω,

f ∈ U, (xi ,Xi (ω)) ∈ Ci , F (ω) ∈ U, ω ∈ Ω.


Stochastic dispatch problem


Discussion

Wong and Fuller (2007) show that λn(ω) is also anoptimal dual variable for the second stage problem inscenario ω with fixed first-stage decisions.So a generator dispatched xi at node n in the first stage ispaid πnxi . Given the first stage dispatch and outcome ω,a generator dispatched Xi (ω) at node n in the secondstage is paid an extra λn(ω)(Xi (ω)− xi ).At these prices, each agent solves a two-stage stochasticprogram to maximize profit. This might involve first-stagedispatch out of merit order, but agents are always in themoney, in expectation.


Remarks

Dispatch and shadow prices yield higher average prices forflexible plant e.g. hydro, and lower average prices forintermittent plant e.g. wind. Incentives to build flexibleplant.Dispatch and shadow prices of SP are revenue adequate inexpectation, but not in every scenario. Zavala et al (2014)study implications of `1 penalty on market outcomes.Question 1: Do all participants share P?Question 2: What are market implications of otherparadigms (robust, chance-constrained)?


Summary

1 Introduction




5 End


Optimization of market participants under uncertainty

Generators maximizing expected profit.Consumers minimizing expected costs.How does risk aversion change this? robustness?What about the information participants have?Start by discussing perfect competition.


The generator problem

Choose a supply function q(p) to maximize profit. Inversesupply function is p(q).In Example 1, the supply function of generator 1 was itsmarginal cost of production defined by q(p) = p. Here thequantity q is offered at p(q) = q, its marginal cost atquantity q.Many markets require p(q) to be a step function definedby a finite set of (quantity,price) pairs. This allows thedispatch problem to be solved as a linear program (or aMIP if there are integer variables).


Perfect competition

Generators do not anticipate the effect of their action onmarket price.The optimal supply function q(p) is then the marginalcost of generation.In this morning’s example, the supply function ofgenerator 1 was its marginal cost of production defined byq(p) = p. Here the quantity p is offered at its marginalcost of p.


What is the marginal cost of a hydro generator?

In perfect competition, agents estimate a stochasticprocess to model future prices.

To coordinate a market, this process might be quite poorlybehaved (Barty et al, 2010).

Solve an (approximate) dynamic programming problemthat maximizes expected profit using the stored water.Offer generation at the expected marginal value of storedwater divided by station production effi ciency (MWh/m3).Many variations on this idea accounting for uncertaininflows, flow constraints, cascading river chains, headeffects, ...In practice, there is a monotonicity problem (Pritchard etal, 2002).


Example 1: Marginal cost approximation(Neame et al, 2003)

Turbine production functions forone, two, and three turbines inoperation in a hydro station.Cusp points correspond toswitching on a new turbine.

Corresponding marginal costfunction is not monotonic.

Globally optimal 5-step supplyfunction is found by dynamic

programming.


Example 2: Bidding model for a single reservoir - HERO(Pritchard et al, 2005)

Consider a single hydro reservoir in a market with randomelectricity prices and random inflows. Assume that we offer thesame curve q(p) in each hour of a week. How to choose q(p)?

The dispatch in each hour depends on the observed price p from a priceduration curve with three sections. Here

P(p ∈ [pi−1, pi ) | Ai ) = Ai (pi − pi−1) where Ai is random and∑i Ai (pi − pi−1) = pm − p0.


Example 2: HERO model (intra-week optimization)

Let ai = E [Ai ] and vij =cov(Ai ,Aj ). Then

g (µ, σ) = max ∑mi=1 ai

∫ pipi−1

pq (p) dps. t. q : [p0, pm ]→ [0, qmax] is a non-decreasing function

Ii =∫ pipi−1

q (p) dp, i = 1, . . . ,m,a>I = µ,

I>VI ≤ σ2,

is the maximum revenue earned from a water release policy thatyields total weekly release with mean µ and variance σ2.


Example 2: HERO model (inter-week optimization)

For a range of (µ, σ) values compute tables for g (µ, σ) in eachweek of the year, and a distribution for the random water releaseUt (µ, σ), we solve a dynamic programming recursion.

Vt (x) = maxµ,σ (gt (µ, σ) +E [Vt+1(x +Wt − Ut (µ, σ))]) .


Example 3: Some of many bidding models for river chains

River chains with Markov prices (Pritchard et al, 2002).River chains with nonconvexities (Conejo et al, 2002).Stochastic mixed integer programming models using scenarios(Fleten et al, 2007).Linear decision rules for reservoir releases (Gronvik et al,2014).MIDAS (Wahid et al, 2014).


Strategic behaviour

Generators anticipate the effect of their action on marketprice to maximize profit.(Price-maker) optimization models need to account forthis.Model is based on the residual demand curve.


Example 1: Best response to a competitor

Cost of dispatch x is C (x) = x2. The demand is fixed at d . Acompetitor offers a supply stack as shown.


Example 1: Residual demand at price p

Residual demand function is d − q(p).



Generator can undercut competitor at its lowest offer price p1 byoffering amount d at price p1 − ε to result in revenue (≈ p1 ∗ d)

equal to green area.



Generator can undercut competitor at its second lowest offer pricep2 by offering amount x at price p2 − ε to result in revenue

(≈ p2 ∗ x) equal to green area. This has lower cost (x2 < d2) andmore revenue than previous solution and is optimal.


Remarks

If we offer at price p1instead of undercutting at p1 − εthen under most market sharing rules, we will have toshare revenue with the competitor. If we offer a singlequantity d at price 0, then we make no revenue. Offeringd − ε at price 0, we make p1(d − ε).The revenue function is not upper semi-continuous as afunction of price, and it is not upper semi-continuous as afunction of quantity.Mathematically, the deterministic optimization problemdoes not have a solution. It is is not well posed.In practice, undercutting behaviour can be observed (e.g.agents offer prices of $49.99/MWh).Finitely many scenarios won’t help too much here - someMIP bidding models just ignore this.Modeling with continuous probability distributions canhelp resolve this mathematically.


Supply function auction with uncertain demand

Suppliers offer supply functions qi (p) indicating what they willsupply at price p. Denote the inverse supply curves by pi (q).Demand is D(p) + h, where h is a random shock. When h isrealized the markets clear at a price p defined by

∑iqi (p) = D(p) + h.


Dispatch point is random (single node)

If other suppliers offer supply functions qj (p) for j 6= i , then indemand shock h, supplier i faces a residual demand curveD(p)−∑j 6=i qj (p) + h. The supply function qi (p) is dispatchedat p where

qi (p) = D(p)−∑j 6=iqj (p) + h.

The profit (rental) earned by supplier i under this outcome isdenoted R(q, p). For example under uniform pricing,R(q, p) = qp − C (q).


Market distribution functionAnderson and Philpott (2002), Wilson (1979)

The residual demand curve is random. The market distributionfunction ψ(q, p) defines the probability that the residualdemand curve passes below and to the left of the point (q, p).

Random residual demand curves faced by a supplier. Here ψ(q, p)is the probability of a curve being red or

Pr(q > D(p)−∑j 6=i qj + h).


Profits at uniform pricesAnderson and Philpott (2002)

The optimal offer curve p(q) for a supplier with profit R(q, p)facing a market distribution function ψ(q, p) maximizes

E[R ] =∫ qM

0R(q, p(q))dψ(q, p(q)) +

∫ P

p(qM )R(qM , p)dψ(qM , p)

+R(qM ,P)(1− ψ(qM ,P))


Optimality conditions for p(q)

q

p

Example showing contours of ψ(q, p) (black). Isoprofit lines for R(q, p)are red. The optimal curve q = S(p) (blue) passes through the points

where these curves have the same slope. (Rq/Rp = ψq/ψp).

Blue curve satisfies the Euler Lagrange equation Z (q, p) = 0where

Z (q, p) =∂R(q, p)

∂q∂ψ(q, p)

∂p− ∂R(q, p)

∂p∂ψ(q, p)

∂q.


Example

Suppose two generators and generator 2 offers q2(p) = p. Findthe best supply function response q(p) when marginal cost is cand inelastic demand h is uniformly distributed on [0, 1].Market distribution function is

ψ(q, p) = Pr(q + q2(p) > h)

= q + p

Z (q, p) = (p − c)∂ψ(q, p)∂p

− q ∂ψ(q, p)∂q

= (p − c)− q = 0.

p(q) = q + c


Proof of global optimality

Z (q, p) =∂R(q, p)

∂q∂ψ(q, p)

∂p− ∂R(q, p)

∂p∂ψ(q, p)

∂q.

Green’s theorem states∫ ∫SZ (q, p) =

∫CR(q, p)dψ(q, p)

Integral of Z over yellow region is negative and so any deviation belowblue Euler curve will decrease expected profit.


Global optimality conditions for p(q)

Z (q, p) =∂R(q, p)

∂q∂ψ(q, p)

∂p− ∂R(q, p)

∂p∂ψ(q, p)

∂q

If the solution toZ (q, p(q)) = 0

is a monotonic nondecreasing curve, and

∂

∂qZ (q, p) ≤ 0

then p(q) is a globally optimal solution to maximizingE[R(q, p)].


Ironing p(q)

If Z (q, p(q)) = 0 is not increasing, then optimal solution isobtained by ironing, so that∫ x

qAZ (q, p)dq ≤ 0,

∫ qB

qAZ (q, p)dq = 0

If∫ xqAZ (q, p)dq > 0 then integral of Z over yellow region is positive

and so any deviation below ironed curve will increase expected profit.


Risk

Electricity prices are volatile.Generators hate low prices.Consumers hate high prices.Markets have derivative products to hedge price risk.Contracts for differences (CFDs) are the most popularderivative.A single CFD at price f is a bet on the future spot priceπ. It pays f−p to the writer of the contract and p−f tothe purchaser.


CFDs under perfect competition

The payout of a CFD is f − p to the writer of the CFDand p − f to the purchaser.The offer curve of an agent does not affect the price.Given a contract price f , whatever they do will not affectthe contract payout.In any given price realization agents should optimizeconsumption or production as if they do not hold acontract.


CFD myths under perfect competition

Myth 1: Generators who have written a CFD for Q MWshould ensure that they are dispatched at least Q to"meet the contract".

the contract pays a coupon independent of generator’sdispatch (assuming this does not affect price).generation should be offered at marginal cost.

Myth 2: Consumers holding a CFD pay a fixed price fortheir energy consumed so they have no need to peakshave.

the contract pays a coupon independent of purchaser’sconsumption (assuming this does not affect price).purchaser should reduce load if observed price is higher thaneconomic (even if they are contracted for this load). They savefrom peak-shaving without affecting contract payment.


Contracts with imperfect competition

Each generator writes Q contracts that each pay (f − p) forsome fixed contract price f .

R(q, p) = pq − C (q) +Q(f − p)ψ(q, p) = Pr(h < q + qj (p)−D(p)) = F (q + qj (p)−D(p))

then

0 =∂R(q, p)

∂q∂ψ(q, p)

∂p− ∂R(q, p)

∂p∂ψ(q, p)

∂q=

(p − C ′(q)

)(q′j (p)−D ′(p))f(q+ qj(p)−D(p))

−(q −Q)f(q+ qj(p)−D(p)).

q = Q +(p − C ′(q)

)(q′j (p)−D ′(p))


Contracts with supply functions

Without contract

q =(p − C ′(q)

)(q′j (p)−D ′(p))

After writing Q contracts, we have

q = Q +(p − C ′(q)

)(q′j (p)−D ′(p))

so the generator uncontracted supply curve shifts Q to theright (assuming the other generator does not changeoffer).We will hear more about this tomorrow.


Optimizing in practice

For optimizing a single offer curve estimate a marketdistribution function by simulation and then solve aprize-collecting dynamic program to compute an optimal stepfunction.

Example of candidate offers at a New Zealand generator. The redcurve is optimal if uncontracted, and the black curve is with retail

load and contracts.


Summary

1 Introduction




5 End


Optimization by a regulator

How do we know that markets are competitive?One approach: compute perfectly competitive prices as abenchmark.How? perfectly competitive partial equilibrium optimizes asocial planning problem...

...so we can compute equilibrium by solving a suitableoptimization problem.

What about systems with hydroelectricity (Brazil, NewZealand, Norway)?We use SDDP to compute perfectly competitive prices asa benchmark.


Suppose a scenario tree models uncertain inflows

Each node n spans a period (week) and corresponds to arealization ω(n) of reservoir inflows in that period.


Social plan minimizes total expected system disbenefit

SSP: min ∑n∈N φ(n)(∑j∈T fj (vj (n))−∑c∈C cc (dc (n))

)+∑n∈L φ(n)∑i∈H Qi (xi (n))

s.t. ∑i∈H gi (ui (n)) +∑j∈T vj (n) ≥ ∑c∈C dc (n), n ∈ N ,

xi (n) = xi (n−)− ui (n)− si (n) +ωi (n), i ∈ H, n ∈N ,

u(n) ∈ U , v(n) ∈ V , x(n) ∈ X , s(n) ∈ S .


Social plan = risk neutral perfectly competitive equilibrium

To minimize Lagrangian for social plan with Lagrangemultipliers φ(n)p(n) we solve each agent problem separately.

HP(i):max ∑n∈N φ(n)p(n)gi (ui (n))−∑n∈L φ(n)Qi (xi (n))s.t. xi (n) = xi (n−)− ui (n)− si (n) +ωi (n), n ∈ N ,

ui (n) ∈ Ui , xi (n) ∈ Xi , si (n) ∈ Si .

TP(j): max ∑n∈N φ(n)(p(n)vj (n)− fj (vj (n))s.t. vj (n) ∈ Vj .

CP(c): max ∑n∈N φ(n) (cc (dc (n))− p(n)dc (n)) .


Social plan = risk neutral perfectly competitive equilibrium

This defines a perfectly competitive equilibrium defined by theindividual optimality conditions and market clearing condition.

CE: ui , xi , si ∈ argmaxHP(i),

vj (n) ∈ argmaxTP(j),

dc (n) ∈ argmaxCP(c),

0 ≤ ∑i∈H gi (ui (n)) +∑j∈T vj (n)−∑c∈C dc (n) ⊥ p(n) ≥ 0.

Solutions to CE can be computed in principle as an equilibriumproblem but it is easier to solve SSP.


How to enact the social plan in practice

Solve the stochastic social planning problem andimplement it (Quebec, British Columbia, France, ...)

OR

Solve the stochastic social planning problem and use it todetermine prices for trading (Brazil, Chile, ...)

OR

Establish a market and let solution to SSP emerge fromtrading (Nordpool, Spain, Colombia, NZ, ...).


Solving the real-world stochastic social planning problem

We seek a policy of releasing water from hydro reservoirs andgenerating from thermal plant that minimizes total generatorand consumer disbenefit. Most models assume an essentiallystagewise independent random inflow process ωt and minimizeexpected disbenefit. Given initial storage x0, this is astochastic control problem:

SSP: min E[∑Tt=1 (f (Vt )− c(Dt )) +QT (XT )]

s.t. Xt = xt−1 − Ut +ωt − St , t = 1, 2, . . . ,T ,

g(Ut ) + Vt ≥ Dt , t = 1, 2, . . . ,T ,

Ut ∈ U , Vt ∈ V , Xt ∈ X , St ∈ S , t = 1, 2, . . . ,T .


Dynamic programming

Assume we observe inflows at the beginning of each stage. LetQt (xt−1,ωt ) be the expected disbenefit from an optimal policythat starts in stage t with reservoir volumes xt−1 and observedinflows ωt . This Bellman function satisfies

Qt (xt−1,ωt ) = min f (vt )− c(dt ) +E[Qt+1(xt ,ωt+1)]

s.t. xt = xt−1 − ut +ωt − st ,

g(ut ) + vt ≥ dt ,

ut ∈ U , vt ∈ V , xt ∈ X , st ∈ S .


How does one solve this dynamic programming problem?(Shapiro, 2011)

In general it is impossible to solve SSP, so we mustapproximate it. When we devise algorithms to solve itsapproximation, we must be clear about what problem weare solving.Given historical inflow data first construct a model thatcan be simulated. We call this the true problem.The true problem is an approximation of reality butprovides a means to test candidate policies by simulation.Easiest model: ωt has a known distribution (e.g.lognormal) and is stagewise independent.Many implementations assume some form of periodicautoregressive process, with stagewise independent errors.Next step is to construct an approximately optimal policy.


Form a scenario tree approximation

Assume stagewise independence for convenience.Sample the random variables in each stage giving asample average approximation (SAA).Independence means we can decouple this sampling i.e.conditional sampling not needed.Gives an astronomically large but finite scenario tree. Theresulting mathematical program is afinite-dimensional convex program.The tree is too big to work with directly using stochasticprogramming techniques.Use (approximate) dynamic programming assumingconvex stage problems (typically LP) and sampling.


Summary


Exact Bellman function computation

Qn(x(n),m) = min(u,v ,d ,x ,s)∈R

∑j∈T

fj (v(m))− ∑c∈C

cc (d(m))

+Ep∈m+[Qm(x(m), p)]


Approximation of Bellman function using cutting planes

Qm(x(n),m) = min(∑j∈T fj (vj (m))−∑c∈C cc (dc (m))

)+ θ(m)

s.t. ∑i∈H gi (ui (m)) +∑j∈T vj (m) ≥ ∑c∈C dc (m),xi (m) = xi (n)− ui (m)− si (m) +ωi (m),θ(m) ≥ α(l ,m) + β(l ,m)>x(m), l = 1, 2, . . . , k,u(m) ∈ U , v(m) ∈ V , x(m) ∈ X , s(m) ∈ S .

β(k, n) =E[∇x Qm(x(n),m)],α(k, n) = E[Qm(x(n),m)] + β(k, n)

>x(n)


E[Qm(x(n),m)] is approximated by θ (x) the pointwise maximumof affi ne functions α(l , n) + β(l , n)>x . Stagewise independent

inflows allows cut sharing in each stage.


Stochastic Dual Dynamic Programming (SDDP)(Pereira & Pinto, 1991, Shapiro, 2011)

SDDP finds an approximately optimal solution to thefinite problem. Improve approximate Bellman functions bysimulating the policy on sample paths in the tree and addingcutting planes at visited states. This gives...

a candidate policy for the true problem;a lower bound on the expected value of an optimal policyfor the finite problem;a lower bound in expectation for the value of the optimalpolicy for the true problem.

(Many variations on this idea exist in the literature (CUPPS, Chen& Powell), AND (Birge & Donohue), DOASA (Guan))

Termination: Run until the lower bound for the finite problemis close to a sample estimate of the expected cost of acandidate policy.


Convergence to solution of SAA(Philpott & Guan, 2008, Giradeau et al, 2014)

SDDP converges to a solution of the SAA problem withprobability 1 when the forward path outcomes are sampledindependently in each iteration.When the stage problems are linear programs, the numberof different cutting planes computed by SDDP can beenumerated.Convergence follows by observing that every outcome willbe visiited infinitely often, so eventually all necessarycutting planes will be created (Philpott and Guan, 2008)When the stage problems are general convex optimizationproblems SDDP also can be shown to converge using amore technical argument (Giradeau et al, 2014).


Expected marginal water values

Estimating marginal water values is diffi cult with noisy valuefunctions.

Lake Pukaki estimated marginal water values.


Backtest estimate of prices over period 2005-2008

South Island prices in market (pink) and counterfactual (green)


SDDP with risk aversion

Water shortages in hydro systems like Chile, Brazil,Colombia, New Zealand are very expensive as there is noway to easily import the volumes of electrical energyneeded in a drought.SDDP can be formulated with a risk-averse objectiveusing a coherent risk measure.We give a brief account of this here.Tomorrow we look at how this relates to competitiveequilibrium. (See also Yves Smeers talk tomorrow).


Coherent risk measures[Artzner et al, 1999]

A coherent risk measure is a mapping ρ from a space Z ofrandom variables to R that satisfies the following axioms forZ1 and Z2 ∈ Z .Subadditivity: ρ(Z1 + Z2) ≤ ρ(Z1) + ρ(Z2);Monotonicity: If Z1 ≤ Z2, then ρ(Z1) ≤ ρ(Z2);Positive homogeneity: If λ ∈ R and λ > 0, then

ρ(λZ1) = λρ(Z1);

Translation equivariance: If λ ∈ R, then

ρ(Iλ+ Z1) = λ+ ρ(Z1).


Dual representation of coherent risk measures(Artzner et al, 1999, Shapiro & Ruszczynski, 2006)

A coherent risk measure of a random disbenefit Z can beexpressed as

ρ(Z ) = supµ∈D

Eµ[Z ]

where D is a convex set of probability measures called the riskset.


Example: three outcomes

Consider possible disbenefit outcomesZ (ω1) < Z (ω2) < Z (ω3). The coherent risk measure

ρ(Z ) =34

E[Z ] +14max(Z )

corresponds to

D=conv(12,14,14), (14,12,14), (14,14,12)

ρ(Z ) = maxµ∈D

Eµ[Z ] =14Z (ω1) +

14Z (ω2) +

12Z (ω3)


Scenario trees and risk measures

Each node m corresponds to a realization ω(m) of reservoirinflows and disbenefit Z (m) in that period.


Dynamic risk measures(Epstein &Schneider, 2003, Artzner et al 2007, Ruszczynski, 2010)

Consider a random sequence of disbenefits Z (n) correspondingto the nodes of the scenario tree. Each node n ∈ N \ L in thescenario tree is endowed with a risk set D(n). Thedynamic risk measure we will use is constructed recursively asfollows. For every leaf node we set the risk-adjusted disbenefit

ρ(n) = Z (n)

and for every other node we set

ρ(n) = Z (n) + maxµ∈D(n) ∑

m∈n+µ(m)ρ(m).


Dynamic optimization under risk

Suppose each node n ∈ N \ L in the scenario tree has risk setD(n). We seek a policy (actions ui (n), xi (n), si (n),vj (n),dc (n))giving disbenefits Z (n) that minimize risk-adjusted disbenefitρ(1), where

ρ(n) = Z (n), n ∈ L,and for every other node we set

ρ(n) = Z (n) + maxµ∈D(n) ∑

m∈n+µ(m)ρ(m).


SDDP with a dynamic coherent risk measure(Philpott and deMatos, 2011, Shapiro et al 2012, Philpott et al, 2013)

The Bellman function at x(n) for scenario m ∈ n+ isapproximated by

Qm(x(n),m) = min(∑j∈T fj (vj (m))−∑c∈C cc (dc (m))

)+ θ(m)

s.t. ∑i∈H gi (ui (m)) +∑j∈T vj (m) ≥ ∑c∈C dc (m),xi (m) = xi (n)− ui (m)− si (m) +ωi (m),θ(m) ≥ α(l ,m) + β(l ,m)>x(m), l = 1, 2, . . . , k.

Here θm gives an outer approximation of the risk adjustedfuture disbenefit ρ(m) in child node m.

µ∗ ∈ arg maxµ∈D(n) ∑

m∈n+µ(m)Qm(x(n),m),

β(k , n) = Eµ∗ [∇x Qm(x(n),m))],α(k , n) = Eµ∗ [Qm(x(n),m)] + β(k, n)>x(n).


Applied to New Zealand system

Simulated volume of water in Lake Pukaki from May 20, 2008 -May 19, 2009, under three release policies. d Many more examples

and papers on this are at www.epoc.org.nz.


Risk averse prices 2005-2008

Weekly average South Island prices from risk averse model with λ = 0.5(green) compared with historical Benmore prices (pink).


Models we have looked at today

Stochastic dispatch for wind.Strategic best response in supply functions.Risk-averse SDDP.All these optimize for a single player.Tomorrow we look at equilibrium.

Nash equilibrium in Cournot games.Supply function equilibrium.Risked competitive equilibrium.


Summary

1 Introduction




5 End


This is the end of tutorial 1

THE END OF TUTORIAL 1


References

Anderson, E. J., & Philpott, A. B. Optimal offer construction inelectricity markets. Mathematics of Operations Research, 2002.Artzner, P. Delbaen, F., Eber, J-M. , & Heath, D., Coherent measures ofrisk, Mathematical Finance, 9, 1999.Barty, K., Carpentier, P., Cohen, G., & Girardeau, P. Price decompositionin large-scale stochastic optimal control. arXiv preprint arXiv:1012.2092,2010.Chen Z.L., Powell W.B. A convergent cutting-plane and partial-samplingalgorithm for multistage linear programs with recourse. J. Optim. TheoryAppl. 1999.Donohue C.J., Birge J.R. The abridged nested decomposition method formultistage stochastic linear programs with relatively complete recourse.Algorithmic Oper. Res. 2006.L.G. Epstein & M. Schneider, Recursive multiple-priors. JET, 2003.Fleten, S. E., & Kristoffersen, T. K. Stochastic programming foroptimizing bidding strategies of a Nordic hydropower producer. EuropeanJournal of Operational Research, 2007.


References

Girardeau, P. et al., On the convergence of decomposition methods formulti-stage stochastic convex programs, Math of OR, 2014.Gronvik, I., Hadziomerovic, A., Ingvoldstad, N., Egging, R., & Fleten, S.E., Feasibility of linear decision rules for hydropower scheduling. InProbabilistic Methods Applied to Power Systems (PMAPS), 2014.Neame, P. J., Philpott, A. B., & Pritchard, G., Offer stack optimizationin electricity pool markets. Operations Research, 2003.Pereira, M. & Pinto, L., Multi-stage stochastic optimization applied toenergy planning, Math Prog, 1991.Philpott, A.B. & Guan, Z., On the convergence of stochastic dualdynamic programming and related methods, OR Letters, 2008.


References

Philpott, A.B. & de Matos, V.L., Dynamic sampling algorithms formulti-stage stochastic programs with risk aversion, EJOR, 2011.Philpott, A.B., et al, On solving multistage stochastic programs withcoherent risk measures Op Res, 2013.Pritchard, G., & Zakeri, G. Market offering strategies for hydroelectricgenerators. Op Res, 2003.Pritchard, G., Zakeri, G., & Philpott, A. A single-settlement, energy-onlyelectric power market for unpredictable and intermittent participants. OpRes, 2010.Ruszczynski, A., Risk-averse dynamic programming for Markov decisionprocesses, Math Prog, 2010.


References

Shapiro, A., Analysis of stochastic dual dynamic programming method,EJOR, 2011.Shapiro, A., et al, Risk neutral and risk averse Stochastic Dual DynamicProgramming method, EJOR, 2012.Shapiro, A., & Ruszczynski, A., Optimization of convex risk functionsMath of OR, 2006.Wong, S., & Fuller, J. D., Pricing energy and reserves using stochasticoptimization in an alternative electricity market. Power Systems, IEEETransactions on, 2007.Zavala, V., Anitescu, M. & Birge, J.R. A Stochastic electricity marketclearing formulation with consistent pricing properties,ANL/MCS-P5110-0314, 2014.

andy philpott electric power optimization ... - epoc.org.nz · 1winter school, kvitfjell, march 24,...

Documents