Department of Telecommunications

MASTER THESIS Nr. 608

INTELLIGENT TRADING AGENT FOR POWER TRADING THROUGH

WHOLESALE MARKETIvo Buljević2012/2013

Zagreb, July 2013

Department of Telecommunications

Contents

¨ Introduction¨ Smart grid¨ Wholesale market¨ CrocodileAgent 2013¨ Conclusion

Zagreb, July 2013 2 of 12

Department of Telecommunications

Introduction

¨ Characteristics of the traditional energy market: Centralized Vertically integrated market structure No competition

¨ Liberalization and deregulation of the traditional energy market

¨ Increased number of renewable energy sources ¨ Progressive transformation of traditional power

systems into evolved systems called smart grids

Zagreb, July 2013 3 of 12

Department of Telecommunications

Smart grid

¨ A modernization concept of the electricity delivery system¨ Enables real-time banacing of energy supply and demand¨ A two-way flow of electricity and information

Zagreb, July 2013 4 of 12

¨ Multi-agent market models Entities are represented by

intelligent software agents Opportunity to test software

solutions in order to prevent market crashes (California 2001)

Department of Telecommunications

Wholesale market

¨ Result of liberalization and deregulation of the traditional energy market, enables energy trade between market entities

¨ Power exchanges and power pools¨ Day-ahead market¨ Examples of wholesale markets:

Chile Great Britain and Wales Nord Pool California

Zagreb, July 2013 5 of 12

Department of Telecommunications

Wholesale market (2)

¨ Energy load forecasting Statistical approach

Similar-day method Exponential smoothing Regression methods

Artifficial intelligence – based tecniques Reinforcement learning

¨ Energy price forecasting Spike preprocessing Time series models with exogenous variables Interval forecasts

Zagreb, July 2013 6 of 12

Department of Telecommunications

CrocodileAgent 2013

¨ Intelligent software agent developed at University of Zagreb

¨ Participant of PowerTAC 2013¨ Main emphasis:

Zagreb, July 2013 7 of 12

Development of wholesale bidding strategy which will minimize negative effects on the balancing market

Responsive and context-aware agent design

Department of Telecommunications

CrocodileAgent 2013Modular architecture

Zagreb, July 2013 8 of 12

CLEARINGINFORMATION

WHOLESALE MARKET

CUSTOMER MARKET

GENCOS OTHER BROKERS

C1 C2 C3

WEATHER

PAST ENERGY USAGE

ALL FORECASTED DATA

PAST CLEARING PRICES

BID/ASK TARIFFS

CONSUMPTION PRODUCTION INTERUPTABLE CONSUMPTION

Office complexVillage types

Centerville homes

SolarWind

Frosty storageHeat Pump

FORECAST MANAGER

TARIFF MANAGER MARKET MANAGER

MARKET REPOSITORY

TARIFF REPOSITORY

MAIN SERVICE (MESSAGE SENDER/

RECEIVER)

OTHER TARIFF SPECIFICATION, TRANSACTION

PUBLISH TARIFFS

PASTUSAGE

FUTURE ENERGY USAGE/PRICES

CURRENT WHOLESALESTATE

BIDS/ASKS

SEND TO SERVER

CrocodileAgent 2013

LEARNING MODULE

BIDDING STRATEGIES

GENERATEDORDERS

ENERGYPRICES

NEEDEDENERGY

Contribution of this master thesis

Department of Telecommunications

CrocodileAgent 2013Learning module

¨ Based on reinforcement learning Erev-Roth method specially adapted for PowerTAC

wholesale market¨ Enables broker to adapt to various market

conditions¨ Key features:

Zagreb, July 2013 9 of 12

Multiple strategies Advanced strategy

evaluation based on its efficiency

RL module Simulator

InitializationChoose strategy

ExecuteResults

Set rewards

Department of Telecommunications

CrocodileAgent 2013Learning module (2)

¨ Uses basic order as an input Generated by forecast module, based on past usage of

subscribers on the retail market Holt-Winters method

¨ Life cycle:

Zagreb, July 2013 10 of 12

Initialization Choose strategy Place order Set reward

¨ Strategies used to model amount of energy and unit price

Department of Telecommunications

CrocodileAgent 2013Results

¨ Broker progressively learns to adapt to current market conditions – manifestation of the learning period Minimization of balancing cost

¨ Broker buys an excessive amount of energy on the wholesale market

Zagreb, July 2013 11 of 12

Results from May trial indicates that broker buys 125% of energy needed on the retail market

A need to optimize basic order generation (energy load forecasting)

Department of Telecommunications

Conclusion

¨ Robustness of the CrocodileAgent’s wholesale module Broker is able to adapt to changes in competition

environment¨ Adapted Erev-Roth algorithm was proved to be

suitable for the PowerTAC wholesale market¨ Future work:

Improvement of energy load forecasting Improvement in unit price calculation Design of intelligent strategies

Zagreb, July 2013 12 of 12