consideration of communication time delays in … › powertech-2019 › wp-content ›...

College of Engineering and Architecture

Consideration of CommunicationTime Delays in

Wide-Area Control

Anjan BoseWashington State University

Pullman, WA, USA

IEEE Powertech 2019

Milan, Italy

June 26, 2019


New England Synchrophasor System

2

PhasorPoint

ROSE

ISO-NE

PDC

DQMS

External

Entity

ISO-NE

Network

• 44 stations

• 90 PMUs

• 429

Phasors

• NYISO

• PJM

• MISO


Challenges of the PMU Data Exchange Scheme

• Raw synchrophasor data exchange only

Large volumes of data• Bandwidth: cost, performance

High maintenance• Each entity has to model and maintain its own and other region’s PMU data

• There is no central PMU Registry

• No data quality information or outage status

Lacks coordination• Each entity processes and analyzes data separately (vendors and applications)

• Operators see different results and displays

• Interpretation discrepancy

• Bi-lateral data exchange structure

Duplicated outgoing data streams because each ISO/RTO has

multiple peers• Network cost, maintenance

3


Proof-of-Concept Cloud Based WAMS• Objective: demonstrate a cloud-hosted distributed

platform for real-time PMU data collection, storage, processing and dissemination to achieve wide-area monitoring Security Network latency Fault tolerance Data consistency Connectivity to powerful cloud-hosted data

analytics tools Potentially low cost: in cloud, we “rent as needed”

rather than “own”

• Project collaborations among ISO New England Inc. Cornell University Washington State University New York Power Authority (Phase II) 4


Proof-of-Concept Project on Cloud-Hosted Wide Area Monitoring

5

Benefits:

• Supplemental and backup to the traditional SE

• A new platform for collaborations between control areas (operators see the same wide view of the bulk system)

• Explore all benefits and concerns of the cloud computing and advance the technology in the power industry

• A new and efficient way for synchrophasor data exchange and repository, further advance the synchrophasor technology


Cloud WAMS Deployment: Live Data Streams

6


Cloud WAMS Deployment: Data Archive + Analytics

7

ISO-NE hosted distribution point

PMU1

PMU2

PMUm-1

PMUm

Re

-pla

ye

d C

37

Da

ta

TCP

Sender

31

PMU

Stream

s

Cornell hosted distribution point

PMU1

PMU2

PMUm-1

PMUm

Re

-pla

ye

d C

37

Da

ta

TCP

Sender

42

PMU

Stream

s

C37.118

Cloud hosted

Ingress point in

“data collector”

role.

Cloud hosted

Ingress point in


role.

Cloud hosted

Ingress point in


role.

Cloud hosted

Ingress point in


role.

Cloud hosted

Ingress point in


role.

Cloud hosted

Ingress point in


role.

Archived

data

Archived

data

Freeze-Frame File

System

Freeze-Frame File

System


Cyber Security

• Amazon Virtual Private Cloud (VPC) Logically isolated section of AWS under

users’ complete control

• SSH Tunnel for data stream

• ISO-NE data source Historical data playback w/ simulated real-

time timestamps Inside firewall

• Cloud Data Storage Encrypted using a key

• Generated by and stored in Amazon AWS• Managed by users

8


Cyber Security Performance Cost

• EC2 Latency (from data sources to LSE) Average = 245ms 1st Percentile = 211ms 99th Percentile = 255ms

• VPC Latency (from data sources to LSE) Average = 261ms 1st Percentile = 228ms 99th Percentile = 270ms

• Delta is approximately +15ms; the numbers do not include SE compute time (75ms-100ms)

• Adding SSH tunnels added less than 2ms

• AES 256 encryption has no impact on performance (noise level) 9


Latency

10

L1: One way from DataSource to CloudRelay to Application (SE)

L3se: Round trip from DataSource to SE to DataSource

L3raw: Round trip from DataSource to CloudRelay to DataSource

L2: Round trip from Ingress Relay to Egress relay


Histogram: L3 Raw Data Round Trip Latencies

11

Num

ber

of O

ccurr

ences

0

500

1000

1500

2000

2500

0 100 200 300 400 500 600

ISO-NE Data Source

Oregon

Viginia


Histogram: L3 Raw Data Round Trip Latencies

12

Num

ber

of O

ccurr

ences

0

500

1000

1500

2000

2500

0 100 200 300 400 500 600

Cornell Data Source

Oregon

Virginia


Histogram: SE Computation Time

13

• Total latency for SE is obtained by adding the raw latency and the SE

Computation time. Typical would be 75-150ms depending on data source and

cloud data center location

• SE Computation time here is from a later SE version with much-improved

computation time relative to that achieved in the original experiments


Histogram: L3 SE Results Round Trip Latencies

14

Num

ber

of O

ccurr

ences

0

500

1000

1500

2000

2500

0 100 200 300 400 500 600

Oregon

Virginia


Fault Tolerance and Data Consistency

15

• Fault Tolerance– Two parallel systems

– Independent• Manual redundancy

• Loss of one data center did

not impact results from

other data center

– Full back-up redundancy

was restored within 3

minutes after data center

shutdowns

• Consistency– No raw data loss

– Returned raw data and LSE

results from the two data

centers were identical

– Within ~100 ms


OpenPDC (Visualizer) Displaying SE Results

16


Freeze Frame File System Role

• A standard POSIX file system, but augmented to provide millisecond accuracy for time-based data access.

• Integrates with the popular Spark data analytics framework via Spark “resilient distributed data” (RDD) objects

These are small scripts written in Python, Java or Scala

Used to extract data, transform it. Automatically cached for speed. Flexible, but in our work, these RDDs usually extract tensors.

Spark has tens of millions of lines of powerful tools that can operate directly on data in this RDD form.

• Includes traditional computational frameworks, like Matlab

• Also more modern ones, like machine learning tools (neural network models, Bayesian learning models, etc)

17

consideration of communication time delays in … › powertech-2019 › wp-content ›...

Documents