cloud data sharing platform project proposal · pserc summer workshop, july 2015 5 dave anderson,...
TRANSCRIPT
Carl Hauser & Anjan Bose (Washington State University)
Ken Birman (Cornell University)
Eugene Litvinov and Xiaochuan Luo (ISO New England)
PSERC Workshop July 2015 1
Cloud Data Sharing Platform Project Proposal
Cloud Data Sharing Platform Concept
PSERC Summer Workshop, July 2015
2
Benefits
ARPAe Project Review: Dec 18. 2013
3
A flexible and extensible platform for deploying new kinds of computational, data-intensive, historical and real-time analytics Automating management tasks associated with initiating
analysis tasks within the cloud
Strong consistency and security guarantees for shared data
Inexpensive continuous availability using remote backup locations
Working toward a demonstration that cloud technology can safely and securely host a critical infrastructure task
Background
ARPAe Project Review: Dec 18. 2013
4
Built on existing ARPA-E GridCloud Project Cloud management
Thousands of simulated (real-time) PMUs on the WECC system
Linear State Estimator (aka Phasor State Estimator)
and ISO-NE Phase-1 Demonstration 73 real (but recorded) PMU data streams
Network planning model
PSE
All cloud resources in a single AWS data center
Application
Tim
e A
lign
men
t
Co
mp
uta
tio
n
Res
ult
s
ARPA-E GridCloud Project
PSERC Summer Workshop, July 2015
5
Dave Anderson, Theodoros Gkountouvas, Carl Hauser
Data Collectors Replica 1
FWDN
FWDN-1
FWD1
…FWD2
Application
Tim
e A
lign
men
t
Co
mp
uta
tio
n
Res
ult
s
Time-Synchronized Data Sources Internet / VPN GridCloud running on Amazon Elastic Compute Cloud
Data Collectors Replica N
FWDN
FWDN-1
FWD1
…FWD2
Application
Tim
e A
lign
men
t
Co
mp
uta
tio
n
Res
ult
s
Results
…
GridCloud Demonstration (6K bus, 3 Replicas)
6
Managed by DMake and ISIS2
GridStat
Simulated Power System Replica 1
Re-
pla
yed
Dat
a Fi
les
PMUN P
PMUN-1 P
PMU1 P
P
…PMU2
S
FE
FE
FE
FE
GridStat (UDP)
GridCloud Replica 1
FWDN
FWDN-1
FWD1
…FWD2
SubstationSE1 P
SubstationSEM P
Control Center SERunning at 5Hz
Tim
e A
lign
men
t
S
Co
mp
uta
tio
n
Res
ult
s
Vis
ual
izat
ion
Simulated Power System Replica 2
Re-
pla
yed
Dat
a Fi
les
PMUN P
PMUN-1 P
PMU1 P
P
…PMU2
S
FE
FE
FE
FE
GridStat (UDP)
GridCloud Replica 2
FWDN
FWDN-1
FWD1
…FWD2
SubstationSE1 P
SubstationSEM P
Simulated Power System Replica 3
Re-
pla
yed
Dat
a Fi
les
PMUN P
PMUN-1 P
PMU1 P
P
…PMU2
S
FE
FE
FE
FE
GridStat (UDP)
GridCloud Replica 3
FWDN
FWDN-1
FWD1
…FWD2
SubstationSE1 P
SubstationSEM P
Time-Synchronized Data Sources Internet GridCloud running on Amazon Elastic Compute Cloud
Visualization Viewer (RDP)
Visualization
Local Machine
4632 Streams
4632 Streams
4632 Streams
291 Substations
291 Substations
291 Substations
175
275
375
475
575
0 100 200 300La
ten
cy (
ms)
Time (Seconds)
175
275
375
475
575
0 100 200 300
Late
ncy
(m
s)
Time (Seconds)
0
2000
4000
6000
8000
10000
0 100 200 300
Late
ncy
(m
s)
Time (Seconds)
175
275
375
475
575
0 100 200 300
Late
ncy
(m
s)
Time (Seconds)
6k Bus System, 144 instance:
Replica Latency Reduction8
Graphs: Latency – Length of time until all necessary data is available
R2 has a likely bad Network
Interface Card that we were
unable to replace prior to
preparation of this graph
0
50
100
150
200
250
175 195 215 235 255 275 295 315 335 355 375 395 415 435 455 475 495 515 535 555 575
Best Replica
0
50
100
150
200
250
175 195 215 235 255 275 295 315 335 355 375 395 415 435 455 475 495 515 535 555 575
Best Replica
Replica 3
Replica 2
Replica 1
6K Bus System, 144 instance:
Replica Latency Reduction9
Graphs: Number of times a particular latency occurs
ISO-NE Phase-1: Heterogeneous Data
10
In Phase 1 we added support any configuration of C37.118
(~24B-65,500B frames), with new challenges:
Framing of data within TCP streams using C37.118 header info
Application (PSE) configuration changes: need to know how each
stream is configured
Also support C37.118 PDC frames via the CloudRelay; PDC
frames can contain many PMUs worth of measurements
which need to be demuxed for use in GridCloud
Component Model of ISO-NE Phase 1
GridStat
Cloud Relay
GridCloud Replica 1
FWDN
FWDN-1
FWD1
…FWD2
SubstationSE1 P
SubstationSEM P
Control Center SERunning at 5Hz
Tim
e A
lign
men
t
S
Co
mp
uta
tio
n
Res
ult
s
Vis
ual
izat
ion
GridCloud Replica 2
FWDN
FWDN-1
FWD1
…FWD2
SubstationSE1 P
SubstationSEM P
GridCloud Replica 3
FWDN
FWDN-1
FWD1
…FWD2
SubstationSE1 P
SubstationSEM P
GridCloud running on Amazon EC2 or VPC
73 Streams
Managed by DMake and ISIS2
73 Streams
73 Streams
TCP or TLS Client
Handler
C37 PDC Stream
Demuxer
Replayed Data
Serving data from 73 PMUs
ISO-NE Internet/VPN
TCP
DataArchive
DataArchive
DataArchive
Key Elements of Supplemental Proposal w/ISO-NE
PSERC Summer Workshop, July 2015
12
Implement Encryption on various links Remote backup sites in other AWS regions; region->region
failover Make PSE output available as data streams that can be
consumed by other applications, in the cloud, or outside the cloud
Document impact of Encryption Network delays Network outages Cloud outages and node overloads
Where next?
ARPAe Project Review: Dec 18. 2013
13
Applications using the platform Current demonstrations: PSE and data archive
Unexplored strength of the cloud platform: dynamic deployment of computational resources
Expressing and enforcing data sharing policies What is each sharing entity willing to share with which
other participants
Acquiring and using real-time and archival topology information
See Anjan Bose or Carl Hauser to discuss further ([email protected]; [email protected])
Discussion
PSERC Summer Workshop, July 2015
14