A Fault Resilient Architecture for Distributed

Cyber-Physical Systems

Fardin Abdi, Brett Robins, Marco Caccamo

University of Illinois at Urbana-ChampaignUrbana-Champaign, USA

{abditag2, robbins3, mcaccamo}@ILLINOIS.EDU

1UIUC

Introduction to problem Preliminary Architecture description

◦ Fault detection◦ Fault handling

Implementation in electric grid evaluation

Outline

2UIUC

Interconnected physical plants that physically affect each other!

State of each nodeis a function of control inputsof other nodesbased on system connection graph

Distributed Cyber Physical Systems

Images : http://geospatial.blogs.com/geospatial/2009/07/alternative-energy-green-nonemitting-clean-renewable-or-low-carbon-.htmlhttp://www.thewatertreatments.com/water/distribution-system/ 3UIUC

Distributed controllers coordinate with other nodes in order to:◦ Reach to the desired state for the entire system◦ Maintain functionality and stability of the system

System relies on Communication◦ North American Electric Reliability Council

report: information system failure is a major reason of cascade failures!

Communication; an essential component

4UIUC

Unpredictable latency in communication Possible failures in communication

channels ◦ Physical disconnection◦ Improper functioning of communication unit

Issues with Current Communication Structure

5UIUC

Replacing the old infrastructure with new infrastructure is expensive therefore the old communication infrastructure is unlikely to be replaced any time soon.

Therefore:◦ Techniques need to be developed for detecting and

handling faults using existing communication technology.

Renovation Cost

6UIUC

Replacing cyber data with physical data to detect and

handle faults

General Idea:

7UIUC

In CPS, in addition to cyber channels, there are also physical channels that can be used as a source of data.◦ Control commands result in a physical change

in the state of a system Red light and street example

◦ Data should match with physical state Water pipe and sensors

We exploit the estimated states of remote nodes to detect communication faults and maintain the overall stability of the CPS.

Unique features of CPS

8UIUC

, Physical connection graph of CPS : physical neighbors of node i

: disconnected neighbors of node i

Preliminary

9UIUC

Connected nodes {1,2,4,5} Partially Connected nodes {3} Totally Disconnected nodes {6,7}

Preliminary

10UIUC

Estimation Unit Communication Unit Switching module Distributed controller Hybrid Controller Local Controller

Architecture

11UIUC

Designed for normal operation mode when reliable data is being received from all the neighbors

For most of the existing distributed cyber-physical systems, their existing controller can be used without any modifications.

Only Access to communication unit

Distributed Controller

12UIUC

Operates only based on estimated state variables of remote nodes and locally measured variables

Only access to estimation unit

Local Controller

13UIUC

When there is both connected and disconnected neighbors.

Has access to both communication and estimation unit

Hybrid Controller

14UIUC

Estimate neighbors state using local measurements and previous knowledge

◦ Example in power:

is previous knowledge and is local measurement

◦ Autonomous Vehicles Using local infrared sensors

◦ Water Distribution system (F: flow rate, R:physical resistance)

Estimation unit

15UIUC

Packetdist :◦ Information required by controllers in order to

take system to desired final state

Packetmeas : ◦ For verification purpose◦ Estimatable for the neighbors

Communication Unit

16UIUC

Periodically checks the following inequality

: maximum estimation error◦ This can be measured using experiments

Xdata : received parameters from neighbors Xest : estimated parameters based on the

local data A communication fault is declared when

the inequality doesn’t hold

Switching Module

17UIUC

No data received◦ Communication unit buffer is not updated in a

while. There would be a deviation between real data and data on communication buffer.

Incorrect data◦ Gap between the estimated and received value

Based on the number of disconnected neighbors, a switch is triggered to hybrid or local controllers.

Fault detection and handling

18UIUC

Sensitivity:

Injecting reactive power lowers the voltage of the node.

Electric Grid Preliminary

19UIUC

Goal: maintain voltages of nodes in the range of

A decentralized network in which each node sends the amount of reactive power that requires for its voltage correction to its neighbors.

Through some iterative steps, each node calculates its own reactive power production.

Decentralized Voltage Regulation Algorithm for Electric grid

20UIUC

When the communication is broken, each node can only use its own reactive power capacity for voltage correction.

Over/under voltages will occur in the nodes with higher needs than their capacity.

Broken Communication

21UIUC

Estimation unit:

Fault declaration:

A fault triggers a switch to Hybrid or Local controllers based on the number of disconnected neighbors.

Fault Resilient DVC

22UIUC

Distributed Controller:◦ Nodes exchange information via communication

channels and come up with value of reactive power production.

Hybrid Controller:◦ For disconnected neighbors, their value of reactive

voltage requirement is estimated based on estimation of their voltage.

Local Controller: ◦ All the reactive power requirements of the neighbors

are estimated. Finally, in order to satisfy requirements of all the neighbors, maximum estimated power is generated by the node.

Controllers

23UIUC

Perfect Communication: All the nodes in the network can generate power for

the node. Broken Communication:

◦ Original DVC algorithm: only the node itself can provide required power

◦ Fault Resilient DVC algorithm: Immediate neighbors can also provide the reactive power.

Comparison:

24UIUC

Scenario 1

25UIUC

Scenario 2

26UIUC