distributed ctrl architecture
DESCRIPTION
DCSTRANSCRIPT
Engineering The Architecture OfDistributed Control Systems
January 2002
Eric RunnerstromMPR Associates
Distributed Systems Architecture
Jan. 2002 2
This methodology was developed by MPR to enable the rigorous design of distributed control systems that will perform as required.
The methodology was developed and implemented as part of the Navy Research Laboratory’s Damage Control Architecture for Reduced Manning (DC-ARM) Program.
The methodology ensures that the technology demonstrated by the DC-ARM program can be effectively applied to other platforms. It also could be be applied to other types of distributed systems, such as distributed sensor systems.
Distributed Systems Architecture
Jan. 2002 3
• Definition Of Supervisory Control System
• Premise
• The Process
• Firemain Example
• Further Development
Outline
Distributed Systems Architecture
Jan. 2002 4
Definition Of Supervisory Control System
A system, automated to some extent, that:
• Monitors and controls multiple, integrated sub-systems
• Enables a human supervisor to interact with the sub-systems
• Provides the information necessary so that the responses ofsub-systems and the actions of personnel can complement oneanother.
Distributed Systems Architecture
Jan. 2002 5
• Projects that implement complex control systems often experience:– Cost overruns– Schedule delays– Performance that does not meet expectations, even after
additional delays and costs, and substantial changes to the system
Premise
Performance problems are exacerbated when systems operate off design or equipment fails.Performance problems are exacerbated when systems operate off design or equipment fails.
Distributed Systems Architecture
Jan. 2002 6
• The principles for designing distributed control systems are evolving– Industry is able to build distributed control systems, but is still
learning how to engineer their design
• Engineering the architecture of distributed control systems will– Provide a basis for preventing the cost, schedule, and performance
problems often experienced during development– Enable optimizing the architecture of the control system with
respect to acquisition program criteria– Enable effective human-systems integration
Premise
A methodology for engineering the architecture of distributed control systems advances the state-of-the-art.
A methodology for engineering the architecture of distributed control systems advances the state-of-the-art.
Distributed Systems Architecture
Jan. 2002 7
� Define Control Decisions
� Develop Control Decision Logical Architecture
� Define Candidate Hardware Architectures
� Evaluate Candidate Hardware Architectures & Select Optimum
� Develop Software Architecture
The Process
Distributed Systems Architecture
Jan. 2002 8
Functional Analysis to define control decisions
1. Define Control Decisions
• System functional requirements driven by top level program requirements
• Fundamental integration of:– Sub-systems with one another
– Sub-systems and controls
– Humans and sub-systems
• Database of attributes for each control decision
Fire Severity?
No Boundary NeededNO FIRE
"SMALL" FIRE
Water Mist Functioning In
Fire Space?
YES"MEDIUM" FIRE
Boundary Needed
NO
Characterize Fire
** TV = Time To Vertical Spread = T(500) + 5 Min.
** TH = Time To Horizontal Spread = T(500) + 9 Min. Characterize
Fire
Control Water Mist System
Water Mist Functioning In
Boundary Spaces?
Compartment Characteristics
Determine Boundary
Compartments
FIRE SEVERITY
Maintain Boundary With
Water MistYES
Maintain Fire Boundary With
Water Mist
Predict Personnel
Performance
Can Personnel Set Boundary Before Fire
Spreads? NO
FIRE PREDICTEDIN COMPARTMENT
Is Boundary Space High Enough Priority To Apply
Personnel?
Define Operational
Priorities
Compartment Characteristics
NO
Maintain Boundary With
Personnel
Maintain Fire Boundary With
Personnel
YES
YES
Calculate Time For Fire Space To Reach
500 C = T(500)
Characterize Damage - Top Level
"LARGE" OR"FULLY DEVELOPED"
FIRE
Compartments Containing Damage
FIRELOCATION
NO
Functional Analysis Enables:
Distributed Systems Architecture
Jan. 2002 9
• Device Level Logic
• System Level Logic
• Ship Level Logic
2. Develop Control Decision Logical Architecture
Other applications could have different boundaries for the control decision logic levels.
– Device Level Control Logic requires only inputs available at the controlled device itself
– Similarly, Compartment Level Control Logic requires only inputs available at the compartment itself
– System Level Control Logic requires inputs from more than one device in the system
– Similarly, Zone Level Control Logic requires inputs from more than one compartment in the zone
– Ship Level Control Logic requires inputs from more than one system or zone
Three levels of control logic for systems or compartments:
Distributed Systems Architecture
Jan. 2002 10
SystemA
Logic
Sensors/InputDevice #3
System Level Logic requiresinputs from more than onedevice in the system.
Device Level Logic requiresonly inputs available at thedevice itself.
Example: DC-ARMSmart Valve usesdevice level logic.
Example: Firemain flowbalance is system level logic.
Ship LevelLogic
Device Logic
Sensors/Input Actuator
SmartDevice #2
Ship Level Logic requires input from more than one system.
Device Logic
Sensors/Input Actuator
SmartDevice #1
Example: Correlation of firemainstatus & compartment status isplant level logic.
SystemB
Logic
2. Develop Control Decision Logical Architecture
Distributed Systems Architecture
Jan. 2002 11
• Functional requirements for the control system are defined by:– Definitions and attributes of control decisions
– Control decision logical architecture
• Consider providing redundant logic for the same function to enhance reliability and survivability– Recognize trade-off of increased complexity and cost
2. Develop Control Decision Logical Architecture
Control decision logical architecture provides a basis for the synthesis of candidate hardware architectures that
meet the same functional requirements.
Control decision logical architecture provides a basis for the synthesis of candidate hardware architectures that
meet the same functional requirements.
Distributed Systems Architecture
Jan. 2002 12
• Hardware Architecture =– Control functions that will be performed on each
processor– Where processors will be located that collect data and
execute control decisions– Redundancy and separation of processors
• Redundant processors can improve reliability• Separated, redundant processors can improve
survivability– Consider need for communications among data sources
and processors• Communications load typically increases
significantly during a casualty
3. Define Candidate Hardware Architectures
Distributed Systems Architecture
Jan. 2002 13
• Processors can be located at:– Device Level; e.g. valves, pumps,
fans, dampers, A/C plants– Compartment Level– System Level; e.g. controllers for
firemain, ventilation, chilled water, electrical distribution
– Zone Level– Ship Level– Other locations
3. Define Candidate Hardware Architectures• Decision logic can be
executed in processors at almost any level; for example, extremes include:– All decisions executed at device
level processors– All decisions executed at a ship
level processor
• Hardware architecture that mirrors decision logical architecture may provide best performance
Communications load, cost, reliability & survivability can vary significantly depending upon the hardware architecture.
Communications load, cost, reliability & survivability can vary significantly depending upon the hardware architecture.
Distributed Systems Architecture
Jan. 2002 14
• Criteria are determined by the acquisition program
• Typical criteria might include:– Acquisition Cost
– Life-Cycle Cost
– Reliability
– Survivability
– Ease of Troubleshooting and Maintenance
– Affordability of Likely Upgrades
4. Evaluate Candidate Hardware Architectures
Select the hardware architecture that provides the best balance among acquisition program criteria and priorities.
Select the hardware architecture that provides the best balance among acquisition program criteria and priorities.
Distributed Systems Architecture
Jan. 2002 15
• Modular software architecture that mirrors control decision logical architecture probably is a good baseline– Provides logically consistent basis for:
• Development• Troubleshooting• Maintenance
– Maximizes “plug and play” capability for upgrades
• Adjust software architecture considering:– Hardware architecture– Communications
5. Develop Software Architecture
Distributed Systems Architecture
Jan. 2002 16
DC-ARMSHADWELL Firemain Example
Following slides are an example of applying the architecture engineering process to the DC-ARM firemain aboard SHADWELL.
Distributed Systems Architecture
Jan. 2002 17
Firemain Example - Functional AnalysisFiremain Self Monitor
Assess Material Condition & Readiness of Firemain
Perform Firemain Maintenance
Link To Attack Major Fire
Monitoring Data**
Preventive & CorrectiveMaintenance Requirements
Does Condition Indicate Probable
Damage?
YESLocation/Compartment of DamageDescription of Damage
Link To Monitor Ship Systems, Compartments, & Pre-Damage
Predictions
Evaluate Readiness of Firemain
Continue
NO
Human Supervisor's Evaluation
Readiness Information for Each Firemain Service; E.G. Fireplugs, Sprinkling, AFFF, Etc.
PM & CM Performed
Component Material Condition & Readiness
Component Tag-Out
**Monitoring Data includes data from sensors for component material condition, component operating data, and system hydraulic data, as needed.
Operating Status of Firemain: Pump & Valve Status; Fluid Pressures, Flows, Temp., Etc., As Needed.
Link To Restore Firemain
Link To Firemain Reflexive Operation
Link To Set Boundaries
Link To Respond To Probable Fire & Extinguish
Minor Fire
Typically, data is passed to links without waiting for human supervisor's assessment. Supervisor's assessment, when available, updates and may override previous assessments.
Firemain
Attack Small, Medium, Large, Or
Fully Developed Fire
Control Firemain & Provide Information To
Isolate Ruptures & Maintain Vital Services
Commands To Operate Pumps & Valves
Pump, Valve, & Pipe Segment AvailabilityPump Status & Valve Position
Valve & Pump Response To CommandsPressures
Predict Personnel
Performance
Maintain Boundary With
Personnel
Fire Plug AvailabilityFire Plugs Vital To Assigned Tasks
Characterize Damage - Top
Level
Control Firemain Maintain Firemain
Distributed Systems Architecture
Jan. 2002 18
• Examples of attributes defined for each action in the functional analysis– Level in control logical hierarchy
• Device, system, or ship– Primary or back-up means of accomplishing the action– Discrete or continuous action
• For discrete actions, need an initiating event– Intended effect of the action– Effects if the action is not performed when it should be– Effects if the action is performed when it should not be– Allocated to machine or human– Inputs required and locations of input sources– Outputs and locations of users of the output
Firemain Example - Functional Analysis
Distributed Systems Architecture
Jan. 2002 19
• Device Level Control Decisions– Isolate ruptures by closing valves closest to the rupture
• Rupture path logic– Inputs and actuator are at the valve
Firemain Example - Decision Logical Architecture
• System Level Control Decisions– Isolate ruptures by closing valves closest to the rupture
• Redundant with device level rupture isolation• Flow balance logic
– Inputs required from multiple valves– Logic is different from device level to minimize common mode failure
– Assess material condition and readiness of firemain• Inputs required from multiple devices
• Ship Level Control Decisions– Provide information to the human supervisor to isolate ruptures and maintain vital
services– Provide information to the human supervisor about the operational status of firemain
components– Start other pump if operating pump is in rupture segment
Distributed Systems Architecture
Jan. 2002 20
• Device Level Processors– Rupture path device level logic - YES
• Maximum survivability• Excellent graceful degradation• Simple• Minimum impact on data communications• Acceptable cost
– Flow balance system level logic - NO• Survivability not enhanced beyond rupture path device level
logic• Logic to achieve graceful degradation likely to become
unmanageably complex• Increases data communications load
– Provide information to the human supervisor to isolate ruptures and about firemain operability - NO• HCI is not applicable to device level processor
Firemain Example - Candidate HW Architectures
Distributed Systems Architecture
Jan. 2002 21
• System Level Processor(s)– Rupture path device level logic - NO
• Execute at device level– Flow balance system level logic - CANDIDATE
• May enhance survivability if:– Separated from ship level processor– Ship level processor executes redundant logic– Dynamic application reallocation is provided
• Some increase in data communications load• Increased cost for: additional computer, associated communications, more
complex software for dynamic application reallocation• Additional redundant computers may increase survivability
– Start other pump if operating pump is in rupture segment - CANDIDATE• May be a candidate for system processor if automated logic can be developed
– Provide information to the human supervisor to isolate ruptures and aboutfiremain operability - NO
• HCI is not applicable to system level processor• Automated assessment of firemain operability could be done at a system level
processor if applicable logic is developed
Firemain Example - Candidate HW Architectures
Distributed Systems Architecture
Jan. 2002 22
• Ship Level Processor– Rupture path device level logic - NO
• Execute at device level– Flow balance system level logic - CANDIDATE
• May enhance survivability if:– Ship level processor is redundant and separated from system level computer
• Some increase in data communications load– Start other pump if operating pump is in rupture segment - CANDIDATE
• Without logic for automation, a human decision is needed - HCI is at the ship level processor
– Provide information to the human supervisor to isolate ruptures and aboutfiremain operability - YES
• HCI is at ship level processor
Firemain Example - Candidate HW Architectures
Distributed Systems Architecture
Jan. 2002 23
• Execute system level logic in redundant, separated computer– More survivable– Less processor usage– More complex– Higher data communications load– More hardware cost– Higher development cost– More equipment to maintain
Firemain Example - HW Architecture Decision• Execute system level logic in ship
level computer– Less survivable– Greater processor usage– Less complex– Lower data communications load– Less hardware cost– Lower development cost– Less equipment to maintain
OR
• Could eliminate redundant system level logic for rupture isolation– No automatic back-up for device level control
– Minimum cost and complexity
• Quantitative analyses could be performed to assess options
• Communications have a significant influence on survivability
Distributed Systems Architecture
Jan. 2002 24
Firemain Example - Software Architecture
Rupture Isolation - Rupture Path Logic
Firemain Device Level Software Modules
Firemain System Level Software Module
Firemain Ship Level Software Module
Redundant Rupture Isolation - Flow Balance LogicFiremain Readiness
Display Information To Isolate Ruptures & Maintain Vital ServicesDisplay Information To Enable Assessment Of Firemain Operability
Remote Control Of Valves & Pumps
Distributed Systems Architecture
Jan. 2002 25
• Extend to compartment-zone structured architecture– Fire detection
– Access closure status
– Watermist actuation
• Investigate integration of device-system structured architecture and compartment-zone structured architecture
Further Development