designing for wide-area situation awareness in future power … · 2016. 6. 22. · abstract...

Designing for Wide-Area Situation Awareness in Future Power GridOperations

by

Fiona F. Tran

A thesis submitted in conformity with the requirementsfor the degree of Master of Applied Science

Graduate Department of Mechanical and Industrial EngineeringUniversity of Toronto

c© Copyright 2016 by Fiona F. Tran

Abstract

Designing for Wide-Area Situation Awareness in Future Power Grid Operations

Fiona F. Tran

Master of Applied Science

Graduate Department of Mechanical and Industrial Engineering

University of Toronto

2016

Power grid operation uncertainty and complexity continue to increase with the rise of electricity market

deregulation, renewable generation, and interconnectedness between multiple jurisdictions. Human op-

erators need appropriate wide-area visualizations to help them monitor system status to ensure reliable

operation of the interconnected power grid. We observed transmission operations at a control centre,

conducted critical incident interviews, and led focus group sessions with operators. The results informed

a Work Domain Analysis of power grid operations, which in turn informed an Ecological Interface De-

sign concept for wide-area monitoring. I validated design concepts through tabletop discussions and a

usability evaluation with operators, earning a mean System Usability Scale score of 77 out of 90. The

design concepts aim to support an operator’s complete and accurate understanding of the power grid

state, which operators increasingly require due to the critical nature of power grid infrastructure and

growing sources of system uncertainty.

ii

Acknowledgements

This research project was made possible with the support of many people. I would like to thank my

supervisor Prof. Greg A. Jamieson for initiating and supporting this project throughout; my colleague

Dr. Antony Hilliard for his mentorship, guidance, and collaboration; and to several people from the

Independent Electricity System Operator (IESO): Len Johnson, David Short, Steven Ferenac, David

Devereaux, Nicola Presutti, Kim Warren, and all the operators and engineers who volunteered to par-

ticipate in interviews, observations, and evaluation discussions. I am grateful to have received generous

support from the IESO, Mitacs, Province of Ontario, and University of Toronto during my studies. Last

but certainly not least, I would like to thank my friends and family for supporting me in my career and

my life outside of work.

iii

Contents

List of Tables viii

List of Figures ix

Abbreviations x

1 Introduction 1

1.1 Power Grid Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Wide-Area Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.2 Human Factors in Power Grid Operations . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Ecological Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Scoping Study of Power Grid Visualizations 4

2.1 Power System Overviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Colour Contours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.2 3D Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.3 Modelling Very Large Interconnected Grids . . . . . . . . . . . . . . . . . . . . . . 6

2.1.4 Correlated Multi-Screen Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.5 Hypermedia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.6 Wide-Area Measurement Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.7 Decision Support Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.8 Electricity Market Monitoring Systems . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Operator Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2.1 Mitigating Situation Awareness Errors . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.2 Distributed Situation Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

iv

2.2.3 Cognitive Workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.4 Effect of Expertise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.5 Operator Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Research Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.4 Limitations of the Scoping Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.5 Scoping Study Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Knowledge Elicitation 17

3.1 Critical Incident Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.1.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.1.2 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2 Focus Group Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4 Work Domain Analysis 23

4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2 Summary of the WDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2.1 Abstraction Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2.2 Means-Ends Links Between Abstraction Levels . . . . . . . . . . . . . . . . . . . . 25

4.2.3 Part-Whole Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.2.4 Topographic/Causal Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.3 Application of the WDA to Wide-Area Monitoring . . . . . . . . . . . . . . . . . . . . . . 26

4.3.1 Design Scope in the Abstraction Hierarchy . . . . . . . . . . . . . . . . . . . . . . 26

4.3.2 Measures and Constraints at Different Abstraction Levels . . . . . . . . . . . . . . 27

4.3.3 Using the Information Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . 28

5 Design and Evaluation of Wide-Area Monitoring Concepts 30

5.1 Preliminary Design Ideas and Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5.1.1 Critical Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.1.2 External Geographical Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.1.3 Modelling the Ontario Power Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.1.4 Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

v

5.1.5 Alarm Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5.1.6 Contours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.2 Design Prototyping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.2.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.2.2 First Interactive Prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.3 Iterated Design Prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.3.1 Wide-Area View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.3.2 System-Wide View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3.3 Detailed Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3.4 Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.4 Usability Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.5 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6 Discussion 47

6.1 Implications for Wide-Area Monitoring and Visualization . . . . . . . . . . . . . . . . . . 47

6.2 Study Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

6.3 Future Research Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

6.4 Industry Partner Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7 Conclusion 52

APPENDICES 52

A Knowledge Elicitation 53

A.1 Control Room Ergonomics Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

A.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

A.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

B Pre-Usability Evaluation Wide-Area Monitoring Design Concept 60

C Usability Evaluation 65

C.1 Participant Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

C.1.1 Positive Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

C.1.2 Implemented Suggestions for Improvement . . . . . . . . . . . . . . . . . . . . . . 66

vi

C.1.3 Suggestions for Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

C.1.4 Suggestions not Implemented . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

C.2 Usability Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

References 71

vii

List of Tables

3.1 Critical incidents identified from operator interviews. . . . . . . . . . . . . . . . . . . . . . 19

5.1 Critical parameters of system operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.2 Number of times each scenario question was answered correctly during the usability eval-

uation (total: 9 participants). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

viii

List of Figures

4.1 Abstraction hierarchy of power grid operations. Elements relevant to wide-area monitoring

are highlighted in green boxes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5.1 Preliminary concept for displaying external jurisdictions that affect the Ontario power grid. 32

5.2 Preliminary design for a one-line diagram showing links between Ontario’s IROLs. . . . . 33

5.3a After tabletop discussion feedback: Stick-and-circle layout for modelling Ontario regions

at the system overview level, showing generation and load numbers. . . . . . . . . . . . . 34

5.3b A toggle in Figure 5.3a would also show the operating reserve and spare generation numbers. 34

5.4 Preliminary design for alarm notification panel. . . . . . . . . . . . . . . . . . . . . . . . . 35

5.5 Iterated design prototype: wide-area view that includes external jurisdictions. . . . . . . . 37

5.6 Iterated design prototype: system view of Ontario power grid. . . . . . . . . . . . . . . . . 39

5.7 Iterated design prototype: overview of the northwest section of the Ontario power grid. . 40

5.8 Iterated design prototype: northwest overview with alarm notification pane showing. . . . 41

5.9 System Usability Scale (SUS) responses on the usability questionnaire. Error bars in all

graphs indicate standard error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.10 Comparison of ease of use between the design prototype and existing tools, as rated by

participants on the usability questionnaire. . . . . . . . . . . . . . . . . . . . . . . . . . . 45

A.1 Responses to the control room ergonomics questionnaire at the focus group sessions. . . . 59

B.1 Previous iteration design prototype: wide-area view that includes external jurisdictions. . 61

B.2 Previous iteration design prototype: system view of the Ontario grid (power flow view). . 62

B.3 Previous iteration design prototype: Hovering over OMTE would show the contribution

of OMTE to the EWTE and FS power flows (i.e. what would happen if the Manitoba

flow was suddenly zero). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

B.4 Previous iteration design prototype: system view of the Ontario grid (generation view). . 63

B.5 Previous iteration design prototype: overview of the northwest section of the Ontario

power grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

ix

Abbreviations

CWA Cognitive Work Analysis.

EID Ecological Interface Design.

EMS Energy Management Systems.

IESO Independent Electricity System Operator.

IROL Interconnection Reliability Operating Limit.

NERC North American Electric Reliability Corporation.

RC Reliability Coordinator.

SA Situation Awareness.

SCADA Supervisory Control and Data Acquisition.

SOL System Operating Limit.

SUS System Usability Scale.

WDA Work Domain Analysis.

x

Chapter 1

Introduction

Electrical power grid operation continues to increase in complexity and uncertainty with the rise of

renewable generation [1], electricity market deregulation [2], and cybersecurity risks [3]. These factors

are exacerbated by the tightly interconnected nature of power grids between multiple jurisdictions [4].

The Reliability Coordinator (RC) for each jurisdiction is responsible for the reliable operation of the

power grid within their jurisdiction. Human control room operators monitor and control the power grid

with the help of automated Energy Management Systems (EMS). With the growing complexity of grid

operations, EMS’s in turn need to adapt to prevent information overload, encourage faster problem-

solving, and reduce error. Operators thus require new visualization tools to summarize and display the

growing body of available and relevant information.

1.1 Power Grid Operations

An EMS forms the human-machine interface that gathers, computes, and displays information about

the the grid state. In addition, it dispatches electricity generation according to supply and demand, and

sets market prices based on near- and long-term forecast demand.

Power grid operations by RCs are tightly regulated by standards such as those set by the North

American Electric Reliability Corporation (NERC) [5] and the North American Energy Standards Board

[6].

1

Chapter 1. Introduction 2

1.1.1 Wide-Area Monitoring

The NERC IRO-003-2 [7] standard mandates that RCs have wide-area views of the Bulk Electric System

- generally defined as all power transmission elements and real/reactive power sources that are operated

or connected at 100 kV or higher, and does not include local power distribution elements [8]. The

wide-area view must include visibility within the RC Area and adjacent RC areas, so that the RC may

determine any potential System Operating Limit (SOL) or Interconnection Reliability Operating Limit

(IROL) violations within their respective RC Area.

Electric blackouts pose a severe risk to the welfare and security of households, health and emergency

services, industry, and more. Large-scale blackouts, though infrequent, may originate anywhere within

or connected to the RC Area and can propagate within milliseconds. For instance, the major North

American blackout in 2003 started with a short-circuit and caused cascading failures that affected 50

million people [9]. Among the U.S.-Canada Power System Outage Task Force’s recommendations was

to improve Reliability Coordinators’ visualization capabilities over wide geographical areas. Wide-area

monitoring and control systems thus play a crucial role by helping determine grid security, and building

operators’ awareness of the system.

1.1.2 Human Factors in Power Grid Operations

While EMS’s are highly automated - human operators take on a supervisory role and monitor the EMS

for abnormalities - human performance is critical for detecting and resolving problems not handled by

the automation. Inadequate operator Situation Awareness (SA) has been a widely cited factor in several

recent large electrical disturbances [10], making it critical that EMS’s are designed to satisfy both the

technical constraints of power operation and the needs of the humans using the system. One way to

accomplish this is through the Ecological Interface Design (EID) [11, 12] approach.

The scoping study in Chapter 2 provides a more detailed overview of the existing body of literature

on human factors in power grid visualizations.

1.2 Ecological Interface Design

The basis of the EID theoretical framework is designing to minimize the opportunity for human error,

and support recovery from errors. EID has been used to develop human-machine interfaces to control

complex systems, in domains including aviation, medicine, manufacturing, and nuclear and hydro power

Chapter 1. Introduction 3

generation [13].

An important part of the EID process is Cognitive Work Analysis (CWA), a framework for modelling

complex systems. CWA consists of 5 phases: Work Domain Analysis (WDA), Control Task Analysis

(ConTA), Strategies Analysis (StrA), Social Organization and Cooperation Analysis, and Worker Com-

petencies Analysis [14]. The CWA conducted in this study was limited to the WDA, as obtaining

information requirements from the WDA is considered the most formalized and systematic approach to

creating ecological displays [15].

To our knowledge, there has been no CWA, nor subset thereof, published in the literature for the

power grid operations domain. Furthermore, EID has been sparsely applied to creating wide-area visu-

alizations - the only examples found in the literature are Rantanen et al.’s application of general display

guidelines [16] and the PEGASE project that modelled the European power grid [17]. While these ex-

amples cite EID guidelines, they do not take information requirements from a WDA, despite this step

being an important part of the EID process.

1.3 Objectives

The Cognitive Engineering Laboratory at the University of Toronto partnered with the Independent

Electricity System Operator (IESO), the RC for the province of Ontario, to work on designing and

evaluating displays for future power grid operations. My work focused on visualizations for wide-area

monitoring.

The objectives of this research were to:

1. Develop an understanding of current human factors research and issues in power grid operation;

2. Analyze operator work in the power grid control room;

3. Design and evaluate novel display concepts for wide-area monitoring using EID.

The research-industry collaboration allowed the IESO to obtain a third-party perspective of human

performance in their operations control room, built on standard human factors principles. In addition,

the display design concepts were proposed for future tools to be deployed in the control room.

Chapter 2

Scoping Study of Power Grid

Visualizations

To gain insight on the current state of the art in power system visualizations for transmission operation,

we conducted a scoping study of the literature.

The scoping study framework was formalized by Arksey and O’Malley in 2005 [18]. It is an extension

of the traditional literature review, incorporating broader research questions with the goal of identifying

gaps in existing research literature. The methodology has been widely used in health care, but research

synthesis in other areas such as software engineering remains relatively untouched [19].

The scoping study consists of the stages: identifying the research question; identifying relevant

studies; study selection; charting the data; and collating, summarizing, and reporting the results.

Stage 1: Research questions: This scoping study pertains to the current solutions for visualizing

growing power system complexity and future opportunities for developing representation aids in this

area.

Stage 2: Relevant studies: Studies and reviews were found from electronic databases (Inspec and

IEEE Xplore), and Google search engines (Scholar and Web). The search terms were different combi-

nations of “power systems”, “electricity markets”, “dispatch”, “visualization”, and “display”. Previous

studies in power systems visualization have largely introduced new techniques and tools. Some have

additionally conducted usability studies; however, there has been no comprehensive review of existing

tools and their usefulness to operators.

4

Chapter 2. Scoping Study of Power Grid Visualizations 5

Stage 3: Study selection: Although our topic was multidisciplinary in nature, the search results were

dominated by power systems and electricity market modelling and optimization algorithms rather than

user interfaces (UI). This may be partly attributed to the ambiguity of the “visualization” and “display”

search terms - for example, an author may visualize their mathematical model by plotting it on a graph.

To select papers for review, a preliminary reading of the abstract, followed by reading the paper, helped

identify which were relevant to the study of control room displays.

Stage 4: Charting the data: Studies were categorized according to their topic of interest in the context

of the electrical transmission control centre. Theoretical papers on future directions of power systems

and electricity market visualization were grouped last.

Stage 5: Results reporting : The results of the scoping study, answering each of the research questions,

are described in the following sections.

2.1 Power System Overviews

Several new visualization techniques have been presented in the literature over the years, although

empirical investigations have only begun in the past decade or so to determine the effectiveness of those

techniques when employed in the control room.

Some visualization techniques for power systems are: tabular, integrated, and colour-contoured one-

line. Tabular representations outline each measurement name and corresponding value. One-line dia-

grams are a form of block diagram showing power flows, and are a simplified graphical representation

of three-phase power systems. Integrated one-line diagrams have the addition of showing voltage values

near the buses on the one-line.

When Overbye et al. compared the effectiveness of these three techniques in the scenario of diagnosing

and solving low-voltage violations [20], they found that the tabular format had the best acknowledgement

response times for low-voltage violations (1-2 seconds faster than one-line), but users solved violations

8-9 seconds faster using the integrated one-line diagram. They attributed this to high display proximity

[21] for the integrated information on the dynamic one-line diagram. Colour contours (discussed below)

fared worst for acknowledgement response time, although contours may be useful for larger real-life

systems than the hypothetical one used in the experiment.


2.1.1 Colour Contours

Colour contours have been investigated for their effectiveness in power system visualizations [22], since

human factors literature suggests colour codes can be interpreted and compared faster than numeric

coding.

Display acknowledgement times do not differ among different configurations of one-line diagrams

(number only, contour only, or number plus contour) for low-complexity violations. For higher complexity

violations (i.e. greater number of violations), the contour-only group was able to more quickly (taking

less than half the time) and accurately identify the low-voltage bus than the number-only group. Solution

times are significantly slower for contour-only displays than number-only ones, however. Overbye et al.

[22] suggest that for contour displays, once a violation has been acknowledged, the contour may be

dimmed out or removed to aid in problem correction.

2.1.2 3D Displays

Three-dimensional (3D) displays of power system information have been compared with conventional

two-dimensional (2D) displays, either one-line diagrams or in tabular format [23]. The 3D display results

in faster solution times than for 2D graphical display, perhaps because of the size and salience of the

generator representations in the 3D format. Precise judgements are not required to solve line flow

violations, so the numerical tabular format would have no advantages.

Overbye et al. [23] cite geographical data views (GDV) as a viable technique for power system

visualization, by integrating information from the power system model and geographic information.

GDVs therefore show a greater range of system information than conventional wide-area visualizations.

2.1.3 Modelling Very Large Interconnected Grids

Power grids are highly interconnected, spanning large areas with multiple independent system operators.

PEGASE is a European project to develop interfaces for power grid control centres that operate very

large interconnected systems [17]. The PEGASE display provides a map of the European power grid, with

countries coloured based on their operating state. The colour scheme uses a traffic light analogy. Possible

operating modes are: normal (green), preventive (yellow), corrective (red), or restorative (magenta). In

addition, black indicates a blackout and grey signifies no information available.


2.1.4 Correlated Multi-Screen Displays

Conventional multi-screen displays simply expand the effective display area, and do not use their full

potential to adjust display contents quickly and conveniently. Zhu et al. developed what they called a

correlated multi-screen display that allows operators to indirectly modify multiple screens by controlling

just one of them [24]. The correlativity between screens (how they would influence what is displayed on

each other) was organized as follows:

• Object correlation: 2 screens showing the same object but different information on it (e.g. 2 maps

of the same area but different overlay content)

• Information correlation: 2 screens showing different objects but the same information type (e.g. 2

maps of different areas but same type of overlay)

• Inheritance correlation: 1 screen showing wide area operating state (main screen) and 1 screen

showing local detail (sub-screen which operator can control)

• Entirety correlation: 2 screens with correlated objects and information, and whatever the operator

does on either screen, the other will change with it

The authors designed and developed an experimental platform using the correlated multi-screen

display philosophy. They hypothesize that their platform would allow operators to more conveniently

control objects on the screens, and perform tasks more quickly. However, their system had not yet

undergone usability testing.

2.1.5 Hypermedia

Hypermedia is the use of interactive multimedia nodes, linked together as a model of information rep-

resentation and management [25]. This linkage is relevant to power grids, whereby buses and lines are

all interconnected to each other to form a complex system. Moreno-Muñoz et al. propose hypermedia

UI design to optimize power grid data management [26], to replace current tools that are mainly data

tables and vector-based graphical displays.

They created a hierarchy of displays, starting with the substation view at the top level, since current

power systems are usually presented on a substation-centric map. Clicking on each node would then lead

to a localized view showing connections to the equipment, analogue measurements, and any additional

database information. The lowest hierarchy level would show tables of raw variable values. Graphical

displays of power quality information would be available from any screen so that operators can explore


subsystems while understanding topology, composition, and current status.

2.1.6 Wide-Area Measurement Systems

Supervisory Control and Data Acquisition (SCADA) has been used in the past few decades for remotely

controlling equipment, and taking measurements every few seconds. As transmission systems continue

to replace their aging assets, phasor measurement units (PMUs) are sought as a replacement to SCADA

as they have much faster scan rates - up to 60 measurements per second - and can directly measure bus

angles across systems [27]. This opens up the possibility of real-time, direct visualization.

A large network that has deployed PMUs for measurement is known as a wide-area measurement

system (WAMS). WAMS has been applied to a grid dispatching system in China, and corresponding

visualization functions deployed on the control centre’s dynamic displays of alarm and non-alarm infor-

mation [28]. The visual displays show basic limit violations, power system disturbances, low-frequency

oscillations, and performance evaluations of adjoining operating units.

2.1.7 Decision Support Systems

Moving on from experimenting with different visualization techniques according to technological capabil-

ities, current research has examined how automation can assist operators in decision-making and reduce

the likelihood of error.

Decision support systems advise the operator on what to do, rather than doing the task itself. Done

incorrectly, decision support can decrease human performance, be it taking longer to make decisions, or

forming independent assessments and succumbing to cognitive anchoring [29]. When cues are targeted

correctly and the system provides advice after the operator has made a decision (i.e. critiquing the

operator’s action rather than teaching the operator what to do next), error rates may drop significantly.

One area of interest for decision support has been alarm visualization. Tripathi et al. [30] developed

an alarm visualization concept that includes a filter to prioritize important alarms. More critical alarms

are displayed in a larger font than non-critical ones. A root cause diagram indicates other affected

devices in the network. Their rationale, supported by subject matter expert feedback, was that the

salience of critical alarms improves alarm identification for operators, and the root cause diagram helps

them analyze the alarm’s impact to resolve faults.


2.1.8 Electricity Market Monitoring Systems

Aside from general power systems monitoring and control, transmission dispatchers are also concerned

with market changes and how they may affect transmission status and constraints. Market dynamics

are of particular concern to operators with the arrival of deregulated electricity markets [2].

Independent system operators use market monitoring systems (MMS) to ensure efficient market

performance, through detecting market inefficiencies, potential abuses, and market power problems (i.e.

for a seller to sell at above the competitive rate). Effective market monitoring software must encompass:

generation and transmission outages; supply, demand, and transmission adequacy; potential or actual

market power abuses, and behavioural monitoring of market participants [31].

One way to visualize market power is through colour contouring [32] to show loading on each trans-

mission line. Line flows above a certain percentage are highlighted, indicating a small generation market

available to a load pocket. Virtual reality environments can qualitatively portray multiple layered sys-

tems on a 3D interface. For example, line flows and power transfer distribution factors (PTDF) values

can be displayed for both the actual system and a proposed transaction for comparison.

The deregulation of electricity markets has increased the complexity of market operations, as they

now require collaboration between multiple operating jurisdictions, existing market participants, and

new entrants [33]. Growth in renewable energy generation, storage, and controllable loads has added to

the compelling need for effective market monitoring and dispatch tools that merge with power system

operation.

2.2 Operator Effectiveness

While work in power systems and market visualization has largely centred around developing new tech-

niques, there are gaps in the literature on how they can impact the effectiveness of human operators.

Routine events are automated in transmission system control, but human operators take a critical role

in managing emergency system operations [34]. The goal is to provide operators with appropriate levels

of real-time information for them to make decisions. Of particular interest are situation awareness and

cognitive workload.


2.2.1 Mitigating Situation Awareness Errors

Situation awareness (SA) has been defined by Endsley [35] as “the perception of the elements in the

environment within a volume of time and space, the comprehension of their meaning, and the projection

of their status in the near future.” SA theory suggests that SA is achieved in UI design by creating a

mental model for human operators that matches reality.

Operator SA is a key to preserving grid reliability [34]. Panteli et al [36] suggest best practices

for supporting human operators for increasingly complex modern power grids, and outline methods for

dealing with problems associated with SA for power systems. Advanced monitoring and decision-support

tools can support adequate SA, such that operators may gather and interpret necessary information for

responding to incidents.

Panteli et al. [36] identified the main sources of SA errors in transmission and distribution control

centres as:

• Software applications: application errors may cause operators to be misinformed about defects,

and thus not react in time to mitigate them.

• Real-time measurements: missing or conflicting information impedes on decision-making.

• Environmental factors: complexity in a graphical user interface (GUI) may make perception and

interpretation difficult.

• Automation: highly automated systems may detach operators from the real system, and have low

awareness of the state. This is known as the out-of-the-loop performance problem [37].

• Communication with others: insufficient communication between operators, in one or more control

rooms.

• Individual factors: operators’ training, experience, and alertness.

Errors in operators’ SA in transmission and distribution control centres have a profound impact on

their ability to maintain system reliability. Reports on several large-scale blackouts in the past decade

have cited SA as a culprit [38].

To deal with these sources of SA errors, Panteli et al. propose the following [36]:

• Detecting and eliminating data inconsistencies during the process of state estimation. Rule-based

techniques can be implemented to detect and correct topology errors in the event of signalling or

communication malfunction.


• Implementing user-centred design principles to solve problems associated with GUI complexity and

limitations. Examples include mapping system functions to the goals of the operator, grouping

data and alarms around critical decisions, and providing system transparency and observability

(i.e. what the system is doing, why, and what next).

• Designing the GUI and information systems to tell the user when human intervention is required,

if the automation malfunctions or is not equipped to handle the scenario. To enhance operator

awareness, systems should automate only when necessary, keep the operator in control, use decision

support between human and automation, and provide automation transparency.

• Enhancing wide-area visualization and communication between transmission system operators in

neighbouring networks.

• Operator training that includes dealing with events outside of their network that might affect their

responsibility area, and frequent training on new technologies, e.g. smart monitoring and security

tools.

• Ensuring functionality of Energy Management System (EMS) applications, including regular test-

ing and hardware maintenance.

2.2.2 Distributed Situation Awareness

Distributed situation awareness (DSA) treats team SA as a characteristic at the overall systems level

rather than the individuals comprised within the system [39]. Although individuals may attain their

own SA for a system and share this with other members of the team, DSA approaches assume cognitive

properties of the entire system that are not present at the individual level.

The DSA concept has been applied to the case study of an electrical distribution system [40]. Control

rooms house teams of operators coordinating their monitoring activities, so SA across the team is of

particular concern. The case study found that efficient communications links allowed DSA to propagate

throughout the network of agents, corroborating with previous research on team SA.

Shared displays can also facilitate shared SA between team members to supplement verbal commu-

nication [29], be they visual (e.g. computer screens) or auditory (e.g. alarm) displays. This is especially

pertinent in a control room where operators rely on visual and auditory cues to understand system

status, and do not have the benefit of other physical cues. Abstracted shared displays according to each

operator’s tasks and shared goals can improve team performance, while completely duplicating the other


operator’s display can be detrimental to performance [41].

2.2.3 Cognitive Workload

Data overload is a prevalent issue across current control rooms. The increasing amount of data available,

such as from PMUs, and the lack of data integration between systems cause high attentional and memory

load on the operator. This leads to a loss of SA, and therefore higher error rates [42].

For example, Schneiders et al. [43] conducted a cognitive task analysis for one such control room,

and found that it required operators to monitor 20 different information readings. Considering that the

capacity for short-term working memory has been known to be 5-9 “chunks” of information [44], the task

required more memory than was available for the human operator. A more appropriate design would

limit the number of key indicators on the display for monitoring; Schneiders et al. used a process of

data reduction to identify the key indicators with highest priority according to operators. In the event

of deviations from normal operation, the display would then show more detailed information. Multiple

displays were implemented in the control room for different operators according to their tasks, to assist

in shared SA.

With routine events being managed by automation, human operators take on system monitoring tasks

to supervise operations, and intervene only when problems arise. These long periods of low temporal

demand leave them vulnerable to vigilance decrement, where they may suffer from a decline in detection

performance after long periods of time. The decline occurs more rapidly for cognitively demanding

environments [45], which power system control would fall under. In addition, vigilance tasks require

hard mental work and induce stress [46]; stress in particular can have negative effects on information

processing including a reduction in working memory and over-arousal in emergency situations, leading

to a speed/accuracy trade-off in performance [47]. As such, efforts should be made to ensure operators

are appropriately trained to deal with any possible scenarios and that their tasks are not designed so as

to be repetitive.

2.2.4 Effect of Expertise

Power systems operators have varying levels of acquired diagnostic reasoning skills. Domain experts are

able to apply cues to reduce the cognitive demands of a task, while novices may rely on their knowledge

of the domain in order to perform a task. Cue utilization may be measured through 4 approaches:


• Feature identification: identifying key features from a complex scene

• Feature discrimination: consistent perception of relative utility of features

• Paired association: response times and discrimination for item ratings (of the item utility)

• Transition tasks: the sequence in which the operator acquires task-related information

Performance in these 4 cue-based cognitive tasks distinguishes controllers into novice, competent, and

expertise groups [48]. Experts are consistently faster, are more accurate, have greater discrimination,

and are less prone to simply view information in the manner it was presented than their peers. Experts

are also more likely to choose the diagnostic response deemed optimal by the subject matter expert.

2.2.5 Operator Training

Power system operators go through an Operator Training Simulator (OTS) program to familiarize them-

selves with the human-machine interfaces in the control room, and also practise how to deal with emer-

gency operations. The simulator goes through possible contingency scenarios so that trainees learn

how to manage power system operations and assess dynamic security [49]. The growing complexity of

power system operations has meant a greater need to ensure proper operator training. Bronzini et al.

[49] developed an OTS approach that measures the knowledge and skills of the operator for emergency

management, in relation to the required knowledge and skills for the job.

2.3 Research Opportunities

Power transmission grids in the present and future face environmental, customer needs, and infrastruc-

ture challenges. Energy production has turned its attention to reducing CO2 emissions and increasing the

use of renewable power generation. Electricity markets need to remain competitive and customers need

to be satisfied with the quality of power supply. Electricity transmission infrastructure often suffers from

underinvestment and consists of aging assets, in spite of growing load demands. Future power grids are

expected to be smart, taking advantage of modern technological advances in sensing, communications,

control, computing, and information technology [33].

Power systems visualizations in the control centre typically display the voltage magnitude on a one-

line diagram. Under abnormal conditions with the risk of voltage collapse, voltage magnitudes are no

longer sufficient [33]. Instead, other indicators of voltage stability - such as tracing changes in local

frequency to remote locations - need to be employed to help operators identify fault locations.


A lack of specific geographical location information for displays and communication has restrained

situation awareness for control room operators [40]. Future displays will need to present geographical

information to better understand where problems occur and to coordinate with other agents. Overbye

et al.’s geographical data view approach [23] is expected to garner more interest in the design of such

displays.

The shift from SCADA to WAMS for remote measurement and control will mean more opportunities

to use real-time visualization and communication for human operators. Wang et al. [28] presented a

visualization concept for a WAMS implementation in China; however, no usability studies have been

conducted yet in this area.

Likewise, some of the other technologies described in previous sections have undergone preliminary

user studies and simulations, but their effectiveness in practice has yet to be empirically determined.

Owing to industry deregulation, power transmission control centres are moving from centralized to

coordinated decentralized decision-making [50]. Whereas information and communication technologies

(ICT) have rapidly evolved in recent decades, control centres rely largely on legacy technologies. System

operators have no analytical tools to support emergency control, due to legacy data acquisition systems

and limited computational power in the control centre. Future control centres are projected to exploit

modern ICT in order to provide real-time information and analytical tools to operators. This includes

taking advantage of distributed computing to offer data and application software that is decentralized

and distributed.

This shift will require further studies on visualization tools that transform these data into graphics,

and how operators can use these effectively. In particular, the tools need to present rapidly growing

amounts of data to enhance the operator’s SA, without overloading the operators’ cognitive resources.

Mobile application interfaces have been explored in the power utility domain for field workers to

access relevant and timely information, specifically in the case of an electrical distribution system [51].

Power transmission controllers have similar goals to field workers, with only the distinction being that

they operate at a higher-level scope; nevertheless, the benefits of multimedia communication for crews

and decentralization of information allow for the development of shared SA across operators working

at different levels of the power transmission and distribution system. Given the modern proliferation

of mobile devices in both consumer and industrial applications, mobile application interfaces will be

another research avenue to explore to improve collaboration and shared SA in the case of power grid

dispatch.


The Ecological Interface Design (EID) approach forms the basis of PEGASE, but has otherwise not

been widely used in power system control compared to other domains such as nuclear power plants [17].

Since EID can improve operator SA especially in the monitoring of unanticipated events [52], there are

more opportunities to apply the framework to power transmission control and evaluate how it impacts

operator SA.

The wealth of recent literature in power systems operator SA is a testament to significant interest in

designing tools and displays for operator effectiveness [53]. The IEEE Power & Energy Magazine featured

situational awareness for energy management analytics and visualization [54]. Hydro-Québec [55] and

ISO New England [34] have undergone UI redesigns of their energy management systems, and other

electrical utilities are expected to follow suit according to the power supply and demand characteristics

of their jurisdictions.

2.4 Limitations of the Scoping Study

This scoping study presents an overview of current work and future opportunities in visualization for

electrical transmission dispatchers in the control room.

This review may not have included all published papers in the literature on power systems visual-

ization techniques and their usefulness to operators, despite efforts to include all relevant search terms

and to use wide-reaching electronic databases. The review only included English-language publications,

and scans of search results only covered the first few result pages. Hand-searching key journals is a

method proposed by Arksey and O’Malley [18] to find articles that may have been missed in database

and reference list searches.

2.5 Scoping Study Conclusions

Electricity market deregulation, the integration of renewable energy sources, and interconnectedness

of the power grid have presented situation awareness and cognitive workload challenges to power grid

operators. To help meet their goals of reliable power supply, new algorithms and representation aids will

need to be implemented in the control centre.

The scoping study has found a great deal of work on improving tools and representation aids for

power systems monitoring; however, many papers did not support their claims of improving situation


awareness with empirical studies. There is potential for more research in integrating modern information

technologies into aging control centres, such as in providing wide-area monitoring data, geographical

location information, and rapid communication between utilities. Future work in user-centred approaches

to control room display design aim to improve human performance and help meet the goals of power

system reliability, quality, and efficient electricity markets.

Chapter 3

Knowledge Elicitation

We elicited the expertise of current control room operators through critical incident interviews, focus

group sessions, and control room observations. In addition, we observed their work in the control room

over a period of 60 non-consecutive days. The knowledge elicitation phase informed our Work Domain

Analysis and subsequent stages of the design process.

The knowledge elicitation and Work Domain Analysis (Chapter 3 and Section 4.2) were done in

conjunction with Dr. Antony Hilliard. I developed the plan and led the critical incident interviews. The

focus group session planning and facilitation, as well as control room observations, were a joint effort

with Dr. Hilliard. The results reporting and analysis described in Chapter 3 are mine.

3.1 Critical Incident Interviews

We conducted in-person interviews with operators, with the aim of understanding operator work in

the control room, and identifying possible information gaps in existing displays and tools. Flanagan’s

Critical Incident Technique [56] is a well-established method for eliciting recounts of critical incidents for

this purpose. Participants were asked to recollect specific incidents that they had experienced first-hand

in the control room, where their involvement had led to either positive or negative outcomes.

17

Chapter 3. Knowledge Elicitation 18

3.1.1 Methodology

Participants

We interviewed 8 current control room operators at the IESO, who had volunteered to participate. At

the time of the interview, 5 participants were off-shift and 3 were on-shift. Participants had a mean of

9 years of operational experience (min = 3, max = 17, SD = 5).

The IESO operations control room is staffed by 7 operators: a market assistant, 2 systems assistants,

a system-for-markets assistant, a systems supervisor, a market supervisor, and a shift superintendent.

We interviewed operators whose most recent control room roles covered all of the above, except for a

market assistant.

Procedure

Participants read and signed an informed consent form prior to the interview, and were also provided

with the list of interview questions in advance. Questions were adapted from the Critical Incident

Technique [56]. Participants were asked to describe their typical day on the job, and what kinds of

unusual situations they would face. They were then asked to recall incidents where they either made a

positive or negative impact. We told them we were particularly interested in incidents where information

was not available or hard to extract from displays, or where team communication played a role. For each

incident, we asked guiding questions to ensure that we got a detailed picture of the circumstances of the

incident, what the operator did to respond, and what outcomes arose. We did not specify a restriction

to how far incidents had occurred in the past. Each interview took approximately 1 hour.

3.1.2 Results and Discussion

All operators interviewed had examples of incidents where tool limitations hindered their performance.

Twenty-seven separate critical incidents were recorded from these interviews, and categorized by prop-

erties shown in Table 3.1. Some incidents fell into more than one category. The incidents occurred

between 2002 and 2015.

The top issues mentioned in the critical incident interviews were: data unobserved, unclear conse-

quences, model inconsistency, and wrong limit calculations. This observation suggests a need for tools

that: (a) attract operator attention to where important activity is occurring, and (b) integrate infor-

mation (such as limit calculations) between the different automated systems used in the control room.


Table 3.1: Critical incidents identified from operator interviews.

Type of Issue CountData unobserved : Displays did not make it clear that a line was in service or thata region could be islanded, misleading the operators

4

Unclear consequences: Operators performed an action, the consequences of whichwere unexpected

3

Telemetry unavailability : Telemetry data were not transmitting, so operators couldnot figure out system status

3

Wrong limit calculation: Voltage limits were calculated incorrectly, leading to un-stable system or inefficient market operation

3

Model inconsistency : Information discrepancies between different automation sys-tems (e.g. EMS and MIS) led to those systems optimizing for different modelparameters

2

Communication: Lack of adequate communication or cooperation between RCsmeant IESO was unaware of a line coming back into service or having to resort todrastic measures to protect assets

2

Error recovery : Manual input errors were not recoverable until after a delay, causingmarket constraints in the meantime

2

Unpredictable conditions: Variability on the power grid (e.g. harsh weather) addedoperator workload and made communication within the control room crucial

2

Data overload : Operator was overloaded with tasks and interruptions, and losttrack of some of the lower-priority tasks

1

Hardware failure: Tools were unavailable for operators during computer hardwarefailure

1

Model inaccuracy : Model inaccurately calculated largest contingency on the grid 1


Operators remarked that some displays were “cluttered” and “hard to read,” slowing their awareness of

contingencies and data inaccuracies. Future operator tools will need to prioritize information visibility,

particularly of unusual activity, and system data integration.

Our observation was also in line with the IESO’s business objectives of working toward better wide-

area monitoring and limit rules tools for operators. As such, my ensuing design work focused on wide-area

monitoring and power grid visualization, while Dr. Antony Hilliard’s work centred on concepts for a

limit rules engine.

3.2 Focus Group Sessions

We led focus group sessions with operators to understand their work in the control room in a team

setting. The focus groups had 3 parts: a critical incident group discussion, a facilitated discussion on

challenges and brainstorming, and an ergonomics questionnaire.

Compared to the traditional one-on-one critical incident interview, the focus group sessions provided

additional insight into team collaboration during incident response, and multiple perspectives on the

incident from the different operator roles. Because we were able to reach a larger audience in the focus

groups, we were able to have a wide range of responses to brainstorming questions and questionnaires.

3.2.1 Methodology

Participants

We led 6 focus group sessions, with a total of 42 operators from the IESO. The focus groups were

incorporated into the schedule of one of the mandatory IESO operator training days, and participants

had the opportunity to opt out of the session. Participants came from all roles in the control room, and

ranged widely in experience (mean = 10.5 years, min =


was, ”Describe an incident that your crew experienced in the IESO control room where you had to take

action on the electricity market or system.” We were particularly interested in events that involved team

collaboration and had different perspectives on the incident. While participants recounted the incident,

we recorded the timeline of events and challenges encountered during the incident, on chart paper.

A facilitated discussion on control room challenges and their impact on day-to-day operations fol-

lowed. We asked for any other challenges that often turn up in day-to-day operation. Once we finished

collecting challenges, participants voted on the top three in their personal daily operation experience,

by frequency and severity. We also asked participants to brainstorm what the concepts of “wide-area

monitoring” and “limit rules engine,” our two topics of design interest, meant to them as an operator.

We ended with a questionnaire on sight, sound, and tools in the control room (Appendix A.1). Each

focus group session lasted 1 hour in total.

3.2.2 Results

Challenges in Day-to-Day Operation

Issues related to wide-area monitoring featured prominently in participants’ voted top challenges in

day-to-day operation. Four out of the six focus groups had top challenges associated with having to

use multiple SCADA screens for a task, and difficulty navigating between screens in order to diagnose

problems.

The top-voted challenges for each of the 6 focus groups (not counting wide-area monitoring if it was

top-voted) were:

• Knowing system status and whether telemetry has failed.

• Knowing the situation and whether the system state is secure.

• Contingency planning and recognizing the situation.

• Performing studies of contingencies is slow and effortful.

• Needing to open lots of screens simultaneously to piece information together.

• Having an overly large workload during contingency situations.

Wide-Area Monitoring Definition

The focus group sessions with operations teams formulated the wide-area monitoring problem as:


• Detecting “problem areas”, both outside of Ontario and within the province, quickly and indepen-

dently from other RCs

• Supporting situation awareness for operators

• Zoom navigation at different levels of geographical scope

• Data consistency across screens (e.g. an outage should be visible on any screen that contains it)

• Viewing interconnections and where a Transmission Load Relief (TLR) request would affect the

wider grid

• Fulfilling the NERC standard on wide-area monitoring

Control Room Ergonomics

Operators rated their experience of ergonomics issues and tool development processes in the IESO control

room. The results are detailed in Appendix A.2.

3.2.3 Discussion

The brainstorming session on ”What does wide-area monitoring mean to you?” was an exercise in

ensuring that operators’ vision of wide-area monitoring would match the design goals set by the research

team and IESO management. During the design process, this definition helped narrow the scope of

wide-area monitoring within the power grid operations domain.

The responses on the control room ergonomics questionnaire were valuable for providing specific

recommendations to the industry partner on human performance improvements.

Chapter 4

Work Domain Analysis

We conducted a WDA of power grid and electricity market operations. The WDA describes the con-

straints that govern the purposes and physical properties of the power grid; the system decomposition;

and the means-ends links between system purposes and components.

The analysis helped describe the information needs of power grid operations in the context of wide-

area monitoring, and thus influenced the design and evaluation of the wide-area monitoring design

concepts.

4.1 Methodology

To gather an understanding of work domain concepts (such as power system theory), we consulted IESO

operator training materials and other technical documentation [57]. We then refined the WDA based

on our literature review, control room observations, interviews, focus groups, and questionnaire. The

WDA [58] was validated with an operator, and 2 managers who were former operators.

4.2 Summary of the WDA

4.2.1 Abstraction Hierarchy

The electrical power grid may be described at the following five different levels of abstraction, in order

of most to least abstract:

23

Chapter 4. Work Domain Analysis 24

1. Functional Purposes

2. Abstract Functions, Values, and Priority Measures

3. Purpose-related Functions

4. Physical Functions

5. Physical Objects

We started with the functional purposes, which came from our discussions with operators. We then

described the power grid at the physical object level, and worked our way up the rest of the levels of the

abstraction hierarchy.

Functional Purposes

The three purposes of power grid operations can be summarized as: (1) reliability of the power system

and enforcing interconnection reliability standards; (2) quality of service, minimizing outages, voltage

variations, and equipment failures; and (3) efficiency of electricity markets. When making critical deci-

sions, operators have to assess the benefits and risks between this set of overarching goals.

Physical Objects

At the most concrete level, the power grid can be described by the physical appearance and geographical

location of the equipment that makes up a power system. Examples include the transmission lines,

stations, and weather patterns.

Physical Functions

Grid elements have functionality most often described by circuit schematic diagrams, which are irre-

spective of the elements’ physical appearance. Examples of physical functions include the transmission

network (which conducts electricity between equipment, and has a characteristic impedance), transform-

ers (which convert voltage between sections of the transmission network), generators, and loads.

Purpose-related Functions

This level of abstraction more broadly describes what the physical functions do, and can help illustrate

potential problem-solving paths that are available to operators. Such purpose-related functions include

power transmission (real and reactive), generation, consumption, and imports/exports.


Abstract Functions, Values, and Priority Measures

This level describes the grid on the basis of general scientific (and economic) understanding. This infor-

mation often comes from simulations, statistical analyses, and technical models developed by planning

engineers for control room use. Priorities are defined by regulatory standards such as those set by NERC.

Examples include maintaining dynamic stability, matching power supply and demand, and minimizing

net inadvertent interchange.

4.2.2 Means-Ends Links Between Abstraction Levels

The abstraction hierarchy diagram in Figure 4.1 illustrates means-ends links between elements of the

power grid. These links represent the potential trade-off effects of equipment on system purposes, and

the tools available to operators in order to meet their overarching goals.

4.2.3 Part-Whole Decomposition

Individual pieces of equipment on the power grid are clearly defined as in component lists, and the

scope of the power grid can expand beyond our system boundary of a single RC Area. Because electric

power grid components are tightly connected and interdependent, intermediate levels of part-whole

decomposition are more complex to describe and depend on the situation.

Functional purposes and abstract functions of the power grid are delineated by the different regulatory

requirements they are associated with, and the concepts that they represent. Purpose-related functions

describe power flow, either between individual lines, grouped in the form of SOLs, or furthermore grouped

into the Area Control Error1 calculation for a whole jurisdiction. The physical function of equipment

may be grouped according to function within an area, and connectivity with other components. At the

physical object level, grid components can be grouped by transmission yard.

4.2.4 Topographic/Causal Links

The electric grid is physically made up of equipment connected together with wires. These links have

the physical function of electrical conduction, and circuit breakers can change the grid topology - for

example, represented in node/breaker models in the EMS. Purpose-related functions are linked through

1Area Control Error (ACE) [59] is a calculated value (in MW) that represents power balance compared to schedulewithin an area, plus a small “bias” obligation to maintain frequency.


power flows, which can be derived in computational breaker models. Abstract functions are linked by

cause-and-effect relationships between elements - for instance, energy bids and asks affect power balance,

which may require changing generator output, and would in turn affect system robustness and stability.

Functional purposes are somewhat interrelated in that unreliability can lead to inefficiency and low

quality, but the reverse may not necessarily be true.

4.3 Application of the WDA to Wide-Area Monitoring

Wide-area monitoring encompasses a large geographical scope, by definition. In order to avoid infor-

mation overload in capturing this large scope, the data should be displayed in the form of high-level

summary views. The WDA was useful for capturing the information requirements at the highest levels

of abstraction, and the potential paths for problem-solving at a wide-area view of the grid.

4.3.1 Design Scope in the Abstraction Hierarchy

As discussed in the wide-area monitoring definition developed during the focus groups (Subsection 3.2.2),

operators were primarily concerned about the impact of grid events within and outside of Ontario, and

at the interconnections. Power flows, outages, and the external model were recurring themes in the

discussion.

My design concept thus focused on a subset of the abstraction hierarchy concerning: 1) matching

power supply and demand, and 2) minimizing net inadvertent interchange. This does not preclude the

development of wide-area displays that capture other abstract functions, but rather that these two most

closely reflect wide-area monitoring as per the NERC standard for the scope of this project.

These two abstract functions help achieve the purpose of quality of service, and in turn, reliability

that partly depends on quality. The means to achieve these two abstract functions (as seen in Figure

4.1) are: real power transmission, real power consumption, operating reserve, real power generation, and

import/export flows.

Since the prototype is intended as a high-level overview display, physical function and physical object

descriptions were outside the design scope.


4.3.2 Measures and Constraints at Different Abstraction Levels

To develop wide-area monitoring information requirements from the abstraction hierarchy [15], the model

has been converted into the following list of variables for each level of abstraction.

Functional Purposes

• Area Control Error (ACE) (MW)

• Frequency (Hz)

Abstract Functions, Values, and Priority Measures

• Power supply (MW)

• Power demand (MW)

• Net interconnect power transmission flows (MW)

• Scheduled import/export flows (MW)

Purpose-related Functions

• Power transmission flows within Ontario (MW)

• Power consumed (MW)

• Operating reserve capacity (MW)

• Spare generation capacity (MW)

• Power generated (MW)

• Tie-line power transmission flows (MW)

• Tie-line import/export schedules (MW)

The ACE for a region is ideally at or close to 0, which would indicate power balance according to

schedule and an acceptable transmission frequency. Any sustained absolute value in the hundreds or

more may indicate a major failure in the system, such as a large generator outage.

Power line frequency in North America is set at 60 Hz. Deviations from this standard are kept to a

minimum, since power system components are designed for the operating frequency. Frequency control

is also an important aspect of power grid operations because it is one measure of the load and generation

balance. Therefore, operators have to monitor system frequency despite it already being a component

of the ACE calculation.


Power supply is constrained by the power generation infrastructure built and connected to the grid.

Supply and demand are constrained by the transmission and distribution systems that deliver power to

consumers and have voltage and current limits.

Import/export flows between RCs are scheduled 24 hours in advance, although they may issue re-

quests to change import/export values in a shorter time frame. Scheduled and actual imports/exports

are constrained by the transmission infrastructure connecting two jurisdictions, and the power flows

within each jurisdiction.

Power flows within each jurisdiction are restricted by System Operating Limits (SOL) developed in

engineering simulations.

4.3.3 Using the Information Requirements

I sketched basic graphics from these information requirements and presented these preliminary design

ideas in tabletop discussions with operators. The feedback from these tabletop discussions, and the

subsequent design and evaluation process, are described in the next chapter.


Fig

ure

4.1:

Ab

stra

ctio

nh

iera

rchy

ofp

ower

gri

dop

erati

ons.

Ele

men

tsre

leva

nt

tow

ide-

are

am

on

itori

ng

are

hig

hli

ghte

din

gre

enb

oxes

.

Chapter 5

Design and Evaluation of Wide-Area

Monitoring Concepts

Using the information requirements gathered from the WDA, I sketched preliminary design concepts

on paper and in a MS PowerPoint slide deck. In tabletop discussions, the slides were presented to 4

operators for their feedback. Operators were asked a series of questions to clarify their information needs

during operation.

The ideas and feedback were used to develop an interactive prototype, also in MS PowerPoint, from

the results of the tabletop discussions. The prototype was validated in a usability evaluation with 9

operators. Their suggestions were incorporated into the next iteration of the design prototype.

This chapter details the process from preliminary design ideas to evaluating the interactive prototype

with operators.

5.1 Preliminary Design Ideas and Feedback

I sketched preliminary design ideas on paper and in MS PowerPoint. During tabletop discussions with

4 off-shift operators, I asked a series of questions about information requirements for control room

operations and solicited feedback on the design ideas. Each operator was consulted individually, and

each session took approximately 30 minutes.

30

Chapter 5. Design and Evaluation of Wide-Area Monitoring Concepts 31

Table 5.1: Critical parameters of system operation.

Parameter Included in prototype?Frequency YesMajor interface flows YesACE values YesLimits (internal and external) Somewhat (external model missing)Topology (critical infrastructure) Somewhat (required at lower levels)Generation• Total generation Yes• Total load Yes• Total wind, solar generation Yes• Major generating stations YesLargest contingency - operating reserve relationship YesFlow gates YesBoundaries (circuits that make up each interface) No (for simplicity)Any interfaces phase-shifted NoVoltages (perhaps compared to historical) No (mixed opinions on necessity)Automatic Generation Control (AGC) No (mixed opinions on necessity)

5.1.1 Critical Parameters

One operator suggestion from our critical incident interviews was to have a set of critical parameters

of system operation to gauge overall system state, similar to what is employed in nuclear power plant

operations [60]. The notion of critical parameters is in line with higher levels of abstraction, and might

include values that operators would want to monitor at any time. The Macomber Map developed by the

Electric Reliability Council of Texas (ERCOT), for example, has a dashboard showing ACE, frequency,

generation, load, wind, number of islands, and Physical Responsive Capability1 (PRC) [61].

Table 5.1 lists what participants considered critical parameters of system operation. Most of these

parameters were implemented in the prototype, though some were not due to:

• Information unavailability: We did not have access to power grid models external to Ontario.

• Appropriate level of detail for an overview display: The wide-area overview would risk being too

cluttered with detail if circuit topology was included.

• Mixed opinions between participants: Some participants disputed the necessity of voltages or AGC

on the overview display.

To follow up, I asked participants, “What would go on a dashboard?” and proposed a set of parameters

that would go on a dashboard seen at the top of every screen: ACE, frequency, total generation, total

1This term appears to be exclusive to ERCOT, and is synonymous with operating reserve.


load, number of islands, largest contingency, and operating reserve. Participants suggested the removal of

number of islands, as it did not provide context, and suggested the addition of wind and solar generation

numbers. Having these parameters appear at the top of every screen would allow operators to gauge the

system state as it reflects the overall functional purposes.

5.1.2 External Geographical Scope

Five jurisdictions have interties with Ontario that would be required in a wide-area overview: Quebec,

New York, Manitoba, Minnesota, and Michigan. The overview would include the ACE (as a summary

of supply/demand), net interconnect power transmission flows, and scheduled import/export flows for

each jurisdiction.

Participants also suggested that they would like summaries for the reliability coordinators Midcon-

tinent Independent System Operator (MISO), which includes Manitoba, Minnesota, and Michigan; and

the PJM Interconnection, which includes Pennsylvania, New Jersey, and Maryland. Although the PJM

Interconnection not directly connected to Ontario, it is large enough to affect the province’s power grid.

Figure 5.1 shows the preliminary concept for displaying these jurisdictions and RCs. This would

later be developed into the wide-area view of the design prototype.

Figure 5.1: Preliminary concept for displaying external jurisdictions that affect the Ontario power grid.


5.1.3 Modelling the Ontario Power Grid

All power flow limits fall into the category of System Operating Limits (SOL). A subset of these SOLs

is considered crucial for maintaining stability in the overall Eastern Interconnect, and these limits are

known as Interconnection Reliability Operating Limits (IROL).

To summarize the Ontario grid at the system view, I proposed a one-line diagram that showed the

links between Ontario’s IROLs (Figure 5.2). The diagram would act as a summary of power transmission

flows within the province. The first participant pointed out an existing zonal demand overview display

already in their EMS, where the province’s grid was separated into “zones,” each with generation and

consumption data.

Figure 5.2: Preliminary design for a one-line diagram showing links between Ontario’s IROLs.

The system level of the prototype borrowed the pre-existing layout of these Ontario zones and their

relationships with IROLs. The stick-and-circle layout (Figure 5.3a) was more suitable for modelling

the Ontario regions compared to the one-line diagram, because different status numbers for each region

could now be displayed in one graphic. These status numbers include: power consumed, operating reserve

capacity, spare generation capacity, and power generated. Knowing the available operating reserve or

spare generation capacity, and generation numbers, will help operators decide what control actions are

available to match power supply and demand in the province.

IROL power flows, by definition, affect flows at the interties between jurisdictions, and operators

monitor IROLs to minimize net inadvertent interchange. As seen in Figure 5.3a, tie-lines are shown on

the Ontario system level, along with their power transmission flows and import/export schedules. For


Figure 5.3a: After tabletop discussion feedback: Stick-and-circle layout for modelling Ontario regions atthe system overview level, showing generation and load numbers.

Figure 5.3b: A toggle in Figure 5.3a would also show the operating reserve and spare generation numbers.

example, a power flow reaching its IROL may cause a tie-line flow to be above its scheduled value; the

display would alert the operator to this inadvertent interchange and the area of the cause.


5.1.4 Generation

One way to help operators match power supply and demand is to allow them to monitor the supply

coming from generators. I proposed a list of generators, with various options for sorting: type of

generation, MW generating, MW capacity, geographical region. Operators commented that a list format

was not helpful to them, considering the importance of being able to draw connections between the

generators and the rest of the grid (as at the physical function level).

Instead, a graphical form that captured the generators and their relationship with critical parameters,

such as power flow limits, would be more relevant to their operational goals. The Ontario zonal layout

was helpful for creating a generation summary page in the prototype that captured generation and load

amounts for each zone (Figure 5.3b).

5.1.5 Alarm Notifications

I proposed an alarm notification panel that would provide context as to what happened, the potential

cause, and links to potential operator actions. An example is shown in Figure 5.4.

Figure 5.4: Preliminary design for alarm notification panel.

The concept was positively received, and participants did not have any other suggestions for notes

to add in the alarm notifications.


5.1.6 Contours

Colour contours were explored as a way to help operators quickly diagnose abnormal states by changes

in colour or contour shape. There are several possibilities for what the contours may capture: voltage

[22], frequency [62], and power flows [63] have been previously proposed.

Between these three proposed parameters for a colour contour, participants felt that voltage would

be the most useful. Frequency deviations rarely occur, and are caused by circuit separation that would

have already triggered other alarms. Power flows were a possibility, but because their limits widely vary

by area (compared to voltages), comparisons between power flows may not necessarily be useful.

Contours were not implemented in the prototype due to technical limitations of the prototyping

software. However, there may be a need for voltage contour maps to supplement the prototype - later

in the design process, one operator did suggest the addition of a voltage monitoring overview.

5.2 Design Prototyping

5.2.1 Context

The wide-area monitoring display concept focuses on visualizing the power grid at the multi-jurisdiction

(Ontario and its neighbours) and system-wide (Ontario) levels. In practice, it would be integrated into

an EMS as a central monitoring tool to help operators detect abnormal events across the power grid.

It would be suitable as a control room wallboard display or as a landing screen for the EMS desktop

interface. The EMS would have other displays, not described here, with finer levels of detail to aid

problem-solving.

5.2.2 First Interactive Prototype

The first interactive prototype was developed in MS PowerPoint. This first iteration (see Appendix

B) was used in the usability evaluation. The hierarchy of detail was: wide-area view – system view –

detailed overview. Further detailed views were outside the design scope of wide-area monitoring.

A detailed description of the iterated design, after incorporating usability evaluation feedback, is

described in the next section.


5.3 Iterated Design Prototype

5.3.1 Wide-Area View

The wide-area view in Figure 5.5 shows the Ontario grid in relation to its neighbours’. It namely features

imports/export schedules and ACE values.

Figure 5.5: Iterated design prototype: wide-area view that includes external jurisdictions.

Imports/exports are shown as a trend chart comparison between actual and scheduled flows. This

allows operators to determine whether net imports and exports are following pre-determined schedules

for the day, and whether any net inadvertent interchange requires a control action. If there is a significant

discrepancy between the two, a bracket (yellow for warnings, red for alerts) labelled with the difference

appears. As these trend charts are for comparing intertie power flows with schedules and for monitoring

trends, the MW vertical axis range is dynamic to ensure trends are visible. The trend charts do not

have axis labels in order to reduce screen clutter.

Arrows between the jurisdictions show the direction of power flow. The thickness of the line corre-


sponds (though a non-linear relationship, so that small flows are not hidden) to the MW of power flow

along all interties between two jurisdictions.

At the top of the screen (and for all others in the prototype) is a dashboard showing ACE, frequency,

total generation, total load (+ dispatchable load), largest contingency vs. operating reserve, and current

wind and solar generation. The alarm button opens up an alarm notification pane, described in Subsec-

tion 5.3.4. A search box allows the operator to go directly to the location or equipment they are looking

for, by name. Constraints on the ACE-frequency relationship and allowed period for ACE deviation are

assumed to be already visualized in other displays like the Balancing Authority ACE Limit radar [64].

5.3.2 System-Wide View

The system-wide display was initially split between a power flow view (focused on monitoring power

flows and their limits) and a generation view (for examining generation across different regions of the

ICG). However, participants suggested combining the two, and showing limits according to the most

constraining (or situationally relevant) ones.

For example, FS (short for “Flow South”) is shown in the example scenario in Figure 5.6 between

the Northeast and Essa, because power flow along this transmission line grouping is flowing south, as is

typical during the daytime. During night times when power flow is going north, the FN (“Flow North”)

flow limit is situationally relevant, so it appears in place of FS.

When power flows are approaching the limit, a bar chart appears as a visual warning; the bar is

yellow when approaching, and red when exceeding the limit. The breakdown of generatio

designing for wide-area situation awareness in future power … · 2016. 6. 22. · abstract...

Documents