Results Based Draftingfor

WECC Drafting Teams

W. Shannon Black

Manager, Standards Processes

Overview

●Why change?

●Changes to the process Scope Early determine of the roadmap

●Changes to the documents New template Where things go

2

Drafting Process / Sequencing

Why change?

● “R”equirements need to improve Current “R”equirements

Are unclear Administrative not results Impossible to achieve! Measure! Voluntary Devoid of drafting conventions

● Failure to identify the work product up front is slowing the Process

● Where are we headed?

Where are we Headed?

● Emphasis on Defining the Scope (product)

Before we draft

Defining the Requirements Before we draft Using proper drafting conventions Using common sense Using a sequencing concept

Defining the Review the process Quality control review at each posting

Define the Scope

Get consensus before writing requirements

• NEED: Why the standard is being developed or modified• GOALS: What must be accomplished to meet the need• OBJECTIVES: Criteria for success• OPERATIONAL CONCEPTS: Day in the life• STAKEHOLDERS: Individuals or groups impacted or

interested• DRIVERS: External constraints

Define the Requirements

Write results-based requirements

Review and Approve

• Collect comments• Revise in response to comments• Recycle per planned schedule

• 45 and 30 day Comment windows• Complete a quality review check list• Submit for approval

• Team votes – forwards to the Standard Committee

Scope

What you need to know and agree upon in What you need to know and agree upon in order to develop a standard and write order to develop a standard and write requirementsrequirements

Gathering the tools for the tool boxGathering the tools for the tool box

Defining Scope

● View the maze from the top! See the problems before wasting time

● You think you know… …but you don’t know you know.

● Defining scope Defines the boundaries Defines the target

Defining ScopeOperational Concepts/ Defining the Problem

• Identify the OPERATIONAL CONCEPTS• Understand the operational fact pattern

• What specifically happened to create the problem your team will try and resolve?

• How do we address it today?• Make a list of the steps that are taken TODAY to meet

the operational concern

• How should we address it tomorrow? • Make a list of “first blush” suggested changes to the

existing operations.

11

Defining Scope

• Identify the STAKEHOLDERS• Who was impacted by the fact pattern?• Who will be impacted by the remedy?

• Identify the DRIVERS• What’s pushing this whole effort?• Get the “agendas” out on the table

12

Defining ScopeWhy identify the stakeholders and drivers?

• The impact is broader than the Functional Model

• Vested Interested• Lineman / Back Office / After-the-Fact / Schedulers• “Mom and Pop”

• Influential Interests• Government• Stock holders

• Participating Interests• Usually the “Applicable Entities”

13

Defining ScopeWhy identify the stakeholders and drivers?

• The end product has to reflect reality

• Consider potential drivers• Technology• Deadlines• Existing Standards / Existing Processes• Personal and Corporate agendas• End user expectations

14

Purpose Statement

What you need to know and agree upon in What you need to know and agree upon in order to develop a standard and write order to develop a standard and write requirementsrequirements

Gathering the tools for the tool boxGathering the tools for the tool box

Defining ScopeFocusing on the Purpose Statement

• Identify the NEED• Why the document is being developed or

modified?• “To ensure uniform maintenance

procedures…..”

• Identify the GOALS/OBJECTIVES• What do we have to do to meet the needs?

• “…by defining maintenance periodicity.”

• Lays out the criteria for success

16

Defining Scope “Purpose” Exercise

17

Defining Scope “Purpose” Exercise● What’s wrong with this picture?

● Purpose Statement for EOP-004-1 “Disturbances or unusual occurrences that

jeopardize the operation of the Bulk Electric System, or result in system equipment damage or customer interruptions, need to be studied and understood to minimize the likelihood of similar events in the future.”

18

Defining Scope / “Purpose” Small Group Exercise● What’s missing?

● What’s included that need not be?

● Would it help to know the fact pattern?

● What’s the intent (goal/objective) of the document? Can you tell? Can an auditor tell?

● How do we know when we “did it”? Can we compare the Purpose statement to the

final document and know “we’re done”?19

Defining Scope “Purpose” statement● Suggested rewrite

NEED “To improve BES reliability…”

GOAL “…by reporting and analyzing Impact Events…”

Objective “…for the purposes of minimizing repeat events.”

● This is just one solution

20

Drafting the PurposeSmall Group Discussion● Review the Team’s Requests

Identify the fact pattern and issues Sometimes called Operational Concept Consider what is being done now and what changes

need to be made

● Identify the NEED E.g.. “To create a standardized process…”

● Identify the GOAL E.g.. “…used by every Balancing Authority…”

● Identify the OBJECTIVE E.g.. “…to measure Time Error Correction.”

21

Requirements

Regulatory Facts of Life

● Requirements are boring This reflects uniformity and pattern

● Requirements look the same “Each BA shall do X.”

● Requirements are a “language” People don’t write like they talk

The spoken word is communicated differently than the written word

People dislike taking time to write at all! Your job is to take the time

23

Regulatory Facts of Life

Things to Correct

● Don’t provide adequate reliability “Documentation heavy”

● Can be interpreted different ways Subjective terms Poor grammar Multiple requirements in a single statement

● Don’t clearly state responsibility

● Can’t be measured

● Can’t be enforced 24

Regulatory Facts of Life

Drafting Conventions

● Avoid Ambiguous Terms Adverbs (words that end in “ly”) Pronouns (it…this…) And / Or Associated / Affected / Accommodate Coordinate / Facilitate Maximize/Minimize Unique / “Mutually agreed upon” “be able to” “capable of”

Wouldn’t you rather they just do it?25

Regulatory Facts of Life

Drafting Conventions

● Do Use the structure Formatted style of language NERC Functional Model Make sure the newest staff member understands

the document

● Don’t Incorporate by reference Create new terms Abbreviate

26

Results Based Requirements

Lock Down the Structure

● Basic components Identifies each affected party

“Each Balancing Authority…”

Is mandatory “…shall…”

Identifies the task “…operate [in a specific fashion].”

To achieve a particular result “…to ensure a balanced interchange.”

Measureable and objective

Results Based Requirements

Create an Affirmative Result● To create an Affirmative Requirement

Meet the Purpose of THE ASSIGNED document Simply because it’s an excellent requirement

doesn’t mean it belongs in “this” standard Increase the reliability of the system

Meet one of the NERC reliability principles Not just admin for the sake of admin

At an affordable cost (Common Sense Test) Is actually feasible / possible / affordable

Internally and externally consistent

Results Based Requirements

Understood the same way by All

FERC

Electrical Providers NERC

If a team sees multiple interpretations – rewrite it!If the newest guy in your shop can’t understand it – rewrite!

If your supervisor can’t understand it – rewrite it!If your friends don’t understand the sentence structure – rewrite!

It’s New!Enhance your Requirements

● It’s acceptable to use Graphs Pictures Charts Examples of how the Requirement

works Sample calculations

When Requirements go wild!Exercise

What’s wrong with this Picture?

● Responsible entities may schedule as appropriate.

● Each TO shall eliminate all lower quality timing loops that occur when two network devices line time off of each other under normal conditions without traceability to a Stratum 1 reference.

● Each Balancing Authorities shall perform all calculations based on the March 10, 2010 Table found at….”

● Each Scheduling Entity shall schedule in accordance with NAESB e-Tag requirements.

Rationale

What is the Rationale block?

34

BB.. RReeqquuiirreemmeennttss aanndd MMeeaassuurreess

R1. Each [Applicability Entity as stated in the current NERC Functional Model] shall [perform what task] to [state “why?” such as “to avoid a Sustained Outage.”]

(Note: Do not draft Sub-requirements. Each Requirement should be free standing. Use of “bullets” indicates “may include any of the following.” Use of numbers indicates “must include all of the following.”

M1. Evidence of violation of Requirement R1 includes: [Note: Measures are now located just below the Requirement. Draft one Measure per each Requirement.]

[Objective measurement] [Objective Measurement]

Rationale

Helps explain the rationale behind the Requirement such as: 1) assumptions made, 2) justification for numbers used or, 3) the specific goal that is to be achieved.

Why have a Rationale block?

“In today’s world of high employee turnover and complex, long-lived products, well-

documented rational is essential to better product development.” Ivy Hooks.

● Benefits Key to understanding Reduce interpretation problems Facilitate maintenance and upgrades Preserve corporate knowledge

35

What goes in the box?

●Why a requirement is needed ●What assumptions were made●What analysis effort drove the

requirement● Information to help understand the

requirement●Source of any numbers

36

What does NOT go in the box?

● A rewrite of the requirement

● Hidden requirements

● Copy of another requirement’s rationale

● Everything you know on the topic

37

The Requirement must Stand Alone

● “Third, we understand that the proposed results-based standard format may include expanded background sections, expanded descriptions of a Reliability Standard’s purpose, and/or explanations about the intent of individual Requirements. This information may provide useful context but it should not contradict or seek to supersede or interpret the requirements within a Reliability Standard. The requirements within the Reliability Standard should govern, and the application of the Standard should be clear without reference to the background or purpose sections.” Docket Nos. RR09-7-000 and AD10-14-000, P. 109

Measures

Measures have Moved

40

BB.. RReeqquuiirreemmeennttss aanndd MMeeaassuurreess

R1. Each [Applicability Entity as stated in the current NERC Functional Model] shall [perform what task] to [state “why?” such as “to avoid a Sustained Outage.”]

(Note: Do not draft Sub-requirements. Each Requirement should be free standing. Use of “bullets” indicates “may include any of the following.” Use of numbers indicates “must include all of the following.”

M1. Evidence of violation of Requirement R1 includes: [Note: Measures are now located just below the Requirement. Draft one Measure per each Requirement.]

[Objective measurement] [Objective Measurement]

Rationale

Helps explain the rationale behind the Requirement such as: 1) assumptions made, 2) justification for numbers used or, 3) the specific goal that is to be achieved.

Measures

● One per customer! One Measure per Requirement Draft upon completing the Requirement

● Each Measure must: Identify the functional entity Tangible, practical, objective Identify what evidence or types of evidence could

be used to show that an entity is compliant with the requirement.

41

Comments

Dealing with Comments

● Important to consider all carefully

● Not important to agree to all

● Don’t waste time playing the assumptions game

● Do not hesitate to ask for clarification

● If many comments from one person or group – ask for priorities.

● Categorize if necessary 43

Questions?

Getting Started

● What’s the issue/fact pattern? Review the Request

● What needs to be fixed?

● How do we do it now? List the “Day in the Life of” today

● How do we propose to fix it? List the proposed “Day in the Life of” for

tomorrow This is the skeleton for our document!