NPPD’s Proposed CIP-007-6 R.2 Patch

Management Program

BackgroundCIP-007 R2

With CIP V3, only NPPD Control Centers are required to comply to CIP-007-3 due to serial connectivity to other sites.

CIP V6 expands the scope to our substations.

With V6, we are improving our current Patch Management spreadsheet only process

All our Medium BES Cyber Systems are without External Routable Connectivity.

Security Patch ManagementCIP-007 R2

R2.1: Security Patch Management Process

Must have a process for tracking, evaluating and installing cyber security patches.

Nothing to do with functionality patches, only cyber security patches.

In a nutshell, it states we have to create a plan, and follow it.

Security Patch ManagementCIP-007 R2

R2.2: Security patches must be evaluated within 35 days of their release.

Must check websites, vendor contacts or email lists for notifications of cyber security related patches a minimum of every 35 days.

Patches found are then evaluated for applicability to our devices.

Security Patch ManagementCIP-007 R2

R2.3: Security patch application process

Within 35 days of a cyber security patch evaluation, you have three choices: Apply the patch Create a dated mitigation plan Modify a dated mitigation plan

A mitigation plan does not necessarily mean applying the patch. It can also mean finding another way to address vulnerabilities resolved by the patch.

Security Patch ManagementCIP-007 R2

R2.4: Mitigation plan for security patches

The plan you create must be followed. If you don’t follow it, a new plan must be created and approved by the CIP senior manager or delegate.

SAP Preventative Maintenance Plan

A program that automatically creates orders and operations in SAP at pre-determined intervals.

Each operation has a start and due date and attention is raised if tasks are not followed, as scheduled.

Once all operations are confirmed, the PM plan is complete.

Using the PM Plan

In order to evaluate a patch within 35 days of its release, you must look for patches a minimum of every 35 days.

Use a PM Plan to create operations that must start every 28 days and be completed within one week.

These activities include: A patch checker activity for each type of device to

find new available patches. A patch reviewer (analyst) activity to review found

patches and evaluate them for security applicability.

Patch Checker Activity

Using our baseline spreadsheet, review email lists, websites, etc. for new patches available.

For each new patch, create an SAP notification stating that a new patch is available for a certain type of equipment. This notification is assigned to the FPC (Facility Protection Committee) planning group.

Update the baseline spreadsheet with the new patch information and notification number.

Confirm SAP PM operation for evidence.

Days 1-7

Patch Reviewer/Analyst Activity

Using the baseline spreadsheet completed by the patch checker, review newly available patches for: Cyber security fixes Applicability to our equipment

Update existing notification with the conclusion (applicable or not applicable)

If not applicable, close notification. Process is complete.

If applicable, the notification is used for the next steps.

Confirm SAP PM operation for evidence.

Days 1-7

CIP-007 R2.2 Completed By checking for patches and evaluating their

applicability to our equipment in a period of less than 35 days, we meet requirement CIP-007 R2.2.

This process must happen in a week’s timeframe to allow other processes to follow their schedule and not fall outside the 35-day window.

Next step is to figure out how to fix the cyber vulnerability the patch addresses.

Entering the Work Management Process

A superior work order is created from each notification remaining after the patch-checking process.

Includes operations for the analyst to test the available patches, create mitigation plans and update baselines.

Days 8-28

Analyze Patches

Patch analyst has to determine if patch can be installed.

Analyst does this by implementing the patch in our test lab and testing the equipment to ensure it performs as needed with no undesired consequences.

From those findings, the analyst creates a detailed plan to implement, or not implement and fix the vulnerabilities without patching, and updates the notification with their recommendation.

If the patch is to be installed, analyst will update the baseline with the patch to be installed.

Days 8-28

Create Plans Another PM plan kicks off to get a group of subject matter

experts together to create dated mitigation plans for each patch deemed applicable from the previous PM plan.

This plan includes operations for different work groups to decide how and when they will patch the equipment.

Ideally, this group would include representatives from each group that maintains cyber assets (compliance, telecom, protection engineering, SCADA, substation maintenance, etc.).

This period of time is also days 1-7 of the next PM for the checkers and analysts.

This group of SME’s meets every four weeks. They are on the same, every 28 days, must be completed in a week, schedule as the patch checkers.

Days 29-35

Facility Protection Committee (FPC) The group of SME’s is called the FPC.

They review notifications assigned to the FPC planning group, which are patch notifications created in the previous PM plan.

Notifications reference each type of equipment. This group must find all pieces of equipment that fall under the types that need patching.

For every piece of equipment that needs patching, a dated plan must be created as a sub-order under the superior order.

Unless a plan to patch said device exists, that plan is simply modified to ensure the new patch is installed.

Days 29-35

Superior and Sub Orders Superior work orders created from the initial patch

notification is for evidence purposes.

The notification can only be tied to one work order, so it gets tied to the superior one.

Superior orders cannot be closed (completed) unless each of its sub orders are closed first. This ensures all work is completed before the evidence (superior) order can be completed.

Once all notifications are turned into superior orders and each device that needs patched has a dated order, this assignment is complete.

Each member can close their operation and the PM plan is completed.

Days 29-35

Timeline(n=0)FPC Meets to screen for FPC notifications. Plansmust be created/modified to address patching.

(n=1)FPC Meets to screen for FPC notifications. Plansmust be created/modified to address patching.

(n=1)PatchChecker/Reviewer Newly available patches must be evaluated. Notifications are created for new patches available.

(n=1) Patch Analyst Patches evaluated as applicable must be analyzed for acceptance in our test equipment. New baselines must be created if a patch is to be installed. Notifications are updated with analysts recommendation for patching/mitigation.

(n=2)PatchChecker/Reviewer Newly available patches must be evaluated. Notifications are created for new patches available.

Planner/Schedulers Screens for operations that are due. Verifies that all responsible persons complete their work as scheduled. Also, screens for work orders to complete patching. If patching does not look like it will be completed on time, a NERC deferral process must be started to modify the current plan with proper approval.

Day 0 Day 7 Day 28 Day 35

CIP-007 R2.3 Completed

By creating orders for each piece of equipment that requires patching in a period of less than 35 days after the patch reviews, we meet requirement CIP-007 R2.3.

This process should have happened in a period of a week so as not to fall outside the 35-day window following the evaluation process.

Next step is to ensure the work is carried out as planned.

What if the plan doesn’t work? Unforeseen events can make patch implementation by the

due date not possible.

If work cannot be completed as planned, it can be rescheduled.

We use a PRC-005 protection system maintenance deferral example already in place to help defer patching on BES Cyber Systems that can’t be completed on schedule.

This deferral process does not allow the interval to exceed the maintenance intervals in PRC-005. The same would be true for the CIP-007 intervals.

Our planner/schedulers monitor schedules and work and will notice if they will not make a due date.

NERC Preventative Maintenance Deferral

If the planner/scheduler notices a patch cannot happen as planned, he/she will kick off the NERC PM Deferral process.

A new plan to complete the patch has to be created and a modification to the sub-order is made.

Approval by the CIP senior manager or delegate is required to complete the new plan.

CIP-007 R2.4 Completed

By following our original plans or getting CIP senior manager or delegate approval to modify our plans, then complete them, we meet requirements of CIP-007 R2.4.

CIP-007 R2.1 Completed

By having this process to complete all cyber security patches to cyber assets, we are compliant with CIP-007 R2.1.

Mission Accomplished

Once all sub orders are completed, the superior order can be closed, and work is considered completed.

It will serve as evidence that we completed our required patching as planned, and we are compliant with CIP-007 R2.