LESSONS LEARNED FROM A $5 MILLION SCADA UPGRADE

Columbus Water Distribution System

Phil Schmidt, PEJohn Gilmore

Overview of Distribution System

• Service to approximately 1.2 Million Population

• 135 MGD Average

• 3,500+ Miles of Water Lines Maintained by Division of Water

• 13 Primary Pressure Districts

• Franklin County and parts of Fairfield, Licking, Delaware, Union, Pickaway

Operations Control Center

Operates and Maintains:• 35 Elevated Storage Tanks• 3 Ground Tanks• 28 Booster Stations • 24 SCADA Controlled Valves• Operate High Service Pumps at 3 water

treatment plants

• 4 crews – Operators, Techs, Mechanics, Electricians

• Management/Support• 40 staff

Need for Upgrades

• Obsolescence of Equipment – Replacement parts and technical support issues

• 18 years old – host system• Pre-9/11 – Security concerns were different

• Risk– Longer term disruption to SCADA system would have likelihood of rolling losses

to thousands of customers’ water service

Overview of Upgrades

• Radio Network Improvements

• Construct remote backup control center

• Renovation of existing main control center

• Upgrade all RTUs in the system

• Improved Security

Radio Network – Existing

Control Center

Booster Station A

Tank 1 Booster Station F

Repeater

PRV 4

Booster Station E

RepeaterBooster

Station D

PRV 3

Repeater

Booster Station C

PRV 2

Repeater

Booster Station B

PRV 1

Existing Network - 900 MHz Entire system dependent on Control Center

Tank 2

Tank 3

Tank 4

Tank 5

MCC

Repeater

Repeater

BCC

Repeater

Repeater

Booster Station D

PRV 4

Tank 4

Booster Station C

Tank 3

PRV 3

Booster Station B

Tank 2

PRV 2

Booster Station A

Tank 1

PRV 1

Radio Network – Redundant Wireless Ring

900 MHz network remains in placeMCC, BCC and Master Repeaters are connected with a 5.8 GHz loop.If any site on the loop goes down, the data can still travel on the ring.

RTU Upgrades• Schneider (Telvent) Sage 2400 boards

• Upgrade from Sage 2200 boards

• Purchased 85 through contract and they were installed at all remote locations by our control center staff

• Improved Security

Backup Control Center (BCC)

• Constructed in a secure remote location

• 2 operator stations• Diesel standby generator• HVAC system• Dimmable LED lighting

• 30 day BCC operational demonstration before starting demolition/remodeling of MCC

Backup Control Center (BCC)

Main Control Center (MCC)• Operator Console

– Adjustable Height– 2 Operator Stations and 1 Engineering Station

• Video Wall Display – Four 55-inch monitors

• Incident Command Room – Conference Table and Wall-Mounted Monitors

• Lockable Wardrobe Style Lockers and File Cabinets for Each Operator

• Kitchenette

• New HVAC system

• Dimmable LED Lighting

Main Control Center – Construction

Main Control Center – Construction

Main Control Center – Construction

Main Control Center – Construction

Main Control Center – Construction

President’s Day Surprise – Water Damage

Newly installed plumbing fixture failed sometime over holiday weekend

Water sprayed into control room until discovered Tuesday morning after holiday

Items Destroyed:• Carpet• Kitchenette Casework • Sink Faucet• Wood Lockers

President’s Day Surprise – Water Damage

President’s Day Surprise – Water Damage

President’s Day Surprise – Water Damage

Challenges During Construction

Tower Variance

Graphics

• Graphics were converted to new format

• Conversion process compatibility issues

• Should have recreated them instead of convert

Electrical Issues

• Generator

• UPS

• Restrictions from existing variance

Self-Healing Radio Ring

Completed Control Room

Completed Control Room

Completed Control Room

Completed Control Room

Completed Control Room

Old New

Completed Control Room

Project Details• Construction Cost is $5.2 Million

$4.6 Million + $600K Contingency

• Prime Contractor – Schneider Electric BuildingsSubcontractors:

Schneider Electric (Telvent)Corna KokosingJ&K CommunicationsMid City Electric

• Design Professional – CDM Smith• Construction Administration - DLZ

Going Forward

Recreate Graphics

More Frequent Upgrade Cycle

More Nodes On Wireless Ring

Slide Number 1Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34