preventing unplanned downtime through alarm...
TRANSCRIPT
Preventing Unplanned Downtimethrough Alarm Management:The Five Most Important Alarm ManagementTechniques for Preventing Unplanned Downtime
The Cost of Bad AlarmManagementAlarm management withinprocess automation systemshas never been moreimportant. A recent studyfound that more than $20billion is lost annually due tounplanned downtime.Approximately 40 percent ofthese losses are linked topreventable human error.Important factors causinghuman errors include theproliferation of alarms and theoperator's ability to respondquickly, efficiently, andcorrectly during upsetconditions.
The results of poor alarmmanagement can influenceyour company’s bottom line inmultiple ways such as loss ofproduct, damage toequipment, loss of life, anddamage to companyreputation. Just in the pastyear, one widely publicizedincident cost 15 lives, andcaused 100 injuries andsignificant property damage.The total OSHA fines exceeded$20 million for this incident.The fine for the alarm systemalone was more than $2million. In addition, companymanagement is under scrutinyfor criminal negligence. TheU.S. federal investigative reportfound that "managersauthorized the start-up of a keyunit despite knowing that keyalarms weren't working."4
processW
hit
e Pa
per
An Alarming Trend in DCS Alarming More is not necessarily better when it comes to alarms inautomated process control systems. Not long ago, whencontrols were hardwired to annunciator panels, engineerswere selective about the number of alarms in a system. Ina hardwired system, a single alarm could cost $1,000 toimplement, so there were typically no more than 30 to 50alarms per unit. Today, alarms are often viewed as “free.”A single I/O point may be configured to activate multiplealarms. These modern Distributed Control Systems (DCS)may include more configured alarms in the system thanthere are measured process values.
As a result, managing these alarms is becoming anincreasingly important role of the process plant operatorwho must interact seamlessly with the DCS. The job of thecontrol system is to maintain the process within the targetrange of performance. The job of the operator is tointervene to keep the process within its normalperformance range. When processes go outside thenormal range, upset conditions prevail. The ability torespond quickly, efficiently, and correctly during upsetconditions can mean the difference between high profits,unplanned downtime, or catastrophic failure.
To help ensure operators are prepared to handle anyalarm situation, companies worldwide are takingadvantage of new alarm management capabilities inmodern control systems. By utilizing recognized industrybest practices, systems like Siemens PCS 7 DCS helpsmanufacturers minimize unplanned downtime, increaseproduction, and more confidently protect workers.
Industry Trends Elevate Alarm ManagementThere are several significant trends in manufacturingfacilities putting pressure on plant operators and elevatingthe alarm management issue.
Operators are being asked to do more than ever before Significant variations exist in the operators' educationlevel, skill sets, and abilitiesInadequate time is allowed for operator training,particularly in responding to alarm/upset conditionsA "brain drain" is occuring whereby many of the plant'smost experienced operators, technicians, maintenancepeople, and control engineers are transitioning out ofthe workforce
Industry leaders have acknowledged the growingimportance of alarm management. In 2003, the UserAssociation of Process Control Technology in Chemicaland Pharmaceutical Industries (NAMUR) issuedrecommendation NA 102 Alarm Management. Mostrecently the ISA working committee SP 18 has issued adraft version of S18.02, a new standard that will serve asthe recommended practice for US industry for design,deployment, and management of alarm systems.
Alarm Management Best PracticesTo understand how to improve alarm management, it isimportant to first review industry best practices. There arenumerous excellent references that document alarmmanagement best practices. One of the most important isthe EEMUA Publication 191, ALARM SYSTEMS – A Guideto Design, Management and Procurement. It providespractical recommendations on alarm management basedon experiences from many end users and human factorstudies. According to the EEMUA, alarms and alarmsystems should be configured with the following basicprinciples in mind.
The single most important characteristic of a good alarmsystem design is that each alarm requires an operatorresponse. If an alarm condition doesn't really require theoperator to take action, then the condition should not bealarmed.
A good alarm subsystem contains many capabilities thathelp end users implement alarm management bestpractices and follow the recommendations of the EEMUA.Typically this includes the capability to:
Focus operator attention on the most important alarmsProvide clear and understandable alarm messagesProvide information on the recommended correctiveaction and allow comments to be recorded regardingthe actions taken or the future actions required Suppress alarms from a field device or from a processarea when they are not meaningfulAnalyze alarm system performance metrics to identifynuisance alarms or scenarios requiring additionaltraining
Table 1Characteristics of Good Alarms/Alarm System [Ref 3]
Draws attention to the most important issues
Focusing
Indicates the action that should be taken (corrective response)
Advisory
Identifies the problem that has occurredDiagnostic
Having a message which is clear and understandable
Understandable
Indicating the importance that the operator deal with the problem
Prioritized
Not long before a response is needed or too late to do anything
Timely
Not duplicating another alarm Unique
Not spurious or of low operational vaueRelevant
ReasoningCharacteristic
Draws attention to the most important issues
Focusing
Indicates the action that should be taken (corrective response)
Advisory
Identifies the problem that has occurredDiagnostic
Having a message which is clear and understandable
Understandable
Indicating the importance that the operator deal with the problem
Prioritized
Not long before a response is needed or too late to do anything
Timely
Not duplicating another alarm Unique
Not spurious or of low operational vaueRelevant
ReasoningCharacteristic
Preventing Unplanned Downtime Through Alarm Management
2
EEMUA Oil & Gas PetroChem Power Other
Average Alarms per Day 144 1200 1500 2000 900
Average Standing Alarms 9 50 100 65 35
Peak Alarms per 10 Minutes 10 220 180 350 180
Average Alarms/10 Minute Interval 1 6 9 8 5
Distribution % (Low/Med/High) 80/15/5 25/40/35 25/40/35 25/40/35 25/40/35
Indicators of Poor Alarm ManagementThere are several tell-tale signs or "symptoms" of pooralarm management. Many manufacturing operationsexhibit one or more of these symptoms. Some of the mostcommon include:
Nuisance alarms – alarm conditions that come and goon a regular basis or intermittentlyAlarm floods – when too many alarms are presentedto the operator during abnormal situationsCascading alarms – when specific alarms alwaysoccur togetherAlarm messages are not useful – when alarmmessages do not provide meaningful information tothe operator concerning the cause of the problem orthe corrective actionExcessive number of high priority alarms – toomany high priority alarms are present in the systemcausing the operator to treat some as lower priorityStanding alarms – too many alarms are presentcontinuously in the system even during steady-stateconditions (and operators ignore them)
Five Ways to Improve Alarm Management and PreventUnplanned DowntimeWhile there are many features and benefits delivered bythe alarm management capabilities of the DCS, there arefive categories aligned with the EEMUA's best practicesthat deliver immediate benefits to any processmanufacturer who wants to avoid unplanned downtime.These features allow plant personnel to:
I. Focus on the most important alarmsII. Suppress meaningless alarms as neededIII. Quickly comprehend the situation based on clear,
consistent, concise, and informative messagesIV. Obtain useful information regarding probable cause
and recommended corrective action V. Evaluate system / operator performance
I. Focus on the Most Important AlarmsAs shown in Table 2, operators are confronted with manyalarms per day (by a factor of 10X) than they canreasonably process (based on the recommendations ofthe EEMUA).
With the potential for an overabundance of alarms, one ofthe most important features of a DCS is its ability to focusthe operator's attention on the most critical alarms. Whena new alarm occurs, multiple visual and audible methodsare typically provided to attract the operator's attentionand indicate the importance of a particular alarmcondition.
Each individual alarm message within the system can beassigned an alarm priority. EEMUA studies have shownthat to maximize operator effectiveness, no more thanthree different sets of alarm priorities should beconfigured in an entire system. This alarm priorityattribute can be used to ensure that the highest priorityunacknowledged alarm is displayed at all times. It alsoprovides one means of filtering alarm display lists so thatoperators can home in on specific critical alarms.
Table 2Alarming Benchmark Performance by Industry
Source: MatrikonRecommended Actual
3
Preconfigured displays listing incoming alarms aretypically one of the operator’s main ways for monitoringalarms. From this display (Figure 1), an operator can filterand/or sort based on any column to fine-tune the displayso that it shows only the most important alarms.
The default HMI symbols (block icons) should be designedto effectively focus the operator's attention in the event ofan alarm. For example, when a new alarm occurs, analarm status grid within the HMI symbol may change color(typically yellow or red) and display text indicating that analarm is present. This dual indication of alarm statuschange (color and text) follows HMI design best practices
to ensure that even color-blind operators, who make up asignificant minority of the male population, can detect thealarm state change within a graphic display.
A built-in alarm horn capability makes it easy for the userto trigger an action when a new alarm condition occurs,such as the playing of a unique .wav file via the PC'ssound card. For example, the horn could be configured tocreate a unique sound based on the occurence of analarm with any combination of specific alarm attributes,including message class, priority, tag name, source, orprocess area (Figure 2).
Figure 1 Filtering and Sorting of Alarms in the Incoming Alarm List
Figure 2 Alarm Horn Configuration Tool
4
II. Alarm SuppressionAnother way to reduce the number of meaningless alarmsis by using an alarm "locking" or suppression capability.Ideally, all alarms from an individual device, or from anentire process area, should be able to be suppressed atthe touch of a button (the lock button). As a result, alarmsfrom deactivated equipment, or from process areas thatare non-operational, can be removed from the view of theoperator. The lock button is typically protected bystandard access security and requires the appropriateprivileges for it to be enabled.
When a device has been locked or taken out-of-service, itscorresponding symbol (block icon) on the process graphicshould reflect this. For example, an “X” is placed in thealarm status grids on Figure 3.
In addition, an "X" is placed in the plant overview buttonbar to indicate that at least one point in a particular areahas been taken out-of-service (locked) (Figure 4).
Alarm "locking" is a powerful capability for eliminatingmeaningless alarms that can impede operatoreffectiveness. However, if left unmonitored, importantalarms could remain out-of-service after they becomemeaningful. To prevent this situation, a list of all devicesthat are currently suppressed (called the "Lock" list) shouldbe provided. Operations personnel can get an up-to-datelist of all the locked alarms in their system, no matter howold, to determine whether it is acceptable to startup orbegin using the equipment (Figure 5).
Figure 3 Standard HMI Symbol with Alarms Suppressed
Figure 4 Operator Interface Showing Alarms Suppressed in a Process Area
5
III. Provide Clear and Understandable Alarm MessagesControl systems like PCS 7 help users conform to alarmingbest practices by making it easy to define clear andunderstandable information in alarm messages. As shownin Figure 6, a typical alarm message display shouldcontain clear, easy-to-understand information, such as tagname, tag description, the process area in which thealarm occurred, message type, alarm state, alarm priority,and corrective action information if it exists.
In particular, the "Event" field is designed to be easilyconfigured to provide information in a format that makessense to a plant operator. In the example shown below,the "Event" field delivers information based on theequipment description or physical device name (RX_ATank). The field provides a clear description of theproblem (Level LowLow Alarm), and also includes asnapshot of the value of the process variable (5.0) whenthe alarm was triggered (dynamic messaging) (Figure 6).
If the system could talk, it would say "A Reactor A TankLowLow Level alarm has occurred in the Reactor A processarea. The tag name for this device is LI_100. The alarm iscurrently Unacknowledged and has a priority of 2 (High).The process variable hit a value of 5.0 feet when thealarm was triggered. No corrective action information isavailable for this alarm."
IV. Recommended Corrective ActionOne of the most significant trends effectingmanufacturing facilities is the attrition of experienced andknowledgeable plant personnel (including operators). TheDCS was first introduced in 1975 and widely adopted inmanufacturing facilities throughout the late 1970s andearly 1980s. In many plants, the best and mostexperienced operators have run the plant for the past 25to 35 years. Consequently, when these people leave theworkforce, significant process knowledge may be lost.
One way to address this operator attrition is by allowingoperational best practices to be captured and documenteddirectly within the system. For each alarm, it is desirablethat a separate message displaying corrective actioninformation is made available to the operator via the "InfoText" field. Many end users take their Standard OperatingProcedures (SOPs), and copy/paste this informationdirectly into DCS via an "Info Text" field. Other facilitiesconfigure these fields based on the recommendedprocedures of their most valued operators, ensuring thateverone in the plant can benefit from his/her experiences,and that this process knowledge is not lost (Figure 7).
In addition, operators should be able to add comments toindividual alarms after they have been acknowledged. Asa result, they can document actions taken to correct asituation, the root cause of the problem, or to flag it forfollow-up attention (by maintenance personnel). This"Comment" field can then be sorted so maintenance cancreate an on-demand "report" of all the annotated alarms.
Figure 5 List of Suppressed Alarms (Locked)
Button for list of suppressed alarms
6
Tagname Alarm Message includes tag
description and Alarm Meaning
Process Area
Message Type (Alarm, Warning or
System Event)
Alarm State
Alarm Priority
Additional Info Exists (Corrective
Action)
Figure 6 Alarm Message Display
Figure 7 Capturing Operational Best Practices Within the System
Tagname
Alarm Messageincludes tag
description and AlarmMeaning
Process Area
Alarm State
Alarm Priority
Additional Info Exists
(Corrective Action)
Message Type(Alarm, Warningor System Event)
7
www.sea.siemens.com
©2006 Siemens Energy & Automation, Inc. All Rights Reserved.Siemens is a registered trademark of Siemens AG. Product names mentioned may be trademarks or registeredtrademarks of their respective companies. Specifications are subject to change without notice.PAWP-00008-0606 New 0606 Printed in USA
Siemens Energy & Automation, Inc.3333 Old Milton ParkwayAlpharetta, GA 30005
V. Analyzing Alarm System/Operator PerformanceEffective alarm management is a dynamic process thatrequires continuous attention. Look for a DCS thatincludes out-of-the-box tools to provide alarm KeyPerformance Indicators (KPI) that engineers andoperations personnel can use to continuously evaluateand improve the performance of the system – as well asresponsible personnel.
The Alarm History (Journal) list should contain usefulinformation such as including how long a particular pointwas in alarm before it was acknowledged. In the screenbelow, the highlighted alarm was active for more thantwo hours and it was never acknowledged. This mightindicate an area where additional operator training isnecessary, or where a "nuisance" alarm is being ignoredby the operations personnel (Figure 8).
An alarm "hit list" provides a frequency analysis of thealarming activity in the system. This display could beused, for instance, to determine which devices went inand out of alarm most frequently over the past 48 hours.The presence of alarm state changes at an unusualfrequency might indicate an intermittent hardwareproblem in the field, or a "nuisance" alarm which can becorrected by adjusting its limit and/or dead band (Figure 9).
ConclusionAlarm management is one of the most undervalued andunderutilized facets of process automation today. Asalarm systems become less effective, they diminish theeffectiveness of the entire automation system thuseffecting your company’s bottom line. By takingadvantage of the alarm management functions of systemslike PCS 7, manufacturers place an investment in riskmanagement. As a result, plants may operate closer to thelimits than ever before while reducing downtime,increasing productivity, and protecting workers.
References1. "Collaborative Process Automation Drives Return on
Assets", Dave Woll, ARC, June 20022. "Alarm Management Strategies", Larry O'Brien, ARC,
November 20043. EEMUA Alarm systems, a guide to design,
management and procurement, Publication 1914. "Investigators Fault BP for More Lapses in Refinery
Safety", Wall Street Journal, August 18, 20055. "Expertise Lost: How Do You Keep the Plant Running
if Nobody Remembers How", CONTROL Magazine,April 2005
Contact: [email protected]
Figure 8 Alarm History Display Provides Useful KPIs
Figure 9 Alarm Frequency Analysis Helps Identify “Nuisance” Alarms
Alarm was active for 2 hours and 15 minutes and was never acknowledged.
Button for Hitlist on Alarm System
Frequency
Average Time Alarmwas Active
Average Time BeforeAlarm was Ack’ed
by Todd Stauffer PCS 7 Marketing Manager Siemens Energy & Automation, Inc.June, 2006