part 21 rept re failure of rosemount transmitters.all ... · in reference (3), rosemount provided a...

- - _ . - -

t

.

N UTIIJTIES o n.,.i Orric . . seiden Street, Berlin, Connecticut.

I wistia sucuens ucmc w" P.O. BOX 270" " ' " ' " " " " " "HARTFORD, CONNECTICUT 06141-0270

J ",j",Z SUjy"cou,", (203) 665-5000k L

,

April 13, 1989

Docket Nos. 50-21350-24550-33650-423B13178

U.S. Nuclear Regulatory CommissionAttention: Document Control DeskWashington, DC 20555

References: (1) E. J. Mroczka (NU) letter to W. T. Russell (NRC), Reportof Substantial Safety Hazard, B12863, March 25, 1988.

(2) Rosemount letter to Northeast Utilities, Rosemount NuclearQualified Transmitters, December 9, 1988.

(3) Rosemount letter to Northeast Utilities, NotificationUnder 10CFR21, February 7, 1989.

Gentlemen:

Haddam Neck PlantMillstone Nuclear Power Station, Unit Nos.1, 2, and 3

Rosemount Transmitters

This information letter is being submitted to provide a summary of our activ-ities involving Rosemount transmitters. Since Rosemount transmitter failureswere first identified at Millstone Unit No. 3 in 1987, Northeast NuclearEnergy Company (NNECO) personnel have been actively involved in evaluating andaddressing this concern. In addition, we have had a number of discussionswith the vendor, NRC inspectors, and various industry representatives, andhave met NRC reporting requirements on this issue. Since our knowledge hassubstantially increased on this issue since the docketing of Reference (1),NNEC0 believes it is appropriate to ensure the NRC Staff is more fullyinformed, by providing this information letter.

Backaround

During the first cycle of Millstone Unit No. 3 operation, five Rosemountcapacitive-type differential pressure transmitters failed in the reactorcoolant system. Twelve such transmitters are used to monitor reactor coolantflow in the primary loops and provide a reactor trip signal to the reactorprotection system. There are three transmitters in each loop. If any two ofthe three transmitters in a loop sense low flow, a trip signal is initiated.The failures occurred individually over a period between March and November1987, such that two transmitters never failed simultaneously in any one loop.

8904240128 890413PDR ADOCK 05000213

7 IS PNV

_- _ _ __ ___-_

_ _ - _ - _

:.

4

U.S. Nuclear Regulatory Commission IB13178/April 13, 1989

Each failed transmitter was taken out of service and the affected channel wasplaced in the trip condition. The transmitters that failed were all Rosemountmodel 1153 HD5PCs.

Attempts to calibrate the failed transmitters were unsuccessful and thetransmitters were replaced. The five failed transmitters were returned toRosemount and destructive testing determined a loss of oil to be the cause ofthe failed condition.

The individual failures of these transmitters were evaluated for deportabilityin accordance with station procedures. The plant remained in compliance withtechnical specifications and no reportable condition existed under 10CFR50.72and 50.73. However, NNECO concluded that the number ' of failures was ofconcern. Accordingly, an evaluation was initiated in late November 1987 todetermine if a Substantial Safety Hazard existed. This included technicalreviews performed by engineering disciplines and operations personnel,meetings with the vendor, deliberations by the Millstone Unit No. 3 NuclearReview Board, and a final determination by a senior .arporate officer. Aconclusion was reached that the five failures represented a substantial SafetyHazard, and the NRC Staff was accordingly notified under-10CFR21 on March 25,1988 (Reference 1). That notification also informed the NRC Staff that theRosemount problem existed at Millstone Unit No. 3 and not at NU's otherplants.

The Haddam Neck Plant does not have Rosemount transmitters. Millstone UnitNo. I has ten safety-related Rosemount transmitters, model 1152, that are usedonly for ATWS mitigation. Millstone Unit No. 2 has Rosemount transmitters,three of which are used in safety-related applications, and two of these aresimilar in model number (1153HD5PA) to the Millstone Unit No. 3 failures.Reference 1 also reported that Millstone Unit Nos. I and 2 have not exper-ienced this problem.

In addition to having replaced all failed transmitters by the end of the firstcycle, NNEC0's corrective actions included the preparation of an in-servicetest procedure, and a monthly test of all twelve (12) Rosemount transmittersin the reactor coolant system throughout Cycle 2 operation. Millstone UnitNo. 3 is scheduled to shut down for the second refueling on May 20, 1989.Since the first cycle, no additional failures of Rosemount transmitters havebeen observed.

Continuina Review

Recognizing that the monthly test may not in itself provide total operabilityinformation, in addition to the monthly surveillance being performed at

| Millstone Unit No. 3 over the past year, NNECO has continued to investigate| the Rosemount transmitter issue. During a meeting on February 6,1989, NNECO

concluded that all new information should be evaluated to ensure operabilityI and deportability requirements were being fully met. Accordingly, new eval-| uations were initiated on February 8,1989 for Millstone Unit Nos.1, 2, and

_ __ - ____

- - _ - _ _ _ _ _ -

t.

.

U.S. Nuclear Regulatory CommissionB13178/April 13, 1989

-3. The evaluations focused on new information provided by Rosemount and oninformation that was learned as a result of increased analysis of data fromMillstone Unit No. 3.

In Reference (2), Rosemount confirmed to NNECO that there was a problemaffecting Rosemount transmitter models 1153 and 1154. The letter stated thata small number of these transmitters may respond sluggishly to input changesor may drift outside normal specifications. The source of the problem wasidentified as a loss of oil within the sensor cell. Several oil loss pathswere possible: the glass-to-metal interface, fill tube and damaged isolatordiaphragm. The letter stated that the failure was a random and low probabil-ity event. Finally, Rosemount identified that increased acceptance criteriawere added to the cell manufacturing and testing process, to assure a reliableproduct.

In Reference (3), Rosemount provided a 10CFR21 Notification to NNEC0 concern- j

ing the problem affecting Rosemount transmitter models 1153 and 1154. In this iletter, Rosemount stated that the loss of oil from the sensing cell may causea reduction in transmitter performance (such as drift, lack of response and anincrease in response time). The letter also stated that the problem may beunidirectional. In addition, no firm limits could be placed upon the perfor-mance reduction of a failed transmitter.

The scope of the problem as indicated in Reference (3) is believed to belimited. Information used in Rosemount's assessment suggests that trans-mitters in service longer than 36 months may not exhibit the loss of oilfailure, i.e., failures exhibit an infant mortality nature. This is becausethe failure is related to a specific manufacturing process and is not directlyservice related. No known loss of oil failure has occurred after 30 months ofservice. The Millstone Unit No. 3 transmitters now have approximately 36months of service.

The information in Reference (3) did not rule out generic applicability. Theletter stated that ... prior to detectable failure, the transmitter may"

continue to provide a signal but not respond over its full range and/or timeresponse may be significantly degraded. This may be a safety concern at yourplant." Rosemount also stated that action has been taken to correct thesource of the problem by " improving the manufacturing process and intensifyingtest criteria." Therefore, ... potential for failures of this nature in"

transmitters currently being produced has essentially been eliminated."

Reference (3) also identified that all failures reported to Rosemount haveoccurred in certain groups of transmitters. Transmitters from these groupsthat were shipped to the Millstone site were identified in an attachment tothe Rosemount letter. A total of 16 transmitters were listed as having comefrom batches with confirmed failures. Our review of this list confirmed that13 of these were used in Millstone Unit No. 3 and three of these were used inMillstone Unit No. 2. We have identified the locations of these transmitters

____________ -

- . _ _ _ ,

.

.

.


in our plants, and determined the effect that reduced transmitter performancemay have in each application as described below.

Millstone Unit No. 3 - 13 Transmitters from Suspect Batches

The identification, function, failure effects and calibration results for eachof the 13 transmitters from manufacturing batches with confirmed failures isdescribed below:

Additional actions that provide increased operability confidence are describedlater in this letter.

Model No. Serial No. Plant Taa No. Function

ll53DB5 408073 3CHS*LT102 Boric Acid Tank Lvl. TK5A11530B5 408074 3CHS*LT104 Boric Acid Tank Lvl. TK5A1153DB5 408076 3CHS*LT106 Boric Acid Tank Lvl. TK5B

These transmitters are used to measure boric acid tank level. Two of thetransmitters are on Tank 5A and one is on Tank 58. These transmitters arelisted in the Unit 3 Technical Specification Section 3.3.3.5 as being part ofthe required Remote Shutdown Instrumentation. There are two sensors on eachtank and the minimum number of sensors required is one. Therefore a failureof any one of these sensors would not jeopardize remote shutdown capability.These transmitters were calibration checked in February 1989 and found to bein noraal working order.

Model No. Serial No. Plant Tao No. Function

ll53DB5 408078 3SWP-FT59A CTMT Recirc Cooler Aoutlet Flow

1153DB5 408079 3WSP-FT59B CTMT Recirc Cooler Boutlet Flow

These transmitters are used to measure service water outlet flow through thecontainment recirculation coolers. They provide only a monitoring function. ,

Failure of the transmitters would not prevent any of the recirculation coolers |from performing their safety function. These transmitters were calibration '

checked in February 1989 and found to be in normal working order.


1153HD5 408188 3RCS*FT424 RCS Flow Loop 21153HD5 408190 3RCS*F1426 RCS Flow Loop 2ll53HD5 408103 3RCS*FT436 RCS Flow Loop 3

These transmitters are used to measure reactor coolant loop flow. There are a1

total of three transmitters installed in each loop. Two out of three logic isused to provide a low flow reactor trip at full power. Two of the above

_ _ _ _ _ _ _ _

_. . _ _ _ _ _ _ . - .

Ii.

o

.

U.S. Nuclear Regulatory Commission i

813178/April 13, 1989

transmitters are installed in loop 2 and one is installed in loop 3. If thetwo transmitters installed in loop 2 fail simultaneously then the plant wouldnot be able to provide a loop 2 low flow reactor trip. In February 1989, allRCS flow transmitters were monitored for performance using high speed dataacquisition equipment and/or using the Offsite facility Information System(0FIS). All transmitters were found to be in normal working order.


1153HD5 408198 3RCS*LT461 Pressurizer Level

This transmitter is used to measure pressurizer level. There are threetransmitters measuring pressurizer level and providing a 2 out of 3 high-levelreactor trip. Should the transmitter fail there would still be two othertransmitters that could provide the reactor trip function. This transmitterwas calibration checked in February 1989 and found to be in normal workingorder.


1153GD8 411114 3RCS*PT403A RCS Pressure Wide Range

This transmitter is used to measure wide range RCS pressure. It is used toprovide indication, alarm, input RHR valve interlocks, and input to theinadequate core cooling monitor. A redundant channel utilizing Foxborotransmitters is provided. The Rosemount transmitter channel was checkedaqainst the Foxboro transmitter channel using 0FIS data and no anomalies werecoserved. This transmitter was calibration checked in February 1989 and foundto be in normal working order.


1153HD5 408197 Not Installed None1153005 410157 Not Installed Nonell54DP4 414993 Not Installed None

The above transmitters are not installed and therefore pose no safety con-cerns.

Millstone Unit No. 2 - Three Transmitters from Suspect Batches

Reference (3) identified three transmitters from suspect batches which wereused at Millstone Unit No. 2. The identification, function and failureeffects of each of these are described below:


ll53HD5PA 411943 RCS-LT-110Y Pressurizer Level

L _ _____ ____. _ _ ___ ________.___ _ _.___ _ _

___ _ _- -__ - - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ _ _ _ _ - _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ . ___-_ - . .

:.

O


This transmitter is one of two transmitters used to provide pressurizer levelindication signals. The level signals are used for control and indication,and they provide no reactor protection or engineered safeguards actuationfunctions. The selection of either of these two transmitters is made man-ually. Technical Specification sections 3.3.3.5 (Remote Shutdown Instrumen-tation) and 3.3.3.8- (Accident Monitoring) both require a minimum of onechannel to be operable in modes 1, 2, and 3. Loss of both transmitters wouldrequire plant shutdown if not restored within seven days. The pressurizerlevel signals are Type A variables per Regulatory Guide 1.97 and they are alsoreferenced in the plant Emergency Operating Procedures as one means of eval-uating RCS inventory conditions during POST-LOCA operation. This transmitterwas calibrated and response checked in February 1989 and found to be in normalworking order.


ll53HD5PA 411942 Not installed Nonell53HD5PA 411944 Not installed None

The above spare transmitters are not installed and therefore pose no safetyconcerns.

Millstone Unit No. 1 - Transmitters from Suspect Batches

Millstone Unit No. I has not received transmitters from the batches inquestion.

Haddam Neck Plant - Transmitters from Susoect Batches

The Haddam Neck Plant has not received any transmitters from batches inquestion, and Rosemount transmitters are not used in any applications at theplant.

Summary of Transmitters from Susoect Batches

Rosemount informed NNEC0 under 10CFR Part 21 that a potential failure couldoccur in Rosemount model 1153 and 1154 transmitters. The loss of sensor fluidfailure mode may cause the transmitters to exhibit reduced performance priorto a detectable failure. Rosemount indicated that the reported failures all !

occurred in certain manufactured batches. Millstone Unit No. 3 received 13transmitters from suspect batches and Millstone Unit No. 2 received 3 trans-mitters. The functions and failure effects of these transmitters have beenidentified. In addition, all of the installed transmitters from these suspectbatches were performance checked in February 1989 and found to be in normalworking order.

I

_ _ _ _ _ _ - _ _ _ _ _ - _ _ i

- - _ _ _- . . _ _ _ _ _. - _ _ _

'O

o

.


Inspection of Rosemount. Inc.

NNEC0 has responsibility under 10CFR50 Appendix B to assure that its suppliersof safety-related components are complying with applicable requirements.Because of our concerns on the Rosemount transmitters, we sent an experiencedmechanical engineer to the Rosemount facilities in February 1989 to reviewtheir transmitter sensor manufacturing and inspection process, and to verifyimprovements that have been made.

The inspection focused on the following specific concerns:

1) Rosemount's determination of the root cause of the failures.

2) The methodology used by Rosemount to restrict the problem to a subset oftransmitters.

Rosemount has concluded, and we concur, that the root cause of failure is thelack of bond between the glass and metal cup within the transducer cell. Thecritical manufacturing operation is the furnace glass seal and its preparatorysteps. In August of 1986, Rosemount undertook a process optimization programwhich, when completed in March of 1987, cut their reject rate from approx-imately 20 percent to 1-to-2 percent. This, in conjunction with increasedtesting and inspection criteria, is the basis for Rosemount's determinationthat the problem will not exist with units manufactured after the firstquarter of 1987.

The failures to date have been numerically small, the first documented failureoccurring with a 1979 shipping date. The latest shipping date for a failedtransmitter was in the first quarter of 1987. There were approximately 300lots totaling just over 14,000 units produced between those dates. As ofFebruary 1989, Rosemount has had 84 reported failed units classified ~as " lowoil" failures. Upon testing, 6 of those were found to be due to other causes.Each of the 78 low oil failures was traced back to its manufacturing lot and20 lots were classed as suspect. There are 1004 units in those 20 lots ofwhich 16 units were supplied to NNECO. Factors mitigating the conservativenature of this data are:

1. The failure is of the infant mortality type; the metal-to-glass bonddelaminates upon cooldown from the furnace temperature to ambient.

2. The o/erall failure rate is low.

3. The failures are due to small process variations that are lot dependent.

Therefore, while both Rosemount and we believe that additional lots may beadded to the suspect list due to additional failure information, the totalnumber of failures will remain low (in the range of one-percent or less of allunits produced). In addition, units in service are less likely to be foundfailed due to low oil as service life increases. Of the known loss of oil

I

_____- _ a

h

.

.

.


.

|

failures, the shortest length of service time was a few days and the longest30 months. The most frequent lengths of service before failure were 12,18,and 27 months. This data is derived from calibration or refuel outage dates

'

and is not truly actual service time. The Millstone Unit No. 3 transmittershave now operated for approximately 36 months.

Additional Actions Taken to Prostde Operability Confidence

In addition to those transmitters specifically listed by serial number andidentified by Rosemount as being of concern, we have reviewed all other

| Rosemount transmitters in safety-related applications. All safety-relatedRosemount transmitters were evaluated by specific function to assess what, ifany, further testing or analysis was appropriate. These fell into two primarycategories, transmitters that for various reasons did not require furthertesting and/or analysis and transmitters that did require further testingand/or analysis. Further testing and analysis consisted of calibrationchecks, review of 0FIS data and/or review of transient analysis data asappropriate to detect any performance anomalies. To date, none of these othertransmitters have exhibited symptoms associated with the loss of sensor fluidproblem. Rosemount has stated in References 2 and 3 that industry data hasshown that the transmitter failures are random with the highest probability ofoccurrence during the beginning of operation.

NNEC0 has not discovered any additional transmitters that have experiencedthis failure mode. We believe this supports the infant mortality conclusionsthat were independently reached as a result of our inspection of Rosemount'sfacilities.

Future monitoring of our Rosemount transmitters includes a program atMillstone Unit No. 3 which will (1) verify transmitter performance utilizing0FIS data and (2) instruct instrument technicians of symptoms to be alertedto, which would be indicative of degraded transmitters.

Conclusions on Failure Symotoms

Our e.aluations and analyses of Rosemount transmitters described in thisletter have lead us to conclusions that any of the following symptoms may bean indication of transmitter failure.

1. Slow drift in either direction on the order of 1/4% per month.

2. "One sided" noise from flow signals.

3. Slow response to a transient or inability to follow a transient.

4. Decrease in the RMS noise level.

5. Deviation from redundant channels.

_____---____ -_____-__ a

. - _ _ _ _ - _ - _ - _

.

.

.

U.S. Nuclear Regulatory Commission( B13178/; April 13, 1989

' Transmitter calibration when exercised over the calibrated range is consideredthe best indicator of a loss of oil at the present time. If any of thefollowing symptoms are observed during calibration, further evaluation isrecommended.

1. Inability to respond over the entire range.

2. Slow response to either an increasing or decreasing hydraulic testpressure. (e.g. Response at either the high or low end of the calibratedrange may be on the order of 2 minutes to 2 hours.)

3. Any drift of greater than 1% from original calibration.

We have shared the above conclusions with the industry via Nuclear Network,

| notification on February 14, 1989 and we have factored these conclusions into! our monitoring program.

Conclusion

We have had a high level of involvement in the Rosemount transmitter issue.Information which we have helped to identify and share with the vendor, theNRC Staff, and various industry representatives, is contributing to the under-standing and resolution of this issue. We nave continued to ensure thatoperability and reporting requirements have also been met.

Our confidence in the continued operability of the Rosemount transmitters isbased on a number of factors including:

o Specific evidence to believe Rosemount has identified and alsocorrected the deficiencies in the manufacturing process.

o Specific evidence to believe Rosemount has identified and improvedtheir inspection process to preclude defective units being shipped.

I

o Infant mortality evidence supported by data collected by Rosemountand by NNEC0. The mortality is such that the transmitters that aresubject to failure can be expected to have already failed,

o Identification and review of all safety-related Rosemount trans-mitters used in our plants, principally to determine their functionand potential failure effects,

o Specific operability verifications provided by transmitter responsetesting, calibration, review of recorded transmitter performance,use of high-speed data acquisition monitoring, and use of OffsiteFacility Information System monitoring,

o A monitoring program that will selectively verify transmitterperformance.

- --

__ _ - - - _ _ _ _ _ - _ - - - _ _ _ _

..ra

U.S. Nuclear Regulatory Commission: jB13178/April 13, 1989

This summary of our activities on the Rosemount transmitter issue is intended| to further contribute to the understanding and resolution of this concern.

Very truly yours,

CONNECTICUT YANKEE ATOMIC POWER COMPANYNORTHEAST NUCLEAR ENERGY COMPANY

| bdWE.J.froczka ./Senior Vice President

cc: W. T. Russell, Region I AdministratorA. B. Wang, NRC Project Manager, Haddam Neck PlantJ. T. Shedlosky, Senior Resident Inspector, Haddam Neck PlantM. L. Boyle, NRC Project Manager, Millstone Unit No.1G. S. Vissing, NRC Project Manager, Millstone Unit No. 2D. H. Jaffe, NRC Project Manager, Millstone Unit No. 3W. J. Raymond, Senior Resident. Inspector, Millstone Unit Nos. 1, 2, and 3

. _ _ _ - - _ _ - - _ - _ _ _ _ _ _ _ _ _ _ - - - - - - -

__j

part 21 rept re failure of rosemount transmitters.all ... · in reference (3), rosemount provided a...

Documents