112 “once is enough” in radiometric calibrations · calibration services, like manufactured...

1. Introduction

The successful development of an OpticalTechnology Division quality system for optical radia-tion measurement services has provided the opportuni-ty to reconsider the existing calibration procedures toimprove quality and reduce costs. This effort leveragesmajor advances in controlling manufacturing quality,led by W. E. Deming, in Japan, following World War II[1, 2]. These advances are equally applicable to thedelivery of services such as calibrations. Deming andothers developed approaches for controlling industrialprocesses to reduce product variability and, conse-quently, eliminate costly final product testing. For suc-cess, their approach requires manufacturers to under-stand and control all the steps in the production processso that variability in the final product is minimized tosome desired tolerance level [3].

The early proponents of quality manufacturingdeveloped statistical process control procedures andtests to help achieve low product failure rates. Theirwork has had a lasting impacting on efficiency andquality in product manufacturing. The control chart

concept of W. A. Shewhart used for tracking somemeasurable output of a process is particularly relevantfor the present discussion [4]. Shewhart stressed theunderstanding of the whole process and control of thevariability in the elements of the process, to minimizeinefficient final product testing. Deming made thispoint, as well, in his renowned 14 points of manage-ment, reproduced in Table 1. These points served as thefoundation for the quality system that Deming devel-oped. Points 3 and 5, in particular, are relevant to theimprovement of the calibration process as they targetthe elimination of piecewise inspection, emphasizing,instead, control and improvement of the entire process.

Calibration services, like manufactured products,can be delivered with improved quality through effec-tive control of the entire process. Such control elimi-nates the needless inefficiencies endemic in calibra-tions performed by government and industry scientists,whereby the entire measurement is typically repeatedthree or more times and averaged to obtain a reportedresult. Calibration scientists justify the repetitionby claiming that the statistical uncertainty in thefinal result is lower, confidence is improved, and it

Volume 112, Number 1, January-February 2007Journal of Research of the National Institute of Standards and Technology

39

[J. Res. Natl. Inst. Stand. Technol. 112, 39-51 (2007)]

“Once is Enough” in Radiometric Calibrations

Volume 112 Number 1 January-February 2007

Gerald T. Fraser, Charles E.Gibson, Howard W. Yoon, andAlbert C. Parr

Optical Technology Division,National Institute of Standardsand Technology,Gaithersburg, MD 20899-8440

[email protected]@[email protected]@nist.gov

The successful development of an OpticalTechnology Division quality system foroptical radiation measurement serviceshas provided the opportunity to reconsiderthe existing calibration procedures toimprove quality and reduce costs. Wehave instituted procedures in ourcalibration programs to eliminateuninformative repetitive measurementsby concentrating our efforts on controllingand understanding the measurementprocess. The first program in ourcalibration services to undergo theserevisions is described in this paper.

Key words: calibration; efficiency;process control; quality measurements.

Accepted: December 22, 2006

Available online: http://www.nist.gov/jres

has always be done this way. Such repeat measurementsare, in part, a legacy of metrology’s early history whenmeasurements where highly subjective, often involving avisual comparison. Details of this history for photo-metric and radiometric measurements will be given inAppendix 1. Here, we discuss the fallacy of maintainingsuch an approach and consider an alternative strategywhereby the calibration is performed only once. Wedenote this approach for convenience as “Once isEnough.”

To demonstrate the benefits of initiating a “Onceis Enough” measurement strategy we consider one of themost popular optical radiation calibration servicesoffered by the Optical Technology Division, the determi-nation of the spectral irradiance of an appropriate lampstandard. Historically, the spectral irradiance of thetest lamp has been measured three or more times over the

entire 250 nm to 2400 nm wavelength region. The varia-tion in the three measurements was used to assign areproducibility uncertainty to the calibration, of unclearphysical origin.

Despite extensive automation, repeat measurementsare time consuming and needlessly tie up expensiveinstrumentation. They also incur approximately threetimes the wear on reference standard lamps, customerlamps, and motion controllers such as monochromatortranslation stages and scanning grating drives.Ultimately, calibration service customers pay for theextra cost associated with these additional measure-ments, without receiving any significant benefit. Thesecustomers tend to follow NIST’s lead and, likewise,perform repeat measurements with little benefit to “their”customers.

40


1. Create constancy of purpose to improve product and service.

2. Adopt new philosophy for new economic age by management learning responsibilities and taking leadership forchange.

3. Cease dependence on inspection to achieve quality; eliminate the need for mass inspection by building quality intothe product.

4. End awarding business on price; instead minimise total cost and move towards single suppliers for items.

5. Improve constantly and forever the system of production and service to improve quality and productivityand to decrease costs.

6. Institute training on the job.

7. Institute leadership; supervision should be to help do a better job; overhaul supervision of management andproduction workers.

8. Drive out fear so that all may work effectively for the organisation.

9. Break down barriers between departments; research, design, sales and production must work together to foreseeproblems in production and use.

10. Eliminate slogans, exhortations and numerical targets for the workforce, such as 'zero defects' or new productivitylevels. Such exhortations are diversory as the bulk of the problems belong to the system and are beyond thepower of the workforce.

11. Eliminate quotas or work standards, and management by objectives or numerical goals; substitute leadership.

12. Remove barriers that rob people of their right to pride of workmanship; hourly workers, management andengineering; eliminate annual or merit ratings and management by objective.

13. Institute a vigorous education and self-improvement programme.

14. Put everyone in the company to work to accomplish the transformation.

Table 1. Deming’s 14 points of quality management

The repeating of a measurement is usually unneces-sary for a well-characterized process, with a clearlyunderstood and verified uncertainty budget for artifactsand a well characterized and understood stability andrepeatability. It is only useful if it adds value to the stateof knowledge of the measured quantity commensuratewith the additional effort required. W. J. Youden, in hisclassic book [5], “Experimentation and Measurement,”comments on the repeating of measurements:

“Many people seem to feel that there is somemagic in the repetition of measurements andthat if a measurement is repeated frequentlyenough that the final result will approach a“true” value. This is what scientists mean byaccuracy.

Suppose that you are in a science class andthat the next two students to come into theroom were a girl five feet ten inches tall and aboy five feet nine inches tall. Let each studentin the class of 30 students already in the classmeasure the new arrivals to the nearestfoot. The answer for both is six feet. Has therepeated measurements improved theaccuracy?”

In Youden’s example, only one measurement isrequired to answer the problem poised. Repeat meas-urements are only necessary if they improve our knowl-edge of the measure and commensurate with the addi-tional time and effort required. Over the last twodecades, the spectral irradiance measurement servicehas improved significantly in performance, aided byadvances in instrumentation, environmental control,and automation; by implementation of a quality system,and greater understanding of the intricacies of themeasurement. The consequence of the improved per-formance is that the “reproducibility” component of themeasurement is now small, eliminating the justificationfor repeat lamp measurements and providing the oppor-tunity to implement “Once is Enough.” Below we dis-cuss more fully the implementation of the “Once isEnough” strategy for spectral irradiance.

2. The Spectral Irradiance CalibrationProcess

The Spectral Irradiance Calibration Services 39030Cto 39046C disseminate lamps with spectral irradiancesmeasured as a function of wavelength. Typically, a1000 W quartz-halogen FEL lamp is calibrated, where

FEL is the lamp-type designation (not an acronym) ofthe American National Standards Institute (ANSI). Aphotograph of an FEL lamp is shown in Fig. 1 and atypical spectral irradiance curve, as measured 50 cmfrom a lamp, is shown in Fig. 2. The lamps are pur-chased from commercial sources and evaluated for suit-ability as standards [6]. The evaluation includes visualinspection of the bulb and filament structure, assess-ment of the spatial uniformity of the output, testing forthe presence of strong atomic emission lines, andexamination of the stability and relighting reproducibil-ity. The lamps are also seasoned by operating at theprescribed current for 10 h to 15 h prior to cleaning andstoring for future calibration.

The technical details of the calibration procedure andthe associated measurement uncertainties are discussedin NIST Special Publication SP250-20 [6]. The calibra-tion procedure and measurement uncertainties differ,somewhat, from that published initially some 20 yearsago, and will be described in detail in an updatedSpecial Publication [7].


41

Fig. 1. Photograph of a 1000 W quartz-halogen FEL lamp dissemi-nated as a standard of spectral irradiance by NIST measurementservices 39030C to 39046C.

Briefly, the spectral irradiance calibration comparesthe spectral output of a known lamp—the primary work-ing standard (PWS)—to that of an unknown lamp, thelamp under test (LUT). Measurements are performed inthe recently completed, second-generation Facility forAutomated Spectroradiometric Calibrations, denoted asFASCAL 2, shown schematically in Fig. 3. The facilityhas four lamp stages, one for the PWS and three for theLUTs to be calibrated. The FASCAL 2 instrument con-sists additionally of a spectroradiometer, integratingsphere, and filter radiometer all mounted on a moveablecarriage to align the entrance of the integrating spherewith either the PWS or with one of the three LUTs. Thespectral irradiance of the PWS is calibrated by compari-son with a blackbody, HTBB, whose radiance tempera-ture is tied, by a set of filter radiometers, to the absolutecryogenic radiometer through the Spectral ComparatorFacility (SCF) [8] or the Spectral Irradiance andRadiance responsivity Calibration with Uniform Sources(SIRCUS) Facility [9], as described by Yoon, Gibsonand Barnes [10].

Delivery of the calibration service involves a numberof steps, designated 1 through 9, as summarized inFig. 4. The process starts with the acceptance by NIST ofa request for calibration, and ends with the sending of thelamp and report to the customer, and addressing anyfollow-up issues, such as customer feedback, or specificlessons learned that can be applied towards improvingthe overall process. Such an analysis helps ensurecontinual improvement of the service and is a criticalcomponent of the associated quality system.

The core of the measurement process consists of steps3, 4, 5, and supporting step 9, used to maintain the stan-dards, with respect to the International System of Units(SI)[11]. These four steps consist of the physical measure-ments and the assignments of their measurement uncer-tainties. The development of a NIST calibration serviceentails the determination and validation of a completeuncertainty budget for the measurements. Validation isaided by international comparisons with other nationalmetrology institutes. The uncertainty budgets are avail-able to customers and other interested parties, as part ofthe public documentation for the service [6, 8].


42

Fig. 2. A typical spectral irradiance curve for an FEL lamp such as pictured in Fig. 1. The line is drawn to aid the reader. Thespectral irradiance (optical power per unit surface area per unit spectral bandwidth) is plotted as a function of wavelength as measured50 cm from the lamp.


43

Fig. 3. Schematic diagram of the second generation Facility for Automated SpectroradiometricCalibrations (FASCAL 2) with the scale realized from the high-temperature blackbody (HTBB)as collected by the integrating sphere receiver (ISR). For the ultra-violet wavelength regions, thephoto-multiplier tube (PMT) is used instead of the Si or the InGaAs detectors.

Fig. 4. The nine steps required for the Spectral Irradiance Measurement Services. Steps 1through 8 are routinely performed for each calibration. Step now is periodically performed toevaluate the irradiance scales maintained by the PWS lamps.

3. Spectral Irradiance CalibrationUncertainty Budget

The present uncertainty budget for the spectral irra-diance calibration service is summarized in Table 2. Itvaries slightly from that shown in Reference [10], butthe differences are small and represent refinements inthe determinations of specific components, rather thana substantial change in the overall measurementapproach. We will briefly discuss these components toprovide the context for the implementation of “Once isEnough.”

Lines 1 through 4 in Table 2 give the componentscontributing to the uncertainty in the radiance tempera-ture of the High Temperature Black Body (HTBB)during calibration of the PWS lamps. The radiance tem-perature of the HTBB is determined using a set of filterradiometers with responsivities tied to the cryogenicradiometer. The HTBB is operated near 2950 K, toapproximately match the spectral irradiance of the FELlamp, thus reducing errors from spectral stray light anddetector nonlinearity. The HTBB is well represented bya single radiance temperature, independent of wave-length, as determined by comparison against the small-aperture, NIST Variable-Temperature Blackbody(VTBB) [12]. The uncertainty associated with this con-clusion is given in line 2 of the table, as “HTBB spec-tral emissivity.” When using the HTBB to calibrate thePWS, additional uncertainty components arise fromspatial variation of the radiance temperature over the

blackbody exit aperture (line 3), and from drift in theradiance temperature of the HTBB during the measure-ments (line 4).

Line 5, “geometric factors in irradiance transfer,”gives the uncertainty in the geometric factors requiredto define an irradiance level from the HTBB at theentrance aperture to the integrating sphere receiver ofthe spectroradiometer. The irradiance is given by

(1)

where L(λ, Tradiance) is the Planck radiance, Tradiance is theradiance temperature of the HTBB, πrBB

2 is the area ofthe aperture in front of the blackbody, the modified dis-tance factor is D2 = d 2 + r 2 + rBB

2, where πr2 is the areaof the entrance aperture to the integrating spherereceiver, and d is the separation between the two aper-tures. The uncertainty in D is dominated by the uncer-tainty in the separation between the two apertures,since d >> r and d >> rBB.

Lines 6 through 9 are the uncertainty components forthe transfer of the blackbody-based irradiance scale toa PWS lamp by

(2)

where Eλ , PWS is the irradiance of the PWS, Sλ , PWS andSλ , HTBB are the spectroradiometer signals when measur-ing the PWS and HTBB, respectively, and fλ is the


44

Table 2. Uncertainty Budget for Spectral Irradiance Calibrations Using FASCAL 2

Relative Expanded Uncertainties (k = 2) [%]

Source of Uncertainty 250 350 450 555 655 900 1600 2000 2300 2400nm nm nm nm nm nm nm nm nm nm

1. HTBB temperature uncertainty (0.86 K at 2950 K (B) 0.57 0.41 0.32 0.26 0.22 0.16 0.09 0.08 0.07 0.072. HTBB spectral emissivity (B) 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.103. HTBB spatial uniformity (B) 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.104. HTBB temporal stability (0.1 K / h) (B) 0.07 0.05 0.04 0.03 0.03 0.02 0.01 0.01 0.01 0.015. Geometric factors in irradiance transfer (B) 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.106. Spectroradiometer responsivity stability (B) 0.60 0.60 0.30 0.30 0.30 0.30 0.30 0.30 0.30 1.007. Wavelength accuracy (0.1 nm) (B) 0.58 0.26 0.13 0.07 0.04 0.01 0.01 0.01 0.01 0.018. Lamp/spectroradiometer transfer (B) 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.10 0.109. Lamp current stability (B) 0.07 0.05 0.04 0.03 0.03 0.02 0.02 0.01 0.01 0.01

Total uncertainty of the primary working standards 1.03 0.80 0.50 0.45 0.43 0.40 0.37 0.37 0.37 1.0210. Lamp-to-lamp transfer (A) 0.50 0.30 0.20 0.20 0.20 0.20 0.20 0.30 0.30 0.4011. Long-term stability of primary working standards (B) 1.31 0.94 0.73 0.59 0.50 0.36 0.20 0.16 0.14 0.14Overall uncertainty of the test with respect to SI units 1.74 1.27 0.91 0.77 0.69 0.57 0.47 0.50 0.49 1.11

Note: The Type A or Type B evaluation of the uncertainty is indicated in parentheses.

( ) 2 2,HTBB radiance BB, / ,E L T r Dλ λ π=

,PWS,PWS ,HTBB

,HTBB

,S

E f ES

λλ λ λ

λ

=

detector linearity correction factor associated with thedifferent magnitudes of the signals measured whenviewing the PWS and HTBB. Line 6, spectroradio-meter responsivity stability, is the uncertainty compo-nent associated with the drift in the integrating-sphere/monochromator/detector system between measurement of the HTBB (Sλ , HTBB) and the PWS (Sλ , PWS). It isestimated using the standard deviation of the irradianceresponsivities determined for three PWS lamps. Line 7encompasses the effect of wavelength error in themonochromator drive on the spectral irradianceassigned to the PWS. The magnitude of this uncertain-ty component is strongly wavelength dependent, due tothe shape of the spectral irradiance distribution of anFEL lamp. Line 8 contains contributions from uncer-tainties in fλ , stray light and spectral scattering. Line 9is the uncertainty due to current instability in the lamppower supply. Note that line 8 and 9 terms are smalland do not significantly affect the results. The terms inlines 1 through 9 are all type B uncertainties, uncorre-lated with each other and with magnitudes assumedindependent of the LUT being calibrated [13]. Since thePWS are measured against the HTBB only on a period-ic basis, minimizing efforts in this aspect of the calibra-tion process will only have minimal effects upon thetotal time spent in the calibration endeavor. The totaluncertainty on the PWS irradiance calibration given inthe table is derived from the root-mean-square sum ofthe individual uncertainty components.

The irradiance of the LUT, Eλ , LUT, is determined bycomparison against the PWS using

[3]

where Sλ , LUT is the spectroradiometer signal whenviewing the LUT. Note that Sλ , PWS in Eq. (2) is not iden-tical to the Sλ , PWS in Eq. (3), since the measurementswere performed at different times. Lines 10 and 11 arethe uncertainty components associated with this irradi-ance transfer from PWS to LUT, which occur at the cal-ibration of each LUT. Line 10 is the subjectof this paper and is discussed in detail in the nextsection. Line 11 represents our experience on thestability of the irradiance scale of the primary workingstandard, as determined by repeat calibrations per-formed over several years, and intercomparison ofPWS lamps [14].

Further work on the uncertainty determination willbe carried out to formally take into account the correla-tions in the data. Correlations will come about becausethe individual wavelength determinations are correla-

ted through the commonality of the reliance upon acommon blackbody temperature. When these effectsare fully considered the uncertainties could possiblylessen.

4. Lamp-to-Lamp Transfer

Initially, lamps disseminated to customers from theFASCAL facility were calibrated in groups of 12, bycomparing their irradiances to that of four PWS lampsover the desired wavelength interval, typically 250 nmto 2400 nm [6]. Three LUTs and one PWS was mount-ed in the four FASCAL lamp stations for each of the 16calibration runs required. According to the previousprocedures, the LUTs and PWSs were permutated sothat each LUT was calibrated four times, each timemounted in a different station and compared against adifferent PWS, also mounted in a different station. Ateach wavelength of the spectroradiometer the fourlamps were measured and the wavelength was incre-mented. This process put about 6 h running time oneach LUT.

The approach was intended to eliminate the possibil-ity of disseminating a lamp calibrated against a PWSthat had drifted out of calibration. It also attempted toreduce the effects of variations in the PWSs and in thelamp stations on the reported LUT irradiance values. Atthe same time it provided a set of four measured valuesfor calculating a standard deviation for the repeatabili-ty of the calibration; a type A uncertainty componentprovided with the other uncertainty components in thecustomer calibration report.

The process was later modified in response to cus-tomer demands for faster and less expensive calibrations.Each LUT was now calibrated three times against threePWSs, instead of four times against four PWSs. Asbefore, the LUT station, PWS, and PWS station werevaried for each calibration of a LUT. Improvements inlamp temporal stability, through better lamp manufactur-ing and selection, reduced the magnitude of the PWSlamp drift during the measurements, providing addition-al opportunities to reduce the calibration time. Thisdrift, which approximately and linearly depended on thenumber of hours the lamp had been operated since cali-bration, was effectively eliminated by changing thecalibration procedure so the spectroradiometer scannedthrough an entire wavelength range once for each lamp,set by the responsivity range of the detector being used.Significant time was saved over the old process ofviewing each lamp prior to changing the wavelength.The modified process continued with the launch ofFASCAL 2.


45

,LUT,LUT ,PWS

,PWS

,S

E ES

λλ λ

λ

=

The requirement to have each LUT continue to be cal-ibrated three times was driven by the desire to derive astandard deviation from the repeat measurements, toquantify PWS and station variability. The repeatcalibrations also provided confidence in the calibrationstability of the PWSs and equivalency of the lampstations. Both of these rationales are insufficient for jus-tifying the extensive amount of time, effort, and instru-ment and lamp hours expended on repeat calibrations,particularly given the level of agreement between differ-ent calibrations of the same lamp. As shown in line 10 ofTable 2, the wavelength dependent lamp-to-lamp trans-fer uncertainty of 0.2 % to 0.5 % (k = 2) is small, relativeto the total calibration uncertainty, which ranges from0.49 % to 1.74 % (k = 2). This uncertainty component isbased on a large number of lamp calibrations performedon FASCAL 2. Since there is a large sample set based onsimilar lamps to determine the repeatability uncertainty,two additional measurements do not add, in any signifi-cant way, to the state of knowledge about the calibrationuncertainty. A typical example of lamp repeatability isshown in Fig. 5, for an LUT alternately positioned inthree different stations (2, 3, and 4) and calibratedagainst the same PWS positioned in station 1. The vari-ation in the lamp-to-lamp calibration is well within thelamp-to-lamp transfer uncertainty listed in Table 2. Also,the lamp-to-lamp repeatability is well inside the overalluncertainty of the measurement given in the last line ofTable 2, and shown in Fig. 5. The low uncertainty on thelamp-to-lamp transfer provides the primary technicaljustification for implementing “Once is Enough.”

5. Implementation of “Once is Enough”

Here, we provide a strategy for successful imple-mentation of “Once is Enough.” The four componentsdescribed in detail below will ensure confidence byboth calibration scientist and calibration servicecustomers in the spectral irradiance values of lampsmeasured only once.

Automation. Automation improves the repeatabilityof a measurement by eliminating subjective humanfactors associated with reading analog meters, aligningthe spectroradiometer optical axis to the lamp position,setting the monochromator wavelength, fixing lampcurrent levels, and reading spectroradiometer outputsignals. As mentioned previously and discussed inAppendix 1, the large number of subjective readingsand manipulations originally performed in radiometricmeasurements, and their associated potential for humanerror and ambiguity, played a major role in institution-alizing repeat measurements in the calibration services.

The present FASCAL 2 system is completely auto-mated, with the only human intervention being themounting and dismounting of the PWS and LUTs intheir stations. The NIST-issued FEL lamps are mount-ed on brass, bi-post bases to be fixed in kinematicmounts, so lamp alignments are reproduced within theuncertainties without any adjustments. The axial posi-tioning and distance setting of the lamp stations havebeen previously established and are not routinelyadjusted with a change in LUT or PWS. Since a non-imaging integrating sphere is used as the collectionsource, the alignment is relatively insensitive to the off-axis tilt of the integrating sphere aperture, with respectto the LUT. After the lamps are mounted, the currentsare turned up to their set points with computer control.After allowing time for the lamp outputs to stabilize,the spectroradiometer is automatically positioned toeach station and swept through the selected wavelengthrange. The spectroradiometer output signals are record-ed by the computer, converted to absolute spectral irra-diances, and inserted into a text file for insertion intothe calibration report template.

Uncertainty budget. A rigorous uncertainty budget iscritical for the implementation of “Once is Enough.”Such an uncertainty budget should be validated throughindependent measurements that are ideally based onboth similar and different measurement methodsperformed by independent scientists. Uncertaintybudgets in radiometry and photometry have conven-tionally included a “repeatability” component based ona simple statistical analysis of repeat measurements. As


46

Fig. 5. A plot showing the variation in the spectral irradianceassigned to an LUT positioned in stations 2, 3, and 4 of FASCAL 2.A single PWS mounted in station 1 was used for the measurements.Also shown in the figure are the total expanded (k = 2) uncertainties(solid lines), along with the expanded uncertainties for the lamp-to-lamp transfer (dashed lines) from line 10 in Table 2.

mentioned earlier in this article, this was the motivationfor repeating the measurement on a particular lamp, upto three or four times, in the belief that the uncertaintycould be decreased. This is somewhat misguided asTable 2, line 10, contains an estimate based on manymeasurements, of the sort, of uncertainty that pertain tothe expected variation in lamp performance; and a fewadditional measurements do not improve the statisticalquality at all.

The exact origin of the repeatability uncertaintycomponent is often not clear, but certainly includescontributions attributed to other components in theoverall uncertainty budget. The spectral irradiance“repeatability” component, denoted as lamp-to-lamptransfer uncertainty in Table 2, includes contributionsfrom lamp-current drift and spectroradiometer respon-sivity stability, which were already accounted for in theprevious lines in Table 2. The “long-term stability ofthe LUT” uncertainty component is not considered inTable 2, as this is the customers responsibility to esti-mate when they use the lamp. We note that such doublecounting unnecessarily increases the magnitude of thereported uncertainties on the calibrated irradiances.

To avoid double counting, a rigorous uncertaintyanalysis should be physics based, clearly tying eachuncertainty component to the measurement equation. Aphysics-based uncertainty analysis provides a frame-work for developing a strategy to reduce the overallmeasurement uncertainty, and in particular, the compo-nents limiting measurement reproducibility. Without atrue physics-based model, it is difficult to determinewith a finite measurement set, whether the repeat meas-urements should be distributed normally about the truemean.

As discussed above, for the FASCAL 2 measure-ments, the “repeatability” lamp-to-lamp transfer uncer-tainty component is small, relative to the total uncer-tainty. This repeatability component conservativelyreflects the repeatability of the measurements as shownin Fig. 5. Its small magnitude and likely over estimate—due to double counting—indicates that little is con-tributed to the knowledge of the spectral irradianceof the LUT by repeating the measurement three times;certainly not commensurating with the additional timeor effort expended.

Measurement process controls. Carefully selectedmeasurement process controls and associated controlcharts help ensure correct instrument operation, scalestability, and final calibration accuracy. They furtherhelp the calibration scientist immediately identifyproblems arising in the measurement, eliminating theexpenditure of time and resources to complete a cali-

bration that will fall outside of acceptable qualitylevels. Process controls and control charts also elimi-nate unnecessary final product testing, typically per-formed through a repeat calibration, to ensure measure-ment quality. As discussed in the Introduction, Demingand Shewhart promulgated the elimination of finalproduct testing and the integration of statistical processcontrol in the manufacturing process, to ensure finalproduct quality and performance.

For spectral irradiance measurements, two levels ofprocess controls are being implemented. These processcontrols provide either a gross or fine level of assessmentof instrument, PWS, and LUT performance throughoutthe calibration process. The process controls directlyassociated with the measurement of the spectral irradi-ances are being documented in a series of three controlchart sets to help provide early warning of potentialproblems in the calibration process.

The first level of control occurs in the lamp selectionprocess. A number of measurements are presentlyperformed on a lamp prior to its acceptance into thecalibration process. The spectral output of the lamp isexamined to determine the presence and strength ofatomic emission lines. These lines are noted in thecalibration report. The lamp output is examined at654.6 nm to ensure that it varies by less than 1 %, for a± 1° angular displacement from the defined lamp axis,and that it drifts by less than 0.5 % over a 24 h period.The former measurement is performed to aid lampusers interested in realizing a spectral radiance scale byillumination of a highly reflective diffuser.

Three check standard (CS) lamps, calibrated againstthe HTBB, are used to assess the stability of the PWSlamps. The CSs are used infrequently to ensure the con-stancy of their spectral irradiance values. The PWSs, ofwhich there are a total of six, are periodically used toassign a spectral irradiance scale to the CSs. The spec-tral irradiances determined for the CSs are required tostay within one FASCAL 2 standard uncertainty (i.e.,half the k = 2 value from the last line of Table 2) of theirinitially calibrated value. The measurements aretracked in a set of control charts for easy referral;denoted here as Control Chart Set 1.

A second set of control charts, Control Chart Set 2, isbeing developed to track the spectral irradiance respon-sivity of the spectroradiometer as a function of time,defined as the ratio of the output voltage of the spectro-radiometer to the input spectral irradiance from a PWS.Such control charts only provide evidence of grosschange in the performance of the radiometer since, frac-tional changes in the responsivity over time are largerthan the relative standard deviation of the reported


47

spectral irradiances. An example of such a plot for twowavelengths is shown in Fig. 6, for FASCAL. Thefigure shows a nearly linear drift in responsivity withtime, except for a sharp break, due to a change in theoperating voltage applied to the photomultiplier tube.The drift is significant and is caused by a continualchange in the photomultiplier tube sensitivity. Presentsilicon detectors used in FASCAL 2, between 350 nmand 1050 nm, have much less drift so that the change inspectral irradiance responsivity of the spectroradio-meter within this wavelength interval is dominated bythe change in the absolute reflectance of the integratingsphere coating; on the order of 0.5 %/day, for a newlycoated sphere in the ultraviolet wavelength region to< 0.1 %/month for the visible and infrared wavelengthregion. Control charts will encompass one wavelengtheach in the ultraviolet, visible, and infrared to track thespectroradiometer performance, when used with thephotomuliplier tube, silicon, and extended InGaAs

detectors. The overall stability of the radiometer systemis well understood and has a long recorded history thatserves as an overall check on the system as significantdeviations from the expected output of the system willbe immediately recognized.

A third set of control charts (Control Chart Set 3)will be developed to track the ratio of the spectralirradiance of the PWSs as a function of time, againat the same three wavelengths. In the absence of astable monitoring detector to assess the absolute outputof a PWS, “Once is Enough” calibrations will beperformed by comparing 2 LUTs against 2 PWSsmounted simultaneously in the four FASCAL 2 lampstations. Sequential calibrations will position thePWSs, so all four stations are tested in the two cali-bration runs. The relative irradiances between PWSsshould be predictable from the uncertainties in Table 2if the instrument and the standards are performingas expected.


48

Fig. 6. Plot showing the change in the spectral irradiance responsivity of FASCAL as a function of time at 330 nm and 555 nm. Thenearly linear growth in responsivity with time is due to change in the sensitivity of the PMT used at that time. For FASCAL 2, a morestable silicon detector is used for wavelengths between 350 nm to 1050 nm, so that the drift in the spectral irradiance responsivity isdominated by drift in the reflectivity of the integrating sphere coating.

The assigned spectral irradiances of the LUT and thePWS can also be checked by filter radiometers calibrat-ed for spectral irradiance responsivity. Since filterradiometers can have long-term stability of responsivi-ty exceeding that of lamps, these filter radiometers canact as separate check of the assigned spectral irradi-ances. Work is being done to have stable filter radio-meters calibrated for spectral irradiance responsivity tomeasure all FEL lamps put through the calibrationprocess.

To perform calibrations in FASCAL 2 of three LUTs,against one PWS, requires that a monitoring filterradiometer be developed to track the absolute irradi-ance of the PWS at a selected wavelength. A silicon-detector-based filter radiometer will be used to obtainthe necessary long-term stability. Ideally, the radio-meter will operate in the ultraviolet where the lampdrift is greatest, to provide the most sensitive trackingof PWS stability. The challenge will be to find a detec-tor-filter package whose long-term absolute responsiv-ity in the ultraviolet is stable to better than 1 %.

Additional controls to track instrument performanceinclude comparing newly assigned spectral irradiances,with previous measurements on a customer-submittedrecalibration, comparing spectral irradiance scales withother standards laboratories through internationalcomparisons and periodic scale realizations [14].

Quality system. A quality system is critical for ensur-ing that a measurement is repeatable with the sameuncertainty budget, independent of operator or time. Aquality system, such as the ISO 17025 standard imple-mented at NIST, provides clear documentation of themeasurement process to be followed by the calibrationscientist. It also provides the foundation for implement-ing Deming’s points 3 and 5 in Table 1. The describedmeasurement procedure should be identical to thatvalidated through measurement intercomparisons.

Peer review. Periodic objective technical peer reviewaids the calibration scientist in developing a technicalrigorous and robust measurement approach and associ-ated uncertainty analysis. It aids the elimination ofaspects of the calibration not well founded in funda-mental physics. For a mature calibration service,exploring, in detail, the origin of each component of theuncertainty analysis typically leads to the conclusionthat measurement repeats are unnecessary.

Furthermore, the peer review often reveals steps inthe measurement process that are done, more, by customrather than for their contribution to the final result. Finally, an objective analysis can lead to useful sugges-

tions to improve the overall calibration service. Suchextensive peer review responds to the spirit ofDeming’s points 1, 5, and 8.

6. Conclusions

“Once is Enough” is being implemented throughoutthe calibration programs within the Optical TechnologyDivision. Calibration programs involved includeoptical properties of materials measurement, photo-metry, and radiometry. The detailed management andtechnical peer review undertaken as part of this efforthas already led to significant improvements in thequality and reliability of the measurements. Suchimprovements include elimination of superfluous stepsin the process, optimization of controls to ensure instru-ment performance, enhanced automation to maximizerepeatability, and increased understanding of the uncer-tainty budget. We anticipate that “Once is Enough” willbe fully implemented, division-wide, by the end of2007. In addition to saving time, as much as 50 %, theincreased focus on controlling and understanding thecauses of variability in each step of the process willresult in an improvement in the overall quality of ourmeasurement services, for the benefit of NIST’scustomers.

We anticipate that the lessons learned within theDivision on the implementation of “Once is Enough”can be disseminated to other calibration services with-in NIST and to various calibration laboratoriesinvolved in supporting the U.S. National MeasurementSystem. The concomitant reduction in time and costswill allow industry to reduce costs and improve meas-urement accuracy, to obtain an increased competitiveadvantage in areas such as manufacturing. Continuedexamination of best practices in manufacturing will,likewise, lead to new strategies to improve the per-formance and dissemination of NIST calibrationservices.

AcknowledgementsOne of us Albert C. Parr (ACP) would like to

acknowledge the enthusiastic support of the calibrationscientists in the Optical Technology Division for theirhelp in the division-wide implementation of “Once isEnough.” ACP also wishes to thank Phil Wychorski ofthe Eastman Kodak Company for an introduction to thework of Shewhart and the application of control chartsand quality systems in calibration labs.


49

Appendix 1.

NIST, and formerly NBS, has been performingradiometric and photometric calibrations since thefounding of the Bureau of Standards in the early 20thcentury [15]. The early radiometric and photometricscales were based upon flame sources such as candlesand gas lamps, later progressing to lamps and black-bodies. Measurements typically relied upon a visualcomparison of a test source with a reference source[16]. The quantification and standardization of the rela-tionship between human visual response and photomet-ric and radiometric quantities [17, 18] allowed meas-urements of photometric quantities with detectorssimilar to those used in radiometry, but with appropri-ate filters to mimic the human visual response. In 1975,Blevin and Steiner proposed a redefinition of thecandela from being based upon blackbody radiationto being based upon the amount of optical power at555 nm, the peak wavelength for the human visualresponse [19]. The 1979 Conférence Générale de Poidset Mesures (CGPM) accepted the 1977 ComitéInternational de Poids et Mesures (CIPM) recommen-dation to adopt this definition for the candela. Thisredefinition initiated a revolutionary change in themeasurement of photometric quantities. Subjectivemeasurements based upon visual inspection werereplaced by objective measurements performed byelectro-optical sensors. NIST eliminated such visualcomparisons in their photometric and radiometricmeasurements and implemented additional step toimprove radiometric and photometric calibrations [20],including the modernization and automation of thelaboratory facilities and the establishment of an ISOGuide 25 compliant quality system [21, 22]. The quali-ty system has since been updated to comply with thenew NIST-wide standards based upon the newer ISOGuide 17025 [23].

7. References[1] W. E. Deming, Elementary Principles of The Statistical Control

of Quality, 1st ed., Nippon Kagaku Gijutsu Remmei, Tokyo(1951) p. 103.

[2] W. E. Deming, Sample Design in Business Research, JohnWiley & Sons, New York (1960) p. 517.

[3] From this perspective, successful manufacturing requiresstringent control over the entire manufacturing process.Consider, for example, a complicated product manufacturedusing a series of 1000 steps and components. If the targetfailure rate for the final product is less than 1 part in 1000 oversome period of time, the failure rates of each of the steps andcomponents must be controlled on average to better than 1 partin 106 over the same period of time.

[4] W. A. Shewhart, Economic Control of Quality of ManufacturedProduct, 1st ed., D. Van Nostrand Company, Inc., New York(1931).

[5] W. J. Youden, Experimentation and Measurement, NIST SpecialPublication 672, 1994, U.S. Government Printing Office.

[6] J. H. Walker, R. D. Saunders, J. K. Jackson, and D. A.McSparron, Spectral Irradiance Calibrations, NIST Special Publi-cation SP250-20, 1987. Available at http://ts.nist.gov/ts/htdocs/230/233/calibrations/Publications/series-pdf/SP250-20.pdf.

[7] H. W. Yoon and C. E. Gibson, Spectral Irradiance Calibrations,in NIST Special Publication SP250-** (in preparation).

[8] T. C. Larason, S. S. Bruce, and A. C. Parr, Spectro-radiometric Detector Measurements: Part I—UltaravioletDetectors, and Part II—Visible to Near-Infrared Detectors,NIST Special Publication SP250-41, 1998. Available athttp://ts.nist.gov/ts/htdocs/230/233/calibrations/Publications/series-pdf/SP250-41.pdf.

[9] S. W. Brown, G. P. Eppeldauer, and K. R. Lykke, NIST facilityfor Spectral Irradiance and Radiance Responsivity CalibrationsWith Uniform Sources, Metrologia 37, 579-582 (2000).

[10] H. W. Yoon, C. E. Gibson, and P. Y. Barnes, Realization of theNational Institute of Standards and Technology detector basedspectral irradiance scale, Applied Optics 41(28), 5879-5890(2002).

[11] The International System for Units (SI), 8th ed., BureauInternational des Poids et Mesures, STEDI MEDIA, Paris(2006).

[12] H. W. Yoon, C. E. Gibson, and J. L. Gardner, Spectral RadianceComparisons of Two Blackbodies With TemperaturesDetermined Using Absolute Detectors and ITS-90 Techniques,in Temperature: Its Measurement and Control in Scienceand Industry, Vol. 7, D. C. Ripple, ed., American Institute ofPhysics, pp. 601-606 (2003).

[13] B. N. Taylor and C. E. Kuyatt, Guide for Evaluating andExpressing the Uncertainty of NIST Measurement Results,NIST Technical Note 1297, 1994.

[14] E. R. Wooliams, N. P .Fox, P. M. Harris, and N. J. Harrison, TheCCPR K1-a key comparison of spectral irradiance from 250 nmto 2500 nm, measurement analysis and results, Metrologia 43,S98-S104 (2006).

[15] E. P. Hyde, A comparison of the unit of luminous intensity ofthe United States with those of Germany, England, and France,Bull. Bur. Stand. 3(1), p. 65 (1907).

[16] E. P. Hyde and P. E. Cady, On the determination of the meanhorizontal intensity of incandescent lamps by the rotating lampmethod, Bull. Bur. Stand. 2(1), 414 (1907).

[17] K. S. Gibson and E. P. T. Tyndall, The visibility of radiantenergy, Sci. Pap. Bur. Stand. 19, 131-191 (1923).

[18] W. W. Coblentz and W. B. Emerson, Relative sensitivity of theaverage eye to light of different colors and some practical appli-cations to radiation problems, Bull. Bur. Stand. 14, 167-236(1918).

[19] W. R. Blevin and B. Steiner, Redefintion of the Candela and theLumen, Metrologia 11, 97-104 (1975).

[20] A. C. Parr, A national measurement system for radiometry,photometry, and pyrometry based upon absolute detectors,NIST Technical Note 1421, U.S. Government Printing Office,Washington, DC 20402 (1996) p. 32.

[21] S. S. Bruce and T. C. Larason, Developing Quality SystemDocumentation Based upon ANSI/NCSL Z540-1-1994—theOptical Technology Division’s Effort, Internal Report 5866,NIST, Gaithersburg MD (1996) p. 50.


50

[22] ISO / IEC Guide 25: General requirements for the competenceof calibration and testing laboratories, 3rd ed., InternationalOrganization for Standardization, Geneva (1990).

[23] General Requirements for the Competence of Testing andCalibration Laboratories, ANSI/ISO/IEC 17025:2000, 1rst ed.,NCSL International, Boulder CO (2000).

About the authors: Gerald Fraser is a supervisoryresearch chemist in the Optical Technology Division atNIST and is the leader of the Optical Thermometryand Spectral Methods group in the Division. CharlesGibson is a physicist in the Optical TechnologyDivision and is responsible for performing the spectralirradiance and radiance calibrations discussed in thisarticle. Howard Yoon is a physicist in the OpticalTechnology Division and does research in radiationtemperature measurements in the Optical Temperatureand Spectral Methods Group. Albert Parr is asupervisory physicist and is the chief of the OpticalTechnology Division. The National Institute ofStandards and Technology is an agency of theTechnology Administration, U.S. Department ofCommerce.


51

112 “once is enough” in radiometric calibrations · calibration services, like manufactured...

Documents