uncertainty of extreme fit evaluation for three-dimensional measurement data analysis

Uncertainty of extreme fitevaluation for three-dimensionalmeasurement data analysisWoncheol Choi and Thomas R Kurfess*

Three-dimensional measurement is the process of obtaininginformation about a measured object in the form of surfacecoordinates. Of course this information must be accompanied by adata analysis procedure that evaluates geometric dimensions fromthe measured data. Currently extreme fits, that are based on anL`

norm estimation, are widely applied to analyze these data. Theextreme fit results reflect the functionality of the part, and they aremore consistent with standard definitions. However, they aresubject to sampling uncertainty as they are sensitive to measure-ment sampling density. Previous studies focused on the evaluationmethodology of the extreme fits; however, the uncertainty of theextreme fits has not been thoroughly addressed. In this paper, weinvestigate the uncertainty of extreme fits, and propose a statisticalapproach to evaluate it. We employ the bootstrap method todetermine the confidence interval of the extreme fit evaluations.The proposed method is independent of the geometry andevaluation method; thus, it can be easily generalized for variousextreme fit evaluations and geometries.q 1998 Elsevier ScienceLtd. All rights reserved

Keywords: metrology, tolerance, precision, manufacturing,quality, sampling

INTRODUCTION

With the implementation of three-dimensional measure-ment capabilities, metrology systems have become sig-nificantly more powerful. A three-dimensionalmeasurement device such as a coordinate measuringmachine (CMM) samples the surface of a measure objectand generates three-dimensional coordinate points from thatsurface. The measurement output is a collection of digitizedsurface points. Compared to traditional two-point measure-ments, the three-dimensional measurement yields morecomprehensive information about the actual part geometry.Thus, more details about surface variation can be observed,and various sections of different geometry features can bemeasured in a single process. Currently, three-dimensional

measurement devices are an essential tool for measuring afree form surface (e.g. turbine blades), and they canserve as a general purpose dimensional measurement deviceas well.

After measurement data are collected by a three-dimensional measuring machine, an independent numericalanalysis must be performed to evaluate geometric variationsor verify tolerance conformance. The measurement outputfor all such measurement systems is processed in a compu-tational environment as a geometric rather than a parametricentity. Being in a geometric form has the advantage that themeasurement output can be directly integrated with CADdesign models. On the other side, one must process themeasured data to extract parametric information. Themeasured data are typically a set of vectors representingthe coordinates of surface points. If one is interested inevaluating the individual deviations of the measured points,an appropriate rigid body transformation must be applied tothe point set, and then the deviation from the design modelmust be obtained. This task is known as data localization.On the other hand, if one is interested in a specific geometricvalue, such as the diameter of a hole feature or a B-splinesurface representing the surface variation, then an appro-priate geometry must be reconstructed from the measuredpoints. As such, the focus of the numerical analysis is totransform the raw coordinate data into a more appropriateform for interpretation (i.e. inspection results).

Least squares and extreme fits are typically utilized incoordinate data numerical analysis and various researchershave developed a number of numerical algorithms for three-dimensional data analysis. Most CMMs include data analy-sis software using the least squares fit. Examples of leastsquares and extreme fits are shown inFigures 1 and 2.Recently, the extreme fit evaluation has generated signifi-cant interest by metrologists as it represents anL` normestimation such as min–max fit. The main advantage forthe extreme fit is that it is more consistent with ASMEdimensioning and tolerancing standards1,2. The geometricconcepts defined in the standards, such as actual matingenvelope or circularity, can only be verified by utilizingextreme fits as shown inFigure 2.

The use of the extreme fit poses a new set of challengingissues related to the inaccuracy of the data analysis results.As recent studies demonstrate3–5, current data analysis tech-niques are not robust and implicitly possess uncertainty.Furthermore, the three-dimensional measurement dataanalysis procedures are not a closed-form evaluation proce-dure; rather, they are a numerical iterative estimation that

Computer-Aided Design, Vol. 30, No. 7, pp. 549–557, 1998q 1998 Elsevier Science Ltd. All rights reserved

Printed in Great Britain0010-4485/98/$19.00+0.00PII: S0010-4485(98)00012-8

549

*To whom all correspondence should be addressedThe George W. Woodruff School of Mechanical Engineering, GeorgiaInstitute of Technology, Atlanta, GA 30332-0405, USAPaper Received: 9 December 1996. Revised: 1 February 1998. Accepted: 1February 1998

inherently involves uncertainty. Since the measurementmust be substantially more precise than the variationbeing characterized, the precision of measurement is criti-cal. As such, a minor error in the data analysis procedure canyield a significant misunderstanding of measurementresults. Therefore, the accuracy problems are not limitedonly to hardware but also to analysis software.

While extreme fits do have significant utility, they alsohave limitations. Unlike the least squares fit, the extreme fitis determined using a few points from the measurement datathat are only a sample of the true surface. If a differentsample of measurement data were taken, the extreme fitevaluation result would be different because the extremepoints in the measurement data would be different.

Figure 3 demonstrates this issue. The graph represents thevariation of the value of a planar minimum zone withrespect to different samples from the same true surface. Ifone judges the part from the result of a single set ofmeasurement data, an incorrect conclusion may be drawn.Nevertheless, such a fact is not thoroughly considered incurrent practice. Because the measured data are a sampleof the true surface, the extreme evaluation result must con-tain a specific amount of uncertainty.

In this paper, we investigate the uncertainty present inextreme fit evaluations, and we provide a methodology toevaluate the uncertainty. We show that the uncertainty ofextreme fit results is mainly due to the plug-in nature of theextreme fit, and that the uncertainty can be evaluated by acomputation-based statistical method. Where the plug-inestimation is the evaluation of a parameter with a finitenumber of samples regarding them as the entire population.Examples are presented to demonstrate the methodology.

LITERATURE REVIEW

Extreme fit problems have been analyzed in order to addressform tolerance verification problems. Since the formtolerances are defined as minimum zones, the traditionalleast squares fit approach is not capable of resolving the

Extreme fit evaluation for 3D measurement: W Choi and T R Kurfess

550

Figure 1 A least squares fit of a circle

Figure 2 Extreme fit examples

Figure 3 Variation of an extreme fit evaluation result with respect to different samples

problem precisely. A number of extreme fit algorithms6

have been proposed in order to directly solve the minimumzone evaluations: Murthy and Abdin7, Goch8,9, Wang10 andKanada11 use non-linear optimization formulations;Etesami and Qiao12, and Huang et al.13,14 exploit acomputational geometry search approach; Carr andFerreira15,16 use a linear programming approach, andChoi17 employs a root finding technique. The algorithmsprimarily solve a geometric problem between a set of pointsand a surface in three-dimensional space. The fact thatthe measured points are a sample of an unknown truesurface is not addressed. Furthermore, since the problem isbased on a non-differential function, the formulation is apurely numerical iteration. Typically, it is computationallyextensive, and an analytic formulation of the result isdifficult.

Sahoo and Menq18 formalized the uncertainty in ameasurement data analysis, proposed a data localizationalgorithm based on a least squares fit, and addressed therobustness of the algorithm. The localization determined arigid body transformation between a set of measured pointsand an original design model. If the measured data con-tained an error in the measurement process, the determinedtransformation parameters were found to have uncertainty.Employing a sensitivity analysis, they formulated thebounds of the transformation parameters with respect tothe measurement error. Their formulation is not based onstatistics, but the underlying concept is similar to the devel-opment of a statistical confidence interval.

A more direct statistical approach can be found in Kurfessand Banks19. In their statistical tolerance conformanceverification method for size tolerance, they fully utilizedthe statistical property of dimensional evaluations. Unlikeprevious attempts, their method built a model that describedthe actual part variation, and the conformance verificationwas made from the part variation model. In order to buildthe model, they used a statistical maximum likelihoodestimation, which is equivalent to a least squares fit. In theprocess of modeling the part variation, they employed acomputation-based statistical method, the jackknifemethod, to evaluate the variance–covariance matrix of theestimating parameters. Then the confidence interval couldbe derived. From the model and the confidence interval, thetolerance conformance could be verified. Thus, the dataanalysis was performed exploiting the statistical propertyof measurement data.

Kurfess and Banks19 also developed a statistical relation-ship to determine the validity of a geometric model. A set ofcriteria was developed to ensure, with a given confidenceinterval, that the point data accurately represent the surfacefrom which they are sampled. The work presented in thispaper has the requirement that the data do, indeed, accu-rately represent the surface geometry of the part as specifiedby these criteria.

Mestre and Abou-Kandil20 approached the problem froma different direction. Recognizing the problem with currentextreme fit evaluations, that ignore the variation inunmeasured sites, they proposed a new method for the flat-ness evaluation. Employing Bayesian prediction theory,they derived the confidence interval of the surface variationat unmeasured sites. Then, the minimum zone was deter-mined with respect to the derived confidence interval. Bydoing so, they could overcome the plug-in estimation errorof extreme fit evaluation, and the minimum zone could becloser to the true flatness value. However, with respect topractical implementation, the formulation had difficulty in

constructing the minimum zone from multiple confidenceinterval hills and valleys.

The above studies recognize the uncertainty problem incurrent three-dimensional measurement data analysis andaddress the problem in different manners, but the uncer-tainty of the extreme fit evaluation has not been explicitlyaddressed. In this paper, we investigate the extreme fituncertainty, and formulate a methodology to evaluate theuncertainty.

CHARACTERIZATION OF EXTREME FITUNCERTAINTY

Uncertainty exists in any measurement data analysis results.For the case of a least squares fit, the uncertainty is wellformalized by statistical theories. A least squares fit problemcan be represented as a regression

minSSR(v) ¼∑n

i ¼ 1distancepi ,S(v)

� 2 (1)

wheren is the number of measured points,p i is a vectorrepresenting theith measured point,S is the model surface,andv is the model parameter vector. Solving eqn (1) as anoptimization, one can obtain the estimating parameterv.The uncertainty ofv can be represented as the confidenceinterval of the estimation, which is the range where theunknown truev value lies. The marginal confidence intervalfor a parametervp can be evaluated as

vp 6 se(vp)tn¹ p,a=2 (2)

wherevp is the estimation for the parametervp, tn¹p, a/2 is theuppera/2 percentile for the Student-t distribution withn¹ pdegrees of freedom and se(vp) is the standard error of theparameter. The joint confidence interval can be representedas:

(v ¹ v)C¹ 1(v ¹ v) # ps2Fp, n¹ p,a (3)

where v is the estimated parameter vector, andC is thevariance–covariance matrix,p is the number of parameters,s is the residual mean squares distance, andFp,n¹p,a is theupper (percentile for Fisher’sF distribution withn andn¹ pdegrees of freedom.

Consider a simple example as shown inFigure 4. The realprofile is a circular geometry with irregular variations.Provided that the fitting geometry has been correctlychosen, that the noise is approximately normal, and thatthe measured points are sufficient to represent the noisedistribution of the true surface, the least squares fit geometryis the highest probable (or maximum likelihood) estimationof the true surface profile. The fitted circle is a statistical


551

Figure 4 Least squares fit from measured points

estimation; thus, it contains an uncertainty. Here, the sourceof uncertainty is the stochastic noise between the fittedcircle and the measured points. Because the measuredpoints are not perfectly located on the fitted circle, there isalso a possibility that the true circle may be slightly differ-ent. Such an uncertainty of the estimated parameters, centercoordinates and the radius of the circle, can be representedas a confidence interval in eqn (3).

For the extreme fit case, the source of uncertainty is dif-ferent. A typical form of the problem is

minM(v) ¼ max[distance pi ,S(v)�

] (4)

Since the functionM(v) is generally non-differentiable, itmust be determined by an iterative search algorithm. Animportant point is that the fitting geometry is determinedby extreme points of measured data. If the fitting geometryis properly defined, there is a unique solution that willsatisfy the optimality condition and will be constrained bythe extreme points. Unlike the least squares fit, the stochas-tic noise in the measurement data does not contribute to theuncertainty of the evaluation. The solution is not an estima-tion based on the probability but a geometric evaluation.Thus, the evaluation includes all the stochastic noise inthe measured data.

The uncertainty associated with extreme fit comes fromthe plug-in nature of the extreme fit. The extreme fit geo-metry is determined from a few extreme points. However,the extreme points in the measurement data that contributeto the extreme fit geometry are not necessarily the extremepoints in true surface profile. As shown inFigure 5, the truecircumscribing circle and the circumscribing circle deter-mined by measured points are not equivalent. This resultis due to the fact that the measurement data fail to capturethe true extreme points. The extreme points in the measure-ment data are samples of the true surface points. Dependingon how the data are sampled from the actual surface, theevaluated extreme fit values are different. Thus, even thoughthe extreme fit evaluation result is determined from theextreme points of the measured data, the result is typicallynot the true value. This discrepancy will always exist as themeasured data are a finite sample from an unknown truesurface.

The plug-in estimation can be formalized as follows. Sup-posen points are measured from a feature surface. Since thefeature is not perfect, its surface has geometric disturbancesfrom the ideal form due to form variation, waviness, andsurface roughness. We represent such disturbances of sur-face profile as a probability distributionG. G can be definedas the distribution of deviations normal to the feature sur-face or a three-dimensional probability density distribution.The completeG is always unknown because it exists only onthe real surface. Then, the measured point setP can bedefined as a sample setP ¼ (Pi , i ¼ 1, 2,…, nlPi [ R3)that is taken from the real surface with probability distri-bution,G. Suppose we are interested in a parameterv that

represents a geometric characteristic of the feature, andv isdefined as the result from a certain evaluation functionu ofG as

v ¼ u(G) (5)

The function,u, evaluates an extreme fit from a continuoussurface that has the probability distributionG. The truevalue of v cannot be determined becauseG is unknown.Thus, we define an empirical distribution functionG as anestimate ofG. It is assumed that the quantity and spatiallocation of the points are sufficient to yield a proper estima-tion of G. In practice this is implemented by generatinglarge dense data sets. This may not be a practical approachfrom an implementation perspective as larger point setsoften require larger amounts of time to gather. (Althoughnew optical laser scanners may eliminate this issue.)G isdefined to be a discrete distribution that assigns probability1/n to each value ofPi

G¼ÿ

P¼ P1

� =n, P¼ P2

� =n, P¼ P3

� =n, …,

3 P¼ Pn

� =nÞ ð6Þ

where #{P ¼ Pi} represents the number ofP that is equal toPi. Typically, all the measured points are distinct; thus,G isthe distribution with the same probability on eachPi. Theplug-in estimate ofv, v, can be defined as the evaluation ofthe parameter with the empirical distribution:

v ¼ u(G) (7)

Thus, the plug-in estimation is a geometric evaluation froma set of points represented byG. The plug-in estimation isclose to the true value when the empirical distributionclosely represents the unknown distributionG. If themeasured extreme points are close to the true extremepoints in the real surface, the discrepancy between thetrue geometry and the geometry determined by the extremefit becomes small. Thus, the density of measurement datahas a great influence on the accuracy of the extreme fitevaluation.

Because the results are dependent on measurement den-sity and surface location distribution, the approach of usingan extreme fit on sampled data is vulnerable to error. Sinceextreme fits are often performed with insufficient measure-ment data, the evaluation results may contain a significantlevel of error that can possibly yield an incorrect conclusionabout the measured part. Simply increasing the measure-ment data density cannot solve the problem, as increasingnumber of measurement points increases the cost (complex-ity and time) of the extreme fit evaluation. Currently imple-mented extreme fit algorithms are computationallyextensive algorithms. They are considerably slower thanthe least squares fit with the same amount of measureddata. Also, measurement time and data storage cost increasewith increasing sample size. Regardless of the sample size,there is always uncertainty in the extreme fit results becausethe measurement data are finite. Without knowing the exactamount of the uncertainty in the results, reducing the uncer-tainty may not be meaningful.

One approach to determining the level of uncertainty inan extreme fit is to directly evaluate it. In Section Section 4,an extreme fit uncertainty evaluation methodology using abootstrap method is presented. Once such an assessment ismade, the appropriateness of the current evaluation can bejudged by examining its uncertainty. If the uncertainty isunacceptable, the measurement data size must be increased.


552

Figure 5 True value versus estimated value by measured data

If the uncertainty level is deemed acceptable, then the datacan be evaluated using an extreme fit. By doing so, one canmake more accurate and economic measurements.

BOOTSTRAP ESTIMATE OF STANDARDERROR

Analytic evaluation of the extreme fit uncertainty is quitedifficult. The parameter evaluation, represented as thefunction u, is actually a numerical evaluation based on anon-differential function. Developing an analytic relationbetween a continuous distribution and a sample distributionwith a non-differential function is a quite challenging task.Even if such a formulation could be derived, it may beapplicable only to a specific case, and it may be difficult togeneralize. Thus, we approach this problem with anumerical-based method and bootstrap method21. This is apopular computation-based statistical approach that can beapplied to evaluating the standard error of a plug-inestimation. The formulation is now presented.

Suppose a point setP ¼ (Pi , i ¼ 1, 2,…, nlPi (R3) repre-sents measured points from a workpiece that has anunknown probability distribution,G. The geometric para-meter of interestv is evaluated by a plug-in estimation fromthe empirical distribution as defined in eqn (6). The error ofthe estimate is unknown because the true distribution,G, isunknown. The bootstrap method can be used to estimate thestandard error from the sampled data. First, a bootstrapsample must be generated from the sampled data. The boot-strap sampleP* is defined to be a random sample of sizendrawn fromG as:

Pp ¼ Pp1, Pp

2, Pp3, Pp

4, DOTSLOW, Ppn

ÿ �and

G → Pp(8)

Ppi are drawn with replacement from the population having the

G distribution. Thus,P* is a resampled version ofP such thatP* may contain some elements ofP, repetitively, but may notinclude others at all. From the bootstrap sampleP*, we eval-uatevp, the bootstrap replication ofv, which is defined as

vp ¼ u(Pp) (9)

The functionu can be the same extreme fit used to findv.The bootstrap estimate of the standard error is the standarderror of vp in the distribution ofG defined as

seG(vp) ¼

��varG(vp)

q(10)

The bootstrap estimate of the standard error can be numeri-cally calculated using the bootstrap samples. In each boot-strap samplePp

b, the estimated parametervpb is evaluated.

The bootstrap estimate of the standard error can be evalu-ated as

esB ¼

∑B

b¼ 1vp

b ¹ vp(·)

� �2(B¹ 1)

8>>><>>>:9>>>=>>>;

1=2

(11)

where

vp(·) ¼

∑B

b¼ 1vp

b

BandB is the number of bootstrap replications.

It is known that

limB→`

esB ¼ seG(vp) (12)

With a reasonably large number of bootstrap replications,the bootstrap estimate of standard error can be evaluated.Thus, the bootstrap method estimates the unknown trueerror through the similar bootstrap resampling error. For asingle parameter, the confidence interval can be evaluated aseqn (2). For the bootstrap method, the standard error isestimated by the bootstrap samples. Thus, the confidenceinterval based on a normal distribution assumption can beobtained as:

v 6 seG(vp)tB¹ 1,a=2 (13)

When more than one parameter is estimated at the sametime, v is a vector representing the estimating parameters.Supposev is a p-dimensional vector. In order to find thejoint confidence interval, the variance–covariance matrixof v must be determined. With the same bootstrap replica-tions, the variance–covariance matrix can be estimated as

C¼

∑B

b¼ 1vp

b ¹ vp(·)

ÿ �vp

b ¹ vp(·)

ÿ �T

(B¹ 1)(14)

where

vp(·) ¼

∑B

b¼ 1

vpb

Band B is the number of bootstrap replications. Then thebootstrap estimate of the 1¹ a joint confidence regionbecomes

(v ¹ v)TC¹ 1(v ¹ v) # pFp, B¹ p,a (15)

The proposed method evaluates the extreme fit uncertaintyas a statistical confidence interval where the unknown truevalue is bounded. The fundamental concept is that thoughthe extreme fit is a geometric solution, there is a possibilitythat the true value might differ from the evaluated value. Ifthe same surface is measured repetitively with the samemeasurement conditions, the evaluated value will beslightly different each time, and it will follow a probabilitydistribution. The probability distribution is estimated fromthe proposed method, and the bounds of the value are deter-mined by the confidence interval. Thus, the possible errorrange is obtained. The implementation of the proposedmethod is demonstrated in the following section.

EXAMPLES

Two examples are demonstrated. The first one is a planeminimum zone evaluation, that can be applied for flatnessverification. The true surface is a 103 10 square on thex–yplane with randomz-directional noise superimposed on thedata (seeFigure 6). The probability distribution of the truesurface can be represented as the probability densityfunction f (z) given by:

f (z) ¼10 0# z# 0:1

0 elsewhere

((16)

One hundred evenly distributed points are sampled from thetrue surface. From these 100 points, the minimum zone,


553

which is a plug-in estimate of the flatness, is constructed(seeFigure 7). From the specific measurement data, theminimum zone value is evaluated as 0.09768. Since thenoise distribution of the true surface is given by eqn (16),the true minimum zone must be 0.1. Comparing this value tothe evaluated value, the discrepancy is about 2.3%, whichmay not be a negligible amount. However, such a discre-pancy is unknown in an actual application. In order to esti-mate the uncertainty of the evaluation, the proposed methodis applied. The empirical distribution is the discrete distri-bution that simply assigns 1/100 probability to each one ofthe initial measurement points. This is reasonable sincethere is uniform probability that a particular point will bechosen. From the empirical distribution, a bootstrap sampleis taken, and the bootstrap replication of the estimatevp isevaluated. The procedure of taking a bootstrap sample andevaluating the bootstrap replication is iterated 100 times.The standard error is estimated from eqn (11) as 0.00145.For a ¼ 0.05, n ¼ 100, the student distributiont99,0.025¼1.98422. Thus, the 95% confidence interval is calculated as

v ¼ 0:097686 0:00287 (17)

This indicates that the true flatness value is between 0.09481and 0.10055 with 95% probability.

The second example is a maximum inscribing cylinderand a minimum circumscribing cylinder, as shown inFigure8. This example demonstrates the application of the evalua-tion method to an actual mating envelope or actual mini-mum material envelope problem. The true surface is acylinder with diameter 10 and height 10. The cylinder sur-face contains noise that has the probability density distribu-tion f(r) given by

f (r) ¼200 4:99, r , 5:01

0 elsewhere

((18)

wherer is defined as normal to the cylinder surface.In order to evaluate the position of a cylindrical feature,

36 points are taken from the true surface. One of twopossible envelopes can be constructed, an inner envelopeand an outer envelope. The outer envelope is the minimumcircumscribing cylinder and the inner envelope is themaximum inscribing cylinder. The outer envelope is theactual mating envelope as an external feature or the actualminimum material envelope as an internal feature. The innerenvelope is the actual minimum material envelope as anexternal feature or the actual mating envelope as an internalfeature. Depending on the actual feature type and thematerial condition, either one of the envelopes can beevaluated.

In this example, the axis of the cylinder is set to parallelthe z axis for the simplicity and the extreme fit is appliedwithout rotation. Therefore, the data points must only betranslated in thex andy directions, which we will call thepositional shift. The positional shift is evaluated in order toexamine the positional variation of the data. The two shiftparameters to be estimated arev ¼ (xs, ys)

T. These are thexand y shifts required to place the data points within thedefined cylindrical zones. From a set of 36 points, the max-imum inscribing and the minimum circumscribing cylindersare evaluated. The positional shifts from the initial cylindercenter are calculated as shown inTable 1.

In order to evaluate the uncertainty of the positionalshifts, the proposed method is applied to both cases asshown in Figure 9. For the outer cylinder case,v ¼(0.00100,¹ 0.00173)T . As in the previous example, 100bootstrap samples are resampled from the measured data,andvp , the bootstrap replications ofv, are evaluated in eachcase. Fromvp, the variance–covariance matrix is evaluatedas in eqn (14). The estimated covariance matrix ofv for theouter cylinder is:

C¼6:311963 10¹ 7 1:135473 10¹ 7

1:135473 10¹ 7 1:973133 10¹ 6

!(19)

The evaluating parameterp ¼ 2, andn ¼ 100. Fora ¼ 0.05,F2,98,0.05¼ 3.0892. The 95% joint confidence region is anellipse in two-dimensional parameter space that can be cal-culated as

(v ¹ v)TC¹ 1(v ¹ v) # 6:1784 (20)

The evaluated positional shift value and the joint confidenceregion are plotted inFigure 10.

For the inner cylinder case,v ¼ (0.000636,¹ 0.00190).The bootstrap estimate of the variance–covariance matrix is


554

Figure 6 Flatness evaluation from measured data

Figure 7 Bootstrap iteration step

Figure 8 Evaluation of position from the minimum circumscribing cylin-der or the maximum inscribing cylinder

Table 1 Position shift of the cylindrical feature

xs ys

Outer cylinder (MCC) 0.001001 ¹ 0.00173Inner cylinder (MIC) 0.000636 ¹ 0.00190

Figure 9 Bootstrap step for positional shift evaluation uncertainty

C¼1:803253 10¹ 6 ¹ 2:78293 10¹ 8

¹ 2:78293 10¹ 8 1:719343 10¹ 6

!(21)

F2,98,0.05¼ 3.0892 as the outer cylinder case. Thev and thejoint confidence interval is shown inFigure 11.

The flatness example indicates that the minimum zoneevaluation results for the flatness value has an estimateduncertainty of62.8%. Depending on the geometric andfunctional specifications, this amount of uncertainty maybe significant. If this is the case, more measurement datamust be taken. If the uncertainty were not evaluated for theflatness measurement, the error of the extreme fit evaluationmight yield an incorrect conclusion from the measurementdata. By evaluating the uncertainty, more information as tothe validity of the result is obtained. It must be noted that

this specific result does not provide a general sampling cri-terion. All flatness values evaluated from 100 measurementpoints do not have the same amount of the uncertainty. Theuncertainty depends on the size of the flatness and the dis-tribution of the surface noise.

The cylinder example results indicate that the positionalshift of the measured cylinder is not significant. The con-fidence interval specified incorporates the entire expectedrange of the zero shift position. This is demonstrated by thefact that the uncertainty is significantly larger than the eval-uated value. The large uncertainty space inFigures 10 and11 indicate that the true positional shift can be substantiallydifferent from thev calculated. Large confidence regions areindicative of the possibility that the true values ofv candiffer significantly from the estimated values ofv. For this


555

Figure 10 95% joint confidence region for the positional shift of the outer cylinder

Figure 11 95% joint confidence region for the positional shift of the inner cylinder

example, if one is interested in a more precise position shift,either more measurement data must be collected or the sig-nificance of the hole cylindricity with respect to the positionshift must be examined. Thus, the uncertainty evaluationcontributes to a broad understanding of measurement data.

The evaluated confidence interval is based on the assump-tion that the unknown error is normally distributed. Thus,the boundary is symmetric, and the size is determined usinga normal distribution. However, the actual error may not benormal and more likely, it may not be symmetric. For exam-ple, as the flatness is evaluated from the minimum zoneevaluation, the extreme fit evaluation always underestimatesthe true flatness value. Thus, the error is always unidirec-tional. This issue can be addressed by estimating the errorprobability density distribution from the bootstrapsamples. Then, a more precise confidence interval can beevaluated. However, that problem is beyond the scope ofthis paper.

SUMMARY AND CONCLUSIONS

The extreme fits currently practiced to evaluate functionalgeometry from three-dimensional measurement data areplug-in estimations that contain a discrepancy from the truevalue. In order to assess the measurement data moreprecisely, the uncertainty in the extreme fit results must becharacterized. This paper suggests a methodology toevaluate the uncertainty in the extreme fit. A computation-based statistical method, the bootstrap method, is employedin this work. The bootstrap method estimates the extreme fitevaluation error by resampling and bootstrap replication ofthe extreme fit evaluations. The uncertainty of the extremefit evaluation can be represented as a statistical confidenceinterval.

There are two major advantages to the proposedmethodology:

(1) The uncertainty of an extreme fit evaluation can beobtained from the measurement data. It is not necessaryto obtain a separate measurement data set or to test themeasurement procedure. The uncertainty evaluation isjust an additional numerical analysis procedure. Thus,the uncertainty evaluation can be performed for anymeasurement data whenever necessary.

(2) The proposed method can be easily generalized. Sincethe method is based on a computational approach, it isindependent of the detailed extreme fit evaluation pro-cedure. Thus, the method can be applied to differentgeometry and different types of evaluations. Besidesthe form error and position error evaluations presentedin the example, the method can be applied to profileerror evaluation, datum reference frame constructionor size tolerance evaluations.

The major disadvantage of the proposed method is mainlythe computational cost. The bootstrap method requires re-evaluation of the extreme fit for multiple times. The extremefit evaluation itself is a computationally extensive proce-dure. Iterating such a calculation for multiple bootstrapreplications will heavily tax computational resources.With current computing power, it may be difficult toapply this approach to every measurement data set. How-ever, we believe that a smart choice of the bootstrap itera-

tion size and increasing computer speed will make widerapplication possible.

An important contribution of this work is the samplingproblem. The sample size can be determined based on theevaluation uncertainty. The sample size is critical, asthe nature of sampling inherently possesses the risk of eva-luation error. Of course one must always be cognizant of thefact that larger sample sizes yield improved confidenceintervals but at increased costs. The optimal sample sizemust be determined while making the evaluation uncer-tainty sufficiently small. However, the problem is notstraightforward because this uncertainty can be character-ized only after the measurement data are taken, and thesample size must be determined before the measurementis performed. This is a significant issue that must beaddressed in future research.

REFERENCES

1. ASME Y14.5MDimensioning and TolerancingAmerican Society ofMechanical Engineers, 1994.

2. ASME Y14.5.1MMathematical Definition of Dimensioning and Tol-erancing PrinciplesAmerican Society of Mechanical Engineers,1994.

3. Waldele, F. et al., Testing of coordinate measuring machine software.Precision Engineering, 1993,15(2), 121–123.

4. Hocken, R., Raja, J. and Babu, U., Sampling issues in coordinatemetrology. Manufacturing Review, 1993,6(4), 282–294.

5. Hopp, T., Computational metrology. Manufacturing Review, 1993,6(4), 295–304.

6. Kanada, T. and Suzuki, S., Evaluation of minimum zone flatness bymeans of nonlinear optimization techniques and its verification. Pre-cision Engineering, 1993,15, 93–98.

7. Murthy, T. S. R. and Abdin, S. Z., Minimum zone evaluation ofsurfaces. International Journal of Machine Tool Design andResearch, 1980,20, 123–136.

8. Goch, G., Effective multi-purpose algorithm for approximation andalignment problems. Annals of the CIRP, 1990,39, 553–556.

9. Goch, G., A universal algorithm for the alignment of sculptured sur-faces. Annals of the CIRP, 1992,10(1), 51–63.

10. Wang, Y., Minimum zone evaluation of form tolerances. Manufactur-ing Review, 1992,5(3), 213–220.

11. Kanada, T. and Suzuki, S., Application of several computing techni-ques for minimum zone straightness. Precision Engineering, 1993,15,274–280.

12. Etesami, F. and Qiao, H., Analysis of two-dimensional measurementdata for automated inspection. Journal of Manufacturing Systems,1990,9, 21–34.

13. Huang, S. T., Fan, K. C. and Wu, J. H., A new minimum zone methodfor evaluating straightness errors. Precision Engineering, 1993,15,158–165.

14. Huang, S. T., Fan, K. C. and Wu, J. H., A new minimum zone methodfor evaluating flatness errors. Precision Engineering, 1993,15, 25–32.

15. Carr, K. and Ferreira, P., Verification of form tolerances. Part I: Basicissues, flatness and straightness. Precision Engineering, 1995, 17,131–143.

16. Carr, K. and Ferreira, P., Verification of form tolerances. Part II:Cylindricity and straightness of a median line. Precision Engineering,1995,17, 144–156.

17. Choi, W., Computational analysis of three dimensional measurementdata. PhD Dissertation, Carnegie Mellon University, 1996.

18. Sahoo, K. C. and Menq, C. H., Localization of 3-D objects havingcomplex sculptured surfaces using tactile sensing and surface descrip-tion. Journal of Engineering for Industry, 1991,113, 85–92.

19. Kurfess, T. and Banks, D., Statistical verification of conformance togeometric tolerance. Computer-Aided Design, 1995,27(5), 353–361.

20. Mestre, M. and Abou-Kandil, H., Measuring the errors of form ofindustrial surfaces: prediction and optimization. Precision Engineer-ing, 1994,16(4), 268–275.

21. Efron, B. and Tibshirani, R.,An Introduction to the Bootstrap. Chap-man and Hall, London, 1993.


556

Thomas R Kurfess received his S.B., S.M. andPh.D. degrees in mechanical engineering fromM.I.T. in 1986, 1987 and 1989, respectively.He also received an S.M. degree from M.I.T. inelectrical engineering and computer science in1988. Following graduation, he joined Carne-gie Mellon University where he rose to therank of Associate Professor. In 1994 hemoved to the Georgia Institute of Technologywhere he became an Associate Professor in theGeorge W. Woodruff School of MechanicalEngineering.

Woncheol Choi received his B.S. degree inMechanical Design and Production Engineer-ing from Seoul National University. He thenreceived M.S. and Ph.D. degrees in Mechan-ical Engineering from Carnegie Mellon Uni-versity. He is currently a research scientist atthe Precision Machinery Design Center inSeoul University and an adjunct professor atthe Industrial Engineering Department ofDongkuk University. His research interestsinclude CAD, metrology, digital image proces-sing and virtual engineering.


557

uncertainty of extreme fit evaluation for three-dimensional measurement data analysis

Documents