interobserver variability of carotid doppler peak velocity measurements among technologists in an...

Interobserver variability of carotid Doppler peakvelocity measurements among technologists in anICAVL-accredited vascular laboratoryMarc M. Corriveau, MD, FRCS(C),a and K. Wayne Johnston, MD, FRCS(C),b Montreal, Quebec; andToronto, Ontario, Canada

Objective: This study was designed to investigate interobserver variability in the measurement of internal carotid artery(ICA) peak systolic velocity (PSV). We hypothesize that the reproducibility of repeated duplex scanning parameters, inthe hands of very experienced vascular technologists in a laboratory accredited by the Intersocietal Commission forAccreditation of Vascular Laboratories, would be excellent.Methods: Thirty-one patients underwent carotid duplex scanning by three vascular technologists using the same duplexscanning system. They examined patients with the laboratory’s standard protocol. Statistical analysis of the sources ofvariation was carried out with two-way analysis of variance. The Altman-Bland method was used to detect bias andevaluate the interval of agreement between technologists for the ICA PSV on a continuous scale. The � statistic enabledmeasurement of agreement for ICA PSV on a categorical scale of stenosis (<50%, 50%-70%, >70%).Results: Patient variability was responsible for 97.2% of the total variance, with only 0.58% (P < .005) attributed to thetechnologists. The level of agreement on a continuous scale between the measurements of ICA PSV by our technologistsis wide. For individual patients it ranged from �25% to 43% between technologists A and B, �27% to 43% betweentechnologists A and C, and �27% to 31% between technologists B and C. When we compared the three technologists, nosystematic overestimation or underestimation of the ICA PSV was found (ie, no fixed bias). The level of agreementbetween the technologists did not depend on the value of the PSV (ie, no proportional bias). However, analysis of ICAPSV agreement on a categorical scale revealed almost perfect agreement (� >0.8).Conclusion: From measurements of PSV, the severity of carotid stenosis can be reproducibly categorized into ranges(<50%, 50%-70%, >70). However, the unacceptably wide interobserver variation of ICA PSV on a continuous scalemakes the interchangeability of our technologists’ measurements problematic for clinical use, as in determination ofprogression of severity of stenosis. When an ICA PSV measurement is in the vicinity of a cutoff value, the diagnosticaccuracy may be improved with the use of additional diagnostic testing. (J Vasc Surg 2004;39:735-41.)

Duplex scanning of the carotid arteries is a standarddiagnostic test for assessing the severity of carotid arterialocclusive disease. The high accuracy of this noninvasivetest, compared with digital subtraction angiography andmagnetic resonance angiography, has made duplex scan-ning the main imaging method for screening, preoperativeassessment, and follow-up. Diagnosis is based on one or acombination of measurements or observations in the re-gion of a stenosis, including peak systolic velocity (PSV),end-diastolic velocity, ratio of PSV in the internal carotidartery (ICA) to common carotid peak velocity, detection ofdisturbed flow (spectral broadening), and severity of plaqueas assessed on the B-mode image. Measurement of ICAPSV is widely used to determine the severity of a stenosis.1-3

From Department of Surgery, McGill University and Royal Victoria Hospi-tal,a Montreal, and Department of Surgery, University of Toronto andToronto General Hospital.b

Supported by the R. Fraser Elliott Chair in Vascular Surgery.Competition of interest: none.Additional material for this article may be found online at www.mosby.

com/jvs.Reprint requests: Dr K. Wayne Johnston, Toronto General Hospital, 5

Eaton, Rm 309, 200 Elizabeth St, Toronto, Ont, Canada M5G 2C4(e-mail: [email protected]).

0741-5214/$30.00Copyright © 2004 by The Society for Vascular Surgery.doi:10.1016/j.jvs.2003.12.017

The overall accuracy for detecting a diameter reductionof 70% or greater is 88% to 95%.3-5 Although these resultsare good, perfect agreement is limited by errors in theduplex scan measurements, including interobserver varia-tion among the technologists, and the inaccuracy of usingangiography as the standard for interpreting the severity ofthe stenosis.6,7

For clinical decision-making, it is important that themeasurement of ICA PSV be accurate, with low interob-server variability. This is particularly important in patientswith a stenosis that is in the vicinity of a cutoff value that willresult in a stenosis being inaccurately reported. An inaccu-rate measurement may result in additional unnecessarydiagnostic testing, or inappropriate intervention or medicaltreatment.

This study was undertaken to determine interobservervariability in the measurement of ICA PSV. One indicatorfor the reliability of a method is its reproducibility. Westudied the reproducibility of repeated duplex scanningparameters in the hands of very experienced vascular tech-nologists in a laboratory accredited by the IntersocietalCommission on Accreditation of Vascular Laboratories(ICAVL). Special attention was paid to the variability thatwould occur in the categorization of stenosis with border-line severity.

735

JOURNAL OF VASCULAR SURGERYApril 2004736 Corriveau and Johnston

BACKGROUND

In making velocity measurements, several factors causeerrors in PSV measurement. These sources of error fall intothree broad categories: factors intrinsic to the artery beingstudied, such as vessel site, size, depth, tortuosity, plaquecalcification, and vessel acoustic impedance (reflection andrefraction)8; variations attributable to equipment, such astransducer beam pattern (steering vs nonsteering), signalprocessing, signal-to-noise ratio, and factors that affectfrequency spectral shape, such as aperture size, transit time,and geometric broadening, sample volume length, andshape, the later being affected by depth (attenuation),acoustic impedance, and frequency; and factors related tothe examination technique by the technologist, such asexperience, accurate sample volume size, and three-dimen-sional placement in the vessel at the site of maximumstenosis, angle of insonation in relation to the velocityvector, which is not necessarily parallel to the vessel axis,and choice of color Doppler and gain settings.9-11

The literature contains a wide range of cutoff values fordiffering degrees of stenosis. However, there are limiteddata on the interobserver variation of duplex scanningparameters for ICA stenosis.12-16

MATERIAL AND METHODS

The study group consisted of 31 patients who con-sented to undergo color-coded duplex ultrasound scanningby the same three technologists. The indications for referralto our laboratory were suspected carotid artery occlusivedisease, follow-up study of existing stenosis, or postopera-tive surveillance. The technologists work in a tertiary uni-versity hospital in an ICAVL-accredited laboratory for ce-rebral vascular diagnosis. Their experience as vasculartechnologists ranges from 13 to 23 years. Each technolo-gist completes more than 300 duplex cerebrovascular ex-aminations per year.

Measurements and recording of results

The first technologist performed a complete bilateralcerebrovascular examination. Each patient was thenscanned sequentially by the other two technologists, whowere not aware of the results obtained by the other tech-nologists. The last two technologists were asked to performcarotid scanning on the side of the most severe carotidstenosis. The order in which the technologists performedthe examination was not randomized, but based on theiravailability.

A duplex scanning system (Acuson 128; MountainView, Calif) equipped with a 5.0-MHz to 7.5-MHz lineararray transducer was used in all examinations. The technol-ogists were instructed to examine the patients as in clinicalpractice. They were allowed to adjust parameters of themachine as necessary to ensure a natural measurementenvironment. The ICA was found in B-mode, and PSV wasrecorded from the Doppler spectrum at the site of maxi-mum stenosis or in the area where color change suggestedthe highest velocity. If no lesion was present in the ICA, a

Doppler measurement was made in the proximal ICA. TheDoppler waveform was obtained with an angle of in-sonation of 60 degrees or less. The location for the record-ing of the common carotid artery was not standardized, butwas at a normal site in the artery. Patients with occludedarteries were excluded from the study. End-diastolic andICA-CCA ratios were also measured.

Statistical analysis

Analysis of sources of variation. Two-way analysis ofvariance (ANOVA) was used. With this technique the totalvariation present in a set of data is partitioned into multiplecomponents. Associated with each of these components is aspecific source of variation, so that in the analysis it ispossible to ascertain the magnitude of the contributions ofeach of these sources to the total variation. Such a modelassumes that the patients are a random sample from thepopulation of all possible patients and that the technolo-gists are a random sample from the population of all possi-ble technologists. As such, we are interested in makinginferences about the population of patients or the popula-tion of technologists, rather than inferences about these 31patients or these three technologists. Our variance compo-nent model partitions the observed variability of a givenmeasurement into variability between patients, variabilitybetween technologists, and residual variability within asubject. Essentially, each measurement deviates from anoverall mean. The deviation can be decomposed into tech-nologist-specific deviation, patient-specific deviation, andwithin-patient deviation.

ANOVA will answer the question of how variable themeasurements between technologists are. Once the vari-ance components model has been fit to the data, thisquestion can be addressed by testing the hypothesis that thebetween-technologist variability is zero. The variance of theresidual term enables assessment of within-subject variationfor a given measurement. Analysis of this residual varianceenables us to see how variable the measurements withinpatients are. ANOVA will not enable comparison of ourmeasurers and detection of disagreement (bias) betweenthe three technologists.

Analysis of agreement for ICA PSV as a continuousvariable. The Altman-Bland method17,18 was used tocompare technologist agreement of ICA PSV as a contin-uous variable. With this method of analysis the main toolfor continuous analysis is the scatter plot. This plot repre-sents graphically the variability between two technologistsmeasuring ICA PSV in the same patient. The arithmeticdifference (y-axis) between the two measures of ICA PSV isplotted against the mean of the two measures (x-axis). Anordinary least-squares regression line is then plotted. If theordinary least-squares regression line fitted to the plot has aslope (b) that differs significantly from 0, then a propor-tional bias exists. Proportional bias would indicate that theagreement between observers is diminished as ICA PSVrises. Also, if the mean value for the differences (d) differssignificantly from 0 on the basis of a one-sample t test, thenthere is fixed bias. Fixed bias indicates that a technologist

JOURNAL OF VASCULAR SURGERYVolume 39, Number 4 Corriveau and Johnston 737

systematically either overestimates or underestimates thevelocity, as compared with the other technologist. It isimportant to point out that the mere absence of bias oftendoes not provide sufficient information to allow the state-ment that one technologist’s measurement can be usedinterchangeably with the others’. Bland and Altman sug-gest the calculation of the possible range of difference for agiven ICA PSV. They calculate the 95% limits of agree-ment, which is

d � tn � 1,2�Sd�(1�1/n)

where d is the mean difference between technologists, Sd isthe standard deviation of the difference between technolo-gists, tn � 1, 2� is the value of t corresponding to two-sidedP � .05 for df � n � 1, and �(1 � 1/n) is an adjustmentfor small sample size.19

When agreement is complete, all points would be onthe zero line of the y-axis, and the 95% limits of agreement

Fig 1. Variability of the internal carotid artery peak systolic veloc-ity among three technologists ICA, Internal carotid artery.

Fig 2. Difference against means for internal carotid artery peaksystolic velocity for technologists A and B after logarithmic trans-formation (Altman-Bland method). Black lines, Upper and lowerlevels of agreement; red dashed line, mean difference; blue dottedline, regression analysis. ICA, Internal carotid artery; PSV, peaksystolic velocity.

would be tight. The greater the spread from the zero line,the greater the discrepancy between the technologists inmeasuring ICA PSV in the same patient. If the differencesare not normally distributed, a log transformation is appliedbefore the limits of agreement are calculated.

Analysis of agreement for ICA PSV as a categoricalvariable. For categorical analysis, the weighted � statisticwas used to test for agreement among technologists. Thistest is designed to be used in cases in which the categoriesare ordered. It introduces the notion of a weighting factorthat allows for differences in the importance of disagree-ments. It is usually intended to exact a greater penalty forgreater degrees of disagreement. Three categories of ICAPSV that correspond to the cutoff values used in ourlaboratory to determine the severity of carotid stenosis wereused. A normal to minimal disease (�50%) scan is consid-ered to have a velocity of less than 140 cm/s, moderate

Fig 3. Difference against means for internal carotid artery peaksystolic velocity for technologists A and C after logarithmic trans-formation (Altman-Bland method). Black lines, Upper and lowerlevels of agreement; red dashed line, mean difference; blue dottedline, regression analysis. ICA, Internal carotid artery; PSV, peaksystolic velocity.

Fig 4. Difference against means for internal carotid artery peaksystolic velocity for technologists B and C after logarithmic trans-formation (Altman-Bland method). Black lines, Upper and lowerlevels of agreement; red dashed line, mean difference; blue dottedline, regression analysis. ICA, Internal carotid artery; PSV, peaksystolic velocity.


stenosis (50%-70%) has a velocity between 140 and 275cm/s, and severe stenosis (�70%) is documented when thevelocity is greater then 275 cm/s. A simple method forevaluating agreement for the � statistic was proposed byLandis and Koch20: � � 0.00, poor; � � 0.00 to 0.20,slight; � � 0.21 to 0.40, fair; � � 0.41 to 0.60, moderate;� � 0.61 to 0.80, substantial; � � 0.81 to 1.00, almostperfect agreement. The � statistic cannot detect bias. Othercategorical velocity ranges were also evaluated, and arepresented in Appendixes A and B, online only.

RESULTS

Technologists A, B, and C measured ICA PSV in 31patients. Fig 1 shows the ICA PSV recorded for eachpatient by the three technologists. For convenience, thepatients have been ordered by increasing average value ofPSV measured by the three technologists.

Sources of variability. As anticipated, ANOVA re-vealed that the greatest proportion of variability was due tothe patients, not the technologists (P � .005; Table I).Only 0.58% of the total variability in measurement of ICAPSV can be attributed to the technologists. Most of thevariability (97.18%) is attributed to the differing patients.

Variability of ICA PSV measurements as a contin-uous variable. Analysis of the differences between theICA PSV measured by the technologists revealed the ab-sence of a normal distribution. Therefore a logarithmic

Table I. ICA PSV*

Source variance Sum of squares df

Technologists 434.34 2Patient 1,087,674.00 30Error (residual) 50,195.66 60Total 1,138,304.00 92

F distribution, � � 0.005; F � 5.79.ICA, Internal carotid artery; PSV, peak systolic velocity; df, degrees of freed*Analysis of variance.

Table II. ICA PSV analysis of limits of agreement

95% Limits of agreement

Technologist A vs B �0.127-0.156Technologist A vs C �0.136-0.154Technologist B vs C �0.135-0.118

ICA, Internal carotid artery; PSV, peak systolic velocity.

Table III. ICA PSV analysis of bias: Fixed and proportion

Mean difference

Technologist A vs B 0.014Technologist A vs C 0.009Technologist B vs C �0.008

ICA, Internal carotid artery; PSV, peak systolic velocity; CI, confidence int

transformation was applied before analysis, using theAltman-Bland method. Figs 2, 3, and 4 show scatter plotsafter logarithmic transformation for technologists A versusB, A versus C, and B versus C. From these plots it can beseen that there is substantial interobserver variation. Weobserved a significant dispersion on the zero line of they-axis. This significant spread from the zero line indicates alarge discrepancy among the measurements recorded bythe technologists. The 95% limits of agreement are shownas straight upper and lower lines. Table II presents the 95%limits of agreement with its anti-log transformation. Theanti-log of the difference between two values on a log scaleis a dimensionless ratio. This limit shows that for 95% ofcases the measurement of ICA PSV will differ by thisanti-log interval. For example, this would mean that themeasurements of technologist B might differ by 27% belowto 43% above the measurements taken by technologist A.This anti-log transformation allows us to view the possiblerange of differences for a given ICA PSV measurement in apercentage form. The mean difference between the observ-ers is shown as a dashed straight line running parallel to thex-axis, close to zero. A regression line (dotted line) was alsofitted to these plots.

Table III shows the mean difference with its 95%confidence interval. For each pair of technologists theinterval does not differ significantly from 0. Table III alsoreveals the slope of the regression analysis with its 95%

riance component (cm/s)2

SD(cm/s)

Varianceration %

217.17 0.58 14.74 0.259255.82 97.18 190.41836.59 2.24 28.92

Anti-log of limits Percentage (range)

0.75-1.43 25% below–43% above0.73-1.43 27% below–43% above0.73-1.31 27% below–31% above

% CI mean Slope 95% CI slope

.100-0.038 �0.047 �0.180-0.021

.016-0.034 �0.007 �0.146-0.132

.029-0.013 0.026 �0.093-0.145

Va

36

om.

al

95

�0�0�0

erval.


confidence interval. For each pair of technologists theinterval does not significantly differ from zero. That theconfidence intervals for the mean and the slope do notdiffer significantly from zero allows us to conclude thatthere is no fixed or proportional bias, respectively. Theabsence of fixed bias indicates that a technologist does notsystematically either overestimate or underestimate theICA PSV as compared with the other technologists. Thatno proportional bias was found indicates that the level ofagreement is maintained between the technologists as ICAPSV rises.

Variability of ICA PSV measurements as a categor-ical variable. The data for categorical analysis are shownin Tables IV through VI. For the three categories of ICAPSV, technologists A and B agreed in 30 of 31 (97%)measurements, technologists A and C agreed in 28 of 31(90%) measurements, and technologists B and C agreedin 28 of 31 (90%) measurements. In these cross-tables itcan be seen that there is substantial agreement betweentechnologists regarding the measurement of stenosis inall three categories. The � values for the three possiblepairs of technologists are shown in Table VII. There isalmost perfect agreement between each pair oftechnologists.

For further analysis of ICA PSV as a categorical vari-able, we used other cutoff values that have been suggestedin the literature for imaging equipment similar to ours.21,22

We found almost perfect agreement between our technol-ogists when other cutoff value points were used for analyz-ing the severity of carotid stenosis (Appendix A, TablesVIII to XI, online only).

We also analyzed the ICA EDV as a continuous andcategorical variable. Our results show that 9.75% of thetotal variability can be attributed to the technologists (Ap-

Table IV. Cross-tabulations of ICA PSV (cm/s) fortechnologists A and B

Technologist A

Technologist B �140 140-275 �275 Total�140 6 6

140-275 15 1 16�275 9 9Total 6 15 10 31


Table V. Cross-tabulations of ICA PSV (cm/s) fortechnologists A and C

Technologist A

Technologist C �140 140-275 �275 Total�140 6 2 8

140-275 13 1 14�275 9 9Total 6 15 10 31

pendix B, Table XII, online only). The agreement on acontinuous scale revealed even greater interobserver vari-ability then observed for ICA PSV (Appendix B, TableXIII, online only). We also found fixed bias between tech-nologists A versus C and B versus C. Technologists A and Bsystematically overestimated the measurements made bytechnologist C. No proportional bias was found (AppendixB, Table XIV, online only). In our categorical analysis wedid not observe substantial agreement in all three categories(Appendix B, Tables XV to XVIII, online only).
DISCUSSION

Because of the important role of the measurements ofPSV with duplex scanning, it is important that the repro-ducibility of duplex parameters from different technologistsbe clinically reliable. We have minimized the potentialsources of error not attributable to the observer by usingone single anatomic site of measurement and by using thesame machine for every patient. ANOVA revealed that themost significant portion of variability was caused by thediffering patients and not by the technologists.

Agreement between technologists for ICA PSV ascontinuous variable. The Altman-Bland analysis permitsanalysis of bias and limits of agreement between technolo-gists. We were unable to find any systematic differencebetween the technologists. In other words, none of thetechnologists consistently recorded values that were higheror lower than those of the others. This level of agreementalso did not depend on the value of the ICA PSV. To ourknowledge, we are the first to describe the notion ofproportional or fixed bias in the analysis of agreementbetween technologists for cerebrovascular disease. Al-though others have described the presence of proportionalbias when evaluating interobserver variability on femoro-popliteal duplex scans,23,24 our results clearly show theabsence of such bias. Fig 1 suggests a trend toward propor-

Table VI. Cross-tabulations of ICA PSV (cm/s) fortechnologists B and C

Technologist B

Technologist C �140 140-275 �275 Total�140 6 2 8

140-275 14 1 15�275 8 8Total 6 16 9 31


Table VII. �-weighted statistic

� SD CI

Technologist A vs B 0.948 0.050 0.098Technologist A vs C 0.848 0.083 0.163Technologist B vs C 0.845 0.085 0.166

CI, Confidence interval.


tional bias when the ICA PSV exceeds 270 cm/s; however,this was not statistically significant. It is possible that thenumber of patients with severe stenosis was too small toenable us to confirm proportional bias. This proportionalbias could be attributed to the presence of a poststenoticturbulent flow jet. The site of maximal spectral broadeningis located beyond the stenosis where the high-speed jetspreads out over a distance of about 1 cm. It is possible thatthe recording of maximal ICA PSV was taken distal to thestenosis in the poststenotic turbulent flow, thereby under-estimating the velocity. This phenomenon probably ex-plains to a large extent the interobserver variation in mea-suring the PSV found with increasing stenosis.

The limits of agreement between the measurementsmade by these three experienced technologists are unac-ceptably wide. This high interobserver variation makes theinterchangeability of our technologists’ measurementsproblematic for clinical use. There are four possible expla-nations for these wide intervals. First, the group sample sizemay be too small, and the measurement of the ICA PSVmight be difficult to investigate with this method. Second,because both the minimal and maximal values are far re-moved from zero, it is almost inevitable that the analysis isattended by a wide interval. This interval can be narrowedonly by studying a much larger sample. A third explanationwould be fluctuation of the patient’s blood pressure be-tween measurements. The blood pressure was not deter-mined before or after each examination. It is possible thatvariation in blood pressure between each examination wasnot recognized. A fourth explanation may be the absence ofmultiple measurements for each technologist for each pa-tient. Without this information we are unable to assess thevariability between measurements for a given technologistand hence have a better estimation of the true measurementfor each technologist.

Agreement between technologists for ICA PSV as acategorical variable. The weighted � statistic provided amethod for comparing the technologists when the ratingscale was categorical and ordered. The � values for all threepossible sets of technologists represent almost perfectagreement. Because our categorical analysis included nor-mal duplex scans, it is possible that our � value might beoverinflated. However, if we exclude the normal scans, westill obtain near perfect agreement in patients with moder-ate or severe stenosis.

Although we have found almost perfect agreementbetween technologists in the analysis of ICA PSV as acategorical variable, we were somewhat disappointed withthe wide limits of agreement in the analysis of ICA PSV asa continuous variable. A tight limit of agreement would bequite helpful when velocity measurements are in the vicinityof cutoff values, and would be necessary to accuratelydetermine whether there is progression of severity of ste-nosis. A simple method of enhancing the reliability of themeasurements taken would be to scan the suspected site ofstenosis a second time by a second technologist. This isfounded on the principle that repeated measurements are ameans of improving the accuracy of a measurement. How-

ever, our wide limits of agreement do not permit us toconclude that repeated measurements of the ICA PSV bydifferent technologists would improve its accuracy. There-fore stenosis with a velocity in the vicinity of a cutoff valuemay benefit from additional diagnostic testing, because ofthe considerable interobserver variation in ICA PSV as acontinuous variable.

Our study attempted to minimize the potential sourcesof error that are not attributable to the observer by usingone single anatomic site of measurement and by using thesame machine for every patient. However, multiplessources of error regarding duplex scanning equipment andvessel anatomy may have contributed to the wide limits ofagreement in the analysis of ICA PSV on a continuousscale. The transducer beam pattern alters maximum veloc-ity measurements by �3% to 61%.25 Errors in measurementcan also arise from the variables in the Doppler scanningequation. It is assumed that the speed of the ultrasoundbeam is constant throughout the different tissue interfacesand that no refraction occurs. Tissue refraction does occur,and will be a source of angle error. The assumption of aconstant speed of sound and no refraction will result in an8% error in measurement of maximal velocity.8 Samplevolume depends on depth (attenuation), size, and three-dimensional positioning in the artery. The sample volumeshape is highly asymmetric and frequency-dependent,which will contribute to measurement error.

From this study we can conclude that in the analysis ofthe sources of variation in the measurement of the ICA PSVonly 0.58% can be attributed to the technologists. Al-though the total variation attributed to the technologists issmall, analysis of ICA PSV as a continuous variable revealedlimits of agreement that were too wide for clinical use. Thusthe interchangeability of ICA PSV as a continuous variableis not possible, because of the wide percentage changeinterval between technologists, for clinical acceptance. Al-though no maximum discrepancy between observers hasbeen officially recorded in the literature, no proportional orfixed bias was observed between our experienced technol-ogists. We were able to quantify the magnitude of theproblem in reporting PSV as a continuous variable. We canassume that a small difference in sequential examinationsdoes not signify disease progression. Our study design doesnot enable us to determine what magnitude of change inthe PSV represents disease progression. When categoricalanalysis was applied to the ICA PSV, almost perfect agree-ment was obtained between technologists, and conse-quently the interpretation of the technologists can be usedinterchangeably. Additional diagnostic testing may be ofvalue for stenosis with velocity in the vicinity of cutoffvalues, to minimize inaccurate clinical decision-making.

The extensive experience of our vascular technologistsmay represent another potential limitation of our study.Their experience may not reflect the situation existing inmost vascular laboratories. Therefore our results may notbe applicable to every vascular laboratory.

In future work a carotid flow model designed to repro-duce carotid artery anatomy, stenosis, depth, tortuosity,


tissue, and vessel acoustic impedance will be necessary tobetter isolate the variance attributable to the observer.Repeated measures for each observer will help to determinewhat magnitude of change in PSV is to be considereddisease progression. Several duplex scanning machine set-tings would need to be standardized to isolate the varioussources of error. Such settings would include transduceraperture size, focal depth, beam steering, gain, and samplevolume.

We thank Professor Mary Chipman, Department ofPublic Health Sciences, Faculty of Medicine, University ofToronto, for help with the statistical analysis, and SueUngaro, RN RVT, Helen Lypka, BSc RVT, and Cris Rossi,BSc RVT, for help in data collection.

REFERENCES

1. Moneta GL, Edwards JM, Papanicolaou G, Hatsukami T, Taylor LM Jr,Strandness DE Jr, et al. Screening for asymptomatic internal carotidartery stenosis: duplex criteria for discriminating 60% to 99% stenosis. JVasc Surg 1995;21:989-94.

2. Alexandrov AV, Brodie DS, McLean A, Hamilton P, Murphy J, BurnsPN. Correlation of peak systolic velocity and angiographic measure-ment of carotid stenosis revisited. Stroke 1997;28:339-42.

3. Carpenter JP, Lexa FJ, Davis JT. Determination of duplex Dopplerultrasound criteria appropriate to the North American SymptomaticCarotid Endarterectomy Trial. Stroke 1996;27:695-9.

4. Moneta GL, Edwards JM, Chitwood RW, Taylor LM Jr, Lee RW,Cummings CA, et al. Correlation of North American SymptomaticCarotid Endarterectomy Trial (NASCET) angiographic definition of70% to 99% internal carotid artery stenosis with duplex scanning. J VascSurg 1993;17:152-7.

5. Faught WE, Mattos MA, van Bemmelen PS, Hodgson KJ, BarkmeierLD, Ramsey DE, et al. Color-flow duplex scanning of carotid arteries:new velocity criteria based on receiver operator characteristic analysis forthreshold stenoses used in the symptomatic and asymptomatic carotidtrials. J Vasc Surg 1994;19:818-27.

6. Pan XM, Saloner D, Reilly LM, Bowersox JC, Murray SP, AndersonCM, et al. Assessment of carotid artery stenosis by ultrasonography,conventional angiography, and magnetic resonance angiography: cor-relation with ex vivo measurement of plaque stenosis. J Vasc Surg1995;21:82-8.

7. Young GR, Humphrey PR, Shaw MD, Nixon TE, Smith ET. Compar-ison of magnetic resonance angiography, duplex ultrasound, and digitalsubtraction angiography in assessment of extracranial internal carotidartery stenosis. J Neurol Neurosurg Psychiatry 1994;57:1466-78.

8. Christopher DA, Burns PN, Hunt JW, Foster FS. The effect of refrac-tion and assumed speeds of sound in tissue and blood on Dopplerultrasound blood velocity measurements. Ultrasound Med Biol 1995;21:187-201.

9. Hoskins PR. Accuracy of maximum velocity estimates made using
Doppler ultrasound systems. Br J Radiol 1996;69:172-7.
10. Howard G, Baker WH, Chambless LE, Howard VJ, Jones AM, TooleJF. An approach for the use of Doppler ultrasound as a screening toolfor hemodynamically significant stenosis (despite heterogeneity ofDoppler performance): a multicenter experience. Asymptomatic Ca-rotid Atherosclerosis Study Investigators. Stroke 1996;27:1951-7.

11. Steinman AH, Tavakkoli J, Myers JG Jr, Cobbold RS, Johnston KW.Sources of error in maximum velocity estimation using linear phased-array Doppler systems with steady flow. Ultrasound Med Biol 2001;27:655-64.

12. Kohler T, Langlois Y, Roederer GO, Phillips DJ, Beach KW, PrimozichJ, et al. Sources of variability in carotid duplex examination: a prospec-tive study. Ultrasound Med Biol 1985;11:571-6.

13. Kohler TR, Langlois Y, Roederer GO, Phillips DJ, Beach KW, Pri-mozich J, et al. Variability in measurement of specific parameters forcarotid duplex examination. Ultrasound Med Biol 1987;13:637-42.

14. Mikkonen RH, Kreula JM, Virkkunen PJ. Reproducibility of Dopplerultrasound measurements. Acta Radiol 1996;37:545-50.

15. Polak JF, Dobkin GR, O’Leary DH, Wang AM, Cutler SS. Internalcarotid artery stenosis: accuracy and reproducibility of color-Doppler-assisted duplex imaging. Radiology 1989;173:793-8.

16. Tessler FN, Kimme-Smith C, Sutherland ML, Schiller VL, Perrella RR,Grant EG. Inter- and intra-observer variability of Doppler peak velocitymeasurements: an in-vitro study. Ultrasound Med Biol 1990;16:653-7.

17. Bland JM, Altman DG. Comparing methods of measurement: whyplotting difference against standard method is misleading. Lancet 1995;346:1085-7.

18. Bland JM, Altman DG. Statistical methods for assessing agreementbetween two methods of clinical measurement. Lancet 1986;1:307-10.

19. Ludbrook J. Statistical techniques for comparing measurers and meth-ods of measurement: a critical review. Clin Exp Pharmacol Physiol2002;29:527-36.

20. Landis JR, Koch GG. The measurement of observer agreement forcategorical data. Biometrics 1977;33:159-74.

21. Filis KA, Arko FR, Johnson BL, Pipinos II, Harris EJ, Olcott C, et al.Duplex ultrasound criteria for defining the severity of carotid stenosis.Ann Vasc Surg 2002;16:413-21.

22. Kuntz KM, Polak JF, Whittemore AD, Skillman JJ, Kent KC. Duplexultrasound criteria for the identification of carotid stenosis should belaboratory specific. Stroke 1997;28:597-602.

23. Ubbink DT, Fidler M, Legemate DA. Interobserver variability in aor-toiliac and femoropopliteal duplex scanning. J Vasc Surg 2001;33:540-5.

24. Winter-Warnars HA, van der Graaf Y, Mali WP. Interobserver variationin duplex sonographic scanning in the femoropopliteal tract. J Ultra-sound Med 1996;15:421-8.

25. Daigle RJ, Stavros TA, Lee RM. Overestimation of velocity and fre-quency values by multielement linear array Dopplers. J Vasc Technol1990;14:206-12.

Submitted Sep 10, 2003; accepted Dec 2, 2003.Available online Feb 24, 2004.

Additional material for this article may be found online
at www.mosby.com/jvs.

interobserver variability of carotid doppler peak velocity measurements among technologists in an...

Documents