Radiotherapy and Oncology 107 (2013) 6–12

Contents lists available at SciVerse ScienceDirect

Radiotherapy and Oncology

journal homepage: www.thegreenjournal .com

Image guided brachytherapy

Uncertainties of target volume delineation in MRI guided adaptive brachytherapyof cervix cancer: A multi-institutional study

Primoz Petric a,b,⇑, Robert Hudej b, Peter Rogelj c, Mateja Blas d, Kari Tanderup e,f, Elena Fidarova g,h,Christian Kirisits h,i, Daniel Berger h, Johannes Carl Athanasios Dimopoulos j, Richard Pötter h,i,Taran Paulsen Hellebust k,l,m

a Radiation Oncology Department, National Center for Cancer Care and Research, Doha, Qatar; b Division of Radiotherapy, Institute of Oncology Ljubljana, Zaloška c. 2; c University ofPrimorska, Faculty of Mathematics, Natural Sciences and Information Technologies, Koper; d Division of Research and Education, Institute of Oncology Ljubljana, Slovenia;e Department of Oncology, Aarhus University Hospital; f Institute of Clinical Medicine, Aarhus University, Denmark; g Division of Human Health, International Atomic Agency, Vienna;h Department of Radiotherapy and Radiobiology, Medical University of Vienna, Austria; i Christian Doppler Laboratory for Medical Radiation Research for Radiation Oncology, MedicalUniversity of Vienna, Austria; j Department of Radiotherapy, Metropolitan Hospital, Athens, Greece; k Department of Medical Physics, Oslo University Hospital, The Radium Hospital;l Department for Radiation Protection and Nuclear Safety, Norwegian Radiation Protection Authority, Østerås; m Department of Physics, University of Oslo, Norway

a r t i c l e i n f o

Article history:Received 2 October 2012Received in revised form 16 January 2013Accepted 29 January 2013Available online 27 February 2013

Keywords:Cervix cancerBrachytherapyMRIDelineation uncertainties

0167-8140/$ - see front matter � 2013 Elsevier Irelanhttp://dx.doi.org/10.1016/j.radonc.2013.01.014

⇑ Corresponding author.E-mail address: ppetric@hmc.org.qa (P. Petric).

a b s t r a c t

Background and aim: We aimed to quantify target volume delineation uncertainties in cervix cancerimage guided adaptive brachytherapy (IGABT).Materials and methods: Ten radiation oncologists delineated gross tumour volume (GTV), high- and

intermediate-risk clinical target volume (HR CTV, IR CTV) in six patients. Their contours were comparedwith two reference delineations (STAPLE-Simultaneous Truth and Performance Level Estimation andEC- expert consensus) by calculating volumetric and planar conformity index (VCI and PCI) and inter-delineation distances (IDD).Results: VCISTAPLE and VCIEC were 0.76 and 0.72 for HR CTV, 0.77 and 0.68 for IR CTV and 0.59 and 0.58 forGTV. Variation was most prominent caudally and cranially in all target volumes and posterolaterally in IRCTV. IDDSTAPLE and IDDEC for HR CTV (3.6 ± 3.5 and 3.8 ± 3.4 mm) were significantly lower than for GTV(4.8 ± 4.2 and 4.2 ± 3.5 mm) and IR CTV (4.7 ± 5.2 and 5.2 ± 5.6 mm) (p < 0.05).Conclusions: Due to lower delineation uncertainties when compared to GTV and IR CTV, HR CTV may beconsidered most robust volume for dose prescription and optimization in cervix cancer IGABT. Adequateimaging, training and use of contouring recommendations are main strategies to minimize delineationuncertainties.

� 2013 Elsevier Ireland Ltd. All rights reserved. Radiotherapy and Oncology 107 (2013) 6–12

Main advantages of 3D image guided adaptive brachytherapy(IGABT) over conventional techniques include the ability for indi-vidualized treatment tailoring and analysis of dose volume histo-gram (DVH) parameters. Ideally, the optimized treatment plandelivers sufficient dose to delineated target volume, while avoidingover-dosage of the organs at risk (OAR). Repetitive clinical andimaging assessment at subsequent BT applications enable the doseadaptation to a target which changes in shape and volume duringtreatment, reflecting the evolution of 3D into 4D IGABT concept.The dose that is delivered to the tissues therefore depends on thedelineated regions of interest. Recording of the DVH parametersand their correlation with the probability of tumour control andmorbidity are a basis for treatment comparisons and developmentof clinical outcome models, radiobiological parameter estimatesand new treatment protocols. The value of these correlations

d Ltd. All rights reserved.

depends on reliability of the DVH parameters and consequentlyon uncertainties introduced during treatment planning anddelivery. Therefore, selection and accurate delineation of the targetvolumes and organs at risk is a precondition for the success ofIGABT. It may represent a major source of uncertainties [1–4] withimpact on clinical outcome, treatment comparisons and interpre-tation of the results of clinical studies.

Numerous reports on delineation variability, mainly dealingwith external beam radiotherapy (EBRT) of various tumour sites,have been published [5–19]. There are a few reports on delineationuncertainties in cervix cancer radiotherapy [4,20–27]. We assessedthe magnitude and topography of uncertainties in target volumedelineation in cervix cancer IGABT.

Material and methods

This study is based on a prospective protocol, developed in theframe of EMBRACE (an intErnational study on MRI-guided

P. Petric et al. / Radiotherapy and Oncology 107 (2013) 6–12 7

BRachytherapy in locally advanced CErvical cancer, www.embracestudy.dk) study. The protocol was reviewed, adapted andapproved at the meetings of the gynaecological (Gyn) GEC ESTROworking group (WG) and EMBRACE study group between March2009 and December 2010.

Study cases and treatment

Six cervix cancer patients, treated with MRI-based IGABT at theInstitute of Oncology Ljubljana (cases L1–L3) and the Medical Uni-versity of Vienna (cases V1–V3), were included. Tumour character-istics at diagnosis and BT are listed in Table 1. Cases were selectedattempting to reflect common situations with locally advanced tu-mours, extending beyond the cervix and with certain characteris-tics important for contouring, such as parametrial and uterineinfiltration. In addition to FIGO requirements, the initial work-upincluded pelvic MRI and a comprehensive documentation on aschematic cartoon. Following EBRT (45–50.4 Gy in 25–28 frac-tions ± concomitant chemotherapy), two insertions of a tandem-ring applicator with interstitial needles were performed. Post-insertion MRI was obtained at a 1.5 T scanner (Magnetom Avanto,Siemens, Erlangen, Germany) in Ljubljana [20,28] and a 0.2 T scan-ner (Magnetom Open-Viva, Siemens, Erlangen, Germany) in Vienna[29]. The first BT insertion was used for this study.

Test delineations

Clinical findings and pelvic MRI at diagnosis and BT of all caseswere distributed to twelve institutions that successfully partici-pated in the EMBRACE study. At each institution, an experiencedradiation oncologist was invited to independently delineate thegross tumour volume (GTV), high risk clinical target volume (HRCTV) and intermediate risk clinical target volume (IR CTV),respecting the GEC ESTRO recommendations [30]. Ten institutionsresponded, submitting 180 sets of test delineations (10 observ-ers � 6 cases � 3 target volumes) to the study office.

Reference delineations

Facing the difficulty of the definition of the ‘‘underlying truth’’or the ‘‘correct delineation’’, two strategies were pursued to createthe reference outlines. (1) An expectation–maximization algorithmfor Simultaneous Truth and Performance Level Estimation (STA-PLE) was used to compute the probabilistic estimate of the truedelineations from the collection of delineations from all observers[31]. (2) Expert Consensus (EC) delineations were created by fourexperts during a joint discussion at a Gyn GEC ESTRO Wg meeting(Aarhus, Denmark; June 2011), taking into account the delinea-tions from observers. The EC delineations were created to elimi-nate eventual inconsistencies of STAPLE contours with GECESTRO recommendations.

Table 1Tumour characteristics of individual cases at diagnosis and brachytherapy. W/T/H, width/

Case Stage and size Infiltration at diagnosis

FIGO W/T/H [mm] PM Vagina

L1 2b 65/50/65 L + R NoL2 2b 38/37/43 L YesL3 3b 55/43/60 L + R YesV1 3b 85/40/45 L + R YesV2 2b 56/50/35 L + R NoV3 3b 55/40/55 L + R Yes

* Response: MRI-based ratio of initial tumour volume to the volume of residual GTbrachytherapy.

Analysis of delineation uncertainties

Using dedicated software, the deviations of the test from thereference contours were assessed. The volumetric conformity in-dex (VCI: the ratio between the common and encompassing vol-ume) is a commonly used measure of overlap [2,32]. WithSTAPLE and EC delineations as the reference, the VCISTAPLE andVCIEC, respectively, were obtained for each target volume by aver-aging the VCI of pairs between the reference and each test delinea-tion. In addition, planar conformity index (PCISTAPLE and PCIEC) wascalculated as the ratio between the common and encompassingsurface of the test and reference delineation on each image sliceand presented as a function of the slice number for each targetvolume.

In spatial analysis of variation, the procedure, described below,was carried out (Fig. 1). On each (para) transverse image (the con-touring plane), the centre of mass was calculated for the referencedelineation. Using the centre of mass as the origin, a coordinatesystem was defined and divided into four angular sectors (anterior:300–60�; left; 60–120�; posterior: 120–240� and right: 240–300�)(Fig. 1). The shortest inter-delineation distance (IDDSTAPLE, IDDEC)from each reference delineation point to the individual test delin-eation was calculated in 72 angular steps of 5 degrees. The IDDswere averaged over all image slices for each observer and studycase, using the absolute values. The mean IDD of each target vol-ume was presented graphically as a function of the angle. The IDDsof the HR CTV, GTV and IR CTV were compared within each angularsector and over all sectors.

Statistical analysis

Continuous numerical variables were presented asmeans, standard deviations (SD) and relative SD (SD/mean).Analysis of IDDs was performed using the linear mixed effectmodel [33,34]. Observers and cases were included into the mod-el as random effects to take the differences among them into ac-count. Target volumes, angular sectors and interaction betweenthe two were included as fixed factors. The lme4 and multcomppackages in the R statistical software (R DevelopmentCore Team, 2011) were used to perform the analysis. Thep-values (two sided) under 0.05 were considered statisticallysignificant.

Results

Descriptive statistics of the VCI and IDD results are presented inTable 2. HR CTV demonstrated most favourable VCI and signifi-cantly lower overall IDDs (p < 0.05) when compared with GTVand IR CTV for both reference delineations (Table 2, Fig. 2). In allstudy cases, lowest IDDs were obtained for the HR CTV. When com-pared with the overall IDDs (Table 2), highest individual IDD valuesfor the GTV, HR CTV and IR CTV were found in cases V1 (IDDSTAPLE:

thickness/height; PM, parametrium; L, left; R, right; BT, brachytherapy.

Residual pathological tissues at BT

Uterus Response* PM Vagina Uterus

No <50% L + R No NoYes >50% L Yes NoYes >50% L + R No NoNo <50% L + R Yes NoNo >50% R No NoYes <50% R Yes Yes

V, the cervix and the residual pathological tissues (the grey zones) at time of

Fig. 1. The coordinate system for spatial assessment of inter-delineation distancesis projected on a single MRI slice containing an example of GTV, HR CTV and IR CTVdelineations from all observers. Expert consensus delineations are presented asdotted lines. Four angular sectors of analysis are shown in colours. Blue: anterior(300–60�); green: left (60–120�); red: posterior (120–240�) and yellow: right (240–300�). Full circle: centre of mass of the reference delineation.

8 Contouring uncertainties in cervix cancer brachytherapy

6.3 ± 5 mm, 4.1 ± 4.5 mm and 5 ± 5.6 mm; IDDEC: 4.9 ± 4.1 mm,4.6 ± 4.4 mm, respectively) and V2 (IDDSTAPLE: 5.8 ± 4.7 mm,4.2 ± 3.5 mm and 5.5 ± 4.9 mm; IDDEC: 4.6 ± 3.1 mm, 4.2 ±3.6 mm and 6.8 ± 6.2 mm, respectively). When compared withGTV and IR CTV, minimal angle-dependent variation of IDD waspresent for the HR CTV (Fig. 2). With STAPLE contour as the refer-ence, the IDD for the GTV was significantly higher than for the HRCTV in all angular sectors (anterior and right: p < 0.0001, posterior:p = 0.005, left: p = 0.026). With EC contour as the reference, it washigher anteriorly and right (p < 0.001 in both sectors) (Fig. 2). TheIDD for the IR CTV was significantly higher than for the HRCTV in all but the anterior sector both for the STAPLE

(posterior and left: p < 0.0001, right: p < 0.002) and EC (posterior,left and right: p < 0.0001) (Fig. 2). The results on the PCI are pre-sented in Fig. 3. For all target volumes, the variations were mostprominent caudally and cranially, while agreement was high inthe middle of the target volumes. The lowest PCI was found forthe GTV (Fig. 3).

Discussion

In the era of high-precision IGABT, the importance of delinea-tion accuracy cannot be overemphasized. Our results offer an esti-mate of the magnitude of delineation uncertainties, indicating thatthey may represent one of the weakest links of cervix cancer IGABTchain. In the current study, HR CTV had less delineation uncertain-ties than GTV and IR CTV.

Imaging modality

While various imaging modalities have been employed inIGABT, only MRI was used in our study. Cervix cancer IGABT ex-ploits an adaptive target concept. The adaptation is based on clin-ical and imaging interpretation of initial tumour spread andresidual pathological tissues after tumour shrinkage duringradio-chemotherapy [29,30,35]. The validity of this concept haslimited clinico-pathological proof and may involve more signifi-cant operator-related uncertainties than the traditional radiother-apy target concepts, based on pre-therapeutic status. MRIdemonstrates excellent soft tissue contrast and has been recom-mended for cervix cancer IGABT [29,30,36–39]. However, due tolimited availability of MRI, CT-based techniques gained an increas-ing role during past years [40–44] and the use of ultrasound hasbeen reported [45]. However, further research is required to vali-date the potential of different imaging modalities, including thefunctional imaging in IGABT.

Volumetric concordance

Direct comparisons of published studies [5–9,12,18–23,46] withour VCI results are challenging for several reasons. First, the use ofcontouring recommendations was inconsistent and various tumoursites and imaging modalities were evaluated in these studies. Var-ious formulations for the volumetric concordance were applied anddifferent number of observers and cases analysed [2]. Recently,CIgen, an index that is independent from the number of observerswas suggested when comparing more than two delineations [32].In the current study, deviations from the reference volumes are de-scribed. Consequently, the overlap between individual pairs of testand reference delineations was calculated. Furthermore, contouringdeviation of a certain magnitude (in mm) will result in more favour-able VCI when analysing large (common in EBRT), compared tosmall (common in BT) volumes. Taking these shortcomings of com-parisons into account, our results (Table 2) may be considered sim-ilar to studies with comparable target volume sizes, reportingconformity indices of around 0.6–0.7 [7,8,12,18,47]. There are fewreports on contouring uncertainties in MRI-based cervix cancerIGABT, demonstrating VCI of 0.6–0.8 for the HR CTV and 0.6–0.7for the GTV and IR CTV [20–22]. Due to the differences in GTV, HRCTV and IR CTV sizes, comparisons between VCIs of these volumesmay be misleading. The sensitivity of VCI to contouring variation ishighest for the GTV, followed by HR CTV and IR CTV. Accordingly,while IDD for the IR CTV was significantly larger when comparedto the HR CTV, this was not reflected by the VCI results (Table 2).

Spatial distribution of uncertainties

BT is characterized by an inhomogeneous dose distribution andvarying steepness of dose-gradient in different directions from the

Fig. 2. Mean inter-delineation distance (IDD) curves of the GTV, HR CTV and IR CTVas a function of the angle. Angular sectors are represented by coloured regions (cf.Fig. 1). (A) Simultaneous truth and performance level estimation (STAPLE) and (B)Expert consenus (EC) delineations were used as the reference.

Table 2Volumes (V), Volumetric conformity indices (VCI) and overall inter-delineation distances (IDD) for the GTV, HR CTV and IR CTV. IDD for HR CTV was significantly lower (p < 0.05)when compared with GTV and IR CTV. SD, standard deviation; rSD, relative standard deviation; STAPLE, Simultaneous Truth and Performance Level Estimation delineations; EC,expert consensus delineations.

Parameter GTV HR CTV IR CTV

Mean (SD) rSD [%] Mean (SD) rSD [%] Mean (SD) rSD [%]

VOBSERVERS [cm3] 20 (16) 80 52 (22) 42 130 (45) 34VSTAPLE [cm3] 20 (16) 80 56 (24) 43 147 (44) 30VEC [cm3] 20 (13) 65 48 (22) 46 113 (37) 33VCISTAPLE 0.59 (0.2) 34 0.76 (0.07) 9 0.77 (0.06) 8VCIEC 0.58 (0.15) 26 0.72 (0.08) 11 0.68 (0.07) 10IDDSTAPLE [mm] 4.8 (4.2) 88 3.6 (3.5) 97 4.7 (5.2) 110IDDEC [mm] 4.2 (3.5) 83 3.8 (3.4) 89 5.2 (5.6) 108

P. Petric et al. / Radiotherapy and Oncology 107 (2013) 6–12 9

applicator. Consequently, the impact of delineation uncertaintieson DVH parameters depends on their magnitude and location[48]. In this study the IDD was calculated as a function of para-transversal direction and will therefore indicate the magnitude ofdelineation uncertainty for different angles. However, it is shouldbe pointed out that since absolute numbers were used the meanIDD cannot be used to estimate systematic effect of the delineationuncertainties.

Our results on the IDD in the contouring plane revealed small-est deviations and minimal angle-dependent variation for the HRCTV (Table 2, Fig. 2). There are no comparable studies for the GTVor IR CTV. Reported uncertainties for the HR CTV are smallerthan in our study with mean IDDs of around 2–3 mm (st. dev.1–2 mm) [20,22]. Two observers with common institutional back-grounds participated in these studies. In contrast, 10 observersfrom different institutions with different experience participatedin the present analysis, which may explain our less favourableresults.

We found most prominent uncertainties at caudal and cranialparts of the target volumes (Fig. 3). This may be attributed to ob-server-specific interpretations of MRI findings due to the partial-volume effects in the vicinity of the ring, fluid collections in thevaginal packing and challenging interpretation of pathologicaltissues in the uterine corpus. However, even marked uncertainties

in the high-dose region of the ring will have small effect on thecommonly reported DVH parameters. Loading of the applicator,i.e. along the tandem above the cranial and below the caudal ex-tent of the target volume (following the tradition of conventionalX-ray based brachytherapy) could be advised as long as the OARdose constraints are not compromised. By applying such regionsof non-conformity, clinical consequences of delineation uncertain-ties in these parts may be avoided. In contrast, translation of con-touring uncertainties to a lateral non-conformity region around theCTV is discouraged.

In the contouring plane, the IR CTV delineation uncertaintieswere most pronounced posterolaterally, corresponding to the re-gions of the parametria and sacrouterine ligaments. In contrast,the deviations were smallest anteriorly, corresponding to the direc-tion of the bladder (Figs. 1 and 2). The IR CTV concept (as opposed toGTV and HR CTV) is not related only to visible pathology at time of BT(30, 35). Integration of initial clinical and radiological findings andregression during treatment is required during its delineation – aprocess which is prone to subjective interpretations. This may ex-plain the more pronounced uncertainties in the region of parametriaand sacrouterine ligaments – major routes of initial tumour spread.

To localize the regions of BT target volume with highest dosi-metric impact of delineation uncertainties, the concept of spatialdosimetric sensitivity analysis was proposed by Hellebust et al.[49]. The outcome of such dosimetric analysis needs to be com-pared with the results of spatial assessment of uncertainties, aspresented here.

Differences between individual cases

Evaluation of results from individual cases revealed most prom-inent delineation variation for V1 and V2. In addition, spatial analy-sis of the IDDs for the HR CTV reflected higher uncertainties in theparametrial regions in V1 (results not shown), which was not re-vealed in the overall analysis (Fig. 2). These findings suggest thatthere are inherent characteristics of individual cases, predisposingto delineation uncertainties. V1 is the case with the largest initial tu-mour dimensions, frank and extensive parametrial infiltration andless than 50% response to EBRT. In addition, planning MRI was per-formed at 0.2 and 1.5 T field strength for V1–3, and L1–3, respec-tively. Studies with specific clinical questions regarding the impactof field strength, tumour characteristics and other parameters ondelineation uncertainties represent interesting fields of furtherinvestigation.

Strategies to reduce delineation uncertainties

Ways to reduce interobserver delineation variability were pro-posed in the past by several authors [1,4] and have been pursuedin the frame of the GEC ESTRO activities. Based on the results ofthis study, they are elaborated below for the specific context of cer-vix cancer IGABT:

Fig. 3. Mean planar conformity index (PCI) for the GTV, HR CTV and IR CTV as afunction of slice number. Slice number begins at the first slice with at least onecontour of the corresponding volume. Expert consenus (EC) delineations were usedas the reference. Results with STAPLE (Simultaneous truth and performance levelestimation) contour as the reference were comparable (not shown).

10 Contouring uncertainties in cervix cancer brachytherapy

[1] Imaging: High quality imaging is a pre-requisite for accuratetarget volume and OAR delineation. Recently, GEC ESTROrecommendations, based on existing experience with 0.2 Tand 1.5 T MRI, were issued to take advantage of the fullpotential of MRI-based cervix cancer IGABT [38]. In thefuture, functional MRI and scanners with high field strength

may further improve the accuracy of target (sub) volumedelineation and predict the ultimate outcome [38]. Ultra-sound and CT represent an avenue of development thatmay make IGABT more widely available. Complementaryinformation from PET CT has been reported to improveinterobserver contouring agreement in various tumours[9,11,50–52] and may help to improve target definition ingynaecological IGABT. To minimize delineation uncertain-ties, eventual new modalities and target concepts need tobe benchmarked against the current modality of choice,which is based on MRI and comprehensive documentationof clinical information. Consistent department-specific pro-cessing of image information in accordance with GEC ESTROrecommendations is currently encouraged to obtain optimalresults.

[2] Training: when delineation in cervix cancer IGABT is per-formed in accordance with GEC ESTRO recommendationsby experienced observers, analysis of interobserver variationreveals encouraging results [20–22]. Adequate training,cooperation with radiologists, gynaecologists and educationat courses, workshops and e-learning platforms cannot beoveremphasized. Participation in multicentre studies whichrequire to pass a dummy run and are supported by centralquality control and delineation reviewing board is encour-aged. During training, attention is advised at predilectionregions for uncertainties (cranial, caudal and posterolateral).

[3] Delineation guidelines: The use of guidelines and delineationprotocols reduces uncertainties [1,2,4,10,53]. Adherence tothe GEC ESTRO recommendations on target volume delinea-tion [30] and the upcoming ICRU/GEC ESTRO recommenda-tions is encouraged to minimize the observer-specificinterpretation of radiological and clinical findings duringcontouring. In practice, the designated team of radiationoncologists is advised to jointly review the target volumedelineations before treatment.

Conclusions

Uncertainties in MRI-based target volume delineation maychallenge the technological gain of increasing treatment precisionand complicate interpretation of clinical studies. In our study, HRCTV was the most robust target volume with less pronounceddelineation uncertainties when compared with GTV and IR CTV.The use of HR CTV may be advised for dose prescription and opti-mization. Simultaneous careful consideration of the doses appliedto the GTV (the most relevant part of HR CTV) and IR CTV (relatedto the initial tumour extent) is essential. High imaging quality, ade-quate training and respecting the recommendations are the mainstrategies to minimize delineation uncertainties.

Conflict of Interest

Gyn GEC ESTRO Network activities are supported by fundingfrom Varian Medical Systems, Inc. and Nucletron, an Elektacompany.

Acknowledgements

The authors are grateful to the observers for all the time and ef-fort, invested in delineation of the study cases. Observers from thefollowing centres participated in the study: Aarhus University Hos-pital, University of Kaposvar, Leiden University Medical Center, StJames’s University Hospital (Leeds), University Hospital Gasthuis-berg (Leuven), MAASTRO Clinic (Maastricht), University MedicalCenter Utrecht, Institute Gustave-Roussy (Villejuif), Tata Memorial

P. Petric et al. / Radiotherapy and Oncology 107 (2013) 6–12 11

Hospital (Mumbai) and Medical College of Wisconsin (Milwaukee).We appreciate the contribution of dr. Christine Haie Meder fromthe Institute Gustave-Roussy during creation of the expert consen-sus contours and the work of Christoffer Lervåg (Ålesund Hospital,Møre and Romsdal Health Trust), who created the STAPLE delinea-tions which made this study possible. The Gyn GEC ESTROacknowledges the support by the Varian Medical Systems, Inc.and Nucletron, an Elekta company. Aarhus University Hospitalwas supported by research Grants from the European School ofOncology (ESO), Danish Cancer Society, Danish Council for Strate-gic Research, and CIRRO – the Lundbeck Foundation Centre forInterventional Research in Radiation Oncology.

References

[1] Njeh CF. Tumor delineation: the weakest link in the search for accuracy inradiotherapy. J Med Phys 2008;33:136–40.

[2] Fotina I, Lütgendorf-Caucig C, Stock M, et al. Critical discussion of evaluationparameters for inter-observer variability in target definition for radiationtherapy. Strahlenther Onkol 2012;188:160–7.

[3] Allozi R, Li A, White J, et al. Tools for consensus analysis of expert’s contours forradiotherapy structure definitions. Radiother Oncol 2010;97:572–8.

[4] Weiss E, Richter S, Krauss T, et al. Conformal radiotherapy planning of cervixcarcinoma: differences in the delineation of target volume. A comparisonbetween gynaecologic and radiation oncologists. Radiother Oncol2003;67:87–95.

[5] Giezen M, Kouwenhouven E, Scholten AN, et al. Magnetic resonance imaging-versus computed tomography-based target volume delineation of theglandular breast tissue (clinical target volume breast) in breast-conservingtherapy: an exploratory study. In J Radiat Oncol Biol Phys 2011;81:804–11.

[6] Hurkmans C, Admiraal M, Van Der SM, et al. Significance of breast boostvolume changes during radiotherapy in relation to current clinicalinterobserver variations. Radiother Oncol 2009;90:60–5.

[7] Landis DM, Luo W, Song J, et al. Variability among breast radiation oncologistsin delineation of the postsurgical lumpectomy cavity. Int J Radiat Oncol BiolPhys 2007;67:1299–308.

[8] Petersen RP, Truong PT, Kader HA, et al. Target volume delineation for partialbreast radiotherapy planning: clinical characteristics associated with lowinterobserver concordance. Int J Radiat Oncol Biol Phys 2007;69:41–8.

[9] Steenbakkers RJ, Duppen JC, Fitton I, et al. Reduction of observer variationusing matched CT-PET for lung cancer delineation: a three-dimensionalanalysis. Int J Radiat Oncol Biol Phys 2006;64:348–435.

[10] Wong EK, Truong PT, Kader HA, et al. Consistency in seroma contouring forpartial breast radiotherapy: impact of guidelines. Int J Radiat Oncol Biol Phys2006;66:372–6.

[11] Fox JL, Rengan R, O’Meara W, et al. Does registration of PET and planning CTimages decrease interobserver and intraobserver variation in delineatingtumor volumes for non-small-cell lung cancer? Int J Radiat Oncol Biol Phys2005;62:70–5.

[12] Struikmans H, Warlam-Rodenhuis C, Stam T. Interobserver variability ofclinical target volume delineation of glandular breast tissue and of boostvolume in tangential breast irradiation. Radiother Oncol 2005;76:293–9.

[13] Giraud P, Elles S, Helfre S, et al. Conformal radiotherapy for lung cancer:different delineation of the gross tumor volume (GTV) by radiologists andradiation oncologists. Radiother Oncol 2002;62:27–36.

[14] Van de Steene J, Linthout N, de Mey J, et al. Definition of gross tumor volume inlung cancer: inter-observer variability. Radiother Oncol 2002;62:37–49.

[15] Hurkmans CW, Borger JH, Pieters BR, Russell NS, Jansen EP, Mijnheer BJ.Variability in target volume delineation on CT scans of the breast. Int J RadiatOncol Biol Phys 2001;50:1366–72.

[16] Weltens C, Menten J, Feron M, et al. Interobserver variations in gross tumorvolume delineation of brain tumors on computed tomography and impact ofmagnetic resonance imaging. Radiother Oncol 2001;60:49–59.

[17] Khoo VS, Adams EJ, Saran F, et al. A comparison of clinical target volumesdetermined by CT and MRI for the radiotherapy planning of base of skullmeningiomas. Int J Radiat Oncol Biol Phys 2000;46:1309–17.

[18] Rasch C, Barillot I, Remeijer P, Touw A, van Herk M, Lebesque JV. Definition ofthe prostate in CT and MRI: a multi-observer study. Int J Radiat Oncol Biol Phys1999;43:57–66.

[19] Fiorino C, Reni M, Bolognesi A, Cattaneo GM, Calandrin R. Intra- and inter-observer variability in contouring prostate and seminal vesicles: implicationsfor conformal treatment planning. Radiother Oncol 1998;47:285–92.

[20] Petric P, Hudej R, Rogelj P, et al. Comparison of 3D MRI with high samplingefficiency and 2D multiplanar MRI for contouring in cervix cancerbrachytherapy. Radiol Oncol 2012;46:242–51.

[21] Dimopoulos JC, De Vos V, Berger D, et al. Inter-observer comparison of targetdelineation for MRI-assisted cervical cancer brachytherapy: application of theGYN GEC-ESTRO recommendations. Radiother Oncol 2009;91:166–72.

[22] Petric P, Dimopoulos J, Kirisits C, et al. Inter- and intraobserver variation in HR-CTV contouring: Intercomparison of transverse and paratransverse image

orientation in 3D-MRI assisted cervix cancer brachytherapy. Radiother Oncol2008;89:164–71.

[23] Viswanathan AN, Dimopoulos JCA, Kirisits C, et al. CT versus MRI-basedcontouring in cervical cancer brachytherapy: results of a prospective trial andpreliminary guidelines for standardized contours. Int J Radiat Oncol Biol Phys2007;68:491–8.

[24] Lang S, Nulens A, Briot E, et al. Intercomparison of treatment concepts for MRimage assisted brachytherapy of cervical carcinoma based on GYN GEC-ESTROrecommendations. Radiother Oncol 2006;78:185–93.

[25] Wu DH, Mayr NA, Karatas Y, et al. Interobserver variation in cervical cancertumour delineation for image based radiotherapy planning among and withindifferent specialties. J Appl Clin Med Phys 2005;6:106–10.

[26] Eskander RN, Scanderberg D, Saenz CC, et al. Comparison of computed tomographyand magnetic resonance imaging in cervical cancer brachytherapy target and normaltissue contouring. Int J Gynecol Cancer 2010;20:47–53.

[27] Saarnak AE, Boersma M, Van Bunnigen BNFM, et al. Inter-observer variation indelineation of bladder and rectum contours for brachytherapy of cervicalcancer. Radiother Onclo 2000;56:37–42.

[28] Petric P, Hudej R, Music M. MRI assisted cervix cancer brachytherapy pre-planning, based on insertion of the applicator in para-cervical anaesthesia:preliminary results of a prospective study. J Contemp Brachyther2009;1:163–9.

[29] Dimopoulos JCA, Schard G, Berger D, et al. Systematic evaluation of MRIfindings in different stages of treatment of cervical cancer: potential of MRI ondelineation of target, patho-anatomical structures and organs at risk. Int JRadiat Oncol Biol Phys 2006;64:1380–8.

[30] Haie-Meder C, Pötter R, Van Limbergen E, et al. Recommendations fromgynaecological (GYN) GEC-ESTRO working group (I): concepts and terms in 3Dimage based 3D treatment planning in cervix cancer brachytherapy with emphasison MRI assessment of GTV and CTV. Radiother Oncol 2005;74:235–45.

[31] Warfield SK, Zou KH, Wells WM. Simultaneous truth and performance levelestimation (STAPLE): an algorithm for the validation of image segmentation.IEEE Trans Med Imaging 2004;23:903–21.

[32] Kouwenhouven E, Giezen M, Struikmans H. Measuring the similarity of targetvolume delineations independent of the number of observers. Phys Med Biol2009;54:2863–73.

[33] Pinheiro JC, Bates DM. Linear mixed-effects models. In: Chambers J, Eddy W,Härdle W, Sheather S, Tierney L, editors. Mixed effects models in S and S-plus. New York: Springer; 2000. p. 1–271.

[34] Brown H, Prescott R. Generalised linear mixed models. In: Senn S, Barnett V,editors. Applied mixed models in medicine. Chichester: Wiley; 2006. p.107–53.

[35] PetriC P, Pötter R, Van Limbergen E, Haie-Meder C. Adaptive contouring of thetarget volume and organs at risk. In: Viswanathan AN, editor. Gynecologicradiation therapy: novel approaches to image-guidance andmanagement. Heidelberg: Springer; 2011. p. 99–118.

[36] Pötter R, Haie-Meder C, Van Limbergen E, et al. Recommendations fromgynaecological (GYN) GEC-ESTRO Working Group: (II): concepts and terms of3D imaging, radiation physics, radiobiology, and 3D dose volume parameters.Radiother Oncol 2006;78:67–77.

[37] Hellebust TP, Kirisits C, Berger D, et al. Recommendations from gynaecological(GYN) GEC-ESTRO working group: considerations and pitfalls incommissioning and applicator reconstruction in 3D image-based treatmentplanning of cervix cancer brachytherapy. Radiother Oncol 2010;96:153–60.

[38] Dimopoulos JC, Petrow P, Tanderup K, et al. Recommendations fromgynaecological (GYN) GEC-ESTRO working group (IV): basic principles andparameters for MRI imaging within the frame of image based adaptive cervixcancer brachytherapy. Radiother Oncol 2012;103:113–22.

[39] Nag S. Controversies and new developments in gynecologic brachytherapy:image-based intracavitary brachytherapy for cervical carcinoma. Semin RadiatOncol 2006;16:164–7.

[40] Guedea F, Ellison T, Venselaar J, et al. Overview of brachytherapy resources inEurope: a survey of patterns of care study for brachytherapy in Europe.Radiother Oncol 2007;82:50–4.

[41] Guedea F, Venselaar J, Hoskin P, et al. Patterns of care for brachytherapy inEurope: updated results. Radiother Oncol 2010;97:514–20.

[42] Guedea F, Ventura M, Londres B, et al. Overview of brachytherapy resources inLatin America: a patterns-of-care survey. Brachytherapy 2011;10:363–8.

[43] Viswanathan AN, Erickson BA. Three-dimensional imaging in gynaecologicalbrachytherapy: a survey of the American brachytherapy society. Int J RadiatOncol Biol Phys 2010;76:104–9.

[44] Van Dyk S, Byram D, Bernshaw D. Use of 3D imaging and awareness of GEC-ESTRO recommendations for cervix cancer brachytherapy throughoutAustralia and New Zealand. J Med Imaging Radiat Oncol 2010;54:383–7.

[45] Van Dyk S, Narayan K, Fisher R, Bernshaw D. Conformal brachytherapyplanning for cervical cancer using transabdominal ultrasound. Int J RadiatOncol Biol Phys 2009;75:64–70.

[46] Altorjai G, Fotina I, Lütgendorf-Caucig C, et al. Cone-beam CT-baseddelineation of stereotactic lung targets: the influence of image modality andtarget size on interobserver variability. Int J Radiat Oncol Biol Phys2012;82(2):265–72.

[47] Weiss E, Wu J, Sleemen W, et al. Clinical evaluation of soft tissue organboundary visualization on cone-beam computed tomographic (CBCT) imaging.Int J Radiat Oncol Biol Phys 2010;78:929–36.

12 Contouring uncertainties in cervix cancer brachytherapy

[48] Hellebust TP, Tanderup K, Lervåg C, et al. Dosimetric impact of interobservervariability in MRI-based delineation for cervical cancer brachytherapy.Radiother Oncol 2013.

[49] Hellebust TP, Petric P, Tanderup K, et al. Spatial dosimetric sensitivity analysisof contouring uncertainties in gyn 3D based brachytherapy. Radiother Oncol2012;104:S34.

[50] Caldwell CB, Mah K, Ung YC, Danjoux CE, Balogh JM, Ganguli SN, et al. Observervariation in contouring gross tumor volume in patients with poorly definednon-small cell lung tumors on CT: The impact of 18FDG-hybrid PET fusion. IntJ Radiat Oncol Biol Phys 2001;51:923–31.

[51] Krengli M, Cannillo B, Turri L, et al. Target volume delineation for preoperativeradiotherapy of rectal cancer: inter-observer variability and potential impactof FDG-PET/CT imaging. Technol Cancer Res Treat 2010;9:393–8.

[52] Metwally H, Courbon F, David I, et al. Coregistration of prechemotherapy PET-CT for planning pediatric Hodgkin’s disease radiotherapy significantlydiminishes interobserver variability of clinical target volume definition. Int JRadiat Oncol Biol Phys 2011;80:793–9.

[53] Bowden P, Fisher R, Mac Manus M, et al. Measurement of lung tumor volumesusing three-dimensional computer planning software. Int J Radiat Oncol BiolPhys 2002;53:566–73.