Prediction of Breast Deformities: A StepForward for Planning Aesthetic Results After

Breast Surgery

Sılvia Bessa(B), Hooshiar Zolfagharnasab, Eduardo Pereira,and Helder P. Oliveira

INESC TEC, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal{snbessa,hooshiar.zolfagharnasab,ejmp,helder.f.oliveira}@inesctec.pt

Abstract. The development of a three-dimensional (3D) planing toolfor breast cancer surgery requires the existence of proper deformablemodels of the breast, with parameters that can be manipulated to obtainthe desired shape. However, modelling breast is a challenging task due tothe lack of physical landmarks that remain unchanged after deformation.In this paper, the fitting of a 3D point cloud of the breast to a parametricmodel suitable for surgery planning is investigated. Regression techniqueswere used to learn breast deformation functions from exemplar data,resulting in comprehensive models easy to manipulate by surgeons. Newbreast shapes are modelled by varying the type and degree of deformationof three common deformations: ptosis, turn and top-shape.

Keywords: Breast deformations · 3D modelling · Regression models

1 Introduction

In a world where perception of body image takes an important role in the self-esteem of most women, the high incidence rates of breast cancer have been jeop-ardizing the sense of femininity and quality-of-life. After being diagnosed withbreast cancer, women not only face the fear of death, but also the fear of breastdisfigurement, especially because surgery is still the primary treatment for thistype of cancer. With the improvements in surgical procedures and oncologicaltreatments, the aesthetic outcomes have become less dramatic. Even so, the mul-titude of surgical options allied with heterogeneous practices still contribute todifferent aesthetic results. The involvement of women in the treatment decisionprocess has been proven benefit to accept the resulting outcomes [1]; however,the discussion of the different surgical options and the predicted cosmetic out-comes, still relies on 2D visualization of drawings, images or the use of simplemorphing capabilities. Some breast surgery planing tools are already available,but modelers for surgery other than breast augmentation are less common.

Alternative 3D approaches have been addressed in later years, but theyusually demand expensive 3D scanners, landmarks positioned in women torso

c© Springer International Publishing AG 2017L.A. Alexandre et al. (Eds.): IbPRIA 2017, LNCS 10255, pp. 267–276, 2017.DOI: 10.1007/978-3-319-58838-4 30

268 S. Bessa et al.

or complicated procedures to obtain 3D data of the patient. Moreover, mostapproaches are not patient-specific: they require the mapping of 3D data to afixed model, which is posteriorly modified to describe breast deformities. This isthe case of the breast modeler proposed by Seo et al. [2]. In this work, breasts aremodelled with user-intuitive attributes, but 3D breast scans obtained with land-marks are mapped to a template mesh. The template mesh has a fixed numberof points and triangle patches, that constitute shape vectors. Breast shapes arethen generated by varying user-supplied parameters, that are translated to shapevector displacements with a linear model. Kim et al. [3] developed a 3D virtualsimulator for breast plastic surgery. The subject’s 3D torso data is obtained from2D orthogonal photographs and a 3D model template is fitted to the breast, u-sing several feature points manually marked on images. The simulation of thesurgery outcome is based on the idea that each subject can be expressed as alinear combination of exemplar data. Each new breast is described as a combi-nation of breast models in a database. The displacements between the pre andpost models is known and they are used to deform the new breast.

As seen before, to develop a breast surgery planning tool, compact repre-sentations of breasts, characterized by a reduced number of parameters, areneeded. A common approach to obtain such models is to fit surfaces to 3D data.Bardinet et al. [4], exploited the fitting of a deformable parametric model to 3Ddata. Initially, a superquadric is fitted to data, followed by a Free Form Defor-mation (FFD) to refine the fitting to unstructured 3D data. Chen et al. [5] alsoexplored the use of superquadrics to model 3D data, but the parametric model ofthe breast is obtained by applying global deformation functions of known breastdeformities, to the primitive superquadric shape. Langrarian mechanics is usedto define the deformation parameters that optimizes the fitting of the model to3D data. In alternative, Pernes et al. [6], proposed a strategy in which the fittingof the same model is accomplished by minimizing a modified version of the leastsquares cost function, using geometric interpretations instead of the minimumdistance between data and model. It presents better fitting results in comparisonwith the physic-based approach of [5]. Other works [7,8] propose a parametriza-tion of the breast by applying Principal Component Analysis (PCA) to datasetsof Nuclear Magnetic Resonances. Breast models are generated by manipulatingprincipal components to obtain the desired breast shape.

In summary, attempts to model breast deformities, usually require the posi-tioning of landmarks on the patients’ body during image acquisition. Theydepend on a limited number of mathematical equations, that describe partic-ular breast deformities, or fail to be patient-specific and use adjustable parame-ters easy to manipulate by the common user. In this paper, it is proposed tomodel breast deformities by using machine-learning strategies to learn deforma-tion parameters of known functions defined by Chen et al. [5].

2 Proposed Methodology

In an attempt to model breast surgery results, it was explored the use of machinelearning techniques to learn the parameters associated with each major breast

Prediction of Breast Deformities 269

Fig. 1. Block diagram of the proposed methodology.

deformation from exemplar data. Simple adjustment parameters, such as thetype and degree of deformation, are proposed to adapt the model to the desirableshape. The proposed methodology is described in the block diagram of Fig. 1.In brief, breasts are modelled by defining the types of deformations that willresult from surgery, as well as the respective degrees of deformation. The user-defined parameters are combined with features automatically extracted fromdata and a set of regression models predict the deformation parameters, thatapplied to the original point cloud result in the desired shape. Considering thateach deformation has to be described by a specific parametric model, there will beas many regression models as types of deformations. The final shape is the resultof the combined effect of several deformations applied to the original breast.

2.1 Database

In the absence of public databases of breast deformities, the proposed methodo-logy was developed using synthetic databases (ptosis, turn and top-shape defor-mations), created using the equations from [5]. In detail, first, the size of theprimitive superellipsoid is defined by generating random axis values. Second,variable sets of deformation parameters are selected from a range of values, pre-viously defined for each type of deformation: ptosis, turn, top-shape, flatten-sideand top/turn deformations. These parameters are then applied to the primitiveto create breasts with different appearances.

In order to train regression models for specific breast deformations, exampleswith varying degrees of deformation were obtained by applying the respectivedeformation function, with different parameters values, to the synthetic breasts.The resultant degree of deformation was subsequently quantified using a distancemetric, Dij , that compared the original, Pi0, and deformed, Pij , points cloudsof the breasts, as defined by Eq. 1:

Dij = ‖Pi0 − P00‖ + ‖Pij − P0j‖ , (1)

where i is a synthetic breast, to which a j set of deformation parameters wasapplied, and the subscript 0 refers to a reference point cloud used to align

270 S. Bessa et al.

original, O and deformed breasts, M . ‖.‖ is the normalized euclidean distancedefined in:

‖o − m‖ =

√(ox − mx)2 + (oy − my)2 + (oz − mz)2√

o2x + o2y + o2z

. (2)

In this paper, regression models were trained to model ptosis, turn and top-shape deformations, described in [5]. Note, however, that the coordinate systemassociated to our breasts is different than the one proposed in [5]: here, z axisprotrudes from the chest wall outward through the nipple (anterior-posterior), yaxis goes up (inferior-superior), and x axis goes from right to left (lateral-medial).All deformations functions were adapted for this system of coordinates.

2.2 Regression Models

In pursuit of a proper regression model to predict the parameters associatedwith each deformation, a preliminary optimization stage was carried for ptosis.Note that ptosis describes the sagging effect of the breast, which mathematicallytranslates to a shift in the y coordinates, defined as a quadratic function of thez coordinates of points, with b0 and b1 parameters: Sy = b0z + b1z

2 (adaptedfrom [5]). For this study, five feature sets were designed (Table 1), and a refe-rence feature set (REF ) was defined containing the displacement in y pointscoordinates (Sy). By using Sy as input feature, the model is biased because thisis exactly what we want to measure. However, it will serve as ground truth tovalidate the models. All features were normalized to the range of 0 and 1. Thebenefit of using PCA for feature selection was also considered.

Table 1. Feature sets of the ptosis study: DD - degree of deformation, uppercase referto vectors of all coordinates, lowercase are average coordinates values and 0 referrersto original coordinates.

Features

Acronym DD v0 Z0 Y0 z0 y0 Sy

REF X X X X

FS1 X X X

FS2 X X X X

FS3 X X X

FS4 X X

FS5 X X X X

For each feature set, Linear Regression (LR), Support Vector Machine (SVM)and Neural Network (NN) regression models were explored and their relativeperformances were compared. The simple LR was considered and polynomial

Prediction of Breast Deformities 271

basis functions up to the 4th degree were tested, either with or without interceptterm. SVM regressions were trained with linear, polynomial and RBF kernels,and a grid-search optimization strategy was used to select the best parame-trization. In particular, exponentially growing sequences of C and γ (only forRBF kernel) were tested: C = 2−5, 2−3, ..., 27, and γ = 2−5, 2−3, ..., 23. Feed-forward NNs with one hidden layer were trained with a varying number of neu-rons, {5, 10, 25, 50, 75}, using the Scaled Conjugate Gradient Backpropagation.All regression models were optimized using 4-fold cross validation in the traindatasets and the Relative Mean Squared Error as evaluation metric. Where f(xi)are the predicted values from regressions applied to testing data xi, whose truetarget values of testing data are yi. The performance of the best parametrizationwas evaluated on test datasets using distances between original (O) and mod-elled (M) breasts as indirect performance metrics. Here, both Hausdorff (Eq. 3)and mean Euclidean (Eq. 2) distances were used.

h(O,M) = maxo∈O

minm∈M

||o − m|| (3)

where O and M are the matrix of points (N points × 3 dimensions X, Y andZ), and ‖.‖ is the normalized euclidean distance defined in Eq. 2. To providea fairly comparison between breasts with different sizes and resolutions withinthe same dataset, distances are normalized by the distance of the original pointcloud to the origin of the coordinate system. Distances are computed betweenoriginal and modelled breasts, and in both directions; the Hausdorff distance isused because it describes the worst case scenario. Finally, the best combinationof feature set, were adapted for turn and top shape deformations.

3 Experimental Results

3.1 Optimization of Regression Models

Regression models were optimized on datasets containing ptosis examples. Allexamples shared the same original appearance, but had different sizes. In thisstage, the influence of several scenarios (Table 2) were considered when analysingthe performance of models to predict single ptosis parameters at time (b0 or b1).Hypothesis tests with a significance level of 5% were conducted to compare anddecide upon the best scenario. Statistical tests were conducted with the RMSEresults of four conditions: single parameters predicted at time (b0 or b1) andmultiple output predictions: b0 > b1 or b1 ≥ b0.

To consider the effect of the number of degrees of deformation, models weretrained and tested with 200/60 or 400/120 (train/test) examples, depending if 4or 8◦ of deformation were used, respectively. Results of the one-sided t-test sug-gest that results of models trained with 8◦ of deformation had significantly lowerRMSE (p = 0.0027). This was expected because, despite the use of categoricalvalues as inputs, regression models are modelled with continuous variables. So,a higher number of degrees should lead to better fittings. LR models alwaysperformed worst than SVM or NN, but no significant differences were found

272 S. Bessa et al.

Table 2. Optimization Scenarios Explored.

Scenario Hypothesis Statistical test p-value

Degrees of deformation {4,8} One-sided t-test 0.0027*

Feature selection {none, PCA} Paired t-test 0.1725

Regression model {SVM, NN} Paired t-test 0.2300

*highlights significant differences with a significance level of 5%.Bold hypothesis were selected for subsequent stages.

between SVM and NN models. NN regression models were introduced envision-ing the task of predicting multiple parameters of the same deformation functionbecause, contrarily to SVM, NN take in consideration the correlation of the out-puts in the learning process. So thorough analysis of the models was carriedconsidering only multiple outputs conditions and it revealed that NN regres-sions trained with the optimal feature set had slightly better results than SVMmodels. Regarding feature selection, results suggest no benefit in using PCA,although the optimal feature set varied whether PCA was applied or not: whenPCA was used, the best feature set was FS2, while without PCA, the optimalfeature was FS5. This was not unexpected, because both FS2 and FS5 con-tain z and y coordinates produced, varying only on the number of points used.So, the similar results only corroborate the high correlation between the samecoordinates of all points. Therefore, preference was given to NN models trainedwithout PCA and using feature set FS5. The use of FS5 avoids interpolationsto apply the regression models to breast point clouds with varying number offeatures, because only the coordinates of the average point are used.

3.2 Prediction of Ptosis, Turn and Top-Shape Deformations

Next, the optimal scenarios used to generate regression models for ptosis, turnand top-shape deformations are presented. Models were trained and tested with384 and 64 examples, respectively, and breasts with varying sizes and shapes wereused. To predict ptosis, regression models were obtained using FS5. An adaptedversion of this feature set was used to predict turn, in which x coordinates areused instead of y coordinates. This is because in turn deformation, points arechanged as a quadratic function of their z coordinates, but the displacementoccurs along the x axis instead: Sx = b0z + b1z

2 (adapted from [5]). In top-shape deformations, points change along the z axis, as function of their angularposition u in relation to the nipple [5], where 4 parameters can be adjustedto model the top-half profile of the breast. However, only the influence of s0and s1 parameters was modelled, which respectively control the slope of breastpoints near the chest wall and the nipple. For modelling top-shape deformations,the y coordinate in FS5 could have been replaced by spherical coordinates but,instead, it was included all Cartesian coordinates and relied on the NN capacityof modelling the angles by itself. Besides, the degree of deformation of top-shape deformations was quantified with a modified version of Eq. 1, in which the

Prediction of Breast Deformities 273

normalized Euclidean distance was replaced by the Hausdorff distance:

Dij = ‖Pi0 − P00‖ + maxpij∈Pij

minp0j∈P0j

||pij − p0j || . (4)

This was a necessary modification because the effect of varying the top shapeslope parameters is specially notorious on the center area of the top breast profile,which makes Hausdorff distance more appropriate to quantify this effect.

Table 3 presents the indirect performances of all deformation models. Resultssuggest that models learned properly and no significant difference is foundwhen measuring the distances in different directions (Modelled ⇒ Original andOriginal ⇒ Modelled). For all outputs, the average Euclidean and Hausdorffdistances are lower than 3% and 10%, respectively. The worst case scenario is

Table 3. Indirect performance metrics of NN regression models predicting breast defor-mations. Relative distances (percentages) are shown.

Deformation Outputs Statistics Modelled ⇒ Original Original ⇒ Modelled

Euclidean Hausdorff Euclidean Hausdorff

Ptosis b0 > b1 μ 1.66 5.69 2.10 6.52

σ 1.38 4.60 2.51 6.66

Min 0.10 0.18 0.10 0.18

Max 6.71 22.22 11.76 30.97

b1 ≥ b0 μ 2.98 9.64 2.08 7.25

σ 5.56 14.44 1.95 5.10

Min 0.16 0.46 0.16 0.46

Max 37.94 95.87 11.90 25.79

Turn c0 > c1 μ 2.21 6.77 2.12 6.53

σ 2.22 6.30 1.86 5.69

Min 0.19 0.41 0.18 0.41

Max 11.09 29.62 8.36 26.63

c1 > c0 μ 1.67 6.04 1.86 6.54

σ 1.31 4.24 1.60 4.90

Min 0.16 0.38 0.16 0.38

Max 7.16 21.81 6.72 20.22

Top-Shape |s0| > |s+1 | μ 0.56 4.66 0.56 4.65

σ 0.37 3.36 0.37 3.34

Min 0.00 0.02 0.00 0.02

Max 1.48 15.07 1.46 14.95

|s1| > |s0| μ 0.77 6.21 0.76 6.22

σ 0.39 3.60 0.37 3.60

Min 0.10 0.71 0.10 0.71

Max 1.59 14.01 1.41 13.88

274 S. Bessa et al.

found in the prediction of b1 > b0, with Euclidean and Hausdorff distances goingup to 38% and 96%. However, the residual analysis of this model (Fig. 2a) con-firmed the existence of an outlier (signaled by a red arrow). Thorough analysisof the results showed that the outlier is an example of deformation caused bya high value, whose degree of deformation was badly assigned when convertingthe distance in Eq. 1 to a categorical value. In spite of the existence of an out-lier, residual analysis of the model still suggests goodness of fitting for ptosis’regression model: residuals are slightly randomly dispersed in scatter plots oftarget, or predicted values, versus residuals; no structure is clearly identifiablein Lag plots and residuals distributions are approximately normal, as impliedby residuals’ histogram and normal plots. Similar analysis could be made forturn deformation, so that was not included in the paper. The performancesof the top shape deformations models suggest the suitability of the proposedmodifications. However, in the residuals analysis of top-shape deformations, thedistribution of target versus predicted values is not as linear as in ptosis or turndeformation models, particularly in the condition s−

1 > s+0 (Fig. 2b): scatterplots of residuals suggest some structure in the residuals, confirmed by a clearlinear distribution of values in the Lag plot. This means that, although the over-all differences between modeled and original top shape deformations are low, thegeneralization of these models as to be carefully considered, perhaps includingadditional features or revisiting the methodology used to define the degree ofdeformation. Nonetheless, examples of the average predictions of ptosis, turnand top-shape deformation parameters shown in Fig. 3 confirm the ability of themodels to predict breast deformations with small errors.

(a) b1 ≥ b0 (b) |s1| > |s0|

Fig. 2. 6-plot residual analysis for ptosis (left) and top-shape (right) models. Red Arrowsignals the presence of and outlier prediction. (Color figure online)

Prediction of Breast Deformities 275

(a) Ptosis, b0 > b1 (b) Turn, c0 > c1 (c) Top-shape, |s0| > |s1|

Fig. 3. Examples of modelled breasts from average predictions (skin color), superim-posed on the target shape (black) ((a) and (b) front views, (c) side-view).

4 Conclusions

In the proposed methodology, breast deformations are modelled using regressionmodels learnt from exemplar data. At first, a complete study was carried onptosis, to prove the usefulness of regression models, to predict parameters of knowdeformation functions, and the type and degree of deformation were suggestedas adaptable parameters to create the desired breast shape. Next, the best typeof regression model and set of features were tested in the prediction of othertypes of deformation, namely turn and top shape deformations. The parametricmodels were able to predict deformation parameters with good performances,using only the degree of deformation as simple and comprehensible parameterto adjust the shape of breast.

Acknowledgements. This work was funded by the Project NanoSTIMA: Macroto Nano Human Sensing: Towards Integrated Multimodal Health Monitoring andAnalytics/NORTE-01-0145-FEDER-000016 financed by the North Portugal RegionalOperational Programme (NORTE 2020), under the PORTUGAL 2020 PartnershipAgreement, and through the European Regional Development Fund (ERDF), andalso by Fundao para a Cincia e a Tecnologia (FCT) within PhD grant numberSFRH/BD/115616/2016.

References

1. Kim, M., Kim, T., Moon, H.-G., Jin, U.S., Kim, K., Kim, J., Lee, J., Lee, E., Yoo,T.-K., Noh, D.-Y., et al.: Effect of cosmetic outcome on quality of life after breastcancer surgery. Eur. J. Surg. Oncol. (EJSO) 41(3), 426–432 (2015)

2. Seo, H., Cordier, F., Hong, K.: A breast modeler based on analysis of breast scans.Comput. Animation Virtual Worlds 18(2), 141–151 (2007)

3. Kim, Y., Lee, K., Kim, W.: 3D virtual simulator for breast plastic surgery. Comput.Animation Virtual Worlds 19(3–4), 515–526 (2008)

4. Bardinet, E., Cohen, L.D., Ayache, N.: A parametric deformable model to fitunstructured 3D data. Comput. Vis. Image Underst. 71(1), 39–54 (1998)

276 S. Bessa et al.

5. Chen, D.T., Kakadiaris, I.A., Miller, M.J., Loftin, R.B., Patrick, C.: Modeling forplastic and reconstructive breast surgery. In: Delp, S.L., DiGoia, A.M., Jaramaz, B.(eds.) MICCAI 2000. LNCS, vol. 1935, pp. 1040–1050. Springer, Heidelberg (2000).doi:10.1007/978-3-540-40899-4 108

6. Pernes, D., Cardoso, J.S., Oliveira, H.P.: Fitting of superquadrics for breast mod-elling by geometric distance minimization. In: 2014 IEEE International Conferenceon Bioinformatics and Biomedicine (BIBM), pp. 293–296. IEEE (2014)

7. Gallo, G., Guarnera, G.C., Milanese, F., Modica, D., Catanuto, G., Pane, F.: Para-metric representation of human breast shapes (2009)

8. Gallo, G., Guarnera, G.C., Catanuto, G.: Human breast shape analysis using PCA.’In: BIOSIGNALS, pp. 163–167. Citeseer (2010)