automatic characterization of nanofiber assemblies by image texture analysis
TRANSCRIPT
This article appeared in a journal published by Elsevier. The attachedcopy is furnished to the author for internal non-commercial researchand education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling orlicensing copies, or posting to personal, institutional or third party
websites are prohibited.
In most cases authors are permitted to post their version of thearticle (e.g. in Word or Tex form) to their personal website orinstitutional repository. Authors requiring further information
regarding Elsevier’s archiving and manuscript policies areencouraged to visit:
http://www.elsevier.com/copyright
Author's personal copy
Automatic characterization of nanofiber assemblies by image texture analysis
Pierantonio Facco a,⁎, Emanuele Tomba a, Martina Roso b, Michele Modesti b,Fabrizio Bezzo a, Massimiliano Barolo a,⁎a CAPE-Lab — Computer-Aided Process Engineering Laboratory, Dipartimento di Principi e Impianti di Ingegneria Chimica, Università di Padova, via Marzolo, 9 — 35131 Padova PD, Italyb Dipartimento di Processi Chimici dell'Ingegneria, Università di Padova, via Marzolo, 9 — 35131 Padova PD, Italy
a b s t r a c ta r t i c l e i n f o
Article history:Received 20 January 2010Received in revised form 26 March 2010Accepted 25 May 2010Available online 1 June 2010
Keywords:Image texture analysisNanofibersNanomaterialsMultivariate statistical methodsMulti-resolution methodsQuality monitoringStatistical process controlElectrospinning
This paper presents a machine vision system for the automatic characterization of the quality properties ofnanofiber assemblies using image texture analysis techniques. The objective is to use a digital image of ananofiber membrane in order to estimate the pore size distribution (PSD), fiber diameter distribution (FDD)and permeability of the assembly. The underlying idea is that, after textural features have been extractedfrom the image in the form of statistical descriptors, these descriptors can be related to the membraneproperties using multivariate regression techniques. Two alternative feature extraction techniques areconsidered, a statistical-based approach (namely, gray-level co-occurrence matrix, GLCM) and a transform-based approach (based on the use of wavelet transforms).Polymer nanofiber membranes fabricated by electrospinning are used as test beds of the proposed system.The estimation results are very satisfactory for all properties with any of the two feature extractiontechniques, with the wavelet transform-based approach slightly outperforming the GLCM one. Whereas forthe estimation of the PSD and FDD a scanning electron microscope image of the product needs to beavailable, it is shown that as far as permeability is concerned an image at a much lower magnification scale(i.e. an optical microscope one) is sufficient to provide an accurate property estimation. Based on theseresults, the proposed system represents a very promising step toward the complete automation of qualityassessment procedures in nanomaterials processing.
© 2010 Elsevier B.V. All rights reserved.
1. Introduction
The interest in nanofiber assemblies has been steadily increasingin the last decade. Thanks to their peculiar features (i.e. high totalporosity, high specific area, good mechanical performances, highfunctionality), nanofiber membranes are used in a variety ofapplications, such as filtration and separation [1], advanced materials([2–4], catalysis [5], tissue engineering and medicine [6,7]).
The suitability of a nanofiber membrane for the application it hasbeen manufactured for is strongly related to its morphological andphysical characteristics. Inter-fiber pore morphology, fiber diametermorphology, and permeability are among the most importantproperties that characterize the quality of a nanofiber assembly.However, these quality properties are difficult to characterize in asystematic way. Laboratory measurement systems usually require toprocess experimentally a sample of the material (possibly withdestructive tests), and almost invariably call for human intervention,which increases the overall manufacturing costs and may lead to lossof repeatability. The hardware measurement instrumentation is
usually expensive, may require large measurement times, andsometimes may not even be available. For example, while experi-mental methods are available to measure the porosity and the poresize distribution in a nanofiber assembly, no hardware instrumenta-tion is available to measure the fiber diameter distribution so far.Presently, the diameter of a nanofiber is determined by visual inspec-tion of scanning electron microscope (SEM) images of the assembly,where an operator measures “manually” the diameter dimension bycounting the pixels between each fiber boundaries [8]. Recentlyproposed artificial vision systems for the automatic measurement ofnanofiber diameter distribution [9] represent a step forward in theautomatic quality characterization of nanofiber membranes, but atpresent they may suffer for long computational times.
Information on the quality characteristics of a product material isembedded in the plurality of pixels constituting a digital image of thatmaterial. The pixel-to-pixel light intensity variability along the imagerepresents the image texture, which can be related to the qualityfeatures of the object represented on the image itself. Under thisperspective, image texture analysis [10] has emerged as a verypowerful technique for product quality classification and monitoringin a variety of applications. Texture analysis techniques can be com-bined to multivariate statistical image analysis techniques and tomulti-resolution filtering techniques to extract the image texturefeatures that are most related to the quality features of the material
Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
⁎ Corresponding authors.E-mail addresses: [email protected] (P. Facco), [email protected]
(M. Barolo).
0169-7439/$ – see front matter © 2010 Elsevier B.V. All rights reserved.doi:10.1016/j.chemolab.2010.05.018
Contents lists available at ScienceDirect
Chemometrics and Intelligent Laboratory Systems
j ourna l homepage: www.e lsev ie r.com/ locate /chemolab
Author's personal copy
being analyzed, so as to develop efficient machine vision systems thatcan accomplish product quality monitoring and control effectively,automatically and with low capital investment [11–16]).
This paper presents a machine vision system for the automaticcharacterization of the quality properties of nanofiber assemblies usingimage texture analysis techniques. Textural features are extracted fromgray-level SEM images of the product using two different approaches:a statistical approach (namely, gray-level co-occurrence matrix, GLCM)and a transform-based one (based on the use of wavelet transforms).The textural features are then related to the quality properties to beestimated (i.e., pore diameter distribution, fiber diameter distribution,and permeability) through a multivariate regression method basedon the projection onto latent structures.
2. Nanofiber membrane manufacturing and characterization
2.1. Manufacturing
The nanofiber membranes considered in this study have beenobtained by electrospinning. By using this processing technique onecan process a vast range of inorganic and organic materials in order totailor the resulting membrane to the specific application it mustbe manufactured for. Electrospinning exploits electrostatic forces toproduce fibers with diameters ranging in the nanometer scale byforcing a solution through a spinneret under the influence of anelectric field (Fig. 1). Details on how this technique works can befound elsewhere [17,18].
In this work, two different sets of polymer electrospun thin non-wovenmembraneswere fabricated using a solution of polyacrylonitrile-co-vinylacetate in N,N-dimethylformamide in a controlled environ-ment at 26 °C and 43% relative humidity. Membrane Set #1 is acollection of 62 membranes fabricated with constant polymer con-centration; these membranes have negligible permeability differences.Membrane Set #2 is a collection of 18 membranes fabricated withvariable polymer concentration in such a way as to vary the membranepermeability without varying the membrane thickness. Some detailson the process parameters for the production of the two sets ofmembranes are given in Table 1.
Images at a magnification of 2400× and 5000× were obtained bySEM (Obducat CamScan MX2500™; Obducat CamScan, Cambridge,UK) from samples of membranes of Set #1 and Set #2, respectively.Therefore, each membrane set has its image set counterpart.Furthermore, SEM images at a magnification of 500× (reachablealso by optical microscopes) were obtained from samples of allmembranes included into membrane Set #2; the related image setwill be denoted as image Set #2bis.
All SEM images are grayscale scenes with a resolution of1024×1280 pixels. Sample images at different magnifications areshown in Fig. 2.
2.2. Characterization
To characterize the quality of a nanofiber membrane bothmorphological parameters and physical properties need specifying.The morphological parameters of interest are the fiber diameterdistribution (FDD) and the inter-fiber pore size distribution (PSD). Inthis study, the pore size is represented by the minor axis of the ellipsehaving the same second central moment as the pore. Other sizeparameters (e.g. the hydraulic diameter; [19]) may be considered,without major change in the reasoning. The PSD and FDD weremeasured automatically for all membranes belonging to Set #1 bymeans of an artificial vision system [9]. The measured PSD and FDD ofa typical membrane belonging to this set (namely, the one shownin Fig. 2a) are shown in Fig. 3a and b, respectively. Recalling that avigintile is any of the values in a series that divides the distribution of
Fig. 1. Schematic of an electrospinning process for the production of nanofibermembranes.
Table 1Characteristic process parameters for the production of the polymer nanofibermembranes considered in this study.
Membraneset
Polymerconcentration(wt.%)
Feed flow rate(mL/h)
Appliedvoltage (kV)
Tip-collectordistance (m)
#1 12 1 15 25×10−2
#2 10–20 0.5 15 27×10−2
Fig. 2. SEMmicrographs of a polymer nanofiber membrane sample: (a) image at 2400×magnification; (b) image at 500× magnification.
67P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
individuals in that series into twenty groups of equal frequency, adifferent representation of the PSD is shown in Fig. 3c, where thesymbols correspond to the 19 vigintiles of the distribution. Similarly,Fig. 3d shows the vigintiles of the FDD. The horizontal bars in Fig. 3c andd identify the measurement accuracy of the artificial vision system,which corresponds to the linear dimension of one pixel (38.5 nm).
The physical property of interest is membrane permeability.Permeability was measured for membranes of Set #2/#2bis followingthe cup B-1 test in accordance to the JIS-L-1099 standard. Themeasuredvalue ranged between 1906.5 g/(m2h) and 3702.9 g/(m2h). Theaccuracy of this measurement is ∼6%.
3. Design of the property estimation system
The proposed system for automatic membrane characterization ispresented in this section. The purpose is to use a digital image of ananofiber membrane in order to estimate the PSD and FDD, as wellas the membrane permeability, in a fast and accurate way. Theunderlying idea is that, after textural features have been extractedfrom the image in the form of statistical descriptors, these descriptorscan be related to the morphological and physical properties of themembrane through multivariate regression techniques. The proposedsystem goes through three main steps: i) image preprocessing,ii) extraction of textural features, and iii) interrogation of a multi-variate regression model. A discussion of these steps is presented inthe following subsections. Wavelet analysis [20,21], multivariate
image analysis [11,22] and projection onto latent structures [23–25]are the mathematical tools through which the above steps are carriedout.
3.1. Image preprocessing
In order to overcome the problem of a possibly different illu-mination between different image scenes, image equalization needsto be carried out as a first preprocessing operation. Let us assume thata grayscale image is represented by I, an [I×K] matrix of lightintensities, where a light intensity level l is associated to each (i, k)position of I, with i=1,2,…, I and k=0,1,2,…, K. Oftentimes, the actuallight intensity levels are limited to an incomplete fraction of the entirelight intensity scale (i.e. 0–255), mainly depending on the lightconditions on the scene. Equalization is a (linear) adjustment ofthe light intensity values to a grayscale map in the entire range ofthe light intensity levels. The result of this operation is the equalizedversion Ieq [I×K] of I.
A further issue that usually needs to be addressed duringpreprocessing is the removal of image noise. The systematic (i.e.structural) part of the signal in Ieq should in fact be separated fromrandom noise, which corrupts the signal itself. Although waveletdenoising [21] was at first used in this study, it was found that thefinal estimation results (i.e. accuracy in membrane property estima-tion) were barely affected by the denoising operation. This is probablydue to the fact that both techniques that were used for extracting
Fig. 3. Measurements in a Set #1 membrane: (a) pore size distribution, (b) fiber diameter distribution, (c) pore size cumulative distribution and (d) fiber diameter cumulativedistribution. The vertical black lines in (a) and (b) represent the mean of the distributions. The symbols in (c) and (d) represent the vigintiles of the pore size and fiber diameterdistributions (respectively). The horizontal bars indicate the measurement accuracy.
68 P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
textural features from the image (Section 3.2) are multi-resolutionones, and can therefore help extracting the textural features onlyat the resolution of interest, thus inherently providing a denoisingaction. Furthermore, the application of multivariate projectionmethods to relate image textural features to membrane propertiesprovides itself an indirect way to filter noise out of the analyzedimage. For these reasons, an explicit denoising operation on theequalized images was not accomplished.
Image filtering by multivariate image analysis (MIA; [12,22]) wascarried out instead. MIA provides a local filtering action thatcomprises averaging of pixel light intensity as well as first-derivativefiltering in the vertical and horizontal directions. It was found that,when an image was preprocessed by MIA, discriminating betweenreliable and unreliable estimations during the subsequent multivar-iate regression session (see Section 3.3) was more effective. HowMIAwas accomplished with the available images is discussed in thefollowing subsection.
3.1.1. Multivariate image analysisTheMIA algorithm applies multiway principal component analysis
(MPCA; [25]) through a two-step procedure requiring: (i) to create athree-dimensional image array Ieq [(I−2)× J×(K−2)], where theequalized image Ieq is associated to shifted versions of the image itself,and then (ii) to summarize the image information content through adimensionality reduction by using principal component analysis. Thethree-way array Ieq is built by putting side by side Ieq and its shiftedversions. Namely, J−1=8 shifted image versions are created bytaking Ieq and shifting it by one pixel in the eight directions of theneighboring pixels (Fig. 4). By proceeding this way, J represents anadditional dimension that is accounted for, i.e. the spatial variablesdimension (spatial shift index). Note that, due to the one-pixel shift,to ensure full overlapping among the stacked images the edge pixelsof each image need to be discarded when building the three-wayarray Ieq (image cropping).
The correlation between neighboring pixels is captured byapplyingMPCA to Ieq. MPCA combines the pixel intensities at differentspatial shift indices to obtain a reduced number of variables (i.e.principal components) that explain the original data variance. In fact,MPCA is equivalent to PCA on an enlarged two-dimensional matrix I2D[((I−2) ⋅(K−2))× J] obtained by image-wise unfolding the three-way array Ieq:
PI
eq
½ðI−2Þ � J � ðK−2Þ�→unfolding I2D
½ððI−2Þ⋅ðK−2ÞÞ�J�→PCA IPCA =XAa=1
tapTa;
ð1Þ
where ta [((I−2)·(K−2))×1] is a score vector, pa [J×1] is a loadingvector, and dim(IPCA)=[(I−2)(K−2)×J]. By reconstructing the originalimage using only Ab J principal components, a filtering action isobtained that accounts for the correlation of light intensity betweenneighboring pixels. A filtered version Ifilt [(I−2)×(K−2)] of the
original image is obtained eventually by folding the first column( J=1) of IPCA. This procedure is very similar to applying maximumautocorrelation factor analysis to Ieq [26,27]. Following the indications ofBharati et al. [12], A=3principal componentswere used in this study. Acheck on the residual sum of squares image confirmed that nosignificant information remained hidden in the residuals.
3.2. Extraction of textural features
Texture analysis techniques are applied on the preprocessed imageIfilt. In this paper we report the results for two different approaches oftextural feature extraction from an image, one belonging to thecategory of statistical texture analysis techniques and the other onebelonging to transform-based texture analysis techniques.
3.2.1. Gray-level co-occurrence matrixThe GLCM approach [28] is a statistical texture analysis method
that explicitly considers the spatial relationship of pixels in theimage. The GLCM is a tabulation of how often different combinationsof pixel light intensity values (gray levels) occur in an image. It takesthe form of a square matrix whose dimensions are equal to thenumber of grayscale levels of the image. In order to relieve thecomputational burden, the number of grayscale levels actuallyconsidered may be lower than the full light intensity scale [15]. Inthis study, the original 256 light intensity levels were discretizedinto Λ=64 levels. Each (l, λ) entry of the GLCM corresponds to thejoint probability Pδ,θ(l,λ) that a pair of pixels with light intensityvalues l and λ, (l,λ=1,2,…, Λ) , located at a distance of δ pixels andat an angle θ from each other, occur in the image.
The selection of the δ and θ parameters is dependent on the type oftexture being analyzed. Fine textures are best analyzed through smallpixel distances, while coarser textures call for larger values of δ. In thisstudy, δ takes the values of {1, 2, 5, 10} pixels, which correspond toactual distances of {38.5, 77.0, 192.5, 385.0} nm on the image scene(note that the larger selected distances are in the range of the smallestexpected fiber diameter). The smaller values of δ (i.e. δ=1 or 2 pixels)are useful to classify neighboring pixels as belonging to a fiber edge ornot, which can be exploited to locally evaluate the existence of fibersor pores. The larger values of δ are useful to estimate the fiberdimension by evaluating whether or not pixels exist that have thesame light intensity level at a distance comparable to the smallestexpected fiber diameter. As for the proper directions θ of analysis, thevalues suggested by Haralick et al. [28] were selected, i.e. θ={0°, 45°,90°, 135°}. Therefore, given the fact that four pixel distances and fourangles were considered, a total number of 4×4=16 GLCMs Pδ,θ wasconsidered.
The image texture can be quantitatively described through severalstatistical descriptors, which can be calculated for each Pδ,θ matrix[28]. Four such descriptors were used in this study, namely:
contrast jδ;θ = ∑Λ
l=1∑Λ
λ=1l−λj j2⋅Pδ;θðl;λÞ; ð2Þ
Fig. 4. Schematic representation of the procedure for multiway filtering of an image according to the algorithm of Bharati et al. [12].
69P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
correlation jδ;θ = ∑Λ
l=1∑Λ
λ=1
ðl−μ lÞ⋅ðλ−μλÞ⋅Pδ;θðl;λÞσl⋅σλ
; ð3Þ
energy jδ;θ = ∑Λ
l=1∑Λ
λ=1Pδ;θðl;λÞ2; ð4Þ
homogeneity jδ;θ = ∑Λ
l=1∑Λ
λ=1
Pδ;θðl;λÞ1 + l−λj j ; ð5Þ
where μl and μλ are (respectively) the means along the row andcolumn directions of Pδ,θ, and σl and σλ are the variances along thesame directions. These features are calculated for each selected valuesof δ and θ, and therefore constitute [4 (descriptors)×4 (distancesδ)×4 (angles θ)]=64 textural parameters identifying the texture ofeach membrane image. Note that, as observed by Tessier et al. [15],the combined use of different descriptors from each distance actuallyamounts to using GLCM in a multi-resolution framework.
3.2.2. Wavelet texture analysisWavelet texture analysis (WTA) is a transform-based method that
converts the image into a new form using the spatial frequencyproperties of the pixel intensity variations. Using two-dimensionaldiscrete wavelet transform (DWT; [21]), an image (namely, Ifilt) canbe processed through low-pass and high-pass filters, which respec-tively identify the low frequency elements (called approximations)and the high frequency elements (known as details) of the originalsignal. The main operation performed by DWT is the convolution ofthe original signal Ifilt with orthonormal bases constituted bytranslated and dilated versions of an assigned wavelet function. Inthis study, Daubachies 8 wavelets were used.
Fig. 5 shows the procedure for image convolution by DWT, whichgoes through a series of steps that are carried out at each resolutionscale (i.e. spatial frequency) s; note that A0= Ifilt. First, the imageundergoes both a low-pass horizontal filter (Lh) and a high-passhorizontal filter (Hh) through which the image rows are filtered; acolumn-wise dyadic downsampling (2↓1) is performed in such a wayas to generate two images (As−1
low and As−1high ) whose column dimension
is one half of the original one. Then, the columns of these images areprocessed by vertical filters (Lv and Hv), and a row-wise dyadicdownsampling (2→1) is carried out in this case.
Therefore, at each resolution scale four downsized images areobtained (As, Ds
h, Dsv, and Ds
d), whose row and column dimensions areone half of the original ones. Image As resulting from the doubleapplication of the low-pass filters is the approximation of As−1 atscale s and contains low frequency information on the original signal.The high frequency features are stored in the three remaining images(details). The algorithm can be iterated at different resolution scales,by convolving at each scale the approximation obtained at theprevious resolution scale. Note that in a detail image, the image scene
Fig. 5. Cascaded application of low-pass and high-pass discrete wavelet filters on an image at resolution scale s.
Table 2Comparison between the GLCM model and the WTA model in the estimation of poresize (image Set #1, validation subset).
Model Number ofregressors
Number ofLVs
Number of unreliableestimations
R2 AMAPE(%)
GLCM 64 11 2 0.9841 5.4WTA 30 10 3 0.9865 3.9
Fig. 6. Comparison between the measured pore size distribution for a validation image(image Set #1) and the distributions estimated by (a) the GLCM model, and (b) theWTA model. The symbols represent the vigintiles of the distribution. The horizontalbars indicate the measurement accuracy.
70 P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
is strongly directional, i.e. it is most highlighted along a particulardirection (horizontal h, vertical v or diagonal d).
Several works have shown that WTA based on image details leadsto the development of quality monitoring models with goodclassification performances [12,29,30]). This is mostly related to thefact that details are representative of finer textures, which is what oneis most interested at when image classification is needed. Imageapproximations are usually disregarded, restricting their role tocapture the variations induced by lighting or illumination. However,much information may be stored in the image approximations whenthe textural features to be inspected are coarse, as is the case of themembranes considered in this study. For this reason, in this studyWTA focuses on image approximations only: after application of DWTat a given resolution scale, the filtered image Ifilt is reconstructed usingonly the approximation at that scale; this reconstruction is indicatedby Rs. Note that elimination of the details at the first approximationscale (D1
h, D1v, and D1
d) amounts to provide for a denoising action onIfilt; the details at scale sN1 are discarded as well, but are indirectlyincluded in image approximation at scale s−1. To characterize thetextural features of the reconstructed image at scale s, five statisticaldescriptors are calculated:
entropy j s = − ∑I−2
i=1∑K−2
k=1PsðlÞ log2½PsðlÞ�; ð6Þ
energy j s = jjRsjj; ð7Þ
standarddeviation j s = σs =
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi∑I−2
i=1∑K−2
k=1rsi;k−μ
� �2
ðI−2ÞðK−2Þ ;
vuuuut ð8Þ
skewness j s =Ms;3
σ3s
; ð9Þ
kurtosis j s =Ms;4
σ4s
: ð10Þ
In the above equations, Ps(l) is the probability of finding a pixel ofcoordinates (i,k) with light intensity l at scale s, ||Rs|| is the Frobeniusnorm of Rs, Ms,3 and Ms,4 are the 3rd and the 4th central moments ofRs, ri,ks is the [i, k] element of Rs, and μ is the mean of the light intensityin Rs. Following Tomba et al. [9], the number of decomposition scaleswas determined by analyzing the correlation between Ifilt and Rs ateach scale. Six decomposition scales were found sufficient to extractthemost considerable signs that themembranemorphology leaves onthe image scene. Because the statistical descriptors are computed ateach resolution scale, a series of [5 (descriptors)×6 (scales)]=30textural parameters can be calculated for each available image.
3.3. Interrogation of a multivariate regression model
Distinct partial least squares (PLS; [24]) regression models weredeveloped to estimate the PSD, the FDD and the permeability. The
Fig. 7. Control charts for the reliability of the estimated pore size distribution (image Set #1, validation subset): (a) GLCMmodel, and (b) WTAmodel. The numbers inside the boxesindicate the image number; the broken lines represent the confidence limits.
Table 3Comparison between the GLCMmodel and theWTAmodel in the estimation of the fiberdiameter (image Set #1, validation subset).
Model Number ofregressors
Number ofLVs
Number of unreliableestimations
R2 AMAPE(%)
GLCM 64 7 1 0.9505 4.5WTA 30 11 2 0.9544 4.1
71P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
image texture indices (Eqs. (2)–(5) in the GLCM approach, and Eqs.(6)–(10) in the WTA approach) are used as regressors (X matrices);therefore, the column dimension of X is 64 in the GLCM approach and30 in the WTA approach. The column dimension of the response (i.e.estimated property) matrix Y is 19 (number of vigintiles) for themodels that estimate the PSD or the FDD, and 1 for the model thatestimates the permeability. The appropriate number of latentvariables (LVs) to be retained into each PLS model was selected byminimizing the root-mean squared error of prediction in the relevantvalidation data subset [15,16].
To evaluate the goodness of the regression when permeability isestimated, the coefficient of determination R2 is used:
R2 = 1−∑N
n=1ðyn− ynÞ2
∑N
n=1yn−
∑N
i=1yi
N
0@
1A
2 ð11Þ
where ŷn is the permeability in the n-th membrane as estimated bythe PLS model, yn is the relevant measured value, and N is the totalnumber of membrane images for which the estimation is carried out.When a distribution is estimated (M=19 vigintiles for anymembraneof Set #1) the goodness of regression is evaluated by:
R2 =1N
∑N
n=11−
∑M
m=1yn;m− yn;m
� �2
∑M
m=1yn;m−
∑M
j=1yn;j
M
0B@
1CA
2
2666666664
3777777775; ð12Þ
where yn,m is them-th vigintile estimated by the PLSmodel in the n-thmembrane, and yn,m is the relevant measured value.
To provide additional engineering insight into the regression results,the predictive power of each regression model was evaluated by calcu-lating the average mean absolute percent error AMAPE of prediction:
AMAPE =1N
∑N
n=1
∑M
m=1
yn;m− yn;m���
���yn;m
M× 100: ð13Þ
Note that AMAPE and R2 are regression performance indicatorsthat can be evaluated only when measurements of the property to beestimated are available. However, in order to provide the estimate of aproperty for a given membrane n, the regression model would beinterrogated in real time, i.e. in the absence of measurementinformation about that property. Therefore, the reliability of theestimate should be assessed together with the value of the estimateitself. To this purpose, the squared prediction error Qn and theHotelling Tn
2 statistics [31] can be calculated for image n. Aconservative approach was taken in this study, i.e. an estimationwas classified as unreliable and discarded if eitherQn or Tn2 violated therelevant 99% confidence threshold [23]; no further investigation wasthen carried out on a deemed unreliable estimation. Note that the R2
and AMAPE values reported in the tables to follow refer to reliableestimations only.
4. Results and discussion
Image Set #1 was split into a calibration data subset of 47 imagesand a validation data subset of 15 images; the images from this setwere used for the estimation of the PSD and of the FDD. Image Set #2was split into a calibration dataset of 13 images and a validation data
subset of 5 images; the images from this set were used for theestimation of membrane permeability.
4.1. Estimation of the pore size distribution
An overall comparison between the performances of the GLCMmodel and WTA model for pore size estimation is summarized inTable 2.
The WTA model slightly outperforms the GLCM one (the meanrelative absolute error of estimation is 3.9% for the former and 5.4% forthe latter), but both models provide a very satisfactory pore sizeestimation. Note that the GLCM model uses almost the same numberof LVs as the WTA model, although the column dimension of itsregressor matrix is much larger than that of the WTAmodel. This mayindicate that the regressor variables in the GLCM model are highlycollinear or that most of them are irrelevant to the pore sizeestimation. Indeed, it was found that contrast and correlation(Eqs. (2) and (3)) are much more predictive of the pore size thanare energy and homogeneity (Eqs. (4) and (5)).
A comparison between the PSD estimation performances of the twomodels for a validation image is shown in Fig. 6. It can be seen that bothmodels do a good job in estimating the PSD. However, the regressionperformance of the WTA model (R2=0.9967) is better than that ofthe GLCM model (R2=0.9872). The GLCM method estimates quiteaccurately the pore diameters for the first ten vigintiles (Fig. 6a), but six
Fig. 8. Comparison between the measured fiber diameter distribution for one validationimage (image Set #1) and the distributions estimated by (a) the GLCM model, and (b)theWTAmodel. The symbols represent the vigintiles of the distribution. The horizontalbars indicate the measurement accuracy.
72 P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
out of the remaining nine vigintiles are estimatedwith a lower accuracythan the measurement instrument. Fig. 6b shows that the WTA modelguarantees a greater accuracy, because for all but one vigintiles theestimated cumulative distribution is well within the accuracy of themeasured profile. Note that the calculation time needed to estimate thewhole distribution is negligible for any model (less than 1 s using astandardpersonal computer). The seemingbias in thePSDprediction forboth models of Fig. 6 is not a typical prediction trend. Predictions forother membranes (i.e. images) display inflections with respect to themeasured PSD, confirming that overall there is no bias in the prediction.
To verify if the PSD estimation for a given image n is reliable, the Qn
statistic and the Hotelling Tn2 statistic need checking. Fig. 7 refers to thewhole validation subset and provides information on the reliabilityof the PSD estimations using the GLCM and the WTA models. Forthe GLCM model (Fig. 7a), the high Qn residuals for validation images#2 and #10 indicate that these images are not suitably representedby the regression model, due to a change in the correlation structurebetween the predictor indices [13]. Otherwise stated, the modelcarries out an extrapolation for these images, and their scarce con-formity to the reference dataset invalidates the estimation. It wasfound that the values of AMAPE for images #2 and #10 are the twohighest ones within the whole validation subset (10.7% and 9.2%,respectively). However, it should be noted that in general unreliabilitydoes not necessarily stand for unsatisfactory estimation accuracy.
In the case of the WTA model, the Tn2 statistics pinpoints the same
outliers; the Tn2 violation indicates that, as far as the PSD estimation is
concerned, the average texture conditions in validation images #2 and#10 are far from the average texture of the reference. An additionalunreliable estimation is found for validation image #3: the Qn
statistics indicates that the estimated PSD for this image is
extrapolated. Also for this model the PSD estimations for images #2and #10 are the worst ones within the whole validation subset(AMAPE=10.6% and 10.7%, respectively), while the value of AMAPEfor image #3 is 6.0%, which is the fifth worst in the subset.
4.2. Estimation of fiber diameter distribution
An overall comparison between the performances of the GLCMmodel andWTAmodel for fiber diameter estimation is summarized inTable 3.
Both models show very good performance, providing an estima-tion of the nanofiber diameter within 4.5% of the measured value. TheGLCM model is more parsimonious in terms of number of latentvariables; on the other hand, the WTA model has a slightly superiorestimation accuracy.
A comparison between the FDD estimation performances of theGLCM model and that of the WTA model is shown in Fig. 8 for thesame validation image as in Fig. 6. Although both models provide agood estimation of the FDD, on the whole the WTA model ensures aslightly superior estimation accuracy. The coefficient of determinationof the estimated distribution in Fig. 8a is R2=0.9660, whileR2=0.9861 for the estimated distribution in Fig. 8b.
As can be seen in Fig. 9a, the GLCM model classifies as unreliablethe FDD estimation for validation image #7 (AMAPE=4.1%), wheretheWTAmodel (Fig. 9b) alerts on two possibly unreliable estimations,namely validation images #2 (AMAPE=10.5%) and #10(AMAPE=5.0%). Note that there is a certain consistency betweenthe results of Figs. 7b and 9b, in that WTA classifies as unreliableimages #2 and #10 with respect to both PSD estimation and FDDestimation.
Fig. 9. Control charts for the reliability of the estimated fiber diameter distribution (image Set #1, validation subset): (a) GLCM model, and (b) WTA model. The numbers inside theboxes indicate the image number; the broken lines represent the confidence limits.
73P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
4.3. Estimation of permeability
The performances of the GLCM and WTA models in the estimationof permeability using images at 5000× magnification are shown inFig. 10. Both models provide a satisfactory estimation accuracy. TheWTAmodel, however, has a remarkably better performance, as can bealso seen from Table 4, where a compact view of the permeabilityestimation results is presented. On average, the membrane perme-ability estimation through the WTA model is accomplished with anerror of 2.4%, about half of that is found with the GLCM model. Notethat, since for image Set #2 the sample size for permeabilityestimation is much smaller than that used for pore and fiber diameter
characterizations, no attempt was made to assess the reliability of theestimated permeability value.
It is interesting to evaluate whether or not the estimation ofpermeability can be accomplished using images at a much lowermagnification. For this reason, image Set #2bis (containing images ofthe same membranes as in Set #2, but at 500× magnification) wasused to estimate the membrane permeability. The overall results arereported in Table 5.
No substantial loss of performance on the accuracy of thepermeability estimation is observed; rather, there is a remarkableimprovement in the regression performance of the GLCM model. Thismight be due to the fact that a wider plane of vision is more predictivefor this property of the material.
Although these results are somewhat preliminary due to thereduced dimension of membrane Set #2, they nevertheless indicatethat even images a lowmagnification scale can be used to estimate thepermeability of a nanofiber membrane. Note that the 500× magni-fication scale can be easily reached by an optical microscope, whichrequires considerably lower capital investment than a SEM, allows fora higher rate of sampling if used online, and calls for less pretreatmentof the inspected sample.
5. Conclusions
The characterization of quality in nanostructured materialsthrough the measurement of morphometric and physical propertiesis a complex task, due to the lack of adequate instrumentation, theneed for dedicated personnel, and the significant economical effort.Therefore, it is highly desirable to develop automatic tools enabling afull morphological and physical characterization of the material understudy.
In this paper, a machine vision system has been proposed for theautomatic characterization of the quality properties of nanofiberassemblies using image texture analysis. Digital images of nanofibermembranes have been used in order to estimate the pore sizedistribution, the fiber diameter distribution and the permeability ofthe membranes from statistical descriptors of the image texture. Theproposed system goes through three main steps: i) image pre-processing, ii) extraction of image textural features, and iii)interrogation of a PLS regression model for property estimation.Two alternative textural feature extraction techniques have beenconsidered, namely a GLCM approach and a WTA approach. WTA is amulti-resolution approach by design; however, the GLCM approachwas developed in such a way that it results in a multi-resolutionanalysis, too.
Themachine vision system has been tested for the characterizationof polymer nanofiber membranes produced by electrospinning. Theestimation results are very satisfactory with any of the twotechniques, with WTA slightly outperforming GLCM. Typical resultsshow that the estimated PSD and FDD are within about 5% of themeasured values, while the accuracy of the permeability estimation iseven better.
It has also been shown that, while for the estimation of the PSD andFDD a SEM image of the product needs to be available, as far aspermeability is concerned an image at a much lower magnificationscale (i.e. an optical microscope one) is sufficient to provide an
Fig. 10. Model performance for the estimation of permeability using images at 5000×magnification (image Set #2): (a) GLCM model, and (b) WTA model. The numbersinside the squares indicate the validation image number; the horizontal bars indicatethe measurement accuracy.
Table 4Comparison between the GLCM model and the WTA model in the estimation of themembrane permeability using images at 5000×magnification (image Set #2, validationsubset).
Model Number of regressors Number of LVs R2 AMAPE (%)
GLCM 64 7 0.8689 4.1WTA 30 5 0.9568 2.4
Table 5Comparison between the GLCM model and the WTA model in the estimation of themembrane permeability using images at 500× magnification (image Set #2bis,validation subset).
Model Number of regressors Number of LVs R2 AMAPE (%)
GLCM 64 9 0.9873 1.1MAWTA 30 2 0.9261 3.5
74 P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75
Author's personal copy
accurate property estimation. This indicates that, for the materialunder study, permeability is a macroscopic property whose estima-tion may take advantage from a wider plane of observation.
The computational time needed to provide a full qualitycharacterization for a given membrane is negligible (less than 1 s ona standard personal computer). Therefore, the proposed machinevision system represents a very promising step toward the develop-ment of realtime systems for effective monitoring and feedbackcontrol of nanomaterials manufacturing systems.
Acknowledgement
Financial support granted to this work by the University of Padovaunder Project # CPDR088921-2008 (“Innovative techniques formultivariate and multiscale monitoring of quality in the industrialproduction of high added-value goods”) is gratefully acknowledged.
References
[1] R. Gopal, S. Kaur, C. Yang Feng, C. Chan, S. Ramakrishna, S. Tabe, T. Matsuura,Electrospun nanofibrous polysulfone membranes as pre-filters: particulateremoval, J. Membr. Sci. 289 (2007) 210–219.
[2] J.-S. Kim, D.H. Reneker, Mechanical properties of composites using ultrafineelectrospun fibers, Polym. Compos. 20 (1999) 124–131.
[3] P. Gibson, H. Schreuder-Gibson, D. Rivin, Transport properties of porousmembranes based on electrospun nanofibers, Colloids Surf. 187–188 (2001)469–481.
[4] B.M. Min, G. Lee, S.H. Kim, Y.S. Nam, T.S. Lee, W.H. Park, Electrospinning of silkfibroin nanofibers and its effect on the adhesion and spreading of normal humankeratinocytes and fibroblasts in vitro, Biomaterials 25 (2004) 1289–1297.
[5] M. Roso, S. Sundarrajan, D. Pliszka, S. Ramakrishna, M. Modesti, Multifunctionalmembranes based on spinning technologies: the sinergy of nanofibers andnanoparticles, Nanotechnology 19 (2008) 1–6.
[6] S. Agarwal, J.H. Wendorff, A. Greiner, Use of electrospinning technique forbiomedical applications, Polymer 49 (2008) 5603–5621.
[7] E.-R. Kenawy, F.I. Abdel-Hay, M.H. El-Newehy, G.E. Wnek, Processing of polymernanofibers through electrospinning as drug delivery systems, Mater. Chem. Phys.113 (2009) 296–302.
[8] D.S. Gomes, A.N.R. da Silva, N.I. Morimoto, L.T.F. Mendes, R. Furlan, I. Ramos,Characterization of an electrospinning process using different PAN/DMF con-centrations, Polím. Ciênc. Tecnol. 17 (2007) 206–211.
[9] E. Tomba, P. Facco, M. Roso, M. Modesti, F. Bezzo, M. Barolo, Artificial visionsystem for the automatic measurement of inter-fiber pore characteristics andfiber diameter distribution in nanofiber assemblies, Ind. Eng. Chem. Res. 49(2010) 2957–2968.
[10] R.C. Gonzales, R.E. Woods, Digital Image Processing, Prentice-Hall, Inc., UpperSaddle River, New Jersey (U.S.A.), 2002.
[11] P. Geladi, H. Grahn, Multivariate Image Analysis, JohnWiley & Sons, Inc., New York(U.S.A.), 1996.
[12] M.H. Bharati, J.J. Liu, J.F. MacGregor, Image texture analysis: methods andcomparisons, Chemom. Intell. Lab. Syst. 72 (2004) 57–71.
[13] J.M. Prats-Montalbàn, A. Ferrer, Integration of colour and textural information inmultivariate image analysis: defect detection and classification issues, J. Chemom.21 (2007) 10–23.
[14] S. Borah, E.L. Hines, M. Bhuyan, Wavelet transform based image texture analysisfor size estimation applied to the sorting of tea granules, J. Food Eng. 79 (2007)629–639.
[15] J. Tessier, C. Duchesne, C. Gauthier, G. Dufour, Estimation of alumina content ofanode cover materials using multivariate image analysis techniques, Chem. Eng.Sci. 63 (2008) 1370–1380.
[16] S. Kucheryavski, I. Belyaev, S. Fominykh, Estimation of age in forensic medicineusing multivariate approach to image analysis, Chemom. Intell. Lab. Syst. 97(2009) 39–45.
[17] A.L. Andrady, Science and Technology of Polymer Nanofibers, John Wiley & SonsInc., New York (U.S.A.), 2008.
[18] G.C. Rutledge, S.V. Fridrikh, Formation of fibers by electrospinning, Advanceddrug. delivery reviews 59 (2007) 1384–1391.
[19] P.R. Johnston, Fluid Sterilization by Filtration-Third Edition, CRC Press LLC, BocaRaton, FL (U.S.A.), 2004.
[20] S.G. Mallat, A theory for multiresolution signal decomposition: the waveletrepresentation, IEEE Trans. Pattern Anal. Mach. Intell. 11 (1989) 674–693.
[21] P.S. Addison, The Illustrated Wavelet Transform Handbook, Taylor & Francis, NewYork (U.S.A.), 2002.
[22] M.H. Bharati, J.F. MacGregor, Multivariate image analysis for real-time processmonitoring and control, Ind. Eng. Chem. Res. 37 (1998) 4715–4724.
[23] J.E. Jackson, A User's Guide to Principal Components, John Wiley & Sons Inc., NewYork (U.S.A.), 1991.
[24] P. Geladi, B. Kowalski, Partial least-squares regression: a tutorial, Anal. Chim. Acta185 (1986) 1–17.
[25] P. Nomikos, J.F. MacGregor, Monitoring batch processes using multiway principalcomponent analysis, AIChE J. 40 (1994) 1361–1375.
[26] A.A. Green, M. Berman, P. Switzer, M.D. Craig, A transformation for orderingmultispectral data in terms of image quality with implications for noise removal,IEEE Trans. Geosci. Remote Sens. 26 (1988) 65–74.
[27] R. Larsen, Decomposition using maximum autocorrelation factors, J. Chemom. 16(2002) 427–435.
[28] R.M. Haralick, K. Shanmugan, I. Dinstein, Textural features for image classification,IEEE Trans. Syst. Man Cybern. 3 (1973) 610–621.
[29] J.J. Liu, J.F. MacGregor, C. Duchesne, G. Bartolacci, Flotation froth monitoring usingmultiresolutional multivariate image analysis, Min. Eng. 18 (2005) 65–76.
[30] J.J. Liu, J.F. MacGregor, On the extraction of spectral and spatial information fromimages, Chemom. Intell. Lab. Syst. 85 (2007) 119–130.
[31] B.M. Wise, N.B. Gallagher, The process chemometrics approach to processmonitoring and fault detection, J. Process Control 6 (1996) 329–348.
75P. Facco et al. / Chemometrics and Intelligent Laboratory Systems 103 (2010) 66–75