postharvest biology and technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... ·...

12
Contents lists available at ScienceDirect Postharvest Biology and Technology journal homepage: www.elsevier.com/locate/postharvbio Detection of blueberry internal bruising over time using NIR hyperspectral reectance imaging with optimum wavelengths Shuxiang Fan a,b , Changying Li b, ,1 , Wenqian Huang a , Liping Chen a a Beijing Research Center of Intelligent Equipment for Agriculture, Beijing, China b School of Electrical and Computer Engineering, College of Engineering, University of Georgia, Athens Georgia, United States ARTICLE INFO Keywords: Blueberry Bruise Classication Hyperspectral imaging ABSTRACT Early detection of internal bruising is one of the major challenges in blueberry postharvest quality sorting processes. The potential of using near infrared (NIR) hyperspectral reectance imaging (9501650 nm) with reduced spectral features was investigated for blueberry internal bruising detection 30 min to 12 h after me- chanical impact. A least squares support vector machine (LS-SVM) was used to develop classication models to compute the spatial distribution of bruising based on the spectra extracted from regions of interest (ROIs) at four measurement times (30 min, 2 h, 6 h, and 12 h after mechanical impact). Three feature selection methods were used to select optimum wavelengths or band ratio images for bruising detection. The classication model, de- veloped using optimum wavelengths selected by competitive adaptive reweighted sampling (CARS) (CARS-LS- SVM model) and full spectra (full spectra-LS-SVM), had similar performance in the identication of bruised blueberries. Band ratio images (1235 nm/1035 nm) achieved a comparable accuracy with the CARS-LS-SVM model at 6 h, and higher accuracy than CARS-LS-SVM and full spectra-LS-SVM models at 12 h. The overall classication accuracies of 77.5%, 83.8%, 92.5%, and 95.0% were obtained by band ratio images for blueberries 30 min, 2 h, 6 h, and 12 h after impact, respectively. In order to evaluate the performance of the proposed methods, additional validation samples were processed by the detection algorithm. The overall discrimination accuracies for healthy and bruised blueberries in the validation set were 93.3% and 98.0%, respectively, for CARS-LS-SVM model, and 93.3% and 95.9%, respectively, for band ratio images. The overall results indicated that NIR reectance imaging can detect blueberry internal bruising as early as 30 min after mechanical impact, and band ratio images computed from two wavelengths showed great potential to detect blueberry internal bruising on the packing line. 1. Introduction Blueberries are soft and prone to mechanical damage during har- vesting and transportation (Xu et al., 2015; Li et al., 2010). Internal bruising caused by mechanical impact could accelerate spoilage, reduce fruit quality, and elevate the risk of pathogen infection, thus negatively aecting consumer purchasing decisions and prot margins for the fruit industry (Hu et al., 2016; Leiva-Valenzuela et al., 2013). Additionally, bruised berries can be neither stored nor transported for prolonged time since a small number of bruised fruit can induce pathogens and po- tentially spread to other healthy fruit (Jiang et al., 2016). Therefore, nondestructive inspection techniques are needed to eectively detect blueberry internal bruising as early as possible. As the skin of blueberry is opaque to most visible light and does not permit one to easily see tissues under the skin, the bruised region below the skin is not visible to the human eyes in the early stage of bruising. Therefore, it is a challenging task to use traditional RGB images to accurately detect blueberry internal bruises. Over the past decades, hyperspectral imaging (HSI) technology, which integrates the ad- vantages of imaging technology and spectral analysis, has emerged as a powerful technique for detecting external qualities of food and agri- cultural products (Zhang et al., 2014). However, due to the long ac- quisition time, high dimensional data, and high costs, it is not feasible for HSI systems to be applied directly for online automated detection. Instead, HSI systems can be used to select optimum wavelengths for multispectral imaging systems, which are lower in cost and have faster computing speeds than HSI systems. Multispectral imaging systems based on HSI-selected wavelengths have been used for online applica- tions, such as bruise inspection of apples (ElMasry et al., 2008) and pears (Lee et al., 2014), and defect detection in apples (Mehl et al., http://dx.doi.org/10.1016/j.postharvbio.2017.08.012 Received 27 April 2017; Received in revised form 11 August 2017; Accepted 13 August 2017 Corresponding author at: 712F Boyd Graduate Studies, 200 D. W. Brooks Drive, University of Georgia, Athens, GA, 30602, United States. 1 Website: http://sensinglab.engr.uga.edu/. E-mail address: [email protected] (C. Li). Postharvest Biology and Technology 134 (2017) 55–66 Available online 01 September 2017 0925-5214/ © 2017 Published by Elsevier B.V. MARK

Upload: others

Post on 14-May-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

Contents lists available at ScienceDirect

Postharvest Biology and Technology

journal homepage: www.elsevier.com/locate/postharvbio

Detection of blueberry internal bruising over time using NIR hyperspectralreflectance imaging with optimum wavelengths

Shuxiang Fana,b, Changying Lib,⁎,1, Wenqian Huanga, Liping Chena

a Beijing Research Center of Intelligent Equipment for Agriculture, Beijing, Chinab School of Electrical and Computer Engineering, College of Engineering, University of Georgia, Athens Georgia, United States

A R T I C L E I N F O

Keywords:BlueberryBruiseClassificationHyperspectral imaging

A B S T R A C T

Early detection of internal bruising is one of the major challenges in blueberry postharvest quality sortingprocesses. The potential of using near infrared (NIR) hyperspectral reflectance imaging (950–1650 nm) withreduced spectral features was investigated for blueberry internal bruising detection 30 min to 12 h after me-chanical impact. A least squares support vector machine (LS-SVM) was used to develop classification models tocompute the spatial distribution of bruising based on the spectra extracted from regions of interest (ROIs) at fourmeasurement times (30 min, 2 h, 6 h, and 12 h after mechanical impact). Three feature selection methods wereused to select optimum wavelengths or band ratio images for bruising detection. The classification model, de-veloped using optimum wavelengths selected by competitive adaptive reweighted sampling (CARS) (CARS-LS-SVM model) and full spectra (full spectra-LS-SVM), had similar performance in the identification of bruisedblueberries. Band ratio images (1235 nm/1035 nm) achieved a comparable accuracy with the CARS-LS-SVMmodel at 6 h, and higher accuracy than CARS-LS-SVM and full spectra-LS-SVM models at 12 h. The overallclassification accuracies of 77.5%, 83.8%, 92.5%, and 95.0% were obtained by band ratio images for blueberries30 min, 2 h, 6 h, and 12 h after impact, respectively. In order to evaluate the performance of the proposedmethods, additional validation samples were processed by the detection algorithm. The overall discriminationaccuracies for healthy and bruised blueberries in the validation set were 93.3% and 98.0%, respectively, forCARS-LS-SVM model, and 93.3% and 95.9%, respectively, for band ratio images. The overall results indicatedthat NIR reflectance imaging can detect blueberry internal bruising as early as 30 min after mechanical impact,and band ratio images computed from two wavelengths showed great potential to detect blueberry internalbruising on the packing line.

1. Introduction

Blueberries are soft and prone to mechanical damage during har-vesting and transportation (Xu et al., 2015; Li et al., 2010). Internalbruising caused by mechanical impact could accelerate spoilage, reducefruit quality, and elevate the risk of pathogen infection, thus negativelyaffecting consumer purchasing decisions and profit margins for the fruitindustry (Hu et al., 2016; Leiva-Valenzuela et al., 2013). Additionally,bruised berries can be neither stored nor transported for prolonged timesince a small number of bruised fruit can induce pathogens and po-tentially spread to other healthy fruit (Jiang et al., 2016). Therefore,nondestructive inspection techniques are needed to effectively detectblueberry internal bruising as early as possible.

As the skin of blueberry is opaque to most visible light and does notpermit one to easily see tissues under the skin, the bruised region below

the skin is not visible to the human eyes in the early stage of bruising.Therefore, it is a challenging task to use traditional RGB images toaccurately detect blueberry internal bruises. Over the past decades,hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology and spectral analysis, has emerged as apowerful technique for detecting external qualities of food and agri-cultural products (Zhang et al., 2014). However, due to the long ac-quisition time, high dimensional data, and high costs, it is not feasiblefor HSI systems to be applied directly for online automated detection.Instead, HSI systems can be used to select optimum wavelengths formultispectral imaging systems, which are lower in cost and have fastercomputing speeds than HSI systems. Multispectral imaging systemsbased on HSI-selected wavelengths have been used for online applica-tions, such as bruise inspection of apples (ElMasry et al., 2008) andpears (Lee et al., 2014), and defect detection in apples (Mehl et al.,

http://dx.doi.org/10.1016/j.postharvbio.2017.08.012Received 27 April 2017; Received in revised form 11 August 2017; Accepted 13 August 2017

⁎ Corresponding author at: 712F Boyd Graduate Studies, 200 D. W. Brooks Drive, University of Georgia, Athens, GA, 30602, United States.

1 Website: http://sensinglab.engr.uga.edu/.E-mail address: [email protected] (C. Li).

Postharvest Biology and Technology 134 (2017) 55–66

Available online 01 September 20170925-5214/ © 2017 Published by Elsevier B.V.

MARK

Page 2: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

2004), peaches (Zhang et al., 2015), oranges (Li et al., 2013) and jujube(Yu et al., 2014a). Multispectral vision systems have been developed tosort apples according to the presence of defects (Kleynen et al., 2005)and bruise (Huang et al., 2015), and to detect citrus canker (Qin et al.,2012) based on the selected wavelengths from HSI systems (Kleynenet al., 2003; Qin et al., 2009). In addition, two-band ratio images fol-lowed by image segmentations using a threshold value is an effectiveand simple image processing method which could accelerate the imageacquisition and analysis process on a commercial line, enhance thecontrast between the defects and healthy skin as well as produce moreuniform responses across the fruit surface (Kim et al., 2007). Thismethod has been successfully used to detect fecal contaminants onapples (Liu et al., 2007), physical damages of pears (Lee et al., 2014),bruises in cucumbers (Ariana et al., 2006), bacterial diseases in onions(Wang et al., 2012a), and common defects of peaches (Li et al., 2016).These studies suggest that it is reasonable to develop an online multi-spectral imaging system that is based on optimum wavelengths ob-tained from an HSI system, and uses image analysis methods.

Spectral analyses of hyperspectral images to detect blueberry in-ternal bruising have been conducted in a few published papers. Onestudy found that although hyperspectral transmittance showed rela-tively superior performance to reflectance and interactance, it wasdifficult to distinguish fresh damage (inflicted within 24 h) in blue-berries using the transmittance mode according to classification resultsbased on the mean spectra of each fruit (Hu et al., 2016). The previouswork conducted in our lab has demonstrated that liquid crystal tunablefilter (LCTF) based hyperspectral reflectance imaging over the spectralrange of 950–1650 nm, combined with a classification model of thespectra from the regions of interest (ROIs), could be implemented todetect blueberry bruises 24 h after impact (Jiang et al., 2016). Zhangand Li (2016) found that it was feasible to threshold the spectral in-tensity of hyperspectral transmittance images at 1070 nm for blueberryinternal bruising detection. Jiang et al. (2016) mentioned that the calyxend of each fruit should be excluded in hyperspectral imaging analysisbecause of its negative effect on the classification accuracy. However,Jiang et al. (2016) assumed that the calyx end was in the middle of thefruit, which is not always the case in practice. Thus, a new algorithmshould be proposed to identify the calyx end of each fruit automatically.In addition, early bruise (less than 12 h) detection should be con-sidered, and optimum wavelengths in 950–1650 nm need to be de-termined and tested for bruise detection of blueberries.

The overall goal of this study was to examine the potential of LCTF-based hyperspectral reflectance imaging for the detection of earlyblueberry bruising. Specific objectives were to: (1) develop classifica-tion models for internal bruise detection using the spectra extractedfrom the ROIs of blueberry samples with four measurement times(30 min, 2 h, 6 h, and 12 h after mechanical impact), (2) evaluate

spatial distribution of the bruises over the entire fruit surface and cal-culate the bruise ratio based on the established classification models onthe spectral images, and (3) identify and evaluate optimum wave-lengths and band ratio images that are most useful for blueberry bruisedetection.

2. Materials and methods

2.1. Sample preparation

Fresh blueberries, of the Jersey variety, harvested in Michigan, USAin August, 2016 were packed in clamshells and put in a cooler with icepacks. After being shipped to the Bio-Sensing and Instrumentation Labat the University of Georgia, all of the samples were stored in a re-frigerator at 5.5 °C until the experiments were conducted.

All of the tested blueberries were taken out of the refrigerator andstored in the air-conditioned laboratory for 2 h before treatment (bruiseor control) to allow the samples to reach room temperature (20 °C). Inour experiment, a 14.8 g steel ball mounted onto a wooden pendulumwhich weighted 29.0 g, was designed to create bruising (the steel ballwas dropped onto the sample from a height of 90 mm) in a specificposition on each blueberry’s surface (see Supplementary Fig. S1). Theimpact energy was about 0.021 J. Because of the soft surface of blue-berries, the pendulum was caught after rebounding off the blueberrysample to avoid additional impact. The holder, made of silicone rubber,was used to immobilize the blueberry as well as to avoid damage at thebottom of the fruit when the bruise was created. In order to evaluate theeffect of time after bruising on detection efficiency, four sets of blue-berries (80 blueberries in each set) were stored in the laboratory for30 min, 2 h, 6 h, and 12 h after bruise creation, respectively, beforethey were imaged with a hyperspectral reflectance imaging system.

For each of the four measurement times, 80 fruit were prepared;therefore, 320 samples were used to collect hyperspectral data. The 80fruit in each measurement time were divided equally into three bruisegroups (stem bruise, equator bruise, and calyx bruise) and one controlgroup (no bruise treatment), with 20 fruit in each group. It should benoted that, compared with other parts of the blueberry, it is harder tocreate bruises at the calyx end due to the vestigial sepals around thecalyx, which can reduce the impact of mechanical damage on the calyxend. The bruises were therefore created under the sepals for calyxbruise groups in this study (Fig. 1b). The fruit were arrayed on a blackcardboard holder (Fig. 1a) in groups of 20 arranged in 5 columns and 4rows (Fig. 1b), with 12 of the fruit to be used for a calibration set (leftside of array) and the remaining 8 fruit to be used for a prediction set(right side of the array).

Fig. 1. Schematic of the LCTF-based hyperspectral reflectance imagingsystem (a) and orientations used for acquiring images of blueberry fruits(b). The red ovals indicate the bruise positions. The spectra of the samplesto the left and right of the green dotted line were used as the calibrationset and prediction set, respectively. (For interpretation of the references tocolour in this figure legend, the reader is referred to the web version of thisarticle.)

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

56

Page 3: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

2.2. Acquisition of hyperspectral images

An NIR hyperspectral imaging system has been set up in our la-boratory and employed for measuring hyperspectral reflectance imagesof blueberries (Fig. 1a). The main components of the system are anLCTF (Model Varispec LNIR 20HC20, Cambridge Research &Instrumentation, MA, USA), an indium gallium arsenide (InGaAs)camera (320 × 256 pixels, 25 μm pitch, Model SUI320KTS1.7RT,GOODRICH, Sensors Unlimited, Inc, NJ, USA) with a 12-bit digitalCamera Link compatible output, and an optical focusing lens (Nikkor50 mm f/1.4D AF, Nikon, Japan) (Wang et al., 2012b). The illuminationunit consisted of two 20-W halogen lamps adjusted manually at anangle of about 45° to illuminate the camera’s field of view. The hy-perspectral image acquisition was controlled by software developed inLabVIEW (National Instruments, Austin, TX) which was installed on acomputer (OptiPlex 620, Intel Pentium D, 2.80 GHz, Dell, USA). All ofthe components (except the computer) were fixed inside a darkchamber to avoid stray light that might affect the reflectance of thesamples.

During image acquisition in our experiment, the 20 fruit from thesame bruise group and measurement time were placed on one speciallydesigned black cardboard holder with the bruised surfaces facing up.However, for the control groups, four fruit orientations (stem, calyx,equator, back equator) were taken into consideration with five samplesfor each orientation (Fig. 1b). Each 3D hyperspectral image (Rraw) wasconstructed with two spatial dimensions (a 320 × 256 pixels imageplane) and one spectral dimension (141 wavelengths ranging from 950to 1650 nm with a spectral interval of 5 nm). In total, 16 hyperspectralimages were acquired, with four images (20 fruit in each image) foreach measurement time (30 min, 2 h, 6 h, and 12 h).

Prior to collecting images, the white reference image (Rw) was ac-quired from a Teflon white panel (SRT-99-050, Labsphere Inc., NorthSutton, NH, USA) with a reflectance of 99%, and the dark referenceimage (Rd) was obtained with the lamps turned off and the optical lenscompletely covered by its cap. The raw hyperspectral images (Rraw)were corrected with the white and dark reference images to remove thenoise caused by the dark current of the camera as well as artifactscaused by non-uniform illumination. The corrected image (Rc) wascalculated according to the following equation:

=−

−R R R

R Rcraw d

w d (1)

In order to verify the performance of the detection algorithm foridentifying blueberry bruises, an additional 64 blueberry fruit wereused as an independent data set. Similarly, four measurement times(30 min, 2 h, 6 h, and 12 h) were evaluated. For each measurementtime, 16 fruit were prepared and divided equally into four groups (stembruise, equator bruise, calyx bruise, and control) with 4 fruit in eachgroup.

2.3. Reference measurements

It takes time for bruises to become discolored due to oxidativebrowning so that they are visible to the human eye. Therefore, blue-berries were cut in half perpendicular to the bruise position to observethe bruising condition of the blueberry fruit 1 h after being scanned bythe HSI system. Then the sliced samples were imaged with a digitalcamera (D40, Nikon, Tokyo, Japan) under ambient illumination. Thenumber of pixels of the discolored (bruised) area and the number ofpixels of the entire cross-section of the sliced blueberry were calculatedaccording to these images. The ratio between the two numbers of pixelswas used as the measured bruise ratio index of the tested sample. Theseoperations were performed in Environment for Visualizing Imagessoftware (ENVI 4.7, Exelis Inc., VA, USA). Furthermore, the sampleswere classified into two groups based on the standard proposed in ourprevious work (Yu et al., 2014b): depending on whether the discolored

area accounted for greater or less than 25% of the pixels of the slicedsurface, the blueberries were classified as bruised or healthy, respec-tively.

2.4. Preprocessing of hyperspectral images

2.4.1. Mask imageIt was necessary to remove the image background before further

hyperspectral image analysis. Firstly, a mask template of each imagewas created by thresholding a single-band grayscale image at 1075 nmbecause the contrast between the fruit and the background in this imagewas most prominent. After refining by removing small objects, the re-fined image was used as a mask to segment individual samples from thebackground. The acquisition of mask images was processed in MATLAB2016b (The MathWorks Inc., MA, USA). Given that the spectral profileof the calyx end was similar to that of bruised tissues, the calyx end wasalways misclassified as bruised tissue and needed to be excluded inimage classification. After some trial and comparison, the two-bandratio between 1200 nm and 1075 nm (1200 nm/1075 nm) and athreshold value (0.6) were used to recognize the sepal positions, whichhad much higher spectral intensity than the other tissues across asample. The calyx end was therefore identifiable because it was sur-rounded by sepal in the image.

2.4.2. Spectra extractionThe region of interest (ROI) function of ENVI software was used to

select the bruised and healthy ROIs on the grayscale images at 1200 nmbecause of the obvious contrast between the healthy tissues and bruisedtissues in this image (Fig. 2). The ROIs were manually drawn onblueberry fruit and the spectrum of each pixel within the ROI was ex-tracted and saved. For each of four images collected in one measure-ment time, the extracted spectra from the left 12 fruit were used as thecalibration set, and the spectra from the remaining 8 samples on theright side were assigned as the prediction set (Fig. 1b). The number ofspectra extracted from different measurement times were summarizedin Table 1. The raw spectra were preprocessed by mean normalizationprior to the development of classification models:

∑′ =

=

SS

λ

m i

11

ii

i (2)

where Sλi is the spectral intensity of a single pixel in the hyperspectralimage at wavelength λi, and m is the total number of wavelengths inone spectrum. Sλ

'i represents the normalized spectral intensity of the

pixel at wavelength λi.

2.4.3. Classification modelAs a powerful methodology in pattern recognition and function

estimation, a least squares support vector machine (LS-SVM) was usedto classify healthy and bruised tissues. An LS-SVM is a reformulation ofa standard SVM, and is capable of dealing with linear and nonlinearmultivariate analysis (Yu et al., 2014a). The use of LS-SVM could re-duce the complexity of calculation and shorten the amount of time

Fig. 2. RGB and grayscale images at 1200 nm of blueberries 12 h after impact.

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

57

Page 4: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

required, hence improving the ability to study high-dimensionalitycharacteristic space. The radial basis function (RBF) was selected as thekernel function used for LS-SVM in our study as this kernel functionusually deals with the nonlinear relationship between the spectra andtarget class and had good performance. All the operations of LS-SVMwere performed in MATLAB with the free LS-SVM v1.8 toolbox (Suy-kens, Leuven, Belgium). The starting points of tuning parameters, in-cluding the regularization parameter γ and RBF kernel parameter σ2,were determined by coupled simulated annealing (CSA) method, andthen these starting points were given to the grid search optimizationalgorithm. The grid search technique, combined with 10-fold crossvalidation was performed to calculate misclassification values for everypossible combination of γ and σ2 in the two-dimensional grid. Thus,according to the minimum of the misclassification value, the optimalcombinations of γ and σ2 were determined and used to develop thecalibration model.

LS-SVM was applied to build the calibration models for bruise de-tection using the extracted spectra from ROIs. The combined model hassuccessfully reduced the effect of temperature fluctuation (Peirs et al.,2003a), biological variability (Peirs et al., 2003b), and measurementposition (Fan et al., 2016) on the performance of near-infrared (NIR)models for soluble solids content (SSC) of apple and eliminated thetemperature influence on SSC detection of watermelon juice by NIRtechnology (Yao et al., 2013). In our study, a combined time model wasbuilt by LS-SVM to try to effectively detect bruising in all of the mea-surement times. In order to balance the number of spectra of bruisedtissues and healthy tissues, a total of 1200 spectra—including 600spectra of healthy tissues and 600 spectra of bruised tissues—wererandomly selected from each of the four calibration sets and combined.Therefore, 4800 spectra were used as a calibration set to establish acombined classification model. The prediction sets measured at dif-ferent times were utilized to verify the detection power of the estab-lished model.

Based on the established classification model, every pixel on thesurface of the individual fruit sample was classified and the pixelclassifications were used to identify bruise positions of individualsamples, creating a classification map of bruised and healthy tissue.According to the classification map, the predicted bruise ratio index foreach sample was calculated and the sample was graded into healthy orbruised group accordingly.

2.4.4. Wavelength selection methodsWavelength selection is an important step in processing hyper-

spectral images due to the high dimensionality and co-linearity of hy-perspectral images and the limitations of computer hardware. Anotherbenefit of wavelength selection is to eliminate irrelevant variables toimprove model performance. Therefore, it is very important to selectoptimum wavelengths that may perform equally well or even betterthan using full spectra for identifying the bruised regions in blueberries.Two representative wavelength selection methods were applied to se-lect optimum wavelengths and compared in their performance onblueberry bruise detection: competitive adaptive reweighted sampling(CARS) and successive projections algorithm (SPA). All wavelengthselection methods were performed on the calibration set by toolboxes inMATLAB and the parameters used in the toolbox were set to default

values.

2.4.4.1. Competitive adaptive reweighted sampling. CARS has thepotential to select an optimal combination of the effective variablesexisting in the full spectrum coupled with partial least square (PLS)regression (Li et al., 2009). Combined with classification methods suchas PLS and LS-SVM, CARS has been successfully used to select optimumvariables for the determination of SSC and dry matter of pears (Traverset al., 2014) and total viable count of salmon flesh (Wu and Sun, 2013).CARS selects N subsets of wavelengths by N sampling runs in aniterative manner. In each sampling run, some samples are firstrandomly chosen in a fixed ratio to build a PLS model. Depending onthe absolute value of regression coefficient for each wavelength, asubset of wavelengths is determined and then retained for the nextsampling run. After N sampling runs, the subset with the lowest rootmean-square error of cross validation (RMSECV) is chosen. Theprocedure of CARS was performed in MATLAB with the libPLStoolbox available at http://www.libpls.net/.

2.4.4.2. Successive projections algorithm. SPA was initially proposed byAraújo et al. (2001) as a technique for variable selection for multiplelinear regression (MLR). Previous studies have indicated that SPA is avery good algorithm for variable selection and can be used incombination with different multivariate calibration methods such asPLS, MLR (Liu et al., 2009), and LS-SVM (Cheng and Sun, 2015) to buildthe linear and nonlinear models in spectral analysis. SPA is a forwardselection method which starts with one wavelength and incorporates anew one at each iteration until a specified number, N, of wavelengths isreached. The main purpose of this algorithm is to select wavelengthswhose information content is minimally redundant. The optimalnumber of variables can be determined on the basis of the smallestmean square error of cross validation (MSECV) in the validation set ofMLR calibration. A graphical user interface for the SPA (GUI_SPA) isavailable at http://www.ele.ita.br/∼kawakami/spa/.

2.4.5. Band ratio imageThis study attempted to use an image ratio method to develop a

bruise detection algorithm for a low-cost and real-time imaging system.A statistical comparison method—one-way ANOVA test—was used toselect the two-band ratio for classifying the bruised and healthy tissuesby comparing the differences of two-band ratios between bruised andhealthy spectra. Each piece of spectrum has 141 wavelengths, so thereare (141 × 141) possible band ratios of two wavelengths. According tothe highest obtained F-value of all possible two-band ratios for thespectra of healthy and bruised tissues, the optimal band ratio was se-lected because a larger F-value indicated a more statistically significantmean separation of the two-band ratio between the bruised and healthytissue spectra. Following band ratio selection, the optimal thresholdwas determined at the point of the highest classification accuracies forboth healthy spectra and bruised spectra. The spectral data in the ca-libration set (4800 spectra) were used to select the optimal two-bandratio for classifying bruised and healthy tissues. ANOVA test was con-ducted using SAS 9.4 (SAS Institute, Cary, NC).

Table 1Number of spectra in calibration and prediction sets of different measurement times.

Measurement times No. of spectra in calibration set No. of spectra in prediction set

Total Sound% Bruised% Total Sound% Bruised%

30 min 2545 50.2% 49.8% 1608 53.7% 46.3%2 h 2789 49.4% 50.6% 2231 25.0% 75.0%6 h 4063 47.8% 52.2% 2025 28.1% 71.9%12 h 3177 49.7% 50.3% 2403 46.9% 53.1%

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

58

Page 5: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

2.5. Algorithm for blueberry bruise detection

A detailed flow chart of the multispectral bruise detection algorithmis shown in Fig. 3. First, multispectral images were acquired at optimumwavelengths that were determined by wavelength selection or bandratio. Afterward, a mask template was created based on a single-bandimage at 1075 nm using a simple threshold value, while a ratio Image1200 nm/1075 nm was used to identify the calyx position. Next, themask image without calyx position was applied to the multispectralimages to remove the background and calyx position. For spectralanalysis, after mean normalization preprocessing, the multispectralimages were used as the inputs of the established classification model toevaluate spatial distribution of the bruises over the entire fruit surface,while for image analysis, two-band ratio image was obtained and thenthe classification map was calculated using threshold value. Finally, thebruised samples were identified according to their predicted bruiseratio indexes.

3. Results and discussion

3.1. Spectra analysis

Two obvious absorption peaks around 970 and 1200 nm, which areassociated with the absorption of water, were observed in the raw re-flectance spectra of blueberries (Fig. 4). The intensity values of the

reflectance spectra of healthy tissues were higher than those of bruisedtissues and the same trend was observed for all measurement times(30 min, 2 h, 6 h, and 12 h after bruising) mainly due to the rupture ofthe cell membranes. Thus, bruised tissues contained more free water inthe intercellular space and the ruptured cell membranes could affectlight scattering in the fruit tissue. The results were also in agreementwith findings in apples (Luo et al., 2012), strawberries (Nanyam et al.,2012), and pears (Lee et al., 2014). According to ANOVA test on thetwo absorption peaks around 970 and 1200 nm, the mean spectra ofhealthy and bruised tissues were statistically separable for each timetreatment, suggesting that the differences of spectral data betweenhealthy and bruised tissues could be used to detect bruised tissues re-lying on chemometric methods. It was also noticed that as the bruisingtime increased, the differences between healthy and bruised tissue re-flectance became more pronounced throughout the 950–1450 nm re-gion, suggesting that bruises were more easily distinguished by thespectral signature over time.

3.2. Classification results

Before the development of classification model, the spectral datawere processed by mean normalization. The overall accuracy using onlythe pixels within ROIs was over 95%, showing that the establishedmodel could effectively distinguish bruised tissues from healthy tissues(Table 2). However, it should be noted that these performances are

Fig. 3. Flow chart of algorithms using multispectral imaging for identifying blueberry internal bruising.

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

59

Page 6: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

based on the spectral data extracted from the selected ROIs and theperformance should also be evaluated on all the pixels on the entireberry surface that was visible to the camera. Therefore, the next im-portant step of analyzing hyperspectral reflectance images was toevaluate spatial distribution of the bruises over the entire fruit surface,which is the main advantage of hyperspectral reflectance imaging overthe traditional near infrared spectroscopy technology.

In this study, assessment of spatial distribution of bruising on thesurface of a berry was carried out by classifying each pixel of theblueberry sample using the established models, and then obtaining theclassification map and bruise ratio indexes for blueberry samples. Theblueberry has dark pigments in the skin, so it is difficult to identifybruised tissues with the naked eye, unless by slicing the fruit and ob-serving the discolored area on the cross section. The internal tissuesunderneath the area where external damage occurred become dis-colored because of oxidative browning, a result of the release of en-zymes during rupture of cell membranes (Mitsuhashi-Gonzalez et al.,2010).

Regression analyses between image-predicted and visually scoredbruising at four different measurement times showed that the imagingmethod followed a similar trend as the visual observation and thecorrelation became stronger as time advanced (Fig. 5). The results de-monstrated the effectiveness of the imaging method in detecting

internal bruising for the entire fruit surface as early as 30 min aftermechanical impact (R2 = 0.7). The increased exuding of cell liquid inthe bruised tissues after impact might explain stronger absorption inbruised tissues over time, resulting in decreases of bruised tissue re-flectance and more pronounced differences between healthy andbruised tissue reflectance (Fig. 4). These increasingly pronounced dif-ferences could explain why the correlation between visually scored andimage predicted bruise ratio indexes became stronger over time (Fig. 5).

Bruises were detected in some of the fruit in the control group byboth the imaging method and visual scoring, but the bruise ratio in-dexes detected by the imaging method were significantly higher thanthose reported by the visual scoring method for several samples. Thislikely occurred because some control samples were moderately bruisedduring transportation prior to the experiments, despite the deliberateeffort to select the most bruise-free samples for the control group. Thesebruises in the control samples were generally minor and close to theskin, but not present in the flesh. Therefore, visual scoring was not ableto detect these minor close-to-the-skin bruises, but the imaging methodwas able to detect them, suggesting that the imaging method could bemore sensitive to internal bruising at early stages than the visualscoring method.

On the other hand, the imaging method estimated lower bruise ratioindexes than the visual scoring method for most samples in the calyxbruise groups of all measurement times (Fig. 5). As the calyx ends wereexcluded in the image processing pipeline, the imaging method couldunderestimate internal bruising if bruises developed under the calyxand were identified on the cross section by visual scoring. If the calyxbruise group and several outliers from the control group were excludedfrom the data set, the RMSE of bruise ratio index obtained by theclassification model would be reduced to 0.137, 0.126, 0.105, and0.100 for blueberries 30 min, 2 h, 6 h, and 12 h after impact,

Fig. 4. Mean reflectance spectra (solid line) and standard deviation (error bar) before preprocessing of bruised tissues and healthy tissues over time: (a) 30 min; (b) 2 h; (c) 6 h; and (d)12 h.

Table 2Classification accuracy on the spectra of ROIs.

Calibration set accuracy Prediction set accuracy

30 min 2 h 6 h 12 h

100% 95.77% 99.06% 99.11% 98.79%

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

60

Page 7: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

respectively. Moreover, there were 39 (65%), 33 (55%), 50 (83.3%),and 36 (60%) blueberries whose bruise ratio indexes were over-estimated by the imaging method at 30 min, 2 h, 6 h, and 12 h, re-spectively. This confirmed again that the imaging method tended to bemore sensitive than the visual scoring method in detecting surfacebruises if the calyx group was not considered.

As the bruise gradually propagates from the outer surface to theinner flesh after impact, some bruises have not become discolored onthe sliced plane of the bruised samples in the early stage of the bruisingprocess (e.g. 30 min after impact). This was proved by the ANOVA testson the visually scored bruise ratio indexes of blueberry samples frombruise groups (stem bruise and equator bruise) of different measure-ment times. The bruise ratio indexes of 2, 6, and 12 h after impact werestatistically higher than those 30 min after impact (Fig. 6). Conse-quently, the sliced plane could not reflect the real bruise condition at30 min and human graders were not able to observe and evaluate thebruised tissues that had not become discolored. Although the statisticaltests showed a similar trend for the imaging method, the image-pre-dicted bruise ratio indexes on average were higher than visually scoredvalues at all measurement times (Fig. 6). These results suggest againthat the imaging method could be more sensitive to internal bruisingthan visual scoring, especially in the early stage (30 min).

Due to the limitation of visually scoring bruising, more researchneeds to be done to explore bruise progression using advanced

Fig. 5. Correlation between visually scored bruise ratio indexes and image-predicted bruise ratio indexes obtained by classification model over time (a) 30 min; (b) 2 h; (c) 6 h; and (d)12 h after impact.

Fig. 6. Comparison between visually scored and image-predicted bruise ratio indexes ofblueberry samples 30 min, 2 h, 6 h, and 12 h after impact. Different uppercase and low-ercase letters indicate significant differences of visually sored and image-predicted bruiseratio indexes respectively between measurement times at the significance level of 0.05.Horizontal line with the ‘*’ represents statistical differences (paired t-test, p < 0.05)between visually scored and image-predicted bruise ratio indexes.

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

61

Page 8: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

techniques such as magnetic resonance imaging (MRI), which has beensuccessfully used in measuring internal bruising and browning of apples(Cho et al., 2008; Zion et al., 1995). With the help of MRI technology,more accurate ground truth about blueberry internal bruising could beobtained.

3.3. Classification model with optimum wavelengths

CARS and SPA were separately carried out on the spectral data ofcalibration set to select optimum wavelengths carrying the most valu-able information related to bruise detection throughout the full spectra.The key wavelengths for blueberry bruise detection were determined byCARS to be the following 16 wavelengths: 960, 980, 995, 1000, 1180,1200, 1205, 1210, 1225, 1405, 1410, 1420, 1430, 1445, 1645, and1650 nm; and by SPA to be the following 24 wavelengths: 960, 990,995, 1030, 1035, 1040, 1045, 1050, 1060, 1070, 1080, 1095, 1100,1120, 1170, 1180, 1185, 1190, 1195, 1200, 1220, 1340, 1395, and1640 nm (Fig. 7). The 16 wavelengths selected by CARS were observedto be concentrated at the water absorption bands, while the 24 wave-lengths obtained by SPA were mainly located at 1050–1100 nm andaround 1200 nm (water absorption band).

In order to estimate the effectiveness of these selected wavelengths,the dimension of the raw spectral matrix in the calibration set was re-duced from 4800 × 141 to 4800 × 16 and 4800 × 24 and the reducedspectral data were mean normalized and used to establish CARS-LS-SVM and SPA-LS-SVM models, respectively. The bruising maps weretherefore calculated with both the CARS-LS-SVM model and SPA-LS-SVM model, separately, to identify every pixel of all blueberry samplesand calculate classification maps. It should be noted that only theimages at the optimum wavelengths (instead of the full spectra) wereused and corrected using mean normalization images. The visualizedclassification map obtained by CARS-LS-SVM model and correspondingpictures of sliced samples after 30 min of bruising are shown in Fig. 8.Through the predicted classification map, the bruise ratio index foreach sample was calculated. The correlation analysis between manuallymeasured bruise ratio indexes and predicted bruise ratio indexes ob-tained by LS-SVM models built based on optimum wavelengths isshown in Table 3. The CARS-LS-SVM model obtained more promisingresults than the SPA-LS-SVM model with respect to the number of op-timum wavelengths and bruise ratio index prediction ability based onthe RMSE values. Through comparing the two models in terms of op-timum wavelengths, it is interesting to observe that the performancedid not improve by increasing the number of wavelengths. This may bebecause there was some redundant or irrelevant information existingamong the variables selected by SPA. As previously discussed, water isthe main component that varies during the bruising process and most ofthe optimal wavelengths selected by CARS were related to water

absorption, resulting in better performance of the CARS-LS-SVM modelcompared with the SPA-LS-SVM model. The performance of the CARS-LS-SVM model was comparable to the full spectra-LS-SVM model interms of RMSE values but it used only about 11% of the variables usedby the full-range spectra (16 vs. 141), thereby simplifying the predic-tion model and satisfying the requirements for online detection. Theresults also implied that the CARS algorithm for informative variableselection was effective in this study.

3.4. Classification result of band ratio image

F-values of one-way ANOVA for all two-band ratios of the twogroups, bruised and healthy tissue spectra, were calculated and plottedin the contour figure (Fig. 9). The largest F-value was selected and thecorresponding two-band ratio (1235 nm/1035 nm) was taken as theoptimal two-band ratio for discriminating between bruised and healthytissues in hyperspectral images of blueberries. The 1235 nm wavelengthwas near the water absorption band selected by CARS (1200 nm), andthe 1035 nm wavelength, which was also selected by SPA, was relatedto the second overtone of NeH band (Liu et al., 2014). The changes ofNeH band might be due to the release of pigments and enzymes duringthe rupture of the cell walls. Those results suggested that the selectedwavelengths were not only statistically significant, but also chemicallyinterpretable.

According to the distribution of 1235 nm/1035 nm band ratio forhealthy spectra and bruised spectra (Fig. 10a), the distribution of thetwo groups overlapped with each other when the 1235 nm/1035 nmvalue was between 0.4 and 0.5, which resulted in misclassificationbetween bruised and healthy spectra. Therefore, a proper cutoff orthreshold value was required for discrimination. A band ratio above thethreshold value indicated that the spectrum was from healthy tissue,whereas a band ratio below the threshold value indicated that thespectrum was from bruised tissue. Then, the classification accuracies forhealthy and bruised spectra were calculated simultaneously with bandratio values increasing from 0.4 and 0.5 with a step of 0.001 to de-termine the threshold value (Fig. 10b). The point at which two accuracylines intersect (0.471) was determined to be the optimal threshold valuebecause higher classification accuracies were obtained for both bruisedspectra and healthy spectra. Finally, the classification map and bruiseratio index for each sample were obtained by using the selected two-band ratio and threshold to classify each pixel on the sample (Fig. 11).When the band ratio method was used to visualize the distribution ofbruise region and to obtain the bruise ratio indexes, the values of R2 foreach time treatment (30 min, 2 h, 6 h, and 12 h) were 0.569, 0.647,0.711, and 0.801, respectively, with corresponding RMSE values of0.195, 0.184, 0.168, and 0.158.

Compared with bruise ratio index prediction results obtained byCARS-LS-SVM and SPA-LS-SVM models in Table 4, the number of wa-velengths has a significant reduction but the performance of two-bandratio was lower in terms of RMSE and R2 values, especially for thesamples 30 min and 2 h after bruising. The reason for this may be thatthe proposed two-band ratio method, which only used two wave-lengths, may not provide sufficient information for early bruise detec-tion.

3.5. Fruit identification results

In postharvest quality sorting and grading, fruit packers normally donot need to measure the exact value of bruise ratio index for each fruit;instead, they are only interested in grouping blueberries into differentclasses according to their bruising conditions. In this study, accordingto the predicted bruise ratio indexes and a threshold value (0.25) pro-posed in our previous studies (Yu et al., 2014b), the blueberries wereclassified as either bruised or healthy samples.

Given a classifier and a set of instances (the test set), a confusionmatrix (also called a contingency table) can be constructed representing

Fig. 7. Distribution of selected variables by CARS ( ) and SPA ( ).

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

62

Page 9: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

the dispositions of the set of instances. According to the confusionmatrix of bruised fruit identification (Table 4), it was noticed that theCARS-LS-SVM model and full-spectra-LS-SVM model had similar per-formance in the identification of bruised blueberries. For the CARS-LS-SVM model, there were 14, 7, 5, and 6 misclassified samples for 30 min,

2 h, 6 h, and 12 h after impact, respectively, with corresponding overallclassification accuracies of 82.5%, 91.3%, 93.8%, and 92.5%, respec-tively. Although more wavelengths were included in the full spectra-LS-SVM model, the identification accuracies were comparable to those ofCARS-LS-SVM model, suggesting the selected wavelengths could re-place the full spectra for distinguishing between healthy and bruisedblueberries.

Furthermore, although the classification accuracy from band ratioimage method based on only two wavelengths was slightly lower thanthat of the CARS-LS-SVM and full spectra-LS-SVM models in the iden-tification of bruised blueberries 2 h after impact, it was similar to thatof CARS-LS-SVM model at 6 h, and slightly higher than that of CARS-LS-SVM and full spectra-LS-SVM models at 12 h. The overall classificationaccuracies of 77.5%, 83.8%, 92.5%, and 95.0% were obtained by two-band ratio images for blueberries 30 min, 2 h, 6 h, and 12 h after im-pact, respectively.

Table 4 indicates that the identification accuracies based on threemodels were much lower at 30 min after impact, with 15, 14 and 18samples misclassified. As previously mentioned, bruises in severalsamples from calyx bruise groups were excessively underestimated.This could explain why 9, 10, and 11 bruised samples were mis-classified as healthy by three classification models. In addition, thethreshold value affected the identification results because samples withvisually scored bruise ratio indexes around the threshold value weremore easily misclassified by classification models. Compared to thesamples 2 h, 6 h, and 12 h after impact, more blueberry samples wereobserved with visually scored bruise ratio indexes around the thresholdvalue (0.25) 30 min after impact. That was another reason why moresamples were misclassified 30 min after impact. When the calyx bruisegroups were excluded in the data set, the identification accuracies im-proved as the false negative rate was decreased dramatically for all fourtimes (see Supplementary Table S1). The results also showed that thecalyx bruises were more challenging to identify compared with stem

Fig. 8. Classification map obtained by the CARS-LS-SVM model and cor-responding pictures of sliced samples 30 min after impact. Red pixels re-present bruised tissues, white pixels represent healthy tissues, and bluepixels represent calyx excluded from classification. (For interpretation ofthe references to colour in this figure legend, the reader is referred to theweb version of this article.)

Table 3Correlation analysis between manually measured bruise ratio index and predicted bruiseratio index using LS-SVM models built on optimum variables.

ClassificationModel

Bruise ratioindex for30 min

Bruise ratioindex for 2 h

Bruise ratioindex for 6 h

Bruise ratioindex for 12 h

R2 RMSE R2 RMSE R2 RMSE R2 RMSE

SPA-LS-SVM 0.699 0.170 0.774 0.158 0.816 0.175 0.870 0.121CARS-LS-SVM 0.681 0.166 0.741 0.147 0.839 0.133 0.847 0.122

Fig. 9. Contour plot of F-value calculated by ANOVA test between two-band ratios ofhealthy and bruised tissue spectra.

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

63

Page 10: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

and equator bruises. The cells around the calyx end have smaller sizeand a more compact cellular structure compared to those adjacent tothe equator or stem (Fig. 12). Alvarez et al. (2000) found that tissueswith smaller cells sustained less damage under the same external load.In addition, cells adjacent to intercellular spaces are more vulnerable todamage due to the lack of cell-to-cell contact (Mitsuhashi-Gonzalezet al., 2010). Thus, our current and previous studies have confirmedthat stem and equator bruises were more severe than calyx bruises(Jiang et al., 2016). The different cellular structure might explain whythe spectra around healthy calyx tissue were similar to those of bruisedtissue around stem and equator. The calyx bruise was therefore difficultto identify.

Brown et al. (1996) reported that 78% and 23% of blueberries were

bruised when they were harvested by mechanical harvesters and byhuman pickers, respectively, suggesting that bruises in blueberries weremainly attributed to the harvest process. It then takes several hours totransport blueberries from the field to the packing line for sorting ac-cording to their qualities. Therefore, bruises would have time to de-velop before the sorting process and the imaging method developed inthis study would most likely be effective in practice. In addition,compared to other spectral analysis methods, the throughput of thetwo-band ratio image would accelerate the image acquisition andanalysis process, and make it less challenging to implement the hard-ware of multispectral imaging systems. A program was developed inMatlab2016b to measure the time taken by CARS-LS-SVM model andtwo-band ratio image to analyze a hyperspectral image of blueberries. It

Fig. 10. (a) Distribution of 1235 nm/1035 nm band ratio for healthy and bruised tissue spectra and (b) variation of classification accuracies for healthy and bruised tissue spectra with the1235 nm/1035 nm band ratio increasing.

Fig. 11. Grayscale images at 1035 nm, 1235 nm,1235 nm/1035 nm and corresponding classificationmap obtained by two-band ratio image method forsamples 30 min after bruising. Red pixels representbruised tissues, white pixels represent healthy tis-sues, and blue pixels represent calyx excluded fromclassification. Corresponding pictures of sliced sam-ples were shown in Fig. 8. (For interpretation of thereferences to colour in this figure legend, the readeris referred to the web version of this article.)

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

64

Page 11: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

took 9.119 s and 0.782 s to calculate the classification map and to ob-tain bruise ratio index of each sample from stem bruise group 30 minafter impact by CARS-LS-SVM and band ratio images, respectively,using the program run on a computer (Dell, Inter(R) Core(TM) i3-3220CPU @3.30 GHz, RAM 8.0GB). Thus, the two-band ratio image(1235 nm/1035 nm) proposed in this study is more promising anduseful than CARS-LS-SVM model to develop a more efficient multi-spectral imaging system for the bruise detection in the packing lineafter harvesting.

The classification accuracies obtained in this study were higher thanthose of another study reported by Hu et al. (2016), who identifiedmechanical bruising of blueberries using the mean spectrum of eachsample extracted from hyperspectral images in the 400–1000 nmspectral range, with classification accuracies of 80.2% and 76.7% forhealthy and damaged blueberries, respectively, 12 h after impact. Dif-ferences in classification accuracy between our study and that of Huet al. (2016) could possibly be attributed to the shortwave infrared(SWIR) system used in the current study, which had shown relativelyhigher accuracy in bruise classification for apples (Lu, 2003) and pears(Lee et al., 2014). In addition, spectral analysis at the pixel level may beanother reason why better classification results were yielded in thisstudy.

3.6. Model validation using independent data set

According to the calculated classification map and predicted bruiseratio index calculated by CARS-LS-SVM and two-band ratio images,each sample was sorted as healthy or bruised fruit. The overall dis-crimination accuracies for healthy and bruised blueberries in validationset were 93.3% and 98.0%, respectively, for CARS-LS-SVM model, and93.3% and 95.9%, respectively, for two-band ratio images. The results

indicate that the proposed bruise identification algorithms, especiallythe two-band ratio images, provided an effective solution for gradingblueberry samples with respect to bruise condition.

The current bruise detection system was limited to one single viewof the blueberries. To get more reliable and comprehensive detectionresults, the current detection system needs to be modified to detect thefull surface of the fruit without flipping it. In addition, the curvature ofblueberry was not considered in this study. The spectra intensity wasmuch lower along the edge of blueberries due to the curvature, makingit difficult to inspect the bruises near the edge. For instance, healthytissues with low reflectance intensities along the edge might be classi-fied as bruised tissues.

4. Conclusion

In this study, an LCTF-based hyperspectral reflectance imagingsystem with a spectral range of 950–1650 nm was successfully appliedfor rapid and non-invasive determination of blueberry internal bruisingat the early stage. The reflectance of bruised tissues in blueberries wasgenerally lower than that of healthy tissues, and the difference in re-flectance between healthy and bruised tissues increased over time, re-sulting in the increased prediction accuracies of bruise ratio indexes.Compared with stem and equator bruises, calyx bruises were morechallenging to identify with the imaging method. The proposed two-band ratio (1235 nm/1035 nm) image was an effective and promisingmethod to distinguish between healthy and bruised blueberries even atthe early stage of bruising, which could be used to develop an imagingsystem for blueberry bruise detection in the packing line. Future workwill be focused on full surface bruise detection and the curvature of thefruit will also be considered to improve the bruise detection accuracy.

Table 4Confusion matrixes of bruised fruit identification using three models at four different measurement times.

Visually scored bruise ratio index Full spectra-LS-SVM CARS-LS-SVM 1235 nm/1035 nm

Healthy Bruised Accuracy Healthy Bruised Accuracy Healthy Bruised Accuracy

30 minHealthy (N) 18 6 75.0% 20 4 83.3% 17 7 70.8%Bruised (N) 9 47 83.9% 10 46 82.1% 11 45 80.3%

2 hHealthy (N) 11 4 71.4% 13 2 86.67% 13 2 86.7%Bruised (N) 2 63 96.92% 5 60 92.31% 11 54 83.1%

6 hHealthy (N) 12 3 80% 12 3 80.0% 12 3 80.0%Bruised (N) 0 65 100% 2 63 97.0% 3 62 95.4%

12 hHealthy (N) 15 1 93.7% 15 1 93.7% 16 0 100%Bruised (N) 5 59 92.2% 5 59 92.2% 4 60 93.8%

N: number of fruit; Rows: actual number of fruit in each category; Columns: number of fruit in each category classified by models.

Fig. 12. Microscopic images of blueberry tissues around (a) equator and(b) calyx end. Images were taken with a microscope system (model BA410,Motic, Xiamen, China) using a 4× objective lens.

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

65

Page 12: Postharvest Biology and Technologysensinglab.engr.uga.edu/wp-content/uploads/2019/11/... · hyperspectral imaging (HSI) technology, which integrates the ad-vantages of imaging technology

Acknowledgements

This work is supported by the USDA National Institute of Food andAgriculture Specialty Crop Research Initiative program grant (AwardNo: 2014-51181-22383). The authors gratefully thank Mr. Yu Jiang andMs. Mengyun Zhang for their assistance in conducting experiments andproviding useful advice on data analyses.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in theonline version, at http://dx.doi.org/10.1016/j.postharvbio.2017.08.012.

References

Alvarez, M.D., Saunders, D., Vincent, J., Jeronimidis, G., 2000. An engineering method toevaluate the crisp texture of fruit and vegetables. J. Texture Stud. 31, 457–473.

Araújo, M.C.U., Saldanha, T.C.B., Galvão, R.K.H., Yoneyama, T., Chame, H.C., Visani, V.,2001. The successive projections algorithm for variable selection in spectroscopicmulticomponent analysis. Chemom. Intell. Lab. Syst. 57, 65–73.

Ariana, D.P., Lu, R., Guyer, D.E., 2006. Near-infrared hyperspectral reflectance imagingfor detection of bruises on pickling cucumbers. Comput. Electron. Agric. 53, 60–70.

Brown, G., Schulte, N., Timm, E., Beaudry, R., Peterson, D., Hancock, J., Takeda, F., 1996.Estimates of mechanization effects on fresh blueberry quality. Appl. Eng. Agric. 12,21–26.

Cheng, J.H., Sun, D.W., 2015. Rapid and non-invasive detection of fish microbial spoilageby visible and near infrared hyperspectral imaging and multivariate analysis. LWTFood Sci. Technol. 62, 1060–1068.

ElMasry, G., Wang, N., Vigneault, C., Qiao, J., ElSayed, A., 2008. Early detection of applebruises on different background colors using hyperspectral imaging. LWT Food Sci.Technol. 41, 337–345.

Fan, S., Zhang, B., Li, J., Huang, W., Wang, C., 2016.. Effect of spectrum measurementposition variation on the robustness of NIR spectroscopy models for soluble solidscontent of apple. Biosyst. Eng. 143, 9–19.

Hu, M.-H., Dong, Q.-L., Liu, B.-L., 2016. Classification and characterization of blueberrymechanical damage with time evolution using reflectance, transmittance and inter-actance imaging spectroscopy. Comput. Electron. Agric. 122, 19–28.

Huang, W., Li, J., Wang, Q., Chen, L., 2015. Development of a multispectral imagingsystem for online detection of bruises on apples. J. Food Eng. 146, 62–71.

Jiang, Y., Li, C., Takeda, F., 2016. Nondestructive detection and quantification of blue-berry bruising using near-infrared (NIR) hyperspectral reflectance imaging. Sci.Rep. 6.

Kleynen, O., Leemans, V., Destain, M.-F., 2003. Selection of the most efficient wavelengthbands for ‘Jonagold’ apple sorting. Postharvest Biol. Technol. 30, 221–232.

Kleynen, O., Leemans, V., Destain, M.-F., 2005. Development of a multi-spectral visionsystem for the detection of defects on apples. J. Food Eng. 69, 41–49.

Lee, W.-H., Kim, M.S., Lee, H., Delwiche, S.R., Bae, H., Kim, D.-Y., Cho, B.-K., 2014.Hyperspectral near-infrared imaging for the detection of physical damages of pear. J.Food Eng. 130, 1–7.

Leiva-Valenzuela, G.A., Lu, R., Aguilera, J.M., 2013. Prediction of firmness and solublesolids content of blueberries using hyperspectral reflectance imaging. J. Food Eng.115, 91–98.

Li, H., Liang, Y., Xu, Q., Cao, D., 2009. Key wavelengths screening using competitiveadaptive reweighted sampling method for multivariate calibration. Anal. Chim. Acta648, 77–84.

Li, C., Krewer, G.W., Ji, P., Scherm, H., Kays, S.J., 2010. Gas sensor array for blueberryfruit disease detection and classification. Postharvest Biol. Technol. 55, 144–149.

Li, J., Rao, X., Wang, F., Wu, W., Ying, Y., 2013. Automatic detection of common surfacedefects on oranges using combined lighting transform and image ratio methods.Postharvest Biol. Technol. 82, 59–69.

Li, J., Chen, L., Huang, W., Wang, Q., Zhang, B., Tian, X., Fan, S., Li, B., 2016.

Multispectral detection of skin defects of bi-colored peaches based on vis–NIR hy-perspectral imaging. Postharvest Biol. Technol. 112, 121–133.

Liu, Y., Chen, Y.-R., Kim, M.S., Chan, D.E., Lefcourt, A.M., 2007. Development of simplealgorithms for the detection of fecal contaminants on apples from visible/near in-frared hyperspectral reflectance imaging. J. Food Eng. 81, 412–418.

Liu, F., Jiang, Y., He, Y., 2009. Variable selection in visible/near infrared spectra forlinear and nonlinear calibrations: a case study to determine soluble solids content ofbeer. Anal. Chim. Acta 635, 45–52.

Liu, D., Sun, D.-W., Zeng, X.-A., 2014. Recent advances in wavelength selection techni-ques for hyperspectral image processing in the food industry. Food BioprocessTechnol. 7, 307–323.

Lu, R., 2003. Detection of bruises on apples using near–infrared hyperspectral imaging.Trans. ASAE 46, 523.

Luo, X., Takahashi, T., Kyo, K., Zhang, S., 2012. Wavelength selection in vis/NIR spectrafor detection of bruises on apples by ROC analysis. J. Food Eng. 109, 457–466.

Mehl, P.M., Chen, Y.-R., Kim, M.S., Chan, D.E., 2004. Development of hyperspectralimaging technique for the detection of apple surface defects and contaminations. J.Food Eng. 61, 67–81.

Mitsuhashi-Gonzalez, K., Pitts, M., Fellman, J., Curry, E., Clary, C., 2010. Bruising profileof fresh apples associated with tissue type and structure. Appl. Eng. Agric. 26,509–517.

Nanyam, Y., Choudhary, R., Gupta, L., Paliwal, J., 2012. A decision-fusion strategy forfruit quality inspection using hyperspectral imaging. Biosyst. Eng. 111, 118–125.

Peirs, A., Scheerlinck, N., Nicolai, B.M., 2003a. Temperature compensation for near in-frared reflectance measurement of apple fruit soluble solids contents. PostharvestBiol. Technol. 30, 233–248.

Peirs, A., Tirry, J., Verlinden, B., Darius, P., Nicola, B.M., 2003b. Effect of biologicalvariability on the robustness of NIR models for soluble solids content of apples.Postharvest Biol. Technol. 28, 269–280.

Qin, J., Burks, T.F., Ritenour, M.A., Bonn, W.G., 2009. Detection of citrus canker usinghyperspectral reflectance imaging with spectral information divergence. J. Food Eng.93, 183–191.

Qin, J., Burks, T.F., Zhao, X., Niphadkar, N., Ritenour, M.A., 2012. Development of a two-band spectral imaging system for real-time citrus canker detection. J. Food Eng. 108,87–93.

Travers, S., Bertelsen, M.G., Petersen, K.K., Kucheryavskiy, S.V., 2014.. Predicting pear(cv: Clara Frijs) dry matter and soluble solids content with near infrared spectro-scopy. LWT Food Sci. Technol. 59, 1107–1113.

Wang, W., Li, C., Tollner, E.W., Gitaitis, R.D., Rains, G.C., 2012a. Shortwave infraredhyperspectral imaging for detecting sour skin (Burkholderiacepacia)-infected onions.J. Food Eng. 109, 38–48.

Wang, W., Li, C., Tollner, E.W., Rains, G.C., Gitaitis, R.D., 2012b. A liquid crystal tunablefilter based shortwave infrared spectral imaging system: design and integration.Comput. Electron. Agric. 80, 126–134.

Wu, D., Sun, D.-W., 2013. Potential of time series-hyperspectral imaging (TS-HSI) for non-invasive determination of microbial spoilage of salmon flesh. Talanta 111, 39–46.

Xu, R., Takeda, F., Krewer, G., Li, C., 2015. Measure of mechanical impacts in commercialblueberry packing lines and potential damage to blueberry fruit. Postharvest Biol.Technol. 110, 103–113.

Yao, Y., Chen, H., Xie, L., Rao, X., 2013. Assessing the temperature influence on thesoluble solids content of watermelon juice as measured by visible and near-infraredspectroscopy and chemometrics. J. Food Eng. 119, 22–27.

Yu, K., Zhao, Y., Li, X., Shao, Y., Zhu, F., He, Y., 2014a. Identification of crack features infresh jujube using Vis/NIR hyperspectral imaging combined with image processing.Comput. Electron. Agric. 103, 1–10.

Yu, P., Li, C., Takeda, F., Krewer, G., 2014b. Visual bruise assessment and analysis ofmechanical impact measurement in southern highbush blueberries. Appl. Eng. Agric.30, 29–37.

Zhang, M., Li, C., 2016. Blueberry Bruise Detection Using Hyperspectral TransmittanceImaging, 2016 ASABE Annual International Meeting. American Society ofAgricultural and Biological Engineers p. 1.

Zhang, B., Huang, W., Li, J., Zhao, C., Fan, S., Wu, J., Liu, C., 2014. Principles, devel-opments and applications of computer vision for external quality inspection of fruitsand vegetables: a review. Food Res. Int. 62, 326–343.

Zhang, B., Li, J., Fan, S., Huang, W., Zhao, C., Liu, C., Huang, D., 2015. Hyperspectralimaging combined with multivariate analysis and band math for detection ofcommon defects on peaches (Prunus persica). Comput. Electron. Agric. 114, 14–24.

S. Fan et al. Postharvest Biology and Technology 134 (2017) 55–66

66