detection of buried mines and explosive objects using dual-band
TRANSCRIPT
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
1/12
Detection of buried mines and explosive objects using dual-band
thermal imageryJason J. Lepley
*and Michael T. Averill
SELEX Galileo, Sigma House, Christopher Martin Road, Basildon, Essex, SS14 3EL, UK
ABSTRACTWe demonstrate the development and use of novel image processing methods to combine dual-band (MWIR and LWIR)images from SELEX GALILEO's Condor II camera to extract characteristics of observed scenes comprising buried
mines and explosive objects. We discuss the development of a statistical processing technique to extract the different
characteristics of the two bands. We further present a statistical classifier used to detect targets on independently trained
images with a high detection probability and low false negative rates and discuss methods to mitigate the impact of falsepositives through the selective processing of image regions and the contextual interpretation of the scene content.
Keywords: Mine detection, thermal imagery, image processing, statistical classifier, IED, dual-band, object detection
1. INTRODUCTIONThere is an acknowledged need to detect buried objects with little or no clearly visible signature on the surface. This has
direct application in addressing the needs of service personnel in current theatres of operation by assisting them in
detecting concealed explosive devices. It may also be applied in humanitarian applications that need a detection
capability in de-mining operations.
Thermal imagery has an established ability to provide contrast of the surface signatures above buried objects such thatthey can be distinguished from surrounding undisturbed earth. This contrast is particularly clear where the surface soil
has been recently disturbed resulting in a change in the soil porosity or moisture content. Single band thermal imagers,
however, provide only a one-dimensional picture of this information that comprises an inseparable mixture of the surfacematerials emissive and thermal properties. In this work we explore the use of a dual-band imager and combine the
information from the two different spectral bands. Each band responds differently to the physical information the scene
provides so, by simultaneously (or near-simultaneously) recording the two band information and processing the data to
contrast the differences, we are able to more readily separate the information relating to an emplaced object from theundisturbed ground.
This work has been enabled by the development, under the UK MOD ALBION programme and subsequently further bySELEX Galileo, of a dual-band thermal imaging camera (Condor). The Condor camera contains a detector able to
record in either the medium (3-5m) or long (8-12m) thermal bands simply by switching the bias voltage on thedetector. By co-locating the pixels of the two bands, the camera is able to record images in each of the two bands
without suffering mis-registration issues. The use of electrical switching between the bands also permits fast (of theorder of millisecond) switching between the bands, which enables the near-simultaneous recording of both bands at the
full 25 fps frame rate of the camera. This greatly reduces the mis-registration that will occur when imaging on the move,
opening the possibility of a mobile implementation of the detection system.
In the early phases of this work (running from 2008 to 2009), the project explored ratiometric techniques to establish the
physical radiometric information (temperature and emissivity) from the scene. The thesis behind which was that the
changes in emissivity and temperature incurred above an emplaced object may provide information on the altered surfacematerial and sub-surface thermal lag resulting from the emplaced device. Whilst some success was achieved with this
approach, the work concluded that the separated radiometric information was too non-linear and insensitive to provide
adequate discrimination of the target material and further that the grey-body assumption resulted in large errors whenapplied to real-world materials. The approach therefore taken was altered to employ statistical processing of the image
information and the results of this study are summarized in this paper.
*[email protected]; phone +44 (0)1268 883470; selexgalileo.com
Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XVI,edited by Russell S. Harmon, John H. Holloway Jr., J. Thomas Broach, Proc. of SPIE
Vol. 8017, 80171V 2011 SPIE CCC code: 0277-786X/11/$18 doi: 10.1117/12.883375
Proc. of SPIE Vol. 8017 80171V-1
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
2/12
2. EXPERIMENTAL RESULTS2.1 Trials DataAs a result of the progress made within this project and associated activities, SELEX were invited to participate in trialsin Arizona and fielded the dual band Condor II camera along with two other thermal cameras. Participation in this trial
was invaluable, since it provided alternative environmental conditions. Figure 1 shows a series of MW and LW images
captured by the Condor camera at the indicated times, over a period of 24 hours.
Figure 1: MW images (left set) and LW images (right set) over 24 hours
2.2 Analysis of Data SetThe imagery contained significant periodic noise from the cooling engine in the imager, which was visible on all thetrials data. To prevent this affecting subsequent statistical analysis a temporal noise removal process was employed. Of
the methods investigated, frame averaging was rejected due to it being sensitive to the outlying nature of the periodic
noise; instead a 10-frame median filter was applied to each pixel to provide a robust estimate of the true signal. The
imagery was taken from a tripod-mounted camera and can thus be considered to be spatially registered frame to frame.
It is envisaged that this noise would not be present in a production-standard imager and thus the requirement for sets ofregistered image frames would not in reality arise. Figure 2 shows an example of the noise removed.
Figure 3: Noise removed from a LW image
A number of basic observations were made by viewing trials imagery over the diurnal cycle. During the day the MW
signal is dominated by reflected solar radiation, yet the LW signal clearly shows signatures relating to buried objects.
During the evening and night the LW signal weakens, often becoming doughnut shaped representing a faint haloaround the burial region, yet the MW signal clearly shows the buried objects as cooler regions. It is clear from this that
the optimum image band to detect the disturbed earth varies over the course of the day. A trained end-user might realise
this and select the optimum image band depending on the period of the day. However, a more optimal solution would be
to intelligently fuse the two image bands in a manner that provides the greatest visibility of the targets of interest.Principle Component Analysis (PCA) allows us to select the projection of the MW/LW signal pair that has maximum
Proc. of SPIE Vol. 8017 80171V-2
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
3/12
variance and is thus a candidate technique for selecting the band that provides the best contrast on the targets. Initial
experiments with this resulted in a projection that was almost always dominated by the MW signal due to the largevariance that results from the spatial non uniformities in the medium wave imagery that are apparent as vignetting. This
non uniformity is largely a result of temperature variations and re-radiation within the pre-production lens optics and
should not be present in a production standard imager with built in NUC. To prevent this upsetting subsequent analysis,the non uniformity, which is very low frequency, was removed using a wavelet filter. The approximation coefficients at
the seventh level of decomposition were set to zero and the signal then reconstructed. An example is shown in Figure 3,the left hand image is uncorrected, the right hand image corrected. Both are scaled from 5% to 95% of the cumulative
greyscale distribution black to white. Clearly in the right hand signal the remaining variation is the more useful part of
the signal.
Figure 3: Example of wavelet uniformity correction (MW, 2-2-2009, 21:15)
Although spatial non uniformity was predominantly a problem with the MW imagery it is also present to a degree in the
LW. Both sets of imagery were therefore corrected prior to carrying out the PCA calculation.
During the Arizona trials, data was gathered in blocks where each block represents a period of time (usually 24 hours)
over which the cameras had a fixed observation of a static scene comprising several targets. Figure 4 shows the angle ofthe principle component against time of day for imagery from block 2 (x), block 3 (+), block 4 (), block 5 (), and
block 6 (). Where the block covered multiple days, colour is varied as indicated in the legend.
Figure 4: Angle of the principal component
An angle of 0 or represents a principal component lying purely in the MW signal; conversely an angle of /2
represents a principal component lying purely in the LW signal. For data blocks 2 to 5 a clear day/night switching isobserved, with the solar-dominated MW selected predominantly during the day. This is opposite to that required given
Proc. of SPIE Vol. 8017 80171V-3
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
4/12
the initial observations, although this is not a problem as the 2nd principle component will be orthogonal and will thus
directly correlate with the observed data. Block 6 (imagery of the adjacent trial line to blocks 2 5) behaves somewhatdifferently; here the MW signal predominates all the time and some fluctuations are noticeable on the final early
morning. The differences in the scenery within block 6 are small; marker posts absent, and a few surface rocks
different. Another potential difference with block 6 is that the weather conditions have altered, with block 6 occurring inthe presence of mostly overcast conditions, thus the surface heating effects of the sun would be somewhat reduced. This
suggests that this PCA technique is sensitive to image content or lighting conditions and would therefore not be a goodselector of the best signal for buried object detection.
To further investigate the properties of the imagery the dual band [MW,LW] signal was normalised using the technique
of sphereing. For each pixel, i, in an image comprising n pixels, we define the signal vector:
i
i
i
LW
MWx [1]
The sample mean and covariance of this are:
i in xx 1 [2]
T
i
i
i
n)()(
1
1xxxxS
[3]
The sphered data can be written as:
)(21
xxZ
i
T
i[4]
with and being the diagonal matrix of eigenvalues and the column matrix of eigenvectors resulting from the spectraldecomposition of the covariance:
TS [5]
This results in a signal with standardised variance in each of the principle axes and allows for easier observation of the
data. Scatter plots of the sphered data have been analysed for manually selected regions of buried object, backgroundsoil, and surface object (rock). Distinct clusters are visible for the different signal types. For consistency in presentation,
the sphered data has been rotated back to the original MW/LW axis. The data points are also range-limited to , to keep
the display sensible. Buried object is red, background soil is green, and surface object (rock) is blue. Figure 6 shows thescatter plot after target emplacement and Figure 5 shows the same image regions before emplacement recorded at the
same time on the previous day. The circular structure visible in blue in Figure 6 is range-limited data, indicating those
points that would otherwise be off the scale. The separation of buried object from background surface is clearly visible.
The surface object is not as clearly separated but appears to be differently distributed. This object only had a small
number of pixels and so it is less clear cut.
Proc. of SPIE Vol. 8017 80171V-4
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
5/12
Figure 5: Scatter before target emplacement (block 4). The colours represent background soil (Green and Red) and a surfacerock (Blue)
Figure 6: Scatter after target emplacement (block 5).
The separation of regions is maintained throughout the diurnal cycle. The following examples (Figure 7) are from block6, showing a 24 hour period with a larger surface rock. The separation of buried items and background surface is
maintained. The surface rock has a more variable signature than those of the surface signatures. Within these scatters,
more unusual objects tend to be represented farther from the centre. Materials with different properties tend to cluster
in different directions. We have attempted to capture these properties in a fused colour display that highlights both theunusualness of objects and the separation of material types. Direction, being represented here as a periodic function,
needs care in being mapped to an image display to avoid discontinuity at the boundary. To this end we have used thecircularly continuous colour palette provide by the HSI colour scheme, mapping direction of each pixel within the scatter
axis to the H component (Hue) and the Mahalanobis distance to the I component (intensity), the S component
(saturation) is fixed at 1. The resulting images, Figure 8, clearly partition the scene throughout the 24hr cycle (note that
this is the same image set as shown in Figure 7). For comparison, the four MW/LW pairs are shown in Figure 9.
Proc. of SPIE Vol. 8017 80171V-5
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
6/12
Figure 7: Scatter plots over a diurnal cycle (block 6, times indicated in the top left of the plot)
19:00 01:00
07:00 13:00
Figure 8: Fused colour image at 4 times thoughout the day {19:00, 01:00, 07:00, 13:00} (block 6)
Proc. of SPIE Vol. 8017 80171V-6
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
7/12
19:00 01:00
07:00 13:00
Figure 9: MW/LW pair, block 6, at four times {19:00, 01:00, 07:00, 13:00)
3. CLASSIFIER DEVELOPMENT3.1 Gaussian Mixture ModelsClusters such as those shown in Figure 6 and Figure 7 are not well modelled by simple Gaussian distributions however
they can be modelled as a weighted mixture of multiple Gaussian distributions, known as a Gaussian Mixture Model
(GMM). The common technique for estimating the parameters of a GMM distribution is the Expectation-Maximisation
algorithm (EM). This is an iterative algorithm that produces a maximum likelihood estimate of the parameters given asample of data. Prior to using the EM algorithm it is necessary to decide how many distributions are included in the
mixture and to make an initial estimate of the parameters. To address this we have implemented a recursive EM
algorithm (RUEM) [1] that simultaneously estimates the parameters of the mixture and the number of components. Thisis followed by the conventional EM algorithm using the full data set using the selected number of components and
parameters as the initial state.
To facilitate a valid experiment using the available data, the two groups have been selected from the Arizona trials data
to be used for training and evaluation respectively. The left hand trial lane is used for training and the right hand trial for
evaluation.
Three applications of GMM to buried object detection have been considered; classification of segmented regions,classification of each image pixel based on its local neighbourhood, and anomaly detection. In the first two of these
cases GMMs are fitted to training data corresponding to target and non target regions and the processed images are
labelled as target/non target. This clearly needs imagery representative of the target and background classes prior tooperation and may need multiple target and non target classes to cope with a wide range of material and environmental
conditions. The third approach considered makes no prior assumptions about the nature of the target or background but
models what is normal by analysis of scene regions. These three methods are discussed in the following three sections.
3.2
Classification of segmented regionsInitial segmentation of target sized objects from the scene is performed using a watershed by reconstructionsegmentation algorithm [4]. The segmentation is used to produce the region boundaries for the truth data described
above.
The watershed by reconstruction segmentation algorithm has been used in a number of previous Selex
detection/recognition studies and has been shown to produce accurate boundaries. In order to control which regions are
segmented we first need to generate appropriate markers. Morphological top-hat by reconstruction operators allow shape
selective extraction of appropriately sized dark or bright regions of the scene. In this case an upper size limit of 100 x 20
Proc. of SPIE Vol. 8017 80171V-7
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
8/12
pixels has been set and a lower size limit of 7 x 3 pixels to mitigate detection of surface texture. The approach to
segment the scene is based on the contrast in the bands. With two wavebands of imagery and two modes of regiondetection (dark and bright) there are potentially four sets of markers to be evaluated. As expected, based on the
observations of the dataset two of these four combinations perform considerably better; long wave bright markers during
the day, and medium wave dark markers at night. Figure 10 and Figure 11 show the performance of initial regionsegmentation using these marker functions on training data as a function of time of day. Note that this is an initial
detection process prior to classification; for this purpose a high probability of detection is more important than a lowfalse alarm rate.
Figure 10: Probability of detection for initial segmentation using two different markers
Figure 11: False alarm rate for initial segmentation using two different markers
A clear switch point between the two modes is observable. As a consequence we have selected to use long wave brightmarkers between 9:30 and 17:30 and medium wave dark markers for the remainder of the day. An example of the initialsegmentation is shown in Figure 12.
Proc. of SPIE Vol. 8017 80171V-8
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
9/12
Figure 12: Initial region segmentation
A set of 48 (target and non-target, 1 each for each hour of the day) GMMs were trained using the training data truthregions. An upper limit of 120000 was placed on the number of samples used for each GMM so as to keep runtimes
sensible. A maximum likelihood classifier was then used to classify the segmented regions from the evaluation data set.
The performance in terms of correctly identifying target and non-target regions within the segmented region set is shownin the Figure 13 as a function of time of day.
Figure 13: Evaluation using segmented regions (trained on truthed regions)
Although the performance for identifying non-target regions is moderate, the target Pd is poor. A possible explanation
for this is that the segmented target regions extend beyond the declared truth regions thus contaminating the sample. Toinvestigate the effect of this contamination, the experiment was re-run whilst restricting the target samples to the truthed
regions. The results (in Figure 14) show a marked improvement in the correct identification of the target regions.
Proc. of SPIE Vol. 8017 80171V-9
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
10/12
Figure 14: Evaluated using truthed subset of segmented regions (trained on truthed regions)
This suggests that either the training or segmentation needs to be adjusted so that the match is better. To address this, theGMMs were retrained using samples generated by the segmentation process, labelled according to whether they
overlapped the declared truth. The results, shown in Figure 15, are better than the original results but can still be
improved upon.
Figure 15: Evaluated using segmented regions (trained on segmented regions)
3.3 Classification of local neighbourhoodsEach pixel within the image was classified using samples from a square 9 x 9 neighbourhood centred on the pixel.
Boundary pixels, where the neighbourhood would be incomplete, were ignored. A maximum likelihood classifier was
used and the GMM were trained using samples from the truth regions as before.
The performance, evaluated on a per pixel basis, is shown in Figure 16 (left) as a function of time of day. These results
are encouraging. The mean probability of detection over the 24 hour cycle is 84% for target pixels and 86% for non-target pixels. Also shown in Figure 16 (right) is an example of a classified image with true positives (correct declaration
of target) in white, true negatives (correct declaration of non-target) in black, false positives (incorrect declaration of
target) in cyan, and false negatives (incorrect declaration of non-target) in red. It is clear from this that all the regions offalse negatives are connected to regions of true positive and as such all targets have been detected. Also a significant
percentage of the false positives are located in the vicinity of the true positive regions. We may have been overly harsh
Proc. of SPIE Vol. 8017 80171V-10
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
11/12
in our definition of the truth. If we consider areas in the image rather than individual pixels we see all the targets have
been detected and there are approximately 3 areas of false positive (cyan areas not contiguous with a white area of abovea reasonable size).
Figure 16: Left, performance of local neighbourhood based classification, and Right, example classified image (truepositives in white, true negatives in black, false positives in cyan and false negatives in red)
3.4 Anomaly detectionA limitation of the first two methods, particularly given the variable, nature of buried objects and emplacement, is the
requirement to have training data sets. It is therefore desirable to be able to detect regions within a scene that are
anomalous without a prior knowledge of the target.
Common multispectral anomaly detectors such as RX [2] find pixels that are locally extreme by measuring theMahalanobis distance with respect to the local spectral distribution. This may be sufficient for locating a small piece of
foreign material, but is unlikely to detect buried objects, as the surface materials will be on the whole similar to those in
the surroundings. The most likely change will be a change in the mixture of materials as the soil and rock from
subsurface will have different grain sizes and slightly different makeup due to the lack of weathering.
A mixture model, such as a GMM, is an ideal way of modelling this mixture of surface materials. By looking for local
changes in probability density function we should be able to detect regions of disturbance. The challenge of thisapproach comes in estimating the parameters of the GMM for small local neighbourhoods. The large number of
parameters would require a very large neighbourhood to provide sufficient samples for a reliable estimate. The number
of parameters increases linearly with the number of mixture components and quadratically with the number of bands. Forexample a 2 band model with 3 components requires 18 parameters to be estimated.
A solution to this local estimation problem is the borrowed strength methodology presented [3]. The key assumption in
this method is that the component distributions are present throughout the larger data set and that the local variation is
primarily in the mixing coefficients. A GMM is estimated for the whole data set. When examining local regions of the
data the component parameters (mean and covariance) are borrowed from the global GMM and just the local mixing
coefficients estimated using EM. In the case of the example in the previous paragraph this reduces the number ofparameters estimated locally from 18 to 3. Local anomalies can then be detected by inspecting the integrated square error
between the global density function and the local density function.
To demonstrate the strength of this technique, an example is given using a synthetically generated single band image.Each pixel in the image is selected at random from one of two Gaussian distributions (=-2, = 1) and (= 2, = 9/16).
For the majority of the image the mixing weights are 0.4 and 0.6 however the central 25x25 pixels the weights are 0.2
and 0.8. Figure 17 shows the input image and Figure 26 the anomaly map based on the ISE.
The algorithm has been implemented to process dual band images and to use a region map from a scene segmentation
routine to define the higher level regions within which to search. By restricting the search to particular scene regions theGMM produced will be more appropriate to each region being searched and should result in better sensitivity.
Proc. of SPIE Vol. 8017 80171V-11
Downloaded from SPIE Digital Library on 21 Sep 2011 to 147.96.22.163. Terms of Use: http://spiedl.org/terms
-
7/30/2019 Detection of Buried Mines and Explosive Objects Using Dual-band
12/12
The current truthed data set, gathered at Arizona, is not ideal for testing this algorithm since there is an unusually high
density of buried objects. In this case it is unlikely that the target regions would be truly anomalous within the context ofthe scene. It is planned to gather more appropriate test data for this algorithm in the trials planned later under this project.
Figure 17: Single band random scene with anomaly (left) and Anomaly map (right)
4. CONCLUSIONSWe present results from experimental trials and subsequent analysis to evaluate the benefits of a Dual-Band thermal IR
camera for the detection of buried objects. We show how, through the use of statistical analysis of the image data, we
are able to detect buried objects with a high probability of detection using a GMM based classification technique trainedusing an independent data set. We further demonstrate the development of a scene anomaly detector for the detection of
potential threats in un-trained data sets.
5. ACKNOWLEDGEMENTSThis work has been funded by the UK MoD through the EMRS DTC.
REFERENCES
[1] Zivkovic, Z, Recursive Unsupervised Learning of finite mixture models, IEEE Trans. on PAMI, Vol. 26(5),(2004).
[2] Hylta, P. C, et al, Anomaly detection in hyperspectral imagery: comparison of methods using diurnal and
seasonal data, J. Applied Remote sensing, Vol. 3, 1-29, (2009).[3] Priebe, C. E., Nonhomogeneity analysis using borrowed strength, J. American Statistical Association, Vol.
91(436), (1996).
[4] Doughert, E. R. (Ed.), Mathematical morphology in image processing, Marcel Decker, Chapter 12 (Beucher
and Meyer), (1993).
P f SPIE V l 8017 80171V 12