dr. martin g. bello alphatech inc. 6 new england executive park burlington, mass. 01803

ALPHATECH, Inc.

Comparison of Support Vector Machines and Multilayer Perceptron Networks in Building Mine Classification Models

Dr. Martin G. Bello

ALPHATECH Inc.

6 New England Executive Park

Burlington, Mass. 01803

August 29, 2003

(Research Funded by ONR as Part of the 6.2 MCM Program Element, with Associated Technical Agent: NSWC Coastal Systems Station, Panama City, FL.)

6 New England Executive Park, Burlington, MA 01803 781-273-33883811 N. Fairfax Dr., Arlington, VA 22203 703-524-6263

4445 Eastgate Mall, San Diego, CA 92121 858-812-7874

3811 N. Fairfax Dr., Arlington VA 22203 703-524-62636 New England Executive Park, Burlington MA 01803 781-273-3388

ALPHATECH, Inc.

4445 Eastgate Mall, San Diego CA 92121 858-812-7874

PRESENTATION OVERVIEW

• Mine Countermeasures Overview

• Mine Hunting Algorithm Structure

• Overview of Multilayer Perceptron, Support Vector Machine Classifier Construction Methodologies

• Alternative Classifier Construction Performance Result Comparisons

• Conclusions


ALPHATECH, Inc.


Mine Countermeasures Overview

SONAR MISSION TAPE

NEW POST MISSION ANALYSIS CONCEPT-ICoastal Systems Station

PMA 2000

Bluefin BPAUV / Klein Sonar

WHOI / REMUS UUV & MSTL Sonar

CSSCAD/CAC

AlphatechCAD/CAC

LockheedCAD/CAC

Display,Analysis,

& Fusion ofContacts

PAYOFF OF FUSING MULTIPLE CAD / CAC ALGORITHMS

• REDUCED FALSE ALARM RATES• ENVIROMENTALLY ROBUST• DIVERSE ALGORITHMS HAVE FEW FALSE ALARMS IN COMMON

SONAR MISSION TAPE

MULTIPLE COMPUTER-AIDED DETECTION & CLASSIFICATION (CAD / CAC) ALGORITHMS

OBJECTIVEINCREASE SPEED & ROBUSTNESSOF POST MISSION ANALYSIS

NEW POST MISSION ANALYSIS CONCEPT-IICoastal Systems Station

PMA 2000

RaytheonCAD/CAC

Display,Analysis,

& Fusion ofContacts

NEW POST MISSION ANALYSIS CONCEPT-IIICoastal Systems Station

PMA 2000

SonarData

CSSCAD/CAC

LockheedCAD/CAC

OperatorMarks

AlphatechCAD/CAC

Bluefin BPAUV / Klein sonar

Mine-Like Objects (MLO’s)


ALPHATECH, Inc.


AGGREGATE DETECTION/CLASSIFICATION ALGORITHM STRUCTURE

• Normalization Algorithm Enforces More Uniform “Local” Background

• Anomaly Screening Extracts Blobs/Tokens Corresponding to Mine-Like(ML) Target Candidates

• Features are “Local” Functionals of Image Calculated for each Blob/Token

• Feature Vector Multilayer Perceptron Neural NetworkLog-Likelihood Calculation or Alternatively Feature VectorSVM score calculation

Input Side-Scan SonarImagery

Image NormalizationAlgorithm

AnomalyScreeningAlgorithm

Feature Calculationfor

Screener Tokens

Token Log-Likelihood

RatioCalculation

TokenRanking/Thresholding


ALPHATECH, Inc.


ANOMALY-SCREENING ALGORITHM STRUCTURE

• Anomaly Statistic Quantifies Deviation from “Local” Background Characteristics

• (Distinct Highlight(H) and Highlight/Shadow(HS) and Shadow(S) Contrast Statistics Have Been Conceived)

• MP, PC = Blob Anomaly Statistic Maximum Intensity and Pixel Count (PC)

• rMP, rPC= Ranks Associated with MP, PC

• Blob Filtering Identifies Candidate ML-Tokens

• Current Screening Algorithm Employs both H, HS, and S Based Segmentation Statistics, Deriving the Final Collection of Screened Tokens as those HS-blobs which Intersect either a H- or S-blob

AnomalyStatistic

CalculationAlgorithm

Image Histograming,Extraction of Top p% Pixels,

and Region Labeling

Calculation of MP,PC Featuresand Associated

Ranks rMP,rPC for eachBlob/Token

Blob/Token FilteringBased on Size andAggregate Rank

Statistic min(rMP,rPC)


ALPHATECH, Inc.


BASELINE CLASSIFICATION FEATURE VECTOR -f DEFINITIONS-I

• f1= PC for HS-segmentation

• f2= min(rPC, rMP) for HS-segmentation

• f3= “Local” Blob Pixel Count for HS-segmentation

• f4= Mean Blob Anomaly Statistic Intensity for HS-segmentation

• f5= Standard Deviation of Blob Anomaly Statistic Intensity for HS-segmentation

• f6= “Local” Blob Count for HS-segmentation

• f7= MP for HS-segmentation

• f8 = (1,0) Valued Indicator for Existence of Intersecting H-segmentation Blob

• f9 = MP for Intersecting H-Segmentation Blob

• f10 = PC for Intersecting H-Segmentation Blob

• f11 = (1,0) Valued Indicator for Existence of Intersecting S-segmentation Blob

• f12 = PC for Intersecting S-Segmentation Blob

Original Feature Set-1992

Highlight Related

Shadow Related


ALPHATECH, Inc.


BASELINE CLASSIFICATION FEATURE VECTOR -f DEFINITIONS-II

• f13 = Mean Normalized Image Over HS-Segmentation Blob

• f14 = Maximum Normalized Image Over HS-Segmentation Blob

• f15 = Standard Deviation of Normalized Image Over HS-segmentation Blob

• f16 = Skewness Coefficient of Normalized Image over HS-Segmentation Blob

• f17 = Kurtosis Coefficient of Normalized Image Over HS-Segmentation Blob

• f18 = Perimeter of HS-Segmentation Blob

• f19 = (16*PC)/(Perimeter*Perimeter) for HS-Segmentation Blob

• f20 = Perimeter/(2*(Bounding-box-width + Bounding-box-height)) for HS-Segmentation Blob

• f21 = (Major-axis – Minor-axis)/ (Major-axis + Minor-axis) for HS-Segmentation Blob

• f22 = Major-axis

• f23 = Orientation of HS-Segmentation Blob

Statistical Intensity Distribution Related

Shape Related


ALPHATECH, Inc.


DISCRETE COSINE TRANSFORM AND PSEUDO-ZERNIKE MOMENT FEATURE DEFINITION

• Discrete Cosine Transform(DCT) Features Defined on Window Centered on HS-Segmentation Blob,

is a vector of quantities obtained by stacking rows of the below defined matrix…

1

0

1

0)/)(*)(*2(),(

Q

m

Q

nQvcucvuG

))*2/(**)1*2cos((*))*2/(**)1*2cos((*),( QvnQumnmg

• Pseudo-Zernike Moment Features(PZM) Defined on Window Centered on HS-Segmentation Blob…

pqMp

rdrdrgrp

D

iq

qpqp eSZ

0,...0

),()()1(

,,

)()( ,,22

, ZZ iqp

rqpqp

)0(),0(,1,,1,1,,,

pqforpqforqpqpqpqpqpqpPZM

f

fDCT


ALPHATECH, Inc.


OVERVIEW OF COMPARED CLASSIFIER CONSTRUCTION STRATEGIES-I

• Traditional Classifier Construction first involves a “Feature Selection” stage where an Information Theoretic Measure, or actual Discrimination Performance of a simple classifier are optimized

• Multilayer Perceptron(MLP) Based Training Algorithm Optimizes Mutual Information Between Feature Vector and “True” Class Using a Recursive “Backpropagation-like” approach

• Cross-Validation Using a Test Set is Employed to Terminate Training when a specified objective function corresponding to the integral over a Test Set Derived Receiver Operating Characteristic Curve(ROC), is maximized

• The above steps may be repeated for on the order of 50-100 network optimizations to arrive at a “best” solution


ALPHATECH, Inc.


OVERVIEW OF COMPARED CLASSIFIER CONSTRUCTION STRATEGIES-II

• Support Vector Machine(SVM) Implementation Adopted is SVMlight , developed by Professor Thorsten Joachims of Cornell University

• The Linear SVM “Soft-Margin” Training Formulation Is defined as a Quadratic Programming Problem, Optimizing a sum of two terms related to the squared norm of the classifier inner-product related parameter vector, and a weighted sum of “slack” variables related to miss-classification of training samples

• SVMlight employs an iterative “working-subset” strategy to solve the dual of the above described Quadratic Programming Problem, avoiding the excessive memory and time requirements of off the shelf Quadratic Programming Implementations, for large training data sets.


ALPHATECH, Inc.


COMPARISON OF SVM AND MLP RESULTS-I

• 415F= Baseline 23F Set + 256F (DCT Transform Related) + 136F (PZM Related)

• 48F = Baseline 23F Set + 25F Selected from DCT, PZM Related Features using Genetic Algorithm Based Approach (NeuralWare Predict Algorithm)

• 6F= Feature Set Selected from 48F using Genetic Algorithm Based Approach (NeuralWare Predict Algorithm)

• 6F,48F, 23F Results Using MLP networks are superior to SVM based Results using Aggregate 415F Set

0.800.810.820.830.840.850.860.870.880.890.900.910.920.930.940.950.960.970.980.991.00

PC

D

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

FAI

Legend

PCD_415F_SVML

PCD_415F_SVML_U

PCD_415F_SVML_L

PCD_48F_MLP

PCD_6F_MLP

PCD_23F_MLP

Average False Alarms per Image

Low False Alarms Desired


ALPHATECH, Inc.


COMPARISON OF SVM AND MLP RESULTS-II

• 6F,48F, 23F SVM Results are superior to SVM based Results using the Aggregate 415F Set

• Baseline 23F Result using MLP network classifier is the best

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

FAI

0.800.810.820.830.840.850.860.870.880.890.900.910.920.930.940.950.960.970.980.991.00

PCD

Legend

PCD_415F_SVML

PCD_415F_SVML_U

PCD_415F_SVML_L

PCD_48F_SVML

PCD_6F_SVML

PCD_23F_SVML

Average False Alarms per Image


ALPHATECH, Inc.


CONCLUSIONS

• MLP and SVM based classifier construction strategies frequently achieve similar performance. In this study, the MLP approach more consistently resulted in the best performance for a feature set of limited size

• There is an advantage to employing the GA based feature selection technique first, as opposed to the blind use of an aggregate collection of features

• SVM Implementation Needs to be generalized to Incorporate Cross-Validation over Weighting Parameter Associated with Miss-Classification Terms

dr. martin g. bello alphatech inc. 6 new england executive park burlington, mass. 01803

Documents

hssegmentation blob

local blob count

local blob pixel count

new england executive

burlington ma

eastgate mall

san diego ca

arlington va