Arab J Sci EngDOI 10.1007/s13369-014-0989-7

RESEARCH ARTICLE - ELECTRICAL ENGINEERING

A Comparative Study of MLP Networks Using BackpropagationAlgorithms in Electrical Equipment Thermography

A. S. Nazmul Huda · Soib Taib

Received: 2 August 2012 / Accepted: 15 February 2013© King Fahd University of Petroleum and Minerals 2014

Abstract Nowadays, infrared thermographic inspection hasincreasingly been utilized for fault diagnosis tools of electri-cal equipment, because it maintains non-interrupting opera-tion of power system and ensures early diagnosis of faults.The article discusses the performance comparison of multi-layer perceptron (MLP) networks using various backpropa-gation algorithms for thermal imaging-based condition mon-itoring of electrical hotspots. The training algorithms areLevenberg–Marquardt, Broyden–Fletcher–Goldfarb–Shannoquasi-Newton, resilient backpropagation, gradient descentwith momentum and adaptive learning rate and standard gra-dient descent training algorithms. The performance of thetraining algorithms are evaluated in terms of percentage ofaccuracy, sensitivity, specificity, false-positive and false-negative results. In the beginning, thermal images of electri-cal equipment are captured using infrared camera. Six statis-tical intensity features, namely mean, maximum, minimum,median, standard deviation and variance extracted from eachhotspot of equipment thermal image are used as the inputsof multilayered perceptron networks to classify the thermalconditions of hotspots into two classes, namely normal anddefective. The comparison of MLP networks using trainingalgorithms proves that the MLP network trained with resilientbackpropagation algorithm gives the best performance inthermal condition recognition.

Keywords Electrical hotspots · Thermography ·Backpropagation algorithm · MLP network

A. S. N. Huda (B) · S. TaibSchool of Electrical and Electronic Engineering,Engineering Campus, Universiti Sains Malaysia, 14300 Nibong Tebal,Pulau Pinang Malaysiae-mail: nhudaeee@yahoo.com

----

1 Introduction

Infrared thermography is one of the popular non-destructivetesting (NDT) and condition monitoring tools which is gen-erally used to investigate the invisible thermal abnormalitieson the surface of the objects in various applications, suchas medical [1], building and infrastructure [2–4], securityand surveillance [5], production testing and monitoring [6],HVAC installations [7–9] and electrical systems [10]. In ther-mography, infrared radiation emitting from the surface of anobject is collected by an infrared camera. The camera gen-erates a thermal image showing the temperature distributionof the object surface [11].

123

Arab J Sci Eng

Thermographic inspection is an early fault diagnosis sys-tem of electrical equipment. It can detect the internal andexternal faults by monitoring the thermal condition of elec-trical equipment [12]. According to thermographic inspec-tion performed during the period 1999–2005 [13], it wasfound that most of the problems (48 %) were associated withconductors’ connection accessories and bolted connections,including problems in bimetallic surface contacts, dirt, oxi-dation and non-adequate use of inhibitory grease, and 45 %anomalies emerged in disconnectors’ contacts arising fromdefected contacts by deformations, deficient pressure of con-tact, incorrect alignment of arms and dirtiness. Only 7 % ofanomalies were found due to other reasons in the electricalequipment. Electrical equipment also shows thermal abnor-malities or hotspots due to degradations.

Infrared thermography technology can see the effect ofabnormal heating on the surface of equipment which is nor-mally invisible to naked eyes [14]. It is a non-contact and pre-dictive maintenance which can identify the location of faultswithout interrupting or shutting down the system [15,16].Therefore, there is no need to disassemble, rebuild or repairof components which are in good operating condition. Also,the faults of electrical equipments can lead to fire in build-ings and industries. As a results, huge amount of money issaved through early diagnosis of faults [17]. The advantagesof infrared thermography system for fault diagnosis in elec-trical equipment over the conventional approaches can befound in [13,15,17,18].

However, for increasing the operation reliability andimproving the inspection system, artificial neural networks(ANN) [20–22] have been used in various fields includingthermography [23–27]. In recent years, several works of elec-trical thermography have been done using ANN. For exam-ple, Almeida et al. [28] proposed an intelligent thermography-based lighting arrestor fault diagnosis system using neuro-fuzzy network. Emissivity, ambient temperature, humidity,maximum temperature, minimum temperature, temperaturevariation, pollution index, rated voltage, material and man-ufacturer were used as the input variables of ANN. It wasable to classify faults into two classes as normal and faulty,and obtained about 90 % of accuracy. Shafi’i and Hamzah[29] developed an intelligent thermography-based internalfault diagnosis of electrical equipment using multilayeredperceptron (MLP) network with Levenberg–Marquardt train-ing algorithm. RGB colour scale data and temperature datawere used as the input variables of ANN. It classified faultsinto four classes as low, intermediate, medium and high andobtained 99.38 % testing accuracy.

In the present study, the performance of MLP networksusing various backpropagation algorithms are compared tofind the best backpropagation algorithm for thermal imaging-based conditions classification of electrical hotspots. Fivetraining algorithms, namely Levenberg–Marquardt (LM),

Broyden–Fletcher–Goldfarb–Shanno quasi-Newton (BFG),resilient backpropagation (RP), gradient descent withmomentum and adaptive learning rate (GDX) algorithm andstandard gradient descent (GD) are used [30]. Section 2describes the fault analysis technique of electrical equip-ment using thermography. Six statistical intensity features,namely mean, maximum, minimum, median, standard devi-ation and variance extracted from each hotspot of electricalequipment thermal image are used as the inputs of MLP net-works. A detailed description of the proposed methodologyhas been given in Sect. 3. The data samples and performancemeasuring parameters are reported in Sect. 4. The systemclassifies the conditions of hotspots into two classes, namelynormal and defective. Finally, the results for classifying con-ditions using MLP networks using five different backpropa-gation algorithm are explained in Sect. 5. The performanceof the algorithms is evaluated in terms of percentage of accu-racy, sensitivity, specificity, false-positive and false-negativeresults.

2 General Description of Fault Diagnosisby Thermography

Thermographic inspection of electrical equipment is a fastand safe technique to identify the faults. The inspection candetect faults within various electrical installations includ-ing electrical cabinets, fuse boxes, switchgear installations,cables, overhead transmission lines, transformers, motors,power factor compensation equipment, busbar, etc [31].However, in the present study we will concentrate on faultswithin electrical equipment inside the electrical cabinet. Usu-ally, an electrical cabinet consists of wires, cables, circuitbreakers, etc. The major causes of overheating in electricalcabinets equipment are loose or oxidized connections, loadimbalance, overloading, contact problems, wiring mistakes,insulation faults, harmonics, etc.

The root causes of loose or poor connections are continualloading, vibration, aging, etc. Various environmental condi-tions such as high surrounding temperature, dirt in the air andchemicals can increase the speed of corrosion. However, dueto the connections problem, there is an increase in the resis-tance at the faulty spot. Increasing resistance also increasesthe heat at the connection. Thermographic inspections caneasily reveal the faults. In a three-phase system, under thesame load, all phases should show the same thermal char-acteristics. Abnormal or dissimilar thermal characteristics atthe connections of phases identify the poor or corroded con-nection. Figure 1a shows a typical poor or rust connectionproblems in a three phase system. The connection at phaseA is warmer than other phases.

Imbalance of current distribution to the loads is definedas load imbalance or overloading. It is reminded that current

123

Arab J Sci Eng

Fig. 1 a Typical problem in connection (A phase) b overloading (B phase) and c normal condition

imbalance is the primary reason for voltage imbalance. Themajor causes of electrical unbalance are unequal distribu-tion of single phase load, unbalanced incoming power sup-ply, unequal impedance in power supply cables, insulationresistance breakdown, etc. Under the same loading condi-tion, each phase should have the same load and heating con-dition. However, in an unbalanced load situation, the moreheavily loaded phase(s) will be hotter than the others, due tothe heat generated by additional resistance. An unbalancedload, overloading, poor connection and harmonic imbalancecan also create a similar temperature pattern [32]. Thermo-graphic inspection is a simple way to identify these kinds offaults. As in a three-phase system, phases are side by side.Therefore, by capturing the picture of heat, we can easilyidentify the unbalance or overloading problem. Figure 1bshows a typical overloading or unbalance loading problem.The phase B is warmer than other phases.

Also, overheating of electrical equipment can be causeddue to the insulation problem of conductors. Due to this prob-lem, two conductors make contact and create a short circuitproblem. Overcurrent flows into the fuse or circuit breaker(CB) to protect the device. But if the fuse or CB fails toopen, the circuit will be overheated. Harmonics distortionand wiring mistakes in the relays and switches can causeoverheating in electrical equipment. Therefore, thermogra-phy can easily find out the problem and diagnose the faultsbefore the eventual failure of equipment. From the currentdata set, some thermal images of electrical equipment show-ing abnormal heat pattern are illustrated in Fig. 2.

The next step is to classify the conditions of equipmentaccording to the severity of faults. Usually, two methods areused for monitoring conditions. One is quantitative methodand the other is qualitative method [33,34]. In the quantita-tive method, the exact temperature value of the target objectis used. This method is often affected by environmental fac-tors such as current ambient temperature, wind, humidityand emissivity. Therefore, for getting an accurate tempera-ture, the surrounding variables should also be considered.

On the other hand, the qualitative classification method iswidely used and briefly known as �T (temperature differ-ence) factor analysis. This Delta T factor is defined as theincrement of temperature of the equipment with respect to adefined reference, which is typically the ambient air tempera-ture, a similar component under the same loading conditionsor the maximum allowable temperature of the component[34]. �T factor, which is found by comparing the hotspotand reference spot temperature, is widely used for condi-tion monitoring of equipment. For this reason, in the presentstudy we will adopt this �T criteria for classifying the equip-ment fault conditions. The hotspots support the temperatureof the faulty components having temperature more than thereference spot, where the reference spot is the same type orload or the same repeated component of the equipment hav-ing minimum temperature. There are several standards for�T factor analysis, namely InterNational Electrical TestingAssociation (NETA) [35] and American Society for Testing& Materials (ASTM)-E [36], etc.

3 Methodology of Present Research

The MLP networks using five backpropagation algorithmswere used to compare the diagnosis performance of electricalequipment thermography. The inputs to the MLP networksare six features from the ROIs of electrical equipment, whichare mean, maximum, minimum, median, standard deviationand variance. The networks will receive these features as theinputs and classify the conditions of equipment into two cat-egories which are defective and normal, as shown in Fig. 1a,b, c. A flowchart containing the steps of methodology is pre-sented in Fig. 3.

3.1 Thermal Image Acquisition

Infrared images were captured using Fluke Ti25 thermalimager with fusion technology. All the infrared images of

123

Arab J Sci Eng

Fig. 2 Thermal images of electrical equipment showing extreme heat (red colour)

electrical equipment were collected from the main switch-board supplying electricity to an office building under thesame loading conditions. For capturing the image, the ther-mal imager orientation directly faces the target electricalequipment to get an accurate measurement. The distancebetween the target electrical equipment and the thermalimager was in the range of 0.5–1.0 m. The emissivity valuewas set at 0.95 as recommended generally for electrical equip-ments [37]. It was noted that the ambient temperature aroundthe equipment was between 30 and 33 ◦C during the inspec-tion. The specifications of the thermal camera are shown inTable 1.

The thermal camera was used to capture the thermalimages of electrical equipment and stored on SD memorycard. The MLP program was loaded into the computer. Wetransferred the photos from the camera to the computer viathe USB extension cable. Six types of statistical features

were extracted from each image of electrical equipment.Each time, we trained the MLP network with a backprop-agation algorithm (suppose LM algorithm) using extractedfeatures and obtained the results. In this way, one by one wetrained the MLP network with the mentioned five backprop-agation algorithms. After that, we made a comparison amongthe performances of MLP networks using backpropagationalgorithms.

3.2 Manual Classification of Conditions

In this research, based on the experience from current inspec-tion, a total of 253 hotspots from 139 electrical equipment areclassified into two categories, namely normal and defective.In the manual classification, 153 hotspots were classified asnormal and 100 hotspots were selected as defective based onthe �T criteria in Table 2.

123

Arab J Sci Eng

Fig. 3 System methodologyflowchart

Selection of no of hidden

nodes

No of output nodes

Thermal image acquisition of equipment

Manual condition classification

Output data

Input data

Manually image analysis

Thermal image processing

Identify hotspots

Extract features

Selection of training and testing data

MLP architecture

Selection of activation

function

Selection of epochs

Selection of learningrate

Trained with LM/BFG/RP/GDX/GD

training algorithms

Input Layer Output Layer Selection of no of hidden

layer

No of input nodes

Classification using LMalgorithm

Classification using BFG algorithm

Classification using GDX algorithm

Classification using RP algorithm

Classification using GD algorithm

Compare performances of all back propagation algorithms

Classificationaccuracy using

LM

Classification accuracy using

BFG

Classification accuracy using

RP

Classificationaccuracy using

GDX

Classificationaccuracy using

GD

Table 1 Thermal imagingcamera specifications (SourceFluke Ti25 system, 2010)

Temperature range −20 to + 350 ◦C Infrared lens type 20 mm F = 0.8 lens

Accuracy ±2 ◦C or 2 % Image frequency 9 Hz refresh rate

Visual camera 640 × 480 resolution Focus Manual

Detector type 160 × 120 focalplane array (FPA)

Minimum focusdistance

Thermal lens:15 cm (6 in); Visible(light) visual lens: 46 cm (18 in)

Field of view 23◦ × 17◦ Spatial resolution 2.5 mRad

Spectral range 7.5–14 µm Thermal sensitivity ≤ 0.1 ◦C at 30 ◦C (100 mK)

123

Arab J Sci Eng

Table 2 Classification criteria for conditions of electrical hotspots

Condition Prioritylevel

�Tsimilarequipments (◦C)

Recommended actions

Defective 1 �T ≥15 Major discrepancy;repair immediately

2 5 < �T < 15 Probable deficiency;repair as time permits

Normal 3 �T ≤ 5 Minor overheating;warrants investigation

Table 2 shows the �T criteria of different priority levelswith the necessary steps to be taken. In this study, levels 1and 2 are considered as defective and level 3 is considered asthe normal condition of the equipment. Figure 1a, b showsthe defective conditions of phases A and B, respectively. Onthe other hand, the red colour of A and B phases of Fig. 1cshows the normal condition with respect to phase C thermalcondition.

3.3 Input Features of MLP Network

Six features namely mean, maximum, minimum, median,standard deviation and variance are extracted from eachhotspot using the digital image analysis and visualizationprogram, Daime [38]. Daime can automatically segment thehot regions based on the thresholding value. In this study,custom thresholding technique, i.e., manually thresholdingvalue is set to generate an image of defect. The value of T athigh was set 255 always and at low was set as the value whichshows the hot defects of grayscale image clearly. After thatthe image is segmented according to the thresholded image.In the present study, two cases were considered for selectingthe desired hotspots: first, the highest pixel intensity objectand second the object having area of equal to or greater thanhalf of the maximum area object. As the maximum tempera-ture region of an image carries maximum intensity value, thecomponent of maximum area is generally the highest pixelintensity region and in some cases there is more than onehotspot in the selected image. The features will be used asthe inputs of the MLP network.

3.4 General Description of the MLP Network

Multilayered perceptron (MLP) network is widely used forpattern recognition and classification process. In this study,the MLP network is a feedforward artificial neural networkconsisting of six input nodes for six input features and twooutput nodes for the two target classes: defective conditionand normal condition. Two output nodes are assigned as ‘10’and ‘01’, respectively. Cybenko [39] and Funashashi [40]

Fig. 4 MLP architecture with one input, output and hidden layer

have proved that one hidden layer MLP network is alwayssufficient to approximate any continuous function up to accu-racy. Therefore, in this study the MLP network with one hid-den layer is considered. A standard MLP network with inputsz1, z2 . . . zn , predicted outputs y1, y2 . . . ym and nh hiddennodes is shown in Fig. 4.

The predicted output of the j-th neurons of the k-th nodein the output layer of the MLP network can be expressed asfollows: [41]

yk(t) =nh∑

j=1

w2jk F

(n∑

i=1

w1i j zi (t) + b j

)

for 1 ≤ k < m and 1 ≤ j < nh (1)

where w2jk is the connected weights between hidden and out-

put layer. w1i j denotes the connected weights between input

and hidden layer. zi and b j denote the inputs that are suppliedto the input layers and thresholds in hidden nodes, respec-tively. Also, ni , m and nh are the number of input, outputand hidden nodes, respectively. The F(.) is sigmoid activa-tion function of the MLP network, which is used in this study.The learning rate was set at 0.001. The weights w1

i j , w2jk and

threshold b j are unidentified and should be chosen for reduc-ing the error in prediction. The prediction error is defined asfollows:

εk(t) = yk(t) − yk(t) (2)

where yk(t) and yk(t) are actual(target) and network out-put(predicted), respectively.

3.5 Backpropagation Algorithms

The MLP network is trained using a supervised learningbackpropagation algorithm (BPA) in which a sum squareerror function is described and the target of learning is todecrease the error between the actual output and predictedoutput to a minimum level. The performance of the MLPnetwork can be enhanced if appropriate error function and

123

Arab J Sci Eng

Table 3 List of BPA used in this study

Acronym Algorithm

LM Trainlm Levenberg–Marquardt

BFG Trainbfg BFGS Quasi-Newton

RP Trainrp Resilient Backpropagation

GDX Traingdx Gradient descent with momentum andadaptive learning rate backpropagation

GD Traingd Gradient descent backpropagation

minimization algorithm are selected. In the present study, asminimization algorithm the five different backpropagationtraining algorithms were chosen. Table 3 lists the algorithmsthat are tested and the acronyms used to indicate them.

So there are five MLP networks, namely MLP networktrained with LM, MLP network trained with RP, MLP net-work trained with BFG, MLP network trained with GDXand MLP network trained with GD. We compared the per-formances of these five networks.

3.5.1 Levenberg–Marquardt Algorithm

The LM algorithm is a Hessian matrix, H -based algorithmfor nonlinear least squares optimization. The first derivativeof the network error with respect to the weights and biasesis called Jacobian matrix, J (cumulative error vector). Thecomputing of J is less complex than computing H . If theperformance function is in the form of sum of square, thenH can be approximated through J as follows:

H = J T J (3)

and if e is a vector of network errors, then the gradient canbe expressed as

g = J T e (4)

The LM algorithm updates the network weights and biasesby using both H and g

Xk+1 = Xk − [H + λI ]−1g (5)

where Xk is a vector of current weights and biases havinglearning rate, λ, and identity unit matrix I . The value of λ iszero in Newton’s algorithm using the approximate Hessianmatrix. If λ is large, then it becomes gradient descent witha small step size. Thus, λ is reduced after each successfuliteration and is increased if any tentative step increases theperformance function [42].

3.5.2 BFGs Quasi-Newton Algorithm

Quasi-Newton Algorithm is based on Newton’s methodwhich does not compute the second derivatives of the Hessianmatrix. The update of the weights and biases calculates an

approximate Hessian matrix at each iteration of the algorithmas a function of the gradient. By putting λ = 0 in Eq. (5), wecan get the basic of Newton’s method as

Xk+1 = Xk − H−1g (6)

where H is the Hessian matrix (second derivatives) of theperformance index at the current values of the weights andbiases [43].

3.5.3 Resilient Backpropagation

The purpose of the resilient backpropagation training algo-rithm is to eliminate the harmful effect of the size of thepartial derivative on the weight step [44]. Therefore, onlythe sign of the derivative is used to the weight/bias updateof the network, since the sigmoid transfer function causessmall changes of weight and biases, as the gradient can havea small magnitude. For this purpose, to determine only theweight update, let us consider each weight w jk and its updatevalue � jk(t). The weight update value is multiplied by a fac-tor α−, where 0 < α− < 1. If the final iteration generatesthe same sign of factor, then � jk(t) is again multiplied bya factor of η+, where α+ > 1. The update values are con-sidered for each weight in the above way. Finally, to reducethe overall error function each weight is changed by its ownupdate value, in the opposite direction of that weight’s partialderivative.

During two successive weight steps, the update value canbe written as follows:

� jk(t)

=

⎧⎪⎪⎨

⎪⎪⎩

α+ · � jk(t − 1), if ∂ E∂w jk

(t) ∂ E∂w jk

(t − 1) > 0

α_ · � jk(t − 1), if ∂ E∂w jk

(t) ∂ E∂w jk

(t − 1) < 0

� jk(t − 1), otherwise

(7)

where 0 < α− < 1 < α+The results from Eq. (7) show that the sign of partial deriv-

ative changes each time, which means that the last � jk is toolarge and the algorithm jumps over a local minimum. Afterreaching weight adaptation, the weight update value followsa simple rule which can be expressed mathematically as fol-lows:

�w jk(t) =

⎧⎪⎪⎨

⎪⎪⎩

−� jk(t), if ∂ E∂w jk

(t) > 0

� jk(t), if ∂ E∂w jk

(t) < 0

0, otherwise

(8)

w jk(t + 1) = w jk + �w jk(t) (9)

123

Arab J Sci Eng

However, the previous weight update will be reverted, if itsatisfies the following conditions:

�w jk(t) = −�w jk(t − 1), if∂ E

∂w jk(t)

∂ E

∂w jk(t − 1) < 0

(10)

By setting ∂ E∂w jk

(t − 1) = 0 in the �jk update rule above, thedouble penalty of the update value is avoided, which mayarise due to the backtracking weight step, as the derivative isassumed to change its sign once again. The partial derivativeof the total error is given as follows:

∂ E

∂w jk(t) = 1

2

p∑

p=1

∂ E p

∂w jk(t) (11)

Hence, the partial derivatives of the errors must be accumu-lated for all P training patterns. This means that the weightswill be updated only after the presentation of all of the train-ing patterns [45].

3.5.4 Gradient Descent with Momentum and AdaptiveLearning Rate Backpropagation

The GDX algorithm updates weight and bias values accord-ing to gradient descent momentum and an adaptive learningrate. Details of the description of the GDX algorithm will befound in [46,47].

3.5.5 Batch Gradient Descent

The GD algorithm updates the weights, and biases areupdated in the direction of the negative gradient of the per-formance function [48].

Finally, the basic algorithms for training the MLP net-works are as follows:

1. Load the data set, consisting of input and target vectors.2. Create a single hidden layer multilayered perceptron net-

work with the log-sigmoid transfer function in the hiddenlayer and linear transfer function in the output layer.

3. Allocate training and testing data set.4. Train the network using backpropagation algorithm.5. Calculate the results of the training algorithm in terms of

accuracy, sensitivity, specificity, false positive and falsenegative.

4 Data Samples and Resulting Parameters

The entire data set containing 253 samples of each input wasdivided into two different sets, i.e., training data set and test-ing data set. From 253 samples, 152 samples were selected astraining data set and the remaining 101 samples were selected

Table 4 Summary of conditions monitoring data

Number of data 253Training data 152Testing data 101Input features, zn = 6 Mean, maximum, minimum, median,

standard deviation and varianceOutput classes, ym = 2 Normal and defective condition

as testing data. For training data set, 92 were the normal con-dition and the remaining 60 were defective condition electri-cal hotspots. On the other hand, for testing data set, 61 werenormal and 40 were the defective conditions. The trainingdata set is presented during training and adjusted the networkaccording to its error. The testing data set has no effect ontraining and so provides an independent measure of networkperformance during and after training.

A summary of electrical equipment condition monitoringdata sets is shown in Table 4.

The MLP network has been trained and compared by usingfive different backpropagation training algorithms, namelyLM, BFG, RP, GDX and GD to find out the best trainingbackpropagation algorithm for classification. The number ofhidden neurons is varied from 1 to 50 for the networks to findthe best performance. The number of epochs is kept constantat 300 for all the networks. To find the optimum number ofhidden nodes, the number of epochs is kept constant whilevarying the number of hidden nodes from 1 to 50. The num-ber of hidden nodes which gives the best performance wasrecorded as the optimum number of hidden nodes for theneural network.

If T is the total number of equipment in the data set, Cis the total number of equipment condition classified cor-rectly, N is the number of normal condition equipment (Nc

correctly classified and Ni incorrectly classified) and D is thetotal number of defective condition equipment (Dc correctlyclassified and Di incorrectly classified). Then the parametersfor comparing the performance of different classifiers can bedefined as follows.

The accuracy of the network is defined as the percentageof total samples classified correctly with respect to the totalnumber of samples in the data set.

Accuracy = T

C× 100 % (12)

The specificity of the network is defined as the percentage ofnormal condition hotspots classified correctly with respectto the total number of normal condition hotspots in the dataset.

Specificity = Nc

N× 100 % (13)

Sensitivity is defined as the percentage of defective conditionhotspots classified correctly with respect to the total number

123

Arab J Sci Eng

of defective condition hotspots in the data set.

Sensitivity = Dc

D× 100 % (14)

False positive is defined as the percentage of normal condi-tion hotspots classified as defective condition with respect tothe total number of normal condition hotspots in the data set.

False positive = Ni

N× 100 % (15)

False negative is defined as the percentage of defective con-dition classified as normal with respect to the total numberof defective condition hotspots in the data set.

False negative = Di

D× 100 % (16)

5 Results and Discussions

To show the performance of backpropagation algorithms atequal hidden node, six sets testing performance of MLPnetworks trained with various backpropagation algorithms,namely LM, BFG, RP, GDX and standard GD algorithm, atthe number of hidden nodes varying from 1 to 50 (i.e., 1, 10,20, 30, 40 and 50) are shown in Table 5. The MLP networktrained with GDX and RP algorithms gave better results com-pared to the results obtained using LM, BFG and standardGD algorithms. The MLP with GDX obtained the highest74.25 % accuracy with the number of hidden nodes 20 and40. Figure 5 displays the relationship between the trainingalgorithms and the percentage of diagnosis performance atsix hidden nodes.

Tables 6 and 7 show the training and testing phase per-formances of MLP networks for each backpropagation algo-rithm which were obtained in the optimum hidden nodes.

In the training phase, the MLP network trained with RPalgorithm achieved the best accuracy of 84.21 % as com-pared to the MLP network with LM, BFG, GDX and GDalgorithms which produced 82.23, 82.90, 83.55 and 75.65 %accuracy, respectively. The MLP network trained with theRP algorithm was able to achieve the highest sensitivity of80 % and the lowest false negative 20 %. Therefore, this net-work can classify correctly the maximum 80 % of defectivecondition in the training phase and the remaining 20 % isclassified as the normal condition. Similarly, the MLP net-work trained with the LM algorithm was able to achieve thehighest specificity (95.65 %) and the lowest false-positivevalue (4.35 %). Therefore, the MLP network trained withthe LM algorithm can classify the maximum 95.65 % of nor-mal condition hotspots correctly in the training phase and therest 4.35 % is classified as the defective condition.

In the testing phase, the results show that the MLP networktrained by the LM algorithm needs lower number of hidden

Table 5 Testing performance of MLP networks with various backprop-agation algorithms

Numberof hiddennodes

Performanceanalysis (%)

MLP network with backprop-agation algorithm

LM BFG RP GDX GD

1 Accuracy 73.26 72.27 72.27 71.3 39.6

Sensitivity 42.50 42.50 42.50 42.50 100.00

Specificity 93.44 91.80 91.80 90.20 0.00

False positive 6.36 8.20 8.20 9.80 100.00

False negative 57.50 57.50 57.50 57.50 0.00

10 Accuracy 68.31 71.28 70.29 73.26 59.4

Sensitivity 50.00 52.50 65.0 60.0 2.50

Specificity 80.30 83.60 73.27 81.96 96.72

False positive 19.69 16.40 26.77 18.04 3.28

False negative 50.00 47.50 35.0 40.0 97.50

20 Accuracy 69.30 69.3 70.29 74.25 60.39

Sensitivity 57.5 55.0 62.5 60.0 10.00

Specificity 77.00 78.68 75.40 83.6 93.44

False positive 23.00 21.32 25.60 16.4 6.56

False negative 42.50 45.0 37.5 40.0 90.00

30 Accuracy 64.35 68.31 68.31 73.26 61.38

Sensitivity 52.50 65.0 60.0 60.0 5.00

Specificity 72.13 70.5 73.77 82.0 98.36

False positive 27.87 29.5 26.33 18.0 1.66

False negative 47.50 35.0 40.0 40.0 95.00

40 Accuracy 66.33 69.30 67.32 74.25 61.38

Sensitivity 50.00 57.5 60.0 60.0 12.50

Specificity 77.04 77.00 72.13 83.6 93.44

False positive 22.96 23.00 27.87 16.4 6.56

False negative 50.00 42.50 40.0 40.0 87.50

50 Accuracy 66.33 71.28 63.36 72.27 68.31

Sensitivity 67.5 52.50 47.50 62.5 57.50

Specificity 65.57 83.60 73.77 78.68 75.40

False positive 34.43 16.40 26.23 21.32 24.60

False negative 32.5 47.50 52.50 37.5 42.50

nodes than the MLP network which is trained using BFG,RP, GDX and GD algorithms, which makes its structure lesscomplex. The MLP network with GD training algorithm pro-duced the worst testing accuracy of 68.31 % as compared tothe MLP network with the LM, BFG, RP and GDX trainingalgorithms which produced 73.26, 73.26, 76.20 and 74.25 %accuracy, respectively. The MLP network with the RP algo-rithm produces the highest testing performance with an accu-racy of 76.20 %. The MLP network trained with the LM algo-rithm was able to achieve the highest specificity (93.44 %)and the lowest false positive (6.56 %). Therefore, this net-work has maximum capacity to classify the normal conditioncorrectly with respect to the total number of normal con-dition hotspots in the testing data set. Similarly, the MLP

123

Arab J Sci Eng

Acc

urac

y va

lues

50403020101

50403020101

50403020101 50403020101

75

70

65

60

55

50

45

40

Var iable

RPGDXGD

LMBFG

No of nodes

No of nodes

No of nodes

No of nodes No of nodes

Sen

sitiv

ity v

alue

s

50403020100

100

80

60

40

20

0

100

80

60

40

20

0

100

80

60

40

20

0

100

80

60

40

20

0

Spe

cific

ity v

alue

s

Fal

se p

ositi

ve v

alue

s

Fal

se n

egat

ive

valu

es

(a) Accuracy (%)

(b) Sensitivity (%) (c) Specificity (%)

(d) False positive (%) (e) False negative (%)

Fig. 5 Relationship between the backpropagation training algorithms and the percentage of diagnosis performance at six hidden nodes. a Accuracy,b sensitivity, c specificity, d false positive and e false negative

network trained with the RP algorithm was able to achievethe highest sensitivity (67.50 %) and the lowest false nega-tive (32.50 %). Therefore, the MLP network trained with theRP algorithm has the highest capacity to classify the defec-tive condition hotspots correctly with respect to the totalnumber of defective condition hotspots in the testing dataset.

Figure 6 shows the charts having the training and testingperformance of all backpropagation algorithms at optimumhidden nodes. From the charts, we can see the clear distinc-tion among the performance of backpropagation algorithms.

By analysis of the training and testing performances ofall the MLP networks at the optimum hidden node, the over-all results show that the MLP network with the RP training

123

Arab J Sci Eng

Table 6 Training performance(%) of the MLP networks at theoptimum hidden node

Types of MLP networks MLP withLM

MLP withBFG

MLP withRP

MLP withGDX

MLP withGD

Hidden node 1 3 4 20 50

Accuracy 82.23 82.90 84.21 83.55 75.65

Sensitivity 61.67 78.33 80.00 70.00 65.00

Specificity 95.65 85.87 86.96 92.40 82.60

False positive 4.35 14.13 13.04 7.60 17.40

False negative 38.33 21.67 20.00 30.00 35.00

Table 7 Testing performance(%) of the MLP networks at theoptimum hidden node

Types of MLP networks MLP withLM

MLP withBFG

MLP withRP

MLP withGDX

MLP withGD

Hidden node 1 3 4 20 50

Accuracy 73.26 73.26 76.20 74.25 68.31

Sensitivity 42.50 50.00 67.50 60.00 57.50

Specificity 93.44 88.50 82.00 83.60 75.40

False positive 6.56 11.50 18.00 16.40 24.60

False negative 57.50 50.00 32.50 40.00 42.50

Fig. 6 Chart showing thetraining and testing performanceof all backpropagationalgorithms at optimum hiddennodes. a Accuracy, b sensitivity,c specificity, d false positive ande false negative

0

20

40

60

80

100

LM BFG RP GDX GD

Accuracy(%)training

testing

0

20

40

60

80

100

LM BFG RP GDX GD

Sensitivitytraining

testing

0

20

40

60

80

100

120

LM BFG RP GDX GD

Specificity(%)

training

testing

0

5

10

15

20

25

30

LM BFG RP GDX GD

False positive(%)training

testing

0

10

20

30

40

50

60

70

LM BFG RP GDX GD

False negative(%)

training

testing

(a) (b)

(c) (d)

(e)

123

Arab J Sci Eng

algorithm produces better performance than MLP networkstraining with the LM, BFG, GDX and GD algorithms. TheMLP network with the RP algorithm presents several advan-tages as compared to the MLP networks with LM, BFG,GDX and GD training algorithms. It produced the highesttraining and testing phase accuracy of 84.21 and 76.20 %,respectively. Therefore, it classified correctly 128 conditionsout of 152, and 77 conditions out of 101 conditions correctlyin the training and testing phases, respectively. The resultsof training and testing accuracy at optimum hidden node areenough to prove that the MLP network trained with RP is thebest performer. Besides, the network also achieved maximumsensitivity in both training and testing phases 80 % (classi-fication correctly of 48 defective conditions out of 60) and67.50 % (correctly classified 27 defective conditions out of40 conditions), respectively. Also, it showed the lowest falsenegative i.e. classification of defective condition as normalcondition 20 % (12 out of 60) and 32.50 % (13 out of 40) inthe training and testing phases, respectively.

6 Conclusion

In the present study, the performance of multilayered per-ceptron networks for various backpropagation algorithms,namely LM, RP, BFG, GDX and GD were evaluated to clas-sify the conditions of electrical hotspots using infrared ther-mography technology. The conditions of hotspots were clas-sified into two, namely normal and defective. Six intensityfeatures (i.e., mean, maximum, minimum, median, standarddeviation and variance) were used as the inputs of MLP net-works for each training algorithm. The performance of thetraining algorithms was evaluated in terms of accuracy, sen-sitivity, specificity, false-positive and false-negative results.The MLP network trained using the RP algorithm was ableto show the best training and testing performance, comparedwith the standard GD backpropagation network and otherbackpropagation algorithms. It obtained 84.21 and 76.2 %training and testing accuracy, respectively.

Acknowledgments The authors gratefully acknowledge the contribu-tion of Dr. Kamarul Hawari bin Ghazali and Mr. Shawal Jadin, UniversitiMalaysia Pahang for helping to capture thermal images. The work wassupported by the Fundamental Research Grant Scheme (203/PELECT/6071211), Universiti Sains Malaysia.

References

1. Lahiri, B.B.; Bagavathiappan, S.; Jayakumar, T.; Philip, J.: Medicalapplications of infrared thermography: a review. Infrared Phys.Technol. 55, 221–235 (2012)

2. Azenha, M.; Faria, R.; Figueiras, H.: Thermography as a techniquefor monitoring early age temperatures of hardening concrete. Con-str. Build. Mater. 25, 4232–4240 (2011)

3. Balaras, C.A.; Argiriou, A.A.: Infrared thermography for buildingdiagnostics. Energy Build. 34, 171–183 (2002)

4. Barreira, E.; de Freitas V. P.: Evaluation of building materials usinginfrared thermography. Constr. Build. Mater. 21, 218–224 (2007)

5. Švantner, M.; Vacikova, P.; Honner, M.: IR thermography heat fluxmeasurement in fire safety applications. Infrared Phys. Technol.55, 292–298 (2012)

6. Kaplan, H.: Practical Applications of infrared thermal sensing andimaging equipment, 2nd edn. SPIE Optical Engineering Press,Bellingham (1999)

7. Al-Kassir, A.R.; Fernandez, J.; Tinaut, F.V.; Castro, F.: Thermo-graphic study of energetic installations. Appl. Therm. Eng. 25 (2–3), 183–190 (2005)

8. Royo, R.; Torrella, E.; Martín, A.: Operating cycle analyses of dif-ferent refrigeration systems using infrared thermography. InfraredTrain Center FLIR Syst., pp. 103–113 (2005)

9. Ge, Z.; Du, X.; Yang, L.; Yang, Y.; Li, Y.; Jin, Y.: Performancemonitoring of direct air-cooled power generating unit with infraredthermography. Appl. Therm. Eng. 31, 418–424 (2011)

10. Jadin, M.S.; Taib, S.: Recent progress in diagnosing the reliabilityof electrical equipment by using infrared thermography. InfraredPhys. Technol. 55, 236–245 (2012)

11. Griffith, B.; Türler, D.; Goudey, H.: Infrared thermographic sys-tems: a review of IR imagers and their use. Wiley, Berkeley (2001)

12. Hou, N.: The infrared thermography diagnostic technique of high-voltage electrical equipments with internal faults. In: Proceedingsof IEEE International Conference on Power System Technology,Beijing, China, pp. 110–115 (1998)

13. Lizak, F.; Kolcun, M.: Improving reliability and decreasing lossesof electrical system with infrared thermography. Acta Electrotech-nica Inf. 8(1), 60–63 (2008)

14. Korendo, Z.; Florkowski, M.: Thermography based diagnostic ofpower equipment. IEEE Power Eng. J. (2001)

15. Epperly, R.A.; Heberlein, G.E.; Eads, L.G.: A tool for reliabilityand safety: predict and prevent equipment failures with thermog-raphy. In: IEEE Conference on Petroleum and Chemical Industry,pp. 59–68 (1997)

16. Jadin, M.S.; Kabir, S.; Taib, S.: Thermal imaging for qualitative-based measurements of thermal anomalies in electrical compo-nents. In: IEEE Conference ECPC, Saudi Arab, 24–26 June, 2011

17. Garnaik, S.P.: Infrared Thermography : a versatile technology forcondition monitoring and energy conservation. Web. http://www.reliabilityweb.com. Accessed 11 Aug 2011

18. Kregg, M.A.: Benefits of using infrared thermography in utilitysubstations. In: Proceedings of SPIE, pp. 249–257 (2004)

19. Braunovic, M.; Myshkin, N.K.; Konchits, V.V.: Electrical ContactFundamentals, Applications and Technology. Ch 11 (MonitoringTechnologies), CRC Press, Taylor and Francis Group, Boca Raton(2007)

20. Rakhshandehroo, G.R.; Vaghefi, M.; Aghbolaghi, M.A.: Forecast-ing ground water level in Shiraz Plain using artificial neural net-works. Arab. J. Sci. Eng. 37, 1871–1883 (2012)

21. Kadri, M.A.: System Identification of a cooling coil using recurrentneural networks. Arab. J. Sci. Eng., pp. 1–11 (2012)

22. Soleymani Yazdi, M.R.; Seyed Bagheri, Gh.; Tahmasebi, M.: finitevolume analysis and neural network modeling of wear during hotforging of a steel splined hub. Arab. J. Sci. Eng. 37, 821–829 (2012)

23. Mitra, P.; Mitra, S.; Pal, S.K.: Staging of cervical cancer with softcomputing. IEEE Trans. Biomed. Eng. 47(7), 934–940 (2000)

24. Mashor, M.Y.; Mat Isa, N.A.; Othman, N.H.: Classification of papsmear tests using neural networks. In: IFMBE Proceedings of Inter-national Conference on Biomedical Engineering, pp. 21–24 (2004)

25. Sudir, D.S.; Ashok, A.; Amol, P.P.: Neural network aided breastcancer detection and diagnosis. In: Proceedings of the 7th WSEASInternational Conference on Neural Networks, USA, pp. 158–163(2006)

123

Arab J Sci Eng

26. Kates, R.; Harbeck, N.; Schmitt, M.: Prospects for clinical decisionsupport in breast cancer based on neural network analysis of clini-cal survival data. In: 4th International Conference on Knowledge-Based Intelligent Engineering System & Allied Technologies, pp.764–767 (2000)

27. Huda, A.S.N.; Taib, S.; Jadin, M.S.; Ishak, D.: A semi-automaticapproach for thermographic inspection of electrical installationswithin buildings. Energy Build. 55, 585–591 (2012)

28. Almeida, C.A.L.; Braga, A.P.; Nascimento, S.; Paiva, V.; Mar-tins, H.J.A.; Torres, R.; Caminhas, W.M.: Intelligent thermographicdiagnostic applied to surge arresters: a new approach. IEEE Trans.Power Deliv. 24(2), 751–757 (2009)

29. Shafi’i, M.A.; Hamzah, N.: Internal fault classification using arti-ficial neural network. In: Proceedings of 4th International PowerEngineering and Optimization Conference, pp. 352–357 (2010)

30. Aggarwal, K.K.; Singh, Y.; Chandra, P.; Puri, M.: Bayesian regu-larization in a neural network model to estimate lines of code usingfunction points. J. Comput. Sci. 1(4), 505–509 (2005)

31. Detecting electrical unbalance and overloads. Web. http://www.fluke.com/Fluke/inen/solutions/thermal-imaging/Thermal-Imaging-Detecting-Electrical-Unbalance-and-Overloads.htm.Accessed 01 Jan 2012

32. Thermography in electrical installation Web. http://vds.de/fileadmin/vds_publikationen/vds_2858en_web.pdf. Accessed 21Mar 2012

33. Huda, A.S.N.; Taib, S.; Ishak, D.: Necessity of quantitative basedthermographic inspection of electrical equipments. In: Progressin Electromagnetics Research Symposium Proceedings, 28–30March, kuala Lumpur, Malaysia, pp. 1852–1855 (2012)

34. Chou, Y.C.; Yao, L.: Automatic diagnosis system of electricalequipment using infrared thermography. In: International Confer-ence of Soft Computing and Pattern Recognition, pp. 155–160(2009)

35. Standard for Infrared Inspection of Electrical Systems &Rotating Equipment, Infraspection Institute. Web. http://www.armco-inspections.com/files/ir/Electrical%20Rotating%20Std.pdf.2008. Accessed 11 June 2011

36. ASTM, ASTM E 1934: Standard Guide for Examining Electri-cal and Mechanical Equipment with Infrared Thermography. WestConshohocken, Pennsylvania, ASTM International (2005)

37. Why thermography is good for your business, Web. http://www.newark.com/pdfs/techarticles/Why_Thermography_is_Good_for_Business.pdf. Accessed 01 Aug 2012

38. Daims, H.; Lucker, S.; Wagner, M.: daime, a novel image analy-sis program for microbial ecology and biofilm research. Environ.Microbiol. 8, 200–213 (2006)

39. Cybenko, G.: Approximations by superposition of a sigmoidalfunction, Math. Control, Signal Syst. 2, 303–314 (1989)

40. Funahashi, K.: On the approximate realisation of continuous map-pings by neural networks. Neural Networks. 2, 183–192 (1989)

41. Mat Isa, N.M.; Al-Batah, M.S.; Zamli, K.Z.; Azizli, K.A.; Joret,A.; Mat Nor, N.R.: Suitable features selection for HMLP and MLPnetworks to identify the shape of aggregate. Constr. Build. Mater.22, 402–410 (2004)

42. Hagan, M.T.; Menhaj, M.: Training feed-forward networks withthe Marquardt algorithm. IEEE Trans. Neural Netw. 5(6), 989–993(1999)

43. Dennis, J.E.; Schnabel, R.B.: Numerical Methods for Uncon-strained Optimization and Nonlinear Equations. Prentice-Hall,Englewood Cliffs (1983)

44. Riedmiller, M.; Braun, H.: A direct adaptive method for faster back-propagation learning: The RPROP algorithm. In: Proceedings ofthe IEEE International Conference on Neural Networks, USA 1993

45. Al-Naima, F.M.; Al-Timemy, A.H.: Resilient back propagationalgorithm for breast biopsy classification based on artificial neuralnetworks. In: Al- Dahoud Ali (ed.) Computational Intelligence andModern Heuristics, InTech, pp. 145–158 (2010)

46. Plagianakos, V.P.; Sotiropoulos, D.G.; Vrahatis, M.N.: AnImproved Backpropagation Method with Adaptive Learning Rate.Technical Report: Dept. of Mathematics, University of Patras, vol.2, pp.1–8 (1998)

47. Bai, Y.; Zhang, H.; Hao, Y.: The performance of the backpropaga-tion algorithm with varying slope of the activation function. Chaos,Solitons Fractals. 40, 69–77(2009)

48. Hagan, M.T.; Demuth, H.B.; Beale, M.H.: Neural Network Design.PWS Publishing, Boston (1996)

123