new thereal-timemobileapplicationforclassifyingofendangered...

Research ArticleThe Real-Time Mobile Application for Classifying of EndangeredParrot Species Using the CNN Models Based onTransfer Learning

Daegyu Choe 1 Eunjeong Choi 1 and Dong Keun Kim 2

1Department of Computer Science Graduate School of Sangmyung University Seoul Republic of Korea2Department of Intelligent Engineering Information for Human and Institute of Intelligent Informatics TechnologySangmyung University Seoul Republic of Korea

Correspondence should be addressed to Dong Keun Kim dkimsmuackr

Received 11 October 2019 Accepted 7 January 2020 Published 9 March 2020

Guest Editor Malik Jahan Khan

Copyright copy 2020 Daegyu Choe et al )is is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Among the many deep learning methods the convolutional neural network (CNN) model has an excellent performance in imagerecognition Research on identifying and classifying image datasets using CNN is ongoing Animal species recognition andclassification with CNN is expected to be helpful for various applications However sophisticated feature recognition is essentialto classify quasi-species with similar features such as the quasi-species of parrots that have a high color similarity )e purpose ofthis study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN modelbased on transfer learning (some parrots have quite similar colors and shapes) We acquired the images in two ways collectingthem directly from the Seoul Grand Park Zoo and crawling them using the Google search Subsequently we have built advancedCNNmodels with transfer learning and trained them using the data Next we converted one of the fully trained models into a filefor execution on mobile devices and created the Android package files )e accuracy was measured for each of the eight CNNmodels)e overall accuracy for the camera of the mobile device was 94125 For certain species the accuracy of recognition was100 with the required time of only 455ms Our approach helps to recognize the species in real time using the camera of themobile device Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area

1 Introduction

With the development of information technology deeplearning-based image processing and classification is widelyused in various applications [1] In particular the demandfor image classification is increasing [2] Deep learning-based classifiers such as a convolutional neural network(CNN) increase the classification performance for variousobjects [2] A common task in image processing is identi-fying similar types of objects with machine learning methodsto classify and cluster animals [3] Systems that automati-cally identify and classify animal species have become es-sential particularly for the study of endangered species [4]During the customs clearance of animals and plants humanscan directly examine the species to identify individualspecies but this can be inefficient in terms of time and cost

To improve the efficiency automated classification of speciescan be conducted on mobile devices However this wouldrequire solving the problems of classifying species withsimilar shades of colors and shapes Hence custom machinelearning models are needed to classify endangered speciesand address the complicated characteristics of animal imagesfor specific applications

Although various machine learning models can classifyimages of different animals it remains a challenge to dis-tinguish animal species )is is because there are somespecies with a high color similarity It is a complicatedprocess that requires expertise even for human beings )eCNN models are efficient modern recognition methodsUnlike the traditional image classification methods [5] aconvolutional neural network uses multilayer convolution toautomatically extract and combine features )ese

HindawiMobile Information SystemsVolume 2020 Article ID 1475164 13 pageshttpsdoiorg10115520201475164

algorithms are designed to be performed independently andare trained to solve specific tasks Moreover the neuralnetwork models have to be rebuilt once the feature-spacedistribution changes To overcome these disadvantages weadopted the transfer learning method to classify the en-dangered parrot quasi-species in this study Transferlearning is a machine learning technique in which a modeltrained for one task is reused for another related task [6]Among many ways to deploy deep learning models inproduction one of the easiest ways is to deploy it on mobiledevices )e advantages are that mobile devices are popularand easy to use Users can get an answer in a few touchesMoreover deep learning models can receive large amountsof data in real time thanks to the camera of the mobiledevice When deploying a deep learning model on a mobiledevice two aspects should be considered model file size andprocess speed If the size is too large it is impossible todeploy the model on a mobile device If the process is slow itwill cause inconvenience for the users

In this study a real-time mobile application was de-veloped to classify endangered parrot quasi-species using theCNN models based on transfer learning To clarify thepurpose of this study we suggested the followinghypotheses

(i) )e designed CNN-based transfer learning modelscan classify endangered parrot quasi-species withhigh color similarity

(ii) )e developed application can embed the designedCNN-based training

)e rest of this paper is organized as follows Section 2presents related work on transfer learning with CNNmodels Section 3 explains our real-time mobile applicationSection 4 presents the experimental results of the classifi-cation of endangered parrot species for the designed mobileapplication Section 5 discusses the contribution of thedesigned mobile application and the classification resultsFinally Section 5 concludes this study

2 Related Work

21 CNN Models and Image Classification for AnimalsMany well-known CNN model architectures exist for var-ious applications In 2016 Microsoft Research presented asolution for the problem of building deep models withshortcut connections [7] Zoph and Le also presented amethod to automatically find a new optimized model ar-chitecture based on policy gradients called neural archi-tecture search at ICLR 2017 [8] Szegedy et al have won theILSVRC 2014 with a top-5 test error of 67 with a modelbuilt on the concept of ldquonetwork in networkrdquo )e idea ofthis model is to reduce the computing cost using dimen-sionality reduction constructing the network by stackingconvolution operations using filters of various sizes andthen combining them later [9] Another model created bySzegedy et al is Inception-ResNet which combines theresidual connections presented by Microsoft Research [10]

Many relevant studies exist to preserve the diversity ofspecies To acquire the data necessary for these studies

unmanned cameraswere installed to acquire images of thecreatures However human resources are wasted on pro-cessing the obtained data Because humanrsquos judgment issubjective the accuracy is inevitably deteriorated )ereforeit is essential to create a system that automatically processesand classifies animal images Norouzzadeh et al in theldquoSnapshot Serengeti Projectrdquo said that processing of in-formation from animal image datasets by human beings istime-consuming hence much data remains unprocessed)ey presented a system in which a machine can determinewhere the images belong to and check the number of entitiesand their behaviors in images [3] Nguyen et al also created aCNNmodel to classify three of themost commonly observedanimal species in Victoria Australia and showed the realtest results [11] Zhuang et al introduced a deep learningmodel that automatically annotates marine biological imagedata without relying on human experts )ey experimentedwith their model with data from SeaCLEF2017 [12] In thisstudy we also propose a system to classify image data ac-quired in real time using the camera of a mobile device

22 Transfer Learning Transfer learning is a state-of-the-arttechnique in deep learning research Before the advent ofthis technique people had to create and train a model fromscratch It was difficult to invent a model with remarkableperformance on a specific task because of the lack ofcomputing infrastructure Moreover it was impossible tocollect enough meaningful data required to train a modelalthough many researchers attempted to gather themHowever various transfer learning methods have beenproposed for transferring knowledge in the context offeatures instant weights parameters or relationship in-formation between data samples in a domain [13ndash16]

Figure 1 shows four steps of creating a complete modelusing transfer learning First we build an architecture of themodel and train it on a large representative dataset Secondwe delete the final layer (known as ldquoloss outputrdquo) )ird wereplace it with another layer whose job is to finish the specifictask Fourth we train a new model with a relatively smalldataset suitable for the purpose Transfer learning is literally totransfer the job of extracting features from data to the pre-trained model For example a model pretrained on theImageNet dataset can detect low-level features on a bird image(such as curves outlines and lines) because these low-levelfeatures are almost the same in other animal images )eremaining task is to tune the high-level layers of the featureextractor and the final layer that classifies the bird (the processis called fine tuning) Some studies have already appliedtransfer learning [17 18] Transfer learning is expected tocompensate for the lack of data time and computing

3 Implementation of a Real-Time MobileApplication to Classify EndangeredParrot Quasi-Species

31 SystemDesign and Image Classification inMobileDevices)e system is divided into four parts as shown in Figure 2First we preprocess the data to prepare it for deep learning

2 Mobile Information Systems

Second we create and train a classifier using the pre-processed data)ird we convert the generated model into afile that can be deployed on a mobile device Finally wedeploy the model In this section we describe data pre-processing and the process of creating and training the deeplearning model

In our study we used Python (Anaconda) for the thirdstep and Android Studio for the final step Data were pre-processed using a Python library called ldquoimgaugrdquo [19] thatprovides image preprocessing methods (ldquoImage-Transformationrdquo ldquoAdditiveGaussianNoiserdquo ldquoCoarseDrop-outrdquo ldquoBilateralBlurrdquo etc) We imported the ldquoimgaugrdquolibrary into our project in the Anaconda Jupyter notebookenvironment and performed data augmentation for theoriginal images )e obtained images were saved in thefolders together with the original images

To develop an application TensorFlow Lite provides amethod that converts the generated model into a Tensor-Flow Lite FlatBuffer format file (tflite) which can bedeployed on a mobile device According to the officialTensorFlow Lite website FlatBuffer is an open-source cross-platform serialization library that serializes data efficientlyTensorFlow Lite supports the conversion of files created byTensorFlow concrete functions and Keras [20] We insertedthis converted file into the demo project provided byTensorFlow Lite and then built the project After this stepwe created an Android package file (APK) and installed theapplication on a device Figure 3 shows the overall processLi et al developed an optimized modeling technique formobile devices using their reduction module group con-volution and self-attention module )ey claimed that thismodel was efficient for mobile applications compared withother models [21] Subsequently we explain how to deploy aCNN model created by TensorFlow Lite on a mobile device

We use the Keras library to create and train deeplearning models Keras is a high-level open-source neuralnetwork API written in Python It was developed as a part ofthe Open-Ended Neuro-Electronic Intelligent Robot Op-erating System (ONEIROS) project A model produced by

Keras is built using a fast and intuitive interface based onTensorFlow CNTK and )eano [22] In the field of com-puter vision some model architectures that can effectivelyclassify images have been previously introduced and Kerasprovides them as open-source code [23] In this study wepropose a way to customize these models train them andverify their performance

32 Data Augmentation One of the biggest limitations indeep learning model development is that it requires a largedataset )ousands millions or even more data samples arerequired to create a reliable deep learning model )eselimitations can be overcome by manipulating and trans-forming a small amount of data )is is called data aug-mentation Data augmentation techniques have been used inmany studies [24 25] )e techniques include randomcropping horizontal flipping brightness modification andcontrast modification As illustrated in Figure 4 we extendedthe dataset by the horizontal and vertical flipping Figure 4shows the extended dataset as a result of four parrot speciesrsquodata augmentation For this task we imported ldquoimgaugrdquoPython library (as explained in Section 31) It contains theldquoSequentialrdquo method and manipulation techniques can beset as the parameters of this method [19] In this studybecause we only wanted to augment the images by thehorizontal and vertical flipping to check if the model canclassify the quasi-species of parrots with a high color sim-ilarity we inserted ldquoFliplrrdquo and ldquoFlipudrdquo objects Finally14000 images including the original data were gathered (seethe details in Section 35)

33 Feature Extraction and theCNNModel Nguyen et al setthe two experimental scenarios on themodel architectures ofLite AlexNet VGG-16 and ResNet50 to classify wildlifeimages [11] )e first scenario was to train the model fromscratch and the second one was to use a technique calledldquofeature extractionrdquo that imports weights that had beenpretrained on large images in ImageNet To monitor andclassify enormous animal image data some pretrainingtechniques are needed to familiarize the model withextracting local features of a new image Feature extractionsolves the problem It customizes the top layer of a model(fully connected layer) and lets the pretrained CNN extractthe characteristics of the image For our study we used thefeature extraction technique we validated its performanceby comparing it with the model with randomly initializedweights )e first model was generated with the pretrainedweights in ImageNet Our purpose was to verify if the modelcan capture the local differences of two species which arevery similar such as ldquoCacatua galeritardquo and ldquoCacatuagoffinianardquo

According to Lin et al the fully connected layer com-monly used in traditional CNN models is likely to overfitdespite using the dropout )ey proposed a global averagepooling (GAP) technique that inserts the average value ofeach feature map into a vector and links it into the input ofthe SoftMax layer directly instead of a fully connectedlayer [26]

Conv1Conv1

Conv2Conv2

Max poolingMax pooling

Fully connectedFully connected

SoftmaxSoftmax

Data and labels(eg ImageNet)

Output Output

Transfer

Features

Targetdata and labels

Figure 1 Diagram of the transfer learning

Mobile Information Systems 3

GAPi 1

mxn1113944

m

a

1113944

n

b

x (1)

Formula (1) presents the approach suggested in theirstudy GAP is a vector of the average values of feature mapsfrom the last convolutional layer GAPi indicates an elementof the vector Herem is the number of rows in a feature mapand n is the number of columns in a feature map )emeaning of the left term is summing all values in the featuremap and then dividing them by m multiplied by n )epurpose is to obtain the average value of the feature mapGAP calculates averages of feature maps that are the out-comes of the convolutional process (Figure 5) Next itcreates a vector that consists of the average values

According to their proposal GAP has the followingadvantages over a fully connected layer First the compu-tational cost can be reduced by decreasing the number ofparameters to be handled by a human (hyperparameters)Second some model parameters can be eliminated to reduceoverfitting)erefore there is no need to rely on dropout Inthis study we will use GAP instead of a traditional fullyconnected layer to take advantage of this technique

We imported the ResNet50 NASNetMobile Incep-tionResNetV2 and InceptionV3 models from the Keraslibrary for feature extraction )e imported models used

convolutional layers initialized with weights that had beenpretrained on ImageNet A global average pooling layer anda dense layer with SoftMax were added after the convolu-tional layers (Figure 6) )e experiment compared two typesof initialization weights of ImageNet and random valuesMoreover we use a hyperparameter search library calledldquoHyperasrdquo to optimize hyperparameters (such as optimizerand learning rate) without the researcherrsquos effort

34 Transfer Learning As explained in Section 2 we canapply the convolutional layers of a pretrained model toanother classifier Because an image consists of pixels thelocal features of the image are almost the same as in otherimages )e convolutional layers can capture these patternsusing the pretrained weights At this point the modelrsquosability to perform the abstraction of local parts affects themodelrsquos performance According to Krizhesky et al the testresults for the models with transfer learning showed thattheir top-5 accuracy was higher than in other cases [27]Transfer learning does not train the convolutional layers butonly lets them extract the features and then passes theextracted features to the classification layers Moreoverthere is an advanced technique to improve the model (calledfine tuning) that trains the high-level layers of the con-volutional layers and the classification layer together In ourstudy we experimented with themodels described in Section33 (ResNet50 NASNetMobile InceptionResNetV2 andInceptionV3) trained by transfer learning using the weightsof ImageNet (Figure 7)

35 Experiments Parrots are among the most commonendangered species in South Korea because of socialproblems such as smuggling Moreover parrots are includedin the list of the most endangered species by the Conventionon International Trade in Endangered Species of Wild Floraand Fauna (CITES) (Table 1) We have previously studiedparrots of distinct colors and shapes with conventional CNNmodels [28] However in this study we hypothesize that the

Detection

Augmentation

Take a picture

Preprocess the image

Normalization

Extract the features

A trained neural networkclassifies the species

Display the results

New image recognition using the camera

Figure 2 System configuration and scenario for classifying endangered parrot species using a mobile device

Trained TensorFlowmodel

TensorFlow liteconverter

TensorFlow lite modelfile (tfkeras)

Android app

JAVA API

C++ API

Interpreter Operators

Android neuralnetworks API

Figure 3 TensorFlow Lite conversion process graph


CNN models with transfer learning can classify the quasi-species well despite similar colors and patterns )is ex-periment used 14420 parrot images)e parrots were of fourspecies and we used 3605 images per species As shown inTable 1 the four parrot species are Cacatua goffinianaCacatua galerita Ara chloroptera and Psittacus erithacusAmong these species Cacatua goffiniana and Cacatuagalerita have a high color similarity Morphological infor-mation is very important to classify the parrot images usingCNN )e morphological features of each species are shownin Table 1 [29] Parrot images were divided into threesubsets training validation and test sets )ey were crawledfrom Google and YouTube )ere were 980 images perspecies originally but we divided these into two groups anduse only 875 for training because of the information leak3500 images were produced by data augmentation 2800images were for training and 700 images were for validation)e test set has 420 images including 100 crawled imagesand 5 images provided by the Seoul Grand Park per eachspecies Because we focused on the color similarity of twospecies we did not do any data augmentation affecting thecolor of images )us 2800 images for the training set and700 images for the validation set were provided to themodels for each species )e test set did not undergo theprocess of data augmentation because it is not effective to usethe augmented data not affecting the color for the actual test)e testing is divided into two steps After the training wecarried out the test of each modelrsquos performance by com-paring the confusion matrix and F1-score values for 420 testsamples Next we converted the file into a FlatBuffer formatdeployed it on a mobile device and then verified the resultsby using the video data obtained from the Seoul Grand Park

Figure 8 depicts the entire experiment process Originaldata were augmented using the ldquoimgaugrdquo library as

described in Section 32 )e image classifier was createdusing the Keras API in TensorFlow a powerful tool toconstruct a deep learning modelWe focused on a pretrainedmodel for transfer learning hence we imported themodels as shown in Figure 7 For example ldquotensor-flowkerasapplicationsresnetResNet50rdquo can set theweights initialization type [30] We can obtain the desiredresults by setting the keyword parameter ldquoweightsrdquo toldquoimagenetrdquo )e models were completed with stacking a GAPlayer and a dense layer Once themodelsrsquo trainingwas completewe evaluated their performancewith the test data using t ldquoscikit-learnrdquo Python library [31 32] Next we converted it into aldquoFlatBufferrdquo file to be deployed on a mobile device [33] Finallywe can see the result on a device as illustrated in Figure 9

4 Results

41 Experimental Results Figure 10 shows the learningcurves of training accuracy for eight models ResNet50NASNetMobile InceptionResNetV2 and InceptionV3 withtwo types of initialization pretrained ImageNet weights orrandom numbers (as described in the previous section) )ehorizontal axis shows the number of training iterations onthe complete train dataset )e vertical axis shows thetraining accuracy (05 means the model correctly classifiedhalf of the data and 1 means a perfect classification) Asdepicted in Figure 10 performance of the models was poorafter the first epoch but additional iterations improved theaccuracy After approximately twenty epochs the accuracyof each model converged at 1 with no noticeable im-provement afterward Notably the models that were ini-tialized with the ImageNet weights and had nontrainableconvolutional layers outperformed the others (we can checkthat the curves are located higher) Besides their accuracy

Augmentation

Augmentation

Augmentation

Augmentation

Figure 4 Data augmentation for images of endangered parrot species


converged faster Figure 11 illustrates the learning curves ofvalidation accuracy for the models )e models were evalu-ated on the validation data after each epoch )erefore theaccuracymeasures the quality of predictions for the validationdata )e curves look relatively uneven compared with theprior ones )is is because the models had never seen thesedata before )e models learned some features of parrotsusing the training images and we tested what they learnedusing the validation data )e models experienced some

failures repeatedly However their accuracy converged to apoint of minimal error Likewise the accuracy of ImageNet-initialized models is typically better than the others Bothgraphs do not show any obvious drop as time passes (look atboth graphs after twenty epochs) )us overfitting did notoccur Overfitting refers to the models that perform well onthe training set but not on the validation set

)e reason why epoch number is thirty is because wechecked that it is useless to exceed thirty We set some

[Featuremap calculation using Relu]

Featuremapabc = max((Weightc)T lowast xab 0)

where (a b) is a pixel index and c indicates a index of the channels

[Global Average Pooling]

where m is the number of rows n is the number of columns in a featuremap

1 3 2

05 0

2 50

A calculated feature map of the last convolutional layer

2

Global averagepooling vector

GAPi = 1mxn (summ

a sumnb xa b)

GAPi = 1mxn (summ

a sumnb xa b)

PPG

Figure 5 Concept diagram of global average pooling

ClassificationFeature extraction

Convolutional layersFeature maps

GAP

Denselayer

Figure 6 Convolutional layers and feature maps for feature extraction of endangered parrot species


Parrotimages

ResNet50 NASNetMobile InceptionResNetV2 InceptionV3

3

Global average pooling

Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02

Cacatua goffiniana 00

Psittacus erithacus 00

Ara chloroptera 01

Cacatua galerita 099



Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers

Input layerParrotimages

Parrotimages

Parrotimages


Table 1 Examples of four endangered parrot species

Picture

Name Red and green macaw Sulphur-crested cockatoo Goffinrsquos cockatoo Gray parrotScientificname Ara chloroptera Cacatua galerita Cacatua goffiniana Psittacus erithacus

Appearance

Flight feathers back rumpdarker red

Tail-coverts blueMedian wing-covertsscapulars tertials greenTail dark red tipped blueBare face with conspicuous

lines of red feathers

Little yellow on ear-coverts orbases to feathers of head and

underparts

Short blunt billLores and bases to feathers ofhead salmon-pink palest blue

Almost white eye-ring

Gray parrot with shortsquarish red tail

Citesappendices Appendix II Appendix II Appendix I Appendix I


Figure 9 Graphical user interface example of the designed system

1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Learning curves for train accuracyResNet50 (ImageNet)NASNetMobile (ImageNet)InceptionResNetV2 (ImageNet)InceptionV3 (ImageNet)

ResNet50 (random)NASNetMobile (random)InceptionResNetV2 (random)InceptionV3 (random)

Figure 10 Learning curves of each modelrsquos train accuracy

Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result

FlatBuffer fileconverter

Figure 8 Overall experiment process


callback functions when we called the ldquomodelfit()rdquo in ourexperiment ldquoEarlyStopping()rdquoand ldquoReduceLROnPlateau()rdquoIt would have been stopped if the validation accuracy hadnot been improved during five epochs We saw that thetraining epoch never exceeded twenty-five so we set thenumber of epochs to thirty Learning rate started from 0001and decreased gradually by 003 if the validation accuracyhad not been improved during three epochs until the

termination of training When we called ldquomodelcompile()rdquowe set loss equals to ldquocategorical_crossentoryrdquo metricsequals to ldquoaccrdquo and optimizer equals to ldquoAdamrdquo

Table 2 shows the confusion matrix for all models Aconfusion matrix is an evaluation approach that checks theperformance of a classifier for all labels Every model inthis study is included and each row shows the perfor-mance of the model depending on the labels For instance

1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Learning curves for validation accuracyResNet50 (ImageNet)NASNetMobile (ImageNet)InceptionResNetV2 (ImageNet)InceptionV3 (ImageNet)


Figure 11 Learning curves of each modelrsquos validation accuracy

Table 2 Confusion matrix

ResNet50 (ImageNetrandom) PredictionActual Ara chloroptera Cacatua galerita Cacatua goffiniana Psittacus erithacus

Ara chloroptera 10092 01 03 510Cacatua galerita 01 9866 638 10Cacatua goffiniana 02 1540 8855 28Psittacus erithacus 54 15 106 8990

NASNetMobile (ImageNetrandom) PredictionActual Ara chloroptera Cacatua galerita Cacatua goffiniana Psittacus erithacus

Ara chloroptera 9995 01 51 18Cacatua galerita 00 10076 223 36Cacatua goffiniana 03 1232 8954 416Psittacus erithacus 30 04 111 10190InceptionResNetV2 (ImageNetrandom) Prediction

Actual Ara chloroptera Cacatua galerita Cacatua goffiniana Psittacus erithacusAra chloroptera 10385 07 12 111Cacatua galerita 00 9874 529 22Cacatua goffiniana 04 819 9572 210Psittacus erithacus 85 04 14 9692

InceptionV3 (ImageNetrandom) PredictionActual Ara chloroptera Cacatua galerita Cacatua goffiniana Psittacus erithacus



10092 in the first row means that the number of correctpredictions is 100 and 92 for the models initialized by theImageNet weights and random weights respectively )enumber of test images for each species is 105 as mentionedearlier Hence ResNet50 with the ImageNet weightscorrectly classified 100 out of 105 samples )e confusionmatrix is an important measure of the true performance ofeach model Because the models were evaluated on

previously unseen data we can verify whether they canrecognize general features of the species )e results showthat the models can classify the images in the training andvalidation sets with more than 90 of accuracy (learningcurves of training and validation) but it does not seem toapply to the confusion matrix of random-number-ini-tialized models (right-side values of the confusion matrix))erefore some pieces of information for validation were

094

A chloroptera C galerita C goffinianan P erithacus

09

ResNet50

059

084089

081

056

094

ResNet50 imageResNet50 random

Figure 12 F1-score of RestNet50 for four different endangered parrot images

087


095

InceptionResNetV2

076

093 097

078068

098

InceptionResNetV2 imageInceptionResNetV2 random

Figure 13 F1-score of InceptionResNetV2 for four different endangered parrot images

095


094

NASNet Mobile

077

0931

083

065

1

NASNet mobile imageNASNet mobile random

Figure 14 F1-score of NASNetMobile for four different endangered parrot images


leaked out during the training hence the models mem-orized the features of validation instead of general featuresof species According to our results the models withImageNet weights classify the images better than the othermethods even though the images are completely new Forexample the results are 9866 and 8855 for ResNet50 inTable 2 )is finding stands not only for ResNet50 but alsofor the other models )e number of correct predictionsfor each model is 100 out of 105 98 out of 105 and 94 outof 105 for Cacatua galerita 88 out of 105 89 out of 105 95out of 105 and 97 out of 105 for Cacatua goffinianarespectively

Figures 12ndash15 show F1-scores of the models F1-score isa way to quantify the results of the confusion matrix F1-score is calculated using precision and recall by

F1 2lowastPrecisionlowastRecallPrecision + Recall

(2)

Precision reflects how many predicted items are correctRecall reflects how many correct items are predicted Pre-cision can be calculated by dividing the number of truepositives by the number of positive predictions For instanceResNet50 with ImageNet classified 105 images as Arachloroptera in the test set )e number of true positives is100 )erefore the precision of ResNet50 is 100 out of 105Recall can be calculated by dividing the number of truepositives by the number of true cases For ResNet50 the totalnumber of true cases is 105 hence the recall of the model is100 out of 105 We can calculate the F1-score by substitutionof the results

2lowast(100105)lowast(100105)

(100105) +(100105)asymp 095 (3)

Figure 12 shows the F1-score of Ara chloroptera )e F1-score is more effective than simple accuracy when wemeasure the modelrsquos performance because it considers thedata distribution (unlike the accuracy) Let us suppose thatwe have 90 images with the first label and ten images with thesecond label We can obtain 90 of accuracy if we classify allimages as ldquothe first label F1-score avoids this problemOverall we conclude that the ImageNet-based models aresuperior to the random-number-initialized models forquasi-species of parrots

42 Mobile Application )e graphical user interface of thereal-time mobile application developed in this study isshown in Figure 9 NASNetMobile model with ImageNetweights was converted into a FlatBuffer file (tflite) andadded to the application Subsequently we used AndroidStudio to edit the code and add visual elements First wechecked that Android Studio SDK version and depen-dences were compatible with TensorFlow Lite After themodel in a FlatBuffer file was located in a project we built itand then an APK was created Finally the application wasinstalled on a device

)e parrot images were captured by the mobile devicersquoscamera Next the trained model classified the image Finallythe application showed the result of the model We cancheck the result at the bottom of the screen as seen inFigure 9 )e first image of Figure 9 shows a preview of aparrot image a text line presents that this parrot is ldquoArachloropterardquo as one hundred percent ldquo345msrdquo is seen at thelowest part of the image it means that it took 345ms toclassify this image)e average turnaround time was 460msthe minimum time was 229ms and the maximum time was671ms for 50 iterations According to our findings theapplication processed jobs under 1 second

5 Discussion

In this paper we proposed classifiers for endangered parrotspecies )e models extract the features of the parrot ap-pearances at the convolutional layer which has been pre-trained on a large amount of data and then we classify theimages at the last layer Our proposed models require arelatively short time to conduct their job )ey are moreaccurate than the models trained from scratch especially forthe species that have a similar color )is is because thepretrained models can already extract the low-level featuresof a new image Another advantage of the models trained bytransfer learning is that the model does not need to draw abounding box to train the last layer )is approach willgreatly reduce the inconvenience for humans by eliminatingmanual processes We expect that the accuracy will be in-creased if fine tuning is applied Finally Tfkeras-basedmodel can be easily deployed on an Android mobile deviceusing the FlatBuffer file converter provided by TensorFlow

09


095

InceptionV3

067

09 095

07058

099

InceptionV3 imageInceptionV3 random

Figure 15 F1-score of InceptionV3 for four different endangered parrot images


Lite To clarify the key points of this study we suggest thefollowing highlights

(i) CNN models with transfer learning can be trainedwithout any special difficulty

(ii) )e designed advanced CNNmodels do not requireany manual preprocessing (such as labeling ordrawing bounding boxes on the images)

(iii) )e CNN models can be easily converted into a filefor deploying in a mobile application using Ten-sorFlow Lite framework

(iv) )e mobile application can classify endangeredquasi-species of parrots having a high color simi-larity in real time

6 Conclusions and Future Work

In our proposed system the mobile application classifies theimage acquired from the device camera in real time To sumup our system works as follows We used two methods tocreate a high-quality model with a small amount of originaldata First we used data augmentation to increase theamount of data by manipulating the original data Secondwe used transfer learning to extract the characteristics of theimage smoothly Specifically we used the convolutionallayers pretrained on a large amount of data Next we usedthe FlatBuffer file converter provided by TensorFlow Lite todeploy this model on a mobile device For quasi-species ofparrots the accuracy of the classification models withtransfer learning is approximately 20 higher than that ofthe models trained from scratch

Based on this study we also expect that further studieson advanced topics could be explored as follows First theresults can be improved when a fine-tuning process is addedas mentioned in Section 5 Second in addition to theclassification of the four species of parrots in this study it ispossible to carry out accurate classifications for parrots onmore than ten species

Data Availability

)e image data used to support the findings of this study areavailable from the corresponding author upon request Butonly some sample data are available because this study isunder Ministry of Environment Republic of Korea

Conflicts of Interest

)e authors declare that they have no conflicts of interest

Acknowledgments

)is research was supported by the Ministry of Environ-ment Republic of Korea (2018000210004)

References

[1] Y Xue S Chen J Qin Y Liu B Huang and H ChenldquoApplication of deep learning in automated analysis of

molecular images in cancer a surveyrdquo Contrast Media ampMolecular Imaging vol 2017 Article ID 9512370 10 pages2017

[2] Z Xie and C Ji ldquoSingle and multiwavelength detection ofcoronal dimming and coronal wave using faster R-CNNrdquoAdvances in Astronomy vol 2019 Article ID 7821025 9 pages2019

[3] M S Norouzzadeh A Nguyen M Kosmala et al ldquoAuto-matically identifying counting and describing wild animalsin camera-trap images with deep learningrdquo Proceedings of theNational Academy of Sciences vol 115 no 25 pp E5716ndashE5725 2018

[4] B Mridula and P Bonde ldquoHarnessing the power of deeplearning to save animalsrdquo International Journal of ComputerApplications vol 179 no 2 2017

[5] A Sadaula Y Raj Pandeya Y Shah D K Pant andR Kadariya Wildlife Population Monitoring Study AmongEndangered Animals at Protected Areas in Nepal IntechOpenLondon UK 2019

[6] Z Huang Z Pan and B Lei ldquoTransfer learning with deepconvolutional neural network for SAR target classificationwith limited labeled datardquo Remote Sensing vol 9 no 9 p 9072017

[7] K He X Zhang S Ren and J Sun ldquoDeep residual learningfor image recognitionrdquo in Proceedings of the 2016 IEEEConference on Computer Vision and Pattern Recognition(CVPR) Las Vegas NV USA June 2016

[8] B Zoph and Q V Le ldquoNeural architecture search with re-inforcement learningrdquo in Proceedings of the InternationalConference on Learning Representations Toulon France April2017

[9] C Szegedy W Liu Y Jia et al ldquoGoing deeper with con-volutionsrdquo in Proceedings of the 2015 IEEE Conference onComputer Vision and Pattern Recognition (CVPR) BostonMA USA June 2015

[10] C Szegedy S Ioffe V Vincent and A Alexander Inception-V4 Inception-ResNet and the Impact of Residual Connectionson Learning AAAI Press San Francisco CA USA 2017

[11] H Nguyen S J Maclagan T D Nguyen et al ldquoAnimalrecognition and identification with deep convolutional neuralnetworks for automated wildlife monitoringrdquo in Proceedingsof the 2017 IEEE International Conference on Data Science andAdvanced Analytics (DSAA) Tokyo Japan October 2017

[12] P Zhuang L Xing Y Liu S Guo and Y Qiao ldquoMarineanimal detection and recognition with advanced deeplearning modelsrdquo in Proceedings of the CLEF 2017 DublinIreland September 2017

[13] S J Pan and Q Yang ldquoA survey on transfer learningrdquo IEEETransactions on Knowledge and Data Engineering vol 22no 10 pp 1345ndash1359 2010

[14] L Torrey and J Shavlik ldquoTransfer learningrdquo in Handbook ofResearch on Machine Learning Applications and Trends Al-gorithms Methods and Techniques E S OlivasJ D M Guerrero M Martinez-Sober J R Magdalena-Benedito and A J S Lopez Eds pp 242ndash264 IGI GlobalHershey PA USA 2010

[15] R Kumar Sanodiya and J Mathew ldquoA novel unsupervisedglobality-locality preserving projections in transfer learningrdquoImage and Vision Computing vol 90 2019

[16] J Ma J C P Cheng C Lin Y Tan J Zhang and J ZhangldquoImproving air quality prediction accuracy at larger temporalresolutions using deep learning and transfer learning tech-niquesrdquo Atmospheric Environment vol 214 Article ID116885 2019


[17] H Ismail Fawaz G Forestier J Weber L Idoumghar andP-A Muller ldquoTransfer learning for time series classificationrdquoin Proceedings of the IEEE International Conference on BigData pp 1367ndash1376 Seattle WA USA December 2018

[18] M Sabatelli M Kestemont D Walter and P Geurts ldquoDeeptransfer learning for art classification problemsrdquo in Pro-ceedings of the the European Conference on Computer Vision(ECCV) Workshops Munich Germany September 2018

[19] J Alexander ldquoimgaugrdquo 2019 httpsgithubcomalejuimgaug

[20] TensorFlow ldquoTensorFlow lite converterrdquo 2019 httpswwwtensorfloworgliteconvert

[21] X Li R Long J Yan K Jin and J Lee ldquoTANet a tinyplankton classification network for mobile devicesrdquo MobileInformation Systems vol 2019 Article ID 6536925 8 pages2019

[22] Keras ldquo)e Python deep learning LIBRARYrdquo 2019 httpskerasio

[23] D Rong L Xie and Y Ying ldquoComputer vision detection offoreign objects in walnuts using deep learningrdquo Computersand Electronics in Agriculture vol 162 pp 1001ndash1010 2019

[24] A Lin J Wu and X Yang ldquoA data augmentation approach totrain fully convolutional networks for left ventricle segmen-tationrdquo Magnetic Resonance Imaging vol 66 pp 152ndash1642019

[25] D Zhao G Yu P Xu and M Luo ldquoEquivalence betweendropout and data augmentation a mathematical checkrdquoNeural Networks vol 115 pp 82ndash89 2019

[26] M Lin Q Chen and S Yan ldquoNetwork in networkrdquo 2013httpsarxivorgabs13124400

[27] A Krizhevsky I Sutskever and G E Hinton ldquoImageNetclassification with deep convolutional neural networksrdquoCommunications of the ACM vol 60 no 6 pp 84ndash90 2017

[28] D G Cheo E Choi E C Lee and K Dong ldquo)e mobileapplications based on vision-object detections for classifyingof endangered parrot species using the CNN deep modelrdquo inProceedings of the 2018 Americas Conference on MedicalImaging and Clinical Research (AMICR 2018) Panama De-cember 2018

[29] J M Forshaw Parrots of the World Princeton UniversityPress Princeton NJ USA 2010

[30] Keras ldquoApplicationsrdquo 2019 httpskerasioapplications[31] Scikit-Learn ldquosklearnmetricsConfusion_matrixrdquo 2019

httpsscikit-learnorgstablemodulesgeneratedsklearnmetricsconfusion_matrixhtml

[32] Scikit-Learn ldquosklearnmetricsClassification_reportrdquo 2019httpsscikit-learnorgstablemodulesgeneratedsklearnmetricsclassification_reporthtml

[33] Tensorflow ldquoTensorflowtensorflowrdquo 2019 httpsgithubcomtensorflowtensorflowblobmastertensorflowlitepythontflite_convertpy


algorithms are designed to be performed independently andare trained to solve specific tasks Moreover the neuralnetwork models have to be rebuilt once the feature-spacedistribution changes To overcome these disadvantages weadopted the transfer learning method to classify the en-dangered parrot quasi-species in this study Transferlearning is a machine learning technique in which a modeltrained for one task is reused for another related task [6]Among many ways to deploy deep learning models inproduction one of the easiest ways is to deploy it on mobiledevices )e advantages are that mobile devices are popularand easy to use Users can get an answer in a few touchesMoreover deep learning models can receive large amountsof data in real time thanks to the camera of the mobiledevice When deploying a deep learning model on a mobiledevice two aspects should be considered model file size andprocess speed If the size is too large it is impossible todeploy the model on a mobile device If the process is slow itwill cause inconvenience for the users

In this study a real-time mobile application was de-veloped to classify endangered parrot quasi-species using theCNN models based on transfer learning To clarify thepurpose of this study we suggested the followinghypotheses

(i) )e designed CNN-based transfer learning modelscan classify endangered parrot quasi-species withhigh color similarity

(ii) )e developed application can embed the designedCNN-based training

)e rest of this paper is organized as follows Section 2presents related work on transfer learning with CNNmodels Section 3 explains our real-time mobile applicationSection 4 presents the experimental results of the classifi-cation of endangered parrot species for the designed mobileapplication Section 5 discusses the contribution of thedesigned mobile application and the classification resultsFinally Section 5 concludes this study

2 Related Work

21 CNN Models and Image Classification for AnimalsMany well-known CNN model architectures exist for var-ious applications In 2016 Microsoft Research presented asolution for the problem of building deep models withshortcut connections [7] Zoph and Le also presented amethod to automatically find a new optimized model ar-chitecture based on policy gradients called neural archi-tecture search at ICLR 2017 [8] Szegedy et al have won theILSVRC 2014 with a top-5 test error of 67 with a modelbuilt on the concept of ldquonetwork in networkrdquo )e idea ofthis model is to reduce the computing cost using dimen-sionality reduction constructing the network by stackingconvolution operations using filters of various sizes andthen combining them later [9] Another model created bySzegedy et al is Inception-ResNet which combines theresidual connections presented by Microsoft Research [10]

Many relevant studies exist to preserve the diversity ofspecies To acquire the data necessary for these studies

unmanned cameraswere installed to acquire images of thecreatures However human resources are wasted on pro-cessing the obtained data Because humanrsquos judgment issubjective the accuracy is inevitably deteriorated )ereforeit is essential to create a system that automatically processesand classifies animal images Norouzzadeh et al in theldquoSnapshot Serengeti Projectrdquo said that processing of in-formation from animal image datasets by human beings istime-consuming hence much data remains unprocessed)ey presented a system in which a machine can determinewhere the images belong to and check the number of entitiesand their behaviors in images [3] Nguyen et al also created aCNNmodel to classify three of themost commonly observedanimal species in Victoria Australia and showed the realtest results [11] Zhuang et al introduced a deep learningmodel that automatically annotates marine biological imagedata without relying on human experts )ey experimentedwith their model with data from SeaCLEF2017 [12] In thisstudy we also propose a system to classify image data ac-quired in real time using the camera of a mobile device

22 Transfer Learning Transfer learning is a state-of-the-arttechnique in deep learning research Before the advent ofthis technique people had to create and train a model fromscratch It was difficult to invent a model with remarkableperformance on a specific task because of the lack ofcomputing infrastructure Moreover it was impossible tocollect enough meaningful data required to train a modelalthough many researchers attempted to gather themHowever various transfer learning methods have beenproposed for transferring knowledge in the context offeatures instant weights parameters or relationship in-formation between data samples in a domain [13ndash16]

Figure 1 shows four steps of creating a complete modelusing transfer learning First we build an architecture of themodel and train it on a large representative dataset Secondwe delete the final layer (known as ldquoloss outputrdquo) )ird wereplace it with another layer whose job is to finish the specifictask Fourth we train a new model with a relatively smalldataset suitable for the purpose Transfer learning is literally totransfer the job of extracting features from data to the pre-trained model For example a model pretrained on theImageNet dataset can detect low-level features on a bird image(such as curves outlines and lines) because these low-levelfeatures are almost the same in other animal images )eremaining task is to tune the high-level layers of the featureextractor and the final layer that classifies the bird (the processis called fine tuning) Some studies have already appliedtransfer learning [17 18] Transfer learning is expected tocompensate for the lack of data time and computing

3 Implementation of a Real-Time MobileApplication to Classify EndangeredParrot Quasi-Species

31 SystemDesign and Image Classification inMobileDevices)e system is divided into four parts as shown in Figure 2First we preprocess the data to prepare it for deep learning










Conv1Conv1

Conv2Conv2



SoftmaxSoftmax


Output Output

Transfer

Features




GAPi 1

mxn1113944

m

a

1113944

n

b

x (1)







Detection

Augmentation

Take a picture


Normalization



Display the results






Android app

JAVA API

C++ API








4 Results


Augmentation

Augmentation

Augmentation

Augmentation











1 3 2

05 0

2 50


2


GAPi = 1mxn (summ

a sumnb xa b)

GAPi = 1mxn (summ

a sumnb xa b)

PPG




GAP

Denselayer



Parrotimages


3


Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02



Ara chloroptera 01




Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers


Parrotimages

Parrotimages



Picture


Appearance





underparts







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References













































Conv1Conv1

Conv2Conv2



SoftmaxSoftmax


Output Output

Transfer

Features




GAPi 1

mxn1113944

m

a

1113944

n

b

x (1)







Detection

Augmentation

Take a picture


Normalization



Display the results






Android app

JAVA API

C++ API








4 Results


Augmentation

Augmentation

Augmentation

Augmentation











1 3 2

05 0

2 50


2


GAPi = 1mxn (summ

a sumnb xa b)

GAPi = 1mxn (summ

a sumnb xa b)

PPG




GAP

Denselayer



Parrotimages


3


Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02



Ara chloroptera 01




Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers


Parrotimages

Parrotimages



Picture


Appearance





underparts







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References





































GAPi 1

mxn1113944

m

a

1113944

n

b

x (1)







Detection

Augmentation

Take a picture


Normalization



Display the results






Android app

JAVA API

C++ API








4 Results


Augmentation

Augmentation

Augmentation

Augmentation











1 3 2

05 0

2 50


2


GAPi = 1mxn (summ

a sumnb xa b)

GAPi = 1mxn (summ

a sumnb xa b)

PPG




GAP

Denselayer



Parrotimages


3


Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02



Ara chloroptera 01




Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers


Parrotimages

Parrotimages



Picture


Appearance





underparts







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References








































4 Results


Augmentation

Augmentation

Augmentation

Augmentation











1 3 2

05 0

2 50


2


GAPi = 1mxn (summ

a sumnb xa b)

GAPi = 1mxn (summ

a sumnb xa b)

PPG




GAP

Denselayer



Parrotimages


3


Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02



Ara chloroptera 01




Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers


Parrotimages

Parrotimages



Picture


Appearance





underparts







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References













































1 3 2

05 0

2 50


2


GAPi = 1mxn (summ

a sumnb xa b)

GAPi = 1mxn (summ

a sumnb xa b)

PPG




GAP

Denselayer



Parrotimages


3


Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02



Ara chloroptera 01




Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers


Parrotimages

Parrotimages



Picture


Appearance





underparts







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References





































Parrotimages


3


Dense


Dense


Dense


Dense

Ara chloroptera 098

Cacatua galerita 02



Ara chloroptera 01




Ara chloroptera 00

Cacatua galerita 03



Ara chloroptera 00

Cacatua galerita 00



1 times 1 64

3 times 3 64

1 times 1 256

relu

relu

relu

Hj-1

Hj

sep3 times 3

identity

avg3 times 3

identity

add add

concat

hellip

Conv Conv Conv Ap

Conv Conv

Conv

Conv

Conv ConvConv

Conv

concat

dropout

Conv

Conv

hellip

1 times 1Con

1 times 1Con

1 times 1Conv

3 times 3maxpool

3 times 3Conv

5 times 5Conv

1 times 1Conv

Concat

Output layer

Task layer

Shared layers


Parrotimages

Parrotimages



Picture


Appearance





underparts







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References






































1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30




Augmenteddata

Originaldata

Dataaugmentation

Imageclassifier

Testdata

Result







1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References








































1

09

08

07

06

05

Epoch1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30















094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References







































094


09

ResNet50

059

084089

081

056

094



087


095

InceptionResNetV2

076

093 097

078068

098



095


094

NASNet Mobile

077

0931

083

065

1







(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References








































(2)



(100105) +(100105)asymp 095 (3)




5 Discussion


09


095

InceptionV3

067

09 095

07058

099












Data Availability




Acknowledgments


References













































Data Availability




Acknowledgments


References





































new thereal-timemobileapplicationforclassifyingofendangered...

Documents