Anil Kumar Gupta Senior Software Engineer

LinkedIn

Image Classification

3

OutlineWhy Image Classification?Classical Method of Image classificationConvolution in ImagesDeep convolutional neural networksSuccess of DCNN on MNISTImageNet ChallengeSuccess of DCNN on ImageNetWhat DCNN learnsDCNN on limited datasetCaffe IntroductionResult of our datasetReferences and Resources

4

Why Image ClassificationIntroduced Group posts with Images

Solutions:

a) Let all Post with Images go through human review

b) Small fraction of post go to human review. This fraction should include any obscene image posted

5

Classical Method of Image classificationCompute bag of featuresBag of features are computed using different Image processing techniques

SmoothingDerivatives of image Blob detectionHOGSIFT…………

Learn some ML model on these features

6

Convolution in ImagesCompute bag of featuresBag of features are computed using different Image processing techniques

SmoothingDerivatives of image Blob detectionHOGSIFT…………

Learn some ML model on these features

7

Convolution in Images

8

9

Convolutional Neural network

10

Success of DCNN on MNIST (LeNet 1998)

Source:GradientBasedLearningAppliedtodocumentRecognition

11

Source:GradientBasedLearningAppliedtodocumentRecognition

12

Ciresan et. al. net on MNIST (2010)

Used lots of synthetic data to train bigger net

Used Model averaging

The right answer is almost always in the top 2 guesses.

Source: Handwritten Digit Recognition with a Committee of Deep Neural Nets on GPUs

13

ImageNet Challenge (2012)The dataset has 1.2 million high-resolution training images.

The classification task: Get the “correct” class in your top 5 bets. There are 1000 classes.

Some of the best existing computer vision methods were tried on this dataset by leading computer vision groups from Oxford, INRIA, XRCE, …

14

ImageNet Challenge Results (2012)University of Tokyo 26.1% Oxford University Computer Vision Group 26.9%INRIA (French national research institute in CS) + XRCE (Xerox Research Center Europe) 27.0% University of Amsterdam 29.5%

University of Toronto (Alex Krizhevsky) 16.4 %

15

ImageNet Challenge Results (2012)Why so many years for another successful result by DCNN ?

16

Success of DCNN on ImageNet (AlexNet 2012)

Source:ImageNetClassificationwithDeepConvolutionalNeuralNetworks

17

18

AlexNet (2012)Use of GPUs to train the network

Use of Data Augmentation

Use of ReLU instead of Sigmoid

Use of DropOut Layers

19

ImageNet Results 2010 - 2015

20

What CNN learns

Source:Richfeaturehierarchiesforaccurateobjectdetectionandsemanticsegmentation

21

DCNN on limited datasetCNN learns hierarchical features

Goes from generic features to task specific features

Can use generic feature learned on different task for your specific task

How transferable are features in deep neural networks?

Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks

Monza: Image Classification of Vehicle Make and Model Using Convolutional Neural Networks and Transfer Learning

Rich feature hierarchies for accurate object detection and semantic segmentation

22

CaffeA DNN library

Let you define DNN architecture in plain text file

Let you define parameters in plain text file

Can train on GPU and CPU

Can train on GPU and deploy in CPU

23

LayerSetup: run once for initialization.

Forward: make output given input.

Backward: make gradient of output- w.r.t. bottom- w.r.t. parameters (if needed)

Reshape: set dimensions.

24

LossClassification:

SoftmaxWithLossHingeLoss

Linear Regression:EuclideanLoss

Multiclassification:SigmoidCrossEntropyLoss

Others…

loss (LOSS_TYPE)

25

BlobBlobs are 4-D arrays for storing and communicating information.hold data, derivatives, and parameterslazily allocate memoryshuttle between CPU and GPU

top blob

bottom blob

26

Our results from Transfer learningTraining image: 5872

Good: 3075 Bad: 2797

Validation images: 1679 Good: 880 Bad: 799

Test images: 831 Good: 436 Bad: 395

Precision ~ 95%Recall ~ 95%

27

In ProductionBad images is ~ 1 in 1000

5 - 7 % send for human review

Have not received any bad image post via user flagging

Latency ~ 600 ms per image

5-6 QPS on single machine

28

Reference & ResourcesGradientBasedLearningAppliedtodocumentRecognitionHandwritten Digit Recognition with a Committee of Deep Neural Nets on GPUsImageNetClassificationwithDeepConvolutionalNeuralNetworksRichfeaturehierarchiesforaccurateobjectdetectionandsemanticsegmentationHow transferable are features in deep neural networks?Learning and Transferring Mid-Level Image Representations using Convolutional Neural NetworksMonza: Image Classification of Vehicle Make and Model Using Convolutional Neural Networks and Transfer LearningDropout: A Simple Way to Prevent Neural Networks from OverfittingMathematicalIntuitionforPerformanceofRectifiedLinearUnitinDeepNeuralNetworksConvolutional Neural Network for Visual RecognitionNeural Networks for Machine LearningPractical Recommendation for Train DNN

©2014 LinkedIn Corporation. All Rights Reserved.©2014 LinkedIn Corporation. All Rights Reserved.