deep learning: solving the detection problem · plan 1 introduction 2 propositions building a large...
TRANSCRIPT
![Page 1: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/1.jpg)
Deep Learning: Solving thedetection problem
M2R ISI in Paris-Dauphine University - Master thesis defenseMaster internship at Image & Pervasive Access Lab (IPAL)
Anne MORVAN
14 septembre 2015
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 1 / 20
![Page 2: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/2.jpg)
Plan
1 Introduction
2 PropositionsBuilding a large datasetSampling algorithmTraining a CNN-based classifier
3 ResultsSampling algorithmTraining a CNN-based classifier
4 Conclusion
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 2 / 20
![Page 3: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/3.jpg)
Introduction
Outline
1 Introduction
2 PropositionsBuilding a large datasetSampling algorithmTraining a CNN-based classifier
3 ResultsSampling algorithmTraining a CNN-based classifier
4 Conclusion
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 3 / 20
![Page 4: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/4.jpg)
Introduction
What is the detection problem ?
Pipeline
to know how classifiy an image into the classespedestrian presence or not : CLASSIFICATIONTASK
to know the localization within the image and even thenumber of instances
Issues
DEEP learning : lack of data, overfitting, computation time
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 4 / 20
![Page 5: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/5.jpg)
Propositions
Outline
1 Introduction
2 PropositionsBuilding a large datasetSampling algorithmTraining a CNN-based classifier
3 ResultsSampling algorithmTraining a CNN-based classifier
4 Conclusion
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 5 / 20
![Page 6: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/6.jpg)
Propositions Building a large dataset
Collecting the data (1/2)
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 6 / 20
![Page 7: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/7.jpg)
Propositions Building a large dataset
Collecting the data (2/2)
Daimler, ETH, INRIA, TudBrussels, USA, MSCOCO, PETA, CBCL, CVC
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 7 / 20
![Page 8: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/8.jpg)
Propositions Sampling algorithm
Sampling algorithm
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 8 / 20
![Page 9: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/9.jpg)
Propositions Training a CNN-based classifier
Training a CNN-based classifier (1/2)
Model architecture
5 convolutional layers
2 fully-connected layers
Cost function : Cross entropy loss
E = −1N
∑Nn=1
∑Kk=1 (pnk . log p̂nk + (1− pnk ) . log (1− p̂nk ))
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 9 / 20
![Page 10: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/10.jpg)
Propositions Training a CNN-based classifier
Training a CNN-based classifier (2/2)
Pre-processing and data augmentation
normalization with mean and std pixel values for each channel, foreach dataset
mirror (proba = 0.5)
shift (max. 10% of height or width)
rotationRad 0.26 0.13 0.07 0.03 0
Prob 0.1 0.1 0.2 0.3 0.3
aspect ratioRatio (1,1) (2,2) (1,2) (2,1)
Prob 0.25 0.25 0.25 0.25
hard negatives or bootstrapping (proba = 0.5 with 2001 samples)
learning rate policy
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 10 / 20
![Page 11: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/11.jpg)
Results
Outline
1 Introduction
2 PropositionsBuilding a large datasetSampling algorithmTraining a CNN-based classifier
3 ResultsSampling algorithmTraining a CNN-based classifier
4 Conclusion
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 11 / 20
![Page 12: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/12.jpg)
Results Sampling algorithm
Sampling algorithm
Parameters for choosing the data
training data : USA with proba. 1
test data : USA with proba. 1
positive crop proba. : 0.5
positive classes : only person
min. dimensions : w = 5, h = 20
no constraints on the distance fromthe bounds or proportions of the wor h
h and w : two dependent normaldistributions
bounding box : 4 rectangles methodwith jaccard index = 0.1
h N (µh, σ2h)
w N (µw , σ2w )
w |h N (µw + σhw
σw(h − µh), σw −
σ2hwσh
)
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 12 / 20
![Page 13: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/13.jpg)
Results Training a CNN-based classifier
Influence of learning rate
lr = 0.01 & lr = 0.02
lr = 0.03 & lr = 0.05
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 13 / 20
![Page 14: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/14.jpg)
Results Training a CNN-based classifier
Role of data augmentation methods (1/2)
nothing mirror
shift aspect ratio
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 14 / 20
![Page 15: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/15.jpg)
Results Training a CNN-based classifier
Role of data augmentation methods (2/2)
rotation hard negatives
learning rate policy everything
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 15 / 20
![Page 16: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/16.jpg)
Results Training a CNN-based classifier
Accuracy and ROC curve (1/2)
With all data augmentation methods + learning rate policy at the 2500-thbatch
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 16 / 20
![Page 17: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/17.jpg)
Results Training a CNN-based classifier
Fresh results
With all data augmentation methods + lr policy at the 2500-th batch
With all data augmentation methods + deformation ( ≈ blurring)
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 17 / 20
![Page 18: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/18.jpg)
Results Training a CNN-based classifier
Accuracy and ROC curve (2/2)
State of the art ”A comparison of differentfeature extraction andclassification methods.Performance of differentclassifiers on(a) PCA coefficients,(b) Haar wavelets, and(c) Local Receptive Field (LRF)features.(d) A performance comparisonof the best classifiersfor each feature type.”
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 18 / 20
![Page 19: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/19.jpg)
Conclusion
Outline
1 Introduction
2 PropositionsBuilding a large datasetSampling algorithmTraining a CNN-based classifier
3 ResultsSampling algorithmTraining a CNN-based classifier
4 Conclusion
Anne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 19 / 20
![Page 20: Deep Learning: Solving the detection problem · Plan 1 Introduction 2 Propositions Building a large dataset Sampling algorithm Training a CNN-based classi er 3 Results Sampling algorithm](https://reader033.vdocuments.mx/reader033/viewer/2022060817/60969c49af9b41147548dd49/html5/thumbnails/20.jpg)
Conclusion
Conclusion & further perspectives
Our work
classification task (sampling algorithm + data augmentation methods+ classifier)
goal : 98% of well-classified images rate
obtained : ≈ 93% for the validation by training with the USA trainingset and validating on the USA test set
Perspectives
merge more and more datasets (KITTY ...)
perform other data augmentation methods from the elastictransformations family (perspective distorsion transformations...)
use synthetic images
use temporal information + motion
define part-based models
improve the quality of the datasetAnne MORVAN M2R ISI in Paris-Dauphine University - Master thesis defense Master internship at Image & Pervasive Access Lab (IPAL)14 septembre 2015 20 / 20