MSBA 6420 Rapid WinnersKaushik Nuvvula, Pankaj Singhal, Wenqiuli Zhang, John Tong, Rohith D

Rapid Approaches

Agenda

Approach

Different techniques used

Techniques that worked

Techniques that did not work

Best Model

Future Scope and Learnings

Approach

Neural Network

Voting (SVM, Neural)

Boosting (Neural)

Bagging (Neural)

Sampling, NormalizationData Pre-processing, Weights, Bagging, Voting, SamplingData Pre-processing, Sampling, Generate AttributesSelect by weights, Data Pre-processing, SamplingData Pre-processing, Sampling

Top 5 Models

Model Techniques Used

Stacking (k-NN, Neural)

F-measure and cost: Top 5 Models

Techniques that worked

Data Processing

Data Preprocessing

Generate Attributes

Attribute Selection Techniques

Attributes Selection

Optimize Parameters

Techniques

PCA, SMOTE

Voting

NormalizationBagging, Boosting,

Stacking

Filter Examples Sampling

6

SMOTE: Resampling Approach

• SMOTE -Synthetic Minority Oversampling combines Informed Oversampling of the minority class with Random Under-sampling of the majority class.

• For each minority Sample– Find its k-nearest minority neighbors– Randomly select j of these neighbors– Randomly generate synthetic samples along the lines joining the minority sample and its j

selected neighbors

*SMOTE currently yields best results as far as re-sampling and modifying probabilistic estimate techniques (Chawla, 2003).

Deep Dive: SMOTE Sampling

: Minority sample

: Synthetic sample

What happens if there is a nearby majority sample?

: Majority sample

Techniques that did not work

• Meta-cost• Forward Selection• Logistic Regression

Best Model

Neural Network

Class 0: Above 0.1

Class 0: Between 0.03 – 0.1

F- Measure and Misclassification Cost Improvement

Scope - ImprovementsFi

lter E

xam

ples

Metric Change Improvement

Average Friend Age

17 to 31 Positive

Tenure > 4 Positive

Songs Listened

> 1 Negative

Age > 8 and <70 Negative

Key Learnings: Warnings

• Remove Oversampling - Bias in the data• Generate Calculated Attributes• complex f-measure• Try to train your models on relatively higher

variability capturing records – Using Filter Examples

Appendix

True 0 True 1

Pred. 0 24259 335

Pred. 1 1302 109

True 0 True 1

Pred. 0 23442 320

Pred. 1 1289 400

F- Measure

Misclassification Cost