anomaly detection
Post on 21-Nov-2014
460 Views
Preview:
DESCRIPTION
TRANSCRIPT
Anomaly Detection
- SALIL NAVGIRE
Introduction• problem of finding patterns in data that
do not conform to expected behavior
• covers diverse disciplines from statistics, machine learning, data mining, information theory, spectral theory
Applications• Intrusion detection- detection of malicious activity• Host based – OS call traces
• Network based – packet level traces
• Fraud detection - detection of criminal activities in commercial organizations• Credit card fraud detection
• Insurance Claim Fraud Detection
• Insider trading detection
• Industrial damage detection
• Anomaly detection in data
• Anomaly detection in sensor networks
Challenges• Defining normal region
• Sometimes malicious agent adapt themselves to appear as normal observation
• Different techniques for different application domain
• Availability of labeled data for training
• Sometimes noise is similar to anomaly and difficult to distinguish
Different aspects of detection techniques• Nature of input data
• Types of Anomaly• Point Anomalies
• Contextual Anomalies
• Collective Anomalies
• Data Labels• Supervised anomaly detection
• Semi-Supervised anomaly detection
• Unsupervised anomaly detection
• Output • Scores
• Labels
Anomaly detection
techniques
Classification
Nearest Neighbor Clustering Spectral Information
theoretic Statistical Time Series
Anomaly Detection Techniques
• Classification• Neural network based
• Bayesian Network based
• Support Vector Machine based
• Rule based
• Nearest Neighbor• KNN
• Relative density
• Clustering• K means
• SOM
• Statistical• Parametric• Gaussian model based• Regression model based• Mixture of parametric distributions based
• Non-parametric• Histogram based• Kernel function based
• Spectral• Dimensionality reduction
top related