telecom network fault prediction
DESCRIPTION
Telecom Network Fault Prediction. H. K. Yuen Department of Management Sciences City University of Hong Kong. Outline. Problem Formulation Variable Selection Model Development Model Implementation. Problem Formulation. Overview - PowerPoint PPT PresentationTRANSCRIPT
1
Telecom Network Fault Prediction
H. K. Yuen
Department of Management Sciences
City University of Hong Kong
2
Outline
• Problem Formulation
• Variable Selection
• Model Development
• Model Implementation
3
Problem Formulation
• Overview– Messages about network performances are
generated from transmission stations– Messages are examined manually– Messages are classified as urgent fault or non-
urgent fault– To build a model to predict whether a received
signals an urgent fault or not
4
Problem Formulation
• The Data– 5,924 past messages were collected– Each message contains 1,082 variables – Each message was examine manually– The decision "Urgent" or "Non-Urgent" was set
as the target variable• Urgent case = "True" Non-Urgent case = "Null"
5
Problem Formulation
• Distribution of the Target Variable
Null True
6
Problem Formulation
• Selection of Cases– Use the Sampling node of Enterprise Miner
(EM) to select a sample
7
Variable Selection
• Using all of the variables in the model is not practical
• Impractical to examine the associations between the target variable and the other input variables manually
• The Tree node and the Variable Selection node of Enterprise Miner were employed
8
Variable Selection• Process flow
9
Variable Selection
• Some results from Tree1
• A total of 23 variables are selected as input
10
Model Development• Data are partitioned into three parts
– Training (50%)– Validation (25%)– Testing (25%)
• Two possible model selection criteria: – The one that most accurately predicts the
response (either "True" or "Null")– The one that generates the highest expected profit
11
Model Development• Modified Profit Vector
• Neural Network models with different setting were developed
• Model Output: Prob(Target variable="True")
12
Model Development
• Process flow
• Model Manager
13
Model Development
• How to choose a model with the most predictive power?– Sensitivity: # of predicted "True" / # actual "True"– Specificity: # of predicted "Null" / # actual "Null" – Cutoff point: Observations with predicted
probability of the target event greater than a cutoff point are classified as "True"
14
Model Development
• Receiver Operating Characteristic Chart (ROC)
Sen
siti
vity
1-Specificity
Cutoff
Higher
Lower
All "True"
All "Null"
Winner
15
Model Development• Correct Classification Chart
– Displays the prediction accuracy for each actual target level across a range of cutoff values
Cutoff
16
Implementing the Model
An incoming signal
with predicted Prob(target variable = "True) = p
Class 1
p 0.5
Send technician
Class 2
else
Examine the signal manually
Class 3
p 0.15
Ignore the signal
17
Implementing the Model• Results of classification
• Benefits:• Saving in manpower• Faster response time to problems
Class 1 Class 2 Class 3
True 50.38% 36.84% 12.78%Null 11.31% 25.68% 63.01%
Overall 12.19% 25.93% 61.88%
Actual