Copyright © 2010 SAS Institute Inc. All rights reserved.
Decision Trees Using SAS
Sylvain Tremblay
SAS Canada – Education
SAS Halifax Regional User GroupApril 29, 2011
2
Copyright © 2010, SAS Institute Inc. All rights reserved.
They come in all shapes and forms
3
Copyright © 2010, SAS Institute Inc. All rights reserved.
They come in all shapes and forms
4
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
What is a decision tree?
Decision trees using SAS Enterprise Miner
Decision trees using JMP
Conclusion / Questions
5
Copyright © 2010, SAS Institute Inc. All rights reserved.
A Decision Tree is
A predictive model
A representation of the relationship between a target (dependant variable) and a set of inputs (independant variables)
A supervised learning method
A recursive partitionning algorithm
Also know by the name of algorithms that were commercialized:
CART (Classification And Regression Tree)
CHAID (CHi-squared Automatic Interaction Detector)
6
Copyright © 2010, SAS Institute Inc. All rights reserved.
Simple Prediction Illustration
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
Predict P(Y=1| X1, X2)
Training Data
...
7
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Tree Split Search
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
0.63
x2<0.63 ≥0.63
Create a partition rule from the best partition across all inputs.
...
8
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Tree Prediction Rules
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
60%55%
70%
x1
<0.52 ≥0.52 <0.51 ≥0.51x1
x2
<0.63 ≥0.63
root node
interior node
leaf node
...
1
2
3
4
1 2
3 4
9
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Tree Prediction Rules
40%
60%55%
x1
<0.52 ≥0.52 ≥0.51
<0.63
0.0 0.50.1 0.2 0.3 0.4 0.6 0.7 0.8 0.9 1.0
0.0
0.5
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1.0
x1
x2
Decision = Estimate = 0.70
70%
<0.51x1
x2
≥0.63
Predict:Predict:
...
10
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
What is a decision tree?
Decision trees using SAS Enterprise Miner
Decision trees using JMP
Conclusion / Questions
11
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Miner
Trees can be created manually or automatically
12
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Minermanually
13
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Minermanually
14
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Minermanually
15
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees - Assessment
Pruning
16
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees - Pruningmanually
17
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Minerautomatically
EM Tree Parameters
18
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
What is a decision tree?
Decision trees using SAS Enterprise Miner
Decision trees using JMP
Conclusion / Questions
19
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS JMP
20
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
What is a decision tree?
Decision trees using SAS Enterprise Miner
Decision trees using JMP
Conclusion / Questions
