covering algorithms. trees vs. rules from trees to rules. easy: converting a tree into a set of...
Post on 21-Dec-2015
216 views
TRANSCRIPT
![Page 1: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/1.jpg)
Covering Algorithms
![Page 2: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/2.jpg)
Trees vs. rulesFrom trees to rules.• Easy: converting a tree into a set of rules
– One rule for each leaf:– Antecedent contains a condition for every node on the path from
the root to the leaf– Consequent is the class assigned by the leaf
From rules to trees• More difficult: transforming a rule set into a tree
– Tree cannot easily express disjunction between rules• Example:
If a and b then xIf c and d then x
– Corresponding tree contains identical subtrees (“replicated subtree problem”)
![Page 3: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/3.jpg)
A tree for a simple disjunction
![Page 4: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/4.jpg)
Covering algorithms• Strategy for generating a rule set directly:
– for each class in turn find a rule set that covers all instances in it (excluding instances not in the class)
• This approach is called a covering approach because at each stage a rule is identified that covers some of the instances
![Page 5: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/5.jpg)
Example: generating a rule
• Possible rule set for class “b”:• More rules could be added for “perfect” rule set
If x 1.2 then class = b
If x > 1.2 and y 2.6 then class = b
y
x
a
b b
b
b
b
bb
b
b b bb
bb
aa
aa
a y
a
b b
b
b
b
bb
b
b b bb
bb
a a
aa
a
x1·2
2·6
y
a
b b
b
b
b
bb
b
b b bb
bb
a a
aa
a
x1·2
![Page 6: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/6.jpg)
A simple covering algorithm
• Generates a rule by adding tests that maximize rule’s accuracy
• Similar to situation in decision trees: problem of selecting an attribute to split on.– But: decision tree inducer maximizes
overall purity
• Each new test reduces rule’s coverage.
space of examples
rule so far
rule after adding new term
![Page 7: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/7.jpg)
Selecting a test• Goal: maximizing accuracy
– t: total number of instances covered by rule
– p: positive examples of the class covered by rule
– t-p: number of errors made by rule
Select test that maximizes the ratio p/t
• We are finished when p/t = 1 or the set of instances can’t be split any further
![Page 8: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/8.jpg)
Example: contact lenses data
![Page 9: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/9.jpg)
Example: contact lenses data
The numbers on the right show the fraction of “correct” instances in the set singled out by that choice.In this case, correct means that their recommendation is “hard.”
![Page 10: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/10.jpg)
Modified rule and resulting data
The rule isn’t very accurate, getting only 4 out of 12 that it covers. So, it needs further refinement.
![Page 11: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/11.jpg)
Further refinement
![Page 12: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/12.jpg)
Modified rule and resulting data
Should we stop here? Perhaps. But let’s say we are going for exact rules, no matter how complex they become.So, let’s refine further.
![Page 13: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/13.jpg)
Further refinement
![Page 14: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/14.jpg)
The result
![Page 15: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/15.jpg)
Pseudo-code for PRISMFor each class C
Initialize E to the instance set
While E contains instances in class C
Create a rule R with an empty left-hand side that predicts class C
Until R is perfect (or there are no more attributes to use) do
For each attribute A not mentioned in R, and each value v,
Consider adding the condition A = v to the left-hand side of R
Select A and v to maximize the accuracy p/t
(break ties by choosing the condition with the largest p)
Add A = v to R
Remove the instances covered by R from E
![Page 16: Covering Algorithms. Trees vs. rules From trees to rules. Easy: converting a tree into a set of rules –One rule for each leaf: –Antecedent contains a](https://reader030.vdocuments.mx/reader030/viewer/2022032521/56649d5c5503460f94a3b154/html5/thumbnails/16.jpg)
Separate and conquer• Methods like PRISM (for dealing with one class) are
separate-and-conquer algorithms:– First, a rule is identified
– Then, all instances covered by the rule are separated out
– Finally, the remaining instances are “conquered”
• Difference to divide-and-conquer methods:
– Subset covered by rule doesn’t need to be explored any further