constructing associative classifiers from decision tables

5/14-5/17, Toronto RSFDGrC - 2007 1

Constructing Associative Constructing Associative Classifiers from Decision TablesClassifiers from Decision Tables

Jianchao HanCalifornia State University Dominguez Hills, USA

T. Y. LinSan Jose State University, USA

Jiye LiUniversity of Waterloo, Canada

Nick CerconeYork University, Canada


AgendaAgenda

• Introduction

• Related Work

• Our Approach

• Algorithm Description

• An Example Demonstration

• Conclusion


IntroductionIntroduction

• Associative classifier– A set of classification rules– Classification rules as a special form of

association rules– Classifier formed by finding constrained

association rules

• Rough set theory used to reduce data set


Related WorkRelated Work

• Rough set theory– Attribute reduct

• Association rules– Rule template– Constrained rules

• Significant rules– Rule importance measurement

• Michalski’s coverage method


Our ApproachOur Approach• Combining three strategies

– Association rules – Rough set theory to find attribute reducts– Coverage method to form a classifier

• Four steps– Finding attribute reducts– Finding constrained association rules– Measuring association rules – Selecting important rules to cover instances


Generate All Attribute ReductsGenerate All Attribute Reducts

• Use existing reduct finding algorithms such as Genetic Reduct generation algorithm in ROSETTA


Find Classification RulesFind Classification Rules

• For each attribute reduct– Use adapted Apriori algorithm to find

constrained association rules, where right side of rules is constrained to a class label

• Carefully determine the thresholds of support and confident for association rules


Measure Importance of RulesMeasure Importance of Rules

• Rule importance definition

• Properties of the rule importance– 0 < Importance(Rule) ≤ 1

– If Rule only contains core attributes, its importance is 1.

reducts ofnumber total

Rule thegenerates that reducts ofnumber (Rule)Importance


Rule PrecedenceRule Precedence• Given two rules R1 and R2 generated, R1 precedes

R2 (R1 has a higher precedence than R2), denoted Precedence(R1)>Precedence(R1), if – Importance(R1) > Importance(R2); or– Importance(R1) = Importance(R2), and

Confidence(R1) > Confidence(R2); or– Importance(R1) = Importance(R2), and

Confidence(R1) = Confidence(R2), and Support(R1) > Support (R2).

• Otherwise R1 and R2 are considered having the same precedence and denoted Precedence(R1)=Precedence(R2).

5/14-5/17, Toronto RSFDGrC - 2007 10

Property of Rule PrecedenceProperty of Rule Precedence

• The precedence relationship is a total order relation

• Thus all rules can be sorted based on their precedence

5/14-5/17, Toronto RSFDGrC - 2007 11

Find Associative ClassifierFind Associative Classifier

• Sort all rules in terms of their precedence consisting of importance, confidence and support

• Select next rule in the sorted sequence• If this rule covers some rows

– Delete all rows covered by this rule– Put this rule in the classifier

• Repeat until all rules are exhausted

5/14-5/17, Toronto RSFDGrC - 2007 12

An ExampleAn ExampleMakemodel cyl door displace compress power trans weight mileage

USA 6 2 Medium High High Auto Medium Medium

USA 6 4 Medium Medium Medium Manual Medium Medium

USA 4 2 Small High Medium Auto Medium Medium

USA 4 2 Medium Medium Medium Manual Medium Medium

USA 4 2 Medium Medium High Manual Medium Medium

USA 6 4 Medium Medium High Auto Medium Medium

USA 4 2 Medium Medium High Auto Medium Medium

USA 4 2 Medium High High Manual Light High

Japan 4 2 Small High Low Manual Light High

Japan 4 2 Medium Medium Medium Manual Medium High

Japan 4 2 Small High High Manual Medium High

Japan 4 2 Small Medium Low Manual Medium High

Japan 4 2 Small High Medium Manual Medium High

USA 4 2 Small High Medium Manual Medium High

5/14-5/17, Toronto RSFDGrC - 2007 13

Find All Attribute ReductsFind All Attribute Reducts

• Genetic reducer in ROSETTA • Four attribute reducts

Reduct # Reduct Attributes

1 Make, compress, power, trans

2 make, cyl, compress, trans

3 make, displace, compress, trans

4 make, cyl, door, displace, trans, weight

5/14-5/17, Toronto RSFDGrC - 2007 14

Constrained Associated RulesConstrained Associated Rules• support threshold = 1% • confidence threshold = 100% • applying the adapted Apriori algorithm

# Rules Rule Preced

12345678910111213

(make, Japan) → (Mileage, High)(Trans, Auto) → (Mileage, Medium)(Compress, High), (Trans, Manual) → (Mileage, High)(make, USA), (Compress, Medium) → (Mileage, Medium)(Displace, Small), (Trans, Manual) → (Mileage, High)(Cyl, 6) → (Mileage, Medium)(USA, Car), (Displace, Medium), (Weight, Medium) → (Mileage, Medium) (make, USA), (Power, High) → (Mileage, Medium)(Compress, Medium), (Power, High) → (Mileage, Medium) (Power, Low) → (Mileage, High) (Door, 4) → (Mileage, Medium)(Weight, Light) → (Mileage, High)(Displace, Small), (Compress, Medium) → (Mileage, High)

4/4, 5 4/4, 43/4 53/4, 52/4, 52/4, 31/4, 61/4, 41/4, 31/4, 21/4, 21/4, 21/4, 1

5/14-5/17, Toronto RSFDGrC - 2007 15

Construct ClassifierConstruct Classifier

• Covering method• Rule 1: Covers 5 rows 9 through 13• Rule 2: Covers 2 rows 8 and 14• Rule 3: Covers 4 rows 1, 3, 6, and 7 • Rule 4: Covers 3 rows 2, 4 and 5 • Since all rows in the original decision table have

been covered by Rules 1 through 4, the final associative classifier contains only these four class association rules

5/14-5/17, Toronto RSFDGrC - 2007 16

ConclusionConclusion

• Introduce an approach to constructing associative classifiers based on– Rough set theory to find attribute reducts– Association rule mining algorithm– Covering method to build classifiers

• Present the rule importance and precedence measurement used in the proposed approach

• Demonstrate an example

constructing associative classifiers from decision tables

Documents