presentation asrul

8/2/2019 Presentation Asrul

1/20

1

Association RulesApriori Algorithm

Machine Learning Overview

Sales Transaction and Association

Rules

Aprori Algorithm

Example


2/20

2

Machine Learning

Common ground of presented methods

Statistical Learning Methods (frequency/similaritybased)

Distinction

Data are represented in a vector space or

symbolically Supervised learning or unsupervised

Scalable, works with very large data or not

Extracting Rules fromExamples

Apriori Algorithm

FP-Growth

Large quantities of stored data (symbolic) Large means often extremely large, and it is

important that the algorithm gets scalable

Unsupervised learning


3/20

3

Cluster Analysis

K-Means

EM (Expectation Maximization) COBWEBClustering

Assesment

KNN (k Nearest Neighbor)

Data are represented in a vector space

Unsupervised learning

Uncertain knowledge

Naive Bayes

Belief Networks (Bayesian Networks) Main tool is the probability theory, which assigns to

each item numerical degree of belief between 0 and 1

Learning from Observation (symbolic data)

Unsupervised Learning


4/20

4

Decision Trees

ID3

C4.5 cart

Learning from Observation (symbolic data)

Unsupervised Learning

Supervised Classifiers- artificial Neural Networks

Feed forward Networks with one layer: Perceptron

With several layers: Backpropagation Algorithm

RBF Networks

Support Vector Machines

Data are represented in a vector space

Supervised learning


5/20

5

Prediction

Linear Regression

Logistic Regression

Sales Transaction Table

We would like to perform a basket analysis ofthe set of products in a single transaction

Discovering for example, that a customer whobuys shoes is likely to buy socks

Shoes" Socks


6/20

6

Transactional Database

The set of all sales transactions is calledthe population

We represent the transactions in one recordper transaction

The transaction are represented by a datatuple

Shoes,Socks,BeltTX4

Shoes,TieTX3

Shoes,Socks,Tie,Belt,ShirtTX2

Shoes,Socks,TieTX1

Sock is the rule antecedent

Tie is the rule consequent

Socks" Tie


7/20

7

Support and Confidence

Any given association rule has a supportlevel and a confidence level

Support it the percentage of thepopulation which satisfies the rule

If the percentage of the population inwhich the antendent is satisfied is s, thenthe confidence is that percentage inwhich the consequent is also satisfied

Transactional Database

Support is 50% (2/4)

Confidence is 66.67% (2/3)

Shoes,Socks,BeltTX4

Shoes,TieTX3

Shoes,Socks,Tie,Belt,ShirtTX2

Shoes,Socks,TieTX1

Socks" Tie


8/20

8

Apriori Algorithm

Mining for associations among items in a largedatabase of sales transaction is an importantdatabase mining function

For example, the information that a customerwho purchases a keyboard also tends to buy a

mouse at the same time is represented inassociation rule below:

Keyboard Mouse

[support = 6%, confidence = 70%]

Association Rules

Based on the types of values, the associationrules can be classified into two categories:Boolean Association Rules and QuantitativeAssociation Rules

Boolean Association Rule:Keyboard Mouse

[support = 6%, confidence = 70%]

Quantitative Association Rule:

(Age = 26 ...30)(Cars =1, 2)[support 3%, confidence = 36%]


9/20

9

Minimum Support threshold

The support of an association pattern is thepercentage of task-relevant data transactionsfor which the pattern is true

support(A" B) = P(A# B)

A" B

support(A" B) =# _ tuples_containing_both_ A_ and_B

total_# _of _ tuples

Minimum Confidence Threshold

Confidence is defined as the measure of certainty ortrustworthiness associated with each discoveredpattern

The probability ofB given that all we know is A

A" B

confidence(A" B) = P(B | A)

confidence(A" B) =# _ tuples_containing_both _ A_ and_B

# _ tuples_containing_ A


10/20

10

Itemset

A set of items is referred to as itemset

An itemset containing kitems is called

k-itemset

An itemset can be seen as a conjunctionof items (or a presdcate)

Frequent Itemset

Suppose min_supis the minimumsupport threshold

An itemset satisfies minimum support if

the occurrence frequency of the itemsetis greater or equal to min_sup

If an itemset satisfies minimum support,then it is a frequent itemset


11/20

11

Strong Rules

Rules that satisfy both a minimumsupportthreshold and a minimumconfidencethreshold are called strong

Association Rule Mining

Find all frequent itemsets

Generate strong association rules fromthe frequent itemsets

Apriori algorithm is mining frequentitemsets for Boolean associations rules


12/20

12

Apriori Algorithm

Lelel-wise search

k-itemsets (itensets with k items) are used to explore(k+1)- itemsets from transactional databases forBoolean association rules

First, the set of frequent 1-itemsets is found (denoted L1)

L1 is used to find L2, the set of frquent 2-itemsets L2 is used to find L3, and so on, until no frequent k-itemsets

can be found

Generate strong association rules from thefrequent itemsets

Example

Database TDB

1st scan

C1

L1

L2

C2 C2

2nd scan

C3

L33rd scan

B, E40

A, B, C, E30

B, C, E20

A, C, D10

ItemsTid

1{D}

3{E}

3{C}

3{B}

2{A}

supItemset

3{E}

3{C}

3{B}

2{A}

supItemset

{C, E}

{B, E}

{B, C}

{A, E}

{A, C}

{A, B}Itemset1{A, B}

2{A, C}

1{A, E}

2{B, C}

3{B, E}

2{C, E}

supItemset

2{A, C}

2{B, C}

3{B, E}

2{C, E}

supItemset

{B, C, E}

Itemset

2{B, C, E}

supItemset

Supmin = 2


13/20

13

The name of the algorithm is based on the factthat the algorithm uses prior knowledge offrequent items

Employs an iterative approach known as level-

wise search, where k-items are used to explorek+1 items

Apriori Property

Apriori property is used to reduce thesearch space

Apriori property: All nonempty subset of

frequent items must be also frequent Anti-monotone in the sense that if a set

cannot pass a test, all its supper sets will failthe same test as well


14/20

14

Apriori Property

Reducing the search space to avoid finding ofeach Lk requires one full scan of the database(Lk set of frequent k-itemsets)

If an itemset Idoes not satisfy the minimumsupport threshold, min_sup, the Iis not

frequent, P(I) < min_sup If an item A is added to the itemset I, then the

resulting itemset cannot occur more frequentthan I, therfor IA is not frequent,

P(IA)< min_sup

Scalable Methods for Mining

Frequent Patterns

The downward closure property of frequentpatterns Any subset of a frequent itemset must be frequent

If {beer, diaper, nuts} is frequent, so is {beer, diaper}

i.e., every transaction having {beer, diaper, nuts} alsocontains {beer, diaper}

Scalable mining methods: Three major approaches Apriori (Agrawal & Srikant@VLDB94)

Freq. pattern growth (FPgrowthHan, Pei & Yin@SIGMOD00)

Vertical data format approach (CharmZaki & Hsiao@SDM02)


15/20

15

Algorithm

1. Scan the (entire) transaction database to get thesupport Sof each 1-itemset, compare Swithmin_sup, and get a set of frequent 1-itemsets, L1

2. Use Lk-1 join Lk-1 to generate a set of candidate k-itemsets. Use Apriori property to prune theunfreqset k-itemset

3. Scan the transaction database to get the support Sof each candidate k-itemset in the final set, compareS with min_sup, and get a set of frequent k-itemsets, Lk

4. Is the candidate set empty, if not goto 2

5 For each frequent itemset l, generate allnonempty subsets of l

6 For every nonempty subset sof l,output the rule if itsconfidence C > min_conf

I={A1,A2,A5}

s" (I# s)

A1" A2# A5

A2" A5# A1

A1" A5# A2

A1" A2# A5

A2" A1# A5

A5" A1# A2


16/20

16

Example

Five transactions from a supermarket

(diaper=fralda)

Beer,Milk,Coca-Cola5

Diaper,Beer,Detergent4Beer,Diaper,Milk3

Diaper,Baby Powder2

Beer,Diaper,Baby Powder,Bread,Umbrella1

List of ItemsTID

Step 1

Min_sup 40% (2/5)

C1 L1

"1/5"Coca-Cola

"1/5"Detergent

"2/5"Milk

"1/5"Umbrella

"1/5"Bread

"2/5"Baby Powder

"4/5"Diaper

"4/5"Beer

SupportItem

"2/5"Milk

"2/5"Baby Powder

"4/5"Diaper

"4/5"Beer

SupportItem


17/20

17

Step 2 and Step 3

C2 L2

"0"Baby Powder,Milk

"1/5"Diaper,Milk

"2/5"Diaper,Baby Powder

"2/5"Beer, Milk

"1/5"Beer, Baby Powder

"3/5"Beer, Diaper

SupportItem

"2/5"Diaper,BabyPowder

"2/5"Beer, Milk

"3/5"Beer, Diaper

SupportItem

Step 4

C3 empty

Min_sup 40% (2/5)

"0"Diaper,Baby Powder,Milk

"0"Beer, Milk,Baby Powder"1/5"Beer, Diaper,Milk

"1/5"Beer, Diaper,Baby Powder

SupportItem


18/20

18

Step 5

min_sup=40% min_conf=70%

100%40%40%Baby Powder, Diaper

100%40%40%Milk,Beer

75%80%60%Diaper,Beer

50%80%40%Diaper,Baby Powder50%80%40%Beer, Milk

75%80%60%Beer, Diaper

ConfidenceSuport ASupport(A,B)Item

Results

support 60%, confidence 70%




Beer" Diaper

Diaper" Beer

Milk" Beer

Baby _Powder" Diaper


19/20

19

Interpretation

Some results are belivable, like Baby Powder Diaper

Some rules need aditional analysis, like MilkBeer

Some rules are unbelivable, like Diaper Beer

This example could contain unreal resultsbecause of the small data

Machine Learning Overview

Sales Transaction and Association

Rules

Aprori Algorithm

Example


20/20

20

How to make Apriori faster?

FP-growth

presentation asrul

Documents