Data Mining:

Concepts and Techniques

(3rd ed.)

Chapter 6
Jiawei Han, Micheline Kamber, and Jian Pei University of Illinois at Urbana-Champaign &

Simon Fraser University

2011 Han, Kamber & Pei. All rights reserved.
1

Chapter 5: Mining Frequent Patterns, Association and Correlations: Basic Concepts and Methods
Basic Concepts Frequent Itemset Mining Methods Which Patterns Are Interesting?Pattern

Evaluation Methods
Summary
2

What Is Frequent Pattern Analysis?

Frequent pattern: a pattern (a set of items, subsequences, substructures,

etc.) that occurs frequently in a data set

First proposed by Agrawal, Imielinski, and Swami [AIS93] in the context of frequent itemsets and association rule mining

Motivation: Finding inherent regularities in data

What products were often purchased together? Beer and diapers?! What are the subsequent purchases after buying a PC?

What kinds of DNA are sensitive to this new drug?

Can we automatically classify web documents? Basket data analysis, cross-marketing, catalog design, sale campaign analysis, Web log (click stream) analysis, and DNA sequence analysis.
3

Applications

Why Is Freq. Pattern Mining Important?

Freq. pattern: An intrinsic and important property of datasets Foundation for many essential data mining tasks Association, correlation, and causality analysis Sequential, structural (e.g., sub-graph) patterns Pattern analysis in spatiotemporal, multimedia, timeseries, and stream data Classification: discriminative, frequent pattern analysis Cluster analysis: frequent pattern-based clustering Data warehousing: iceberg cube and cube-gradient Semantic data compression: fascicles Broad applications
4

Basic Concepts: Frequent Patterns

Tid 10 20 30 40 50 Items bought Beer, Nuts, Diaper Beer, Coffee, Diaper Beer, Diaper, Eggs Nuts, Eggs, Milk Nuts, Coffee, Diaper, Eggs, Milk Customer buys both Customer buys diaper

Customer buys beer

itemset: A set of one or more items k-itemset X = {x1, , xk} (absolute) support, or, support count of X: Frequency or occurrence of an itemset X (relative) support, s, is the fraction of transactions that contains X (i.e., the probability that a transaction contains X) An itemset X is frequent if Xs support is no less than a minsup threshold
5

Basic Concepts: Association Rules

Tid

Items bought

10 20
30 40 50

Beer, Nuts, Diaper Beer, Coffee, Diaper

Beer, Diaper, Eggs Nuts, Eggs, Milk
Nuts, Coffee, Diaper, Eggs, Milk Customer buys both

Customer buys diaper

Find all the rules X Y with minimum support and confidence support, s, probability that a transaction contains X Y confidence, c, conditional probability that a transaction having X also contains Y
{Beer, Diaper}:3

Let minsup = 50%, minconf = 50% Freq. Pat.: Beer:3, Nuts:3, Diaper:4, Eggs:3,

Customer buys beer

Association rules: (many more!) Beer Diaper (60%, 100%) Diaper Beer (60%, 75%)
6

Closed Patterns and Max-Patterns

A long pattern contains a combinatorial number of subpatterns, e.g., {a1, , a100} contains (1001) + (1002) + + (110000) = 2100 1 = 1.27*1030 sub-patterns! Solution: Mine closed patterns and max-patterns instead

An itemset X is closed if X is frequent and there exists no super-pattern Y X, with the same support as X (proposed by Pasquier, et al. @ ICDT99)
An itemset X is a max-pattern if X is frequent and there exists no frequent super-pattern Y X (proposed by Bayardo @ SIGMOD98) Closed pattern is a lossless compression of freq. patterns

Reducing the # of patterns and rules

7

Closed Patterns and Max-Patterns

Exercise. DB = {<a1, , a100>, < a1, , a50>}

Min_sup = 1. <a1, , a100>: 1

< a1, , a50>: 2 <a1, , a100>: 1 !!
8

What is the set of closed itemset?

What is the set of max-pattern?

What is the set of all patterns?

Computational Complexity of Frequent Itemset Mining

How many itemsets are potentially to be generated in the worst case?

The number of frequent itemsets to be generated is senstive to the minsup threshold When minsup is low, there exist potentially an exponential number of frequent itemsets The worst case: MN where M: # distinct items, and N: max length of transactions Ex. Suppose Walmart has 104 kinds of products

The worst case complexty vs. the expected probability

The chance to pick up one product 10-4 The chance to pick up a particular set of 10 products: ~10-40 What is the chance this particular set of 10 products to be frequent 103 times in 109 transactions?
9

Chapter 5: Mining Frequent Patterns, Association and Correlations: Basic Concepts and Methods
Basic Concepts Frequent Itemset Mining Methods Which Patterns Are Interesting?Pattern

Evaluation Methods
Summary
10

Scalable Frequent Itemset Mining Methods

Apriori: A Candidate Generation-and-Test Approach

Improving the Efficiency of Apriori

FPGrowth: A Frequent Pattern-Growth Approach ECLAT: Frequent Pattern Mining with Vertical

Data Format
11

The Downward Closure Property and Scalable Mining Methods

The downward closure property of frequent patterns Any subset of a frequent itemset must be frequent If {beer, diaper, nuts} is frequent, so is {beer, diaper} i.e., every transaction having {beer, diaper, nuts} also contains {beer, diaper} Scalable mining methods: Three major approaches Apriori (Agrawal & Srikant@VLDB94) Freq. pattern growth (FPgrowthHan, Pei & Yin @SIGMOD00) Vertical data format approach (CharmZaki & Hsiao @SDM02)
12

Apriori: A Candidate Generation & Test Approach

Apriori pruning principle: If there is any itemset which is infrequent, its superset should not be generated/tested! (Agrawal & Srikant @VLDB94, Mannila, et al. @ KDD 94) Method:

Initially, scan DB once to get frequent 1-itemset Generate length (k+1) candidate itemsets from length k frequent itemsets Test the candidates against DB Terminate when no frequent or candidate set can be generated
13

The Apriori AlgorithmAn Example

Database TDB
Tid
10 20 30 40

Supmin = 2 C1 1st scan

Itemset {A} {B} {C} {D} {E}

sup 2 3 3 1 3 sup 1 2 1 2 3 2

Itemset

sup 2

Items
A, C, D B, C, E A, B, C, E B, E

L1

{A}

{B}
{C} {E}

3
3 3

C2

L2

Itemset {A, C} {B, C} {B, E} {C, E}

sup 2 2 3 2

Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E}

C2 2nd scan

Itemset {A, B} {A, C} {A, E} {B, C} {B, E} {C, E}

C3

Itemset {B, C, E}

3rd scan

L3

Itemset {B, C, E}

sup 2
14

The Apriori Algorithm (Pseudo-Code)

Ck: Candidate itemset of size k Lk : frequent itemset of size k L1 = {frequent items}; for (k = 1; Lk != ; k++) do begin Ck+1 = candidates generated from Lk; for each transaction t in database do
increment the count of all candidates in Ck+1 that are contained in t Lk+1 = candidates in Ck+1 with min_support end return k Lk;
15

Implementation of Apriori

How to generate candidates?

Step 1: self-joining Lk Step 2: pruning

Example of Candidate-generation

L3={abc, abd, acd, ace, bcd} Self-joining: L3*L3

abcd from abc and abd acde from acd and ace
acde is removed because ade is not in L3

Pruning:

C4 = {abcd}
16

How to Count Supports of Candidates?

Why counting supports of candidates a problem?

The total number of candidates can be very huge One transaction may contain many candidates Candidate itemsets are stored in a hash-tree

Method:

Leaf node of hash-tree contains a list of itemsets and

counts

Interior node contains a hash table Subset function: finds all the candidates contained in
a transaction
17

Counting Supports of Candidates Using Hash Tree

Subset function 3,6,9 1,4,7
2,5,8 1+2356 13+56 145 12+356 124 457 234 567 136

Transaction: 1 2 3 5 6

345

356 357 689

367 368

125 458

159

18

Candidate Generation: An SQL Implementation

SQL Implementation of candidate generation Suppose the items in Lk-1 are listed in an order Step 1: self-joining Lk-1 insert into Ck select p.item1, p.item2, , p.itemk-1, q.itemk-1 from Lk-1 p, Lk-1 q where p.item1=q.item1, , p.itemk-2=q.itemk-2, p.itemk-1 <

q.itemk-1

Step 2: pruning forall itemsets c in Ck do forall (k-1)-subsets s of c do if (s is not in Lk-1) then delete c from Ck Use object-relational extensions like UDFs, BLOBs, and Table functions for efficient implementation [See: S. Sarawagi, S. Thomas, and R. Agrawal. Integrating association rule mining with relational database systems: Alternatives and implications. SIGMOD98]

19

Scalable Frequent Itemset Mining Methods

Apriori: A Candidate Generation-and-Test Approach Improving the Efficiency of Apriori FPGrowth: A Frequent Pattern-Growth Approach

ECLAT: Frequent Pattern Mining with Vertical Data Format

Mining Close Frequent Patterns and Maxpatterns
20

Further Improvement of the Apriori Method

Major computational challenges

Multiple scans of transaction database Huge number of candidates Tedious workload of support counting for candidates

Improving Apriori: general ideas

Reduce passes of transaction database scans

Shrink number of candidates Facilitate support counting of candidates

21

Partition: Scan Database Only Twice

Any itemset that is potentially frequent in DB must be frequent in at least one of the partitions of DB Scan 1: partition database and find local frequent patterns Scan 2: consolidate global frequent patterns A. Savasere, E. Omiecinski and S. Navathe, VLDB95

DB1 sup1(i) < DB1

DB2 sup2(i) < DB2

DBk

DB

supk(i) < DBk

sup(i) < DB

DHP: Reduce the Number of Candidates

A k-itemset whose corresponding hashing bucket count is below the

threshold cannot be frequent

count
35 88

itemsets
{ab, ad, ae} {bd, be, de}

Candidates: a, b, c, d, e Hash entries

{bd, be, de}

102

{yz, qs, wt}

Frequent 1-itemset: a, b, d, e

Hash Table

ab is not a candidate 2-itemset if the sum of count of {ab, ad, ae}

is below support threshold

J. Park, M. Chen, and P. Yu. An effective hash-based algorithm for mining association rules. SIGMOD95
23

. . .

. . .

{ab, ad, ae}

Sampling for Frequent Patterns

Select a sample of original database, mine frequent patterns within sample using Apriori Scan database once to verify frequent itemsets found in

sample, only borders of closure of frequent patterns are

checked

Example: check abcd instead of ab, ac, , etc.

Scan database again to find missed frequent patterns H. Toivonen. Sampling large databases for association

rules. In VLDB96
24

DIC: Reduce Number of Scans

ABCD

ABC ABD ACD BCD AB AC A BC B C AD D BD CD

Once both A and D are determined frequent, the counting of AD begins Once all length-2 subsets of BCD are determined frequent, the counting of BCD begins

Transactions

Apriori

1-itemsets 2-itemsets 1-itemsets 2-items

{} Itemset lattice
S. Brin R. Motwani, J. Ullman, DIC and S. Tsur. Dynamic itemset counting and implication rules for market basket data. SIGMOD97

3-items

25

Scalable Frequent Itemset Mining Methods

Apriori: A Candidate Generation-and-Test Approach Improving the Efficiency of Apriori FPGrowth: A Frequent Pattern-Growth Approach

ECLAT: Frequent Pattern Mining with Vertical Data Format

Mining Close Frequent Patterns and Maxpatterns
26

Pattern-Growth Approach: Mining Frequent Patterns Without Candidate Generation

Bottlenecks of the Apriori approach

Breadth-first (i.e., level-wise) search Candidate generation and test

Often generates a huge number of candidates

The FPGrowth Approach (J. Han, J. Pei, and Y. Yin, SIGMOD 00)

Depth-first search Avoid explicit candidate generation

Major philosophy: Grow long patterns from short ones using local frequent items only

abc is a frequent pattern Get all transactions having abc, i.e., project DB on abc: DB|abc d is a local frequent item in DB|abc abcd is a frequent pattern
27

Construct FP-tree from a Transaction Database

Items bought (ordered) frequent items {f, a, c, d, g, i, m, p} {f, c, a, m, p} {a, b, c, f, l, m, o} {f, c, a, b, m} min_support = 3 {b, f, h, j, o, w} {f, b} {b, c, k, s, p} {c, b, p} {a, f, c, e, l, p, m, n} {f, c, a, m, p} {} Header Table 1. Scan DB once, find f:4 c:1 Item frequency head frequent 1-itemset (single f 4 item pattern) c 4 c:3 b:1 b:1 2. Sort frequent items in a 3 b 3 frequency descending a:3 p:1 m 3 order, f-list p 3 m:2 b:1 3. Scan DB again, construct FP-tree p:2 m:1 F-list = f-c-a-b-m-p
28

TID 100 200 300 400 500

Partition Patterns and Databases

Frequent patterns can be partitioned into subsets according to f-list F-list = f-c-a-b-m-p Patterns containing p Patterns having m but no p Patterns having c but no a nor b, m, p Pattern f Completeness and non-redundency
29

Find Patterns Having P From P-conditional Database

Starting at the frequent item header table in the FP-tree Traverse the FP-tree by following the link of each frequent item p Accumulate all of transformed prefix paths of item p to form ps conditional pattern base
{} f:4 c:3 a:3 m:2 p:2 b:1 m:1 b:1 c:1 b:1 p:1 Conditional pattern bases item c a cond. pattern base f:3 fc:3

Header Table Item frequency head f 4 c 4 a 3 b 3 m 3 p 3

b
m p

fca:1, f:1, c:1

fca:2, fcab:1 fcam:2, cb:1
30

From Conditional Pattern-bases to Conditional FP-trees

For each pattern-base Accumulate the count for each item in the base Construct the FP-tree for the frequent items of the pattern base
{} f:4 c:1
m-conditional pattern base: fca:2, fcab:1 All frequent patterns relate to m {} m, f:3 fm, cm, am, fcm, fam, cam, c:3 fcam

Header Table Item frequency head f 4 c 4 a 3 b 3 m 3 p 3

c:3
a:3 m:2 p:2

b:1

b:1
p:1

b:1 m:1

a:3
m-conditional FP-tree
31

Recursion: Mining Each Conditional FP-tree

{} {} f:3 c:3 a:3
m-conditional FP-tree

Cond. pattern base of am: (fc:3)

f:3 c:3

am-conditional FP-tree

Cond. pattern base of cm: (f:3)

{} f:3

cm-conditional FP-tree

{}

Cond. pattern base of cam: (f:3)

f:3

cam-conditional FP-tree
32

A Special Case: Single Prefix Path in FP-tree

Suppose a (conditional) FP-tree T has a shared single prefix-path P Mining can be decomposed into two parts

{}
a1:n1 a2:n2 a3:n3
b1:m1 C2:k2

Reduction of the single prefix path into one node

Concatenation of the mining results of the two parts
{}
r1

C1:k1 C3:k3

r1

a1:n1

a2:n2
a3:n3

b1:m1 C2:k2

C1:k1 C3:k3
33

Benefits of the FP-tree Structure

Completeness

Preserve complete information for frequent pattern mining Never break a long pattern of any transaction Reduce irrelevant infoinfrequent items are gone

Compactness

Items in frequency descending order: the more frequently occurring, the more likely to be shared
Never be larger than the original database (not count node-links and the count field)
34

The Frequent Pattern Growth Mining Method

Idea: Frequent pattern growth Recursively grow frequent patterns by pattern and database partition Method For each frequent item, construct its conditional pattern-base, and then its conditional FP-tree Repeat the process on each newly created conditional FP-tree Until the resulting FP-tree is empty, or it contains only one pathsingle path will generate all the combinations of its sub-paths, each of which is a frequent pattern
35

Scaling FP-growth by Database Projection

What about if FP-tree cannot fit in memory?

DB projection

First partition a database into a set of projected DBs Then construct and mine FP-tree for each projected DB

Parallel projection vs. partition projection techniques

Parallel projection

Project the DB in parallel for each frequent item

Parallel projection is space costly

All the partitions can be processed in parallel

Partition projection

Partition the DB based on the ordered frequent items

Passing the unprocessed parts to the subsequent partitions
36

Partition-Based Projection

Parallel projection needs a lot of disk space Partition projection saves it

Tran. DB
fcamp fcabm fb cbp fcamp

p-proj DB
fcam cb fcam

m-proj DB
fcab fca fca

b-proj DB
f cb

a-proj DB
fc

c-proj DB
f

f-proj DB

am-proj DB
fc fc fc

cm-proj DB
f f f

37

Performance of FPGrowth in Large Datasets

100 90 80
D1 FP-grow th runtime D1 Apriori runtime

140 120
Runtime (sec.)

D2 FP-growth D2 TreeProjection

Run time(sec.)

70 60 50 40 30 20 10 0 0 0.5 1 1.5 2 Support threshold(%) 2.5 3

100 80 60 40 20 0 0 0.5 1 Support threshold (%) 1.5 2

Data set T25I20D10K

Data set T25I20D100K

FP-Growth vs. Apriori

FP-Growth vs. Tree-Projection

38

Advantages of the Pattern Growth Approach

Divide-and-conquer:

Decompose both the mining task and DB according to the frequent patterns obtained so far
Lead to focused search of smaller databases No candidate generation, no candidate test Compressed database: FP-tree structure

Other factors

No repeated scan of entire database

Basic ops: counting local freq items and building sub FP-tree, no pattern search and matching FPGrowth+ (Grahne and J. Zhu, FIMI'03)
39

A good open-source implementation and refinement of FPGrowth

Further Improvements of Mining Methods

AFOPT (Liu, et al. @ KDD03)

A push-right method for mining condensed frequent pattern (CFP) tree

Carpenter (Pan, et al. @ KDD03)

Mine data sets with small rows but numerous columns Construct a row-enumeration tree for efficient mining Efficiently Using Prefix-Trees in Mining Frequent Itemsets, Proc. ICDM'03 Int. Workshop on Frequent Itemset Mining

FPgrowth+ (Grahne and Zhu, FIMI03)

Implementations (FIMI'03), Melbourne, FL, Nov. 2003

TD-Close (Liu, et al, SDM06)

40

Extension of Pattern Growth Mining Methodology

Mining closed frequent itemsets and max-patterns CLOSET (DMKD00), FPclose, and FPMax (Grahne & Zhu, Fimi03) Mining sequential patterns PrefixSpan (ICDE01), CloSpan (SDM03), BIDE (ICDE04) Mining graph patterns gSpan (ICDM02), CloseGraph (KDD03) Constraint-based mining of frequent patterns Convertible constraints (ICDE01), gPrune (PAKDD03) Computing iceberg data cubes with complex measures H-tree, H-cubing, and Star-cubing (SIGMOD01, VLDB03) Pattern-growth-based Clustering MaPle (Pei, et al., ICDM03) Pattern-Growth-Based Classification Mining frequent and discriminative patterns (Cheng, et al, ICDE07)
41

Scalable Frequent Itemset Mining Methods

Apriori: A Candidate Generation-and-Test Approach Improving the Efficiency of Apriori FPGrowth: A Frequent Pattern-Growth Approach

ECLAT: Frequent Pattern Mining with Vertical Data Format

Mining Close Frequent Patterns and Maxpatterns
42

ECLAT: Mining by Exploring Vertical Data Format

Vertical format: t(AB) = {T11, T25, }

tid-list: list of trans.-ids containing an itemset

t(X) = t(Y): X and Y always happen together t(X) t(Y): transaction having X always has Y

Deriving frequent patterns based on vertical intersections

Using diffset to accelerate mining

Only keep track of differences of tids t(X) = {T1, T2, T3}, t(XY) = {T1, T3} Diffset (XY, X) = {T2}

Eclat (Zaki et al. @KDD97) Mining Closed patterns using vertical format: CHARM (Zaki & Hsiao@SDM02)
43

Scalable Frequent Itemset Mining Methods

Apriori: A Candidate Generation-and-Test Approach Improving the Efficiency of Apriori FPGrowth: A Frequent Pattern-Growth Approach

ECLAT: Frequent Pattern Mining with Vertical Data Format

Mining Close Frequent Patterns and Maxpatterns
44

Mining Frequent Closed Patterns: CLOSET

Flist: list of all frequent items in support ascending order

Flist: d-a-f-e-c

Min_sup=2
TID 10 20 30 40 50 Items a, c, d, e, f a, b, e c, e, f a, c, d, f c, e, f

Divide search space

Patterns having d
Patterns having d but no a, etc.

Find frequent closed pattern recursively

Every transaction having d also has cfa cfad is a frequent closed pattern

J. Pei, J. Han & R. Mao. CLOSET: An Efficient Algorithm for Mining Frequent Closed Itemsets", DMKD'00.

CLOSET+: Mining Closed Itemsets by Pattern-Growth

Itemset merging: if Y appears in every occurrence of X, then Y is merged with X Sub-itemset pruning: if Y X, and sup(X) = sup(Y), X and all of Xs descendants in the set enumeration tree can be pruned Hybrid tree projection

Bottom-up physical tree-projection Top-down pseudo tree-projection

Item skipping: if a local frequent item has the same support in several header tables at different levels, one can prune it from the header table at higher levels Efficient subset checking

MaxMiner: Mining Max-Patterns

1st scan: find frequent items

Tid 10 20 30

Items A, B, C, D, E B, C, D, E, A, C, D, F

A, B, C, D, E

2nd scan: find support for

AB, AC, AD, AE, ABCDE

BC, BD, BE, BCDE CD, CE, CDE, DE

Potential max-patterns

Since BCDE is a max-pattern, no need to check BCD, BDE, CDE in later scan

R. Bayardo. Efficiently mining long patterns from databases. SIGMOD98

CHARM: Mining by Exploring Vertical Data Format

Vertical format: t(AB) = {T11, T25, }

tid-list: list of trans.-ids containing an itemset t(X) = t(Y): X and Y always happen together t(X) t(Y): transaction having X always has Y

Deriving closed patterns based on vertical intersections

Using diffset to accelerate mining

Only keep track of differences of tids

t(X) = {T1, T2, T3}, t(XY) = {T1, T3} Diffset (XY, X) = {T2}

Eclat/MaxEclat (Zaki et al. @KDD97), VIPER(P. Shenoy et al.@SIGMOD00), CHARM (Zaki & Hsiao@SDM02)

Visualization of Association Rules: Plane Graph

49

Visualization of Association Rules: Rule Graph

50

Visualization of Association Rules (SGI/MineSet 3.0)

51

Chapter 5: Mining Frequent Patterns, Association and Correlations: Basic Concepts and Methods
Basic Concepts Frequent Itemset Mining Methods Which Patterns Are Interesting?Pattern

Evaluation Methods
Summary
52

Interestingness Measure: Correlations (Lift)

play basketball

eat cereal [40%, 66.7%] is misleading not eat cereal [20%, 33.3%] is more accurate,

The overall % of students eating cereal is 75% > 66.7%.

play basketball

although with lower support and confidence

Measure of dependent/correlated events: lift

lift
lift( B, C )

P( A B) P( A) P( B)
0.89

Basketball Cereal Not cereal Sum(col.) 2000 1000 3000

Not basketball 1750 250 2000

Sum (row) 3750 1250 5000

2000 / 5000 3000 / 5000 * 3750 / 5000

lift( B, C )

1000 / 5000 3000 / 5000 *1250 / 5000

1.33

53

Are lift and

Good Measures of Correlation?

Buy walnuts buy milk [1%, 80%] is

misleading if 85% of customers buy milk Support and confidence are not good to indicate correlations Over 20 interestingness measures have been proposed (see Tan, Kumar, Sritastava

@KDD02)

Which are good ones?

54

Null-Invariant Measures

55

Comparison of Interestingness Measures

Null-(transaction) invariance is crucial for correlation analysis Lift and 2 are not null-invariant 5 null-invariant measures
Milk No Milk ~m, c ~m, ~c ~m Sum (row) c ~c

Coffee No Coffee Sum(col.)

m, c m, ~c m

Null-transactions w.r.t. m and c

Kulczynski measure (1927)

Null-invariant

November 15, 2011

Data Mining: Concepts and Techniques

Subtle: They disagree56

Analysis of DBLP Coauthor Relationships

Recent DB conferences, removing balanced associations, low sup, etc.

Advisor-advisee relation: Kulc: high, coherence: low, cosine: middle

Tianyi Wu, Yuguo Chen and Jiawei Han, Association Mining in Large Databases: A Re-Examination of Its Measures, Proc. 2007 Int. Conf. Principles and Practice of Knowledge Discovery in Databases (PKDD'07), Sept. 2007
57

Which Null-Invariant Measure Is Better?

IR (Imbalance Ratio): measure the imbalance of two itemsets A and B in rule implications

Kulczynski and Imbalance Ratio (IR) together present a clear picture for all the three datasets D4 through D6 D4 is balanced & neutral D5 is imbalanced & neutral D6 is very imbalanced & neutral

Chapter 5: Mining Frequent Patterns, Association and Correlations: Basic Concepts and Methods
Basic Concepts Frequent Itemset Mining Methods Which Patterns Are Interesting?Pattern

Evaluation Methods
Summary
59

Summary

Basic concepts: association rules, supportconfident framework, closed and max-patterns Scalable frequent pattern mining methods

Apriori (Candidate generation & test) Projection-based (FPgrowth, CLOSET+, ...)

Vertical format approach (ECLAT, CHARM, ...)

Which patterns are interesting?

Pattern evaluation methods

60

Ref: Basic Concepts of Frequent Pattern Mining

(Association Rules) R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. SIGMOD'93 (Max-pattern) R. J. Bayardo. Efficiently mining long patterns from databases. SIGMOD'98

(Closed-pattern) N. Pasquier, Y. Bastide, R. Taouil, and L. Lakhal. Discovering frequent closed itemsets for association rules. ICDT'99
(Sequential pattern) R. Agrawal and R. Srikant. Mining sequential patterns. ICDE'95

61

Ref: Apriori and Its Improvements

R. Agrawal and R. Srikant. Fast algorithms for mining association rules. VLDB'94 H. Mannila, H. Toivonen, and A. I. Verkamo. Efficient algorithms for discovering association rules. KDD'94

A. Savasere, E. Omiecinski, and S. Navathe. An efficient algorithm for mining association rules in large databases. VLDB'95
J. S. Park, M. S. Chen, and P. S. Yu. An effective hash-based algorithm for mining association rules. SIGMOD'95 H. Toivonen. Sampling large databases for association rules. VLDB'96 S. Brin, R. Motwani, J. D. Ullman, and S. Tsur. Dynamic itemset counting and implication rules for market basket analysis. SIGMOD'97

S. Sarawagi, S. Thomas, and R. Agrawal. Integrating association rule mining with relational database systems: Alternatives and implications. SIGMOD'98
62

Ref: Depth-First, Projection-Based FP Mining

R. Agarwal, C. Aggarwal, and V. V. V. Prasad. A tree projection algorithm for generation of frequent itemsets. J. Parallel and Distributed Computing, 2002. G. Grahne and J. Zhu, Efficiently Using Prefix-Trees in Mining Frequent Itemsets, Proc. FIMI'03 B. Goethals and M. Zaki. An introduction to workshop on frequent itemset mining implementations. Proc. ICDM03 Int. Workshop on Frequent Itemset Mining Implementations (FIMI03), Melbourne, FL, Nov. 2003 J. Han, J. Pei, and Y. Yin. Mining frequent patterns without candidate generation. SIGMOD 00 J. Liu, Y. Pan, K. Wang, and J. Han. Mining Frequent Item Sets by Opportunistic Projection. KDD'02 J. Han, J. Wang, Y. Lu, and P. Tzvetkov. Mining Top-K Frequent Closed Patterns without Minimum Support. ICDM'02 J. Wang, J. Han, and J. Pei. CLOSET+: Searching for the Best Strategies for Mining Frequent Closed Itemsets. KDD'03
63

Ref: Vertical Format and Row Enumeration Methods

M. J. Zaki, S. Parthasarathy, M. Ogihara, and W. Li. Parallel algorithm for discovery of association rules. DAMI:97.

M. J. Zaki and C. J. Hsiao. CHARM: An Efficient Algorithm for Closed Itemset Mining, SDM'02.

C. Bucila, J. Gehrke, D. Kifer, and W. White. DualMiner: A Dual-Pruning Algorithm for Itemsets with Constraints. KDD02.

F. Pan, G. Cong, A. K. H. Tung, J. Yang, and M. Zaki , CARPENTER: Finding

Closed Patterns in Long Biological Datasets. KDD'03.

H. Liu, J. Han, D. Xin, and Z. Shao, Mining Interesting Patterns from Very High Dimensional Data: A Top-Down Row Enumeration Approach, SDM'06.

64

Ref: Mining Correlations and Interesting Rules

S. Brin, R. Motwani, and C. Silverstein. Beyond market basket: Generalizing association rules to correlations. SIGMOD'97. M. Klemettinen, H. Mannila, P. Ronkainen, H. Toivonen, and A. I. Verkamo. Finding interesting rules from large sets of discovered association rules. CIKM'94. R. J. Hilderman and H. J. Hamilton. Knowledge Discovery and Measures of Interest. Kluwer Academic, 2001. C. Silverstein, S. Brin, R. Motwani, and J. Ullman. Scalable techniques for mining causal structures. VLDB'98.

P.-N. Tan, V. Kumar, and J. Srivastava. Selecting the Right Interestingness Measure for Association Patterns. KDD'02.
E. Omiecinski. Alternative Interest Measures for Mining Associations. TKDE03. T. Wu, Y. Chen, and J. Han, Re-Examination of Interestingness Measures in Pattern Mining: A Unified Framework", Data Mining and Knowledge Discovery, 21(3):371397, 2010
65