BIDE : Efficient Mining of Frequent Closed Sequences
Jianyong Wang and Jiawei Han Proc. 2004 Int. Conf. on Data Engineering (ICDE'04), Boston, MA, March 2004.
Opening
{,,} sup = 5 {,,,} sup = 5 sup < 5 sup > 5
{, ,} sup = 5 {,, ,} sup = 5
Review of Algorithms
Problem was first introduced by Agrawal and Srikant 1995. Mostly for mining frequent sequences (closed and non-closed). frequent sequences :
Apriori-based (Apriori, AprioriAll, GSP) Sequence-Enumeration Tree/Lattice-based (Max-Miner, Spam, Spade) Constraint-based (SPIRIT Family)
frequent closed sequences :
CloSpan : The first algorithm for mining frequent closed sequences (Based on Prefix-Span). BIDE
Apriori-based algorithms
Make multiple passes over the database.
First Pass:
Find the 1-frequent sequences Join them to build the candidate 2-sequences
For step k:
Join the set of frequent (k-1)-sequences Scan the SDB to check which of the k-sequences are actually frequent
Stop at step m:
No frequent m-sequence is found
Tree-based algorithms (1/2)
Use a sequence enumeration tree to generate all the candidate sequences. Traverse the tree (DFS). Use pruning techniques to avoid traversing sub-trees.
Apriori-principle: If a sequence is non-frequent, its supersequences can not be frequent.
If an extension event leads to a non-frequent sequence, this event is no longer used in other sequence extensions.
Tree-based algorithms (2/2)
I = {a, b} S-step:
add item to sequence
I-step: add item to event
Definition of Terms (1/2)
Items set: I = {i1, i2, ,in} Ordered Events set (Sequence) : S = {e1, e2, , em} Length : m-sequence Subsequence : Sa = {a1, a2, , an} Super-sequence : Sb = {b1, b2, , bm}
a1= bi1, a2 = bi2, , an = bin 1 i1 <i2<< in m
Definition of Terms (2/2)
SDB : sequence database tuple (sid, S) S : is a subsequence of S. supSDB(Sa) : Absolute support supSDB(Sa) / |SDB| : Relative support min_sup : supSDB(Sa) min_sup
S is closed: if there exists no other super-sequence with the same support.
Frequent Closed Sequences
Find the complete set of frequent closed sequences :
SDB
I = { A, B, C } min_sup = 50%
Previous Algorithms : 17 Frequent Sequences
BIDE (and CloSpan) :
6 Frequent Closed Sequences
CloSpan
Maintains the set of already mined frequent closed patterns in memory.
Sub-pattern checking: new pattern can be absorbed by an already mined frequent pattern. Super-pattern checking: new pattern can absorb some already mined frequent patterns.
To save space, CloSpan stores a superset of the frequent patterns in a hash-tree indexed structure and then prunes the tree to get the actual set of frequent closed patterns.
BIDE (1/2)
BIDE : An efficient algorithm for discovering the complete set
of frequent closed sequences.
BI-Directional Extension: new paradigm for mining frequent closed sequences without candidate maintenance.
Back-Scan pruning method.
Back-Skip optimization technique. Performance study: BIDE can be over an order of magnitude faster than the previous algorithms and consumes orders of magnitude less memory.
BIDE (2/2)
How to enumerate the complete set of frequent sequences?
Upon getting a frequent sequence, how to check if it is
closed?
How to design some search space pruning methods or other optimization techniques to accelerate the mining process?
First instance
S =CAABC Sp = A B First instance of a prefix sequence Sp = C A A B Projected Sequence of a prefix sequence Sp = C Projected Database of a prefix sequence Sp = Sp_SDB.
i-th last-in-first appearance
S =CAABC Sp = C A first instance of a prefix sequence Sp = C A
2nd last-in-first appearance = the last A of first instance
i-th semi-maximum period : SMP i( Sp )
S =ABCB Sp = A C the end of the first instance of prefix e1e2ei-1 in S = A between the 2nd last-in-first appearance = C SMP 2( A C) = B SMP 1( A C) =
Back-Scan search space pruning method
Theorem : Let the pre-fix sequence be an n-sequence, Sp = e1e2 . . . en. If i (1 i n), there exists an item e which appears in each of the i-th SMP of the prefix Sp in SDB, we can safely stop growing prefix Sp.
I = { A, B, C } min_sup = 50%
Sp = B : 4 SMP1 = {CAA, A, CA, A} Prune sub-tree under B
BI-Directional Extension closure checking
Sp = e1e2en Sp* = e1e2en e supSDB(Sp) = supSDB(Sp*) Sp is non-closed, item e is a forward extension event. Sp = e1e2en Sp2 = e1e2ei e ei+1en Sp = e1e2en Sp2 = e e1e2en supSDB(Sp) = supSDB(Sp*) Sp is non-closed, item e is a backward extension event.
or
Theorem : If there exists no forward extension event nor backward extension event w.r.t. Sp, Sp must be closed; otherwise it must be non-closed.
Last instance
S =CABDCABBA Sp = A B Last instance of a prefix sequence Sp = C A B D C A B B
i-th last-in-last appearance
S =CAABC Sp = A B Last instance of a prefix sequence Sp = C A A B
1st last-in-last appearance = the last A of last instance
i-th maximum period : MP i( Sp )
S =ABCB Sp = A B the end of the first instance of prefix e1e2ei-1 in S = A between the 2nd last-in-last appearance = B MP 2( A B) = B C MP 1( A B) =
Backward-Extension event checking
Lemma If there exists an item e which appears in each of the i-th maximum periods of Sp in SDB, then e is a backward extension event w.r.t. Sp. SDB I = { A, B, C } min_sup = 50%
Sp = A C : 4
MP2 = {AB, B, B, BB} Backward Extension Event for Sp(AC) is B. AC:4 is a frequent non-closed sequence. Sp = A B C : 4 No backward-extension item for Sp. No forward-extension item for Sp. ABC:4 is a frequent closed sequence.
Scan-Skip optimization
SDB I = { A, B, C } min_sup = 50% Sp = ABC with support 4.
MP1 = {CA, , C, }
skip last two
MP2 = {A, , , B}
skip last two
MP3 = {, , , B}
skip last three
Pruning and optimization
Projected Sequence & Projected Database
SDB
Sequence identifier
1 2 3 4
Sp_SDB = {C, CB, C, BCA}
Sequence
CAABC ABCB CABC ABBCA
[Sp = AB]
Sequence identifier 1 2 3 4
Sequence C CB C BCA
Frequent Sequence Enumeration (Prefix-Span)
Forward-extension event checking
Lemma For a pre-fix sequence Sp, its complete set of forward-extension events is equivalent to the set of its locally frequent items whose supports are equal to supSDB(Sp) .
SDB I = { A, B, C } min_sup = 50%
Sp1 = AB : 4 Sp1_SDB = {C, CB, C, BCA} Forward Extension Event for AB is C. Sp2 = CAB : 2 SP2_SDB = {C, , C, } Forward Extension Event for CAB is C.
Scheme of Bide Algorithm
SDB min_sup
Sp (Start from frequent 1-sequences)
Grow Sp with extension event.
BackScan pruning
Stop growing Sp.
BI-Directional Extension closure checking Backward-Extension event checking Find extension events. frequent non-closed sequences Forward-extension event checking No extension event. frequent closed sequences
The BIDE algorithm (1/2)
Scan the database once in order to find all the frequent 1sequences.
For each frequent 1-sequence build a pseudo-projected database and check if it can be pruned: Back-Scan. For every sequence that cannot be pruned compute the number of backward-extension items and then call subroutine bide.
The BIDE algorithm (2/2)
Subroutine bide:
For prefix Sp scan its projected DB. Compute the number of forward-extension items. If there is no forward-extension item nor backward-extension item, Sp is closed. Grow Sp with each locally frequent item to get a new prefix. Build the pseudo-projected DB for the new prefix. For each new prefix:
Check if it can be pruned If not, compute the number of backward-extension items and call itself
Performance Evaluation
BIDE can significantly outperform PrefixSpan and SPADE. BIDE consumes much less memory and can be faster than CloSpan. BackScan pruning method is effective in enhancing the performance. Experiments on three datasets.
SPADE vs. PrefixSpan vs. CloSpan vs. BIDE
CloSpan vs. BIDE(1/3)
Gazelle Dataset
CloSpan vs. BIDE(2/3)
Snake Dataset
CloSpan vs. BIDE(3/3)
Pi Dataset
Conclusions
Closed sequence mining: More compact result set Significantly better efficiency BIDE: Avoids the curse of candidate maintenance Prunes search space more deeply Consumes much less memory than CloSpan in closure checking Future Work: Push constraints into the mining process
Thank you !
