bitSPADE: A Lattice-based Sequential Pattern Mining Algorithm Using Bitmap Representation

Aseervatham, Sujeevan; Osmani, Aomar; Viennet, Emmanuel

doi:10.1109/icdm.2006.28

Cited by 24 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the more general setting, Site Si has a random subset of features mi ⊂ m 4. Existence of this clock is of interest only for theoretical analysis.…”

mentioning

confidence: 99%

Consensus-based modeling using distributed feature construction with ILP

Dutta

Srinivasan

2017

Mach Learn

View full text Add to dashboard Cite

A particularly successful role for Inductive Logic Programming (ILP) is as a tool for discovering useful relational features for subsequent use in a predictive model. Conceptually, the case for using ILP to construct relational features rests on treating these features as functions, the automated discovery of which necessarily requires some form of first-order learning. Practically, there are now several reports in the literature that suggest that augmenting any existing features with ILPdiscovered relational features can substantially improve the predictive power of a model. While the approach is straightforward enough, much still needs to be done to scale it up to explore more fully the space of possible features that can be constructed by an ILP system. This is in principle, infinite and in practice, extremely large. Applications have been confined to heuristic or random selections from this space. In this paper, we address this computational difficulty by allowing features to be constructed in a distributed manner. That is, there is a network of computational units, each of which employs an ILP engine to construct some small number of features and then builds a (local) model. We then employ a consensus-based algorithm, in which neighbouring nodes share information to update local models. For a category of models (those with convex loss functions), it can be shown that the algorithm will result in all nodes converging to a consensus model. In practice, it may be slow to achieve this convergence. Nevertheless, our results on synthetic and real datasets that suggests that in relatively short time the "best" node in the network reaches a model whose predictive accuracy is comparable to that obtained using more computational effort in a non-distributed setting (the best node is identified as the one whose weights converge first).

show abstract

“…In the more general setting, Site Si has a random subset of features mi ⊂ m 4. Existence of this clock is of interest only for theoretical analysis.…”

mentioning

confidence: 99%

Consensus-based modeling using distributed feature construction with ILP

Dutta

Srinivasan

2017

Mach Learn

View full text Add to dashboard Cite

show abstract

“…Ayres et al then proposed SPAM [4] to search a lexicographic sequence tree in depth-first manner and use a vertical bitmap data layout to support simple and efficient counting process. Aseervatham et al presented bitSPADE [3] using a lattice-based bitmap representation for sequential pattern mining. In addition, there are also several works on adding constraints to find sequential patterns [13], [17], [28], [33], closed sequential patterns [11], [36], [38], maximal sequential pattern mining [22], spatiotemporal sequential pattern mining [6], sequential pattern mining on specific data domain [14], [37], sequential pattern mining on stream data [18], [24], [25], frequent episode mining [20], [23], and path traversal pattern mining [9].…”

Section: Related Workmentioning

confidence: 99%

“…The progressive sequential pattern mining deals with a progressive database, which not only adds new data to the original database but also removes obsolete data from the database. As will be surveyed in Section 2.2, the sequential pattern mining with a static database finds the sequential patterns in the database in which data do not change over time [2], [3], [4], [16], [32], [35], [41]. On the other hand, the sequential pattern mining with an incremental database corresponds to the mining process where there are new data arriving as time goes by (i.e., the sequences database is incremental) [7], [10], [21], [26], [27], [31].…”

Section: Introductionmentioning

confidence: 99%

A General Model for Sequential Pattern Mining with a Progressive Database

Huang

Tseng

et al. 2008

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Although there have been many recent studies on the mining of sequential patterns in a static database and in a database with increasing data, these works, in general, do not fully explore the effect of deleting old data from the sequences in the database. When sequential patterns are generated, the newly arriving patterns may not be identified as frequent sequential patterns due to the existence of old data and sequences. Even worse, the obsolete sequential patterns that are not frequent recently may stay in the reported results. In practice, users are usually more interested in the recent data than the old ones. To capture the dynamic nature of data addition and deletion, we propose a general model of sequential pattern mining with a progressive database while the data in the database may be static, inserted, or deleted. In addition, we present a progressive algorithm Pisa, which stands for Progressive mIning of Sequential pAtterns, to progressively discover sequential patterns in defined time period of interest (POI). The POI is a sliding window continuously advancing as the time goes by. Pisa utilizes a progressive sequential tree to efficiently maintain the latest data sequences, discover the complete set of up-to-date sequential patterns, and delete obsolete data and patterns accordingly. The height of the sequential pattern tree proposed is bounded by the length of POI, thereby effectively limiting the memory space required by Pisa that is significantly smaller than the memory needed by the alternative method, Direct Appending (DirApp). Note that the sequential pattern mining with a static database and with an incremental database are special cases of the progressive sequential pattern mining. By changing Start time and End time of the POI, Pisa can easily deal with a static database or an incremental database as well. Complexity of algorithms proposed is analyzed. The experimental results show that Pisa not only significantly outperforms the prior methods in execution time by orders of magnitude but also possesses graceful scalability.

show abstract

“…The most powerful feature of SPAM is bitmap data structure for counting sequence supports [8] . Then semi-vertical bitmap representation was presented in SPADE which was a vertical database by Sujeevan Aseervatham et al [9] . The speed of counting supports was improved by using semi-vertical bitmap representation.…”

Section: Introductionmentioning

confidence: 99%

Sliding-Window Filtering with Constraints of Compactness and Recency in Incremental Database

Ren

Tian

2008

2008 Fourth International Conference on Networked Computing and Advanced Information Management

View full text Add to dashboard Cite

In true-life the database is changed continually in many applications. Incremental mining technique has been developed to avoid rescanning database for knowledge discovery. Recent and compact constraints also are developed for frequent patterns mining. We store the database with a time-vertical bitmap representation, therefore the supports of frequent pattern and recent pattern can be computed fast. Link and bitmap are adopted, so a mass of running time can be saved during incremental mining process. Besides, to mine more efficiently in the incremental database, two concepts of recency and compactness are introduced into sliding-window filtering (denoted as SWF). In essence, an incremental database is divided into several partitions, and a filtering threshold is employed in each partition to handle candidate itemsets generation under constraints of recency and compactness. By employing SWF with constraints of compactness and recency, user satisfactory CFRpatterns (compactness, frequency and recency) can be discovered. Experimental result shows that the running time can be reduced.

show abstract

bitSPADE: A Lattice-based Sequential Pattern Mining Algorithm Using Bitmap Representation

Cited by 24 publications

References 12 publications

Consensus-based modeling using distributed feature construction with ILP

Consensus-based modeling using distributed feature construction with ILP

A General Model for Sequential Pattern Mining with a Progressive Database

Sliding-Window Filtering with Constraints of Compactness and Recency in Incremental Database

Contact Info

Product

Resources

About