Mining sequential patterns with regular expression constraints

Abstract

AbstractÐDiscovering sequential patterns is an important problem in data mining with a host of application domains including medicine, telecommunications, and the World Wide Web. Conventional sequential pattern mining systems provide users with only a very restricted mechanism (based on minimum support) for specifying patterns of interest. As a consequence, the pattern mining process is typically characterized by lack of focus and users often end up paying inordinate computational costs just to be inundated with an overwhelming number of useless results. In this paper, we propose the use of Regular Expressions (REs) as a flexible constraint specification tool that enables user-controlled focus to be incorporated into the pattern mining process. We develop a family of novel algorithms (termed SPIRITÐSequential Pattern mIning with Regular expressIon consTraints) for mining frequent sequential patterns that also satisfy user-specified RE constraints. The main distinguishing factor among the proposed schemes is the degree to which the RE constraints are enforced to prune the search space of patterns during computation. Our solutions provide valuable insights into the trade-offs that arise when constraints that do not subscribe to nice properties (like antimonotonicity) are integrated into the mining process. A quantitative exploration of these trade-offs is conducted through an extensive experimental study on synthetic and real-life data sets. The experimental results clearly validate the effectiveness of our approach, showing that speedups of more than an order of magnitude are possible when RE constraints are pushed deep inside the mining process. Our experimentation with real-life data also illustrates the versatility of REs as a user-level tool for focusing on interesting patterns.

MARBLES: Mining Association Rules Buried in Long Event Sequences

“…Harms et al [12], meanwhile, experiment with closed serial episodes. In another attempt to trim the output, Garofalakis et al [9] proposed a family of algorithms called Spirit which allow the user to define regular expressions that specify the language that the discovered patterns must belong to.…”

“…Consider sequence s = abxywzcbaxywzc and parallel episode X = {a, b, c}. s contains three minimal windows of this episode, namely s [1,7], s [7,9] and s [8,14]. Note that the second minimal window overlaps with both the first and the third, so the disjointwindow frequency of X would be equal to 2.…”

“…However, if we want to maximise the sum of the inverse lengths of the windows, we cannot simply take the maximal number of non-overlapping windows and then compute this sum. In our example, we have two possible sets of non-overlapping windows, s [1,7] and s [8,14], and s [7,9] alone. In the first case, we have two windows of size 7, and the resulting sum would be 2 7 .…”

Constraint-Based Mining of Sequential Patterns over Datasets with Consecutive Repetitions

“…indicating that it appears in sequence 3 and starts at timestamp 2. The interval [4,9] means that it represents several occurrences ending from 4 to 9. The gmax value of 7 notifies that occurrences ending from 4 to 7 satisfy the maxGap constraint, while for occurrences ending strictly after timestamp 7 only the prefix of the occurrence satisfies maxGap.…”

“…Independently, the use of user-defined constraints to reduce the search space during sequential pattern extraction has been developed (e.g., [11,9,4,2]). Indeed, it has also been integrated in the occurrence list approach in the cSpade algorithm [13], resulting in one of the most efficient algorithms proposed for constraint-based mining of sequential patterns.…”