Large-Scale Frequent Episode Mining from Complex Event Sequences with Hierarchies

Abstract

Frequent Episode Mining (FEM), which aims at mining frequent sub-sequences from a single long event sequence, is one of the essential building blocks for the sequence mining research field. Existing studies about FEM suffer from unsatisfied scalability when faced with complex sequences as it is an NP-complete problem for testing whether an episode occurs in a sequence. In this article, we propose a scalable, distributed framework to support FEM on “big” event sequences. As a rule of thumb, “big” illustrates an event sequence is either very long or with masses of simultaneous events. Meanwhile, the events in this article are arranged in a predefined hierarchy. It derives some abstractive events that can form episodes that may not directly appear in the input sequence. Specifically, we devise an event-centered and hierarchy-aware partitioning strategy to allocate events from different levels of the hierarchy into local processes. We then present an efficient special-purpose algorithm to improve the local mining performance. We also extend our framework to support maximal and closed episode mining in the context of event hierarchy, and to the best of our knowledge, we are the first attempt to define and discover hierarchy-aware maximal and closed episodes. We implement the proposed framework on Apache Spark and conduct experiments on both synthetic and real-world datasets. Experimental results demonstrate the efficiency and scalability of the proposed approach and show that we can find practical patterns when taking event hierarchies into account.

Learn Smart with Less: Building Better Online Decision Trees with Fewer Training Examples

“…The success of Mem-ES is particularly exciting, since the idea of applying resampling techniques like non-parametric bootstrap on data streams had always been considered very difficult in the past due to its cumbersome nature and high time and space complexity. But, this first of its kind realization and experimental validation of approximate bootstrapping can invite further research investigations in other stream mining algorithms used for frequent pattern mining [Das and Zaniolo, 2016], episode mining [Ao et al, 2019;2018], complex pattern detection and ranking [Gu et al, 2016], where bootstrapping can be useful.…”

CrowdMed-II: a blockchain-based framework for efficient consent management in health data sharing

“…Lastly, the limitations of blockchain technology remain a potential obstacle to our framework. With continuous improvement of blockchain technology, we hope that the blockchain system and network can overcome the current problems of throughput and latency, such as those dealing with streaming data [ 4 , 13 , 16 ]. In emergency situations, health data is often urgently needed.…”

A Scalable Analytical Framework for Complex Event Episode Mining With Various Domains Applications

“…Recently Xiang et al proposed a big data algorithm for episode mining called LA-FRMH [46], but this algorithm did not handle the case of simultaneous events. Oualid et al proposed an algorithm called NONEPI [47] for episode rule mining using the concept of non-overlapping frequency.…”