e-NSP: Efficient negative sequential pattern mining

Abstract

As an important tool for behavior informatics, negative sequential patterns (NSP) (such as missing medical treatments) are critical and sometimes much more informative than positive sequential patterns (PSP) (e.g. using a medical service) in many intelligent systems and applications such as intelligent transport systems, healthcare and risk management, as they often involve non-occurring but interesting behaviors. However, discovering NSP is much more difficult than identifying PSP due to the significant problem complexity caused by non-occurring elements, high computational cost and huge search space in calculating negative sequential candidates (NSC). So far, the problem has not been formalized well, and very few approaches have been proposed to mine for specific types of NSP, which rely on database re-scans after identifying PSP in order to calculate the NSC supports. This has been shown to be very inefficient or even impractical, since the NSC search space is usually huge. This paper proposes a very innovative and efficient theoretical framework: set theory-based NSP mining (ST-NSP), and a corresponding algorithm, e-NSP, to efficiently identify NSP by involving only the identified PSP, without re-scanning the database. Accordingly, negative containment is first defined to determine whether a data sequence contains a negative sequence based on set theory. Second, an efficient approach is proposed to convert the negative containment problem to a positive containment problem. The NSC supports are then calculated based only on the corresponding PSP. This not only avoids the need for additional database scans, but also enables the use of existing PSP mining algorithms to mine for NSP. Finally, a simple but efficient strategy is proposed to generate NSC. Theoretical analyses show that e-NSP performs particularly well on datasets with a small number of elements in a sequence, a large number of itemsets and low minimum support. e-NSP is compared with two currently available NSP mining algorithms via intensive experiments on three synthetic and six real-life datasets from aspects including data characteristics, computational costs and scalability. e-NSP is tens to thousands of times faster than baseline approaches, and offers a sound and effective approach for efficient mining of NSP in large scale datasets by directly using existing PSP mining algorithms.

Mining negative sequential patterns from infrequent positive sequences with 2-level multiple minimum supports

“…The method in [19] only defines three forms of NSP, i.e., (¬A, B), (A, ¬B) and (¬A, ¬B) and it requires A ∩ B = ∅. This is a general requirement in negative association rule mining, but it is very strict in NSP mining [1].…”

“…The number of NSC is often large, but many of them are meaningless. In this paper, we use two constraints to limit the number of NSC [1].…”

“…The definition of negative containment is very important to mine NSP because it affects the speed to calculate the NSC's supports. We use the same definition as e-NSP [1]. Before introducing the definition of negative containment, we first give an important related definition Positive Partner.…”

“…Negative sequential patterns (NSP) considering both occurring items and non-occurring items (also called positive and negative items) play a very important role in many intelligent systems and applications [1] [2][3] [4][5] [6]. For instance, nsp =< xy¬zF > is a negative sequential patterns, where x, y and z stand for drug codes, and F stands for disease status.…”

Mining negative sequential patterns from frequent and infrequent sequences based on multiple level minimum supports

“…How to understand behaviors, especially the non-occurring behaviors (NOB) is very important in behavioral research [1] [2]. Mining negative sequential patterns (NSP) is one of few methods that can well understand NOB [3]. Different from only considering occurring (positive) behaviors in positive sequential patterns (PSP) [4], NSP take into account both non-occurring (negative) and occurring behaviors.…”