Online Association Rule Mining over Fast Data

The virtual machine (VM) allocation problem in cloud computing has been widely studied in recent years, and many algorithms have been proposed in the literature. Most of them have been successfully applied to batch processing models such as MapReduce; however, none of them can be applied to streaming workflow well because of the following weaknesses: 1) failure to capture the characteristics of tasks in streaming workflow for the short life cycle of data streams; 2) most algorithms are based on the assumptions that the price of VMs and traffic among data centers (DCs) are static and fixed. In this paper, we propose a streaming workflow allocation algorithm that takes into consideration the characteristics of streaming work and the price diversity among geo-distributed DCs, to further achieve the goal of cost minimization for streaming big data processing. First, we construct an extended streaming workflow graph (ESWG) based on the task semantics of streaming workflow and the price diversity of geo-distributed DCs, and the streaming workflow allocation problem is formulated into mixed integer linear programming based on the ESWG. Second, we propose two heuristic algorithms to reduce the computational space based on task combination and DC combination in order to meet the strict latency requirement. Finally, our experimental results demonstrate significant performance gains with lower total cost and execution time.

show abstract

“…The size and type of w ind ow sequ ence is beyond the scop e of this w ork, please refer to [16] [17] for d etail.…”

Section: Efinition 1 (Window Sequence)mentioning

confidence: 99%

Cost-Aware Streaming Workflow Allocation on Geo-Distributed Data Centers

Chen

Paik

2016

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…number of records, q = 2 d − 1 is the total item set count, d is the unique item count, and w is the maximum record length [34]. On the other hand, the time complexity of the FP-growth algorithm is O(n.d 2 ) , where n is the number of records and d is the number of unique items [35].…”

Section: Mtarm: Multitask Association Rule Minermentioning

confidence: 99%

Multitask-based association rule mining

Taşer¹,

Birant²,

Birant³

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Recently, there has been a growing interest in association rule mining (ARM) in various fields. However, standard ARM algorithms fail to discover rules for multitask problems as they do not consider task-oriented investigation and, therefore, they ignore the correlation among the tasks. Considering this situation, this paper proposes a novel algorithm, named multitask association rule miner (MTARM), that tends to jointly discover rules by considering multiple tasks. This paper also introduces two novel concepts: single-task rule and multiple-task rule. In the first phase of the proposed approach, highly frequent local rules (single-task rules) are explored for each task separately and then these local rules are combined to produce the global result (multitask rules) using a majority voting mechanism. Experiments were conducted on four different real-world multitask learning datasets. The experimental results indicated that the proposed MTARM approach discovers more information than that of traditional ARM algorithms by jointly considering the relationships among multiple tasks. This work is licensed under a Creative Commons Attribution 4.0 International License. 933 YILDIRIM TAŞER et al./Turk J Elec Eng & Comp Scisubset of them, are familiar to one mutual rule set with a small difference. The proposed approach can discover common association rules by responding to different tasks and by applying an algorithm to consider all the tasks collectively. Accordingly, the rule X → Y can have high support and confidence in a task. A rule may not be frequent in the entire dataset; however, it may be frequent in several specific tasks. Therefore, finding task-based frequent rules in an effective way is important. This paper proposes a novel algorithm, named multitask association rule miner (MTARM), that tends to discover rules by jointly considering multiple tasks. The proposed algorithm mines datasets that include task information. The MTARM approach consists of two phases. In the first phase, the algorithm discovers local frequent rules (single-task rules) from the data of each task separately, and in the second phase, it combines these local rules to produce the global result (multitask rules). The discovered multitask rules can be used to deploy information systems that support the execution of their associated task or to improve decision-making in applications.The main contributions and novelty of this study are threefold: (i) this is the first study that applies ARM to multitask problems; (ii) it proposes the MTARM approach, which attempts to find intertask rules based solely on the task identities and the observed data for each task; and, finally (iii) it also introduces two novel concepts: single-task rule and multitask rule. This paper also provides a brief survey of ARM as it is an active research area of data mining. In addition, it presents the results of experimental studies conducted on four different real-world datasets to demonstrate the capability of the proposed algorithm.The rest of the paper is organized...

show abstract

“…Yew-Kwong et al (2001) proposed a fast online dynamic association rule mining algorithm for dynamic e-commerce dataset and outperformed classical Apriori algorithm. In the literature, several online association rule mining techniques were proposed (Hidber 1999, Ölmezoğulları andAri 2013). These techniques mainly focus on mining streaming data, allow users to adjust the threshold values and generally do not deal with emerging and vanishing characteristics of the rules.…”

Section: Emerging and Vanishing Association Pattern Mining In Hydroclmentioning

confidence: 99%

Untitled

Çelik

Dokuz²,

Dadaşer-Çelik

2018

KSEJ

View full text Add to dashboard Cite

mining algorithms do not consider how the strengths of patterns change over time. Emerging and vanishing association patterns are association patterns, whose frequencies get stronger and weaker over time, respectively. These patterns provide more information about how the strengths of patterns change in the temporal dimension. Discovering emerging and vanishing patterns is beneficial for time-evolving financial analyses, identification of temporal associations among various parameters related to diseases, or the relationships among hydroclimatic parameters under changing climatic conditions. In this study, we focus on discovering emerging and vanishing association patterns in hydroclimatic datasets.

show abstract

Online Association Rule Mining over Fast Data

Cited by 25 publications

References 12 publications

Cost-Aware Streaming Workflow Allocation on Geo-Distributed Data Centers

Cost-Aware Streaming Workflow Allocation on Geo-Distributed Data Centers

Multitask-based association rule mining

Untitled

Contact Info

Product

Resources

About