MEFASD-BD: Multi-objective evolutionary fuzzy algorithm for subgroup discovery in big data environments - A MapReduce solution

Pulgar-Rubio, F.; Rivas, Antonio Jesús Rivera; Pérez-Godoy, María Dolores; González, Pedro; Carmona, C. J.; Jesus, María José del

doi:10.1016/j.knosys.2016.08.021

Cited by 35 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, those algorithms that optimize the interpretability of the model are not able to achieve state-of-the-art results in terms of accuracy [17]. There are also other contributions that, from our point of view, do not consider enough datasets to assess these aspects in Big Data environments [13], [18], [20].…”

Section: Related Workmentioning

confidence: 94%

CFM-BD: A Distributed Rule Induction Algorithm for Building Compact Fuzzy Models in Big Data Classification Problems

Elkano

Sanz

Barrenechea

et al. 2020

IEEE Trans. Fuzzy Syst.

View full text Add to dashboard Cite

Interpretability has always been a major concern for fuzzy rule-based classifiers. The usage of human-readable models allows them to explain the reasoning behind their predictions and decisions. However, when it comes to Big Data classification problems, fuzzy rule-based classifiers have not been able to maintain the good trade-off between accuracy and interpretability that has characterized these techniques in non-Big Data environments. The most accurate methods build too complex models composed of a large number of rules and fuzzy sets, while those approaches focusing on interpretability do not provide state-of-the-art discrimination capabilities. In this paper, we propose a new distributed learning algorithm named CFM-BD to construct accurate and compact fuzzy rulebased classification systems for Big Data. This method has been specifically designed from scratch for Big Data problems and does not adapt or extend any existing algorithm. The proposed learning process consists of three stages: 1) pre-processing based on the probability integral transform theorem; 2) rule induction inspired by CHI-BD and Apriori algorithms; 3) rule selection by means of a global evolutionary optimization. We conducted a complete empirical study to test the performance of our approach in terms of accuracy, complexity, and runtime. The results obtained were compared and contrasted with four state-ofthe-art fuzzy classifiers for Big Data (FBDT, FMDT, Chi-Spark-RS, and CHI-BD). According to this study, CFM-BD is able to provide competitive discrimination capabilities using significantly simpler models composed of a few rules of less than 3 antecedents, employing 5 linguistic labels for all variables.

show abstract

Section: Related Workmentioning

confidence: 94%

CFM-BD: A Distributed Rule Induction Algorithm for Building Compact Fuzzy Models in Big Data Classification Problems

Elkano

Sanz

Barrenechea

et al. 2020

IEEE Trans. Fuzzy Syst.

View full text Add to dashboard Cite

show abstract

“… Pulgar-Rubio et al (2017) introduced a novel method, named MEFASD-BD, for subgroup discovery. It is the first big data approximation in evolutionary fuzzy systems for subgroup discovery.…”

Section: Background and Related Workmentioning

confidence: 99%

Artificial intelligence approaches and mechanisms for big data analytics: a systematic study

Rahmani

Azhir

Ali

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

Recent advances in sensor networks and the Internet of Things (IoT) technologies have led to the gathering of an enormous scale of data. The exploration of such huge quantities of data needs more efficient methods with high analysis accuracy. Artificial Intelligence (AI) techniques such as machine learning and evolutionary algorithms able to provide more precise, faster, and scalable outcomes in big data analytics. Despite this interest, as far as we are aware there is not any complete survey of various artificial intelligence techniques for big data analytics. The present survey aims to study the research done on big data analytics using artificial intelligence techniques. The authors select related research papers using the Systematic Literature Review (SLR) method. Four groups are considered to investigate these mechanisms which are machine learning, knowledge-based and reasoning methods, decision-making algorithms, and search methods and optimization theory. A number of articles are investigated within each category. Furthermore, this survey denotes the strengths and weaknesses of the selected AI-driven big data analytics techniques and discusses the related parameters, comparing them in terms of scalability, efficiency, precision, and privacy. Furthermore, a number of important areas are provided to enhance the big data analytics mechanisms in the future.

show abstract

“…HDFS is a decentralized storage system which provides reliable, scalable and faults tolerant data storage (Jena, Gourisaria, Rautaray, & Pandey, 2017). It works with MapReduce computing framework based on the requests and queries (Matsuzaki, 2017;Park, Min, & Shim, 2017;Pulgar-Rubio, Rivera-Rivas, Prez-Godoy, Gonzlez, Carmona, & del Jesus, 2017;Sha, Liang, Jiang, Chen, & Zhuge, 2017, pp. 585-592).…”

Section: Hadoop Distributed File System (Hdfs)mentioning

confidence: 99%

An implementation of real-time air quality and influenza-like illness data storage and processing platform

Yang

Chen

Tsan

et al. 2019

Computers in Human Behavior

View full text Add to dashboard Cite

A B S T R A C TRecently, air pollution has become the primary concern in Taiwan as it significantly affected people's health. Some air pollution monitoring, analysis, and prediction systems were proposed to solve the problem. However, there is very little research to see whether the air quality is associated with the Influenza-Like Illness (ILI) disease or not. In this study, a system is needed, in which the air quality data and the influenza-like illness data can be analyzed together to determine their associations accurately and effectively. In this work, a novel integrated platform was implemented by building a cluster environment based on Hadoop, Spark and a visualization environment with ELK Stack as well as a backup storage system based on Ceph object storage architecture. Also, Sqoop and Alluxio were used to solve the inefficiency problem in processing vast amounts of data. The experimental results showed the visualization of air quality and influenza-like illness data collected from 2016 to 2017 in Taichung, Taiwan. Besides, the association analyses and discussion between air quality and influenzalike illness were also presented.

show abstract

MEFASD-BD: Multi-objective evolutionary fuzzy algorithm for subgroup discovery in big data environments - A MapReduce solution

Cited by 35 publications

References 34 publications

CFM-BD: A Distributed Rule Induction Algorithm for Building Compact Fuzzy Models in Big Data Classification Problems

CFM-BD: A Distributed Rule Induction Algorithm for Building Compact Fuzzy Models in Big Data Classification Problems

Artificial intelligence approaches and mechanisms for big data analytics: a systematic study

An implementation of real-time air quality and influenza-like illness data storage and processing platform

Contact Info

Product

Resources

About