Methodological framework for data processing based on the Data Science paradigm

Pacheco, Fannia; Rangel, C.; Aguilar, José; Cerrada, Mariela; Altamiranda, Junior

doi:10.1109/clei.2014.6965184

Cited by 26 publications

(16 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This speed of data generation will continue in the coming years and is expected to increase at an exponential level, according to International Data Corporation (IDC) recent survey [1]. The most fundamental challenge is to explore the large volumes of data and extract useful knowledge for future actions through data mining and data science [2], [3].…”

Section: Introductionmentioning

confidence: 99%

“…The data science is focused in the representation, analysis, anomalies of data, and relations among variables [3], from a process with the next steps: raw data collected, data processing, clean data, exploratory data analysis, models and algorithms, construction of reports, and build data product [6]. The data science process flowchart is shown in Fig.…”

Section: Data Sciencementioning

confidence: 99%

“…[6]. Although the data mining methodologies and data sciences defined the steps for data treatment, these not tackle the issues in data quality clearly, leaving out relevant activities [3]. It has been agreed that poor data quality in data mining, machine learning and data science will impact the quality of results of analyses and that it will therefore impact on decisions made on the basis of these results.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Conceptual Framework for Data Quality in Knowledge Discovery Tasks (FDQ-KDT): A Proposal

Corrales¹,

Ledezma²,

Corrales³

2015

JCP

View full text Add to dashboard Cite

Large Volume of Data is growing because the organizations are continuously capturing the collective amount of data for better decision-making process. The most fundamental challenge is to explore the large volumes of data and extract useful knowledge for future actions through data mining and data science methodologies. Nevertheless these not tackle the issues in data quality clearly, leaving out relevant activities. We proposed a conceptual framework for data quality in knowledge discovery tasks based on CRISP-DM, SEMMA and Data Science, considering the issues of ESE Taxonomy.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Data Sciencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Conceptual Framework for Data Quality in Knowledge Discovery Tasks (FDQ-KDT): A Proposal

Corrales¹,

Ledezma²,

Corrales³

2015

JCP

View full text Add to dashboard Cite

show abstract

“…Recently, the IT divisions of enterprises are centered on taking advantage of the significant amount of data to extract useful knowledge and supporting decision-making [3,4]. These benefits facilitate the growth of organizational locations, strategies, and customers.…”

Section: Introductionmentioning

confidence: 99%

From Theory to Practice: A Data Quality Framework for Classification Tasks

2018

View full text Add to dashboard Cite

Abstract:The data preprocessing is an essential step in knowledge discovery projects. The experts affirm that preprocessing tasks take between 50% to 70% of the total time of the knowledge discovery process. In this sense, several authors consider the data cleaning as one of the most cumbersome and critical tasks. Failure to provide high data quality in the preprocessing stage will significantly reduce the accuracy of any data analytic project. In this paper, we propose a framework to address the data quality issues in classification tasks DQF4CT. Our approach is composed of: (i) a conceptual framework to provide the user guidance on how to deal with data problems in classification tasks; and (ii) an ontology that represents the knowledge in data cleaning and suggests the proper data cleaning approaches. We presented two case studies through real datasets: physical activity monitoring (PAM) and occupancy detection of an office room (OD). With the aim of evaluating our proposal, the cleaned datasets by DQF4CT were used to train the same algorithms used in classification tasks by the authors of PAM and OD. Additionally, we evaluated DQF4CT through datasets of the Repository of Machine Learning Databases of the University of California, Irvine (UCI). In addition, 84% of the results achieved by the models of the datasets cleaned by DQF4CT are better than the models of the datasets authors.

show abstract

“…The most fundamental challenge is to explore the large volumes of data and extract useful knowledge for future actions through knowledge discovery tasks as classification, clustering, etc. [3][4], however many data suffer from a lack of quality. It has been agreed that poor data quality will impact the quality of results of analyses in knowledge discovery tasks and that it will therefore impact on decisions made on the basis of these results [5][6].…”

Section: Introductionmentioning

confidence: 99%

A systematic review of data quality issues in knowledge discovery tasks

Corrales

Ledezma

Corrales

2016

Rev. ing. univ. Medellín

View full text Add to dashboard Cite

Large volume of data is growing because organizations are continuously capturing the collective amount of data for a better decision-making process. The most fundamental challenge is to explore the large volumes of data and extract useful knowledge for future actions through knowledge discovery tasks, nevertheless many data has poor quality. We presented a systematic review of the data quality issues in knowledge discovery tasks and a case study applied to agricultural disease named coffee rust. ResumenHay un gran crecimiento en el volumen de datos porque las organizaciones capturan permanentemente la cantidad colectiva de datos para lograr un mejor proceso de toma de decisiones. El desafío mas fundamental es la exploración de los grandes volúmenes de datos y la extracción de conocimiento útil para futuras acciones por medio de tareas para el descubrimiento del conocimiento; sin embargo, muchos datos presentan mala calidad. Presentamos una revisión sistemática de los asuntos de calidad de datos en las áreas del descubrimiento de conocimiento y un estudio de caso aplicado a la enfermedad agrícola conocida como la roya del café.

show abstract

Methodological framework for data processing based on the Data Science paradigm

Cited by 26 publications

References 5 publications

A Conceptual Framework for Data Quality in Knowledge Discovery Tasks (FDQ-KDT): A Proposal

A Conceptual Framework for Data Quality in Knowledge Discovery Tasks (FDQ-KDT): A Proposal

From Theory to Practice: A Data Quality Framework for Classification Tasks

A systematic review of data quality issues in knowledge discovery tasks

Contact Info

Product

Resources

About