Hard Data Analytics Problems Make for Better Data Analysis Algorithms: Bioinformatics as an Example

Abstract

Data mining and knowledge discovery techniques have greatly progressed in the last decade. They are now able to handle larger and larger datasets, process heterogeneous information, integrate complex metadata, and extract and visualize new knowledge. Often these advances were driven by new challenges arising from real-world domains, with biology and biotechnology a prime source of diverse and hard (e.g., high volume, high throughput, high variety, and high noise) data analytics problems. The aim of this article is to show the broad spectrum of data mining tasks and challenges present in biological data, and how these challenges have driven us over the years to design new data mining and knowledge discovery procedures for biodata. This is illustrated with the help of two kinds of case studies. The first kind is focused on the field of protein structure prediction, where we have contributed in several areas: by designing, through regression, functions that can distinguish between good and bad models of a protein's predicted structure; by creating new measures to characterize aspects of a protein's structure associated with individual positions in a protein's sequence, measures containing information that might be useful for protein structure prediction; and by creating accurate estimators of these structural aspects. The second kind of case study is focused on omics data analytics, a class of biological data characterized for having extremely high dimensionalities. Our methods were able not only to generate very accurate classification models, but also to discover new biological knowledge that was later ratified by experimentalists. Finally, we describe several strategies to tightly integrate knowledge extraction and data mining in order to create a new class of biodata mining algorithms that can natively embrace the complexity of biological data, efficiently generate accurate information in the form of classification/regression models, and extract valuable new knowledge. Thus, a complete data-to-information-to-knowledge pipeline is presented.

Data Science and Plant Metabolomics

“…A lack of domain knowledge can lead to inaccurate or inappropriate models or result in misinterpreted results. Data science requires a combination of technical and domain knowledge, and a lack of the latter can significantly impact the accuracy of models and analyses [ 170 ].…”

Establishing a database for sickle cell disease patient mapping and survival tracking: The sickle pan-african research consortium Nigeria example

“…Applications of these techniques have been discussed in the literature (Hand et al, 2001;Lei and Karniadakis, 2013;Solanki, 2014;Khalaf et al, 2016). The use of Pattern Recognition techniques in medicine have also been studied (Benamrane et al, 2005;Bacardit et al, 2014). An interesting case of the application of text and data mining in SCD research is the development of an information exploration system, Dragon Exploration System for SCD (DESSCD), which aims to promote the easy exploration of SCD related data.…”

An Analytical Research on the Big Data Tools and Techniques

“…Apart from that the diversification of information is another key issue of massive data analytics. Considering the constantly advancing technology surrounding datasets, data mining tasks have incredibly developed [6]. Moreover, the selection and reduction of data are among the fundamental tasks that are inhibited in the process of dealing with massive sets of data.…”