Data Analysis for Information Discovery

Abstract

Artificial intelligence applications are becoming increasingly popular and are producing better results in many areas of research. The quality of the results depends on the quantity of data and its information content. In recent years, the amount of data available has increased significantly, but this does not always mean more information and therefore better results. The aim of this work is to evaluate the effects of a new data preprocessing method for machine learning. This method was designed for sparce matrix approximation, and it is called semi-pivoted QR approximation (SPQR). To best of our knowledge, it has never been applied to data preprocessing in machine learning algorithms. This method works as a feature selection algorithm, and in this work, an evaluation of its effects on the performance of an unsupervised clustering algorithm is proposed. The obtained results are compared to those obtained using, as preprocessing algorithm, principal component analysis (PCA). These two methods have been applied to various publicly available datasets. The obtained results show that the SPQR algorithm can achieve results comparable to those obtained using PCA without introducing any transformation of the original dataset.

Smart City Based on Open Data: A Survey

“…To solve these two problems, there are two popular techniques that reduce data without losing enough information: PCA (principal component analysis) [40], [86], [90] and LDA (linear discriminant analysis) [40], [108]. However, Amato and Lecce [109] proposed a new method called semi-pivoted QR approximation, to efficiently reduce a data set, and they demonstrated its performance compared to PCA.…”