Visualization and Integration of Databases Using Self-Organizing Map

Abstract

Abstract-With the growing computer networks, accessible data is becoming increasingly distributed. Understanding and integrating remote and unfamiliar data sources are important data management issues. In this paper, we propose to utilize self-organizing maps (SOM) clustering to aid with the visualization of similar columns, and integration of relational database tables and attributes based on the content. In order to accommodate heterogeneous data types found in relational databases, we extended the TFIDF measure to handle, in addition to text, numerical attribute types for coincident meaning extraction. We present a SOM clustering based visualization algorithm allowing the user to browse the heterogeneously typed database attributes and discover semantically similar clusters. Additionally, we propose a new algorithm Common Item Based Classifier (CIBC) to smoothen the homogeneity of the clusters obtained by SOM. The discovered semantic clusters can significantly aid in manual or automated constructions of data integrity constraints in data cleaning or schema mappings in data integration.

Clustering Relational Database Entities Using K-means

“…The user of unsupervised clustering methods does not need to specify the expected number of clusters, while he must specify the expected number of clusters with the supervised clustering methods. It has already been demonstrated that HDM-UV works well with the unsupervised clustering method SOM [1]. We are aiming to demonstrate that the HDM-UV is applicable to algorithms other than SOM such as K-means.…”

“…It appears that when two data types, such as numerical and textual, are simultaneously processed by HDM, the use of unified vectorization (UV) leads to better convergent semantic clustering results [1]. In spite of good results, the development and the use of similar data weighting measures to represent these heterogeneous data types in a unified VSM matrix improves the clustering results [2].…”

“…This interest seems to be proportional to the heterogeneity of the data, i.e., the more heterogeneous the data types are, and the greater the interest there is in automatically processing it [1]. This is likely due to the availability and abundance of such data.…”

“…Heterogeneous Data Mining by Unified Vectorization (HDM-UV) appeared to be successful when applied to textual and numerical data using Self Organizing-Map (SOM) for data integration applications [1]. Furthermore, when the heterogeneous data types are converted into algebraic models using similar weighting measures, the clustering results are enhanced when SOM is used [2].…”

Efficiency and Effectiveness of Clustering Algorithms for High Dimensional Data

“…Where N j is elements of cluster j and N i is the number of elements of class i for class i and cluster j, N ij is the numbers of elements of class i in cluster j [12].…”