Mining constrained gradients in large databases

Abstract

Many data analysis tasks can be viewed as search or mining in a multidimensional space (MDS). In such MDSs, dimensions capture potentially important factors for given applications, and cells represent combinations of values for the factors. To systematically analyze data in MDS, an interesting notion, called "cubegrade" was recently introduced by Imielinski, et al. [IKA02], which focuses on the notable changes in measures in MDS by comparing a cell (which we refer as probe cell) with its gradient cells, namely its ancestors, descendants and siblings. We call such queries gradient analysis queries (GQs). Since an MDS can contain billions of cells, it is important to answer GQs efficiently. In this study, we focus on developing efficient methods for mining GQs constrained by certain (weakly) anti-monotone constraints. Instead of conducting an independent gradient-cell search once per probe cell, which is inefficient due to much repeated work, we propose an efficient algorithm, LiveSet-Driven. This algorithm finds all good gradient-probe cell pairs in one search pass. It utilizes measure-value analysis and dimensionmatch analysis in a set-oriented manner, to achieve bi-directional pruning between the sets of hopeful probe cells and of hopeful gradient cells. Moreover, it adopts a hyper-tree structure and an H-cubing method to compress data and to maximize sharing of computation. Our performance study shows that this algorithm is efficient and scalable. In addition to data cubes, we extend our study to another important scenario: mining constrained gradients in transactional databases where each item is associated with some measures such as price. Such transactional databases can be viewed as sparse MDSs where items represent dimensions, although they have significantly different characteristics than data cubes. We outline efficient mining methods for this problem in this paper.

Semi-Automated Exploration of Data Warehouses

“…To do so, we compare it to four methods, three of which come from the machine learning literature. Our first baseline, Exact, is similar to Claude, but we removed the approximation scheme presented in 4 -instead we compute the exact the mutual information, as in Equation 6 The method should be slower, but more accurate.…”

Survey on a Trust-Aware System for Personalized User Recommendations in Social Networks

“…Experimental comparison, using data from a real collaborative tagging system (Last.fm), against both recent tag-aware and traditional (non tag aware) item recommendation algorithms indicates significant improvements in recommendation quality. [3] Mining Constrained Gradients in Large Databases: liveset-Driven algorithm finds all good gradient-probe cell pairs in one search pass. It utilizes measure-value analysis and dimension-match analysis in a set-oriented manner, to achieve bidirectional pruning between the sets of hopeful probe cells and of hopeful gradient cells.…”

Towards intensional answers to OLAP queries for analytical sessions

“…This includes (but is not limited to) outlier detection in multidimensional data [11,19,21], cubegrade generation [10], constrained gradient analysis [6], cube mining [22,13], cube modeling and compression [2,3,12,19], and query answer approximation [2,4,16].…”