Diverse Dimension Decomposition of an Itemset Space

Abstract

We introduce the problem of diverse dimension decomposition in transactional databases. A dimension is a set of mutually-exclusive itemsets, and our problem is to find a decomposition of the itemset space into dimensions, which are orthogonal to each other, and that provide high coverage of the input database. The mining framework we propose effectively represents a dimensionality-reducing transformation from the space of all items to the space of orthogonal dimensions. Our approach relies on information-theoretic concepts, and we are able to formulate the dimension-finding problem with a single objective function that simultaneously captures constraints on coverage, exclusivity and orthogonality. We describe an efficient greedy method for finding diverse dimensions from transactional databases. The experimental evaluation of the proposed approach using two real datasets, flickr and del.icio.us, demonstrates the effectiveness of our solution. Although we are motivated by the applications in the collaborative tagging domain, we believe that the mining task we introduce in this paper is general enough to be useful in other application domains.

dbTrento

“…The information theoretic mining framework we have proposed together with Yahoo! Research can be interpreted as a dimensionalityreducing transformation from the space of all tags to the space of orthogonal dimensions [53,54].…”