Biclustering: An application of Dual Topic Models

Abstract

Biclustering is a data mining technique that allows simultaneous clustering of two variables. A common biclustering task for categorical variables is to find 'heavy' biclusters, i.e., biclusters with high co-occurrence values. Although algorithms have been proposed to extract heavy biclusters, they provide little information about relative importance of each bicluster, as well as importance of the variables for each bicluster. To address these problems, there have been attempts to apply mixture models using information theory or Bayesian method. Although they are able to rank the biclusters and the variables for each bicluster, they do not target at extracting heavy biclusters. Furthermore, these models constrain the search for biclusters in such a way that every cell in the matrix must participate in some bicluster. We attempt to alleviate these limitations using dual topic models. First of all, we develop a generalized LDA topic model that extracts dual topics, i.e., topics in opposite directions -row-and column-topics. To obtain better topics, it applies mutual reinforcement, i.e., considering column-topics while constructing row-topics, and vice versa. Heavy biclusters, the high co-occurred relationship, are extracted using thresholds. We show that our model Dual Topic to Biclusters (DT2B) is effective in extracting heavy biclusters by experimenting over a simulated data, a text corpus (NIPS author-document) and a microarray gene expression data. Results show that biclusters extracted by DT2B are better.

Comodularity and detection of co-communities

“…Studying relationships between variables of the same type is naturally very useful; its simplest generalization is to study relationships between variables of a different type; this is * thomas.bartlett.10@ucl.ac.uk known as the coclustering problem [22][23][24][25], and is of much current interest in application areas from genomics to natural language processing [26][27][28]. The coclustering problem can also be approached nonparametrically, as is made clear by [23] and [25].…”

A study on dual topic models and bayesian topic model inference

“…Bi-clustering: An application of dual topic models. In Proceedings of the 2017 SIAM International Conference on Data Mining, Houston, Texas, USA, April 27-29, 2017., pages 453-461…”

“…We note that, in addition to latent Dirichlet allocation (LDA), RCA can be also applied to inference of other probabilistic topic models, such as dual topic model (DT2B) [90], relational topic model [22], and author-topic model [87], etc. In contrast to many other topic models, DT2B models the pair-wise dependency between two types of latent Algorithm 4 Recursively Compound Allocation for LDA…”

“…i = i + 1 9: end while 10: Estimate θ and φ using Equation 5.5. return θ, φ 5.1 Recursively Compound Allocation 95 topics [90]. In this section, we turn our attention to applying RCA to DT2B, and may shed some light on the inference procedure of such kind of topic models.…”

“…We note that although PTM includes the modeling of user-topics using the parameters ψ z , the topic assignment is constrained to be one value for an observed word, user pair. Instead, in our proposed model, users and words have its own dedicated topic assignment k i 6.2 Future Work 137 and z i respectively [90]. Figure 6.2 illustrates the graphical model for our proposed extension to PTM.…”