Familia: A Configurable Topic Modeling Framework for Industrial Text Engineering

Abstract

In the last decade, a variety of topic models have been proposed for text engineering. However, except Probabilistic Latent Semantic Analysis (PLSA) and Latent Dirichlet Allocation (LDA), most of existing topic models are seldom applied or considered in industrial scenarios. This phenomenon is caused by the fact that there are very few convenient tools to support these topic models so far. Intimidated by the demanding expertise and labor of designing and implementing parameter inference algorithms, software engineers are prone to simply resort to PLSA/LDA, without considering whether it is proper for their problem at hand or not.In this paper, we propose a configurable topic modeling framework named Familia, in order to bridge the huge gap between academic research fruits and current industrial practice. Familia supports an important line of topic models that are widely applicable in text engineering scenarios. In order to relieve burdens of software engineers without knowledge of Bayesian networks, Familia is able to conduct automatic parameter inference for a variety of topic models. Simply through changing the data organization of Familia, software engineers are able to easily explore a broad spectrum of existing topic models or even design their own topic models, and find the one that best suits the problem at hand. With its superior extendability, Familia has a novel sampling mechanism that strikes balance between effectiveness and efficiency of parameter inference. Furthermore, Familia is essentially a big topic modeling framework that supports parallel parameter inference and distributed parameter storage. After being open-sourced at Github, Familia rapidly becomes the second most popular project in topic model area, and is widely used in both industrial community and academia. The utilities and necessity of Familia are demonstrated in real-life industrial applications. Familia would significantly enlarge software engineers' arsenal of topic models and pave the way for utilizing highly customized topic models in real-life problems.

Learning to Select Context in a Hierarchical and Global Perspective for Open-domain Dialogue Generation

“…Discussion of GDS: Since GDS is only utilized during the training process, we calculate the relevance score between each contextual utterance and the ground-truth response. Af- ter applying Familia [20] over the entire conversation, the relevance scores are 0.1502, 0.1388, 0.1602, 0.1548, 0.0979, 0.1343, and 0.1638 for X 1 to X 7 , which is consistent with humans' intuition. Besides, inspired by Zhang et al [6], we randomly sample 300 context-response pairs from JDDC.…”

“…The kappa value is 0.568, which indicates the moderate consistency among different annotators. We then pick out samples that is labeled the same by at least two annotators, and then calculate the kappa value between humans' judgement and the outputs from Familia [20] on these cases. The value 0.863 reflects "Substantial agreement" between them.…”

“…The head number of multi-head attention is set to 8. The high-quality pre-trained sentence-embedding model we used is Infersent [19]/Familia [20] for Ubuntu/JDDC. These models are both pre-trained on large-scale datasets in either English or Chinese, and perform well on our datasets.…”

TopicOcean: An Ever-Increasing Topic Model with Meta-Learning

“…Despite the achieved progress, to conduct topic modeling for a new corpus, the state-of-the-art methods typically train a new topic model from scratch. Empirically a massive amount of in-domain data and considerable human labour is usually involved in obtaining a high-quality topic model [6]. Considering the great effort made in training a high-quality topic model, it is desirable to develop a framework that can take full advantages of previously welltrained topic models and transfer the knowledge in these models to the scenario of topic modeling on a new corpus, in order to save cost and improve effectiveness.…”