Additive regularization of topic models

Semisupervised and unsupervised systems provide operators with invaluable support and can tremendously reduce the operators' load. In the light of the necessity to process large volumes of video data and provide autonomous decisions, this paper proposes new learning algorithms for activity analysis in video. The activities and behaviors are described by a dynamic topic model. Two novel learning algorithms based on the expectation maximization approach and variational Bayes inference are proposed. Theoretical derivations of the posterior estimates of model parameters are given. The designed learning algorithms are compared with the Gibbs sampling inference scheme introduced earlier in the literature. A detailed comparison of the learning algorithms is presented on real video data. We also propose an anomaly localization procedure, elegantly embedded in the topic modeling framework. It is shown that the developed learning algorithms can achieve 95% success rate. The proposed framework can be applied to a number of areas, including transportation systems, security, and surveillance.

show abstract

“…where (a) + def = max(a, 0) [27]; β x , α y and γ z are the elements of the hyperparameter vectors β, α and γ, respectively; and…”

Section: A Learning: Em-algorithm Schemementioning

confidence: 99%

“…The corresponding Dirichlet distributions with all used parameters are presented in Figure 4. Note that parameter learning is an ill-posed problem in topic modeling [27]. This means there is no unique solution for parameter estimates.…”

Section: A Setupmentioning

confidence: 99%

Learning Methods for Dynamic Topic Modeling in Automated Behavior Analysis

Isupova

Kuzin

Mihaylova

2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…A number of papers is devoted to the sparseness of the target distributions, e.g., [29]. In [156] different forms of regularisation are presented to overcome the problem of non-uniqueness of the matrix factorisation in topic modeling.…”

Section: Extensions Of Conventional Modelsmentioning

confidence: 99%

“…where (a) + def = max(a, 0) [156]; η w , α z , κ b , and υ b are the elements of the hyperparameter vectors η, α, κ and υ, respectively, and:…”

Section: Expectation-maximisation Learningmentioning

confidence: 99%

Machine Learning Methods for Behaviour Analysis and Anomaly Detection in Video

Isupova¹

2018

Springer Theses

View full text Add to dashboard Cite

Behaviour analysis and anomaly detection are key components of intelligent vision systems. Anomaly detection can be considered from two perspectives: abnormal events can be defined as those that violate typical activities or as a sudden change in behaviour. Topic modeling and change point detection methodologies, respectively, are employed to achieve these objectives.The thesis starts with development of novel learning algorithms for a dynamic topic model. Topics extracted by the learning algorithms represent typical activities happening within an observed scene. These typical activities are used for likelihood computation. The likelihood serves as a normality measure in anomaly detection decision making. A novel anomaly localisation procedure is proposed. In the considered dynamic topic model a number of topics, i.e., typical activities, should be specified in advance. A novel dynamic nonparametric hierarchical Dirichlet process topic model is then developed where the number of topics is determined from data. Conventional posterior inference algorithms require processing of the whole data through several passes. It is computationally intractable for massive or sequential data. Therefore, batch and online inference algorithms for the proposed model are developed. A novel normality measure is derived for decision making in anomaly detection. The latter part of the thesis considers behaviour analysis and anomaly detection within the change point detection methodology. A novel general framework for change point detection is introduced. Gaussian process time series data is considered and a change is defined as an alteration in hyperparameters of the Gaussian process prior. The problem is formulated in the context of statistical hypothesis testing and several tests suitable both for offline and online data processing and multiple change point detection are proposed. Theoretical properties of the proposed tests are derived based on the distribution of the test statistics. i ACKNOWLEDGMENTS

show abstract

“…First, there is a natural way to learn document embeddings. Second, additive regularization of topic models [43] can be used to meet further requirements. In this paper we employ…”

Section: Additive Regularization and Embeddings For Multiple Modalitiesmentioning

confidence: 99%

Interpretable Probabilistic Embeddings: Bridging the Gap Between Topic Models and Neural Networks

Potapenko¹,

Popov²,

Vorontsov³

2017

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

We consider probabilistic topic models and more recent word embedding techniques from a perspective of learning hidden semantic representations. Inspired by a striking similarity of the two approaches, we merge them and learn probabilistic embeddings with online EM-algorithm on word co-occurrence data. The resulting embeddings perform on par with Skip-Gram Negative Sampling (SGNS) on word similarity tasks and benefit in the interpretability of the components. Next, we learn probabilistic document embeddings that outperform para-graph2vec on a document similarity task and require less memory and time for training. Finally, we employ multimodal Additive Regularization of Topic Models (ARTM) to obtain a high sparsity and learn embeddings for other modalities, such as timestamps and categories. We observe further improvement of word similarity performance and meaningful inter-modality similarities.

show abstract

Additive regularization of topic models

Cited by 91 publications

References 24 publications

Learning Methods for Dynamic Topic Modeling in Automated Behavior Analysis

Learning Methods for Dynamic Topic Modeling in Automated Behavior Analysis

Machine Learning Methods for Behaviour Analysis and Anomaly Detection in Video

Interpretable Probabilistic Embeddings: Bridging the Gap Between Topic Models and Neural Networks

Contact Info

Product

Resources

About