The Infinite-Order Conditional Random Field Model for Sequential Data Modeling

Abstract

Abstract-Sequential data labeling is a fundamental task in machine learning applications, with speech and natural language processing, activity recognition in video sequences, and biomedical data analysis being characteristic such examples, to name just a few. The conditional random field (CRF), a log-linear model representing the conditional distribution of the observation labels, is one of the most successful approaches for sequential data labeling and classification, and has lately received significant attention in machine learning, as it achieves superb prediction performance in a variety of scenarios. Nevertheless, existing CRF formulations can capture only one-or few-timestep interactions, and neglect higher-order dependencies, which are potentially useful in many real-life sequential data modeling applications. To resolve these issues, in this paper we introduce a novel CRF formulation, based on the postulation of an energy function which entails infinitely-long time-dependencies between the modeled data. Building blocks of our novel approach are: (i) the sequence memoizer, a recently proposed nonparametric Bayesian approach for modeling label sequences with infinitely-long time dependencies; and (b) a mean-field-like approximation of the model marginal likelihood, which allows for the derivation of computationally efficient inference algorithms for our model. The efficacy of the so-obtained infinite-order CRF (CRF ∞ ) model is experimentally demonstrated.Index Terms-Conditional random field, sequential data, sequence memoizer, mean-field principle.

Semi-Supervised Morphosyntactic Classification of Old Icelandic

“…Conditional Random Fields [27], [32] is an undirected graphical model often used for tagging sequential data. A CRF assigns probabilities to output nodes based on the values of input nodes.…”

Determining the prevalence of cannabis, tobacco, and vaping device mentions in online communities using natural language processing

“…Second, we used a Conditional Random Field (CRF) algorithm (Lafferty et al, 2001). CRFs are a popular method for sequence-based labeling (Chatzis and Demiris, 2013), and have been widely used for language processing applications. We used the CRFsuite (Okazaki, 2016) implementation of CRFs for this work.…”

Data-Driven Modeling and Prediction of Complex Spatio-Temporal Dynamics in Excitable Media

“…Such systems are well-suited for feature extraction [28], but their linear structure does not allow a direct modeling of temporal changes or the possibility to process a sequence of data. To enable temporal modeling, we employ linear-chain conditional random fields [31] that will be introduced in the next section.…”