Hidden tree Markov networks: Deep and wide learning for structured data

Abstract

The paper introduces the Hidden Tree Markov Network (HTN), a neuro-probabilistic hybrid fusing the representation power of generative models for trees with the incremental and discriminative learning capabilities of neural networks. We put forward a modular architecture in which multiple generative models of limited complexity are trained to learn structural feature detectors whose outputs are then combined and integrated by neural layers at a later stage. In this respect, the model is both deep, thanks to the unfolding of the generative models on the input structures, as well as wide, given the potentially large number of generative modules that can be trained in parallel. Experimental results show that the proposed approach can outperform state-of-the-art syntactic kernels as well as generative kernels built on the same probabilistic model as the HTN.

Synthesizing of models for identification of teletraffic Markov chains by artificial neural networks and decision tree method

“…Fact is the creation of the hidden tree Markov network (HTN) -a neural hybrid architecture which combines the power of generative models for trees with the incremental and discriminative learning capabilities of the neural network. In this way a greater depth of unfolding of the generative models on the input structures is guaranteed [8].…”

Deep Tree Transductions - A Short Survey

“…There it has been formalized the idea of extending recurrent models to perform a bottom-up processing of the tree, unfolding the network recursively over the tree structure so that the hidden state of the current tree node depends from that of its children. The same approach has been taken by a number of models from different paradigms, such as the probabilistic bottom-up Hidden Markov Tree model [4], the neural Tree Echo State Network [14] or the neural-probabilistic hybrid in Hidden Tree Markov Networks (HTNs) [3]. Another approach can be taken based on inverting the direction of parsing of the tree, by processing this top-down from the root to its leaves.…”

Bayesian mixtures of Hidden Tree Markov Models for structured data clustering

“…An alternative approach is that put forward in the Tree Echo State Network (TreeESN) [10] where the recursive neurons are randomly initialized according to some dynamic system stability criterion and their weights are not adjusted by the training procedure. Recently, the Hidden Tree Markov Networks (HTNs) [11] have been proposed as an hybrid approach integrating probabilistic bottomup models within a neural architecture and learning scheme.…”