Generative Kernels for Tree-Structured Data

Abstract

This paper presents a family of methods for the design of adaptive kernels for tree-structured data that exploits the summarization properties of hidden states of hidden Markov models for trees. We introduce a compact and discriminative feature space based on the concept of hidden states multisets and we discuss different approaches to estimate such hidden state encoding. We show how it can be used to build an efficient and general tree kernel based on Jaccard similarity. Furthermore, we derive an unsupervised convolutional generative kernel using a topology induced on the Markov states by a tree topographic mapping. This paper provides an extensive empirical assessment on a variety of structured data learning tasks, comparing the predictive accuracy and computational efficiency of state-of-the-art generative, adaptive, and syntactical tree kernels. The results show that the proposed generative approach has a good tradeoff between computational complexity and predictive performance, in particular when considering the soft matching introduced by the topographic mapping.

Hidden tree Markov networks: Deep and wide learning for structured data

“…The results highlight how such an approach can outperform other methods for imbuing discriminative power into generative models, e.g. through the definition of generative kernels [4]. The modular structure of the network allows to apply a wide range of performance and training optimization tricks which might prove effective when dealing with large scale problem.…”

Complex Data: Learning Trustworthily, Automatically, and with Guarantees

“…Still, learning from graphs poses many open research challenges like further improving efficiency by reservoir computing [6] or incremental approaches [5], analyzing in depth the properties related to the informative content of layers [10], studying long-term dependencies issues [11], and understanding causal factors encoded in the layers [12]. Cross contamination between different approaches is also needed by data-dependent representations kernel definition [13] via multiple kernel learning [14] or by incorporating priors provided by graph kernels in graph neural networks [15,16]. Finally, enriching these methods with trustworthiness and automatization into a unified framework with guarantees on both technical and human-relevant metrics remains an open problem.…”

Complexity vs. Performance in Granular Embedding Spaces for Graph Classification

“…Amongst the implicit embedding techniques, kernel methods emerge (Schölkopf and Smola, 2002;Shawe-Taylor and Cristianini, 2004): kernel methods exploit the so-called kernel trick (i.e., the inner product in a reproducing kernel Hilbert space) in order to measure similarity between patterns. In the literature, have been proposed several graph kernels (Kondor and Lafferty, 2002;Borgwardt and Kriegel, 2005;Vishwanathan et al, 2010;Shervashidze et al, 2011;Livi and Rizzi, 2013;Neumann et al, 2016;Ghosh et al, 2018;Bacciu et al, 2018) that, for example, consider substructures such as (random) walks, trees, paths, cycles in order to measure similarity or exploit propagation/diffusion schemes. Conversely, as explicit embedding techniques are concerned, dissimilarity spaces are one of the main approaches (Pękalska and Duin, 2005).…”