The inner and outer approaches to the design of recursive neural architectures

Abstract

Feedforward neural network architectures work well for numerical data of fixed size, such as images. For variable size, structured data, such as sequences, d dimensional grids, trees, and other graphs, recursive architectures must be used. We distinguish two general approaches for the design of recursive architectures in deep learning, the inner and the outer approach. The inner approach uses neural networks recursively inside the data graphs, essentially to "crawl" the edges of the graphs in order to compute the final output. It requires acyclic orientations of the underlying graphs. The outer approach uses neural networks recursively outside the data graphs and regardless of their orientation. These neural networks operate orthogonally to the data graph and progressively "fold" or aggregate the input structure to produce the final output. The distinction is illustrated using several examples from the fields of natural language processing, chemoinformatics, and bioinformatics, and applied to the problem of learning from variable-size sets.

Deep learning long-range information in undirected graphs with wave networks

“…Inner recursive neural networks operate on undirected graphs by enumerating a unique spanning tree for each node and applying an architecture similar to Tree LSTMs. These networks have shown promise in several chemical informatics tasks [2]. Unfortunately, neither of these techniques can operate on undirected graphs with cycles without loss of information.…”

Jet flavor classification in high-energy physics with deep neural networks

“…A natural approach to handling variable-sized input is to use recursive neural networks. Broadly speaking, there are two classes of approaches for designing such architectures, the inner approach and the outer approach [38]. In the inner approach, neural networks are used inside the data graphs to crawl the corresponding edges and compute the final output.…”

KDEEP: Protein–Ligand Absolute Binding Affinity Prediction via 3D-Convolutional Neural Networks

“…Recently, deep learning-based , approaches have rapidly emerged to provide state-of-the-art performance in fields such as computer vision, natural language processing, and generative modeling . However, the promise of deep learning in structural biology and computational chemistry has yet to be fully developed, but it is becoming increasingly common. − Driven by these developments and to find out how this class of models perform in molecular scoring tasks, we hereby propose an end-to-end framework, named K DEEP , based on 3D-convolutional neural networks for predicting protein–ligand absolute affinities. We extensively compare its performance to other existing approaches in several data sets such as the PDBbind v.2016 benchmark, several CSAR data sets, and other recently published congeneric series sets.…”