MEMO: A Deep Network for Flexible Combination of Episodic Memories

Abstract

Recent research developing neural network architectures with external memory have often used the benchmark bAbI question and answering dataset which provides a challenging number of tasks requiring reasoning. Here we employed a classic associative inference task from the memory-based reasoning neuroscience literature in order to more carefully probe the reasoning capacity of existing memoryaugmented architectures. This task is thought to capture the essence of reasoningthe appreciation of distant relationships among elements distributed across multiple facts or memories. Surprisingly, we found that current architectures struggle to reason over long distance associations. Similar results were obtained on a more complex task involving finding the shortest path between nodes in a path. We therefore developed MEMO, an architecture endowed with the capacity to reason over longer distances. This was accomplished with the addition of two novel components. First, it introduces a separation between memories/facts stored in external memory and the items that comprise these facts in external memory. Second, it makes use of an adaptive retrieval mechanism, allowing a variable number of 'memory hops' before the answer is produced. MEMO is capable of solving our novel reasoning tasks, as well as match state of the art results in bAbI.

TransDreamer: Reinforcement Learning with Transformer World Models

“…Transformers have been shown to outperform RNNs in many tasks in both NLP and computer vision. In particular, their ability to directly access historical states and to learn complex interactions among them, has been shown to excel in tasks that require complex long-term temporal dependencies such as memory-based reasoning (Ritter et al, 2020;Banino et al, 2020). Furthermore, they have been shown to be effective for temporal generation in both language and visual domains.…”

“…To deal with partial observability (Kaelbling et al, 1998), the dynamics models in MBRL have been implemented using recurrent neural networks (RNNs) (Hafner et al, 2019;Schrittwieser et al, 2020;Kaiser et al, 2019). However, Transformers (Vaswani et al, 2017;Dai et al, 2019) have shown to be more effective than RNNs in many domains requiring long-term dependency and direct access to memory for a form of memory-based reasoning (Ritter et al, 2020;Banino et al, 2020). Also, it has been shown that training complex policy networks based on transformers using only rewards is difficult (Parisotto et al, 2020), so learning a transformer-based world model where the training signal is more diverse may facilitate learning.…”

Understanding in Artificial Intelligence

“…In spirit, methods for Program induction tend to be closer to neural networks than to symbolic computing. For instance, architectures such as the Neural Turing Machine [197,198], the Differential Neural Computer [198,199], the Neural programmer [200], Neural programmer-interpreters [201,195], Neural Program Lattices [202], the Neural State Machine [200], and most recently MEMO [203] extend neural networks with external memory, and can infer simple algorithms such as adding numbers, copying, sorting and path finding. [165] illustrates the use of DeepProbLog to solve three program induction tasks and compare their results to Differentiable Forth (∂4) [204].…”

Generating Radiology Reports via Memory-driven Transformer

“…For example, Lample et al (2019) proposed to solve the under-fitting problem of Transformer by introducing a product-key layer that is similar to a memory module. Banino et al (2020) proposed MEMO, an adaptive memory to reason over long-distance texts. Compared to these studies, the approach proposed in this paper focuses on leveraging memory for decoding rather than encoding, and presents a relational memory to learn from previous generation processes as well as patterns for long text generation.…”