Augmenting Robot Knowledge Consultants with Distributed Short Term Memory

Abstract

Human-robot communication in situated environments involves a complex interplay between knowledge representations across a wide variety of modalities. Crucially, linguistic information must be associated with representations of objects, locations, people, and goals, which may be represented in very different ways. In previous work, we developed a Consultant Framework that facilitates modality-agnostic access to information distributed across a set of heterogeneously represented knowledge sources. In this work, we draw inspiration from cognitive science to augment these distributed knowledge sources with Short Term Memory Buffers to create an STM-augmented algorithm for referring expression generation. We then discuss the potential performance benefits of this approach and insights from cognitive science that may inform future refinements in the design of our approach.

Toward Forgetting-Sensitive Referring Expression Generationfor Integrated Robot Architectures

“…For example, Hawes et al (2007) leverage a model of WM within the CoSy architecture, with concessions made to accommodate the realistic needs of integrated robot architectures, in which specialized representations are stored and manipulated within distributed, parallelized, domain-specific components (see also . Similarly, in our own work within the DIARC architecture (Scheutz et al, 2019), we have demonstrated (as we will further discuss in this paper) the use of distributed WM buffers associated with such architectural components (Williams et al, 2018b), as well as hierarchical models of common ground jointly inspired by models of WM (e.g. Cowan, 2001) and models of givenness from natural language pragmatics (e.g.…”

“…These concerns are addressed by our previously proposed model of robotic short-term memory (Williams et al, 2018b), in which speakers rely on the contents of WM for initial attribute selection and, if their selected referring expression is not fully discriminating, select additional properties using a variant of the IA. While this does not align with dual-process models of cognition, it does account for both encoding and maintenance of WM, and provides a potentially more cognitively plausible account of REG with respect to WM dynamics.…”

“…Similarly, if R is the target referent during REG, and if property P holds for R and is considered for inclusion in the generated RE, then for any distractors for which P also holds, P will be added to those distractors' WM buffers at the point where it is affirmed that P cannot be used to rule out those distractors because it holds for them as well. Once Reference Resolution is completed, if the robot decides to respond to the human utterance, it does so through a process that is largely the inverse of language understanding, including a dedicated REG module, which uses the properties cached in WM, along with properties retrievable from Long-Term Memory, to translate this intention into text (Williams & Scheutz, 2017;Williams et al, 2018b).…”

Enabling Morally Sensitive Robotic Clarification Requests

“…Another example of a consultant is the agent consultant, which stores information about other agents (like humans) with which a robot might interact. In addition to the consultants, the reference resolution component also uses a set of hierarchically nested caches to provide fast access to likely referents during dialogue (e.g., objects that were recently referenced) [58].…”