Using Grounded Word Representations to Study Theories of Lexical Concepts

Abstract

The fields of cognitive science and philosophy have proposed many different theories for how humans represent "concepts". Multiple such theories are compatible with state-of-theart NLP methods, and could in principle be operationalized using neural networks. We focus on two particularly prominent theories-Classical Theory and Prototype Theory-in the context of visually-grounded lexical representations. We compare when and how the behavior of models based on these theories differs in terms of categorization and entailment tasks. Our preliminary results suggest that Classicalbased representations perform better for entailment and Prototype-based representations perform better for categorization. We discuss plans for additional experiments needed to confirm these initial observations.

Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games

“…Their training scheme is more complex as they first separately train the AE for each modality and then fuse them, which we avoid by adopting a single end-to-end architecture. Ebert and Pavlick (2019) used VAEs to learn grounded representations for lexical concepts. However, as discussed in Section 3, VAEs are not as well suited as RAEs to representation learning for our imagination module.…”