Meta-Representations as Representations of Processes

Abstract

In this study, we explore how the notion of meta-representations in Higher-Order Theories (HOT) of consciousness can be implemented in computational models. HOT suggests that consciousness emerges from meta-representations, which are representations of first-order sensory representations. However, translating this abstract concept into a concrete computational model, such as those used in artificial intelligence, presents a theoretical challenge. For example, a simplistic interpretation of meta-representation as a representation of representation makes the notion rather trivial and ubiquitous. Here, we propose a refined interpretation of meta-representations. Contrary to the simplistic view of meta-representations as mere transformations of the first-order representational states or confidence estimates, we argue that meta-representations are representations of the processes that generate first-order representations. This presents a process-oriented view whereby meta-representations capture the qualitative aspect of how sensory information is transformed into first-order representations. To concretely illustrate and operationalize thus formulated notion of meta-representation, we constructed "meta-networks" designed to explicitly model meta-representations within deep learning architectures. Specifically, we constructed meta-networks by implementing autoencoders of first-order neural networks. In this architecture, the latent spaces embedding those first-order networks correspond to the meta-representations of first-order networks. By applying meta-networks to embed neural networks trained to encode visual and auditory datasets, we show that the meta-representations of first-order networks successfully capture the qualitative aspects of those networks by separating the visual and auditory networks in the meta-representation space. We argue that such meta-representations would be useful for quantitatively compare and contrast the qualitative differences of computational processes. While whether such meta-representational systems exist in the human brain remains an open question, this formulation of meta-representation offers a new empirically testable hypothesis that there are brain regions that represent the processes of transforming a representation in one brain region to a representation in another brain region. Furthermore, this form of meta-representations might underlie our ability to describe the qualitative aspect of sensory experience or qualia.