Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks

Abstract

Figure 1. [Better seen in color]. Overview of the proposed solution. Our Agglomerator is a novel architecture for vision applications, in which column structure (c) mimics hyper-columns typical of the human visual cortex [14]. The input data (a) is fed to the columns using a patch-based embedding (b). The Agglomerator architecture iteratively routes the information across its structure, creating a neural representation of each image, similar to neural fields [37]. In the neural representation, part-whole hierarchies (d) emerge at different levels of the columns. The same column can represent the same patch of the image with different levels of abstraction (e.g., the ears, the head, and the dog) corresponding to each level in the column. Neighbor columns agree on a part representation (e.g ears, head) at lower levels, ideally representing the same whole (e.g. dog) at the top level. The resulting feature space represents the conceptual-semantic relationships between data (e) resembling the human hierarchical organization [38]. Samples belonging to the same super-class (e.g., animals, vehicles) are clustered together, with conceptually close categories (e.g., birds and airplanes) represented on the edge of the super-classes.