Signatures of cross-modal alignment in children's early concepts

Abstract

Whether supervised or unsupervised, human and machine learning is usually characterised as event-based. However, learning may also proceed by systems alignment in which mappings are inferred between systems, such as visual and linguistic systems. Systems alignment is possible because items that share similar visual contexts, such as a car and a truck, will also tend to share similar linguistic contexts. Because of the mirrored similarity relationships, the visual and linguistic systems and can be aligned at some later time absent either input. We considered whether children's early concepts are learned by systems alignment. We found that children's early concepts are close to optimal for inferring novel concepts through system alignment. Structurally, children's early concepts are distinguished by their dense semantic neighbourhoods. Artificial agents using these structural features were highly effective, including in domains that exclude children's early concepts. For children, system alignment and event-based learning appear complementary. Artificial systems can benefit from incorporating these developmental principles.