Getting Real about Systematicity

Connecting Twenty-First Century Connectionism and Wittgenstein

“…The volume of the information is greatly reduced in memory storage, i.e., the dimensionality of the tensor product is of the same dimensionality as the constituents, but even after further manipulation the constituent information can be retrieved from memory with very little degradation (Plate 2003) and symbols/ representations can continue to change to some extent as more related instances are added to a VSA memory. 23 Contrary to classicist assumptions, there is evidence that linguistic structure can be built in connectionist architectures via the development of symbols without the need for anything like grammatical rules (Elman 2014;Frank 2014). These lexicon approaches work with the ability of symbols to bind together into meaningful units on the basis of their own complex internal structure (Bod et al 2003).…”

“…Regularities in use reinforced through training; and the subsequent ability to combine in different ways provides the compositionality that allows for productivity without classical symbols and rules. Thus Stephan Frank, building on Elman's work shows a way to develop what Hadley (1994) described as "strong systematicity" with connectionist symbols, although Frank notes that a symbols and rules approach combined with statistical probability still worked slightly better in mimicking our language use than his connectionist model (Frank 2014).…”