Systematically Grounding Language through Vision in a Deep, Recurrent Neural Network

What are the computational correlates of consciousness?

“…The hypothesis that symbol grounding is a computational correlate of consciousness makes neurocomputational systems that learn to ground auditory symbols by associating them with visual images potentially relevant to the issue of consciousness (Weems and Reggia, 2006). Such models, while quite limited at present, can learn to produce the meaning of sentences describing a very simple external world that is observed, including for novel objects and novel descriptive sentences (Monner and Reggia, 2011). They sometimes learn/discover a latent symbol system -a system for symbol processing that was not built into the structure of the network but instead emerged as part of its learned distributed representation (Monner and Reggia, 2012…”

Getting Real about Systematicity

“…And although there have been many attempts to empirically demonstrate (lack of) systematicity in connectionist models, it remains doubtful how these demonstrations bear upon reality considering that they are always restricted to handcrafted, miniature domains. This is the case irrespective of whether they are presented by supporters of connectionist systematicity 1 (Bodén, 2004;Brakel & Frank, 2009;Chang, 2002;Christiansen & Chater, 1994;Elman, 1991;Farkaš & Crocker, 2008;Fitz & Chang, 2009;Frank, 2006aFrank, , 2006bFrank & Čerňanský, 2008;Frank, Haselager, & van Rooij, 2009;Hadley, Rotaru-Varga, Arnold, & Cardei, 2001;Jansen & Watter, 2012;McClelland, St.John, & Taraban, 1989;Miikkulainen, 1996;Monner & Reggia, 2011;Niklasson & Van Gelder, 1994;Voegtlin & Dominey, 2005;Wong & Wang, 2007) or by those who are more skeptical (Marcus, 2001;Phillips, 1998; Van der Velde, Van der Voort van der Kleij, & De Kamps, 2004).…”

Getting Real about Systematicity

“…And although there have been many attempts to empirically demonstrate (lack of) systematicity in connectionist models, it remains doubtful how these demonstrations bear upon reality considering that they are always restricted to handcrafted, miniature domains. This is the case irrespective of whether they are presented by supporters of connectionist systematicity 1 (Bodén, 2004;Brakel & Frank, 2009;Chang, 2002;Christiansen & Chater, 1994;Elman, 1991;Farkaš & Crocker, 2008;Fitz & Chang, 2009;Frank, 2006aFrank, , 2006bFrank & Čerňanský, 2008;Frank, Haselager, & van Rooij, 2009;Hadley, Rotaru-Varga, Arnold, & Cardei, 2001;Jansen & Watter, 2012;McClelland, St.John, & Taraban, 1989;Miikkulainen, 1996;Monner & Reggia, 2011;Niklasson & Van Gelder, 1994;Voegtlin & Dominey, 2005;Wong & Wang, 2007) or by those who are more skeptical (Marcus, 2001;Phillips, 1998; Van der Velde, Van der Voort van der Kleij, & De Kamps, 2004).…”