Cognitive programs: software for attention's executive

Abstract

What are the computational tasks that an executive controller for visual attention must solve? This question is posed in the context of the Selective Tuning model of attention. The range of required computations go beyond top-down bias signals or region-of-interest determinations, and must deal with overt and covert fixations, process timing and synchronization, information routing, memory, matching control to task, spatial localization, priming, and coordination of bottom-up with top-down information. During task execution, results must be monitored to ensure the expected results. This description includes the kinds of elements that are common in the control of any kind of complex machine or system. We seek a mechanistic integration of the above, in other words, algorithms that accomplish control. Such algorithms operate on representations, transforming a representation of one kind into another, which then forms the input to yet another algorithm. Cognitive Programs (CPs) are hypothesized to capture exactly such representational transformations via stepwise sequences of operations. CPs, an updated and modernized offspring of Ullman's Visual Routines, impose an algorithmic structure to the set of attentional functions and play a role in the overall shaping of attentional modulation of the visual system so that it provides its best performance. This requires that we consider the visual system as a dynamic, yet general-purpose processor tuned to the task and input of the moment. This differs dramatically from the almost universal cognitive and computational views, which regard vision as a passively observing module to which simple questions about percepts can be posed, regardless of task. Differing from Visual Routines, CPs explicitly involve the critical elements of Visual Task Executive (vTE), Visual Attention Executive (vAE), and Visual Working Memory (vWM). Cognitive Programs provide the software that directs the actions of the Selective Tuning model of visual attention.

Learning a New Selection Rule in Visual and Frontal Cortex

“…Theoretical studies suggested that feedback signals about the chosen action to visual cortex may play a role in the guidance of neuronal plasticity, by focusing synaptic changes on the visual representations that are related to the choice ( Roelfsema et al 2010 ; Rombouts et al 2012 , 2015 ). The feedback signals are delayed relative to the onset of the visual stimulus, presumably because they depend on recurrent interactions within and between areas of the frontal, parietal, and visual cortex where neurons jointly select one of the 2 curves ( Bruce and Goldberg 1985 ; Schall et al 1995 ; Khayat et al 2009 ; Pooresmaeili et al 2014 ) and in accordance with theories that stress the importance of reentrant processing ( Di Lollo et al 2000 ; Lamme and Roelfsema 2000 ; Roelfsema 2005 ; Cichy et al 2014 ; Tsotsos and Kruijne 2014 ).…”

Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs

“…Fig 2(B) lists the instruction set of VCC, and their mapping to the different controllers. The instruction set of the VCC was heavily influenced by the primacy of objects and spatial primitives in human cognition (29), and by the elemental operations that have been previously proposed (11,13). Looping constructs loop start, loop end are constrained to loop over the currently attended objects in a scene.…”

“…In this paper we hypothesize that human concepts are programs, termed cognitive programs (11), implemented on a computer with certain constraints and architectural biases (eg. a 'Human Turing Machine' (12)) that are fundamentally different compared to the prevalent von Neumann architecture.…”

Complexity Level Analysis Revisited: What Can 30 Years of Hindsight Tell Us about How the Brain Might Represent Visual Information?

“…A smaller additional cost would be the process of actual tuning. (Reproduced from Tsotsos and Kruijne, 2014 ).…”