Significance Our expanding understanding of the brain at the level of neurons and synapses, and the level of cognitive phenomena such as language, leaves a formidable gap between these two scales. Here we introduce a computational system which promises to bridge this gap: the Assembly Calculus. It encompasses operations on assemblies of neurons, such as project, associate, and merge, which appear to be implicated in cognitive phenomena, and can be shown, analytically as well as through simulations, to be plausibly realizable at the level of neurons and synapses. We demonstrate the reach of this system by proposing a brain architecture for syntactic processing in the production of language, compatible with recent experimental results.

Assemblies are large populations of neurons believed to imprint memories, concepts, words and other cognitive information. We identify a repertoire of operations on assemblies. These operations correspond to properties of assemblies observed in experiments, and can be shown, analytically and through simulations, to be realizable by generic, randomly connected populations of neurons with Hebbian plasticity and inhibition. Operations on assemblies include: projection (duplicating an assembly by creating a new assembly in a downstream brain area); reciprocal projection (a variant of projection also entailing synaptic connectivity from the newly created assembly to the original one); association (increasing the overlap of two assemblies in the same brain area to reflect cooccurrence or similarity of the corresponding concepts); merge (creating a new assembly with ample synaptic connectivity to and from two existing ones); and pattern-completion (firing of an assembly, with some probability, in response to the firing of some but not all of its neurons). Our analytical results establishing the plausibility of these operations are proved in a simplified mathematical model of cortex: a finite set of brain areas each containing n excitatory neurons, with random connectivity that is both recurrent (within an area) and afferent (between areas). Within one area and at any time, only k of the n neurons fire -an assumption that models inhibition and serves to define both assemblies and areaswhile synaptic weights are modified by Hebbian plasticity, as well as homeostasis. Importantly, all neural apparatus needed for the functionality of the assembly operations is created on the fly through the randomness of the synaptic network, the selection of the k neurons with the highest synaptic input, and Hebbian plasticity, without any special neural circuits assumed to be in place. Assemblies and their operations constitute a computational model of the brain which we call the Assembly Calculus, occupying a level of detail intermediate between the level of spiking neurons and synapses, and that of the whole brain. As with high-level programming languages, a computation in the Assembly Calculus (that is, a coherent sequence of assembly operations accomplishing a task) can ultimately be reduced -"compiled down" -to computation by neurons and synapses; however, it would be far more cumbersome and opaque to represent the same computation that way. The resulting computational system can be shown, under assumptions, to be in principle capable of carrying out arbitrary computations. We hypothesize that something like it may underlie higher human cognitive functions such as reasoning, planning, and language. In particular, we propose a plausible brain architecture based on assemblies for implementing the syntactic processing of language in cortex, which is consistent with recent experimental results.

We revisit the planning problem in the blocks world, and we implement a known heuristic for this task. Importantly, our implementation is biologically plausible, in the sense that it is carried out exclusively through the spiking of neurons. Even though much has been accomplished in the blocks world over the past five decades, we believe that this is the first algorithm of its kind. The input is a sequence of symbols encoding an initial set of block stacks as well as a target set, and the output is a sequence of motion commands such as "put the top block in stack 1 on the table". The program is written in the Assembly Calculus, a recently proposed computational framework meant to model computation in the brain by bridging the gap between neural activity and cognitive function. Its elementary objects are assemblies of neurons (stable sets of neurons whose simultaneous firing signifies that the subject is thinking of an object, concept, word, etc.), its commands include project and merge, and its execution model is based on widely accepted tenets of neuroscience. A program in this framework essentially sets up a dynamical system of neurons and synapses that eventually, with high probability, accomplishes the task. The purpose of this work is to establish empirically that reasonably large programs in the Assembly Calculus can execute correctly and reliably; and that rather realistic -if idealized -higher cognitive functions, such as planning in the blocks world, can be implemented successfully by such programs.