A quantitative model of ensemble perception as summed activation in feature space

Abstract

Ensemble perception is a process by which we extract and consolidate redundancies in the environment, getting the 'gist' of a set of objects. This ability to summarize complex scenes is thought to underlie our ability to construct robust memory representations, guide attention and classify features or objects when many objects are present. Despite the relevance of ensemble perception to everyday cognition, there are few computational models that provide a formal, process-level account of ensemble perception across a range of experimental conditions and stimuli. In the current work, we take a conceptually novel approach towards developing such a model. According to the proposed theory of how ensemble representations are computed, items evoke distributed patterns of familiarity over feature space (e.g., a color evokes a pattern of familiarity over all color channels), and ensemble representations reflect the global sum of these signals across all individual items. We leverage this set of minimal assumptions to formally connect a model of memory for individual items to a model for ensembles. Our approach involves using parameter estimates of performance on a visual memory task for individual items, to predict individual differences in performance on an ensemble continuous report task. That is, we account for the full distribution of errors on the ensemble task with zero free parameters. We evaluate our ensemble model against a set of newly developed alternative models in five ensemble tasks. Our top-down modeling approach is the first to formally unify models of memory for individual items and ensembles, and opens a novel venue for building and comparing models of distinct memory processes and representations. We discuss our model and results in the context of current theories and models of ensemble processing, gist memory and neural population coding.Human perception and cognition are grounded in a capacity limited system (e.g., Baddaley, 1992;Miller, 1956;Pashler, 1984;Simon, 1990). For this reason, a basic question across research areas in the behavioral and social sciences is how people effectively represent and navigate an environment that should far exceed their processing capabilities (e.g., Kahneman, 2003). A widely accepted answer to this question is that perceptual and cognitive systems take advantage of redundancies in the environment, for example, by making use of a condensed summary or gist rather than representing all of the details of the world (Ariely, 2001;Brady & Oliva, 2008;Goldstein, et al., 2010). In the visual domain, the ability of people to extract summaries by making use of such redundancies is typically referred to as ensemble perception (Whitney & Yamanashi Leib, 2018). In a typical laboratory ensemble task, participants are shown a set of stimuli that share properties in a specific feature dimension (e.g., color, orientation, motion direction, face emotion), and are instructed to report on their average along that dimension. For instance, as depicted in Figure 1, participants might be s...

Similarity in feature space dictates the efficiency of attentional selection during ensemble processing

“…Recent models of ensemble perception, where ensemble reports are generated by reading out the peak of population activity in the relevant neural representational space (Baek & Chong, 2020;Robinson & Brady, 2023;Utochkin et al, 2023), are well suited to explain our results. In the overlapping conditions in our experiment, the similarity of the item sets may lead them to summate to a single peak in orientation space initially, requiring selective processes to upweight or downweight target or distractor activity, resulting in an overall shifted peak and thus a biased orientation report.…”

“…Thus, discarding irrelevant item or ensemble information requires processing time, and does not occur in the initial feedforward sweep of visual processing. Notably, while population activity models would predict biases towards irrelevant items because those items naturally contribute to the initial peak in population activity (e.g., Robinson & Brady, 2023), none of these models explicitly incorporate the type of attentional processes that are necessary for selection or suppression of entire item sets as in the current task, nor do they account for the additional temporal dynamics revealed in our Experiment 2. Such effects would be fruitful avenues for future modeling efforts, and may help quantify the relative balance of target activity enhancement versus distractor suppression that would explain our observed biases.…”

A signal-detection account of item-based and ensemble-based visual change detection: A reply to Harrison, McMaster, and Bays

“… Utochkin et al (2023) suggest a pooling and population coding model (akin to earlier pooling models considering related phenomena, e.g., Harrison & Bex, 2015 ; Webb, Ledgeway, & McGraw, 2010 ) that captures the variety of ensemble representations, including the computation of average and variability, as well as setting boundary conditions for all items being perceived as belonging to one or several clusters. Robinson and Brady (2023) conceptualize the ensemble representation as a pooled familiarity signal produced by individual items in noisy feature-selective channels ( Schurgin, Wixted, & Brady, 2020 ), an idea having a strong reference to population coding. Udale, Gramm, Husain, & Manohar (2021) suggest that population codes can apply both to absolute values of individual features and to feature relations at different hierarchical levels of processing and that observers rely on a combination of these population responses when they perform a visual working memory task.…”