A Turing test for crowds

Abstract

The accuracy and believability of crowd simulations underpins computational studies of human collective behaviour, with implications for urban design, policing, security and many other areas. Accuracy concerns the closeness of the fit between a simulation and observed data, and believability concerns the human perception of plausibility. In this paper, we address both issues via a so-called ‘Turing test’ for crowds, using movies generated from both accurate simulations and observations of real crowds. The fundamental question we ask is ‘Can human observers distinguish between real and simulated crowds?’ In two studies with student volunteers ( n = 384 and n = 156), we find that non-specialist individuals are able to reliably distinguish between real and simulated crowds when they are presented side-by-side, but they are unable to accurately classify them. Classification performance improves slightly when crowds are presented individually, but not enough to out-perform random guessing. We find that untrained individuals have an idealized view of human crowd behaviour which is inconsistent with observations of real crowds. Our results suggest a possible framework for establishing a minimal set of collective behaviours that should be integrated into the next generation of crowd simulation models.

Identification of Lifelike Characteristics of Human Crowds Through a Classification Task

“…To summarise, our previous work established the existence of features that are present in real crowds but not in simulated crowds; the aim of the current paper is to identify those features. In (Webster and Amos, 2020) we argue that "Our results suggest a possible framework for establishing a minimal set of collective behaviours that should be integrated into the next generation of crowd simulation models." Here, we use the "Turing test" classification task to identify that specific set of features that allow trained viewers to reliably classify (not just partition) "Real" and "Simulated" crowds.…”

“…We performed two sets of Turing test experiments; the first (Test 1) was an online-only repetition of the second (classification) test from (Webster and Amos, 2020), with entirely new participants. We attracted 232 participants, who were recruited via social media.…”

“…As we argue in (Webster and Amos, 2020), there is still a need for more realistic behavioural/movement models in crowd simulation, and "This is motivated by a widelyacknowledged need for crowd simulations to include more "lifelike" features derived from individual and social psychology (such as group-level behaviours, indecision, etc.) (Lemercier and Auberlet, 2016;Seitz et al, 2017;Templeton et al, 2015), which are generally not included in software packages, and which give rise to rather unrealistic or "robotic" patterns of behaviour at the population level".…”

“…Our previous work (Webster and Amos, 2020) showed that crowd simulations that employ the most commonlyused movement model are valid (in terms of their outputs having the same statistical properties as observed crowds), but they still possess a "signature" that allows them to be distinguished from real crowds. Simply put, to human observers, simulated crowds are still perceived differently to real crowds.…”

“…We previously used the same approach to investigate crowd simulations, basing our approach on a related Turing test for collective motion in fish (Herbert-Read et al, 2015). In (Webster and Amos, 2020), we describe the results of initial experiments, using a total of 540 inperson participants. The first set of trials presented individuals with a sequence of paired movies, using a side-by-side representation.…”