Perceiving and remembering speech depend on multifractal nonlinearity in movements producing and exploring speech

Abstract

Speech perception and memory for speech require active engagement. Gestural theories have emphasized mainly the effect of speaker's movements on speech perception. They fail to address the effects of listener movement, focusing on communication as a boundary condition constraining movement among interlocutors. The present work attempts to break new ground by using multifractal geometry of physical movement as a common currency for supporting both sides of the speaker–listener dyads. Participants self-paced their listening to a narrative, after which they completed a test of memory querying their narrative comprehension and their ability to recognize words from the story. The multifractal evidence of nonlinear interactions across timescales predicted the fluency of speech perception. Self-pacing movements that enabled listeners to control the presentation of speech sounds constituted a rich exploratory process. The multifractal nonlinearity of this exploration supported several aspects of memory for the perceived spoken language. These findings extend the role of multifractal geometry in the speaker's movements to the narrative case of speech perception. In addition to posing novel basic research questions, these findings make a compelling case for calibrating multifractal structure in text-to-speech synthesizers for better perception and memory of speech.

Ergodic descriptors of non-ergodic stochastic processes

“…Theoretically, the form and behaviour that distinguish life as an evolving, innovating and adaptive process are bound towards novelty and divergence. Empirically, measurements of living and thinking systems widely fail tests of ergodicity at all levels of organization: motion of protein molecules with a cell [8][9][10][11][12], haemodynamics and intracellular and extracellular transport of complex media in biological systems, such as cytoplasm and nucleoplasm [13,14], intravascular blood flow [15], coupled networks of heterogeneous neurons [16,17] and cognitive processes involved in a variety of perceptuomotor behaviour [18][19][20][21][22]. Description by averages is at odds with our expectations of what life is and as noted above, what is at stake is not a mathematical nicety but rather the issue that applying ergodic models to non-ergodic measurement privileges expedience and convention over access to the cause.…”

Multifractal test for nonlinearity of interactions across scales in time series

“…However, all of the preceding discussion offers a roadmap to approach nonlinearity with a sober view of the possibilities. For instance, the mounting evidence that nonlinearity matters to psychological experience is intriguing and worth further and broader attention (Bloomfield et al, 2021;Carver et al, 2017;Doyon et al, 2019;Jacobson et al, 2020;Kelty-Stephen et al, 2016Olthof, Hasselman, Wijnants, et al, 2020;Schiepek, 2009;Teng et al, 2016;Ward & Kelty-Stephen, 2018). We hope that multifractality in our measures speaks directly to the capacity of developing systems to blend processes across different timescales, producing emergent structures (e.g., Gottlieb 2002).…”

“…An important point, though, is that the tstatistic's significance is thus only part of the information we need to worry about. The tstatistic becomes a continuous measure of multifractal nonlinearity, and our recent work has successfully used this t-statistic as a predictor of psychological outcomes-perceptual (Carver et al, 2017) and cognitive (Bloomfield et al, 2021) outcomes alike. Important to note, this t-statistic serves as a significant predictor for all values of the t-statistic-that is, a significant predictor in a regression model, no matter whether each value of the tstatistic is itself significantly large for each row of the dataset supporting that regression model.…”

Better than DFA? A Bayesian Method for Estimating the Hurst Exponent in Behavioral Sciences

“…RT time series were reanalyzed from a published study [139]. Data was collected on twenty adults (nine men and eleven women, M ± SD age = 20.10 ± 1.29 years) after obtaining informed consent.…”