CDRNN: Discovering Complex Dynamics in Human Language Processing

Shain, Cory

doi:10.18653/v1/2021.acl-long.288

Cited by 11 publications

(15 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach—the continuous-time deconvolutional regressive neural network (CDR-NN)—uses deep learning to relax the key simplifying assumptions above (discrete time, linearity, stationarity, and homoscedasticity) in order to estimate, visualize, and test properties of the response structure of a complex process from data. Our study expands significantly upon an earlier proposal of the CDR-NN approach (Shain, 2021 , see SI A for detailed comparison). We evaluate CDR-NNs on a range of synthetic data, as well as on publicly available behavioral and neural data from studies of human language processing.…”

Section: Introductionsupporting

confidence: 68%

“…Formally, the parameters s for the response distribution 𝓕 are computed as the sum of (i) the temporal convolution of X ′ with G 1 , …, G N and (ii) learned bias vector (intercept) s 0 , where each transposed row

, 1 ≤ n ≤ N of X ′ is vertically concatenated with a bias. This bias, which we have called rate in prior work (Shain, 2021 ; Shain & Schuler, 2018 , 2021 ), serves to capture general effects of stimulus timing, or, equivalently, the baseline response of the system to a stimulus, without regard to stimulus properties. Rate can therefore be regarded as a kind of “deconvolutional intercept”, i.e., a baseline response that is added to any stimulus-specific responses.…”

Section: The Cdr-nn Modelmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes

Shain,

Schuler

2024

Open Mind

Self Cite

View full text Add to dashboard Cite

The dynamics of the mind are complex. Mental processes unfold continuously in time and may be sensitive to a myriad of interacting variables, especially in naturalistic settings. But statistical models used to analyze data from cognitive experiments often assume simplistic dynamics. Recent advances in deep learning have yielded startling improvements to simulations of dynamical cognitive processes, including speech comprehension, visual perception, and goal-directed behavior. But due to poor interpretability, deep learning is generally not used for scientific analysis. Here, we bridge this gap by showing that deep learning can be used, not just to imitate, but to analyze complex processes, providing flexible function approximation while preserving interpretability. To do so, we define and implement a nonlinear regression model in which the probability distribution over the response variable is parameterized by convolving the history of predictors over time using an artificial neural network, thereby allowing the shape and continuous temporal extent of effects to be inferred directly from time series data. Our approach relaxes standard simplifying assumptions (e.g., linearity, stationarity, and homoscedasticity) that are implausible for many cognitive processes and may critically affect the interpretation of data. We demonstrate substantial improvements on behavioral and neuroimaging data from the language processing domain, and we show that our model enables discovery of novel patterns in exploratory analyses, controls for diverse confounds in confirmatory analyses, and opens up research questions in cognitive (neuro)science that are otherwise hard to study.

show abstract

Section: Introductionsupporting

confidence: 68%

Section: The Cdr-nn Modelmentioning

confidence: 99%

A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes

Shain,

Schuler

2024

Open Mind

Self Cite

View full text Add to dashboard Cite

show abstract

“…More flexible finite impulse response models (FIR; Dale, 1999;Glover, 1999) require fitting a large number of parameters and are computationally brittle. Thus, instead of sticking with linearized approaches, some researchers are suggesting to model the HRF shape explicitly within a family of nonlinear functions motivated by physiological data (see, for instance, Lindquist & Wager, 2007;Shain et al, 2020;Shain, 2021). By using a constrained space of nonlinear mappings (rather than an unconstrained space of linear mappings, as in FIR), one can estimate the veridical shape of the HRFs using a relatively small number of parameters.…”

Section: Incorporate Measurement-related Considerationsmentioning

confidence: 99%

Beyond linear regression: mapping models in cognitive neuroscience should align with research goals

Ivanova

Schrimpf

Anzellotti

et al. 2021

Preprint

View full text Add to dashboard Cite

Advances in cognitive neuroscience are often accompanied by an increased complexity in the methods we use to uncover new aspects of brain function. Recently, many studies have started to use large feature sets to predict and interpret brain activity patterns. Of crucial importance in this paradigm is the mapping model, which defines the space of possible relationships between the features and neural data. Until recently, most encoding and decoding studies have used linear mapping models. However, some researchers have argued that the space of linear mappings is overly constrained and advocated for the use of more flexible nonlinear mapping models. Here, we discuss the choice of a mapping model in the context of three overarching goals: predictive accuracy, interpretability, and biological plausibility. We show that, contrary to popular intuition, these goals do not map cleanly onto the linear/nonlinear divide. Moreover, we argue that, instead of categorically treating the mapping models as linear or nonlinear, we should instead aim to estimate the complexity of these models. We show that, in most cases, complexity provides a more accurate reflection of restrictions imposed by various research goals and outline several complexity metrics that can be used to effectively evaluate mapping models.

show abstract

“…To this end, one approach, called Monte Carlo dropout (MCD), has seen increasing use within the domain of neuroimaging classification [13]. MCD has been used in a variety of studies, including those focused on cortex parcellation [32], dynamics estimation [33], [34], and classification of autism spectrum disorder [35] and Parkinson’s disease [36]. A more recently developed alternative to MCD that has seen comparatively little use in the domain of neuroimaging classification is Monte Carlo batch normalization (MCBN) [14].…”

Section: Introductionmentioning

confidence: 99%

Towards Greater Neuroimaging Classification Transparency via the Integration of Explainability Methods and Confidence Estimation Approaches

Ellis

Miller

Calhoun

2022

Preprint

View full text Add to dashboard Cite

The field of neuroimaging has increasingly sought to develop artificial intelligence-based models for neurological and neuropsychiatric disorder automated diagnosis and clinical decision support. However, if these models are to be implemented in a clinical setting, transparency will be vital. Two aspects of transparency are (1) confidence estimation and (2) explainability. Confidence estimation approaches indicate confidence in individual predictions. Explainability methods give insight into the importance of features to model predictions. In this study, we integrate confidence estimation and explainability approaches for the first time. We demonstrate their viability for schizophrenia diagnosis using resting state functional magnetic resonance imaging (rs-fMRI) dynamic functional network connectivity (dFNC) data. We compare two confidence estimation approaches: Monte Carlo dropout (MCD) and MC batch normalization (MCBN). We combine them with two gradient-based explainability approaches, saliency and layer-wise relevance propagation (LRP), and examine their effects upon explanations. We find that MCD often adversely affects model gradients, making it ill-suited for integration with gradient-based explainability methods. In contrast, MCBN does not affect model gradients. Additionally, we find many participant-level differences between regular explanations and the distributions of explanations for combined explainability and confidence estimation approaches. This suggests that a similar confidence estimation approach used in a clinical context with explanations only output for the regular model would likely not yield adequate explanations. We hope that our findings will provide a starting point for the integration of the two fields, provide useful guidance for future studies, and accelerate the development of transparent neuroimaging clinical decision support systems.

show abstract

CDRNN: Discovering Complex Dynamics in Human Language Processing

Cited by 11 publications

References 69 publications

A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes

A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes

Beyond linear regression: mapping models in cognitive neuroscience should align with research goals

Towards Greater Neuroimaging Classification Transparency via the Integration of Explainability Methods and Confidence Estimation Approaches

Contact Info

Product

Resources

About