cottoncandy: scientific python package for easy cloud storage

Núñez-Elizalde, Anwar O.; Gao, James; Zhang, Tianjiao; Gallant, Jack L.

doi:10.21105/joss.00890

Cited by 4 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we quantified modality-specific features to capture additional sources of variance. For videos, these included motion energy computed using a pyramid of spatio-temporal Gabor filters as implemented in the pymoten package 102 . We also included two video-specific action features which were not present or discernible in the sentences: tool use (binary feature: whether the action was tool-mediated), and effectors (binary vectors indicating the body parts involved in each action: face/head, hands, arms, legs, and torso).…”

Section: Methodsmentioning

confidence: 99%

Shared representations of human actions across vision and language

Dima,

Janarthanan,

Culham

et al. 2023

Preprint

View full text Add to dashboard Cite

Humans can recognize and communicate about many actions performed by others. How are actions organized in the mind, and is this organization shared across vision and language? We collected similarity judgments of human actions depicted through naturalistic videos and sentences, and tested four models of action categorization, defining actions at different levels of abstraction ranging from specific (action verb) to broad (action target: whether an action is directed towards an object, another person, or the self). The similarity judgments reflected a shared semantic organization across videos and sentences, determined mainly by the target of actions, even after accounting for other semantic features. Large language model features predicted the behavioral similarity of action videos and sentences, and captured information about the target of actions alongside unique semantic information. Together, our results show how modality-invariant action concepts are organized in the human mind and in large language model representations.

show abstract

Section: Methodsmentioning

confidence: 99%

Shared representations of human actions across vision and language

Dima,

Janarthanan,

Culham

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Extract motion-energy features from the stimuli (optional). This optional notebook describes how to use pymoten [Nunez-Elizalde et al, 2021] to extract motion-energy features from the stimulus. Motion-energy features are low-level visual features extracted from the movie clips stimuli using a collection of spatio-temporal Gabor filters.…”

Section: Understand Ridge Regression and Hyperparameter Selection (Op...mentioning

confidence: 99%

The Voxelwise Modeling framework: a tutorial introduction to fitting encoding models to fMRI data

Dupré la Tour,

Visconti di Oleggio Castello,

Gallant

2024

Preprint

View full text Add to dashboard Cite

Voxelwise modeling (VM) is a powerful framework for functionally mapping the brain. In the VM framework, features are extracted from the stimulus (or task) and used in an encoding model to predict brain activity. If the encoding model is able to predict brain activity in some part of the brain, then one may conclude that some information represented in the features is also encoded in the brain. In VM, a separate encoding model is fitted on each spatial sample (i.e. each voxel). VM has many benefits compared to other methods for analyzing and modeling neuroimaging data. Most importantly, VM can use large numbers of features simultaneously, which enables the analysis of complex naturalistic stimuli and tasks. Therefore, VM can produce high-dimensional functional maps that reflect the selectivity of each voxel to large numbers of features. Moreover, because model performance is estimated on a separate test dataset not used during fitting, VM minimizes overfitting and inflated Type I error confounds that plague other approaches, and the results of VM generalize to new subjects and new stimuli. Despite these benefits, VM is still not widely used in neuroimaging, partly because no tutorials on this method are available currently. To demystify the VM framework and ease its dissemination, this paper presents a series of hands-on tutorials accessible to novice practitioners. The VM tutorials are based on free open- source tools and public datasets, and reproduce the analysis presented in previously published work.

show abstract

“…The data analysis pipeline for Model Connectivity was implemented in Python (modelconn), and is publicly available [link will be provided here upon publication]. The package modelconn builds on the scientific Python ecosystem, using numpy 118 , matplotlib 119 , scipy 108 , scikit-learn 120 , umap-learn 117 , nipype 121 , nibabel 122 , pycortex 107 , cottoncandy 123 , and himalaya 44 with a pytorch backend 124 .…”

Section: Data Analysis Implementationmentioning

confidence: 99%

Model connectivity: leveraging the power of encoding models to overcome the limitations of functional connectivity

Meschke

Castello

Tour

et al. 2023

Preprint

View full text Add to dashboard Cite

Functional connectivity (FC) is the most popular method for recovering functional networks of brain areas with fMRI. However, because FC is defined as temporal correlations in brain activity, FC networks are confounded by noise and lack a precise functional role. To overcome these limitations, we developed model connectivity (MC). MC is defined as similarities in encoding model weights, which quantify reliable functional activity in terms of interpretable stimulus- or task-related features. To compare FC and MC, both methods were applied to a naturalistic story listening dataset. FC recovered spatially broad networks that are confounded by noise, and that lack a clear role during natural language comprehension. By contrast, MC recovered spatially localized networks that are robust to noise, and that represent distinct categories of semantic concepts. Thus, MC is a powerful data-driven approach for recovering and interpreting the functional networks that support complex cognitive processes.

show abstract

cottoncandy: scientific python package for easy cloud storage

Cited by 4 publications

References 2 publications

Shared representations of human actions across vision and language

Shared representations of human actions across vision and language

The Voxelwise Modeling framework: a tutorial introduction to fitting encoding models to fMRI data

Model connectivity: leveraging the power of encoding models to overcome the limitations of functional connectivity

Contact Info

Product

Resources

About