A studyforrest extension, an annotation of spoken language in the German dubbed movie “Forrest Gump” and its audio-description

Abstract

Here we present an annotation of speech in the audio-visual movie “Forrest Gump” and its audio-description for a visually impaired audience, as an addition to a large public functional brain imaging dataset (studyforrest.org). The annotation provides information about the exact timing of each of the more than 2500 spoken sentences, 16,000 words (including 202 non-speech vocalizations), 66,000 phonemes, and their corresponding speaker. Additionally, for every word, we provide lemmatization, a simple part-of-speech-tagging (15 grammatical categories), a detailed part-of-speech tagging (43 grammatical categories), syntactic dependencies, and a semantic analysis based on word embedding which represents each word in a 300-dimensional semantic space. To validate the dataset’s quality, we build a model of hemodynamic brain activity based on information drawn from the annotation. Results suggest that the annotation’s content and quality enable independent researchers to create models of brain activity correlating with a variety of linguistic aspects under conditions of near-real-life complexity.

Processing of visual and non-visual naturalistic spatial information in the "parahippocampal place area"

“…For the analysis of the audio-description stimulus, we extended a publicly available annotation of its speech content 21 by classifying concrete and countable nouns that the narrator uses to describe the movie’s absent visual content. An initial annotation was performed by one individual, and minor corrections were applied after comparing with a second categorization done by the author.…”

“…To answer this question, we operationalized the perception of both visual and auditory spatial information using two naturalistic stimuli (see reviews 17 – 19 ). The current operationalization of visual spatial perception is based on an annotation of cuts and depicted locations in the audio-visual movie “Forrest Gump” 20 , while the operationalization of non-visual spatial perception is based on an annotation of speech occurring in the movie’s audio-description 21 . The movie stimulus shares the stimulation in the visual domain with classical localizer stimuli, while featuring real-life-like visual complexity and naturalistic auditory stimulation.…”

Reduced dimension stimulus decoding and column-based modeling reveal architectural differences of primary somatosensory finger maps between younger and older adults

“…In addition, highly homogenous stimulation is a requirement, which was possible here by applying highly controlled sensory stimulations to individual fingers. This was ensured in earlier studies by, for example, asking participants to view the same movie (Guntupalli et al, 2016; Häusler and Hanke, 2021). However, when this decoding analysis is applied to other systems or paradigms, such as a motor task using an effective movement tracking system, or in dynamic non-synchronized cognitive tasks, it has to be clarified how subtle inter-subject variances in movement speed or cognitive computation can be accounted for by this method.…”

“…The large number of voxels and the smaller number of volumes (TRs or time points) makes the high dimensional space sparse by nature, due to which statistical analyses are difficult and underpowered. Shared response modeling (SRM) (Chen et al, 2015) is a possible solution for the problems outlined above because SRM projects the fMRI time series of each participant to a low-dimensional space, which captures the temporal variance shared across participants when exposed to the same stimulus or task sequence (for example: watching a movie (Häusler and Hanke, 2021)). The experimental manipulation or stimulus induces a series of cognitive or sensory states, like visual, auditory, or semantic, and shared variance is used to highlight the common variance related to these specific states between participants.…”

Social Media and Chronic Pain: What Do Patients Discuss?

“…To remove stop words, the commonly used stop words in English as implemented in the Natural Language Toolkit (NLTK) were combined with a manual addition of stop words that were present in the extracted text. Afterwards, lemmatisation (the process of combining the inflected forms of a word in order to work with the base form (root) of a word [ 20 ]) was performed.…”