On the Advantages of Word Frequency and Contextual Diversity Measures Extracted from Subtitles: The Case of Portuguese

Abstract

We examined the potential advantage of the lexical databases using subtitles and present SUBTLEX-PT, a new lexical database for 132,710 Portuguese words obtained from a 78 million corpus based on film and television series subtitles, offering word frequency and contextual diversity measures. Additionally we validated SUBTLEX-PT with a lexical decision study involving 1920 Portuguese words (and 1920 nonwords) with different lengths in letters (M = 6.89, SD = 2.10) and syllables (M = 2.99, SD = 0.94). Multiple regression analyses on latency and accuracy data were conducted to compare the proportion of variance explained by the Portuguese subtitle word frequency measures with that accounted by the recent written-word frequency database (Procura-PALavras; P-PAL; Soares, Iriarte, et al., 2014 ). As its international counterparts, SUBTLEX-PT explains approximately 15% more of the variance in the lexical decision performance of young adults than the P-PAL database. Moreover, in line with recent studies, contextual diversity accounted for approximately 2% more of the variance in participants' reading performance than the raw frequency counts obtained from subtitles. SUBTLEX-PT is freely available for research purposes (at http://p-pal.di.uminho.pt/about/databases ).

The ERP signature of the contextual diversity effect in visual word recognition

“…This database provides not only the token account of each word (i.e., word frequency), but also the proportion of films in which a word appears. As in previous research, the CD variable was operationalized as the proportion of films [documents] in which a word appears (see also Soares et al, 2015).…”

“…In the past years, the effect of CD has received increasing attention in the field of word recognition. The basic finding is that the higher the number of contexts in which a word appears, the faster the word identification times (see also Cai & Brysbaert, 2010;Dimitropoulou, Duñabeitia, Avilés, Corral, & Carreiras, 2010;Perea, Soares, & Comesaña, 2013;Soares et al, 2015, for converging evidence). This effect is not restricted to single word identification tasks.…”

Disentangling stimulus plausibility and contextual congruency: Electro-physiological evidence for differential cognitive dynamics

“…Furthermore, we also made sure that the effect of plausibility was not confounded by other sources of information such as the lexical frequency of target words, which is known to mediate stimulus processing (see Rayner and Duffy (1986) for an example in reading research). In particular, we used the SUBTLEX-PT (Soares et al, 2015), which is the largest lexical database for the Portuguese language to date, computed the lexical frequency of plausible (2065 ± 3263) and implausible (1895 ± 4011) words and found no difference between conditions (t = .71, p = .5).…”

The Minho Word Pool: Norms for imageability, concreteness, and subjective frequency for 3,800 Portuguese words

“…Indeed, contrary to the words' objective proprieties, mostly obtained from automatic (computational) procedures applied to large corpora (see, e.g., Soares, Machado, et al, 2015;Soares, Medeiros, et al, 2014, for recent examples of these procedures), collecting subjective proprieties is more demanding and time-consuming. Typically this implies conducting large-scale studies, and thus asking a great number of participants to rate a set of words in a given subjective dimension.…”