Web-based language production experiments: Semantic interference assessment is robust for spoken and typed response modalities

Abstract

For experimental research on language production, temporal precision and high quality of the recorded audio-files are mandatory. These requirements are a considerable challenge if language production is to be investigated online. However, online research has a huge potential regarding efficiency, ecological validity and diversity of study-populations in psycholinguistic and related research, also beyond the current situation. Here, we supply confirmatory evidence that language production can be investigated online and demonstrate that the written naming responses (using the keyboard) are a reliable and efficient alternative to typical overt verbal responses. To assess semantic interference effects in both modalities we performed two pre-registered experiments (n=30 each) in online settings using the participants’ web-browsers. A cumulative semantic interference (CSI) paradigm was employed that required naming several exemplars of semantic categories within a seemingly unrelated sequence of objects. Reaction time (RT) is expected to increase linearly for each additional exemplar of a category. In experiment 1, CSI effects in naming times described in lab-based studies were replicated. In experiment 2, the responses were typed on participants’ computer keyboards and the first correct key press was used for RT analysis. This novel response assessment yielded a qualitatively similar, very robust CSI effect. Besides technical ease of application, collecting typewritten responses and automatic data preprocessing substantially reduce the work load for language production research. Results of both experiments open new perspectives for research on RT-effects in language experiments across a wide range of contexts. JavaScript- and R-based implementations for data collection and processing are available for download.

Conducting Language Production Research Online: A Web-based Study of Semantic Context and Name Agreement Effects in Multi-Word Production

“…Thus, our initial concern that speech onset latencies would be unreliable turned out to be unwarranted. Other recent studies using internet-based paradigms have provided similar evidence for the reliability of onset latencies, as they replicated several key findings of the speech production literature, including the word frequency effect (Fairs & Strijkers, 2021), the cumulative semantic interference effect (Stark et al, 2021), and the semantic interference effect in the picture-word interference paradigm (Vogt et al, 2021). Fairs and Strijkers (2021) compared the results of their webbased study to those of an otherwise identical study run in the laboratory.…”

“…The current study therefore explored the usefulness of dependent measures that did not depend on this synchrony, but were derived from the durations and fluency of the participants' utterances. Meanwhile, recent speech production studies have shown that onset latencies can in fact be measured with good accuracy in web-based platforms (e.g., Fairs & Strijkers, 2021;Stark et al, 2021;Vogt et al, 2021). We review these studies in the Discussion section.…”

Moving Experimental Psychology Online: How to Maintain Data Quality When We Can’t See Our Participants

“…Similar early reassurance came from studies of performance on linguistic tasks such as acceptability judgements (Sprouse, 2011) and self-paced reading (Enochson & Culbertson, 2015). Since then, researchers have extensively explored the validity and reliability of online data collection methods (e.g., see Stark et al, 2022 for recent demonstration of a classic semantic interference effect in word production as measured by key press responses; Angele et al, 2022 for a replication of classic masked priming effects; Ratcliff & Hendrickson, 2021 for exploration of lexical decision data). Importantly, several studies have directly compared the data from the same task when it was performed in the lab using standardized equipment or in participants' own homes with their own hardware.…”