Internet based language production research with overt articulation: Proof of concept, challenges, and practical advice

Vogt, Anne B.; Hauber, Roger Christoph; Kuhlen, Anna K.; Rahman, Rasha Abdel

doi:10.31234/osf.io/cyvwf

Cited by 8 publications

(19 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a picture-naming paradigm, Fairs & Strijkers (2021) replicated lab-based effects of word frequency in an online experiment run on the platform FindingFive (FindingFive Team, 2019) with 100 participants. For the picture word interference (PWI) effect (Bürki et al, 2020;Lupker, 1979), which requires naming object pictures overlayed with semantically related or unrelated distractor words, Vogt et al (2021) showed online-feasibility reproducing the lab-based findings. They implemented the experiment on two different platforms (jsPsych, de Leeuw & Motz, 2016;and SoSciSurvey, Leiner, 2019) with different participant cohorts (each n = 48).…”

mentioning

confidence: 68%

“…Recent evidence, however, suggests that studying language production via online platforms is possible. Overt naming responses acquired online have been demonstrated to be precise enough to detect speech-onset reaction time effects in the critical range of ~10-20 milliseconds (Fairs & Strijkers, 2021a;Vogt et al, 2021). In a picture-naming paradigm, Fairs & Strijkers (2021) replicated lab-based effects of word frequency in an online experiment run on the platform FindingFive (FindingFive Team, 2019) with 100 participants.…”

mentioning

confidence: 99%

“…Latencies of key presses can be detected with basic methods, and are, next to mouse clicks, the main means of reaction time detection in both labbased and online experiments. They have been implemented in a wide variety of online experiments, including psycholinguistic experiments using categorization tasks (Mathôt & March, 2021;Vogt et al, 2021). Additionally, it has already been shown that typewritten texts can be preprocessed automatically and a wide range of different procedures exists, even controlling for typing errors (Borrie et al, 2019;Bosker, 2021;Navarro, 2001).…”

mentioning

confidence: 99%

“…In the current study, we therefore compared spoken and typewritten naming responses in a browser-based online experiment using a reaction-time sensitive and well-established paradigm, the cumulative semantic interference (CSI) paradigm. In experiment 1, we set out to replicate lab-based language production effects using the CSI paradigm in the same experimental platform and audio-recording method as described in Vogt et al (2021), Experiment 1 (SoSciSurvey). The CSI paradigm requires to name several exemplars of semantic categories within a seemingly unrelated sequence of objects.…”

mentioning

confidence: 99%

“…However, when put into practice, the overall noise seems to affect data quality less than expected. Being relatively stable for a single participant, we can assume high precision for within-subject comparisons with a sufficient amount of trials (Bridges et al, 2020;Pinet et al, 2017b;Vogt et al, 2021; see Baker et al, 2020 for a recent article on how sample size and number of trials affect statistical power). Regarding keypresses, by comparing the actual time elapsing between successive keystrokes with the time measured with a JavaScript-based implementation, one study found a delay of ~60 ms on average in absolute latencies detected in an online experiment, but no delay (0 ms) in relative latencies (Pinet et al, 2017).…”

mentioning

confidence: 99%

See 4 more Smart Citations

Web-based language production experiments: Semantic interference assessment is robust for spoken and typed response modalities

Stark¹,

C²,

Obrig³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

For experimental research on language production, temporal precision and high quality of the recorded audio-files are mandatory. These requirements are a considerable challenge if language production is to be investigated online. However, online research has a huge potential regarding efficiency, ecological validity and diversity of study-populations in psycholinguistic and related research, also beyond the current situation. Here, we supply confirmatory evidence that language production can be investigated online and demonstrate that the written naming responses (using the keyboard) are a reliable and efficient alternative to typical overt verbal responses. To assess semantic interference effects in both modalities we performed two pre-registered experiments (n=30 each) in online settings using the participants’ web-browsers. A cumulative semantic interference (CSI) paradigm was employed that required naming several exemplars of semantic categories within a seemingly unrelated sequence of objects. Reaction time (RT) is expected to increase linearly for each additional exemplar of a category. In experiment 1, CSI effects in naming times described in lab-based studies were replicated. In experiment 2, the responses were typed on participants’ computer keyboards and the first correct key press was used for RT analysis. This novel response assessment yielded a qualitatively similar, very robust CSI effect. Besides technical ease of application, collecting typewritten responses and automatic data preprocessing substantially reduce the work load for language production research. Results of both experiments open new perspectives for research on RT-effects in language experiments across a wide range of contexts. JavaScript- and R-based implementations for data collection and processing are available for download.

show abstract

mentioning

confidence: 68%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Web-based language production experiments: Semantic interference assessment is robust for spoken and typed response modalities

Stark¹,

C²,

Obrig³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

REPP: A robust cross-platform solution for online sensorimotor synchronization experiments

Anglada-Tort

Harrison

Jacoby

2021

Preprint

View full text Add to dashboard Cite

Sensorimotor synchronization (SMS), the rhythmic coordination of perception and action, is a fundamental human skill that supports many behaviors, from daily repetitive routines to the most complex behavioural coordination, including music and dance (Repp 2005; Repp & Su, 2013). Research on SMS has been mostly conducted in the laboratory using finger tapping paradigms, where participants typically tap with their index finger to a rhythmic sequence of auditory stimuli. However, these experiments require equipment with high temporal fidelity to capture the asynchronies between the time of the tap and the corresponding cue event. Thus, SMS is particularly challenging to study with online research, where variability in participants’ hardware and software can introduce uncontrolled latency and jitter into recordings. Here we present REPP (Rhythm ExPeriment Platform), a novel technology for measuring SMS in online experiments that can work efficiently using the built-in microphone and speakers of standard laptop computers. The audio stimulus (e.g., a metronome or a music excerpt) is played through the speakers and the resulting signal is recorded along with participants’ responses in a single channel. The resulting recording is then analyzed using signal processing techniques to extract and align timing cues with high temporal accuracy. This analysis is fully automated and customizable, enabling researchers to monitor online experiments in real time and to implement a wide variety of SMS paradigms. In this paper, we validate REPP through a series of calibration and behavioural experiments. We demonstrate that our technology achieves high temporal accuracy (latency and jitter within 2 ms on average), high test-retest reliability both in the laboratory (r = .87) and online (r = .80), and high concurrent validity (r = .94). We also suggest methods to ensure high data quality in online SMS experiments using REPP while minimizing recruitment costs. REPP can therefore open new avenues for research on SMS that would be nearly impossible in the laboratory, reducing experimental costs while massively increasing the reach, scalability and speed of data collection.

show abstract

Conducting Language Production Research Online: A Web-based Study of Semantic Context and Name Agreement Effects in Multi-Word Production

Meyer

Creemers

et al. 2021

Collabra: Psychology

View full text Add to dashboard Cite

Few web-based experiments have explored spoken language production, perhaps due to concerns of data quality, especially for measuring onset latencies. The present study highlights how speech production research can be done outside of the laboratory by measuring utterance durations and speech fluency in a multiple-object naming task when examining two effects related to lexical selection: semantic context and name agreement. A web-based modified blocked-cyclic naming paradigm was created, in which participants named a total of sixteen simultaneously presented pictures on each trial. The pictures were either four tokens from the same semantic category (homogeneous context), or four tokens from different semantic categories (heterogeneous context). Name agreement of the pictures was varied orthogonally (high, low). In addition to onset latency, five dependent variables were measured to index naming performance: accuracy, utterance duration, total pause time, the number of chunks (word groups pronounced without intervening pauses), and first chunk length. Bayesian analyses showed effects of semantic context and name agreement for some of the dependent measures, but no interaction. We discuss the methodological implications of the current study and make best practice recommendations for spoken language production research in an online environment.

show abstract

Internet based language production research with overt articulation: Proof of concept, challenges, and practical advice

Cited by 8 publications

References 20 publications

Web-based language production experiments: Semantic interference assessment is robust for spoken and typed response modalities

Web-based language production experiments: Semantic interference assessment is robust for spoken and typed response modalities

REPP: A robust cross-platform solution for online sensorimotor synchronization experiments

Conducting Language Production Research Online: A Web-based Study of Semantic Context and Name Agreement Effects in Multi-Word Production

Contact Info

Product

Resources

About