The Acceptability and Validity of AI-Generated Psycholinguistic Stimuli

Abstract

Sentence stimuli pervade psycholinguistics research. Yet limited attention has been paid to the automatic construction of sentence stimuli. Given their linguistic capabilities, this study investigated the efficacy of ChatGPT/AI tools in generating sentence stimuli. In three psycholinguistic experiments, this study examined the acceptability and validity of AI-formulated auditory sentences and written sentences in two languages: English and Arabic. Participants gave English AI-generated stimuli similar to or higher acceptability ratings than human-composed stimuli, but the opposite was observed for Arabic AI-generated stimuli. The validity of AI-developed stimuli relied on the study design, with only Experiment 2 demonstrating the target psycholinguistic effect (L1/L2 morphosyntactic prediction). These results highlight the promising role of AI as a stimuli developer which could facilitate psycholinguistic research and increase its diversity. Implications for psycholinguistic research were discussed.