Toward a replication culture: Speech production research in the classroom

Abstract

Our understanding of human sound systems is increasingly shaped by experimental studies. What we can learn from a single study, however, is limited. It is of critical importance to evaluate and substantiate existing findings in the literature by directly replicating published studies. Our publication system, however, does not reward direct replications in the same way as it rewards novel discoveries. Consequently, there is a lack of incentives for researchers to spend resources on conducting replication studies, a situation that is particularly true for speech production experiments, which often require resourceful data collection procedures and recording environments. In order to sidestep this issue, we propose to run direct replication studies with our students in the classroom. This proposal offers an easy and inexpensive way to conduct large-scale replication studies and has valuable pedagogical advantages for our students. To illustrate the feasibility of this approach, we report on two classroom-based replication studies on incomplete neutralization, a speech phenomenon that has sparked many methodological debates in the past. We show that in our classroom studies, we not only replicated incomplete neutralization effects, but our studies yielded effect magnitudes comparable to laboratory experiments and meta analytical estimates. We discuss potential challenges to this approach and outline possible ways to help us substantiate our scientific record.

Modelling L1 and the artificial language during artificial language learning

“…Some scientific subfields including linguistics discuss a so-called replication crisis since replicability rates for empirical studies have been found to be very low (Nieuwland et al, 2018;Stack et al, 2018). This problems calls for convincing accumulating evidence for the effects we report (Roettger and Baer-Henney, 2019;Nicenboim et al, 2018), and the present paper allows us to show valuable additional evidence for an effect that has been reported for English speakers and has replicated the original identity effect in German and Mandarin speakers. This study has therefore provided cross-linguistic evidence that the identity effect is potentially universal.…”

Modeling Dimensions of Prosodic Prominence

“…Incomplete neutralization in German is a classic case that questions the plausibility of this chain: The final obstruents of <Rad> /Kad/ ("wheel") and <Rat> /Kat/ ("advice") should be completely indistinguishable for the phonetic implementation module after the neutralization rule described for German has turned both forms into [Kat h ]. Numerous studies demonstrated that this is not the case and that there are indeed systematic acoustic differences between the two words such as voice onset time, closure duration, or the duration of the preceding vowel (see among others Port and O'Dell, 1985, Port and Crawford, 1989, Roettger et al, 2014, and Roettger and Baer-Henney, 2018, for Dutch: Ernestus and Baayen, 2006. In the modular view, the phonetic component should not be able to produce different signals based on the two phonological representations because they are identical.…”

Online pragmatic interpretations of scalar adjectives are affected by perceived speaker reliability

“…Because speakers vary in their ways of encoding their intentions into words, it is considered effective for comprehenders to flexibly adjust their interpretation of this mapping according to who the input is coming from. Although talker-based adaptation has been attested in speech perception (e.g., [ 42 , 43 , 52 – 58 ]) and syntactic priming and processing (e.g., [ 59 – 62 ]), it has only begun to be understood with respect to pragmatics.…”

“…Rather, comprehenders seem to be sensitive to the linguistic evidence of pragmatic unreliability or uncooperativeness and are able to spontaneously alter their implicit interpretive behaviors in response to the observed idiosyncrasy. We must note, however, that the current statistical approach, based on the frequentist view on statistical significance, comes with clear limitations with respect to interpretations of null effects [ 52 ]. What we can report here is that the results from the current experiments provide reasonable evidentiary support for our main hypothesis, while replications of the results will be desirable.…”