Recent empirical studies have highlighted the large degree of analytic flexibility in data analysis that can lead to substantially different conclusions based on the same data set. Thus, researchers have expressed their concerns that these researcher degrees of freedom might facilitate bias and can lead to claims that do not stand the test of time. Even greater flexibility is to be expected in fields in which the primary data lend themselves to a variety of possible operationalizations. The multidimensional, temporally extended nature of speech constitutes an ideal testing ground for assessing the variability in analytic approaches, which derives not only from aspects of statistical modeling but also from decisions regarding the quantification of the measured behavior. In this study, we gave the same speech-production data set to 46 teams of researchers and asked them to answer the same research question, resulting in substantial variability in reported effect sizes and their interpretation. Using Bayesian meta-analytic tools, we further found little to no evidence that the observed variability can be explained by analysts’ prior beliefs, expertise, or the perceived quality of their analyses. In light of this idiosyncratic variability, we recommend that researchers more transparently share details of their analysis, strengthen the link between theoretical construct and quantitative system, and calibrate their (un)certainty in their conclusions.

No abstract

This exploratory study gives a first glance at the development of the perception of the Spanish /e/-/ei/ contrast by heritage learners in comparison to that of L2 learners in the classroom. To this end, two types of semester-long, explicit phonetic instruction training are compared: High Phonetic Variability Training (HPVT) with exposure to multiple sources of speech, and regular standalone phonetics courses with low variability of speech input (LPVT). Data from two identical pre-test and post-test ABX perceptual discrimination tasks were obtained from 27 students, as well as 7 control speakers whose primary language is Spanish. Results show that heritage learners perceive the contrast better than L2 learners, and that HPVT significantly improves the perception of the /e/-/ei/ contrast. Although heritage learners perform close to a native ceiling and do not significantly differ from native controls, the improvement from pre-test to post-test is larger in heritage learners enrolled in HPVT than LPVT training. These results suggest that, although the discrimination accuracy of Spanish /e/ and /ei/ is already high for heritage learners at the pre-test stage, High Phonetic Variability Training can be beneficial in the perceptual development of their heritage language, even matching their accuracy to that of native speakers.