Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction Research

Plonsky, Luke; Gass, Susan M.

doi:10.1111/j.1467-9922.2011.00640.x

Cited by 213 publications

(284 citation statements)

References 102 publications

(134 reference statements)

Supporting

Mentioning

268

Contrasting

Unclassified

Order By: Relevance

“…Methods for the Systematic Review Searching We focused our search on academic, peer-reviewed journals because we wanted to examine the extent of self-labeled replication in this medium, which has been identified as the primary channel for disseminating L2 research (Smith & Lafford, 2009;VanPatten & Williams, 2002). We therefore excluded replications in books, dissertations, conference proceedings, and the like, following procedures used in previous syntheses in the field (e.g., Plonsky & Gass, 2011;Plonsky, 2013). Admittedly, this left our sample susceptible to the effects of potential publication bias among journals.…”

Section: A Systematic Review Of Self-labeled Replication Aimsmentioning

confidence: 99%

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

et al. 2018

View full text Add to dashboard Cite

Despite its critical role for the development of the field, little is known about replication in second language (L2) research. To better understand replication practice, we first provide a narrative review of challenges related to replication, drawing on recent developments in psychology. This discussion frames and motivates a systematic review, building on syntheses of replication in psychology, education, and L2 research. We coded 67 self-labeled L2 replication studies found across 26 journals for 136 characteristics. We estimated a mean rate of 1 published replication study for every 400 articles, with a mean of 6.64 years between initial and replication studies and a mean of 117 citations of the initial study before a replication was published. Replication studies had an annual mean of 7.3 citations, much higher than averages in linguistics and education. Overlap in authorship between initial and replication studies and the availability of the initial materials both increased the likelihood of a replication supporting the initial findings. Our sample contained no direct (exact) replication attempts, and changes made to initial studies were numerous and wide ranging, which likely obscured, if not undermined, the interpretability of replication studies. To improve the amount and quality of L2 replication research, we propose 16 recommendations relating to rationale, nomenclature, design, infrastructure, and incentivization for collaboration and publication.

show abstract

Section: A Systematic Review Of Self-labeled Replication Aimsmentioning

confidence: 99%

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Bearing this in mind, the following suggestions for English SBI research are offered, many of which encompass previous suggestions for more general reform of EFL research efforts (see, e.g., Ellis;Norris & Ortega, 2000Plonsky & Gass, 2011): First of all, the value of English SBI depends on whether its effects last over time or not; few study samples in this meta-analysis included delayed posttests; thus, the need for more measurements of the persisting summary effects of English SBI is felt.…”

Section: Resultsmentioning

confidence: 99%

The Effectiveness of Strategy-Based Instruction in Teaching English as a Second or Foreign Language: A Meta-analysis of Experimental Studies

Shirvan¹,

Ghonsooly²,

Fatemi³

2016

UCFL

View full text Add to dashboard Cite

A large number of single research studies on the effects of strategy-based instruction (SBI) teaching English as a foreign or second language have been conducted so far. However, the lack of a comprehensive meta-analysis of experimental studies targeting the effectiveness of English language SBI is observed. Therefore, this study aimed to meta-analyze the effects of SBI experimental studies conducted in the field of learning English as a foreign or second language to clarify the aroused controversial issues regarding their overall effectiveness and their related variables. After a comprehensive search to collect the population of experimental SBI of English language studies, effect sizes under random effects model in terms of Hedge's g were calculated for 26 primary studies, contributing a total of 84 independent effect sizes. The findings indicated that summary effect size of studies with learners at different proficiency levels turned out to be bigger than 0.60. Furthermore, SBI in English as a foreign language context was more effective than the one under English as a second language context. Finally, SBI study samples with treatment time of more than two weeks and English language learners over the age of 14 were more effective than those with less than two week treatment time and learners under the age of 14.

show abstract

“…However, findings of recent surveys of the SLA research literature (e.g. Plonsky & Gass, 2011;Plonsky, 2013;Lindstromberg, 2016) do not suggest that the community of SLA researchers has the resources to conduct enough large-scale RCTs to address a good-sized proportion, let alone all, of the many worthwhile hypotheses that have been, and are being, generated. Finally, we have looked at the sampling variation of p values derived from the IS t-test.…”

Section: What An Initial P Value Foretells About a Replication P Valuementioning

confidence: 99%

“…the reliability of measurements, experimental design, and the method of statistical analysis). Focusing on observational and quasi-experimental studies relating to Long's (1983) interaction hypothesis, Plonsky and Gass (2011) surveyed 174 articles in 14 applied linguistics journals (plus two edited volumes) from the period 1980 to 2010, finding that the mean number of learners per learning condition was 22. In a survey embracing 606 quantitative studies published in two top applied linguistics journals, Plonsky (2013) found that the median number of learners per learning condition was 19.…”

Section: Introductionmentioning

confidence: 99%

The particular need for replication in the quantitative study of SLA: A case study of the mnemonic effect of assonance in collocations

Lindstromberg¹,

Eyckmans

2017

Journal of the European Second Language Association

View full text Add to dashboard Cite

. The particular need for replication in the quantitative study of SLA: A case study of the mnemonic effect of assonance in collocations. Journal of the European Second Language Association, 1(1), 126-136, DOI: https://doi.org/10.22599/jesla.26 SYSTEMATIC REVIEW AND META-ANALYSESThe particular need for replication in the quantitative study of SLA: A case study of the mnemonic effect of assonance in collocations Seth Lindstromberg * and June Eyckmans † Recent surveys of published reports of quasi-experimental studies of second language acquisition (SLA) indicate that low statistical power is pervasive owing in large part to small average sample sizes. The surveys do not indicate a marked trend toward samples that are larger. After illustrating the problem of low power in SLA research, we review arguments that increased replication of original studies can enable small-sample quantitative researchers to make firmer contributions to the field of SLA, especially if estimation of effect sizes and the practice of on-going statistical meta-analysis become routine. As a case study, we describe a series of small-sample quasi-experiments of which the first five found a short term positive mnemonic effect of interword, intra-phrase vowel repetition (or assonance) on learners' retention of the forms of L2 collocations (e.g. strong bond vs. firm hold), whereas a sixth study newly reported here found negative effects. The case study illustrates the roles of replication and meta-analysis in successive re-adjustments of an original estimate. More specifically, the case study illustrates a metaanalytic approach to making sense of conflicting outcomes. All in all, it illustrates why small-sample researchers need to adopt a more long-term view.

show abstract

Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction Research

Cited by 213 publications

References 102 publications

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

The Effectiveness of Strategy-Based Instruction in Teaching English as a Second or Foreign Language: A Meta-analysis of Experimental Studies

The particular need for replication in the quantitative study of SLA: A case study of the mnemonic effect of assonance in collocations

Contact Info

Product

Resources

About