Improving Data Analysis in Second Language Acquisition by Utilizing Modern Developments in Applied Statistics

Larson‐Hall, Jenifer; Herrington, Richard

doi:10.1093/applin/amp038

Cited by 86 publications

(57 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies that examined whether power analyses were reported found that only a very small percentage of studies did so (2% in Plonsky & Gass, ; 1% in Plonsky, , and 7% in Ziegler, ). The proportion of studies that has reported checking statistical assumptions prior to employing inferential statistics is similarly small (3% in Plonsky & Gass, ; 17% in Plonsky, , and 36% in Ziegler, ), a result we find disturbing given the high co‐occurrence of parametric statistics on the one hand and non‐normal distributions, small samples, and outliers on the other (Larson‐Hall, ; Larson‐Hall & Herrington, ; Phakiti, ; Plonsky, ; Larson‐Hall, ; Plonsky et al., in press).…”

Section: Descriptive Statistics and Measures Of Study Qualitymentioning

confidence: 76%

Reporting and Interpreting Quantitative Research Findings: What Gets Reported and Recommendations for the Field

2015

Self Cite

View full text Add to dashboard Cite

This paper presents a set of guidelines for reporting on five types of quantitative data issues: (1) Descriptive statistics, (2) Effect sizes and confidence intervals, (3) Instrument reliability, (4) Visual displays of data, and (5) Raw data. Our recommendations are derived mainly from various professional sources related to L2 research but motivated by results from investigations into how well the field as a whole is following these guidelines for best methodological practices, and illustrated by L2 examples. Although recent surveys of L2 reporting practices have found that more researchers are including important data such as effect sizes, confidence intervals, reliability coefficients, research questions, a priori alpha levels, graphics, and so forth in their research reports, we call for further improvement so that research findings may build upon each other and lend themselves to meta-analyses and a mindset that sees each research project in the context of a coherent whole.Keywords confidence intervals; effect sizes; language; L2; mean; meta-analysis; methodology; quantitative; raw data; reporting practices; standard deviation IntroductionThorough and consistent reporting practices are critical to advancing and informing second language (L2) theory and practice. In this chapter we call for quantitative L2 researchers to adopt a synthetic mindset-understanding that no research stands alone, that progress as a field happens best if we can pool research findings, and that reporting practices can either facilitate or impede Larson-Hall and PlonskyWhat Gets Reported and Recommendations for the Field this type of research. However, currently researchers have little in the way of L2-specific guidance or direction on what to present and how. As a baseline from which to assess L2 quantitative research, we will first introduce reporting practices based on recommendations from various sources related to L2 research. Next, we will review how well the field as a whole follows these guidelines by presenting results from secondary/meta-analytic research. We then propose a set of recommendations for improving the means by which quantitative L2 data are reported and interpreted. We do this for five main areas of quantitative reporting: (1) Descriptive statistics and other measures of study quality such as power, (2) Effect sizes and confidence intervals (CIs), (3) Instrument reliability, (4) Visual displays of data, and (5) Raw data.

show abstract

Section: Descriptive Statistics and Measures Of Study Qualitymentioning

confidence: 76%

Reporting and Interpreting Quantitative Research Findings: What Gets Reported and Recommendations for the Field

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…Chaudron, 2001;Larson-Hall & Herrington, 2009;Nassaji, 2012;Norris, 2015;Norris & Ortega, 2000, 2006. This review joins the chorus.…”

Section: Key Findings Of Other Recent Journal Surveysmentioning

confidence: 97%

Inferential statistics in Language Teaching Research: A review and ways forward

Lindstromberg

2016

Language Teaching Research

View full text Add to dashboard Cite

This article reviews all (quasi)experimental studies appearing in the first 19 volumes (1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015) of Language Teaching Research (LTR). Specifically, it provides an overview of how statistical analyses were conducted in these studies and of how the analyses were reported. The overall conclusion is that there has been a tight adherence to traditional methods and practices, some of which are suboptimal. Accordingly, a number of improvements are recommended. Topics covered include the implications of small average sample sizes, the unsuitability of p values as indicators of replicability, statistical power and implications of low power, the non-robustness of the most commonly used significance tests, the benefits of reporting standardized effect sizes such as Cohen's d, options regarding control of the familywise Type I error rate, analytic options in pretest-posttest designs, 'meta-analytic thinking' and its benefits, and the mistaken use of a significance test to show that treatment groups are equivalent at pretest. An online companion article elaborates on some of these topics plus a few additional ones and offers guidelines, recommendations, and additional background discussion for researchers intending to submit to LTR an article reporting a (quasi)experimental study. KeywordsEffect sizes, L2 quantitative research, pretest-posttest designs, (quasi)experimental studies, robust methods, small sample sizes, statistical analysis, statistical power, testing for baseline balance I IntroductionThis article reports a survey of all issues of Language Teaching Research (LTR) from the first issue in 1997 through the latest issue (at the time of writing) of 2015, including special issues. The main aims are (1) to outline of how LTR authors have subjected data

show abstract

“…Researchers regularly violate the basic assumptions of quantitative reasoning, and application of quantitative inferential models lags well behind the mounting complexities of language science. Over the years, isolated voices have urged caution, revision, and evolution in our quantitative practices (e.g., Brown, ; Gries, ; Hudson, ; Larson‐Hall & Herrington, ; Norris & Ortega, , ; Plonsky, ; Plonsky & Gass, ), though the mainstream of L2 quantitative researchers has paid scant attention to their recommendations. At the same time, new approaches to using quantitative models and inferential techniques for capturing the complexity of L2 data are emerging (e.g., Cunnings, ; Gries, ; Gudmestad, House, & Geeslin, ; Verspoor, de Bot, & Lowie, ), but the extent to which they may be taken up by applied linguists is limited by a generalized lack of quantitative training.…”

Section: Improving Quantitative Reasoningmentioning

confidence: 99%