Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication

Php, Hanel; Mehler, David Marc Anton

doi:10.31234/osf.io/3mv26

Cited by 10 publications

(14 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our opinion, standardized effect sizes can be useful tools for interpreting data when thoughtfully employed by the scientists reporting them. However, sport and exercise scientists should be careful when selecting the appropriate SMD or effect size, and ensure that their choice effectively communicates the effect of interest ( Hanel & Mehler, 2019 ). Herein, we will discuss things to consider when reporting an SMD, and we will close by providing general recommendations and examples that we believe sport and exercise scientists will find useful.…”

Section: Should I Report a Standardized Mean Difference?mentioning

confidence: 99%

A case against default effect sizes in sport and exercise science

Caldwell

Vigotsky

2020

PeerJ

View full text Add to dashboard Cite

Recent discussions in the sport and exercise science community have focused on the appropriate use and reporting of effect sizes. Sport and exercise scientists often analyze repeated-measures data, from which mean differences are reported. To aid the interpretation of these data, standardized mean differences (SMD) are commonly reported as a description of effect size. In this manuscript, we hope to alleviate some confusion. First, we provide a philosophical framework for conceptualizing SMDs; that is, by dichotomizing them into two groups: magnitude-based and signal-to-noise SMDs. Second, we describe the statistical properties of SMDs and their implications. Finally, we provide high-level recommendations for how sport and exercise scientists can thoughtfully report raw effect sizes, SMDs, or other effect sizes for their own studies. This conceptual framework provides sport and exercise scientists with the background necessary to make and justify their choice of an SMD.

show abstract

Section: Should I Report a Standardized Mean Difference?mentioning

confidence: 99%

A case against default effect sizes in sport and exercise science

Caldwell

Vigotsky

2020

PeerJ

View full text Add to dashboard Cite

show abstract

“…CLES is particularly useful because it directly conveys the direction and variability of change scores without suggesting that the mean difference itself is small or large. Further, current evidence would suggest that the CLES is easier for readers to comprehend than a signal-to-noise SMD (Hanel and Mehler, 2019).…”

Section: Alternative To the Signal-to-noise Standardized Mean Differencementioning

confidence: 87%

“…Going back to our earlier example (d z = 11.62 and 0.25 respectively), the CLES would be approximately > 99% and 59.9%, or the probability of a randomly sample individual undergoing an improvement is > 99% or 59.9% for intervention 1 and 2, respectively. As Hanel and Mehler (2019) demonstrated, the CLES may be a more intuitive description of the signal-to-noise SMD.…”

Section: Recommendations For Reporting Effect Sizesmentioning

confidence: 96%

“…In our opinion, standardized effect sizes can be useful tools for interpreting data when thoughtfully employed by the scientists reporting them. However, sport and exercise scientists should be careful when selecting the appropriate SMD or effect size, and ensure that their choice effectively communicates the effect of interest (Hanel and Mehler, 2019). Herein, we will discuss things to consider when reporting an SMD, and we will close by providing general recommendations and examples that we believe sport and exercise scientists will find useful.…”

Section: Comments On Standardized Mean Differences In Sport and Exercmentioning

confidence: 99%

See 1 more Smart Citation

Peer Review #1 of "A case against default effect sizes in sport and exercise science (v0.2)"

Jukic¹

2020

View full text Add to dashboard Cite

Recent discussions in the sport and exercise science community have focused on the appropriate use and reporting of effect sizes. Sport and exercise scientists often analyze repeated-measures data, from which mean differences are reported. To aid the interpretation of these data, standardized mean differences (SMD) are commonly reported as description of effect size. The use of SMDs has been criticized by sport and exercise scientists as well as those in other disciplines. However, we believe, when thoughtfully applied, SMDs can be a useful analytical tool for sport and exercise scientists. In this manuscript, we hope to alleviate some confusion by demonstrating how effect sizes-primarily SMDs-are calculated, and describing their statistical properties. Finally, we provide high-level recommendations for how sport and exercise scientists can thoughtfully report raw effect sizes, SMDs, or other effect sizes for their own studies.

show abstract

“…), results such as "sprint speed increased (d = 0.47)", or "Hedge's g indicated the effect was large" we suspect are not easily interpreted by coaches, practitioners, or athletes. In fact, some research has shown that though non-scientists find certain standardised effect sizes more 'informative' than others, 38 most are relatively unim-pressed by even 'large' standardised effects 39 . The use of standardised effect sizes likely makes decision making difficult regarding the problem that a coach is looking to solve and ultimately is counterintuitive to the goal of sports scientists.…”

Section: Barriers To Answering This Question (Between Person) Correlamentioning

confidence: 99%

Does increasing an athletes’ strength improve sports performance? A critical review with suggestions to help answer this, and other, causal questions in sport science

Steele¹,

Fisher²,

Crawford³

2020

Journal of Trainology

View full text Add to dashboard Cite

Researchers and practitioners in sports science aim to generate, and apply, knowledge to improve sports performance. One area of interest is the role that muscular strength, and thus approaches to improve this (i.e. resistance training), has upon sports performance. In this review we briefly consider the evidence regarding an answer to the causal question "Does increasing an athletes' strength improve sports performance?". Design & Methods: We first consider the Applied Research Model for the Sport Sciences (ARMSS) to frame the problem and answer this. We then highlight barriers to answering it (and other causal questions) before offering suggestions to address these. Results: Muscular strength typically differentiates elite and non-elite athletes, and is correlated with proxy measures of sports performance. However, there is insufficient evidence to make a definitive statement regarding the causal effect of muscular strength upon sports performance. Conclusions: Considering the ARMSS, evidence is lacking whether improving muscular strength is causally related to sports performance. Present evidence is primarily observational and cross-sectional, experimental evidence is limited and focused upon proxy measures of sports performance, primarily conducted in small samples, and with little consideration regarding meaningfulness of effects. Suggestions to help improve research in this area and better answer this question include: larger sample sizes, determination of smallest effect sizes of interest for outcomes including muscular strength and proxy measures of sports performance (using both anchoring and/or expert opinion), and use of causal inference methods for observational data (actual sports performance, performance indicators, and fitness measures) including graphical causal diagrams and mediation analysis.

show abstract

Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication

Cited by 10 publications

References 22 publications

A case against default effect sizes in sport and exercise science

A case against default effect sizes in sport and exercise science

Peer Review #1 of "A case against default effect sizes in sport and exercise science (v0.2)"

Does increasing an athletes’ strength improve sports performance? A critical review with suggestions to help answer this, and other, causal questions in sport science

Contact Info

Product

Resources

About