Researchers often need to consider the practical significance of a relationship. For example, interpreting the magnitude of an effect size or establishing bounds in equivalence testing requires knowledge of the meaningfulness of a relationship. However, there has been little research exploring the degree of relationship among variables (e.g., correlation, mean difference) necessary for an association to be interpreted as meaningful or practically significant. In this study, we presented statistically trained and untrained participants with a collection of figures that displayed varying degrees of mean difference between groups or correlations among variables and participants indicated whether or not each relationship was meaningful. The results suggest that statistically trained and untrained participants differ in their qualification of a meaningful relationship, and that there is significant variability in how large a relationship must be before it is labeled meaningful. The results also shed some light on what degree of relationship is considered meaningful by individuals in a context-free setting.

An effect size (ES) provides valuable information regarding the magnitude of effects, with the interpretation of magnitude being the most important. Interpreting ES magnitude requires combining information from the numerical ES value and the context of the research. However, many researchers adopt popular benchmarks such as those proposed by Cohen. More recently, researchers have proposed interpreting ES magnitude relative to the distribution of observed ESs in a specific field, creating unique benchmarks for declaring effects small, medium or large. However, there is no valid rationale whatsoever for this approach. This study was carried out in two parts: (1) We identified articles that proposed the use of field-specific ES distributions to interpret magnitude (primary articles); and (2) We identified articles that cited the primary articles and classified them by year and publication type. The first type consisted of methodological papers. The second type included articles that interpreted ES magnitude using the approach proposed in the primary articles. There has been a steady increase in the number of methodological and substantial articles discussing or adopting the approach of interpreting ES magnitude by considering the distribution of observed ES in that field, even though the approach is devoid of a theoretical framework. It is hoped that this research will restrict the practice of interpreting ES magnitude relative to the distribution of ES values in a field and instead encourage researchers to interpret such by considering the specific context of the study.

The over-reliance on the null hypothesis significance testing framework and its accompanying tools has recently been challenged. An example of such a tool is statistical power analysis, which is used to determine how many participants are required to detect a minimally meaningful effect size in the population at a given level of power and Type I error rate. To investigate how power analysis is currently used, this study reviews the reporting of 443 power analyses in high impact psychology journals in 2016 and 2017. We found that many pieces of information required for power analyses are not reported, and effect sizes selected for the procedure are often chosen based on an inappropriate rationale. In light of these findings, we argue that the power analysis procedure forces researchers to compromise in the selection of the different pieces of information required. We offer that researchers should focus on tools beyond traditional power analysis when sample planning, such as precision-based power analysis or simply collecting the largest sample size possible.

Reporting and interpreting effect sizes (ESs) has been recommended by all major bodies within the field of psychology. In this systematic review, we investigated the reporting of effect sizes in six social-personality psychology journals from 2018, given that this area has been at the center of psychology's replication crisis. Our results highlight that although ES reporting is near perfect (even for follow-up tests), interpreting the magnitude of ESs, including confidence intervals for ESs, and interpreting the precision of the confidence intervals needs development.We also highlight widespread confusion regarding the interpretations of the magnitude of ESs within the context of the research.

No abstract