Is Omega Squared Less Biased? a Comparison of Three Major Effect Size Indices in One-Way Anova

Okada, Kensuke

doi:10.2333/bhmk.40.129

Cited by 91 publications

(90 citation statements)

References 23 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous simulation study results (Carroll & Nordholm, 1975;Keselman, 1975;Okada, 2013;Skidmore & Thompson, 2013) appear to support these recommendations. For example, Skidmore and Thompson conducted a simulation study to evaluate the bias of the effect size estimators when assumptions of ANOVA are violated, and summarized their results as follows: BOverall, our results corroborate the limited previous research (Carroll & Nordholm, 1975;Keselman, 1975) and suggest that η 2 should not be used as an ANOVA effect size estimator, because across the range of conditions we examined, η 2 had considerable sampling error bias^(p. 544).…”

supporting

confidence: 60%

“…Following previous simulation studies (Keselman, 1975;Okada, 2013), we consider a one-factor, between-subjects ANOVA design with four levels and manipulate three experimental factors: (a) three levels of population effect size, (b) two levels of population mean variability, and (c) Kelley (1935) SS sum of squares, df degrees of freedom, and MS mean squares. For the subscripts, T total, M treatment (or means), and E error three levels of sample size.…”

Section: Methodsmentioning

confidence: 99%

“…Following previous simulation studies (Keselman, 1975;Okada, 2013), two conditions were included as experimental factors. Although Cohen also considered minimum variability, it is not used in the present study.…”

Section: Population Mean Variabilitymentioning

confidence: 99%

See 2 more Smart Citations

Negative estimate of variance-accounted-for effect size: How often it is obtained, and what happens if it is treated as zero

Okada

2016

Behav Res

Self Cite

View full text Add to dashboard Cite

Researchers recommend reporting of biascorrected variance-accounted-for effect size estimates such as omega squared instead of uncorrected estimates, because the latter are known for their tendency toward overestimation, whereas the former mostly correct this bias. However, this argument may miss an important fact: A bias-corrected estimate can take a negative value, and of course, a negative variance ratio does not make sense. Therefore, it has been a common practice to report an obtained negative estimate as zero. This article presents an argument against this practice, based on a simulation study investigating how often negative estimates are obtained and what are the consequences of treating them as zero. The results indicate that negative estimates are obtained more often than researchers might have thought. In fact, they occur more than half the time under some reasonable conditions. Moreover, treating the obtained negative estimates as zero causes substantial overestimation of even bias-corrected estimators when the sample size and population effect are not large, which is often the case in psychology. Therefore, the recommendation is that researchers report obtained negative estimates as is, instead of reporting them as zero, to avoid the inflation of effect sizes in research syntheses, even though zero can be considered the most plausible value when interpreting such a result. R code to reproduce all of the described results is included as supplemental material.

show abstract

supporting

confidence: 60%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Negative estimate of variance-accounted-for effect size: How often it is obtained, and what happens if it is treated as zero

Okada

2016

Behav Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…Omega-squared ( " ) was used as an effect size measure for ANOVAs, as it is a less biased alternative to the more common eta-squared ( " ) (Okada, 2013).…”

Section: Resultsmentioning

confidence: 99%

Attitudes of Austrian psychotherapists towards process and outcome monitoring

Kaiser¹,

Schmutzhart²,

Laireiter³

2017

Preprint

View full text Add to dashboard Cite

While monitoring systems in psychotherapy have become more common, little is known about the attitudes that mental health practitioners have towards these systems. In an online survey among 111 Austrian psychotherapists and trainees, attitudes towards therapy monitoring were measured. A well-validated questionnaire measuring attitudes towards outcome monitoring, the outcome measurement questionnaire (OMQ), was used. Clinicians' theoretical orientations as well as previous knowledge and experience with monitoring systems were associated with positive attitudes towards monitoring. Possible factors that may have led to these findings, like the views of different theoretical orientations or obstacles in Austrian public health care, are discussed.

show abstract

“…If this meta-analysis included all studies that were left unpublished (although a few were located), the averaged effect size potentially would be lower, as we see with some metaanalytic techniques that correct for selective reporting. Okada (2013) posited that the use of eta squared is not necessarily the best estimator to use, with eta squared being slightly biased, especially in the case of small sample sizes. Even after pooling primary effect sizes by inverse variance and controlling for experiments with small sample sizes, biased eta squared estimates still could have inflated the meta-analytic estimates of the current project.…”

Section: Limitationsmentioning

confidence: 99%