Background
Researchers often misinterpret and misrepresent statistical outputs. This abuse has led to a large literature on modification or replacement of testing thresholds and P-values with confidence intervals, Bayes factors, and other devices. Because the core problems appear cognitive rather than statistical, we review some simple methods to aid researchers in interpreting statistical outputs. These methods emphasize logical and information concepts over probability, and thus may be more robust to common misinterpretations than are traditional descriptions.
Methods
We use the Shannon transform of the P-value p, also known as the binary surprisal or S-value s = −log2(p), to provide a measure of the information supplied by the testing procedure, and to help calibrate intuitions against simple physical experiments like coin tossing. We also use tables or graphs of test statistics for alternative hypotheses, and interval estimates for different percentile levels, to thwart fallacies arising from arbitrary dichotomies. Finally, we reinterpret P-values and interval estimates in unconditional terms, which describe compatibility of data with the entire set of analysis assumptions. We illustrate these methods with a reanalysis of data from an existing record-based cohort study.
Conclusions
In line with other recent recommendations, we advise that teaching materials and research reports discuss P-values as measures of compatibility rather than significance, compute P-values for alternative hypotheses whenever they are computed for null hypotheses, and interpret interval estimates as showing values of high compatibility with data, rather than regions of confidence.
Our recommendations emphasize cognitive devices for displaying the compatibility of the observed data with various hypotheses of interest, rather than focusing on single hypothesis tests or interval estimates. We believe these simple reforms are well worth the minor effort they require.
Relatively few investigations have reported purposeful overfeeding in resistance-trained adults. This preliminary study examined potential predictors of resistance training (RT) adaptations during a period of purposeful overfeeding and RT. Resistance-trained males (n = 28; n = 21 completers) were assigned to 6 weeks of supervised RT and daily consumption of a high-calorie protein/carbohydrate supplement with a target body mass (BM) gain of ≥0.45 kg·wk−1. At baseline and post-intervention, body composition was evaluated via 4-component (4C) model and ultrasonography. Additional assessments of resting metabolism and muscular performance were performed. Accelerometry and automated dietary interviews estimated physical activity levels and nutrient intake before and during the intervention. Bayesian regression methods were employed to examine potential predictors of changes in body composition, muscular performance, and metabolism. A simplified regression model with only rate of BM gain as a predictor was also developed. Increases in 4C whole-body fat-free mass (FFM; (mean ± SD) 4.8 ± 2.6%), muscle thickness (4.5 ± 5.9% for elbow flexors; 7.4 ± 8.4% for knee extensors), and muscular performance were observed in nearly all individuals. However, changes in outcome variables could generally not be predicted with precision. Bayes R2 values for the models ranged from 0.18 to 0.40, and other metrics also indicated relatively poor predictive performance. On average, a BM gain of ~0.55%/week corresponded with a body composition score ((∆FFM/∆BM)*100) of 100, indicative of all BM gained as FFM. However, meaningful variability around this estimate was observed. This study offers insight regarding the complex interactions between the RT stimulus, overfeeding, and putative predictors of RT adaptations.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.