Small Sample Size Solutions

Abstract

Researchers often have difficulties collecting enough data to test their hypotheses, either because target groups are small or hard to access, or because data collection entails prohibitive costs. Such obstacles may result in data sets that are too small for the complexity of the statistical model needed to answer the research question. This unique book provides guidelines and tools for implementing solutions to issues that arise in small sample research. Each chapter illustrates statistical methods that allow researchers to apply the optimal statistical model for their research question when the sample is too small. This essential book will enable social and behavioral science researchers to test their hypotheses even when the statistical model required for answering their research question is too complex for the sample sizes they can collect. The statistical models in the book range from the estimation of a population mean to models with latent variables and nested observations, and solutions include both classical and Bayesian methods. All proposed solutions are described in steps researchers can implement with their own data and are accompanied with annotated syntax in R. The methods described in this book will be useful for researchers across the social and behavioral sciences, ranging from medical sciences and epidemiology to psychology, marketing, and economics.

Dwarfs on the Shoulders of Giants: Bayesian Analysis With Informative Priors in Elite Sports Research and Decision Making

“…However, it may be argued that if there is sufficient evidence in support of a specific intervention to be tested in elite athletes, this “range of expectable values” may also merit to be considered as part of a new state of knowledge after data have been collected. In other words, Bayesian analysis with informative priors is particularly appropriate if the analytical aim is on decision making rather than on generalizable inference (Senn, 2011 ), available data on the level of interest is scarce (Sottas et al, 2007 ; Van de Schoot, 2020 ), and supposedly applicable prior knowledge is available (Aitken and Taroni, 2004 ; Hecksteden et al, 2017 ). As already pointed out in the introduction, this situation regularly occurs in elite sports.…”

“…Prior distributional parameters for the efficacy of whole-body CWI (β CWI ) are based on the metaanalysis published by Poppendieck et al ( 2013 ) the rationale being to characterise the range of plausible values pre-trial (Wagenmakers et al, 2016 ; Hecksteden et al, 2017 ; Van de Schoot, 2020 ). A detailed account is provided as a Supplementary File .…”

“…A comprehensive discussion of dealing with small sample sizes in an elite sports context has been recently published elsewhere (Hecksteden et al, 2021 ). Two particularly promising options are study designs that include some form of replication on the individual level (e.g., a replicated cross-over design) (Hilgers et al, 2018 ; Hecksteden et al, 2021 ) and a switch from mainstream frequentist to Bayesian statistics (Greenland, 2006 ; Greenland et al, 2016 ; Van de Schoot, 2020 ). Collecting several observations per individual reduces the number of study participants needed to get enough datapoints for group-based analyses (Hilgers et al, 2018 ; Hecksteden et al, 2021 ).…”

“…Bayesian statistics offer several advantages for small trials, most prominently the possibility to formally include pre-existing knowledge about the effect of interest (e.g., the efficacy of an intervention). This “informative prior” may be derived from a variety of sources e.g., from larger trials in sub-elite athletes, meta-analyses or in some cases from routine data (Hecksteden et al, 2017 ) or expert judgement (Van de Schoot, 2020 ). As a result, data analysis for a given trial (or one specific athlete) (Sottas et al, 2007 ; Hecksteden et al, 2017 ; Barth et al, 2019 ) does not have to start from zero.…”

“…The frequentist counterparts, confidence intervals, are often interpreted that way, but their true meaning is less straight forward (“If the experiment was repeated many times, a proportion equal to the confidence level would include the true value”). Finally, the Bayesian notion of probability as a subjective believe contingent on the current state of knowledge is less dependent on many datapoints as compared to the frequentist “long run relative frequency.” Although this does not eliminate the advantages of a high number of observations, Bayesian analyses are generally less dependent on sample size (Van de Schoot, 2020 ). Box 1 summarises relevant advantages of Bayesian analyses with informative priors in elite sports.…”