Estimation of standard deviations and inverse‐variance weights from an observed range

Abstract

A variety of methods have been proposed to estimate a standard deviation, when only a sample range has been observed or reported. This problem occurs in the interpretation of individual clinical studies that are incompletely reported, and also in their incorporation into meta‐analyses. The methods differ with respect to their focus being either on the standard deviation in the underlying population or on the particular sample in hand, a distinction that has not been widely recognized. In this article, we contrast and compare various estimators of these two quantities with respect to bias and mean squared error, for normally distributed data. We show that unbiased estimators are available for either quantity, and recommend our preferred methods. We also propose a Taylor series method to obtain inverse‐variance weights, for samples where only the sample range is available; this method yields very little bias, even for quite small samples. In contrast, the naïve approach of simply taking the inverse of an estimated variance is shown to be substantially biased, and can place unduly large weight on small samples, such as small clinical trials in a meta‐analysis. Accordingly, this naïve (but commonly used) method is not recommended.

Standard error estimation in meta-analysis of studies reporting medians

“…see the open peer review report of Hozo et al 2 ), the downstream consequences of using naïve SE estimators has been largely unexplored in the literature because most simulation studies evaluating the transformation-based approaches (e.g. see the authors [2][3][4][5][6][7][8][9][10][11][12] ) have almost entirely focused on the performance of these methods at the study-level (i.e. for estimating the mean and standard deviation of the outcome distribution for a given study) rather than at the meta-analytic level.…”

“…Such approaches, which we refer to as transformationbased approaches, were first proposed and systematically evaluated by Hozo et al 2 and have been further developed by a number of authors in recent years. [3][4][5][6][7][8][9][10][11][12][13][14] Reflecting their widespread application, Google Scholar lists over 10,000 articles citing these transformation-based approaches [2][3][4][5][6][7][8][9][10][11][12][13][14] as of 1 May 2022.…”

Unified approach to optimal estimation of mean and standard deviation from sample summaries

“…While the work on this problem can be traced back all the way to Tippett, 5 interest in the systematic research of the estimators resurfaced in Hozo et al 6 and a significant body of the literature soon followed. [7][8][9][10][11][12][13][14][15][16][17][18][19] Typically, three scenarios are investigated, depending on the summaries being reported in an addition to a sample size: Scenario 1: { min , median, max}, Scenario 2: {first quartile, median, third quartile}, or Scenario 3: { min , first quartile, median, third quartile, max}.…”

Standard error estimation in meta-analysis of studies reporting medians

“…Then, one applies standard meta-analytic approaches with inverse-variance weighting based on the (imputed) study-specific sample means and standard deviations. Such approaches, which we refer to as transformation-based approaches, were first proposed and systematically evaluated by Hozo et al [2] and have been further developed by a number of authors in recent years [3,4,5,6,7,8,9,10,11,12]. Reflecting their widespread application, Google Scholar lists over 10,000 articles citing these transformation-based approaches (i.e., [2,3,4,5,6,7,8,9,10,11,12]) as of May 1 2022.…”

“…Even more explicitly, Hozo et al [2], Luo et al [6], McGrath et al [7], and Shi et al [9] used the the naïve SE estimator in data applications illustrating how to apply their proposed transformation-based approaches, for instance. While some have raised concerns over potential issues of the naïve SE estimator (e.g., see the open peer review report of Hozo et al [2]), the downstream consequences of using naïve SE estimators has been largely unexplored in the literature because most simulation studies evaluating the transformation-based approaches (e.g., see [2,3,4,5,6,7,8,9,10,11]) have almost entirely focused on the performance of these methods at the study-level (i.e., for estimating the mean and standard deviation of the outcome distribution for a given study) rather than at the meta-analytic level.…”