Constructing Orthogonal Replications for Variance Estimation

Gurney, Margaret; Jewett, Robert S.

doi:10.1080/01621459.1975.10480308

Cited by 25 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The variance estimate (eq ) is similar to that used in the method of balanced repeated replications , for estimating the variance of a nonlinear statistic in stratified random sampling, where orthogonal replications are employed instead of random replications as above. Note that the variance estimate of Δ F̂ could be S /4, because the variance of Δ F̂ ( j 1 , ..., j M ) is that of Δ F̂ multiplied by 4 according to different sample sizes.…”

Section: Methodsmentioning

confidence: 99%

Ligand Binding Thermodynamic Cycles: Hysteresis, the Locally Weighted Histogram Analysis Method, and the Overlapping States Matrix

Cui

Zhang

Tan

et al. 2019

J. Chem. Theory Comput.

View full text Add to dashboard Cite

Free energy perturbation (FEP) simulations have been widely applied to obtain predictions of the relative binding free energy for a series of congeneric ligands binding to the same receptor, which is an essential component for the lead optimization process in computer-aided drug discovery. In the case of several congeneric ligands forming a perturbation map involving a closed thermodynamic cycle, the summation of the estimated free energy change along each edge in the cycle using BAR (Bennett acceptance ratio) usually will deviate from zero due to systematic and random errors, which is the hysteresis of cycle closure. In this work, the advanced reweighting techniques UWHAM (binless weighted histogram analysis method) and LWHAM (locally weighted histogram analysis method) are applied to provide statistical estimators of the free energy change along each edge in order to eliminate the hysteresis effect. As an example, we analyze a closed thermodynamic cycle involving four congeneric ligands which bind to HIV-1 integrase, a promising target which has emerged for antiviral therapy. We demonstrate that compared with FEP and BAR, more accurate and hysteresis-free estimates of free energy differences can be achieved by using UWHAM to find a single estimate of the density of states based on all of the data in the cycle. Furthermore, by comparison of LWHAM results obtained from the inclusion of different numbers of neighboring states with UWHAM estimation involving all the states, we show how to determine the optimal neighborhood size in the LWHAM analysis to balance the tradeoffs between computational cost and accuracy of the free energy prediction. Even with the smallest neighborhood, LWHAM can improve the BAR free energy estimates using the same input data as BAR. We introduce an overlapping states matrix that is constructed by using the global jump formula of LWHAM and plot its heat map. The heat map provides a quantitative measure of the overlap between pairs of alchemical/thermodynamic states. We explain how to identify and improve the FEP calculations along the edges that most likely cause large systematic errors by using the heat map of the overlapping states matrix and by comparing the BAR and UWHAM estimates of the free energy change.

show abstract

Section: Methodsmentioning

confidence: 99%

Ligand Binding Thermodynamic Cycles: Hysteresis, the Locally Weighted Histogram Analysis Method, and the Overlapping States Matrix

Cui

Zhang

Tan

et al. 2019

J. Chem. Theory Comput.

View full text Add to dashboard Cite

show abstract

“…assumption, which limits the applicability of empirical evaluation. Note that derivation present in this paper is adapted from [10,4,29].…”

Section: Balanced Half Sampling Methodsmentioning

confidence: 99%

“…The condition for GBHS method is that number of PSU must be a prime integer and n samples are selected from each stratum. Based on [10,29], an extension on Equation 7 of population mean with GBHS method can be defined,…”

Section: Balanced Half-sample Variance Estimationmentioning

confidence: 99%

Efficient benchmarking of content-based image retrieval via resampling

Shen

Shepherd

2006

Proceedings of the 14th ACM International Conference on Multimedia

View full text Add to dashboard Cite

While content-based image retrieval (CBIR) is an expanding field, and new approaches to ever more effective retrieval are frequently proposed, relatively little attention has so far been paid to the process of evaluating the effectiveness of CBIR methods. Most of the reported evaluations use standard IR evaluation methodologies, with little consideration of their statistical significance or appropriateness for CBIR, which makes it difficult to assess the precise impact of individual methods. In this paper, we present a new approach for evaluating CBIR systems which provides both efficient and statistically-sound performance evaluation. The approach is based on stratified sampling, and provides a significant improvement over existing evaluation approaches. Comprehensive experiments using our approach to evaluate a range of CBIR methods have shown that the approach reduces not only the estimation error, but also reduces the size of the test data set required to achieve specific estimation error levels.

show abstract

“…The set of R balanced half-samples used may be defined by an R x L design matrix (a;), 1 5 r 5 R and 1 5 h 5 L, where 6; = + 1 or -1 depending on whether the first or second sample psu in the hth stratum is in the rth half-sample and xr = 0 for all h # h'. Gurney and Jewett (1975) extended BRR to nh = q (a prime) for all h, but we confine ourselves in this article to the important special case of nh = 2 for all h.…”

Section: Brr Estimators Ofmentioning

confidence: 98%

Inference from Stratified Samples: Second-Order Analysis of Three Methods for Nonlinear Statistics

Rao

1985

Journal of the American Statistical Association

View full text Add to dashboard Cite

For stratified samples and nonlinear statistics that can be expressed as functions of estimated totals, second-order asymptotic expansions of the linearization, jackknife, and balanced repeated-replication variance estimators are obtained. Based on these, comparisons are made in terms of their biases. Some higher order asymptotic equivalence results are also established. The special case of a combined ratio estimator is investigated in detail. Some results on bias reduction achieved by the jackknife and balanced repeated-replication estimators of a nonlinear function of totals are also given.

show abstract

Constructing Orthogonal Replications for Variance Estimation

Cited by 25 publications

References 1 publication

Ligand Binding Thermodynamic Cycles: Hysteresis, the Locally Weighted Histogram Analysis Method, and the Overlapping States Matrix

Ligand Binding Thermodynamic Cycles: Hysteresis, the Locally Weighted Histogram Analysis Method, and the Overlapping States Matrix

Efficient benchmarking of content-based image retrieval via resampling

Inference from Stratified Samples: Second-Order Analysis of Three Methods for Nonlinear Statistics

Contact Info

Product

Resources

About