Modeling error in experimental assays using the bootstrap principle: understanding discrepancies between assays using different dispensing technologies

Abstract

All experimental assay data contains error, but the magnitude, type, and primary origin of this error is often not obvious. Here, we describe a simple set of assay modeling techniques based on the bootstrap principle that allow sources of error and bias to be simulated and propagated into assay results. We demonstrate how deceptively simple operations—such as the creation of a dilution series with a robotic liquid handler—can significantly amplify imprecision and even contribute substantially to bias. To illustrate these techniques, we review an example of how the choice of dispensing technology can impact assay measurements, and show how large contributions to discrepancies between assays can be easily understood and potentially corrected for. These simple modeling techniques—illustrated with an accompanying IPython notebook—can allow modelers to understand the expected error and bias in experimental datasets, and even help experimentalists design assays to more effectively reach accuracy and imprecision goals.

Dynamic, structural and thermodynamic basis of insulin-like growth factor 1 kinase allostery mediated by activation loop phosphorylation

“… 54 At each λ value, a total of 1 ns simulation was carried out and the last 800 ps was used for free energy calculations by thermodynamic integration (TI). 58 To remove bias when estimating the free energy change, uncorrelated data were extracted based on their correlation time and the bootstrapping protocol 59 , 60 was applied to obtain the free energy error with 1000 repetitions for each transformation. The convergence of the determined free energy values was checked by extending the simulation length to 2 ns for each system.…”

Automated Acoustic Dispensing for the Serial Dilution of Peptide Agonists in Potency Determination Assays

“…While acoustic dispensing does not completely rule out the use of plasticware, it does significantly reduce it. Furthermore, the removal of tip-based pipetting also eliminates sample carryover, and errors are not propagated throughout the entire serial dilution, allowing for both reliable and reproducible preparation of concentration-response curves 14,15 .…”

Measuring experimental cyclohexane-water distribution coefficients for the SAMPL5 challenge

“…Good approaches to uncertainty analysis propagate all known sources of experimental error into the final estimates of uncertainty. To accomplish this, we developed a parametric bootstrap model [26] of the experiment based on earlier work [27], with the goal of propagating pipetting volume and technical replicate errors through the complex analysis procedure to estimate their impact on the overall estimated logD measurements.…”

“…Uncertainties in pipetting operations were modeled based on manufacturer descriptions [29, 30], following the work of Hanson, Ekins and Chodera [27]. Technical replicate variation was modeled by calculating the coefficient of variation (CV) between individual experimental replicates.…”