Beyond the t-Test: Statistical Equivalence Testing

Abstract

2 2 1 ( ) ( ( ) ( ) ( ) -0 the t-Test: Statistical Equivalence Testing BEYONDO ne of the most common questions considered by analytical chemists is whether replicate measurements are the same or significantly different from each other. The determination of significantly different results can be used to argue that a phenomenon is novel or to justify a claim of a significant improvement in a technique, process, or product. Science and technology are also driven by determinations of sameness, such as equivalence, control, or ruggedness.Given the variability inherent to most instrument systems, the question of whether a measurable difference is "real" can be difficult to answer. In some cases, intuition, experience, and knowledge of the practical context of the data can be used to inspect or "eyeball" the data to assess whether a true difference exists. For example, most of us would agree that a difference of 100% in a measurement that typically exhibits a precision of 1% is a real difference, and we would also agree that a difference of 0.01% is not significant for the same measurement. But what about less clear-cut cases in which the difference between two sets of data is similar to the precision? How much difference is "too much"? Not only does simple inspection fail in these circumstances, but a subjective process can be biased, is difficult to justify, and, most importantly, can lead to the wrong conclusion.

Ecosystem metabolism in salt marsh tidal creeks and ponds : applying triple oxygen isotopes and other gas tracers to novel environments

“…Triple oxygen isotope and O2 derived rates of GPP from the chamber experiments were equivalent (p=0.0017 and 0.0096 for a two one-sided t-test, θ=0.65; Limentani et al 2005). GPPTOI and GPPO2 from the open pond for just the chamber experiment days and periods are also equivalent (p=0.0364 and 0.0237 for a two one-sided t-test, θ=0.24).…”

Control of untethered soft grippers for pick-and-place tasks

“…In order to determine whether the performance difference among different grippers can be considered statistically negligible, we performed a Two One-Sided T-test (TOST) [32]. The tests revealed statistical equivalence, with 90% interval confidence, suggesting that all soft grippers behave similarly.…”

Utilizing Virtual Environments for the Evaluation of Lighting Controls

“…An alternative test, the two one-sided ttest (TOST) "begins with a null hypothesis that the two mean values are not equivalent, then attempts to demonstrate that they are equivalent within a practical, preset limit" (Limentani, Ringo, Ye, Bergquist & McSorley 2005). TOST was originally designed for bioequivalence testing of pharmaceutical products but has recently expanded into process engineering, psychology, medicine, chemistry and environmental science (Limentani et al, 2005). The challenge for the researcher applying this test is to define an acceptance criterion based on previous knowledge and the intended application.…”