Implications and Recommendations for Equivalence Testing in Measures of Movement Behaviors: A Scoping Review

Abstract

Equivalence testing may provide complementary information to more frequently used statistical procedures because it determines whether physical behavior outcomes are statistically equivalent to criterion measures. A caveat of this procedure is the predetermined selection of upper and lower bounds of acceptable error around a specified zone of equivalence. With no clear guidelines available to assist researchers, these equivalence zones are arbitrarily selected. A scoping review of articles implementing equivalence testing was performed to determine the validity of physical behavior outcomes; the aim was to characterize how this procedure has been implemented and to provide recommendations. A literature search from five databases initially identified potentially 1,153 articles which resulted in the acceptance of 19 studies (20 arms) conducted in children/youth and 40 in adults (49 arms). Most studies were conducted in free-living conditions (children/youth = 13 arms; adults = 22 arms) and employed a ±10% equivalence zone. However, equivalence zones ranged from ±3% to ±25% with only a subset using absolute thresholds (e.g., ±1,000 steps/day). If these equivalence zones were increased or decreased by ±5%, 75% (15/20, children/youth) and 71% (35/49, adults), they would have exhibited opposing equivalence test outcomes (i.e., equivalent to nonequivalent or vice versa). This scoping review identifies the heterogeneous usage of equivalence testing in studies examining the accuracy of (in)activity measures. In the absence of evidence-based standardized equivalence criteria, presenting the percentage required to achieve statistical equivalence or using absolute thresholds as a proportion of the SD may be a better practice than arbitrarily selecting zones a priori.

Disagreements in physical activity monitor validation study guidelines create challenges in conducting validity studies

“…Consistent with some guidelines ( 1 , 4 ), we encourage that equivalence testing be conducted and that the equivalence zone required for the two measures to be deemed statistically equivalent reported. This avoids the use of arbitrary a priori thresholds (e.g., ±10% or ±20%) that produce dichotomous outcomes that are sensitive to minor deviations in threshold selection ( 14 ). Specifically, a review on the topic demonstrated that a 5% change in threshold selection altered the conclusions of 75% and 71% of validation studies in children/youth and adults, respectively ( 14 ).…”

“…This avoids the use of arbitrary a priori thresholds (e.g., ±10% or ±20%) that produce dichotomous outcomes that are sensitive to minor deviations in threshold selection ( 14 ). Specifically, a review on the topic demonstrated that a 5% change in threshold selection altered the conclusions of 75% and 71% of validation studies in children/youth and adults, respectively ( 14 ). In the absence of clinically acceptable equivalence zones, we also recommend that researchers consider reporting the zone required for the measures to be deemed equivalent as a percentage and/or as a proportion of the SD (e.g., 0.5 SD).…”

“…A primer on equivalence testing is presented elsewhere ( 13 ). While challenges to implementing equivalence testing exist, such as the establishment of thresholds to denote “equivalence” ( 14 , 15 ), the overarching idea of establishing whether two measures statistically produce values within an acceptable level of error seems pertinent to device based validation studies. Despite the heterogeneous recommendations, it is our position that equivalence testing be implemented alongside bias testing (i.e., Bland-Altman), difference of means, and MAPE.…”

Validation of an automated sleep detection algorithm using data from multiple accelerometer brands

“…The ±10% equivalence zone of the reference mean was selected based on previous validation studies of physical behaviours measures. However, as presented by O'Brien (2021), the selection of an equivalence zone (i.e., narrow versus broad) influences equivalence test outcomes. Therefore, as there is no evidence of a standardised equivalence criteria, this study also reported a minimum equivalence zone for the PSG estimates that include 90% CI of the accelerometer sleep estimates.…”

Comparing the activPAL CREA and GHLA Algorithms for the Classification of Postures and Activity in Free-Living Children

“…For instance, the use of a 10% zone can be strict when values are highly variable and smaller but lax when values are very high and across a narrow range of values [ 29 ]. Therefore, the absolute equivalence zone needed to reach equivalence is provided alongside the relative equivalence zone presented as a proportion of the SD [ 30 ]. To aid interpretations, a strict threshold of 3%, indicative of excellent agreement between algorithms, was used to aid interpretations of the absolute equivalence zone [ 26 ].…”