Permuting incomplete paired data: a novel exact and asymptotic correct randomization test

Abstract

Various statistical tests have been developed for testing the equality of means in matched pairs with missing values. However, most existing methods are commonly based on certain distributional assumptions such as normality, 0-symmetry or homoscedasticity of the data. The aim of this paper is to develop a statistical test that is robust against deviations from such assumptions and also leads to valid inference in case of heteroscedasticity or skewed distributions. This is achieved by applying a novel randomization approach. The resulting test procedure is not only shown to be asymptotically correct but is also finitely exact if the distribution of the data is invariant with respect to the considered randomization group. Its small sample performance is further studied in an extensive simulation study and compared to existing methods. Finally, an illustrative data example is analyzed.

A cautionary tale on using imputation methods for inference in matched-pairs design

“…Simulation results of type-I-error of the T M L -permutation test as well as the paired t-test based on imputation are summarized in Table 1 under various covariance structures and residual distributions for (n 1 , n 2 , n 3 ) = (10, 10, 10). Starting with the permutation test T M L of Amro and Pauly (2017) we see that it tends to be an adequate exact level α test among all considered ranges of ρ for homoscedastic covariance structures and most heteroscedastic Figure 1: NRMSE for the Normal, Exponential, χ 2 df =30 and Laplace distribution using 10, 000 Monte-Carlo runs under H 0 and the balanced scheme with n 1 = n 2 = n 3 = 10 under various correlations ρ ∈ {−0.9, −0.5, −0.1, 0.1, 0.5, 0.9} for (1) RFMI, (2) RFMICE, (3) PMM and (4) NORM.…”

“…Based on a simulation study (Amro et al, 2018), the two permutation methods showed similar behavior in most of the considered cases. Therefore, we only consider the method of Amro and Pauly (2017) for further analysis. Their suggested test statistic consists of the following weighting scheme:…”

“…2· ) 2 the corresponding empirical variances. A recommended value of a is 2n 1 /(n + n 1 ) (Amro and Pauly, 2017). This choice guarantees the correct treatment in the extreme scenarios of n 1 = n and n 1 = 0.…”

“…Furthermore, the sample sizes were chosen as (n 1 , n 2 , n 3 ) ∈ {(10, 10, 10), (30, 10, 10)} and later increased by adding k1 ⊤ 3 for k ∈ {1, ..., 500} to the balanced sample size (10, 10, 10) and multiplying the unbalanced setting (30, 10, 10) with a factor k ∈ {1, ..., 50}. In order to assess the power of the paired-t-test for imputation methods and the permutation test of Amro and Pauly (2017), we set µ = [δ, 0] ⊤ with shifting parameter δ ∈ {0, 1/2, 1}. Here, the permutation method was based on B = 1, 000 permutation runs.…”

Multiplication‐combination tests for incomplete paired data

“…Moreover, these procedures are usually nonrobust to deviations and may result in inaccurate decisions caused by possibly inflated or conservative type‐I error rates . To overcome these problems, the typical recommendation is to use adequately weighted studentized test statistics for the underlying paired and unpaired two sample problem; see, eg, the works of Samawi and Vogel or Amro and Pauly for the mean functional and the works of Gao, Konietschke et al, and Fong et al for nonparametric situations. In case of small or moderate sample sizes, however, estimating large quantiles of the test statistics' distributions is challenging, leading to possibly inflated type‐I error rates.…”