Controlling decision errors with minimal costs: The sequential probability ratio t test.

Abstract

Waldian t tests: Sequential Bayesian t tests with controlled error probabilities [Manuscript submitted for publication].

Efficiency in Sequential Testing: Comparing the Sequential Probability Ratio Test and the Sequential Bayes Factor Test

“…Previous research has presented the SBFT and SPRT as two distinct methodologies for sequential hypothesis testing (e.g., Schnuerch & Erdfelder, 2020;Pramanik et al, 2020). In the following sections, we will demonstrate that the methods are, in fact, part of the same overarching hypothesis testing framework.…”

“…For example, in the one-sided SPRT t-test, the likelihood functions for the null and alternative model are both t-distributions with ν = n 1 + n 2 − 2 degrees of freedom, where n 1 and n 2 are the sample sizes of the two groups that should be compared. The data are summarized by a t-statistic (Hajnal, 1961;Schnuerch & Erdfelder, 2020). The null model posits that Cohen's δ 0 , the standardized mean difference between the groups, equals zero.…”

“…On average, sequential hypothesis tests yield substantially smaller sample sizes than conventional hypothesis tests that assume a fixed sample size based on a statistical power analysis. Studies investigating the efficiency of sequential designs have consistently found reductions in sample sizes of 50% or more compared to these fixed sample size designs (e.g., Wald, 1945;Pocock, 1977;Schönbrodt et al, 2017;Schnuerch & Erdfelder, 2020). This makes sequential hypothesis testing an attractive choice when resources are scarce.…”

“…Recently, two existing sequential hypothesis testing methods have been brought to the attention of psychology researchers: the Sequential Probability Ratio Test (SPRT; Schnuerch & Erdfelder, 2020) and the Sequential Bayes Factor Test (SBFT; Schönbrodt et al, 2017). Both methods are applicable to a wide range of hypothesis tests (Wald, 1945;Rouder, 2014), but have been discussed with a particular focus on the independent samples t-test that is frequently applied in psychological research (Wetzels et al, 2011).…”

“…In a recent article, Schnuerch & Erdfelder (2020) contrasted the SPRT t-test and the sequential Bayesian t-test and concluded that the two main advantages of the SPRT are that (1) it is more efficient than the sequential Bayesian test, and (2) it allows for explicit control of error rates. A similar claim was made by Pramanik et al (2020) based on their comparison of the SBFT and a modified SPRT procedure that allows researchers to set a maximum sample size.…”