A Bayesian Decision Theoretic Model of Sequential Experimentation with Delayed Response

Abstract

We propose a Bayesian decision theoretic model of a fully sequential experiment in which the real-valued primary end point is observed with delay. The goal is to identify the sequential experiment which maximizes the expected benefits of technology adoption decisions, minus sampling costs. The solution yields a unified policy defining the optimal 'do not experiment'-'fixed sample size experiment'-'sequential experiment' regions and optimal stopping boundaries for sequential sampling, as a function of the prior mean benefit and the size of the delay. We apply the model to the field of medical statistics, using data from published clinical trials.

Cost-effective clinical trial design: Application of a Bayesian sequential model to the ProFHER pragmatic trial

“…We call a rule which meets this objective an 'Optimal Bayes Sequential policy' and obtain such a policy using dynamic programming methods. 13 There are two scenarios in which it is not optimal to enter Stage II: (1) the expected benefit from entering Stage II is less than that of running a trial with a fixed number of pairwise allocations in the range (0, t). In this scenario, the Optimal Bayes Sequential policy selects the same sample size as a trial designed to maximise the difference between the expected value of sample information and the cost of sampling.…”

“…Where there exists a large degree of uncertainty over the values of a particular parameter, sensitivity analysis may be carried out. 13 The application…”

Costly sequential experimentation and project valuation with an application to health technology assessment

“…While Draper (2013) argues that this is unnecessary, it is precisely this assumption that allows us to derive many analytical properties, even in a more complicated case with delayed observations. Papers without this feature, such as, for example, Hampson and Jennison (2013) or Chick et al (2015), typically have to resort to numerical solutions. In short, what we lose in realism we gain in analytical clarity.…”

“…An important issue in measuring health benefits as part of a clinical trial is that these are typically not observed instantaneously but, rather, with a delay. This issue has recently been investigated by Chick et al (2015) who use a Bayesian sequential estimation method. Unlike their approach, in our sequential testing framework, incorporating delays in observations can be achieved as a straightforward extension of the basic model by following the approach introduced in Øksendal Suppose that the delay in information becoming available is denoted by δ > 0.…”

Optimal sequential sampling with delayed observations and unknown variance

“…We are interested in knowing for how long sampling should continue before stopping, observing the remaining pipeline data, and making an implementation decision. Chick, Forster, and Pertile (2015) (hereafter CFP) derive a Bayesian decision-theoretic model of optimal sequential sampling which allows for samples to arrive with a fixed delay when the sampling variance is known. The authors apply the model to clinical trials and health technology assessments which seek to compare two alternatives, one a 'new' health technology, the other an existing one.…”