Preference option randomized design (PORD) for comparative effectiveness research: Statistical power for testing comparative effect, preference effect, selection effect, intent-to-treat effect, and overall effect

Abstract

Comparative effectiveness research trials in real-world settings may require participants to choose between preferred intervention options. A randomized clinical trial with parallel experimental and control arms is straightforward and regarded as a gold standard design, but by design it forces and anticipates the participants to comply with a randomly assigned intervention regardless of their preference. Therefore, the randomized clinical trial may impose impractical limitations when planning comparative effectiveness research trials. To accommodate participants' preference if they are expressed, and to maintain randomization, we propose an alternative design that allows participants' preference after randomization, which we call a "preference option randomized design (PORD)". In contrast to other preference designs, which ask whether or not participants consent to the assigned intervention after randomization, the crucial feature of preference option randomized design is its unique informed consent process before randomization. Specifically, the preference option randomized design consent process informs participants that they can opt out and switch to the other intervention only if after randomization they actively express the desire to do so. Participants who do not independently express explicit alternate preference or assent to the randomly assigned intervention are considered to not have an alternate preference. In sum, preference option randomized design intends to maximize retention, minimize possibility of forced assignment for any participants, and to maintain randomization by allowing participants with no or equal preference to represent random assignments. This design scheme enables to define five effects that are interconnected with each other through common design parameters-comparative, preference, selection, intent-to-treat, and overall/as-treated-to collectively guide decision making between interventions. Statistical power functions for testing all these effects are derived, and simulations verified the validity of the power functions under normal and binomial distributions.

Equine-assisted services for individuals with substance use disorders: a scoping review

“…Additionally, as acknowledged by Gatti et al [ 30 ] who had 7 participants withdraw consent based on randomization assignment, it is reasonable to assume that EAS would be most effective for those actively seeking this type of treatment. Therefore, innovative randomized experimental designs that take into consideration patient preference may more accurately reflect real world implementation and effectiveness [ 36 ]. Additionally, however, these observed dropouts may have also been the result of randomizing individuals in a single treatment setting where participants are acutely aware of counterparts’ differing treatment conditions.…”

Authors' reply: Letter to the Editor: Preference option randomized design (PORD) for comparative effectiveness research: Statistical power for testing comparative effect, preference effect, selection effect, intent-to-treat effect, and overall effect (SMMR, Vol. 28, Issue 2, 2019)

“…Following their logic in Box 2 and their computation process in Table 2 (Box 3), we have: p2 ¼ 0.4, p3 ¼ 0.4, p4 ¼ 0.4, p5 ¼ 0.5; and a A ¼ 0.1 and b B ¼ 0.2. First, here a A þ b B ¼ 0.3 which contradicts equation (1). Second, if we follow their equation (1), then we have:…”

“…

We appreciate Dr. Walter and his colleagues' pointing out a limitation concerning (A2) assumption. 1 In short, we adopted the A2 assumption because in preference option randomized design (PORD), there are no participants who prefer treatment B in A S (even though there are participants who prefer treatment B in R A , that is B A ), or those who prefer treatment A in B S (even though there are participants who prefer treatment A in R B , that is A B ). In theory, however, it might have been more reasonable to assume that P(A A jA S ) ¼ P(A B jB R [A B ) ¼ P(A B jR B )/ (1 À P(B B jR B )) ¼ P(A B jR B )/(1 À P(B A jR A )), this equation holding by a strict NSBR (no selection bias from randomization) assumption, and likewise P(B B jB S ) ¼ P(B A jR A )/(1 À P(A B jR B )).

…”

Comment to the reply letter: Letter to the Editor: Preference option randomized design (PORD) for comparative effectiveness research: Statistical power for testing comparative effect, preference effect, selection effect, intent-to-treat effect, and overall effect (SMMR, Vol. 28, Issue 2, 2019)

“…feel does provide a valid approach to the scenario that Heo et al have considered. 2 In brief, Heo et al have either not fully read our paper, or they have mis-understood it, leading to a number of false claims, as we now enumerate.First, Heo et al describe the five preference groups in our Table 1 as 'exhaustive'. In fact, Heo et al have failed to notice that our 2006 paper explicitly lists four other possible preference patterns: people who would refuse whichever treatment (A or B) is offered to them (so-called refusers); people who insist on A if offered B, or insist on B if offered A (defiers); and people who decline treatment if offered A, but insist on A if they are offered B, and vice versa (two groups of contrarians); see both the Methods and Discussion sections of our paper.…”

“…feel does provide a valid approach to the scenario that Heo et al have considered. 2 In brief, Heo et al have either not fully read our paper, or they have mis-understood it, leading to a number of false claims, as we now enumerate.…”