Fidelity of Delivery and Contextual Factors Influencing Children’s Level of Engagement: Process Evaluation of the Online Remote Behavioral Intervention for Tics Trial

Abstract

Background The Online Remote Behavioral Intervention for Tics (ORBIT) study was a multicenter randomized controlled trial of a complex intervention that consisted of a web-based behavioral intervention for children and young people with tic disorders. In the first part of a two-stage process evaluation, we conducted a mixed methods study exploring the reach, dose, and fidelity of the intervention and contextual factors influencing engagement. Objective This study aims to explore the fidelity of delivery and contextual factors underpinning the ORBIT trial. Methods Baseline study data and intervention usage metrics from participants in the intervention arm were used as quantitative implementation data (N=112). The experiences of being in the intervention were explored through semistructured interviews with children (n=20) and parent participants (n=20), therapists (n=4), and referring clinicians (n=6). A principal component analysis was used to create a comprehensive, composite measure of children and young people’s engagement with the intervention. Engagement factor scores reflected relative uptake as assessed by a range of usage indices, including chapters accessed, number of pages visited, and number of log-ins. The engagement factor score was used as the dependent variable in a multiple linear regression analysis with various contextual variables as independent variables to assess if there were any significant predictors of engagement. Results The intervention was implemented with high fidelity, and participants deemed the intervention acceptable and satisfactory. The engagement was high, with child participants completing an average of 7.5 of 10 (SD 2.7) chapters, and 88.4% (99/112) of participants completed the minimum of the first four chapters—the predefined threshold effective dose. Compared with the total population of children with tic disorders, participants in the sample tended to have more educated parents and lived in more economically advantaged areas; however, socioeconomic factors were not related to engagement factor scores. Factors associated with higher engagement factor scores included participants enrolled at the London site versus the Nottingham site (P=.01), self-referred versus clinic referred (P=.04), higher parental engagement as evidenced by the number of parental chapters completed (n=111; ρ=0.73; P<.001), and more therapist time for parents (n=111; ρ=0.46; P<.001). A multiple linear regression indicated that parents’ chapter completion (β=.69; t110=10.18; P<.001) and therapist time for parents (β=.19; t110=2.95; P=.004) were the only significant independent predictors of child engagement factor scores. Conclusions Overall, the intervention had high fidelity of delivery and was evaluated positively by participants, although reach may have been constrained by the nature of the randomized controlled trial. Parental engagement and therapist time for parents were strong predictors of intervention implementation, which has important implications for designing and implementing digital therapeutic interventions in child and adolescent mental health services. International Registered Report Identifier (IRRID) RR2-10.1186/s13063-019-3974-3

Digital and remote behavioral therapies for treating tic disorders: Recent advances and next steps

“…This may constitute specific features such as video demonstrations of therapy, animations, the ability to visualize which tics are increasing or decreasing in severity and frequency which may be especially engaging and enjoyable for children. Indeed, these interactive components were identified as key features of the ORBIT intervention and seemed to be used most (32). This is consistent with evidence that interactive elements, including attractive audio-visual material to be amongst the most highly used features of DHIs as they tend to keep users' interest (48,49).…”

“…Thus, future RCTs evaluating DHIs for people with tic disorders should consider conducting a mixed methods process evaluation concurrently with trial delivery, as this would be useful in addressing the intervention's implementation, mechanisms of impact and context. Such findings were crucial in understanding the extent to which ORBIT was both implemented with a high degree of quality (32) and the mechanisms through which it achieved impact (33).…”

“…It does seem that parents play a crucial role in engaging and ensuring the adherence of treatment content within these interventions. Indeed, the ORBIT trial's process evaluation found that parental engagement significantly influenced child's level of engagement (32) and efficacy (33). Several systematic reviews have also noted the crucial role parents have in positive outcomes for children and adolescents across a range of treatments for a variety of conditions (41)(42)(43).…”

Characteristics and Potential Challenges of Digital-Based Interventions for Children and Young People: Scoping Review

“…Guidance components provided users with educational information and were included in 22 studies. Of these, 19 provided users with education, for example, cognitive behavioral therapy, behavioral interventions, social story interventions, parent training programs, coping skills, or physical activity education [11,12,[14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30]. Two studies provided users with tailored educational information, including personalized health and wellness information for someone with Down syndrome [31] and nutritional information with individualized meal plans [32].…”

“…Kollins et al [34], Sosnowski et al [9], Zheng et al [15], Hanrahan et al [16], Hardy et al [35], Ko et al [36], Klee et al [42], Edridge et al [39], Dobias et al [17], Haug et al [43], Alfonsi et al [41], Topooco et al [18,23], Jesus et al [38], Aspvall et al [19], Cliffe et al [20], Voss et al [10], Lei et al [32], Shedrief et al [33], Ranney et al [22], Gallen et al [37], Khan et al [24], Kenny et al [25], Lenhard et al [26], Osborn et al [40], Knox et al [27], Nordh et al [28], Schmidt et al [29], and Lindqvist et al [12] 29 (85) Children or adolescents only Chung et al [31] and Sourander et al [21] 2 (6) Caregivers only…”