Methodological Challenges in Randomized Controlled Trials on Smartphone-Based Treatment in Psychiatry: Systematic Review

Abstract

BackgroundSmartphone-based technology is developing at high speed, and many apps offer potential new ways of monitoring and treating a range of psychiatric disorders and symptoms. However, the effects of most available apps have not been scientifically investigated. Within medicine, randomized controlled trials (RCTs) are the standard method for providing the evidence of effects. However, their rigidity and long time frame may contrast with the field of information technology research. Therefore, a systematic review of methodological challenges in designing and conducting RCTs within mobile health is needed.ObjectiveThis systematic review aimed to (1) identify and describe RCTs investigating the effect of smartphone-based treatment in adult patients with a psychiatric diagnosis, (2) discuss methodological challenges in designing and conducting individual trials, and (3) suggest recommendations for future trials.MethodsA systematic search in English was conducted in PubMed, PsycINFO, and EMBASE up to August 12, 2019. The search terms were (1) psychiatric disorders in broad term and for specific disorders AND (2) smartphone or app AND (3) RCT. The Consolidated Standards of Reporting Trials electronic health guidelines were used as a template for data extraction. The focus was on trial design, method, and reporting. Only trials having sufficient information on diagnosis and acceptable diagnostic procedures, having a smartphone as a central part of treatment, and using an RCT design were included.ResultsA total of 27 trials comprising 3312 patients within a range of psychiatric diagnoses were included. Among them, 2 trials were concerning drug or alcohol abuse, 3 psychosis, 10 affective disorders, 9 anxiety and posttraumatic stress disorder, 1 eating disorder, and 1 attention-deficit/hyperactivity disorder. In addition, 1 trial used a cross-diagnostic design, 7 trials included patients with a clinical diagnosis that was subsequently assessed and validated by the researchers, and 11 trials had a sample size above 100. Generally, large between-trial heterogeneity and multiple approaches to patient recruitment, diagnostic procedures, trial design, comparator, outcome measures, and analyses were identified. Only 5 trials published a trial protocol. Furthermore, 1 trial provided information regarding technological updates, and only 18 trials reported on the conflicts of interest. No trial addressed the ethical aspects of using smartphones in treatment.ConclusionsThis first systematic review of the methodological challenges in designing and conducting RCTs investigating smartphone-based treatment in psychiatric patients suggests an increasing number of trials but with a lower quality compared with classic medical RCTs. Heterogeneity and methodological issues in individual trials limit the evidence. Methodological recommendations are presented.

Leveraging Implementation Science to Understand Factors Influencing Sustained Use of Mental Health Apps: a Narrative Review

“…These realities exist in stark opposition to the nature of apps, which must frequently change to retain users’ attention and remain compatible with smartphone technologies. Funding mechanisms must be nimble, dynamic, adaptive, and supportive of frequent iteration to ensure that the product being implemented is actually usable (Glasgow et al 2014 ; Mohr et al 2017a ; Tønning et al 2019 ). Grants that partner with industry, such as NIH Small Business Innovation Research grants, are critical in acknowledging that MH researchers will likely not also be experts in app design; such mechanisms can allow the strengths of both academia and industry to shine through (Hidalgo-Mazzei et al 2020 ).…”

Brief Acceptance and Commitment Therapy for Fibromyalgia: Feasibility and Effectiveness of a Replicated Single-Case Design

“…The literature on clinical research in general and chronic pain in particular has been largely dominated by large-scale randomized clinical trials in which the analysis of changes, mostly with group interventions, is explored considering average group scores and a reduced number of assessments [ 39 , 40 ]. While large-scale randomized clinical trials are clearly informative and can address very important questions including responder and moderation analyses, the overrepresentation of this type of studies in the literature can also be problematic for a number of reasons.…”

Smartphone-Based Self-Monitoring, Treatment, and Automatically Generated Data in Children, Adolescents, and Young Adults With Psychiatric Disorders: Systematic Review

“…In conclusion, the findings from this review strongly emphasize the need for a larger number of studies as well as studies with a larger number of participants, including RCTs investigating the potential positive and negative effects of fine-grained smartphone-based self-monitoring and treatment over prolonged periods of time. Such RCTs should provide details on the methodology, reporting, and interpretation of findings, as recently described by our group [ 31 ], thereby making it easier to compare studies and to facilitate future meta-analyses.…”