Audio-digital recordings for surveillance in clinical trials of major depressive disorder

Abstract

Ratings surveillance is used in clinical trials to assure ratings reliability of site-based scores. One surveillance method employs audio-digital recordings of site-based clinician interviews to obtain remote, site-independent scores for assessment of paired scoring concordance and interview quality. We examined the utility of this surveillance strategy using paired site-independent scores derived from recorded site-based Montgomery-Asberg depression rating scale (MADRS) interviews obtained from patients with major depressive disorder (MDD) participating in 5 clinical trials.High correlations were noted between the 3736 paired site-based and site-independent scores across all visits. Some rater “outliers” were identified whose ratings performance improved following remediation. In 3 studies with available outcome data, the blinded remote ratings yielded a high predictive value (91.2%) for replicating treatment response rates.The magnitude of the total MADRS scores affected the directionality of paired scoring deviations in each of the 5 studies. Across all visits, site-based raters scored the more severe MADRS scores (≥30) higher than site-independent raters and the less severe MADRS scores (<20) lower than site-independent raters. Individual MADRS items were similarly affected by the directionality of symptom severity.This analysis affirms the utility of audio-digital recording of site-based interviews as a surveillance strategy for quality assurance (monitoring and remediation). In addition, the high predictive value of blinded remote ratings to replicate site-based treatment outcomes may be useful to affirm primary site-based results when there is a potential of functional unblinding. The use of remote ratings as a primary measure beyond its utility for quality assurance needs further exploration.

Real Time Quality Assurance of Depression Ratings in Psychiatric Clinical Trials

“…This result, to our knowledge, provides first evidence that this method is practical and implementable with complex psychiatric patients with bipolar depression and subacute suicidal ideation or behavior. This result also replicates and extends the findings of Targum and Catania (2019), who examined concordance between site and site-independent raters using digital audio recording of 3,736 MADRS interviews. They report concordance rates between 89.5% and 95.8% with lower concordance occurring during earlier visits and higher concordance occurring at later visits.…”

“…15 However, Targum and Catania defined discordance as a deviation of greater than 6 points on the MADRS, which was equal to one standard deviation of the mean total MADRS score. 14 In contrast to Targum, the SRMS method used a more rigorous definition of 3 points to achieve a similar concordance rate of 93.4%. If the SRMS criteria were relaxed to define 6 points as the discordant cutoff, only 3 discordant pairs would occur out of 133 assessments yielding a 97.7% IRR rate.…”

“…16 Targum and Catania reported that for MADRS scores equal to or greater than 30, site raters tend to assign higher (more severe scores) than site-independent raters. 14 The converse is true for MADRS scores less than 20. Our results confirm this finding.…”

Patterns of activity correlate with symptom severity in major depressive disorder patients

“…We selected RMSE as measure of accuracy because it penalizes all deviations and is sensitive to outlier values, and did not use average deviation, where the leverage of large positive and negative outliers can balance out and yield a misleading small average. Reports available in the literature estimate 95% CI = 7 face-to-face vs. telephone interview [ 34 ] and up to 2.75 units difference across testing occasions [ 35 ]. In our datasets, restricting to RMSE < 3 yielded an average accuracy of 2.7, and an absolute error below 3 in 75% of the cases.…”

A qualitative investigation of the Montgomery–Åsberg depression rating scale: discrepancies in rater perceptions and data trends in remote assessments of rapid-acting antidepressants in treatment resistant depression

“…This shift raises concerns, particularly for items like ‘Apparent sadness,’ which rely on observational assessment ( 19 – 24 ). Despite its widespread use and proven interrater reliability across various languages, the appropriateness of MADRS for rapid-acting antidepressants, especially in remote settings, warrants examination ( 25 – 35 ). In addition to that RAADs can generate responses within hours or days, rather than weeks or months, while our rating instruments were designed to assess mood symptoms over a 7-day time frame, typically.…”