Minimal effect of messaging on engagement in a digital anxiety intervention.

Abstract

This study evaluated the effectiveness of different recruitment messages for encouraging enrollment in a digital mental health intervention (DMHI) for anxiety among 1,600 anxious patients in a large health care system. Patients were randomly assigned to receive a standard message, or one of five messages designed to encourage enrollment: three messages offered varying financial incentives, one message offered coaching, and one message provided consumer testimonials. Patients could then click a link in the message to visit the DMHI website, enroll, and start the first session. We examined the effects of message features and message length (short vs. long) on rates of site clicks, enrollment, and starting the first session. We also tested whether demographic and clinical factors derived from patients' electronic health records were associated with rates of enrollment and starting the first session to understand the characteristics of patients most likely to use DMHIs in this setting. Across messages, 19.4% of patients clicked a link to visit the DMHI website, but none of the messages were significantly associated with rates of site clicks, enrollment, or starting the first session. Females (vs. males) had a greater probability of enrollment. No other demographic or clinical variables were significantly associated with enrollment or starting the first session. Findings provide guidance for resource allocation decisions in larger scale DMHI implementations in health care settings. Public Significance StatementAmong anxious patients in a large health care setting, resource-intensive methods of promoting DMHI engagement (e.g., offering financial incentives or coaching) did not affect rates of clicks on a link to visit a DMHI website, enrollment, or starting the first session. Findings suggest these high-cost promotional methods may not add value in terms of increasing DMHI uptake or use when offering DMHIs at a larger scale.

Minimal Effect of Culturally Informed Marketing Messages on Engagement With a Digital Anxiety Intervention Among Latinx Individuals

“…In contrast, model-predicted rates of enrollment and starting the first session were relatively similar across message features. While these findings are contrary to hypotheses, they align with prior studies (one with a Latinx sample, and one with a predominantly non-Latinx white sample) that observed small and nonsignificant differences across message conditions in rates of enrollment (less than .01%; Graham et al, 2012;2.74%;Silverman et al, 2023) and starting the first session (3.30%; Silverman et al, 2023). It is not surprising that the message features had the greatest impact at the initial stage of the recruitment process (i.e., clicking the link to visit the DMHI website) rather than subsequent stages (i.e., enrolling and starting the first session) since messages were delivered at this point in the recruitment pipeline.…”

“…The difference in model-predicted rates of site clicks between the best performing message feature (9.5% for the testimonials message) compared to the poorest performing message feature (6.2% for the somatization message) is slightly larger than those found in two previous studies conducted among Latinx individuals (1.11%; Barrera et al, 2014; . 03% Graham et al, 2012), but more than two times smaller than those found in two studies conducted among general samples (9.10%; Birnbaum et al, 2017;8.71%;Silverman et al, 2023). While small, differences of this size could be potentially meaningful (in terms of attracting thousands of more individuals to visit the DMHI website for no extra cost) if marketing materials were scaled to reach millions of Spanish-speaking Latinx individuals, which becomes possible when using targeted social media marketing.…”

“…For example, in prior studies that have examined the effectiveness of different online DTC marketing messages for increasing rates of clicks to visit mental and behavioral health websites, observed differences between the best and worst performing messages have been variable, ranging from .03% (Graham et al, 2012) to 9.10% (Birnbaum et al, 2017). Other studies have observed small and non-significant differences in rates of enrollment between the best and worst performing messages (e.g., less than .01%; Graham et al, 2012;2.74%;Silverman et al, 2023), highlighting the challenge of using DTC marketing messages to promote treatment-seeking behaviors. As such, additional research is needed to understand the potential for DTC marketing efforts to create "pull demand" (i.e., to increase treatment-seeking behaviors and uptake) for anxiety services.…”

“…Along these lines, DTC marketing campaigns designed to promote cognitive behavior therapy for anxiety in unselected samples have been found to positively impact perceptions of treatment (e.g., Schofield et al, 2019;Schofield et al, 2020). However, these studies did not examine the impact of DTC materials on actual treatment-seeking behaviors (e.g., requests for more information, mental health website visits) or uptake (e.g., enrollment rates); and findings from the few studies that have tested this question have been relatively mixed (e.g., Barrera et al, 2014;Birnbaum et al, 2017;Graham et al, 2012;Silverman et al, 2023). For example, in prior studies that have examined the effectiveness of different online DTC marketing messages for increasing rates of clicks to visit mental and behavioral health websites, observed differences between the best and worst performing messages have been variable, ranging from .03% (Graham et al, 2012) to 9.10% (Birnbaum et al, 2017).…”

“…Focus group members indicated that it would be helpful for marketing materials to include testimonials from individuals who had previously used the DMHI to help increase the perceived trustworthiness and credibility of the program (Calicho-Mamani et al, 2019). While the use of testimonials was viewed favorably among focus group participants, scientific evidence for the effectiveness of this marketing strategy for promoting DMHI engagement has been relatively mixed (e.g., Silverman et al, 2023;Simenec et al, 2023). Nonetheless, testing the effectiveness of testimonials among Spanish-speaking Latinx individuals remains important, given the support for this promotional strategy among focus group participants.…”