Review of Evaluation Metrics Used in Digital and Traditional Tobacco Control Campaigns

Abstract

Background Mass media campaigns for public health are increasingly using digital media platforms, such as web-based advertising and social media; however, there is a lack of evidence on how to best use these digital platforms for public health campaigns. To generate this evidence, appropriate campaign evaluations are needed, but with the proliferation of digital media–related metrics, there is no clear consensus on which evaluation metrics should be used. Public health campaigns are diverse in nature, so to facilitate analysis, this review has selected tobacco control campaigns as the scope of the study. Objective This literature review aimed to examine how tobacco control campaigns that use traditional and digital media platforms have been evaluated. Methods Medicine and science databases (Medical Literature Analysis and Retrieval System Online [MEDLINE], EMBASE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature [CINAHL], and Scopus), and a marketing case study database (World Advertising Research Center) were searched for articles published between 2013 and 2018. Two authors established the eligibility criteria and reviewed articles for inclusion. Individual campaigns were identified from the articles, and information on campaigns and their evaluations were supplemented with searches on Google, Google Scholar, and social media platforms. Data about campaign evaluations were tabulated and mapped to a conceptual framework. Results In total, 17 campaigns were included in this review, with evaluations reported on by 51 articles, 17 marketing reports, and 4 grey literature reports. Most campaigns were from English-speaking countries, with behavioral change as the primary objective. In the process evaluations, a wide range of metrics were used to assess the reach of digital campaign activities, making comparison between campaigns difficult. Every campaign in the review, except one, reported some type of engagement impact measure, with website visits being the most commonly reported metric (11 of the 17 campaigns). Other commonly reported evaluation measures identified in this review include engagement on social media, changes in attitudes, and number of people contacting smoking cessation services. Of note, only 7 of the 17 campaigns attempted to measure media platform attribution, for example, by asking participants where they recalled seeing the campaign or using unique website tracking codes for ads on different media platforms. Conclusions One of the key findings of this review is the numerous and diverse range of measures and metrics used in tobacco control campaign evaluations. To address this issue, we propose principles to guide the selection of digital media–related metrics for campaign evaluations, and also outline a conceptual framework to provide a coherent organization to the diverse range of metrics. Future research is needed to specifically investigate whether engagement metrics are associated with desired campaign outcomes, to determine whether reporting of engagement metrics is meaningful in campaign evaluations.

Using social media influencers to increase knowledge and positive attitudes toward the flu vaccine

“…Most studies using influencers to improve health behaviors have relied on campaign awareness or digital metrics to evaluate success. However, these metrics are often simply a reflection of digital ad spending, rather than an evaluation of behavior change [52,53]. While the evaluation methodology employed in this study contained its limitations, it presents an alternative strategy of measuring success that merits further examination.…”

Social Media Use for Health Purposes: Systematic Review

“…Research effort has also focused on improving the design and evaluation of social media-based health interventions. Studies have examined approaches to evaluating campaign outcomes [56], proposed theoretical frameworks for effective health interventions design [57], explored social media user segments to inform intervention development [58], and examined user acceptance and suggestions of receiving social media-based interventions [59]. In general, empirical evidence shows that integrating social media in health interventions is accepted by target populations [59] and effective in increasing health knowledge, reducing risky behaviors, and adopting health behaviors [60].…”

“Study Natural” without Drugs: An Exploratory Study of Theory-Guided and Tailored Health Campaign Interventions to Prevent Nonmedical Use of Prescription Stimulants in College Students

“…To the best of our knowledge, this is the first study examined the efficacy of theory-guided and tailored campaign interventions in changing college students' attitudes and behavior towards NMUPS. Findings of our study can help address issues prevalent and persistent in the campaign literature: (1) a lack of theory-guided and tailored campaign interventions [30,31], (2) a lack of campaign studies that offer detailed information on how the campaign strategies and messages were designed and developed [84,[86][87][88][89]105], and (3) a lack of campaign interventions in the context of NMUPS tailored to emerging adults such as college students [81][82][83].…”

“…We believe important understandings could be gained by answering the research question raised above, as there is an urgent need to fill the existing research gap and identify approaches that could effectively address and alleviate students' engagement in NMUPS activities [81][82][83]. In addition, there is also a need to develop theory-guided interventions that can be replicated and applied in other health contexts to further strengthen the campaign intervention literature [84][85][86][87][88][89].…”