Harnessing Artificial Intelligence for Health Message Generation: The Folic Acid Message Engine

Abstract

Background Communication campaigns using social media can raise public awareness; however, they are difficult to sustain. A barrier is the need to generate and constantly post novel but on-topic messages, which creates a resource-intensive bottleneck. Objective In this study, we aim to harness the latest advances in artificial intelligence (AI) to build a pilot system that can generate many candidate messages, which could be used for a campaign to suggest novel, on-topic candidate messages. The issue of folic acid, a B-vitamin that helps prevent major birth defects, serves as an example; however, the system can work with other issues that could benefit from higher levels of public awareness. Methods We used the Generative Pretrained Transformer-2 architecture, a machine learning model trained on a large natural language corpus, and fine-tuned it using a data set of autodownloaded tweets about #folicacid. The fine-tuned model was then used as a message engine, that is, to create new messages about this topic. We conducted a web-based study to gauge how human raters evaluate AI-generated tweet messages compared with original, human-crafted messages. Results We found that the Folic Acid Message Engine can easily create several hundreds of new messages that appear natural to humans. Web-based raters evaluated the clarity and quality of a human-curated sample of AI-generated messages as on par with human-generated ones. Overall, these results showed that it is feasible to use such a message engine to suggest messages for web-based campaigns that focus on promoting awareness. Conclusions The message engine can serve as a starting point for more sophisticated AI-guided message creation systems for health communication. Beyond the practical potential of such systems for campaigns in the age of social media, they also hold great scientific potential for the quantitative analysis of message characteristics that promote successful communication. We discuss future developments and obvious ethical challenges that need to be addressed as AI technologies for health persuasion enter the stage.

Towards AI-Driven Healthcare: Systematic Optimization, Linguistic Analysis, and Clinicians’ Evaluation of Large Language Models for Smoking Cessation Interventions

“…Results revealed that temperature sampling (version 5, t=0.7) and a hybrid approach (version 1, p=0.9, k=50, t=0.8; version 8, k=40, t=0.7) exhibited perplexity scores closely to that of the expert-writing condition. Consistent with prior studies [ 28 , 33 , 73 , 80 ], decoding methods with higher p values (version 2, p=0.9; version 3, p=0.95) or higher temperature settings (version 4, t=0.9) were found to sample too many unlikely tokens, resulting in messages that were diverse yet incoherent. The perplexity findings also echoed the repetition filtering process: decoding methods (version 1, 5, 8) that exhibited lower and approximating-reference perplexity scores had a higher proportion of messages (7.2% to 7.7%) discarded due to repetition; conversely, decoding methods with higher perplexity scores (version 2, 3, 4) demonstrated lower rates of repetition (3.7% to 4.5%).…”

“…Similarly, Holtzman et al [ 34 ] observed that temperatures exceeding 0.9 yielded messages diversity akin to human writing, measured by self-BLEU scores, and temperatures above 0.7 effectively mitigated repetition issues. In the healthcare context, Schmalzle and Wilcox [ 80 ] conducted a pilot test employing temperature settings of 0.3, 0.5, 0.7, and 1 with a fine-tuned 355M GPT-2 model, aiming to create messages about folic acid. Their results indicated that T=0.7 produced the most balanced outputs in terms of both quality and diversity, whereas T=1 led to incoherent outputs and T<0.5 produced text that was highly homogeneous with the training text.…”

“…Specifically, in version 3, we adopted the top-p sampling value recommended by [ 33 ] and reduced the value from 0.95 (as in version 3) to 0.9 (in version 2) to examine potential improvements in model performance. Version 5 employed a temperature setting of 0.7, as suggested by [ 80 ] for message generation tasks, and we increased the value to 0.9 in version 4 to explore the impact of a temperature variation on model performance. For top-k sampling, we followed [ 73 ]’s advice, using k=40 in version 6 and adjusting it to k=30 in version 7 to study its effects.…”

A study on the correlation between MTHFR and folic acid combined with trace elements for the prevention of fetal malformations in the first trimester of pregnancy

“…As an essential nutrient, its significance for the well-being of pregnant women is self-evident. [30][31][32][33] In this study, the selected subjects exhibit varying degrees of FA supplementation, contributing to a reduction in the risk of fetal malformations among pregnant women to a certain extent.…”

Artificial intelligence for health message generation: an empirical study using a large language model (LLM) and prompt engineering

“…To our knowledge, only one paper in communication has used NLP models to generate health awareness messages. Schmälzle and Wilcox (2022) introduced a message machine capable of creating new awareness messages. Specifically, they used a GPT2-LM to generate short topical text messages, now commonly called tweets.…”