Estimating Patient Satisfaction Through a Language Processing Model: Model Development and Evaluation

Matsuda, Shinichi; Ohtomo, Takumi; Okuyama, Masahiko; Miyake, Hiraku; Aoki, Kotonari

doi:10.2196/48534

Cited by 3 publications

(1 citation statement)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The applications of deep learning for natural language processing have expanded dramatically in recent years [25]. Since the development of a high-performance deep learning model in 2018 [26], attempts to apply cutting-edge deep learning models to various kinds of patient-generated text data for the evaluation of safety events or the analysis of unscalable subjective information from patients have been accelerating [27][28][29][30][31]. Most studies have been conducted to use patients' narrative data for pharmacovigilance [27,[32][33][34][35], while few have been aimed at improvement of real-time safety monitoring for individual patients.…”

Section: Introductionmentioning

confidence: 99%

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models (Preprint)

Nishioka,

Watabe,

Yanagisawa

et al. 2023

Preprint

View full text Add to dashboard Cite

BACKGROUND Early detection of adverse events and their management are crucial to improving anticancer treatment outcomes, and listening to patients’ subjective opinions (patients’ voices) can make a major contribution to improving safety management. Recent progress in deep learning technologies has enabled various new approaches for the evaluation of safety-related events based on patient-generated text data, but few studies have focused on the improvement of real-time safety monitoring for individual patients. In addition, no study has yet been performed to validate deep learning models for screening patients’ narratives for clinically important adverse event signals that require medical intervention. In our previous work, novel deep learning models have been developed to detect adverse event signals for hand-foot syndrome or adverse events limiting patients’ daily lives from the authored narratives of patients with cancer, aiming ultimately to use them as safety monitoring support tools for individual patients. OBJECTIVE This study was designed to evaluate whether our deep learning models can screen clinically important adverse event signals that require intervention by health care professionals. The applicability of our deep learning models to data on patients’ concerns at pharmacies was also assessed. METHODS Pharmaceutical care records at community pharmacies were used for the evaluation of our deep learning models. The records followed the SOAP format, consisting of subjective (S), objective (O), assessment (A), and plan (P) columns. Because of the unique combination of patients’ concerns in the S column and the professional records of the pharmacists, this was considered a suitable data for the present purpose. Our deep learning models were applied to the S records of patients with cancer, and the extracted adverse event signals were assessed in relation to medical actions and prescribed drugs. RESULTS From 30,784 S records of 2479 patients with at least 1 prescription of anticancer drugs, our deep learning models extracted true adverse event signals with more than 80% accuracy for both hand-foot syndrome (n=152, 91%) and adverse events limiting patients’ daily lives (n=157, 80.1%). The deep learning models were also able to screen adverse event signals that require medical intervention by health care providers. The extracted adverse event signals could reflect the side effects of anticancer drugs used by the patients based on analysis of prescribed anticancer drugs. “Pain or numbness” (n=57, 36.3%), “fever” (n=46, 29.3%), and “nausea” (n=40, 25.5%) were common symptoms out of the true adverse event signals identified by the model for adverse events limiting patients’ daily lives. CONCLUSIONS Our deep learning models were able to screen clinically important adverse event signals that require intervention for symptoms. It was also confirmed that these deep learning models could be applied to patients’ subjective information recorded in pharmaceutical care records accumulated during pharmacists’ daily work.

show abstract

Section: Introductionmentioning

confidence: 99%

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models (Preprint)

Nishioka,

Watabe,

Yanagisawa

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models

Nishioka,

Watabe,

Yanagisawa

et al. 2024

J Med Internet Res

View full text Add to dashboard Cite

Background Early detection of adverse events and their management are crucial to improving anticancer treatment outcomes, and listening to patients’ subjective opinions (patients’ voices) can make a major contribution to improving safety management. Recent progress in deep learning technologies has enabled various new approaches for the evaluation of safety-related events based on patient-generated text data, but few studies have focused on the improvement of real-time safety monitoring for individual patients. In addition, no study has yet been performed to validate deep learning models for screening patients’ narratives for clinically important adverse event signals that require medical intervention. In our previous work, novel deep learning models have been developed to detect adverse event signals for hand-foot syndrome or adverse events limiting patients’ daily lives from the authored narratives of patients with cancer, aiming ultimately to use them as safety monitoring support tools for individual patients. Objective This study was designed to evaluate whether our deep learning models can screen clinically important adverse event signals that require intervention by health care professionals. The applicability of our deep learning models to data on patients’ concerns at pharmacies was also assessed. Methods Pharmaceutical care records at community pharmacies were used for the evaluation of our deep learning models. The records followed the SOAP format, consisting of subjective (S), objective (O), assessment (A), and plan (P) columns. Because of the unique combination of patients’ concerns in the S column and the professional records of the pharmacists, this was considered a suitable data for the present purpose. Our deep learning models were applied to the S records of patients with cancer, and the extracted adverse event signals were assessed in relation to medical actions and prescribed drugs. Results From 30,784 S records of 2479 patients with at least 1 prescription of anticancer drugs, our deep learning models extracted true adverse event signals with more than 80% accuracy for both hand-foot syndrome (n=152, 91%) and adverse events limiting patients’ daily lives (n=157, 80.1%). The deep learning models were also able to screen adverse event signals that require medical intervention by health care providers. The extracted adverse event signals could reflect the side effects of anticancer drugs used by the patients based on analysis of prescribed anticancer drugs. “Pain or numbness” (n=57, 36.3%), “fever” (n=46, 29.3%), and “nausea” (n=40, 25.5%) were common symptoms out of the true adverse event signals identified by the model for adverse events limiting patients’ daily lives. Conclusions Our deep learning models were able to screen clinically important adverse event signals that require intervention for symptoms. It was also confirmed that these deep learning models could be applied to patients’ subjective information recorded in pharmaceutical care records accumulated during pharmacists’ daily work.

show abstract

Predicting patient reported outcome measures: a scoping review for the artificial intelligence-guided patient preference predictor

Balch,

Chatham,

Hong

et al. 2024

Front. Artif. Intell.

View full text Add to dashboard Cite

BackgroundThe algorithmic patient preference predictor (PPP) has been proposed to aid in decision making for incapacitated patients in the absence of advanced directives. Ethical and legal challenges aside, multiple practical barriers exist for building a personalized PPP. Here, we examine previous work using machine learning to predict patient reported outcome measures (PROMs) for capacitated patients undergoing diverse procedures, therapies, and life events. Demonstrating robust performance in predicting PROMs for capacitated patients could suggest opportunities for developing a model tailored to incapacitated ones.MethodsWe performed a scoping review of PubMed, Embase, and Scopus using the PRISMA-ScR guidelines to capture studies using machine learning to predict PROMs following a medical event alongside qualitative studies exploring a theoretical PPP.ResultsSixty-eight studies used machine learning to evaluate PROMs; an additional 20 studies focused on a theoretical PPP. For PROMs, orthopedic surgeries (n = 33) and spinal surgeries (n = 12) were the most common medical event. Studies used demographic (n = 30), pre-event PROMs (n = 52), comorbidities (n = 29), social determinants of health (n = 30), and intraoperative variables (n = 124) as predictors. Thirty-four different PROMs were used as the target outcome. Evaluation metrics varied by task, but performance was overall poor to moderate for the best reported scores. In models that used feature importance, pre-event PROMs were the most predictive of post-event PROMs. Fairness assessments were rare (n = 6). These findings reinforce the necessity of the integrating patient values and preferences, beyond demographic factors, to improve the development of personalized PPP models for incapacitated patients.ConclusionThe primary objective of a PPP is to estimate patient-reported quality of life following an intervention. Use of machine learning to predict PROMs for capacitated patients introduces challenges and opportunities for building a personalized PPP for incapacitated patients without advanced directives.

show abstract

Estimating Patient Satisfaction Through a Language Processing Model: Model Development and Evaluation

Cited by 3 publications

References 37 publications

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models (Preprint)

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models (Preprint)

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models

Predicting patient reported outcome measures: a scoping review for the artificial intelligence-guided patient preference predictor

Contact Info

Product

Resources

About