Evaluation of an international medical E-learning course with natural language processing and machine learning

Abstract

Background In the context of the ongoing pandemic, e-learning has become essential to maintain existing medical educational programmes. Evaluation of such courses has thus far been on a small scale at single institutions. Further, systematic appraisal of the large volume of qualitative feedback generated by massive online e-learning courses manually is time consuming. This study aimed to evaluate the impact of an e-learning course targeting medical students collaborating in an international cohort study, with semi-automated analysis of feedback using text mining and machine learning methods. Method This study was based on a multi-centre cohort study exploring gastrointestinal recovery following elective colorectal surgery. Collaborators were invited to complete a series of e-learning modules on key aspects of the study and complete a feedback questionnaire on the modules. Quantitative data were analysed using simple descriptive statistics. Qualitative data were analysed using text mining with most frequent words, sentiment analysis with the AFINN-111 and syuzhet lexicons and topic modelling using the Latent Dirichlet Allocation (LDA). Results One thousand six hundred and eleventh collaborators from 24 countries completed the e-learning course; 1396 (86.7%) were medical students; 1067 (66.2%) entered feedback. 1031 (96.6%) rated the quality of the course a 4/5 or higher (mean 4.56; SD 0.58). The mean sentiment score using the AFINN was + 1.54/5 (5: most positive; SD 1.19) and + 0.287/1 (1: most positive; SD 0.390) using syuzhet. LDA generated topics consolidated into the themes: (1) ease of use, (2) conciseness and (3) interactivity. Conclusions E-learning can have high user satisfaction for training investigators of clinical studies and medical students. Natural language processing may be beneficial in analysis of large scale educational courses.

Potential Pitfalls With Automatic Sentiment Analysis: The Example of Queerphobic Bias

“…NRC and AFINN were used by two papers. Ultimately, we opted to investigate AFINN because it is older, although still used in recent papers such as Borakati, (2021), and so we felt it was more likely to record queer terms as negative and be potentially biasing contemporary findings. We also found VADER, LIWC and AFINN to be widely used by papers in the SCOPUS database.…”

Discussions of Asperger Syndrome on Social Media: Content and Sentiment Analysis on Twitter

“…The AFINN tool has been used to analyze the sentiment included in tweets about Asperger [ 5 ], and to compare the sentiment towards different COVID-19 vaccines expressed in social media posts [ 27 ]. The AFINN tool has also been used to analyze free short text, such as answers given in surveys [ 28 , 29 ], web-based reviews [ 30 ], self-reported notes [ 31 ], or descriptions of public health campaigns [ 32 ].…”

Analysis of Social Needs for Doctors and Medicine through a Keyword Analysis of Newspaper Articles (2016–2020)

“…This technique encompasses methods such as latent semantic analysis, probabilistic latent semantic analysis, and latent Dirichlet allocation (LDA), with LDA being the most widely recognized topic model [15]. Recently, there has been a surge in research papers employing LDA techniques within the realm of medical education [16][17][18]. LDA topic modeling identifies the extent to which words in one newspaper article co-occur in others, as well as the intermediary relationships between these words.…”