AI Medical School Tutor: Modelling and Implementation

Afzal, Shazia; Dhamecha, Tejas I.; Gagnon, Paul; Nayak, Akash; Shah, Ayush; Carlstedt‐Duke, Jan; Pathak, Smriti; Mondal, Sneha; Gugnani, Akshay; Zary, Nabil; Chetlur, Malolan

doi:10.1007/978-3-030-59137-3_13

Cited by 6 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the tutoring system should not merely be a knowledge-transfer system, but one designed on current educational principles such as cased-based, problem-based, and complex scenario-based gamification that can move beyond managing a patient with a single condition under contextless circumstances. Interaction with the student can utilise Natural Language Processing (NLP) ( Chary et al , 2019 ), and these systems can then present flexible and realistic virtual patients, constructed by AI, based upon real patient records ( Wijayarathna and Zary, 2019 ; Afzal et al , 2020 ), requiring critical thinking and clinical reasoning, and adjusting the levels of complexity to better suit the level at which the student is expected to perform. One can then have effective virtual (or even robotic) patient case simulation for general use ( Hayasaka, Fujikura and Kashimura, 2018 ; Laleye et al , 2020 ) or within specific environments, such as single or multi-centre PBL ( Caudell et al , 2003 ; Hamdy et al , 2017 ).…”

Section: Administration and Methodologymentioning

confidence: 99%

Artificial Intelligence developments in medical education: a conceptual and practical framework

Masters¹

2020

MedEdPublish

View full text Add to dashboard Cite

As Artificial Intelligence (AI) develops in medicine, there is a greater awareness that medical education may also benefit from AI. Many AI projects are already underway, and many more are still to come. Most medical education administrators and educators are aware of AI, but are not necessarily familiar enough with it to understand the areas of possible application in both using AI in medical education, and also in the content areas that need to be raised with their students. As such, they are at a disadvantage of not understanding the current lie of the land, and are at an even greater disadvantage of not being able to influence and guide future AI projects. This paper attempts to provide an AI conceptual and practical framework for medical education administrators and educators, so that they may have a clearer understanding of the current situation, and may be better placed to guide future AI developments to meet their needs in medical education.

show abstract

Section: Administration and Methodologymentioning

confidence: 99%

Artificial Intelligence developments in medical education: a conceptual and practical framework

Masters¹

2020

MedEdPublish

View full text Add to dashboard Cite

show abstract

“…Furthermore, through adequate metrics obtained from the analyses of the user's logged actions, VPSs may generate a multidimensional representation of the students' medical competence, thus providing teachers with potentially valuable didactical information [9][10][11][12][13]. VPSs may include the use of natural language processing (NLP) techniques to better mimic physician-patient interactions and facilitate the use of these techniques by medical school students [13][14][15].…”

Section: Introductionmentioning

confidence: 99%

“…In many VPSs, metrics are set up to merely assess sectorial aspects of the overall patient's diagnostic management, such as history taking [14] or clinical examination [13], whereas, in other VPSs, crucial diagnostic activities such as conducting a physical examination and ordering medical tests are not considered [15]. Therefore, many VPSs and their relative metrics aim to address specific didactical items rather than embracing the overall clinical diagnostic approach.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, many VPSs and their relative metrics aim to address specific didactical items rather than embracing the overall clinical diagnostic approach. The latter is crucial in undergraduate medical training as most diagnostic errors made by junior physicians are caused by flaws in data collection or data integration [15]. There is a need for novel VPSs that target all areas of the diagnostic process while maintaining the user-friendly features provided by NLP techniques.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Analytics Applied to Clinical Diagnostic Reasoning Using a Natural Language Processing–Based Virtual Patient Simulator: Case Study

Furlan¹,

Gatti²,

Menè³

et al. 2022

JMIR Med Educ

View full text Add to dashboard Cite

Background Virtual patient simulators (VPSs) log all users’ actions, thereby enabling the creation of a multidimensional representation of students’ medical knowledge. This representation can be used to create metrics providing teachers with valuable learning information. Objective The aim of this study is to describe the metrics we developed to analyze the clinical diagnostic reasoning of medical students, provide examples of their application, and preliminarily validate these metrics on a class of undergraduate medical students. The metrics are computed from the data obtained through a novel VPS embedding natural language processing techniques. Methods A total of 2 clinical case simulations (tests) were created to test our metrics. During each simulation, the students’ step-by-step actions were logged into the program database for offline analysis. The students’ performance was divided into seven dimensions: the identification of relevant information in the given clinical scenario, history taking, physical examination, medical test ordering, diagnostic hypothesis setting, binary analysis fulfillment, and final diagnosis setting. Sensitivity (percentage of relevant information found) and precision (percentage of correct actions performed) metrics were computed for each issue and combined into a harmonic mean (F1), thereby obtaining a single score evaluating the students’ performance. The 7 metrics were further grouped to reflect the students’ capability to collect and to analyze information to obtain an overall performance score. A methodological score was computed based on the discordance between the diagnostic pathway followed by students and the reference one previously defined by the teacher. In total, 25 students attending the fifth year of the School of Medicine at Humanitas University underwent test 1, which simulated a patient with dyspnea. Test 2 dealt with abdominal pain and was attended by 36 students on a different day. For validation, we assessed the Spearman rank correlation between the performance on these scores and the score obtained by each student in the hematology curricular examination. Results The mean overall scores were consistent between test 1 (mean 0.59, SD 0.05) and test 2 (mean 0.54, SD 0.12). For each student, the overall performance was achieved through a different contribution in collecting and analyzing information. Methodological scores highlighted discordances between the reference diagnostic pattern previously set by the teacher and the one pursued by the student. No significant correlation was found between the VPS scores and hematology examination scores. Conclusions Different components of the students’ diagnostic process may be disentangled and quantified by appropriate metrics applied to students’ actions recorded while addressing a virtual case. Such an approach may help teachers provide students with individualized feedback aimed at filling competence drawbacks and methodological inconsistencies. There was no correlation between the hematology curricular examination score and any of the proposed scores as these scores address different aspects of students’ medical knowledge.

show abstract

“…ITS technologies can be adapted to students' specific learning needs, thus potentially increasing the simulator's teaching effectiveness [15][16][17]. Natural language processing (NLP) may complement and support medical education techniques [18], particularly where the diagnostic reasoning aspect is concerned [15,[19][20][21][22]. Notably, the combined use of NLP and ITS technologies in the simulation of virtual patients might promote students' learning by making the student-software interaction more similar to a real-life scenario, while simultaneously giving the student appropriate feedback after every simulated medical activity.…”

Section: Introductionmentioning

confidence: 99%

A Natural Language Processing–Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process: Simulator Development and Case Study

Furlan¹,

Gatti²,

Menè³

et al. 2021

JMIR Med Inform

View full text Add to dashboard Cite

Background Shortage of human resources, increasing educational costs, and the need to keep social distances in response to the COVID-19 worldwide outbreak have prompted the necessity of clinical training methods designed for distance learning. Virtual patient simulators (VPSs) may partially meet these needs. Natural language processing (NLP) and intelligent tutoring systems (ITSs) may further enhance the educational impact of these simulators. Objective The goal of this study was to develop a VPS for clinical diagnostic reasoning that integrates interaction in natural language and an ITS. We also aimed to provide preliminary results of a short-term learning test administered on undergraduate students after use of the simulator. Methods We trained a Siamese long short-term memory network for anamnesis and NLP algorithms combined with Systematized Nomenclature of Medicine (SNOMED) ontology for diagnostic hypothesis generation. The ITS was structured on the concepts of knowledge, assessment, and learner models. To assess short-term learning changes, 15 undergraduate medical students underwent two identical tests, composed of multiple-choice questions, before and after performing a simulation by the virtual simulator. The test was made up of 22 questions; 11 of these were core questions that were specifically designed to evaluate clinical knowledge related to the simulated case. Results We developed a VPS called Hepius that allows students to gather clinical information from the patient’s medical history, physical exam, and investigations and allows them to formulate a differential diagnosis by using natural language. Hepius is also an ITS that provides real-time step-by-step feedback to the student and suggests specific topics the student has to review to fill in potential knowledge gaps. Results from the short-term learning test showed an increase in both mean test score (P<.001) and mean score for core questions (P<.001) when comparing presimulation and postsimulation performance. Conclusions By combining ITS and NLP technologies, Hepius may provide medical undergraduate students with a learning tool for training them in diagnostic reasoning. This may be particularly useful in a setting where students have restricted access to clinical wards, as is happening during the COVID-19 pandemic in many countries worldwide.

show abstract

AI Medical School Tutor: Modelling and Implementation

Cited by 6 publications

References 16 publications

Artificial Intelligence developments in medical education: a conceptual and practical framework

Artificial Intelligence developments in medical education: a conceptual and practical framework

Learning Analytics Applied to Clinical Diagnostic Reasoning Using a Natural Language Processing–Based Virtual Patient Simulator: Case Study

A Natural Language Processing–Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process: Simulator Development and Case Study

Contact Info

Product

Resources

About