Diabot: A Predictive Medical Chatbot using Ensemble Learning

Abstract

Accessibility to medical knowledge and healthcare costs are the two major impediments for common man. Conversational agents like Medical chatbots, which are designed keeping in view medical applications can potentially address these issues. Chatbots can either be generic or disease-specific in nature. Diabetes is a non-communicable disease and early detection of the same can let people know about the serious consequences of this disorder and help save human lives. In this paper, we have developed a generic text-to-text ‘Diabot’ – a DIAgnostic chatBOT which engages patients in conversation using advanced Natural Language Understanding (NLU) techniques to provide personalized prediction using the general health dataset and based on the various symptoms sought from the patient. The design is further extended as a DIAbetes chatBOT for specialized Diabetes prediction using the Pima Indian diabetes dataset for suggesting proactive preventive measures to be taken. For prediction, there exists multiple classification algorithms in Machine Learning which can be used based on their accuracy. However, rather than considering only one model and hoping this model is the best or most accurate predictor we can make, the novelty in this paper lies in Ensemble learning, which is a meta-algorithm that combines a myriad of weaker models and averages them to produce one final balanced and accurate model. From literature reviews, it is observed that very little research has happened in ensemble methods to increase prediction accuracy. The paper presents a state-of-the art Diabot design with an undemanding front-end interface for common man using React UI, RASA NLU based text pre-processing, quantitative performance comparison of various machine learning algorithms as standalone classifiers and combining them all in a majority voting ensemble. It is observed that the chatbot is able to interact seamlessly with all patients based on the symptoms sought. The accuracy of Ensemble model is balanced for general health prediction and highest for diabetes prediction among all weak learners considered which provides motivation for further exploring ensemble techniques in this domain.

The role of knowledge graphs in chatbots

“…Beyond NLP, other machine learning algorithms such as ensemble learning, artificial neural networks, binary regression and classification were also used to analyze the inputs and answer the user queries. For example, Bali et al. (2019) used ensemble learning to predict user disease based on the user’s symptoms in the token or processed string format.…”

A Systematic Literature Review: A Comparison Of Available Approaches In Chatbot And Dialogue Manager Development

“…Recurrent Neural Network (RNN), as referenced in [22], [23], is particularly well-suited for text-based chatbots, excelling in natural language understanding. Ensemble Learning ( [19], [20], [21]) and Long-Short Term Memory (LSTM) ( [24], [25], [26], [27]) algorithms offer enhanced performance, making them valuable choices for chatbot development in specific domains. Dialogue Manager is the core component in chatbot development, overseeing conversations and interactions.…”

“…This approach is usually found with unique purposes, such as fashion chatbots or diabetes treatment chatbot [20], [22]. Ensemble Learning can help fortify weaker models and as computational power continues to grow, its use case can become even more common [19].…”

“…Ensemble Learning can help fortify weaker models and as computational power continues to grow, its use case can become even more common [19]. A recent study has shown that the use of metaalgorithms such as ensemble learning can help chatbots achieve superior generalization ability and help map strong correlations better in a humane and conversational context [20]. Other than that, RNN is also another popular algorithm that is used in the development of chatbots.…”

“…Other than that, using ensemble learning techniques can help make a more robust model that is content aware in a dialogue management system. By combining models via ensemble learning, the chatbot developed can have more generalization over a humane conversation and help map strong correlation efficiently [20]. These understanding can help optimize and harness the true potential of each algorithm to enhance the performance and effectiveness of algorithms for different use cases.…”