Detection of Questions from Text Data Using LSTM-Deep Learning Model

Abstract

This paper discusses the importance of detecting questions in textual data for various applications in natural language processing (NLP), such as question answering and chatbot creation. The proposed approach employs long short-term memory (LSTM) models to accurately identify questions by leveraging the sequential nature of language.The paper highlights that LSTM models address challenges like ambiguous language and varying sentence structures. They allow the model to learn from sequential patterns, crucial for understanding the intent behind the text. The preprocessing steps, including tokenization, embedding, and padding, are detailed to prepare the data for training and testing. The study investigates the impact of hyperparameters like hidden layers, hidden states, and optimizer choice on the LSTM algorithm’s performance. In experiments on benchmark datasets, the proposed LSTM-based approach consistently outperforms conventional machine learning models, achieving a remarkable accuracy of 99.25% on the test dataset. The paper concludes by suggesting future directions, including applyingthe approach to other NLP tasks like named entity recognition, sentiment analysis, and text classification. Further optimization for specific datasets or domains is also encouraged. Overall, this research contributes to robust question detection models in NLP, with potential applications in various fields.