ALBERT over Match-LSTM Network for Intelligent Questions Classification in Chinese

Wang, Xiaomin; Wang, Haoriqin; Guo-cheng, Zhao; Liu, Zhichao; Wu, Haiyuan

doi:10.3390/agronomy11081530

Cited by 10 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning techniques (R-BLS and G-BLS) outperform LSTM in text categorization. Entity discovery in complex queries is addressed using semantic features and classification [61][62][63][64][65].…”

Section: Literature Reviewmentioning

confidence: 99%

Detection of Questions from Text Data Using LSTM-Deep Learning Model

Nadir Hussain,

Dr. Sheikh Muhammad Saqib,

Hamza Arif

et al. 2024

VAWKUM trans. comput. sci.

View full text Add to dashboard Cite

This paper discusses the importance of detecting questions in textual data for various applications in natural language processing (NLP), such as question answering and chatbot creation. The proposed approach employs long short-term memory (LSTM) models to accurately identify questions by leveraging the sequential nature of language.The paper highlights that LSTM models address challenges like ambiguous language and varying sentence structures. They allow the model to learn from sequential patterns, crucial for understanding the intent behind the text. The preprocessing steps, including tokenization, embedding, and padding, are detailed to prepare the data for training and testing. The study investigates the impact of hyperparameters like hidden layers, hidden states, and optimizer choice on the LSTM algorithm’s performance. In experiments on benchmark datasets, the proposed LSTM-based approach consistently outperforms conventional machine learning models, achieving a remarkable accuracy of 99.25% on the test dataset. The paper concludes by suggesting future directions, including applyingthe approach to other NLP tasks like named entity recognition, sentiment analysis, and text classification. Further optimization for specific datasets or domains is also encouraged. Overall, this research contributes to robust question detection models in NLP, with potential applications in various fields.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Detection of Questions from Text Data Using LSTM-Deep Learning Model

Nadir Hussain,

Dr. Sheikh Muhammad Saqib,

Hamza Arif

et al. 2024

VAWKUM trans. comput. sci.

View full text Add to dashboard Cite

show abstract

“…Zhou and Zhang [44] developed a medical QA model based on bidirectional encoder representations from transformers (BERT), generative pre-trained transformer 2 (GPT-2), and text-to-text transfer transformer (T5) models, thereby showing improved performance compared to existing systems. Wang et al [45] suggested a classification method based on a lite BERT (ALBERT) and match-long short-term memory (match-LSTM) models to improve the performance of data classification.…”

Section: General Question Answeringmentioning

confidence: 99%

A Question-Answering Model Based on Knowledge Graphs for the General Provisions of Equipment Purchase Orders for Steel Plants Maintenance

Lee

Choi

2023

Electronics

View full text Add to dashboard Cite

Recently, equipment replacement and maintenance repair and operation (MRO) optimization have substantially increased owing to the aging and deterioration of industrial plants, such as steel-making factories in Korea. Therefore, plant owners are required to quickly review equipment supply contracts, i.e., purchase order (PO) documents, with suppliers and vendors. Currently, there is inconsistency in the time and quality required for the PO document review process by engineers, depending on their manual skills and practice. This study developed a general provisions question-answering model (GPQAM) by combining knowledge graph (KG) and question-answering (QA) techniques to search for semantically connected contract clauses through the definition of relationships between entities during the review of equipment purchase contracts. The PO documents analyzed in this case study were based on one steel-making company’s general provisions (GP). GPQAM is a machine learning (ML)-based model with two sub-models (i.e., KG and QA) that automatically generates the most relevant answers to semantic search questions through a cypher query statement in GP for the PO engineers. First, based on the developed GP lexicon and its classifying taxonomy to be stored in the Neo4j graph database (GDB), the KG sub-model finds the corresponding synonyms and consequently shows GP-related information in a graphic form. Second, the QA sub-model is a function to find and answer contract information within the KG and applies pattern-matching technology based on the Aho–Corasick (AC) algorithm. Third, nodes with the meaning most similar to the question are selected using similarity measurement if a response cannot be extracted through the pattern-matching process. Forty-five pilot test questions were created and applied to the GPQAM model evaluation. The F1 score was 82.8%, indicating that the unsupervised training methods developed in this study could be better applied to a semantic QA process in plant engineering documents, where sufficient training data are limited and bargained. An expert survey of PO practitioners confirmed that the semantic QA capability of GPQAM might be efficient and useful for their work. As the first case of applying KG technology to semantic QA for plant equipment PO contracts, this study might be a meaningful contribution to the steel plant industry and, therefore, extended to construction and engineering contract applications.

show abstract

“…In order to compare the performance of Bleem model and other models in extracting document elements, this study selected text similarity model (Islamaj R, 2019) [22] , BI CNN (Yin W, 2015) [23] , ABCNN (Type 3) (Yin W, 2016) [24] , match_ Lstm (Wang X, 2021) [25] , Bert (Uthirapathy S E 2023) [5] serve as the baseline.…”

Section: Performance Comparison and Analysis Of Different Modelsmentioning

confidence: 99%

Design of intelligent legal text analysis and information retrieval system based on BERT model

Li,

Wang

2023

Preprint

View full text Add to dashboard Cite

The amalgamation of information retrieval systems and soft computing techniques establishes a robust framework to confront the challenges and seize the opportunities presented by the vast expanse of big data. As the volume, diversity, and velocity of data continue to proliferate, further advancements in this domain are poised to significantly contribute to various realms, encompassing healthcare, finance, e-commerce, and scientific research, ultimately propelling innovation and facilitating decision-making in the epoch of big data. The application of artificial intelligence (AI) technology to the analysis and comprehension of legal documents holds the potential to expedite the acquisition of case-specific information by legal researchers, thereby expediting their research endeavors. This paper proposes a legal text analysis and retrieval model, rooted in the Bleem model (Bert-based Legal Paper's Element Extracting Model). Initially, our model employs Bert as the coding layer to extract the semantic information embedded within document sentences and element exemplars. Subsequently, we leverage the Attention mechanism to align the semantic essence of element example sentences with document sentences, while simultaneously computing their respective attention weights. Comparative experiments and attention visualization are then employed to validate the efficacy of the Bleem model. The experimental results corroborate the superiority of the Bleem model in terms of accuracy and F1 scores. The visualization of the attention mechanism effectively reveals the inner workings of the Bleem model and unveils its capacity to explore pertinent fragments within document sentences and element examples. Building upon the aforementioned model, we have devised an intelligent legal text analysis and retrieval system, empowering legal researchers to swiftly acquire pivotal data through case briefs. This application has effectively fostered the integration of legal services within the holistic management of public risks.

show abstract

ALBERT over Match-LSTM Network for Intelligent Questions Classification in Chinese

Cited by 10 publications

References 22 publications

Detection of Questions from Text Data Using LSTM-Deep Learning Model

Detection of Questions from Text Data Using LSTM-Deep Learning Model

A Question-Answering Model Based on Knowledge Graphs for the General Provisions of Equipment Purchase Orders for Steel Plants Maintenance

Design of intelligent legal text analysis and information retrieval system based on BERT model

Contact Info

Product

Resources

About