Bridge inspection named entity recognition via BERT and lexicon augmented machine reading comprehension neural model

Li, Ren; Mo, Tianjin; Yang, Jianxi; Li, Dong; Jiang, Shixin; Wang, Di

doi:10.1016/j.aei.2021.101416

Cited by 34 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Named entities refer to those words or phrases that contain special meaning or strong references [11]. Under normal circumstances, the entity types include some names, place names, and organization names.…”

Section: Nermentioning

confidence: 99%

Legal Text Recognition Using LSTM-CRF Deep Learning Model

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In legal texts, named entity recognition (NER) is researched using deep learning models. First, the bidirectional (Bi)-long short-term memory (LSTM)-conditional random field (CRF) model for studying NER in legal texts is established. Second, different annotation methods are used to compare and analyze the entity recognition effect of the Bi-LSTM-CRF model. Finally, other objective loss functions are set to compare and analyze the entity recognition effect of the Bi-LSTM-CRF model. The research results show that the F1 value of the model trained on the word sequence labeling corpus on the named entity is 88.13%, higher than that of the word sequence labeling corpus. For the two types of entities, place names and organization names, the F1 values obtained by the Bi-LSTM-CRF model using word segmentation are 67.60% and 89.45%, respectively, higher than the F1 values obtained by the model using character segmentation. Therefore, the Bi-LSTM-CRF model using word segmentation is more suitable for recognizing extended entities. The parameter learning result using log-likelihood is better than that using the maximum interval criterion, and it is ideal for the Bi-LSTM-CRF model. This method provides ideas for the research of legal text recognition and has a particular value.

show abstract

Section: Nermentioning

confidence: 99%

Legal Text Recognition Using LSTM-CRF Deep Learning Model

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…For example, in tourism domain, NER is used to label the point of interest, reviews, hotels, location, etc. (Chantrapornchai and Tunsakul, 2021); in medical domain, NER identifies clinical entities in electronic medical records and assigns them to previously defined categories, such as disease, image review, laboratory examination, operation, drug and anatomy (Kong et al , 2021); in commerce domain, NER is introduced to identify entity name of the cross border e-commerce commodity (Luo et al , 2020); in architecture domain, NER also successfully recognizes bridge names, structural members, member elements, locations of members or elements, structural defects and negative descriptions in bridge inspection reports (Li et al , 2021).…”

Section: Related Workmentioning

confidence: 99%

“…For the former, the bidirectional LSTM (BiLSTM)-CRF model is well-known and it works to extract forward and backward text sequence features simultaneously using two stacked LSTM networks (Huang et al , 2015). This model is considered to be the cornerstone of several modified NER models (Li et al , 2021). Besides the RNN-based models, CNN-based models are adopted for NER because of CNN's clear computational advantages, which famously included CNN-CRF (Collobert et al , 2011) and Iterated Dilated Convolutional Neural Network (IDCNN)-CRF (Strubell et al , 2017).…”

Section: Related Workmentioning

confidence: 99%

Identifying business information through deep learning: analyzing the tender documents of an Internet-based logistics bidding platform

2023

DTA

View full text Add to dashboard Cite

PurposeThe tender documents, an essential data source for internet-based logistics tendering platforms, incorporate massive fine-grained data, ranging from information on tenderee, shipping location and shipping items. Automated information extraction in this area is, however, under-researched, making the extraction process a time- and effort-consuming one. For Chinese logistics tender entities, in particular, existing named entity recognition (NER) solutions are mostly unsuitable as they involve domain-specific terminologies and possess different semantic features.Design/methodology/approachTo tackle this problem, a novel lattice long short-term memory (LSTM) model, combining a variant contextual feature representation and a conditional random field (CRF) layer, is proposed in this paper for identifying valuable entities from logistic tender documents. Instead of traditional word embedding, the proposed model uses the pretrained Bidirectional Encoder Representations from Transformers (BERT) model as input to augment the contextual feature representation. Subsequently, with the Lattice-LSTM model, the information of characters and words is effectively utilized to avoid error segmentation.FindingsThe proposed model is then verified by the Chinese logistic tender named entity corpus. Moreover, the results suggest that the proposed model excels in the logistics tender corpus over other mainstream NER models. The proposed model underpins the automatic extraction of logistics tender information, enabling logistic companies to perceive the ever-changing market trends and make far-sighted logistic decisions.Originality/value(1) A practical model for logistic tender NER is proposed in the manuscript. By employing and fine-tuning BERT into the downstream task with a small amount of data, the experiment results show that the model has a better performance than other existing models. This is the first study, to the best of the authors' knowledge, to extract named entities from Chinese logistic tender documents. (2) A real logistic tender corpus for practical use is constructed and a program of the model for online-processing real logistic tender documents is developed in this work. The authors believe that the model will facilitate logistic companies in converting unstructured documents to structured data and further perceive the ever-changing market trends to make far-sighted logistic decisions.

show abstract

“…For example, the previous work [ 42 ] constructed bridge structure and health monitoring ontology using Semantic Web technology, and realized multi-angle fine-grained modeling of bridge structure, SHM system, sensor, and perception data, which lays a foundation for the semantic ontology construction of bridge maintenance. With the in-depth research, Li et al [ 43 ] proposed a dictionary-enhanced machine reading comprehension NER neural model for identifying planes and nested entities from Chinese bridge detection texts. In the previous work [ 44 ], a new entity related attention neural network model was proposed for joint extraction of entities and relationships in bridge inspection.…”

Section: Related Workmentioning

confidence: 99%

Complex Knowledge Base Question Answering for Intelligent Bridge Management Based on Multi-Task Learning and Cross-Task Constraints

Yang

et al. 2022

Entropy

Self Cite

View full text Add to dashboard Cite

In the process of bridge management, large amounts of domain information are accumulated, such as basic attributes, structural defects, technical conditions, etc. However, the valuable information is not fully utilized, resulting in insufficient knowledge service in the field of bridge management. To tackle these problems, this paper proposes a complex knowledge base question answering (C-KBQA) framework for intelligent bridge management based on multi-task learning (MTL) and cross-task constraints (CTC). First, with C-KBQA as the main task, part-of-speech (POS) tagging, topic entity extraction (TEE), and question classification (QC) as auxiliary tasks, an MTL framework is built by sharing encoders and parameters, thereby effectively avoiding the error propagation problem of the pipeline model. Second, cross-task semantic constraints are provided for different subtasks via POS embeddings, entity embeddings, and question-type embeddings. Finally, using template matching, relevant query statements are generated and interaction with the knowledge base is established. The experimental results show that the proposed model outperforms compared mainstream models in terms of TEE and QC on bridge management datasets, and its performance in C-KBQA is outstanding.

show abstract

Bridge inspection named entity recognition via BERT and lexicon augmented machine reading comprehension neural model

Cited by 34 publications

References 26 publications

Legal Text Recognition Using LSTM-CRF Deep Learning Model

Legal Text Recognition Using LSTM-CRF Deep Learning Model

Identifying business information through deep learning: analyzing the tender documents of an Internet-based logistics bidding platform

Complex Knowledge Base Question Answering for Intelligent Bridge Management Based on Multi-Task Learning and Cross-Task Constraints

Contact Info

Product

Resources

About