Chinese named entity recognition method for the finance domain based on enhanced features and pretrained language models

Zhang, Han; Wang, Xinyu; Liu, Junxiu; Zhang, Lei; Ji, Lixia

doi:10.1016/j.ins.2022.12.049

Cited by 25 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our experiment mainly uses the Pytorch and Huggingface libraries. The pretrained model of T5 is the Mengzi‐t5‐base model [32]. The latent variable z dimension in CVAE is set to 768, and the hidden linear size of the CVAE network is also set to 768.…”

Section: Dataset and Simulation Parametersmentioning

confidence: 99%

A semantic and emotion‐based dual latent variable generation model for a dialogue system

Yan

Lou

Chan

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

With the development of intelligent agents pursuing humanisation, artificial intelligence must consider emotion, the most basic spiritual need in human interaction. Traditional emotional dialogue systems usually use an external emotional dictionary to select appropriate emotional words to add to the response or concatenate emotional tags and semantic features in the decoding step to generate appropriate responses. However, selecting emotional words from a fixed emotional dictionary may result in loss of the diversity and consistency of the response. We propose a semantic and emotion-based dual latent variable generation model (Dual-LVG) for dialogue systems, which is able to generate appropriate emotional responses without an emotional dictionary. Different from previous work, the conditional variational autoencoder (CVAE) adopts the standard transformer structure. Then, Dual-LVG regularises the CVAE latent space by introducing a dual latent space of semantics and emotion. The content diversity and emotional accuracy of the generated responses are improved by learning emotion and semantic features respectively. Moreover, the average attention mechanism is adopted to better extract semantic features at the sequence level, and the semi-supervised attention mechanism is used in the decoding step to strengthen the fusion of emotional features of the model. Experimental results show that Dual-LVG can successfully achieve the effect of generating different content by controlling emotional factors.

show abstract

Section: Dataset and Simulation Parametersmentioning

confidence: 99%

A semantic and emotion‐based dual latent variable generation model for a dialogue system

Yan

Lou

Chan

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

show abstract

“…For instance, Srivastava [16] performed named entity recognition based on word embedding and deep learning models for web information security texts. Similarly, Zhang [17] proposed a pre-trained Chinese financial domain named entity recognition model that contains two sub-models for financial entity boundary delineation and financial entity classification. Puccetti [18] provided a patented text named entity recognition system that combines rule-based, gazetteer, and deep learning techniques.…”

Section: Related Workmentioning

confidence: 99%

Named Entity Identification in the Power Dispatch Domain Based on RoBERTa-Attention-FL Model

et al. 2023

View full text Add to dashboard Cite

Named entity identification is an important step in building a knowledge graph of the grid domain, which contains a certain number of nested entities. To address the issue of nested entities in the Chinese power dispatching domain’s named entity recognition, we propose a RoBERTa-Attention-FL model. This model effectively recognizes nested entities using the span representation annotation method. We extract the output values from RoBERTa’s middle 4–10 layers, obtain syntactic information from the Transformer Encoder layers via the multi-head self-attention mechanism, and integrate it with deep semantic information output from RoBERTa’s last layer. During training, we use Focal Loss to mitigate the sample imbalance problem. To evaluate the model’s performance, we construct named entity recognition datasets for flat and nested entities in the power dispatching domain annotated with actual power operation data, and conduct experiments. The results indicate that compared to the baseline model, the RoBERTa-Attention-FL model significantly improves recognition performance, increasing the F1-score by 4.28% to 90.35%, with an accuracy rate of 92.53% and a recall rate of 88.12%.

show abstract

“…The majority of the biomedical text mining community has been using deep learning-based and specifically Transformerbased methods [Miranda-Escalada et al, 2023] to perform several tasks, including NER, RE and Question-Answering. Moreover, Large Language Models have recently emerged as an alternative also for tasks other than Question-Answering [Wang et al, 2023]. We wanted to explore how we could take advantage of deep learning-based methods to improve the JensenLab suite of web resources.…”

Section: Introductionmentioning

confidence: 99%

Improving dictionary-based named entity recognition with deep learning

Nastou,

Koutrouli,

Pyysalo

et al. 2023

Preprint

View full text Add to dashboard Cite

Dictionary-based named entity recognition (NER) allows terms to be detected in a corpus and normalized to biomedical databases and ontologies. However, adaptation to different entity types requires new high-quality dictionaries and associated lists of blocked names for each type. The latter are so far created by identifying cases that cause many false positives through manual inspection of individual names, a process that scales poorly.In this work we aim to improve block lists by automatically identifying names to block, based on the context in which they appear. By comparing results of three well-established biomedical NER methods, we generated a dataset of over 12.5 million text spans where the methods agree on the boundaries and type of entity tagged. These were used to generate positive and negative examples of contexts for four entity types, namely genes, diseases, species and chemicals, which were then used to train a Transformer-based model (BioBERT) to perform entity type classification. Application of the best model (F1-score=96.7%) allowed us to generate a list of problematic names that should be blocked. Introducing this into our system doubled the size of the previous list of corpus-wide blocked names. Additionally, we generated a document-specific list that allows ambiguous names to be blocked in specific documents. These changes boosted text mining precision by ∼5.5% on average, and over 8% for chemical and gene names, positively affecting several biological databases utilizing this NER system to extract associations between biomedical entities, like the STRING database, with only a minor drop in recall (0.6%).

show abstract

Chinese named entity recognition method for the finance domain based on enhanced features and pretrained language models

Cited by 25 publications

References 24 publications

A semantic and emotion‐based dual latent variable generation model for a dialogue system

A semantic and emotion‐based dual latent variable generation model for a dialogue system

Named Entity Identification in the Power Dispatch Domain Based on RoBERTa-Attention-FL Model

Improving dictionary-based named entity recognition with deep learning

Contact Info

Product

Resources

About