Named Entity Recognition for Sensitive Data Discovery in Portuguese

Dias, Mariana; Boné, João; Ferreira, Joao C Ferreira or Joao; Ribeiro, Ricardo; Maia, Rui Leandro

doi:10.3390/app10072303

Cited by 27 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also released an annotated data set to help researchers make studies to improve their results. In 2020, Mariana et al [20] employed a hybrid approach that consists of rule sets, lexical based models, and machine learning algorithms to detect and make notifications about personal data for Portuguese language. This study extracts only 11 types of personal data entities in 3 categories.…”

Section: Personal Data Discoverymentioning

confidence: 99%

Deep learning-aided automated personal data discovery and profiling

YAYIK,

AYBAR,

APIK

et al. 2022

Turkish Journal of Electrical Engineering and Computer Sciences

View full text Add to dashboard Cite

In Turkey, Turkish Personal Data Protection Rule (PDPR) No. 6698, in force since 2016, provides protection to citizens for the legal existence of their personal data. Although the law provides excellent guidance, companies currently face challenges in complying with its regulations in terms of storing, sharing, or monitoring personal data. Since any specially designed software with wide industrial usage is not on the market, almost all of the companies have no other choice but to take expensive and error-prone operations manually to ensure their compliance. In this paper, we present an automated solution to facilitate and accelerate PDPR compliance. In a structured or unstructured document, words or phrases that could include personal data entities are tagged with the help of a Bi-LSTM based named entity recognition model and a rule-based matching component that employs contextual analysis. To find associations in personal data and obtain individual personal profiles, these entities are divided into categories according to their confidence levels. Personal profiles are constructed using an approach similar to clustering. It treats the personal data categories with high identification levels as separate clusters and finds related personal data entities at the left and/or right of its contexts. We evaluated the system on a data set formed of 70 documents of different types and personal data entities. We obtained 91.76 % micro-averaged F1-measure for personal data entity classification and 72.46 % accuracy for profile extraction. We also performed experiments related to the performance of the named entity recognition and to the time complexity of the overall system on a data set formed of 33K documents.

show abstract

Section: Personal Data Discoverymentioning

confidence: 99%

Deep learning-aided automated personal data discovery and profiling

YAYIK,

AYBAR,

APIK

et al. 2022

Turkish Journal of Electrical Engineering and Computer Sciences

View full text Add to dashboard Cite

show abstract

“…However, more investigations are required to verify these hypotheses, and the effects of NLP models to recognise entities of different classes deserves more research. A possible research path involves the combination of CRFs with BiLSTMs, which can achieve state-ofthe-art performance for specific language resources [10] and be potentially useful in the identification of entities of the ENAMEX group, e.g., [7,31]. In addition, methodological approaches involving latent factor analysis may also contribute to improve computational efficiency and prediction accuracy of trained models for unseen data [32][33][34][35][36] .…”

Section: Influence Of Self-training On Model Performancementioning

confidence: 99%

Iterative Named Entity Recognition with Conditional Random Fields

et al. 2021

View full text Add to dashboard Cite

Named entity recognition (NER) constitutes an important step in the processing of unstructured text content for the extraction of information as well as for the computer-supported analysis of large amounts of digital data via machine learning methods. However, NER often relies on domain-specific knowledge, being conducted manually in a time- and human-resource-intensive process. These can be reduced with statistical models performing NER automatically. The current work investigates whether Conditional Random Fields (CRF) can be efficiently trained for NER in German texts, by means of an iterative procedure combining self-learning with a manual annotation–active learning–component. The training dataset increases continuously with the iterative procedure. Whilst self-learning did not markedly improve the performance of the CRF for NER, the manual annotation of sentences with the lowest probability of correct prediction clearly improved the model F1-score and simultaneously reduced the amount of manual annotation required to train the model. A model with an F1-score of 0.885 was able to be trained in 11.4 h.

show abstract

“…BiLSTM is a new network structure improved on the basis of LSTM [30][31][32]. It consists of the following four layers: input layer, forward LSTM layer, backward LSTM layer and the output layer.…”

Section: Extracting Semantic Information Based On Bilstmmentioning

confidence: 99%

Defect Texts Mining of Secondary Device in Smart Substation with GloVe and Attention-Based Bidirectional LSTM

et al. 2020

View full text Add to dashboard Cite

In the process of the operation and maintenance of secondary devices in smart substation, a wealth of defect texts containing the state information of the equipment is generated. Aiming to overcome the low efficiency and low accuracy problems of artificial power text classification and mining, combined with the characteristics of power equipment defect texts, a defect texts mining method for a secondary device in a smart substation is proposed, which integrates global vectors for word representation (GloVe) method and attention-based bidirectional long short-term memory (BiLSTM-Attention) method in one model. First, the characteristics of the defect texts are analyzed and preprocessed to improve the quality of the defect texts. Then, defect texts are segmented into words, and the words are mapped to the high-dimensional feature space based on the global vectors for word representation (GloVe) model to form distributed word vectors. Finally, a text classification model based on BiLSTM-Attention was proposed to classify the defect texts of a secondary device. Precision, Recall and F1-score are selected as evaluation indicators, and compared with traditional machine learning and deep learning models. The analysis of a case study shows that the BiLSTM-Attention model has better performance and can achieve the intelligent, accurate and efficient classification of secondary device defect texts. It can assist the operation and maintenance personnel to make scientific maintenance decisions on a secondary device and improve the level of intelligent management of equipment.

show abstract

Named Entity Recognition for Sensitive Data Discovery in Portuguese

Cited by 27 publications

References 26 publications

Deep learning-aided automated personal data discovery and profiling

Deep learning-aided automated personal data discovery and profiling

Iterative Named Entity Recognition with Conditional Random Fields

Defect Texts Mining of Secondary Device in Smart Substation with GloVe and Attention-Based Bidirectional LSTM

Contact Info

Product

Resources

About