Automatic Deidentification of French Electronic Health Records: A Cost-Effective Approach Exploiting Distant Supervision and Deep Learning Models

Abstract

Background: Electronic health records (EHRs) contain valuable information for clinical research; however, the sensitive nature of healthcare data presents security and confidentiality challenges. Deidentification is therefore essential to protect personal data in EHRs and comply with government regulations. Named entity recognition (NER) methods have been proposed to remove personal identifiers, with deep learning-based models achieving better performance. However, manual annotation of training data is time-consuming and expensive. The aim of this study was to develop an automatic deidentification pipeline for all kinds of clinical documents based on a distant supervised method to significantly reduce the cost of manual annotations and to facilitate the transfer of the deidentification pipeline to other clinical centers. Methods: We proposed an automated annotation process for French clinical deidentification, exploiting data from the eHOP clinical data warehouse(CDW) of the CHU de Rennes and national knowledge bases, as well as other features. In addition, this paper proposes an assisted data annotation solution using the Prodigy annotation tool. This approach aims to reduce the cost required to create a reference corpus for the evaluation of state-of-the-art NER models. Finally, we evaluated and compared the effectiveness of different NER methods. Results: A French deidentification dataset was developed in this work, based on EHRs provided by the eHOP CDW at Rennes University Hospital, France. The dataset was rich in terms of personal information, and the distribution of entities was quite similar in the training and test datasets. We evaluated a Bi-LSTM + CRF sequence labeling architecture, combined with Flair + FastText word embeddings, on a test set of manually annotated clinical reports. The model outperformed the other tested models with a significant F1 score of 96,96%, demonstrating the effectiveness of our automatic approach for deidentifying sensitive information. Conclusions: This study provides an automatic deidentification pipeline for clinical notes, which can facilitate the reuse of EHRs for secondary purposes such as clinical research. Our study highlights the importance of using advanced NLP techniques for effective de-identification, as well as the need for innovative solutions such as distant supervision to overcome the challenge of limited annotated data in the medical domain.

Anonymizing medical documents with local, privacy preserving large language models: The LLM-Anonymizer

“…29 , LLMs fine-tuned for medical purposes demonstrated a slightly worse performance (precision of 0.91, a recall of 0.95) in anonymizing medical documents. 27 We are the first to show the high performance of inference from local LLMs’, specifically Llama-2 and −3 models, in extracting PII from medical documents, whereas others achieved only insufficient results: Liu et al tried zero-shot medical text anonymization with GPT-4 and Llama models, but the Llama models failed to generate any relevant anonymization output for tested medical documents with an accuracy of 0.61. GPT-4 demonstrated an superior accuracy of 0.908 for implicit and 0.99 for explicit shot prompting on their synthetic medical dataset.…”

“…Existing NLP NER approaches have demonstrated notable performance for anonymizing medical documents (rule based: recall 0.95, precision 0.93). 27 Recent advancements in transformer models have showcased similar capabilities in various NLP tasks (precision and recall 0.94). 28 LLMs like GPT-4 have proven to possess advanced anonymization skills.…”