Secure Record Linkage of Large Health Data Sets: Evaluation of a Hybrid Cloud Model

Abstract

Background The linking of administrative data across agencies provides the capability to investigate many health and social issues with the potential to deliver significant public benefit. Despite its advantages, the use of cloud computing resources for linkage purposes is scarce, with the storage of identifiable information on cloud infrastructure assessed as high risk by data custodians. Objective This study aims to present a model for record linkage that utilizes cloud computing capabilities while assuring custodians that identifiable data sets remain secure and local. Methods A new hybrid cloud model was developed, including privacy-preserving record linkage techniques and container-based batch processing. An evaluation of this model was conducted with a prototype implementation using large synthetic data sets representative of administrative health data. Results The cloud model kept identifiers on premises and uses privacy-preserved identifiers to run all linkage computations on cloud infrastructure. Our prototype used a managed container cluster in Amazon Web Services to distribute the computation using existing linkage software. Although the cost of computation was relatively low, the use of existing software resulted in an overhead of processing of 35.7% (149/417 min execution time). Conclusions The result of our experimental evaluation shows the operational feasibility of such a model and the exciting opportunities for advancing the analysis of linkage outputs.

An Effective Entity Resolution Approach for Big Data

“…As Soundex is vulnerable to errors that happen at the prefix of the encoded text, the proposed protocol deploys an optimization to the algorithm by encoding the reverse of the original text with the second phonetic algorithm. Brown et al [22] presented a new hybrid cloud model for PPRL. They used containers to distribute the record linkage workload across multiple nodes.…”

“…AtyImo is implemented over Apache Spark. No blocking or pruning techniques are implemented in [21], [22], [23] except for the last one as Different predicts have been analysed for blocking selection. Chen et al [24] examine the use of Spark-SQL for efficient parallel entity resolution.…”

Validating a novel deterministic privacy-preserving record linkage between administrative & clinical data: applications in stroke research

“…When dealing with health care data, in particular, the lack of direct identifiers often means that a privacy-preserving record linkage (PPRL) is required to link the databases [12,13]; this method ensures that no personal data are revealed in the process of combining the datasets. Due to the potential errors and variation in indirect identifiers (e.g., a patient's name which could match as "Elizabeth", "Elisabeth", or "Liz"), probabilistic privacy-preserving linkages, often using Bloom filter encoding [14,15], have shown great success in health care datasets [13,[16][17][18][19]. Deterministic PPRLs, or combinations between probabilistic and deterministic algorithms, have also become more common and have had demonstrated success using healthcare data [20][21][22].…”