A Graph Matching Method for Historical Census Household Linkage

Fu, Zhichun; Christen, Peter; Zhou, Jun

doi:10.1007/978-3-319-06608-0_40

Cited by 25 publications

(20 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose a graph-based solution to address the scalability problem. 2,[7][8][9][10][11][12] We define nodes as unique sets of patient identifying information as recorded in the NHLS database. We define weighted edges as the scored comparisons between these nodes, with scores calculated using a modified version of the Fellegi-Sunter approach.…”

Section: Using Graphs To Guide Record Linkagementioning

confidence: 99%

Building a National HIV Cohort from Routine Laboratory Data: Probabilistic Record-Linkage with Graphs

Macleod

Oleinik

et al. 2018

Preprint

View full text Add to dashboard Cite

Background. Chronic disease management requires the ability to link patient records across multiple interactions with the health sector. South Africa's National Health Laboratory Service (NHLS) conducts all routine laboratory monitoring for the country's national public sector HIV program. However, the absence of a validated patient identifier has limited the potential of the NHLS database for epidemiological research, policy evaluation, and longitudinal patient care. We developed and validated a record linkage algorithm, creating a unique patient identifier and enabling analysis of the NHLS database as a national HIV cohort. To our knowledge, this is the first national HIV cohort in any low-or middle-income country. Methods.We linked data on all CD4 counts, HIV viral loads (VL), and ART workup laboratory tests from 2004-2016. Each NHLS laboratory test result is associated with a name, sex, date of birth (DOB), gender, and facility. However, due to typographical and other errors and patient mobility between facilities, different patient specimens may be associated with different sets of identifying information. We developed a graph-based probabilistic record linkage algorithm and used it to construct a unique identifier for all patients with laboratory results in the national HIV program. We used standard probabilistic linkage methods with Jaro-Winkler string comparisons and weights informed by response frequency. We also used graph concepts to guide the linkage in Results. As of December 2016, the NHLS database contained 117 million patient specimens with a CD4, VL, or other laboratory test used in HIV care. These specimens had 63 million unique combinations of patient identifying information. From these data, our matching algorithm identified 11.6 million unique HIV patients who had at least one CD4 count or VL result. These patients 70.9 million total specimens, with a median of 3 specimens per patient (IQR 1 to 8). Sensitivity and PPV of the algorithm were estimated to be 93.7% and 98.6% in manually-matched data, compared to 64.1% and 100.0% for the existing NHLS identifier. We estimated that in 2016 there were 3.35 million patients on ART and virologically monitored, similar to the National Department of Health estimate of 3.50 million. Conclusion.We constructed a South African National HIV Cohort by applying novel graph-based probabilistic record linkage techniques to routinely collected laboratory data, with high sensitivity and positive predictive value. Information on graph structure can guide record linkage in large populations when identifying data are limited.

show abstract

Section: Using Graphs To Guide Record Linkagementioning

confidence: 99%

Building a National HIV Cohort from Routine Laboratory Data: Probabilistic Record-Linkage with Graphs

Macleod

Oleinik

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Most projects in historical record linkage are challenged by low data quality (due to scanning and transcription errors of handwri en forms), as well as a lack of ground truth data (which is difficult and expensive to obtain). erefore, research in this area has concentrated on either exploiting the structure in such data sets (such as households and families) and developed group linkage methods [8,13,14,24] or Algorithm 1: Pair-wise similarity graph generation Input: -R:…”

Section: Related Workmentioning

confidence: 99%

“…As with other historical data sets [1,14], this birth data set has a very small number of unique name values (2,055 first names and only 547 last names). As Figure 3 shows, the frequency distributions of names are also very skewed.…”

Section: Experimental Evaluationmentioning

confidence: 99%

Robust Temporal Graph Clustering for Group Record Linkage

Nanayakkara

Christen

Ranbaduge

2019

Advances in Knowledge Discovery and Data Mining

Self Cite

View full text Add to dashboard Cite

Research in the social sciences is increasingly based on large and complex data collections, where individual data sets from different domains are linked and integrated to allow advanced analytics. A popular type of data used in such a context are historical censuses, as well as birth, death, and marriage certificates. Individually, such data sets however limit the types of studies that can be conducted. Specifically, it is impossible to track individuals, families, or households over time. Once such data sets are linked and family trees spanning several decades are available it is possible to, for example, investigate how education, health, mobility, employment, and social status influence each other and the lives of people over two or even more generations. A major challenge is however the accurate linkage of historical data sets which is due to data quality and commonly also the lack of ground truth data being available. Unsupervised techniques need to be employed, which can be based on similarity graphs generated by comparing individual records. In this paper we present results from clustering birth records from Scotland where we aim to identify all births of the same mother and group siblings into clusters. We extend an existing clustering technique for record linkage by incorporating temporal constraints that must hold between births by the same mother, and propose a novel greedy temporal clustering technique. Experimental results show improvements over non-temporary approaches, however further work is needed to obtain links of high quality.

show abstract

“…census records (Fu et al 2011) and publication records . census records (Fu et al 2011) and publication records .…”

Section: Identification Of Duplicatesmentioning

confidence: 99%

Population Reconstruction

Kirby¹,

Carson²,

Dunlop³

et al. 2015

View full text Add to dashboard Cite

PrefacePeople shape societies. They are linked to each other by family ties and networks with social, economic and religious dimensions. People live together in households and form communities. Some own a house, land and other properties, often related to their profession. And all this is in continuous change. People are born, marry, have children and die, and they change houses and addresses, and build careers. For the study of a society in all aspects, people are at the heart of the problem and should be known in the context of their complex relationships. Even today, it is not easy to get this information in an all-enfolding way, but for populations in the past, it is a real challenge. And that is what this book is about. The book addresses the problems that are encountered, and solutions that have been proposed, when we aim to identify people and to reconstruct populations under conditions where information is scarce, ambiguous, fuzzy and sometimes erroneous.It is not a single discipline that is involved in such an endeavour. Historians, social scientists, and linguists represent the humanities through their knowledge of the complexity of the past, the limitations of sources and the possible interpretations of information. The availability of big data from digitised archives and the need of complex analyses to identify individuals require the involvement of computer scientists. With contributions from all these fields, often in direct cooperation, this book is at the heart of digital humanities and hopefully a source of inspiration for future investigations.The process from handwritten registers to a reconstructed digitised population has three major phases which shape the three sections of this book. The first phase is that of data transcription and digitisation while structuring the information in a meaningful and efficient way. Little of this phase can be automated. With archives that comprise easily tens of millions of records, the help of volunteers for transcription and digitisation is indispensable, but requires a rigorous management. Experiences from Denmark demonstrate the complexity of this task in Chap. 1. Spelling variation, aliases, abbreviations, errors and typos all generate difficulties in further processing and require data cleaning. Similarity measures can be helpful to v

show abstract

A Graph Matching Method for Historical Census Household Linkage

Cited by 25 publications

References 17 publications

Building a National HIV Cohort from Routine Laboratory Data: Probabilistic Record-Linkage with Graphs

Building a National HIV Cohort from Routine Laboratory Data: Probabilistic Record-Linkage with Graphs

Robust Temporal Graph Clustering for Group Record Linkage

Population Reconstruction

Contact Info

Product

Resources

About