Automatic Record Linkage of Individuals and Households in Historical Census Data

Abstract

Linking historical census data is an important task for the study of the social, economic, and demographic aspects of families and society in the past. Although various (semi-) automatic linking methods have been proposed, stateof-the-art methods have only been targeted at linking records that correspond to individuals. In this paper, we introduce an automatic method aimed at linking both individuals and households across several historical census datasets. The proposed method contains several steps, including data quality analysis and enhancement, household identity detection, as well as individual and household record linking. We have applied this method to a set of six census datasets collected from the district of Rawtenstall in North-East Lancashire in the United Kingdom between 1851 and 1901. Experimental results show that the proposed method can greatly reduce the ambiguity arising from the individual record linkage, and facilitate the accurate matching of households across several decades.

Dynamic data science and official statistics

“…In other situations there may be no applicable numeric identifier. An interesting article by Fu et al () develops a methodology for automatic linkage of persons through linkage of households in historic census data. First, in two successive censuses that have been transcribed electronically and cleaned, households are delineated, and persons are matched (with some ambiguity) by automatic supervised learning or threshold techniques on name (approximate), location, sex, and birth year (approximate).…”

Population Reconstruction

“…A first wave of studies using manual techniques (Steckel 1988;Knights 1991;Ferrie 1996Ferrie , 1999 has been followed by the use of machine-learning methodology (Ruggles 2006;Goeken et al 2011;Fu et al 2014). A first wave of studies using manual techniques (Steckel 1988;Knights 1991;Ferrie 1996Ferrie , 1999 has been followed by the use of machine-learning methodology (Ruggles 2006;Goeken et al 2011;Fu et al 2014).…”

Mining microdata: Economic opportunity and spatial mobility in Britain and the United States, 1850–1881

“…We use both the open-source "Freely Extensible Biomedical Record Linkage" (FEBRL) software [47][48][49][50] and a new implementation of distance function routines written by Guelph post-doctoral researcher Luiza Antonie customized for large historical datasets [51][52][53]. Other linking variables-such as birthplace and sex-do not pose string comparison problems because they are numerically coded to eliminate spelling variation.…”