Using a Probabilistic Model to Assist Merging of Large-Scale Administrative Records

Enamorado, Ted; Fifield, Benjamin; Imai, Kosuke

doi:10.2139/ssrn.3214172

Cited by 95 publications

(105 citation statements)

References 45 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…Next, the use of LexisNexis linkage provided augmented certification records obtained from the Florida State FMO. We also employed the latest linkage software (FastLink 21 ) which in addition to having superior computational efficiency has the advantage of generating error (or “confusion”) tables estimating linkage accuracy. It was estimated that our FCDS linkage was over 99% accurate with a corresponding sensitivity and specificity of 98.9% and 100%, respectively 65 .…”

Section: Discussionmentioning

confidence: 99%

Cancer risk among career male and female Florida firefighters: Evidence from the Florida Firefighter Cancer Registry (1981‐2014)

Lee

Koru‐Sengul

Hernandez

et al. 2020

American J Industrial Med

View full text Add to dashboard Cite

Background Firefighters are at increased risk for select cancers. However, many studies are limited by relatively small samples, with virtually no data on the cancer experience of female firefighters. This study examines cancer risk in over 100,000 career Florida firefighters including 5000 + females assessed over a 34‐year period. Methods Florida firefighter employment records (n = 109 009) were linked with Florida Cancer Data System registry data (1981‐2014; ~3.3 million records), identifying 3760 male and 168 female‐linked primary cancers. Gender‐specific age and calendar year‐adjusted odds ratios (aOR) and 95% confidence intervals for firefighters vs non‐firefighters were calculated. Results Male firefighters were at increased risk of melanoma (aOR = 1.56; 1.39‐1.76), prostate (1.36; 1.27‐1.46), testicular (1.66; 1.34‐2.06), thyroid (2.17; 1.78‐2.66) and late‐stage colon cancer (1.19;1.00‐1.41). Female firefighters showed significantly elevated risk of brain (2.54; 1.19‐5.42) and thyroid (2.42; 1.56‐3.74) cancers and an elevated risk of melanoma that approached statistical significance (1.68; 0.97‐2.90). Among male firefighters there was additional evidence of increased cancer risk younger than the age of 50 vs 50 years and older for thyroid (2.55; 1.96‐3.31 vs 1.69; 1.22‐2.34), prostate (1.88; 1.49‐2.36 vs 1.36; 1.26‐1.47), testicular (1.60; 1.28‐2.01 vs 1.47; 0.73‐2.94), and melanoma (1.87; 1.55‐2.26 vs 1.42; 1.22‐1.66) cancers. Conclusion Male career firefighters in Florida are at increased risk for five cancers with typically stronger associations in those diagnosed younger than the age of 50, while there was evidence for increased thyroid and brain cancer, and possibly melanoma risk in female firefighters. Larger cohorts with adequate female representation, along with the collection of well‐characterized exposure histories, are needed to more precisely examine cancer risk in this occupational group.

show abstract

Section: Discussionmentioning

confidence: 99%

Cancer risk among career male and female Florida firefighters: Evidence from the Florida Firefighter Cancer Registry (1981‐2014)

Lee

Koru‐Sengul

Hernandez

et al. 2020

American J Industrial Med

View full text Add to dashboard Cite

show abstract

“…To this end, the exploratory framework described above can be thought of as modular, in that different steps can be replaced with others as the open‐source code is refined further (such as a methodological advancement similar to Enamorado et al . ). While research involving increasingly bigger, innovative types of open data will only become more important over time (within this domain and others), data originators too have an important role to play in minimizing the costs associated with aggregating and cleaning the data at source.…”

Section: Resultsmentioning

confidence: 97%

“…The gains from the approximate routine are considerably higher than the 30% improvement achieved by Enamorado et al . () when compared against exact matches only. Figure shows what matches occur across which register, highlighting the dominance of the private sector in the local government procurement process.…”

Section: Methodsmentioning

confidence: 99%

“…One especially relevant piece of work is Enamorado et al . (), which develops ‘a fast and scalable algorithm to implement the canonical probabilistic model of record linkage’ able to ‘efficiently handle millions of observations while accounting for missing data and measurement error’. While the scope of this literature is too broad to discuss in detail here, we refer where appropriate to the applicable work in this field at the relevant points in our discussion throughout the remainder of this paper.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

The Keys to Unlocking Public Payments Data

Rahal¹

2018

Kyklos

View full text Add to dashboard Cite

Summary We mechanize some of the richest yet significantly under‐utilized data resources within developed, ‘Open Data' economies. We show how it is possible to scrape, parse, clean and merge tens of thousands of disaggregated public payments datasets in an attempt to bridge the methodological gap between newly available data from the administrative sphere and applications in empirical social science research. We outline techniques to unambiguously link records to various freely available institutional registers. In particular, we offer guidance on overcoming the substantial challenges of heterogeneous provision and administrative recording errors in the absence of Uniform Resource Identifiers, namely in the form of an approximate, domain‐specific ‘record‐linkage' type matching algorithm. As an illuminating example, we construct a cleaned database of 24,581,192 local government payments subject to the Local Transparency Codes which total £169.87bn in value. We overcome various challenges in a detailed examination of the procurement of services by local government from the voluntary sector: an important contemporary issue due to the rise of the ‘Big Society’ political ideology of the early 21st century. Finally, we motivate future work in this area and discuss potential international applications and practical advancements.

show abstract

“…Since then, however, a number of approaches have been developed which rely heavily on data mining and machine learning. One especially relevant piece of work is Enamorado et al (2017), 2 which develops 'a fast and scalable algorithm to implement the canonical probabilistic model of record linkage' able to 'efficiently handle millions of observations while accounting for missing data and measurement error'. While the scope of this literature is too broad to discuss in detail here, we refer where appropriate to the applicable work in this field at the relevant points in our discussion throughout the remainder of this paper.…”

Section: Technical Literaturementioning

confidence: 99%

The Keys to Unlocking Public Payments Data

Rahal

2016

SSRN Journal

View full text Add to dashboard Cite

We [economists] have been using the same data sets over and over again, and since we wanted new answers, we have been developing new econometric techniques to try to transform the data, and get more meaningful information out of them. Just having a fresh, new data set brings a whole new perspective, and I think people are starting to realize that, and gradually people are becoming more interested in data collection itself."

show abstract

Using a Probabilistic Model to Assist Merging of Large-Scale Administrative Records

Cited by 95 publications

References 45 publications

Cancer risk among career male and female Florida firefighters: Evidence from the Florida Firefighter Cancer Registry (1981‐2014)

Cancer risk among career male and female Florida firefighters: Evidence from the Florida Firefighter Cancer Registry (1981‐2014)

The Keys to Unlocking Public Payments Data

The Keys to Unlocking Public Payments Data

Contact Info

Product

Resources

About