Automatic identification of variables in epidemiological datasets using logic regression

Abstract

BackgroundFor an individual participant data (IPD) meta-analysis, multiple datasets must be transformed in a consistent format, e.g. using uniform variable names. When large numbers of datasets have to be processed, this can be a time-consuming and error-prone task. Automated or semi-automated identification of variables can help to reduce the workload and improve the data quality. For semi-automation high sensitivity in the recognition of matching variables is particularly important, because it allows creating software which for a target variable presents a choice of source variables, from which a user can choose the matching one, with only low risk of having missed a correct source variable.MethodsFor each variable in a set of target variables, a number of simple rules were manually created. With logic regression, an optimal Boolean combination of these rules was searched for every target variable, using a random subset of a large database of epidemiological and clinical cohort data (construction subset). In a second subset of this database (validation subset), this optimal combination rules were validated.ResultsIn the construction sample, 41 target variables were allocated on average with a positive predictive value (PPV) of 34%, and a negative predictive value (NPV) of 95%. In the validation sample, PPV was 33%, whereas NPV remained at 94%. In the construction sample, PPV was 50% or less in 63% of all variables, in the validation sample in 71% of all variables.ConclusionsWe demonstrated that the application of logic regression in a complex data management task in large epidemiological IPD meta-analyses is feasible. However, the performance of the algorithm is poor, which may require backup strategies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12911-017-0429-1) contains supplementary material, which is available to authorized users.

A Novel Method for Computationally Efficacious Linear and Polynomial Regression Analytics of Big Data in Medicine

“…However, none of these studies implemented our data transform method to boost linear or polynomial regression models. Since the last decade, there have been several attempts in the existing peer-reviewed literature to implement linear models as well as other machine-learning methods in combination with the data transform function, including logistic regression, regression trees and Fourier transform, logistic regression with Log10 transformation, logistic regression with Ln transformation, multiple linear regression with log10 transformation, cycling regression model with Fourier transform, proportional hazards Cox regression model, time-series analytics regression with Fourier transform, logistic regression with square root and log10 transformation, and proportional hazards model in combination with logistic regression (Lorenz et al, 2017;Menotti, Puddu, & Lanti, 2002;Shaban-Nejad, Michalowski, & Buckeridge, 2018).…”

Using logic regression to characterize extreme heat exposures and their health associations: a time-series study of emergency department visits in Atlanta

“…Logic regression estimates a decision tree constructed using Boolean combinations of binary predictors. Logic regression has been utilized extensively in genetic association studies for identifying high-dimensional interactions [30,31], and has recently been extended to other exposures [32][33][34]. We show how logic regression provides a data-driven approach to construct a daily extreme heat exposure indicator that is binary (i.e., presence versus absence of the exposure) and can capture impacts of sustained extreme exposure over several days (i.e., heat waves).…”