Pay Attention to the Ignorable Missing Data Mechanisms! An Exploration of Their Impact on the Efficiency of Regression Coefficients

Despite recent improvements in collected drilling data quality and volume, the actual number of wells being used in studies remain low and are often limited to a single source and oil field, producing results that are prone to overfitting and are non-transferable. In our study, we access oil drilling data from 5 of more than 20 oil drilling companies collected from 2005 to 2016 from our industrial partner to create well drilling duration models for well planning. This project could lead to the creation of more generalized models from larger datasets than others in literature. However, the data is difficult to process without expert knowledge, further complicated by properties such as unharmonized, source-locked, semantic heterogeneity, sparse and unlabelled. Conventional automated methods for feature selection, propositionalization, multi-source, or block-wise missing techniques could not be used. In this paper, we describe our method to assist the Knowledge Discovery in Databases (KDD) Selection stage of the abovementioned data - Feature Selection before Propositionalization (FSbP) via Database Attribute Health Feature Reduction (DAHFR) and Report Feature Correlation Matrix (RFCM), collectively known as FvDR. DAHFR and RFCM are filter-type feature selection techniques that could measure relational missingness and keyword correlations respectively despite the complexity of multi-source oil drilling data. FvDR successfully reduced the scope from 700 tables containing 20,000 columns to 22 tables containing fewer than 707 columns while successfully selecting 13 of 16 relevant tables suggested by literature. Despite the loss of information from limitations of subsequent KDD procedures, preliminary models show promising results with over half the test predictions falling within the 20% error margin required for well planning. FvDR proves to be indispensable in KDD as a FSbP framework as it reduces features for examination and streamlines the research process necessary to understand business rules for data harmonization and propositionalization.

show abstract

Many nonnormalities, one simulation: Do different data generation algorithms affect study results?

Fairchild,

Yin,

Baraldi

et al. 2024

Behav Res

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Pay Attention to the Ignorable Missing Data Mechanisms! An Exploration of Their Impact on the Efficiency of Regression Coefficients

Abstract: This is the supplementary material for the manuscript entitled "Pay attention to the ignorable missing data mechanisms! An exploration of their impact on the efficiency of regression coefficients".

Cited by 3 publications

References 32 publications

Tackling Challenges in Data Pooling: Missing Data Handling in Latent Variable Models with Continuous and Categorical Indicators

Tackling Challenges in Data Pooling: Missing Data Handling in Latent Variable Models with Continuous and Categorical Indicators

Feature selection before propositionalization of multi-source oil drilling data

Many nonnormalities, one simulation: Do different data generation algorithms affect study results?

Contact Info

Product

Resources

About