Mechanism-aware imputation: a two-step approach in handling missing values in metabolomics

Abstract

When analyzing large datasets from high-throughput technologies, researchers often encounter missing quantitative measurements, which are particularly frequent in metabolomics datasets. Metabolomics, the comprehensive profiling of metabolite abundances, are typically measured using mass spectrometry technologies that often introduce missingness via multiple mechanisms: (1) the metabolite signal may be smaller than the instrument limit of detection; (2) the conditions under which the data are collected and processed may lead to missing values; (3) missing values can be introduced randomly. Missingness resulting from mechanism (1) would be classified as Missing Not At Random (MNAR), that from mechanism (2) would be Missing At Random (MAR), and that from mechanism (3) would be classified as Missing Completely At Random (MCAR). Two common approaches for handling missing data are the following: (1) omit missing data from the analysis; (2) impute the missing values. Both approaches may introduce bias and reduce statistical power in downstream analyses such as testing metabolite associations with clinical variables. Further, standard imputation methods in metabolomics often ignore the mechanisms causing missingness and inaccurately estimate missing values within a data set. We propose a mechanism-aware imputation algorithm that leverages a two-step approach in imputing missing values. First, we use a random forest classifier to classify the missing mechanism for each missing value in the data set. Second, we impute each missing value using imputation algorithms that are specific to the predicted missingness mechanism (i.e., MAR/MCAR or MNAR). Using complete data, we conducted simulations, where we imposed different missingness patterns within the data and tested the performance of combinations of imputation algorithms. Our proposed algorithm provided imputations closer to the original data than those using only one imputation algorithm for all the missing values. Consequently, our two-step approach was able to reduce bias for improved downstream analyses.

Missing values are informative in label-free shotgun proteomics data: estimating the detection probability curve

“…There has been much discussion in the literature about the relevance of Donald Rubin's MAR, MCAR and MNAR classification for proteomics data [26,15,16,27,28,17,29,30]. Our analysis shows unambiguously that intensity is a strong predictor of the rate of missingness.…”

Neither random nor censored: estimating intensity-dependent probabilities for missing values in label-free proteomics

“…More generally, the slope of the DPC quantifies the amount of information that can be theoretically extracted from the missing value frequencies for estimating peptide intensities or assessing differential expression. Some authors have proposed classifying individual missing values as MAR or left-censored ( Lazar et al 2016 ; Wei et al 2018 ; Liu and Dongre 2021 ; Dekermanjian et al 2022 ) but our work shows that the same DPC can be applied to all values.…”

“…There has been much discussion in the literature about the relevance of Donald Rubin’s missing at random (MAR) and missing not at random (MNAR) classification for mass spectrometry data ( Karpievitch et al 2012 ; Webb-Robertson et al 2015 ; Lazar et al 2016 ; Wang et al 2020 ; Gardner and Freitas 2021 ; Liu and Dongre 2021 ; Dekermanjian et al 2022 ; Shen et al 2022 ). Our work shows that missing intensities are MNAR but that the dependence of missing value frequency on intensity is gradual.…”

Metabolomics and modelling approaches for systems metabolic engineering

“…It is also challenging to identify meaningful variables from the large pool. In this regard, ML approaches can be applied to reduce the feature size by projecting the high dimensional data to a lower dimensional space ( Hira and Gillies, 2015 ; Dekermanjian et al, 2022 ; Faquih et al, 2020 ). Some examples of such ML methods are non-negative matrix factorization (NMF), principal component analysis (PCA), t-distributed stochastic neighbour embedding (t-SNE), and autoencoder (a type of ANN), with the last two especially suited for analysing non-linear data.…”