Evaluating feature-selection stability in next-generation proteomics

Abstract

Identifying reproducible yet relevant features is a major challenge in biological research. This is well documented in genomics data. Using a proposed set of three reliability benchmarks, we find that this issue exists also in proteomics for commonly used feature-selection methods, e.g. [Formula: see text]-test and recursive feature elimination. Moreover, due to high test variability, selecting the top proteins based on [Formula: see text]-value ranks - even when restricted to high-abundance proteins - does not improve reproducibility. Statistical testing based on networks are believed to be more robust, but this does not always hold true: The commonly used hypergeometric enrichment that tests for enrichment of protein subnets performs abysmally due to its dependence on unstable protein pre-selection steps. We demonstrate here for the first time the utility of a novel suite of network-based algorithms called ranked-based network algorithms (RBNAs) on proteomics. These have originally been introduced and tested extensively on genomics data. We show here that they are highly stable, reproducible and select relevant features when applied to proteomics data. It is also evident from these results that use of statistical feature testing on protein expression data should be executed with due caution. Careless use of networks does not resolve poor-performance issues, and can even mislead. We recommend augmenting statistical feature-selection methods with concurrent analysis on stability and reproducibility to improve the quality of the selected features prior to experimental validation.

GFS: fuzzy preprocessing for effective gene expression analysis

“…The idea of fuzzification has also been used earlier in a few gene expression profile analysis methods [7, 8] and also proteomic profile analysis methods [6, 9]. However, these works merely use it as a component of their respective methods, and do not study its role and effectiveness as a normalization procedure.…”

Fuzzy-FishNET: a highly reproducible protein complex-based approach for feature selection in comparative proteomics

“…Although subnets or clusters are predictable from large biological networks, real biological complexes are enriched for biological signal, far outperforming predicted complexes/subnets from reference networks [19, 31, 33, 34]. Here, known human protein complexes derived from the CORUM database are used [35].…”

“…QPSP, and the rank-based network approaches (RBNAs), SNET (SubNET) [29], FSNET (Fuzzy SNET) and PFSNET (Paired FSNET) [30] have been shown to be highly stable and robust, these techniques are similar in that they use a fuzzy weighting system on proteins ranked by expression [31] (see Methods). …”

Class‐paired Fuzzy SubNETs: A paired variant of the rank‐based network analysis family for feature selection based on protein complexes

“…The advent of network-based analysis methods as featureselection methods can help resolve irreproducibility [4,[6][7][8][9]. Whilst design is nontrivial and requires proper integration of bio-statistics, networks and proteomics [4,[6][7][8][9], networkbased approaches are already contributing towards resolving idiosyncratic coverage and consistency problems in clinical proteomics [10][11][12]. Soh et al [13] and Lim et al [14,15] have further demonstrated that network-based feature-selection methods are highly reproducible, and select phenotypically relevant features.…”

“…The advent of network‐based analysis methods as feature‐selection methods can help resolve irreproducibility . Whilst design is nontrivial and requires proper integration of bio‐statistics, networks and proteomics , network‐based approaches are already contributing towards resolving idiosyncratic coverage and consistency problems in clinical proteomics . Soh et al.…”