Resting-State Functional Network Scale Effects and Statistical Significance-Based Feature Selection in Machine Learning Classification

Abstract

In recent years, functional brain network topological features have been widely used as classification features. Previous studies have found that network node scale differences caused by different network parcellation definitions significantly affect the structure of the constructed network and its topological properties. However, we still do not know how network scale differences affect the classification accuracy, performance of classification features, and effectiveness of the feature selection strategy using P values in terms of the machine learning method. This study used five scale parcellations, involving 90, 256, 497, 1003, and 1501 nodes. Three local properties of resting-state functional brain networks were selected (degree, betweenness centrality, and nodal efficiency), and the support vector machine method was used to construct classifiers to identify patients with major depressive disorder. We analyzed the impact of the five scales on classification accuracy. In addition, the effectiveness and redundancy of features obtained by the different scale parcellations were compared. Finally, traditional statistical significance (P value) was verified as a feature selection criterion. The results showed that the feature effectiveness of different scales was similar; in other words, parcellation with more regions did not provide more effective discriminative features. Nevertheless, parcellation with more regions did provide a greater quantity of discriminative features, which led to an improvement in the accuracy of the classification. However, due to the close distance between brain regions, the redundancy of parcellation with more regions was also greater. The traditional P value feature selection strategy is feasible with different scales, but our analysis showed that the traditional P < 0.05 threshold was too strict for feature selection. This study provides an important reference for the selection of network scales when applying topological properties of brain networks to machine learning methods.

Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis

“…Nearly half of the models (48.3% [268 of 555]) were found in studies authored by those with academic training in computers and data science (eTable 2 in Supplement 1). Schizophrenia (25.4% [141 of 555 models])…”

“…In the predictors domain, 187 of 555 models (33.7%; 95% CI, 29.9%- 37.6%) were rated with high ROB (Table 1). Defining predictors by knowing the outcome of these models was the unique source of the high ROB in this domain (ie, signaling question 2.2: were predictor assessments made without knowledge of outcome data?).…”

“…Defining predictors by knowing the outcome of these models was the unique source of the high ROB in this domain (ie, signaling question 2.2: were predictor assessments made without knowledge of outcome data?). In the outcome domain, high ROB was scored for 198 of 469 models (35.7%; 95% CI, 31.8%-39.7%) (Table 1). These models had a high ROB because the outcome knowledge of testing data sets was leaked into the predictors of the training set (ie, signaling question 3.3: were predictors excluded from the outcome definition?…”

Feature Selection is Critical for 2-Year Prognosis in Advanced Stage High Grade Serous Ovarian Cancer by Using Machine Learning

“…Classification problems typically involve a high time complexity and low performance when many features are used but will have a low time complexity and high performance for a minimum size and the most effective features. 33 HGSOC prognosis is a complex matter and failure to address this, can lead to a less meaningful interpretation of outcome data. Nevertheless, our effort allowed us to minimise redundancy and identify those discriminant features with the maximal relevance to the 2-year prediction estimation.…”

“…Starting with simple classifiers and then gradually proceeding with more complex classifiers, remains one of the ML principles, which could potentially affect the prediction accuracy of the model. 33 Nonetheless, the ML approach is proving versatile. Both recall and precision, often inversely related, were greater than 80%.…”