LANDMark: an ensemble approach to the supervised selection of biomarkers in high-throughput sequencing data

Abstract

Background Identification of biomarkers, which are measurable characteristics of biological datasets, can be challenging. Although amplicon sequence variants (ASVs) can be considered potential biomarkers, identifying important ASVs in high-throughput sequencing datasets is challenging. Noise, algorithmic failures to account for specific distributional properties, and feature interactions can complicate the discovery of ASV biomarkers. In addition, these issues can impact the replicability of various models and elevate false-discovery rates. Contemporary machine learning approaches can be leveraged to address these issues. Ensembles of decision trees are particularly effective at classifying the types of data commonly generated in high-throughput sequencing (HTS) studies due to their robustness when the number of features in the training data is orders of magnitude larger than the number of samples. In addition, when combined with appropriate model introspection algorithms, machine learning algorithms can also be used to discover and select potential biomarkers. However, the construction of these models could introduce various biases which potentially obfuscate feature discovery. Results We developed a decision tree ensemble, LANDMark, which uses oblique and non-linear cuts at each node. In synthetic and toy tests LANDMark consistently ranked as the best classifier and often outperformed the Random Forest classifier. When trained on the full metabarcoding dataset obtained from Canada’s Wood Buffalo National Park, LANDMark was able to create highly predictive models and achieved an overall balanced accuracy score of 0.96 ± 0.06. The use of recursive feature elimination did not impact LANDMark’s generalization performance and, when trained on data from the BE amplicon, it was able to outperform the Linear Support Vector Machine, Logistic Regression models, and Stochastic Gradient Descent models (p ≤ 0.05). Finally, LANDMark distinguishes itself due to its ability to learn smoother non-linear decision boundaries. Conclusions Our work introduces LANDMark, a meta-classifier which blends the characteristics of several machine learning models into a decision tree and ensemble learning framework. To our knowledge, this is the first study to apply this type of ensemble approach to amplicon sequencing data and we have shown that analyzing these datasets using LANDMark can produce highly predictive and consistent models.

Decision Tree Ensembles Utilizing Multivariate Splits Are Effective at Investigating Beta-Diversity in Medically Relevant 16S Amplicon Sequencing Data

“…In contrast, these structures did not exist in LANDMark (Oracle) models, implying the learning of a smoother boundary. This is consistent with other work involving this class of classifiers (14,37).…”

“…Regardless of which classifier was used, the first principal component in each PCoA projection explained a large amount of the variance in the decision space. This suggests that each classifier can learn good decision rules which separate different classes of samples (14,37). However, due to the small number of samples, the PCoA results for the higher components should be interpreted with some caution.…”

“…Therefore, the impact of uninformative ASVs tends to be minimized since they are not selected as often. LANDMark (Oracle) extends this idea by identifying which linear or non-linear model is best at discriminating between classes using a randomly selected coalition of ASVs (37).…”

“…In contrast, we used 128 trees and split our data into training and testing sets using repeated split-half cross-validation. Previous work has demonstrated that after 128 trees the performance of a RF tends to plateau (30,37). Some additional testing using the various subsets of the IMID dataset demonstrated that adding additional trees to our analysis is unlikely to result in substantially better performance (Suppl Table 1).…”

“…This form of tree construction has also been shown to have a strong regularizing effect, which could limit the amount of available information upon which decisions are made (57). While it is likely that a regularization effect similar to that observed in RF and ET occurs in LANDMark, the strength of this effect may be more muted because LANDMark considers more features at each node (37).…”