A Benchmarking Between Deep Learning, Support Vector Machine and Bayesian Threshold Best Linear Unbiased Prediction for Predicting Ordinal Traits in Plant Breeding

Abstract

Genomic selection is revolutionizing plant breeding. However, still lacking are better statistical models for ordinal phenotypes to improve the accuracy of the selection of candidate genotypes. For this reason, in this paper we explore the genomic based prediction performance of two popular machine learning methods: the Multi Layer Perceptron (MLP) and support vector machine (SVM) methods vs. the Bayesian threshold genomic best linear unbiased prediction (TGBLUP) model. We used the percentage of cases correctly classified (PCCC) as a metric to measure the prediction performance, and seven real data sets to evaluate the prediction accuracy, and found that the best predictions (in four out of the seven data sets) in terms of PCCC occurred under the TGLBUP model, while the worst occurred under the SVM method. Also, in general we found no statistical differences between using 1, 2 and 3 layers under the MLP models, which means that many times the conventional neuronal network model with only one layer is enough. However, although even that the TGBLUP model was better, we found that the predictions of MLP and SVM were very competitive with the advantage that the SVM was the most efficient in terms of the computational time required.

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

“…DL is relatively straightforward to implement (https://keras.io/whyuse-keras/), but optimum performance depends on an adequate hyperparameter choice, which is not trivial and requires considerable computational resources (Young et al, 2015;Chan et al, 2018). Although previous, limited evidence does not show a consistent advantage of DL over penalized linear methods for genomic prediction (GP) purposes (González-Recio et al, 2014;Ma et al, 2017;Bellot et al, 2018;Montesinos-López et al, 2018a;Montesinos-López et al, 2018b;Montesinos-López et al, 2019a), more efforts are needed to fully understand the behavior and potential constraints and capabilities of DL in GP scenarios.…”

A multiple-trait Bayesian Lasso for genome-enabled analysis and prediction of complex traits

“…It is risky to make sweeping statements arguing in favor of a specific treatment of data as outcomes are heavily dependent on the biological architecture of the traits considered, and on the data structure as well. The picture emerging from two decades of experience in genome-enabled prediction in the fields of animal and plant breeding is that is largely futile to categorize methods in terms of expected predictive performance using broad criteria, in view of the large variability of performance with respect to data structure for any given prediction machine (Morota and Gianola 2014; Gianola and Rosa 2015; Momen et al 2018; Montesinos-López et al 2019 a,b,c,d; Azodi et al 2019).…”

“…However, it was found that MBL was better than MT Bayesian BLUP for the two pine tree traits. After almost two decades of genome-enabled prediction it is now clear that no universally best prediction machine exists (Gianola et al 2011; Heslot 2012; de los Campos et al 2013; Momen et al 2018; Bellot et al 2018; Montesinos-López et al 2018a, b, c, d) even when non-parametric or deep learning techniques are brought into the comparisons.…”

A stacking ensemble learning framework for genomic prediction

“…A large number of researches had tried to apply single machine learning methods in genomic prediction [11,14,37,38]. However, the single machine learning methods in the most previous studies only performed well on several traits [13,14,38,39]. Therefore, we proposed a new strategy to utilize machine learning methods in genomic prediction.…”

“…Ogutu et al compared the prediction accuracy of random forest (RF), boosting and support vector machine (SVM) with rrBLUP in simulated dataset, in which rrBLUP outperformed the three machine learning methods [13]. Montesinos-López et al compared the prediction performance of multi-layer prediction, support vector machine with the Bayesian threshold genomic best linear unbiased prediction (TGBLUP) and believed that the reliability of two machine learning methods was comparable to TGBLUP, in some case, outperformed TGBLUP [14]. Even though the achievement of ML in GS had not been fantastic, the breeders still had the confidence in exploration of ML because of its outstanding performance in other majors.…”