BeyondZar: the use and abuse of classification statistics for otolith chemistry

Abstract

Classification method performance was evaluated using otolith chemistry of juvenile Atlantic menhaden Brevoortia tyrannus when assumptions of data normality were met and were violated. Four methods were tested [linear discriminant function analysis (LDFA), quadratic discriminant function analysis (QDFA), random forest (RF) and artificial neural networks (ANN)] using computer simulation to determine their performance when variable-group means ranged from small to large and their performance under conditions of typical skewness to double the amount of skewness typically observed. Using the kappa index, the parametric methods performed best after applying appropriate data transformation, gaining 2% better performance with LDFA performing slightly better than QDFA. RF performed as well as QDFA and showed no difference in performance between raw and transformed data while the performance of ANN was the poorest and worse with raw data. All methods performed well when group differences were large, but parametric methods outperformed machine-learning methods. When data were skewed the performance of all methods declined and worsened with greater skewness, but RF performed consistently as well or better than the other methods in the presence of skewness. The parametric methods were found to be more powerful when assumptions of normality can be met and can be used confidently when skewness and kurtosis are minimized. When these assumptions cannot be minimized, then machine-algorithm methods should also be tried.

Otolith microchemistry: a useful tool for investigating stock structure of yellowfin tuna (Thunnus albacares) in the Indian Ocean

“…However, in recent years, machine learning techniques have emerged as promising classification tools in otolith-related studies (Zhang et al 2016;Tournois et al 2017;Bouchoucha et al 2018). The accuracy of each technique will depend on the nature of the data analysed, and the results of this study agree with those of recent studies encouraging the use of machine learning methods when otolith chemical data are not multivariate normal or exhibit skewed distributions (Mercier et al 2011;Jones et al 2017). The reasonably easy use of RF, together with a lack of distributional assumption requirements and the robustness of RF against overfitting (Breiman 2001), make this technique a useful approach for population structure analyses of yellowfin tuna in the Indian Ocean.…”

“…LDA and QDA were completed using lda and qda functions from the MASS library; RF was implemented using the randomForest function from the randomForest library, using default settings (Liaw and Wiener 2002) and ANN was performed using the nnet function from the nnet library, using default settings and 30 hidden neurons at the intermediate layer (Venables and Ripley 2002). Detailed descriptions of each technique and their assumptions can be found in Jones et al (2017) and Mercier et al (2011). In all cases, data were randomly split into a training dataset (75%) and a testing dataset (25%) to perform a cross-validation procedure and measure prediction quality.…”

“…Kappa values are always #1, with a value of 1 implying perfect agreement. The kappa index is unbiased when the number of samples differs between groups, and can present more reliable results because it considers the possibility of the agreement occurring by chance (Fielding and Bell 1997;Jones et al 2017). Following Mercier et al (2011), the elemental combination maximising habitat discrimination was selected by performing a stepwise forward variable selection procedure using the Wilk's lambda criterion for LDA and QDA.…”

Combining sale records of landings and fishers knowledge for predicting métiers in a small-scale, multi-gear, multispecies fishery

“…WEKA (Witten and Frank, 2005) implements an exhaustive collection of machine learning algorithms within which there are 17 binary classification algorithms. The algorithm selected was which showed the largest cross-validated predictive capability as measured using kappa index (Jones et al, 2017). This process was fully automatized using the Rweka library (Hornik et al, 2009) from R.…”

Frontiers in otolith chemistry: insights, advances and applications

“…In an important assessment of this and alternative classification methods, Jones et al . () assess classification performance of discriminant function analyses (linear and quadratic) as well as machine‐algorithm methods when otolith data were non‐normal and skewed with and without data transformations. When parametric assumptions were met, the traditional parametric classification methods performed best, indicating that when data can be transformed to meet assumptions, such as with the Box‐Cox transformation, workers should proceed with parametric classifiers.…”