Assessment of current taxonomic assignment strategies for metabarcoding eukaryotes

Abstract

The effective use of metabarcoding in biodiversity science has brought important analytical challenges due to the need to generate accurate taxonomic assignments. The assignment of sequences to genus or species level is critical for biodiversity surveys and biomonitoring, but it is particularly challenging as researchers must select the approach that best recovers information on species composition. This study evaluates the performance and accuracy of seven methods in recovering the species composition of mock communities by using COI barcode fragments. The mock communities varied in species number and specimen abundance, while upstream molecular and bioinformatic variables were held constant, and using a set of COI fragments. We evaluated the impact of parameter optimization on the quality of the predictions. Our results indicate that BLAST top hit competes well with more complex approaches if optimized for the mock community under study. For example, the two machine learning methods that were benchmarked proved more sensitive to reference database heterogeneity and completeness than methods based on sequence similarity. The accuracy of assignments was impacted by both species and specimen counts (query compositional heterogeneity) which ultimately influence the selection of appropriate software. We urge researchers to: (i) use realistic mock communities to allow optimization of parameters, regardless of the taxonomic assignment method employed; (ii) carefully choose and curate the reference databases including completeness; and (iii) use QIIME, BLAST or LCA methods, in conjunction with parameter tuning to better assign taxonomy to diverse communities, especially when information on species diversity is lacking for the area under study.

Just keep it simple? Benchmarking the accuracy of taxonomy assignment software in metabarcoding studies

“…The benchmarking results from Hleap et al (2021) also convey an important warning about the interdependence of taxonomy assignment workflows and reference databases. Achieving adequate taxonomic coverage in reference databases is of paramount importance for all metabarcoding workflows, especially for studies of poorly sampled taxonomic groups and geographic regions -notably, only BLAST and the QIIME2 feature classifier were able to sufficiently cope with existing gaps in reference databases.…”

“…Mock communities were designed to be compositionally heterogeneous (including varying numbers of taxonomic groups and individuals per taxa), thus mimicking community DNA obtained from natural ecosystems. Next, Hleap et al (2021) manually constructed a well-annotated reference database of COI barcodes, including realistic subsampling and gene alignments required for phylogenetic and probabilistic taxonomy classifiers. Finally, the performance of taxonomy assignment software was objectively compared using a suite of defined metrics, including scoring of false discovery rates (the proportion of false predictions), true positive rates (the proportion of true positive predictions), and the Matthews correlation coefficient (a metric comparing the observed versus predicted taxonomy classification), as well as other composite metrics.…”

“…The methods implemented here have obvious relevance to other metabarcoding loci and taxonomic groups. Hleap et al (2021) placed strong emphasis on reproducibility so that these benchmarking exercises can be easily extended to other data sets: all scripts, pipelines, and study outputs have been made freely available in an open source GitHub repository (https://github.com/jshle ap/TA_pipes).…”

“…The results of Hleap et al (2021) challenge the assumption that complex mathematical approaches return more accurate taxonomy assignments for metabarcoding amplicons. In fact, "simple" tools outperformed all other methods, with the QIIME2 feature classifier and BLAST top hit ranking first and second in accuracy of taxonomic classification, respectively.…”

Community metabarcoding reveals the relative role of environmental filtering and dispersal in metacommunity dynamics of soil microarthropods across a mosaic of montane forests

“…Subsequently, we ran blast to compare all ASVs against a combined database composed of the NCBI nt collection (accessed November 2020) and a curated reference catalogue including the 344 Sanger sequences of the 'voucher' specimens plus 561 previously available sequences corresponding to soil lineages of Acari, Collembola and Coleoptera (Arribas et al 2016(Arribas et al , 2021b. Based on the blast output we assigned the ASVs to high-rank taxonomic levels, by applying the weighted lowest common ancestor algorithm in megan6 (Huson et al, 2016; see also Hleap, Littlefair, Steinke, Hebert, & Cristescu, 2021). Only ASVs assigned to Acari, Collembola or Coleoptera were retained and used for downstream analyses.…”