Detecting and correcting misclassified sequences in the large-scale public databases

Abstract

Motivation As the cost of sequencing decreases, the amount of data being deposited into public repositories is increasing rapidly. Public databases rely on the user to provide metadata for each submission that is prone to user error. Unfortunately, most public databases, such as non-redundant (NR), rely on user input and do not have methods for identifying errors in the provided metadata, leading to the potential for error propagation. Previous research on a small subset of the non-redundant (NR) database analyzed misclassification based on sequence similarity. To the best of our knowledge, the amount of misclassification in the entire database has not been quantified. We propose a heuristic method to detect potentially misclassified taxonomic assignments in the NR database. We applied a curation technique and quality control to find the most probable taxonomic assignment. Our method incorporates provenance and frequency of each annotation from manually and computationally created databases and clustering information at 95% similarity. Results We found more than 2 million potentially taxonomically misclassified proteins in the NR database. Using simulated data, we show a high precision of 97% and a recall of 87% for detecting taxonomically misclassified proteins. The proposed approach and findings could also be applied to other databases. Availability Source code, dataset, documentation, Jupyter notebooks, and Docker container are available at https://github.com/boalang/nr. Supplementary information Supplementary data are available at Bioinformatics online.

Impacts of environmental stress on resistance and resilience of algal‐associated bacterial communities

“…Determining taxa-function robustness would provide further insight into the ecological resilience of the bacterial community (Eng & Borenstein, 2018). However, functional redundancy is not uniform across different bacterial groups (Griffiths & Philippot, 2013) and the level of functional redundancy within environmental systems remains unclear as the functional capacities of many bacterial groups are yet to be described, or even incorrectly annotated (Allison & Martiny, 2008;Bagheri et al, 2020). Moreover, describing bacterial functional capacities does not elucidate functional expression and individual microbes may modulate their metabolic performance in response to environmental stress (Eng & Borenstein, 2018).…”

Comprehensive Analysis of Non Redundant Protein Database

“…As shown in the previous work [12], the provenance information could be utilized to clean the NR database by assigning more weight to the manually reviewed annotations.…”

“…As it can be seen in Table 2, some proteins have thousands of taxonomic assignments. We previously explored the NR database for the taxonomic misclassified sequences [12]. The non-redundant version of annotations in the NR database improves the usage and querying of the NR database.…”

“…This work is built on top of previous work that we introduced BoaG as a domain-specific language and shared data science Hadoop-based infrastructure for genomic data [12]. We demonstrated the computational power of BoaG on a proof of concept dataset, RefSeq, on a VirtualBox and Docker container.…”

“…We demonstrated the computational power of BoaG on a proof of concept dataset, RefSeq, on a VirtualBox and Docker container. We also showed a use case of BoaG in detecting and correcting misclassified sequences in the NR database [12].…”