A novel alignment-free method for detection of lateral genetic transfer based on TF-IDF

Abstract

Lateral genetic transfer (LGT) plays an important role in the evolution of microbes. Existing computational methods for detecting genomic regions of putative lateral origin scale poorly to large data. Here, we propose a novel method based on TF-IDF (Term Frequency-Inverse Document Frequency) statistics to detect not only regions of lateral origin, but also their origin and direction of transfer, in sets of hierarchically structured nucleotide or protein sequences. This approach is based on the frequency distributions of k-mers in the sequences. If a set of contiguous k-mers appears sufficiently more frequently in another phyletic group than in its own, we infer that they have been transferred from the first group to the second. We performed rigorous tests of TF-IDF using simulated and empirical datasets. With the simulated data, we tested our method under different parameter settings for sequence length, substitution rate between and within groups and post-LGT, deletion rate, length of transferred region and k size, and found that we can detect LGT events with high precision and recall. Our method performs better than an established method, ALFY, which has high recall but low precision. Our method is efficient, with runtime increasing approximately linearly with sequence length.

Robust Inference of Genetic Exchange Communities from Microbial Genomes Using TF-IDF

“…(D) If the average frequency of k -mers in a potential lateral segment is less than that of all k -mers in the target genome’s own group, then we consider it an inferred lateral segment . Step B implements the IDF component, and step D the TF component (Cong et al, 2016a,b). Pseudocode is available in the Supplementary Material to Cong et al (2016a), and the TF-IDF source code at https://github.com/congyingnan/TF-IDF.git.…”

“…Recently we (Cong et al, 2016a,b) introduced term frequency-inverse document frequency (TF-IDF) as an accurate, scalable approach to infer LGT among microbial genomes. Using TF-IDF, edges represent only lateral signal and can be inferred directly from whole genomes without first parsing them into individual genes.…”

“…Our previous results make it clear that edges, hence dense edge sets in LGT graphs and their biological interpretations, can be sensitive to the choice of TF-IDF parameters. Notably, precision and recall can be sensitive to the size of k (Cong et al, 2016a), and edges to the structure and delineation of groups (Cong et al, 2016b). It may be that different values of k are more sensitive to different assumptions or biological processes; because of this, we are interested in inferences of GECs that are robust to change of k .…”

Recapitulating phylogenies using k-mers: from trees to networks

“…the use of degenerate k -mers, scoring match lengths rather than k -mer composition, and grammar-based techniques; see recent reviews 16, 17 for more detail. Methods for inferring lateral genetic transfer have also been developed 18, 19 . Importantly, evolutionary relationships can also be depicted as a network, with taxa and relationships represented respectively as nodes and edges 20– 24 , rather than as a strictly bifurcating tree.…”

Information extraction from text messages using data mining techniques

“…A SVM model represents examples as points in space, different classes of examples are divided by a certain gap which must be as wide as possible. New examples when mapped into the space are predicted to belong to a class of examples based on which side of the gap they fall [10,18,22]. • KNN Algorithm The output obtained from Support Vector Machines Algorithm [9] are clusters of two sentiments with class labels "normal" and "critical".…”