Assessment of transfer methods for comparative genomics of regulatory networks in bacteria

Abstract

BackgroundComparative genomics can leverage the vast amount of available genomic sequences to reconstruct and analyze transcriptional regulatory networks in Bacteria, but the efficacy of this approach hinges on the ability to transfer regulatory network information from reference species to the genomes under analysis. Several methods have been proposed to transfer regulatory information between bacterial species, but the paucity and distributed nature of experimental information on bacterial transcriptional networks have prevented their systematic evaluation.ResultsWe report the compilation of a large catalog of transcription factor-binding sites across Bacteria and its use to systematically benchmark proposed transfer methods across pairs of bacterial species. We evaluate motif- and accuracy-based metrics to assess the results of regulatory network transfer and we identify the precision-recall area-under-the-curve as the best metric for this purpose due to the large class-imbalanced nature of the problem. Methods assuming conservation of the transcription factor-binding motif (motif-based) are shown to substantially outperform those assuming conservation of regulon composition (network-based), even though their efficiency can decrease sharply with increasing phylogenetic distance. Variations of the basic motif-based transfer method do not yield significant improvements in transfer accuracy. Our results indicate that detection of a large enough number of regulated orthologs is critical for network-based transfer methods, but that relaxing orthology requirements does not improve results. Using the transcriptional regulators LexA and Fur as case examples, we also show how DNA-binding domain sequence similarity can yield confounding results as an indicator of transfer efficiency for motif-based methods.ConclusionsCounter to standard practice, our evaluation of metrics to assess the efficiency of methods for regulatory network information transfer reveals that the area under precision-recall (PR) curves is a more precise and informative metric than that of receiver-operating-characteristic (ROC) curves, confirming similar findings in other class-imbalanced settings. Our systematic assessment of transfer methods reveals that simple approaches to both motif- and network-based transfer of regulatory information provide equal or better results than more elaborate methods. We also show that there are not effective predictors of transfer efficacy, substantiating the long-standing practice of manual curation in comparative genomics analyses.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-1113-7) contains supplementary material, which is available to authorized users.

Flexible comparative genomics of prokaryotic transcriptional regulatory networks

“…Experimental information on TF-binding specificity is often available in different reference bacterial species, yet the problem of how to transfer and combine this information to target species in a comparative genomics analysis remains largely unaddressed. TF-binding motif information can be transferred across species, but the efficacy of this process decays with evolutionary distance [ 19 ]. CGB takes in prior knowledge in the form of a list of TF instances (NCBI protein accessions) in different bacterial strains, together with reported (or inferred) TF-binding sites for each of these TF instances.…”

“…Similarly, formal methods to define what constitutes a functional TF-binding site prediction, and the integration of such predictions across multiple genomes to define what constitutes a conserved binding site, have not been implemented in available tools [ 17 , 18 ]. Other issues concern the automated integration of multiple sources of experimental information and the generation of interpretable probabilistic results for gene regulation in the light of operon reorganization [ 19 ]. Here we present CGB, an integrated pipeline for comparative genomics of bacterial regulatory networks with minimal external dependencies that provides a flexible environment for comparative genomics analyses while introducing a formal probabilistic framework for the integration and interpretation of analysis results.…”

CoryneRegNet 7, the reference database and analysis platform for corynebacterial gene regulatory networks

“…The transfer of TRNs has been hindered by the limited availability of experimentally validated data on bacterial TRNs which are available for only a few model organisms such as E. coli, B. subitilis and C. glutamicum 4,14 . TRN transfer from one organism to another largely depends on known regulatory interactions in the model organism as well as genome similarity between model and target organism 4,11,13,14 . Thus, the more experimental TRN data is available for a greater diversity of bacterial species, the higher the quality of any predicted TRNs.…”

“…Thus, the more experimental TRN data is available for a greater diversity of bacterial species, the higher the quality of any predicted TRNs. Furthermore, the ability of detecting regulatory interactions acquired by horizontal gene transfer (HGT) becomes relevant 4,11,14 considering regulatory interactions related to life-style may not be identified by using only one model organism. Using several model organisms allows us to identify these regulations.…”

“…1b. More extensive explanations on genome-scale TRN transfer methods can be found in Baumbach et al 11 and Kiliç et al 14 .…”