Precious2GPT: the combination of multiomics pretrained transformer and conditional diffusion for artificial multi-omics multi-species multi-tissue sample generation

Abstract

Synthetic data generation in omics mimics real-world biological data, providing alternatives for training and evaluation of genomic analysis tools, controlling differential expression, and exploring data architecture. We previously developed Precious1GPT, a multimodal transformer trained on transcriptomic and methylation data, along with metadata, for predicting biological age and identifying dual-purpose therapeutic targets potentially implicated in aging and age-associated diseases. In this study, we introduce Precious2GPT, a multimodal architecture that integrates Conditional Diffusion (CDiffusion) and decoder-only Multi-omics Pretrained Transformer (MoPT) models trained on gene expression and DNA methylation data. Precious2GPT excels in synthetic data generation, outperforming Conditional Generative Adversarial Networks (CGANs), CDiffusion, and MoPT. We demonstrate that Precious2GPT is capable of generating representative synthetic data that captures tissue-and age-specific information from real transcriptomics and methylomics data. Notably, Precious2GPT surpasses other models in age prediction accuracy using the generated data, and it can generate data beyond 120 years of age. Furthermore, we showcase the potential of using this model in identifying gene signatures and potential therapeutic targets in a colorectal cancer case study.Biological synthetic data generation in the context of omics refers to the creation of artificial datasets that mimic the characteristics of real biological data, particularly in genomics, transcriptomics, proteomics, and other high-throughput biological technologies 1 . Generating synthetic data is valuable for various reasons, including the development and validation of computational methods, protection of privacy in sensitive datasets, and augmentation of limited real-world data. Generative adversarial networks (GANs) have been introduced as unique models to generate synthetic genomic data, ranging from DNA sequences to bulk RNA-seq data 2,3 . Copula-based methods are other examples of classical statistical approaches in generating synthetic omics data, especially microarray gene expression data 4 . Moreover, Diffusion models are a recent addition to deep learning for synthetic data generation by simulating a diffusion process, which gradually transforms a simple noise distribution into the target data distribution 5 . Large language models (LLMs), exemplified by Generative Pre-trained Transformer 2 (GPT-2), have also garnered substantial interest built upon the Transformer architectures, capturing their significant contributions to the analysis of sequential data, and capabilities in modeling and advanced language understanding, generation and prediction 6 . Although these models have shown promising results in