Many deep learning approaches have been proposed to connect DNA sequence, epigenetic profiles, chromatin organization and transcription activities. While these approaches achieve satisfactory performance in predicting one modality from another, the representations learned are not generalizable across predictive tasks or across cell types. In this paper, we propose a deep learning approach named EPCOT which employs a pre-training and fine-tuning framework, and comprehensively predicts epigenome, chromatin organization, transcriptome, and enhancer activity in one framework, which is also generalizable to new cell types. EPCOT not only achieves superior predictive performance in individual predictive tasks, it also produces globally optimized sequence representations that are generalizable across different predictive tasks. Interpreting EPCOT model also allows us to provide a number of tools and services to the research community including mapping between different genomic modalities, identifying TF sequence binding patterns, and analyzing cell-type specific TF impacts to enhancer activity.