Joint inversion has drawn considerable attention due to the availability of multiple geophysical datasets, ever-increasing computational resources, development of advanced algorithms, and its ability to reduce inversion uncertainty. A key issue of joint inversion is to develop effective strategies to link different geophysical data in a unified mathematical framework, where the information obtained from different models can complement each other. In this paper, we propose a deep learning enhanced (DLE) joint inversion framework to simultaneously reconstruct different physical models by fusing different types of geophysical data. Traditionally, structure similarity constraints are pursued by joint inversion algorithms using manually crafted formulations (e.g. cross gradient). In this work, the constraint is constructed by a deep neural network (DNN) during the learning process. The framework is designed to combine the DNN and a traditional independent inversion workflow and improve the joint inversion result iteratively. The network can be easily extended to incorporate multi-physics without structural changes. Numerical experiments on the joint inversion of 2D DC resistivity data and seismic travel time are used to validate the proposed method. In addition, this learning-based framework demonstrates excellent generalization abilities when tested on datasets using different geological structures. It can also handle different sensing configurations and nonconforming discretization.