Cognitive radio networks (CRNs) are expected to improve spectrum utilization significantly by allowing secondary users (SUs) to opportunistically access the licensed spectrum of primary users (PUs). In an integrated network consisting of multiple heterogeneous CRNs, SUs with multiple interfaces may have to conduct inter-system or intra-system spectrum handoff due to the arrival of PUs or performance degradation on serving spectrum. In this case, designing an optimal spectrum handoff scheme which offers quality of service (QoS) guarantee and performance enhancement of the SUs is of particular importance. On the other hand, resource allocation strategy on target channel also plays an important role in affecting the transmission performance of handoff SUs. In this paper, we jointly design spectrum handoff and resource allocation strategy for handoff SUs in heterogeneously integrated CRNs. To achieve joint resource management among various CRNs, we propose a joint radio resource management architecture, based on which the proposed spectrum handoff and resource allocation scheme can be conducted. Jointly considering the transmission performance of the handoff SUs, we formulate the total energy efficiency of the SUs and design an optimization problem which maximizes the energy efficiency subject to spectrum handoff, QoS, and power constraints of the SUs. An iterative algorithm is proposed to solve the formulated nonlinear fractional optimization problem. Within each iteration, the optimization problem is transformed equivalently into two subproblems, i.e., power allocation subproblem of each SU-spectrum pair and spectrum handoff subproblem for all the SUs. The two subproblems are solved, respectively, through applying Lagrange dual method and the Kuhn-Munkres (K-M) algorithm. Numerical results demonstrate the efficiency of the proposed algorithm.