The interplay between the unavoidable various nonlinearities of the direct conversion transmitter, such as local oscillator leakage, power amplifier (PA) nonlinearities, and in-phase and quadrature (I/Q) branch imbalance, and so forth will degrade the communication system performance seriously. To overcome these nonlinear interactive effects, an accurate adaptive sparse behavioral model is proposed for the joint compensation of the transmitter impairments in this article. First, a three-input nonlinear joint compensation model, which is composed of the nonlinear frequency-dependent cross terms between the I and Q branches as well as the magnitudes of the input signal, is developed.Second, to prune the redundant terms and reduce the computational complexity of the full three-input model, an efficient robust quasi-newton-based adaptive greedy algorithm is developed. To verify the performance of the proposed method, the different imperfect transmitters based on single-device GaN Class-F PA and GaN Doherty PA are used for experimental verification and analysis.Experiment results show that the proposed method can efficiently construct a sparse joint compensation model with improved modeling and distortion mitigation capability than the reported I/Q imbalance model, where nonlinear distortion and I/Q imbalance characteristics in the transmitter can be almost completely removed.