Automatic Modulation Recognition (AMR) is a fundamental research topic in the field of signal processing and wireless communication, which has widespread applications in cognitive radio, non-collaborative communication, etc. However, current AMR methods are mostly based on unimodal inputs, which suffer from incomplete information and local optimization. In this paper, we focus on the modality utilization in AMR. The proxy experiments show that different modalities achieve a similar recognition effect in most scenarios, while the personalities of different inputs are complementary to each other for particular modulations. Therefore, we mine the universal and complementary characteristics of the modality data in the domain-agnostic and domain-specific aspects, yielding the Universal and Complementary subspaces accordingly (dubbed as UCNet). To facilitate the subspace construction, we propose universal and complementary losses accordingly, where the former minimizes the heterogeneous feature gap by an adversarial constraint and the latter consists of an orthogonal constraint between universal and complementary features. The extensive experiments on the RadioML2016.10A dataset demonstrate the effectiveness of UCNet, which has achieved the highest recognition accuracy of 93.2% at 10 dB, and the average accuracy is 92.6% at high SNR greater than zero.