In this work, a multi-task convolutional neural network with multi-input (MIMT-CNN) is proposed for electromagnetic interference (EMI) signals recognition and electromagnetic environment risk evaluation of the data link of unmanned aerial vehicle (UAV). The visualized performance parameters, short-time Fourier transform (STFT) spectrograms, and constellation diagrams are obtained by experiment on the electromagnetic susceptibility of UAV’s datalink. In particular, the constellation diagram is further enhanced by calculating the density distribution of sampling points to obtain the normalized density constellation. Taking the above different categories of images as the input of the expected model, the multi-element and high correlation EMI features are extracted and fused in the MIMT-CNN. Besides, the structure of series-parallel connection is adopted in the trained model and the Bayesian optimization is also used to select hyperparameters. In this case, the perception model with higher reliability can be obtained. On this basis, the performance and complexity of the obtained model with different input channels are compared. The results show that with the input of constellation diagram, especially the normalized density constellation, can significantly improve the accuracy of the model. Besides the normalized density constellation, the model with visualized performance parameters and STFT spectrogram as inputs has a much better performance.