Automatic modulation classification (AMC) is an important stage in intelligent wireless communication receivers. It is a necessary process after signal detection, and before demodulation. It plays a vital role in various applications. Blind modulation classification is a very difficult task without information about the transmitted signal and the receiver parameters like carrier frequency, signal power, timing information, phase offset, existence of frequency-selective multipath fading, and time-varying channels in real-world applications. The AMC methods are divided into traditional and advanced methods. Traditional methods include likelihood-based (LB) and feature-based (FB) methods. The advanced methods include deep learning (DL) methods. In addition, the AMC methods are used to classify different modulation schemes such as ASK, PSK, FSK, PAM, and QAM with different orders and different signal-to-noise ratios (SNRs). This paper focuses on summarizing the AMC methoods, comparing between them, presenting the commercial software packages for AMC, and finally considering the new challenges in the implementation of AMC. K E Y W O R D S automatic modulation classification (AMC), deep learning (DL), feature-based (FB) methods, likelihood-based (LB) methods | INTRODUCTIONAutomatic modulation classification (AMC) is important in wireless communication systems used in military and civilian applications to enhance the efficiency of the spectrum utilization, redue the overhead, and resolve the shortage problems. Unfortunately, the restricted spectrum resources barely satisfy the ever-increasing demand for 5G 1,2 and Internet of Things (IoT) networks. 3 The AMC can be used for better management of the available spectrum. A simple block diagram of a communication system based on AMC is presented in Figure 1. 4 The AMC architecture contains two steps: signal preprocessing and a proper algorithm for classification. The preprocessing tasks involve reduction of noise, carrier frequency estimation, symbol period estimation, equalization, and signal power evaluation. On the other hand, the AMC methods comprise traditional methods including decisiontheoretic methods and feature-based methods 4,5 along with advanced methods 6 as shown in Figure 2.

Recently, automatic modulation classification (AMC) has extensively and commonly been utilized in several modern wireless communication systems as a significant tool of signal detection for civilian and military applications and cognitive radio systems. Although several methods have been established to identify the modulation scheme of a received signal, they show a difficulty of learning radio characteristics for most conventional machine learning algorithms. This article focuses on the deep learning (DL) classification technique to solve these problems. To improve the classification accuracy of a communication signal modulation type, we apply a new model that combines Gabor filtering and thresholding with the help of convolution filters implemented in DL. A basic convolutional neural network, AlexNet, and a residual neural network are used for being compatible with constellation diagrams in order to achieve a superior classification performance. Moreover, the Gabor filter can effectively extract spatial information, including edges and textures. In terms of classification accuracy, the proposed AMC system improves the signal modulation classification accuracy significantly, and achieves competitive results. We use seven modulation types over the range of signal-noise ratio (SNR) values from −10 to 30 dB. The performed experiments reveal that the proposal guarantees a remarkable classification accuracy of approximately 100% at a 10 dB SNR over AWGN and Rayleigh fading channels. Therefore, to prove the functional viability of our proposed method, it can be applied in adaptive modulators that can be used in many devices in applications such as Internet-of-Things (IoT). INTRODUCTIONDue to the increasingly growing demand for wireless radio spectrum bandwidth, better approaches for utilization of the radio spectrum are essential. 1 The automatic modulation classification (AMC) is one of the most common techniques for identifying frequency spectra, spectrum management, electronic warfare, cognitive radio networks, interference