Mobile cognitive radio networks provide a new platform to implement and adapt wireless cellular communications, increasing the use of the electromagnetic spectrum by using it when the primary user is not using it and providing cellular service to secondary users. In these networks, there exist vulnerabilities that can be exploited, such as the malicious primary user emulation (PUE), which tries to imitate the primary user signal to make the cognitive network release the used channel, causing a denial of service to secondary users. We propose a support vector machine (SVM) technique, which classifies if the received signal is a primary user or a malicious primary user emulation signal by using the signal-to-noise ratio (SNR) and Rényi entropy of the energy signal as an input to the SVM. This model improves the detection of the malicious attacker presence in low SNR without the need for a threshold calculation, which can lead to false detection results, especially in orthogonal frequency division multiplexing (OFDM) where the threshold is more difficult to estimate because the signal limit values are very close in low SNR. It is implemented on a software-defined radio (SDR) testbed to emulate the environment of mobile system modulations, such as Gaussian minimum shift keying (GMSK) and OFDM. The SVM made a previous learning process to allow the SVM system to recognize the signal behavior of a primary user in modulations such as GMSK and OFDM and the SNR value, and then the received test signal is analyzed in real-time to decide if a malicious PUE is present. The results show that our solution increases the detection probability compared to traditional techniques such as energy or cyclostationary detection in low SNR values, and it detects malicious PUE signal in MCRN.