Cognitive radio (CR) networks allow secondary users (SUs) to access and use the spectrum opportunistically. However, some misbehavior users mislead SUs to evacuate the spectrum by launching a primary user emulation attack (PUEA), which is a denial of service (DoS) attack. During the service of an SU, the arrival of the malicious misbehavior users (MMUs) will force the SU under service to leave the system. Another type of attack, different from the MMU, refers to the selfish misbehavior users (SMUs) who launch PUEA to interrupt the serving SU and then take over the spectrum for their own use. In this paper, we consider two types of misbehavior users, i.e., MMU and SMU. We first investigate the noncooperative behavior of SUs that maximize their own benefit and obtain a unique individual Nash equilibrium joining strategy. Next, we present a socially optimal joining strategy from the perspective of social planners. To eliminate the gap between individual equilibrium and socially optimal strategies, a fee imposed on SUs is proposed. We carry out a sensitivity analysis of joining strategies of SUs and social welfare regarding some main parameters of the system. Interestingly, we observe that the individual equilibrium and socially optimal joining probabilities along with the social welfare increase when the MMUs invade the system more frequently.INDEX TERMS Cognitive radio system, noncooperative game, cooperative game, queueing system, primary user emulation attack.

No abstract