The optimization of cognitive radio (CR) system using an enhanced firefly algorithm (EFA) is presented in this work. The Firefly algorithm (FA) is a nature-inspired algorithm based on the unique light-flashing behavior of fireflies. It has already proved its competence in various optimization problems, but it suffers from slow convergence issues. To improve the convergence performance of FA, a new variant named EFA is proposed. The effectiveness of EFA as a good optimizer is demonstrated by optimizing benchmark functions, and simulation results show its superior performance compared to biogeography-based optimization (BBO), bat algorithm, artificial bee colony, and FA. As an application of this algorithm to real-world problems, EFA is also applied to optimize the CR system. CR is a revolutionary technique that uses a dynamic spectrum allocation strategy to solve the spectrum scarcity problem. However, it requires optimization to meet specific performance objectives. The results obtained by EFA in CR system optimization are compared with results in the literature of BBO, simulated annealing, and genetic algorithm. Statistical results further prove that the proposed algorithm is highly efficient and provides superior results.