This paper proposes an inter-frame dynamic double threshold (IF-DDT) spectrum sensing algorithm in order to improve the sensing performance based on energy detection (ED) in cognitive radios (CRs). Based on both the activity model of the primary user (PU) and the sensing mechanism of the secondary user (SU), the proposed algorithm exploits the relationship between two adjacent sensing frames and designs dynamic double thresholds for each sensing frame to enhance spectrum sensing performance when the received energy cannot give a reliable local decision. The detection probability and false alarm probability of the proposed sensing scheme are analyzed, and an algorithm for searching the optimal dynamic double thresholds is derived with very low complexity according to the Neyman-Pearson (NP) test criterion. Theoretical analysis and simulation results show that the proposed algorithm outperforms the ED algorithm.