Nowadays, unmanned aerial vehicles (UAVs) have achieved massive improvement, which brings great convenience and advantage. Meanwhile, threats posed by them may damage public security and personal safety. This article proposes an architecture of intelligent anti-UAVs low-altitude defense system. To address the key problem of discovering UAVs, research based on multisensor information fusion is carried out. Firstly, to solve the problem of probing suspicious targets, a fusion method is designed, which combines radar and photoelectric information. Subsequently, single shot multibox detector model is introduced to identify UAV from photoelectric images. Moreover, improved spatially regularized discriminative correlation filters algorithm is used to elevate real-time and stability performance of system. Finally, experimental platform is constructed to demonstrate the effectiveness of the method. Results show better performance in range, accuracy, and success rate of surveillance.