This paper proposes a system for ultra-light drone (ULD) auto–detection using only one non-static optical PTZ camera. The system includes multi-stages of suspect objects detection, clarification, and distance estimation. An AI model for detection and clarification stages is designed based on the YOLOv3 architecture and trained with a practical dataset. In the detection stage, the camera continuously pans, tilts, and zooms to take panoramic images of the detection zone and pass them to the AI model. Once the AI model detects a suspect object, it will switch to the verification stage. In this stage, the camera controlled by the AI model’s output focuses on the target to clarify and estimate the distance to ULD. The proposed solution was implemented and tested with popular fly cams. The results show that the system can auto-detect ultra-light drones effectively with high accuracy.