The rapid development and availability of drones has raised growing interest in their numerous applications, especially for aerial remote-sensing tasks using the Internet of Drones (IoD) for smart city applications. Drones image a large-scale, high-resolution, and no visible band short wavelength infrared (SWIR) ground aerial map of the investigated area for remote sensing. However, due to the high-altitude environment, a drone can easily jitter due to dynamic weather conditions, resulting in blurred SWIR images. Furthermore, it can easily be influenced by clouds and shadow images, thereby resulting in the failed construction of a remote-sensing map. Most UAV remote-sensing studies use RGB cameras. In this study, we developed a platform for intelligent aerial remote sensing using SWIR cameras in an IoD environment. First, we developed a prototype for an aerial SWIR image remote-sensing system. Then, to address the low-quality aerial image issue and reroute the trajectory, we proposed an effective lightweight multitask deep learning-based flying model (LMFM). The experimental results demonstrate that our proposed intelligent drone-based remote-sensing system efficiently stabilizes the drone using our designed LMFM approach in the onboard computer and successfully builds a high-quality aerial remote-sensing map. Furthermore, the proposed LMFM has computationally efficient characteristics that offer near state-of-the-art accuracy at up to 6.97 FPS, making it suitable for low-cost low-power devices.