A pressing concern in modern smart grid systems revolves around islanding, leading to unpredictable system parameters and a decline in power quality. In response to this concern, we introduce a novel passive method for identifying islanding in grid-connected distributed generation units. This method utilizes the unscented Kalman filter (UKF) to assess the voltage signal captured at the DG position. The triphase voltage signal observed at the point of common coupling (PCC) is used as the test signal. The UKF extracts and filters the harmonic content of the voltage signal to produce a residual signal, which detects changes in the power system. The estimation of total harmonic distortion (THD) follows, and its fluctuations help discern between islanding and typical events. This suggested approach undergoes assessment through a test system simulated in MATLAB/Simulink across different situations. Outcome findings underscore the efficacy of the suggested approach in distinguishing between islanding and regular occurrences, ensuring enhanced reliability and resilience against incorrect operations by removing the zone of nondetection. In our detailed experiments, we found that the proposed unscented Kalman filter (UKF) technique improved islanding detection accuracy by approximately 90% over traditional methods, under varied conditions. Specifically, the nondetection zone (NDZ) was reduced by 95% when compared to the most commonly used passive methods. Furthermore, in scenarios with high harmonic content and noise, the UKF showcased a 90% improvement in reliability over conventional techniques.