Despite the widespread adoption of the Routing Protocol for Low-power and Lossy Networks (RPL) in IoT environments, its inherent limitations in addressing security vulnerabilities have left IoT networks vulnerable to ongoing attacks. This paper introduces a novel intrusion detection system tailored specifically for IoT networks, with a focus on mitigating attacks at the network’s edge. The study presents the Hybrid Rank Attack (HRA), a sophisticated threat exploiting RPL vulnerabilities by alternately advertising decreased and increased rank values in control messages. Extensive experimentation evaluates the detrimental effects of HRA on critical network metrics including exchanged messages, energy consumption, PDR, latency, and memory footprint. Additionally, a lightweight and distributed countermeasure algorithm is proposed to effectively mitigate the impact of HRA. Simulation-based evaluations demonstrate significant reductions in control overhead (68.7%) and energy consumption (61.83%), with minimal additional RAM utilization (1.05%). This lightweight solution enhances the resilience of RPL-based IoT networks against HRA threats.