Internet of Things (IoT) has been thriving in recent years, playing an important role in a multitude of various domains, including industry 4.0, smart transportation, home automation, and healthcare. As a result, a massive number of IoT devices are deployed to collect data from our surrounding environment and transfer these data to other systems over the Internet. This may lead to cybersecurity threats, such as denial of service attacks, brute-force attacks, and unauthorized accesses. Unfortunately, many IoT devices lack solid security mechanisms and hardware security supports because of their limitations in computational capability. In addition, the heterogeneity of devices in IoT networks causes nontrivial challenges in detecting security threats. In this article, we present a collaborative intrusion detection system (IDS), namely, MidSiot, deployed at both Internet gateways and IoT local gateways. Our proposed IDS consists of three stages: (1) classifying the type of each IoT device in the IoT network; (2) differentiating between benign and malicious network traffic; and (3) identifying the type of attacks targeting IoT devices. The last two stages are handled by the Internet gateways, whereas the first stage is on the local gateway to leverage the computational resources from edge devices. The evaluation results on three popular IDS datasets (IoTID20, CIC-IDS-2017, and BOT-IoT) indicate our proposal could detect seven common cyberattacks targeting IoT devices with an average accuracy of 99.68% and outperforms state-of-the-art IDSs. This demonstrates that MidSiot could be an effective and practical IDS to protect IoT networks.