This paper proposes an attention-based convolutional neural network (ABCNN) for intrusion detection in the Internet of Things (IoT). The proposed ABCNN employs an attention mechanism that aids in the learning process for low-instance classes. On the other hand, the Convolutional Neural Network (CNN) employed in the ABCNN framework converges toward the most important parameters and effectively detects malicious activities. Furthermore, the mutual information technique is employed during the pre-processing stage to filter out the most significant features from the datasets, thereby improving the effectiveness of the ABCN model. To assess the effectiveness of the ABCNN approach, we utilized the Edge-IoTset, IoTID20, ToN_IoT, and CIC-IDS2017 datasets. The performance of the proposed architecture was assessed using various evaluation metrics, such as precision, recall, F1-score, and accuracy. Additionally, the performance of the proposed model was compared to multiple ML and DL methods to evaluate its effectiveness. The proposed model exhibited impressive performance on all the utilized datasets, achieving an average accuracy of 99.81%. Furthermore, it demonstrated excellent scores for other evaluation metrics, including 98.02% precision, 98.18% recall, and 98.08% F1-score, which outperformed other ML and DL models.