Internet of Things (IoT) protection refers to the software field related to securing the Internet of Things and associated linked devices and systems. The IoT is a system of interconnected computers, sensors, actuators, or people on the World Wide Web (WWW). All these different devices have a unique identity in the IoT and must convey data across the network automatically. If computers are not adequately secured, allowing them to connect to the Internet exposes them to a range of serious vulnerabilities. Because the consequences of IoT failures are severe, it is necessary to observe and analyze security issues related to IoT. The prime goal of IoT security is to protect personal safety, while also guaranteeing and ensuring accessibility. In the context of IoT technology, the present study conducts a systematic literature review that analyzes the security problems associated with commercial and educational applications of home automation and details the technical possibilities of IoT with respect to the network layer. In this systematic review, we discuss how current contexts result in the inability of designers of IoT devices to enhance their cyber-security initiatives. Typically, application developers are responsible for training themselves to understand recent security advancements. As a result, active participation on the ridge scale with passive improvement can be achieved. A comparative analysis of the literature was conducted. The main objective of this research is to provide an overview of current IoT security research in home automation, particularly those using authentication methods in different devices, and related technologies in radio frequency identification (RFID) on network layers. IoT security issues are addressed, and various security problems in each layer are analyzed. We describe cross-layer heterogeneous integration as a domain of IoT and demonstrate how it can provide some promising solutions.