With the rapid growth of internet-connected devices and their resource-constrained capabilities, the current authentication mechanisms are unable to meet the complex IoT application requirements, such as in the Industrial Internet of Things (IIoT), due to the increased computation, communication, and storage overhead arising from these mechanisms. In the IIoT, machine-to-machine (M2M) communication is an underlying technology where devices (e.g., sensors, actuators, and controllers) can be enabled to exchange information autonomously; thus, the massive data generated by these devices can increase latency, network congestion, and the complexity of security management. Message queue telemetry transport (MQTT) is one of the promising M2M protocols used in the IoT that could encounter such issues because it relies on a central broker in the cloud and implements a heavyweight authentication mechanism based on TLS. Therefore, this paper proposes an MQTT architecture with multi-tier brokers based on fog computing, where each broker is deployed with an authentication manager. In addition, the paper presents a lightweight mutual authentication scheme based on hash function and XOR operation. Comparing the results given in the benchmark, the overall performance of our scheme shows that storage and communication overheads are reduced to 89% and 23%, respectively. Furthermore, our system can resist against several cyberattacks and provide scalability.