With the maturity and popularization of the Internet of Things, we saw the emergence of the Internet of Vehicles. This collects and processes real-time traffic information, alleviates traffic congestion, and realizes intelligent transportation. However, sensitive information, such as real-time driving data of vehicles, are transmitted on public channels, which are easily to steal and manipulate for attackers. In addition, vehicle communications are vulnerable to malicious attacks. Therefore, it is essential to design secure and efficient protocols. Many studies have adopted asymmetric cryptosystems and fog computing to in this environment, but most of them do not reflect the advantages of fog nodes, which share the computational burden of cloud servers. Therefore, it is challenging to design a protocol that effectively uses fog nodes. In this paper, we design an authentication protocol based on a symmetric encryption algorithm and fog computing in the Internet of Vehicles. In this protocol, we first propose a four-layer architecture that significantly reduces the computational burden of cloud servers. To resist several well-known attacks, we also apply Intel software guard extensions to our protocol. This is because it can resist privileged insider attacks. We prove the security of the proposed protocol through the Real-Or-Random model and informal analysis. We also compare the performance of the proposed protocol with recent protocols. The results show better security and a lower computational cost.