Vehicular Ad Hoc Network (VANET) is a wireless Mobile Ad Hoc Network that is used for communication between vehicles, vehicles and fixed access points, and vehicles and pedestrians. However, because of the use of open wireless channels, VANET is more vulnerable. Therefore, VANET security is critical for safe driving and user privacy protection. Authentication and key agreement are crucial for ensuring security. Numerous authentication schemes have been proposed between vehicles and roadside units (RSUs). Many solutions are authentication and key negotiation between the vehicle and a single RSU. The vehicle passing through a region needs to complete authentication and key agreement with multiple RSUs separately, which brings a great burden to the vehicle. In order to simplify the authentication process of vehicles and multiple RSUs and improve the efficiency of authentication and key agreement, an efficient authentication and key agreement scheme based on RSU cache is proposed when the vehicle moves from one RSU to another RSU region. In the proposed scheme, RSUs are divided into regions, and each region has a RSU cluster head. When the vehicle enters a certain region and authenticates with a RSU successfully, the RSU submits the authentication information to the RSU cluster head. The RSU cluster head shares the authentication information with other RSUs in the region using the shared key. Other RSUs record the authentication information in the cache. When the vehicle communicates with other RSUs, the authentication is not necessary; the session key can be negotiated by simply exchanging information. After using the cache, the calculation and communication cost of the authentication and key negotiation between the vehicle and other RSU can be significantly saved, the calculation cost is reduced by 37%, and the traffic is reduced by 35%. The random oracle model is used to prove the security of the scheme. The results revealed that the authentication overhead of the proposed scheme is considerably lower than those of other schemes. Compared with the related schemes, the computational cost of the proposed scheme is reduced by 34% on average; the communication cost is close to other related schemes. Moreover, the security analysis shows that the proposed scheme provides better security compared to other related schemes.