In recent years, the network technology known as Internet of Vehicles (IoV) has been developed to improve road safety and vehicle security, with the goal of servicing the digital demands of car drivers and passengers. However, the highly dynamical network topology that characterizes these networks, and which often leads to discontinuous transmissions, is one of the most significant challenges of IoV. To address this issue, IoV infrastructure-based components known as roadside units (RSU) are designed to play a critical role by providing continuous transmission coverage and permanent connectivity. However, the main challenges that arise when deploying RSUs are balancing IoVs’ performances and total cost so that optimal vehicle service coverage is provided with respect to some target Quality of Service (QoS) such as: service coverage, throughput, low latency, or energy consumption. This paper provides an in-depth survey of RSU deployment in IoV networks, discussing recent research trends in this field, and summarizing of a number of previous papers on the subject. Furthermore, we highlight that two classes of RSU deployment can be found in the literature—static and dynamic—the latter being based on vehicle mobility. A comparison between the existing RSU deployment schemes proposed in existing literature, as well as the various networking metrics, are presented and discussed. Our comparative study confirms that the performance of the different RSU placement solutions heavily depends on several factors such as road shape, particularity of road segments (like accident-prone ones), wireless access methods, mobility model, and vehicles’ distribution over time and space. Besides that, we review the most important RSU placement approaches, highlighting their strengths and limitations. Finally, this survey concludes by presenting some future research directions in this domain.