Internet of Things (IoT) provides a wide range of services in domestic and industrial environments. Access control plays a crucial role in granting access rights to users and devices when an IoT device is connected to a network. However, many challenges exist in designing and implementing an ideal access control solution for the IoT due to the characteristics of the IoT including but not limited to the variety of the IoT devices, the resource constraints on the IoT devices, and the heterogeneous nature of the IoT. This paper conducts a comprehensive survey on access control in the IoT, including access control requirements, authorization architecture, access control models, access control policies, access control research challenges, and future directions. It identifies and summarizes key access control requirements in the IoT. The paper further evaluates the existing access control models to fulfill the access control requirements. Access control decisions are governed by access control policies. The existing approaches on dynamic policies’ specification are reviewed. The challenges faced by the existing solutions for policies’ specification are highlighted. Finally, the paper presents the research challenges and future directions of access control in the IoT. Due to the variety of IoT applications, there is no one-size-fits-all solution for access control in the IoT. Despite the challenges encountered in designing and implementing the access control in the IoT, it is desired to have an access control solution to meet all the identified requirements to secure the IoT.