Due to the dramatic increase in high data rate services and in order to meet the demands of the fifth-generation (5G) networks, researchers from both academia and industry are exploring advanced transmission techniques, new network architectures and new frequency spectrum such as the visible light and the millimeter wave (mmWave) spectra. Visible light communication (VLC) particularly is an emerging technology that has been introduced as a promising solution for 5G and beyond, owing to the large unexploited spectrum, which translates to significantly high data rates. Although VLC systems are more immune against interference and less susceptible to security vulnerabilities since light does not penetrate through walls, security issues arise naturally in VLC channels due to their open and broadcasting nature, compared to fiber-optic systems. In addition, since VLC is considered to be an enabling technology for 5G, and security is one of the 5G fundamental requirements, security issues should be carefully addressed and resolved in the VLC context. On the other hand, due to the success of physical layer security (PLS) in improving the security of radio-frequency (RF) wireless networks, extending such PLS techniques to VLC systems has been of great interest. Only two survey papers on security in VLC have been published in the literature. However, a comparative and unified survey on PLS for VLC from information theoretic and signal processing point of views is still missing. This paper covers almost all aspects of PLS for VLC, including different channel models, input distributions, network configurations, precoding/signaling strategies, and secrecy capacity and information rates. Furthermore, we propose a number of timely and open research directions for PLS-VLC systems, including the application of measurement-based indoor and outdoor channel models, incorporating user mobility and device orientation into the channel model, and combining VLC and RF systems to realize the potential of such technologies.