Due to the malicious attacks in wireless networks, physical layer security has attracted increasing concerns from both academia and industry. The research on physical layer security mainly focuses either on the secrecy capacity/achievable secrecy rate/capacity-equivocation region from the perspective of information theory, or on the security designs from the viewpoints of optimization and signal processing. Because of its importance in security designs, the latter research direction is surveyed in a comprehensive way in this paper. The survey begins with typical wiretap channel models to cover common scenarios and systems. The topics on physical-layer security designs are then summarized from resource allocation, beamforming/precoding, and antenna/node selection and cooperation. Based on the aforementioned schemes, the performance metrics and fundamental optimization problems are discussed, which are generally adopted in security designs. Thereafter, the state of the art of optimization approaches on each research topic of physical layer security is reviewed from four categories of optimization problems, such as secrecy rate maximization, secrecy outrage probability minimization, power consumption minimization, and secure energy efficiency maximization. Furthermore, the impacts of channel state information on optimization and design are discussed. Finally, the survey concludes with the observations on potential future directions and open challenges.