The growing interconnection between information networks and power grids enables a more efficient and economical operation, but also introduces significant challenges in modern cyber-physical power systems, such as malicious cyber-attacks. Several cyber-attacks have been reported in the recent decade, affecting hundreds of thousands of people. These attacks can lead to large power system blackouts, and optimization models are essential tools for optimal decision-making in reliable and secure operation of cyber-physical power systems. In this paper, optimization models in cyber-physical power systems are extensively reviewed and classified based on their applications, including cyber-security and optimal operation. One major application of optimization is in cyber-security evaluation of smart grids. This paper investigates the models to implement cyber-attacks against state estimation, coordinated cyber-physical attacks aiming to cover each other, and financially-motivated attacks in electricity markets; optimal defense strategies and interactions between the attacker and the defender are also introduced. Furthermore, optimization models used in the operation and dispatching of cyber-physical power and energy systems, optimal routing of information networks, and privacy-preserving models are presented. Finally, as solving optimization models is a crucial step in optimal decision-making, solvers in the literature are also introduced.