This review aims to study the various theoretical and numerical investigations in the optimization of heat engines. The main focus is to discuss the procedures to derive the efficiency of heat engines under different operating regimes (or optimization criteria) for different models of heat engines such as endreversible models, stochastic models, low-dissipation models, quantum models etc. Both maximum power and maximum efficiency operational regimes are desirable but not economical, so to meet the thermo-ecological considerations, some other compromise-based criteria have been proposed such as Ω criterion (ecological criterion) and efficient power criterion. Thus, heat engines can be optimized to work at an efficiency which may not be the maximum (Carnot) efficiency. The optimization efficiency obtained under each criterion shows a striking universal behaviour in the near-equilibrium regime. We also discussed a multi-parameter combined objective function of heat engines. The optimization efficiency derived from the multi-parameter combined objective function includes a variety of optimization efficiencies, such as the efficiency at the maximum power, efficiency at the maximum efficiency-power state, efficiency at the maximum criterion, and Carnot efficiency. Thus, a comparison of optimization of heat engines under different criteria enables to choose the suitable one for the best performance of heat engine under different conditions.