Operation optimization of natural gas pipelines has received increasing attentions, due to such advantages as maximizing the operating economic benefit and the gas delivery amount. This paper provides a review on the most relevant research progress related to the steady-state operation optimization models of natural gas pipelines as well as corresponding solution methods based on stochastic optimization algorithms. The existing operation optimization model of the natural gas pipeline is a mixed-integer nonlinear programming (MINLP) model involving a nonconvex feasible region and mixing of continuous, discrete, and integer optimization variables, which represents an extremely difficult problem to be solved by use of optimization algorithms. A survey on the state of the art demonstrates that many stochastic algorithms show better performance of solving such optimization models due to their advantages of handling discrete variables and of high computation efficiency over classical deterministic optimization algorithms. The essential progress mainly with regard to the applications of the Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), Simulated Annealing (SA) algorithms, and their extensions is summarized. The performances of these algorithms are compared in terms of the quality of optimization results and the computation efficiency. Furthermore, the research challenges of improving the optimization model, enhancing the stochastic algorithms, developing an online optimization technology, researching the transient optimization, and studying operation optimization of the integrated energy network are discussed.