Parallel pump systems are widely used in industries. However, pumps are often operating under off-design conditions due to many reasons, which reduces the efficiency and reliability of the pumps. This paper aims at reducing the power consumption of parallel pump systems and improving the reliability of pumps by improving the operation points of pumps. To achieve this goal, a combination of selecting the proper number of pumps to put into operation and their speed is needed. Also, a control valve is adopted to further improve the operation points. The optimization model is built for a parallel pump system. Particle swarm optimization algorithm is used to solve the problem. Experimental verification shows that the proposed method improves the reliability and can also reduce power consumption for some cases compared to conventional PID speed regulation method. But for cases with large flow, extra power might be needed because the control valve introduces extra flow resistance in order to improve the operation point. The proposed method can be easily extended to applications based on their requirements on reliability.