The stepless flow control system offers important energy-saving technology for reciprocating compressors in the petrochemical and oil refining industries. Optimizing the movement characteristics of the unloader and suction valve is vital to improving the energy-saving level of the system and reducing the overall operating cost. However, the system has many of the characteristics of multi-system coupling and multi-parameter crossover; thus, it is difficult to optimize the key control parameters. In this study, to optimize the system inlet pressure, return pressure, and return spring stiffness parameters, a working model of the flow control system based on multi-system coupling was established. Using the ejection and withdrawal speeds of the unloader, the flow displacement deviation, and the gas work deviation of the control system as the optimization parameters, we used the response surface method to establish an optimization proxy model between the objective function and key parameters. Additionally, verification of the model’s accuracy and sensitivity analyses were completed. Finally, a double optimization scheme based on a non-dominated genetic algorithm (NSGA-II) was proposed. Simulation and experimental results show that with optimization of the return spring, oil inlet pressure, and oil return pressure, the unloader’s kinematic characteristics were also optimized at full load. The impact energy of the ejector and withdrawal speed of the unloader were reduced, and the compressor flow control error was less than 5%, which effectively improved the comprehensive working performance of the stepless flow control system.