The marine diesel engine propulsion system is a nonlinear system with time delay. In order to realize the accurate and real-time control of the marine diesel engine speed, a new method based on state-compensate extended state observer, backstepping method and beetle antennae search algorithm, that is, adaptive state-compensate extended state observer-backstepping, is proposed. First of all, the response relationship model between the engine speed and the fuel injection is established on the basis of the mean value model of diesel engine. Then, to deal with the load disturbances and model parameter perturbation of diesel engine, a state-compensate extended state observer is used to estimate lumped disturbances and states of the diesel engine, and a backstepping method combined with the state-compensate extended state observer, namely state-compensate extended state observer-backstepping, is used to control the marine diesel engine speed. Then, an adaptive state-compensate extended state observer-backstepping controller is proposed by introducing the beetle antennae search algorithm for online optimization of the control parameters. Finally, simulation experiments based on the model of the 12K98ME marine diesel engine are conducted to verify the effectiveness of the proposed controller under conditions of random disturbances, sudden dumping load and parameter perturbation. The experiment results show that the proposed adaptive state-compensate extended state observer-backstepping control method has a better control effect and stronger disturbance rejection ability in comparison of the standard linear active disturbance rejection control.