Distributed energy sources are usually interfaced to the grid using power electronic converters, and lack of inertia in inverter dominated microgrids can affect the system stability. This paper presents a new method for virtual synchronous generator (VSG) control in order to solve the low system inertia and support the grid frequency problem. In this paper, the VSG based on electromagnetic transient characteristics is improved and an adaptive command filter back-stepping controller is designed. Firstly, the rotor swing equation and power part are modeled to complete the controller design for achieving system stability in the islanded, grid-connected and transition modes. In addition, a limited-amplitude command filter is used to deal with computational complexity and nonlinear saturation problems in the design process. Secondly, projection operator, and adaptive inertia and damping control are introduced to reduce the modeling error and disturbance caused by changing parameters. This ensures the boundedness of the estimated value and further improves the frequency response, especially in the transition mode. Finally, simulation results show that the proposed controller is more effective than the traditional control method for achieving power stability and frequency improvement.