The tracking performance of an actuation transfer system in a real-time hybrid simulation (RTHS) frequently faces accuracy and robustness challenges under constraints and complicated environments with uncertainties. This study proposes a novel control approach based on the deep deterministic policy gradient algorithm in reinforcement learning (RL) combined with feedforward (FF) compensation, which emphasizes the implementation of shaking table control and substructure RTHS. The proposed method first describes the control plant within the RL environment. Then, the agent is trained offline to develop optimized control policies for interaction with the environment. A series of validation tests were conducted to assess the performance of the proposed method, starting with the dynamic testing of underwater shaking table control and then a virtual RTHS benchmark problem. For complex systems, such as controlling the underwater shaking table, the proposed algorithm, FF, and adaptive time series (ATS) compensation methods are compared under various water depths and motions. The results show better performance and wider broadband frequency applicability under different shaking table dynamic-coupling effects. Next, a controller based on the proposed method was designed by extending the virtual RTHS via the configuration of the control plant and substructure division, as provided in the RTHS benchmark problem. The proposed RL controller also improved the tracking accuracy and robustness of conventional FF compensators against unmodeled dynamics and perturbation uncertainties. This controller can be extended to further advanced control strategies as a component of model-based control methods. K E Y W O R D Sdeep deterministic policy gradient algorithm, hybrid control, real-time hybrid simulation, reinforcement learning, underwater shaking table Ning Li and Jichuan Tang have equal contributions and share first authorship.