Classical active flow control (AFC) methods based on solving the Navier-Stocks equations are laborious and computationally intensive even with the use of reduced-order models. Data-driven methods offer a promising alternative for AFC and have been applied successfully to reduce the drag of two-dimensional bluff bodies using deep reinforcement learning (DRL) paradigms. However, the standard DRL method tends to result in large fluctuations in the unsteady forces acting on the cylinder as the Reynolds number increases. In this study, a Markov decision process (MDP) with time delays is introduced to quantify the action delays in the DRL process along with the use of a first-order autoregressive policy (ARP). This hybrid DRL method is applied to control the vortex shedding process from a two-dimensional circular cylinder with four synthetic jet actuators at a freestream Reynolds number of 400. Compared to the standard DRL method, this method is shown to reduce the magnitude of drag and lift fluctuations by approximately 90% at the same actuation frequency while achieving a similar level of drag reduction. Although both methods suppress the vortex-induced forces by extending the recirculation zone behind the circular cylinder, the hybrid method leads to a steadier and more elongated recirculation zone, and hence a weaker vortex shedding process. This study demonstrates the necessity of accurately quantifying the delay in the MDP and the benefits of introducing ARPs to achieve a smooth control of unsteady forces in AFC.