We revisit the problem involving two constantly accelerating Unruh-DeWitt detectors using Open Effective Field Theory methods. We study the time evolution of the joint detector state using a Markovian approximation which differs from the standard one taken in the literature. We show that this Markovian limit already implies complete positivity of the dynamical evolution map without the rotating wave approximation (RWA), albeit in a narrower region of parameter space than normally considered. This shows that the standard microscopic derivation of the Gorini-Kossakowski-Sudarshan-Lindblad master equation (which usually invokes the RWA) is at best valid over a much smaller parameter space than what the Markovian regime allows for. We provide explicit validity bounds for when the Markovian approximation holds, which show that two well-known special cases studied in the literature, namely the "stacked trajectory" limit (when detector trajectories are taken to be on top of one another) and large gap-to-acceleration ratio violate the validity of Markovian approximation. We also show that Markovian dynamics without RWA can lead to rather different qualitative predictions for entanglement dynamics compared to RWA-based solutions, thus emphasizing further the need to track the regime of validity of all approximations.