We study inclusive and diffractive dijet production in electron-proton and electron-nucleus collisions within the Color Glass Condensate effective field theory. We compute dijet cross sections differentially in both mean dijet transverse momentum P and recoil momentum ∆, as well as the anisotropy in the relative angle between P and ∆. We use the nonlinear Gaussian approximation to compute multiparticle correlators for general small x kinematics, employing running coupling Balitsky-Kovchegov evolution to determine the dipole amplitude at small x. Our results cover a much larger kinematic range than accessible in previous computations performed in the correlation limit approximation, where it is assumed that |P | |∆|. We validate this approximation in its range of applicability and quantify its failure for |P | |∆|. We also predict significant targetdependent deviations from the correlation limit approximation for |P | > |∆| and |P | Qs, which offers a straightforward test of gluon saturation and access to multi-gluon distributions at a future electron ion collider. arXiv:1912.05586v1 [nucl-th]