Conservation farming practices are known for their capacity to mitigate runoff and erosion, but the magnitude of their effectiveness is highly variable across studies. In order to better understand the contribution of environmental and management factors to their effectiveness, up to 37 studies reporting 271 individual trials were collated for a quantitative review regarding 3 common conservation agriculture‐related practices, at the plot scale and in a Western European context. Two different methods suitable for hierarchically structured data sets were used for the meta‐analyses—hierarchical nonparametric bootstrapping and linear random effects models—, yielding nearly identical average outcomes but differing in terms of confidence intervals. We found that, on average, winter cover crops reduce cumulative seasonal (autumn‐winter) runoff by 68% and soil losses by 72% compared with a bare soil. The occurrence and intensity of stubble tillage on the control plot is a key explanatory variable for the mitigation effect of winter cover crops. In potato crops, tied ridging reduces cumulative seasonal (spring–summer) runoff by a mean of 70% and soil erosion by 92%. Conservation (non‐inversion) tillage techniques alleviate cumulative seasonal overland flow by 27% and associated sediments losses by 66%, but strong evidence of publication bias was detected for this farming practice, probably leading to an overestimation of its effectiveness. These mitigation effects are shown to be much greater for spring crops than for winter crops, and to increase with time since ploughing was stopped. The type of conservation tillage scheme strongly affects the ability to attenuate surface flows. Intensive non‐inversion tillage systems relying on repeated use of (powered) tillage operations appear to be the least effective for reducing both water and sediment losses. The best performing scheme against runoff would be a deep (non‐inversion) tillage (−61%), while against erosion it would be a no‐till system (−82%). Although several explanatory factors were identified, there remains a high (unexplained) variability between trials effect sizes, thus not attributable to pure sampling variability. Meanwhile, this review provides farm advisors or policy makers with guidance on the contexts in which implementation of such conservation practices should be supported so as to maximize expected benefits.