Magnetic abrasive finishing (MAF) shows a high potential for use on computerized numerical control (CNC) machine tools as a standard tool to polish workpieces directly after the milling process. This paper presents a new MAF tool with a single, large permanent magnet and a novel top cover structure for finishing the plain ferromagnetic workpieces. The top cover structure of the MAF tool, combined with an optimized working gap, ensures the effect of mechanical powder compaction, which leads to a significant increase in process capability and surface roughness reduction. The influence of the process parameters such as feed rate, equivalent cutting speed, working gap (including for three grain sizes) and the gap to the magnet was investigated. In addition, the influence of the initial surface after face milling, end milling, ball end milling and grinding on the surface quality after MAF was investigated. Furthermore, three typical surfaces after milling and MAF were analyzed. By magnetic abrasive finishing, a significant surface quality improvement of the initial milled surfaces to roughness values up to Ra = 0.02 µm and Rz = 0.12 µm in one processing step could be achieved.