Improving the material removal rate (MRR) has recently become one of the most important issues in electrical discharge machining. During the discharge process, a large portion of molten material cannot be sufficiently expelled from the molten pool but re-solidifies, ultimately resulting in low energy utilization and machining efficiency. Unlike existing methods that primarily focus on optimizing general discharge parameters, this study aims to enhance molten material expulsion and MRR through discharge plasma regulation by employing a redesigned late-stage oscillating discharge current. During a single-pulse discharge process, this kind of discharge current firstly remains constant to ensure stable heat transfer from the plasma to the workpiece, then transitions to periodic oscillations to enhance plasma movement and facilitate molten material expulsion. High-speed plasma observations and heat-flow coupling simulations are conducted to analyze the effects of the discharge currents on material removal, and the optimal oscillation start time is obtained. Experimental results in machining stainless steel demonstrate that the use of the late-stage oscillating discharge current, in comparison to the conventional rectangular discharge current, results in a 74% increase in material removal volume per unit of energy and a 56% in average recast layer thickness.