This paper presents a closed-loop detection strategy with significant potential to solve the detection defects existed in the residence-times difference (RTD) method of the coupled core fluxgate magnetic sensors. The external magnetic field (e.g. Earth's magnetic field), as the main source of these detection defects, has been analyzed by the theoretical investigation. To overcome such deficiencies, the close-loop detection method based on the analog-type RTD readout method and magentic flux-balance detection is utilized to implement external magnetic field real-time compensation and achieve a near-zero-field working state of the magnetic material. The fluxgate with the optimized detection strategy and vertical arrangement structure has been designed, fabricated and examined, and the preliminary results showed steady operation and a detection resolution of ±0.10 nT. The optimized detection method based on the balance detection concept proposed in this manuscript demonstrated a great potential to obtain the optimal detection performances which are embedded in the couplinginduced oscillation of nonlinear dynamics.