The acoustic-based internal defect detection is essential to ensure the quality of arc magnets efficiently. Swarm decomposition (SWD) is conducive to processing acoustic signals, but it is still confronted with threshold optimization problems. Especially, the existing optimization methods for the SWD thresholds are merely available for a single signal with exclusive characteristics, instead of the various signals with similar characteristics. Therefore, a threshold-optimized SWD using grey wolf optimizer (GWO) is proposed to solve these issues and applied to detect the internal defects of arc magnets. In this method, a fitness function is designed to indicate the relationship between the SWD thresholds and the overall decomposition effect of similar signals. The minimum value of it corresponds to the threshold setting yielding the optimal decomposition. GWO is used for searching such a minimum value, and the obtained optimal threshold setting allows SWD to decompose any signal into a series of oscillatory components. The frequency information in the two oscillatory components with the highest energy ratio is extracted as the internal defect features. Random forest is carried out to identify these features. Experimentally, the detection accuracy reaches above 97%, and the detection speed per single arc magnet does not exceed 3.4 seconds. The proposed method cannot only determine the unified threshold setting of SWD for similar signals but also achieve an accurate, rapid detection for the internal defects.