Cavities are typical features in aeronautical structural parts and molds. For high-speed milling of multi-cavity parts, a reasonable processing sequence planning can significantly affect the machining accuracy and efficiency. This paper proposes an improved continuous peripheral milling method for multi-cavity based on ant colony optimization algorithm (ACO). Firstly, by analyzing the mathematical model of cavity corner milling process, the geometric center of the corner is selected as the initial tool feed position. Subsequently, the tool path is globally optimized through ant colony dissemination and pheromone perception for path solution of multi-cavity milling. With the advantages of ant colony parallel search and pheromone positive feedback, the searching efficiency of the global shortest processing path is effectively improved. Finally, the milling programming of an aeronautical structural part is taken as a sample to verify the effectiveness of the proposed methodology. Compared with zigzag milling and genetic algorithm (GA)-based peripheral milling modes in the computer aided manufacturing (CAM) software, the results show that the ACO-based methodology can shorten the milling time of a sample part by more than 13%.