SiCp/Al composites are a kind of particle-reinforced composite material, which has been widely used in various fields due to its excellent performance. However, during the machining, the damage and failure of the SiC particles, the aluminum matrix, and the interface phase will cause many surface and edge defects. These defects will seriously affect the application of SiCp/Al composites. In this paper, a finite element model of randomly distributed multi-cell SiCp/Al composites with longitudinal-torsional ultrasonic vibration-assisted milling was established to analyze the formation mechanism of edge defects of the material. The simulation results showed that the main reason for the formation of edge defects was that the interface, SiC particles, and the Al matrix will produce cracks during machining, and these cracks will propagate and cause particle breakage, matrix tearing, and edge gaps. According to different machined parameters, the ultrasonic vibration-assisted milling experiment was carried out. The test results showed that the actual processed workpiece does have defects such as edge gaps. The depth of cut and the feed per tooth had a serious influence on the edge defect value, while the cutting speed had a small effect. Moreover, under the condition of applying appropriate ultrasonic amplitude, the phenomenon of edge defects and the surface quality were significantly improved. Therefore, the application of ultrasonic vibration-assisted milling can improve the surface and edge quality of SiCp/Al composites.