Salt erosion has an adverse impact on the durability of asphalt pavements. Porous asphalt concrete is particularly susceptible to the influence of salt. In this study, a finite element model was developed to investigate the fracture behavior of PAC exposed to salt erosion. The 2D heterogeneous structure of PAC was generated with an image-aided approach to computationally study the fracture behavior of PAC. Laboratory SCB tests were conducted to validate the finite element model. The simulation results of the SCB tests indicate that the peak load of the PAC decreased by 21.8% in dry-wet cycles and 26.1% in freeze-thaw cycles compared to the control group. The salt solution accelerated the degradation of the durability of PAC under both dry-wet cycles and freeze-thaw cycle conditions, which is consistent with laboratory tests. After flushing treatment before the drying phase, the peak load of the PAC in salt environments increased by 5.3% compared to that of the samples with no flushing. Salt erosion also results in a higher average value of scalar stiffness degradation (SDEG), and the damaged elements were primarily the cohesive elements in the fracture of the PAC. Additionally, the influence of crucial factors including the void content, adhesion and cohesion, and loading rate on the fracture behavior of the PAC was analyzed. As the void content increases, the average SDEG value of the cohesive elements increases and surpasses the average SDEG value of the adhesive elements at a void content of approximately 9%. The performance of the fine aggregate matrix (FAM) has a much greater impact than the FAM-aggregate interface on the durability of the PAC. And there were more damaged CZM elements with the increase in the loading rate. Salt erosion results in higher SDEG values and a larger number of cohesive damaged elements at each loading rate.