Chloride ion attack is a major cause of concrete durability problems, and existing studies have rarely addressed the effects of damage zones. In this paper, an improved mesoscale model including five phases was constructed using the finite element software ABAQUS to study the diffusivity of chloride ions in cracked concrete. It was found that the damage zone is negligible when the crack width is less than 50 μm, while the width and depth of the damage zone are about 15 times the crack width and 15% of the crack depth when the crack is greater than 50 μm. The results show that the diffusion of chloride is greatly influenced by the crack width, while it is little-influenced by the crack shape. Low water–cement ratio and adequate hydration of the concrete are also key factors affecting chloride diffusion. In contrast, regular rounded aggregates have a positive effect on reducing chloride diffusion compared to irregularly shaped aggregates, and this effect becomes weaker with increasing service time. In addition, the protective layer can effectively prevent the diffusion of chloride in concrete. Therefore, when designing marine concrete, efforts should be made to ensure that the concrete has a low water–cement ratio, adequate hydration, less cracking and a protective layer.