The void distribution characteristics of the drainage asphalt mixture have a certain influence on its durability. In this paper, X-ray CT technology and digital image processing technology are used to study the void distribution characteristics of three different graded asphalt mixtures, and the void ratio, void quantity, void equivalent diameter, and void fractal dimension of drainage asphalt mixtures are determined. On this basis, the damage characteristics of the asphalt mixture are analyzed with the split strength as the index, and the relationship between the void fractal dimension and the split strength is established. The relationship between the void diameter and the durability of the OGFC mixture is determined by the influence of the void diameter on the void fractal dimension. The results show that the void distribution characteristics of the OGFC mixture have a significant impact on its durability. The void larger than 3 mm in OGFC asphalt mixture has a greater impact on the durability of asphalt mixture. Even if the void ratio of different OGFC mixtures is relatively close, its durability is also very different. The difference in durability is mainly due to the difference of the void structure and the proportion of the number of large voids above 5 mm.