The air void content is one of the most important volumetric properties of asphalt matrixes, such as asphalt mixtures or mortars, because it can greatly affect the performance of the matrix. At the mixture level, there are standardized methods for measuring the air void content, which is an important design parameter of a mixture. However, at the mortar scale, no unified method has been proposed to determine the air void content corresponding to the asphalt mixture. Therefore, this research aims to, first, characterize the air void distribution within the mortar of an asphalt mixture and then develop an updated theoretical method for calculating the air void content of asphalt mortar. The internal structures of air voids from three commonly used asphalt mixtures with different gradations were captured using an industrial CT scanning technique and then morphologically characterized using the image analysis method. Three-dimensional models of the air voids were reconstructed, and the air void content calculated from the CT images was also verified by density tests. The scanning results show that the air void content and air void size fit the cumulative Weibull curve. The results show that the nominal maximum particle size (NMPS) of mortar and the mixture air void content were the main parameters that affected the mortar’s air voids. The mortar air void content had linear relationships with both the mixture air void content and the mixture asphalt aggregate ratio. The mortar air void content was more sensitive to the NMPS of the asphalt mixture compared with the air void content or asphalt aggregate ratio of the mixture. This research provides a quantitative method to calculate the air void content of asphalt mortar within a mixture, which may help in the matrix design of mortar.