The permeability coefficient of construction materials plays a crucial role in engineering quality and durability. In this study, a microstructure model based on real aggregate shape and digital image technology is proposed to predict the permeability coefficient of concrete. A two-dimensional, three-component finite element model of cement concrete was established considering the interfacial transition zone (ITZ) between aggregate and mortar. The permeability coefficient prediction model was developed by the finite element method. The accuracy of the model was verified by experimental data, and the influence of the water−cement ratio on the permeability coefficient of concrete was analyzed. The results show that this method has good prediction accuracy with a relative error of 1.73%. According to the verified model, the influences of aggregate content, aggregate characteristics, aggregate location, ITZ thickness, and other factors on the permeability of concrete were explored. The higher the water−cement ratio, the higher the permeability coefficient. With the increase in aggregate content, the permeability coefficient decreases. Aggregate permeability has a significant influence on the effective permeability coefficient of concrete within a certain range. The greater the roundness of aggregate, the greater the permeability of concrete. On the contrary, the larger aggregate size causes lower permeability. The permeability coefficient of concrete with segregation is lower than that with uniform distribution. At the same time, the permeability increases with the increase of ITZ thickness.