The durability of concrete irrigation infrastructures decreases easily due to environmental effects, such as the freeze-thawed process. The degree of damage in concrete is, in most cases, evaluated by an unconfined compression test or ultrasonic test. The limiting states of structures are the result of a gradual accumulation of microdefects during the in-service period, which leads to the initiation and development of macroscopic cracks, loss of performance, and Failure. Therefore, to maintain such structures, it is necessary to properly determine the degree of damage by non-destructive testing. For effective maintenance and management of in-service structures, it is necessary to evaluate not only such mechanical properties as strength but also the degree of damage. Quantitative damage evaluation of concrete is proposed by applying the X-ray CT method and related non-destructive inspection, which is based on estimating cracking damage effects. In this study, the damage evaluation of concrete-core samples is investigated by X-ray CT parameters and the non-destructive parameters. The samples tested were taken from the damaged concrete headwork in Niigata, Japan, about 50 years after its construction. The geometric properties of the concrete matrix were calculated by the analysis of X-ray CT images. After X-ray CT measurement, an ultrasonic test and a resonant frequency test were performed. The dynamic modulus of elasticity was calculated by using ultrasonic pulse velocity and resonant frequency. As a result, based on the relationship between the geometric properties of the concrete matrix and the non-destructive parameters, the accumulation of damage in the samples tested was correlated with the ratio of the total area of cracks to the observation area. In damaged conditions, the high value of its ratio is demonstrated under low ultrasonic pulse velocity. These results suggest that the most sensitive components in a concrete body to the degree of damage are cracks in mortar and the ratio of the total area of cracks to the observation area. These values are affected by the internal actual cracks. Thus, the damage of concrete could be quantitatively evaluated by the geometric properties of the concrete matrix and its physical properties.