The qualitative measurement is a common practice in infrastructure condition inspection when using Infrared Thermography (IRT), as it can effectively locate the defected area non-destructively and non-contact. However, a quantitative evaluation becomes more significant because it can help decision makers figure out specific compensation plans to deal with defects. In this work, an IRT-based novel damage index, damage density, was proposed to quantify the significance of subsurface defects. This index is extracted from IR images using our thermography analytics framework. The proposed framework includes thermal image processing, defect edge detection, and thermal gradient map calculations. A modified root mean square error (mRMSE), which is a novel modification to the existing RMSE, was compared to evaluate the performance of image processing methods. The results show that the histogram equalization performs better than the other methods in the image processing part as the mRMSE is the lowest among them. The Pearson correlation coefficient between the developed index and the volume of subsurface defects is 0.94, which indicates a positive linear relationship between them. Thus, the proposed damage index can be used to guide the engineering practices and maintenance decisions for the subsurface determination in the civil infrastructure.