Pulsed thermography is an established nondestructive evaluation technology that excels at detecting and characterizing subsurface defects within specimens. A critical challenge in this domain is the accurate estimation of defect depth. In this paper, a new publicly accessible pulsed infrared dataset for PVC specimens is introduced. It was enriched with 3D positional information to advance research in this area. To ensure the labeling quality, a comparative analysis of two distinct data labeling methods was conducted. The first method is based on human domain expertise, while the second method relies on 3D CAD images. The analysis showed that the CAD-based labeling method noticeably enhanced the precision of defect dimension quantification. Additionally, a sophisticated deep learning model was employed on the data, which were preprocessed by different methods to predict both the two-dimensional coordinates and the depth of the identified defects.