Detection and characterization of hidden defects, impurities, and damages in homogeneous materials like aluminum die casting materials, as well as composite materials like Fiber–Metal Laminates (FML), is still a challenge. This work discusses methods and challenges in data-driven modeling of automated damage and defect detectors using measured X-ray single- and multi-projection images. Three main issues are identified: Data and feature variance, data feature labeling (for supervised machine learning), and the missing ground truth. It will be shown that simulation of synthetic measuring data can deliver a ground truth dataset and accurate labeling for data-driven modeling, but it cannot be used directly to predict defects in manufacturing processes. Noise has a significant impact on the feature detection and will be discussed. Data-driven feature detectors are implemented with semantic pixel Convolutional Neural Networks. Experimental data are measured with different devices: A low-quality and low-cost (Low-Q) X-ray radiography, a typical industrial mid-quality X-ray radiography and Computed Tomography (CT) system, and a state-of-the-art high-quality μ-CT device. The goals of this work are the training of robust and generalized data-driven ML feature detectors with synthetic data only and the transition from CT to single-projection radiography imaging and analysis. Although, as the title implies, the primary task is pore characterization in aluminum high-pressure die-cast materials, but the methods and results are not limited to this use case.