A comprehensive approach to understand the mechanical behavior of materials involves costly and time-consuming experiments. Recent advances in machine learning and in the field of computational material science could significantly reduce the need for experiments by enabling the prediction of a material’s mechanical behavior. In this paper, a reliable data pipeline consisting of experimentally validated phase field simulations and finite element analysis was created to generate a dataset of dual-phase steel microstructures and mechanical behaviors under different heat treatment conditions. Afterwards, a deep learning-based method was presented, which was the hybridization of two well-known transfer-learning approaches, ResNet50 and VGG16. Hyper parameter optimization (HPO) and fine-tuning were also implemented to train and boost both methods for the hybrid network. By fusing the hybrid model and the feature extractor, the dual-phase steels’ yield stress, ultimate stress, and fracture strain under new treatment conditions were predicted with an error of less than 1%.