Both number manually-counting method and traditional Machine-Vision (MV) number counting strategy are laborious and very time-consuming (sometimes several hours). Thus a new deep learning (DL) fusion model is proposed, which includes object detection and semantic segmentation. It can solve the problems of end-face localization and segmentation of steel bars at the same time. In this fusion model, firstly, an improved data augmentation method namely, Sliding Window Data Augmentation (SWDA) is adopted to compensate less training data concerning object detection, based on which a new object-detection architecture, Inception-RFB-FPN is presented to improve the accuracy and inference time. Secondly, a novel AI labeling method, Fibonacci-incremental mask labeling method (FIMLM) is introduced to accelerate the generation of annotation mask. Furthermore, by contrast, three FCN (Fully Convolutional Networks) architectures of data segmentation, namely, VGG16-FCN, ResNet18-FCN, and ResNet34-FCN are used to conduct the end-face segmentations of steel bars separately. Finally, a series of experiments show that the proposed Inception-RFB-FPN model can reach 98.17% in F1 score (harmonic mean value of precision and recall) with respect to object detection, and its inference time only needs 0.0306 seconds, much faster than some related reports. In addition, the FIMLM-based ResNet34-FCN model can reach 97.47% in mean Intersection-Over-Union (mIOU) with respect to semantic segmentation, higher than both VGG16-FCN and ResNet18-FCN.