Diamond grinding wheels (DGWs) have a central role in cutting-edge industries such as aeronautics or defense and spatial applications. Characterizations of DGWs are essential to optimize the design and machining performance of such cutting tools. Thus, the critical issue of DGW characterization lies in the detection of diamond grits. However, the traditional diamond detection methods rely on manual operations on DGW images. These methods are time-consuming, error-prone and inaccurate. In addition, the manual detection of diamond grits remains challenging even for a subject expert. To overcome these shortcomings, we introduce a deep learning approach for automatic diamond grit segmentation. Due to our small dataset of 153 images, the proposed approach leverages transfer learning techniques with pre-trained ResNet34 as an encoder of U-Net CNN architecture. Moreover, with more than 8600 hyperparameter combinations in our model, manually finding the best configuration is impossible. That is why we use a Bayesian optimization algorithm using Hyperband early stopping mechanisms to automatically explore the search space and find the best hyperparameter values. Moreover, considering our small dataset, we obtain overall satisfactory performance with over 53% IoU and 69% F1-score. Finally, this work provides a first step toward diamond grinding wheel characterization by using a data-driven approach for automatic semantic segmentation of diamond grits.