The majority of fine chemical and pharmaceutical processes includes some form of crystallization steps. For process optimization and control of further downstream steps, the crystal size distribution of the product is a crucial factor. To identify characteristic particle size classes from a large number of measurements, each individual probe has to be separated from the mother liquor and manually analyzed. In this contribution a deep learning‐based method is presented using microscopic images as input for crystal size analysis. Additionally, a data augmentation approach was investigated to limit the data necessary for learning. A high segmentation accuracy of the crystals was achieved with 93.02 %. To evaluate the classification performed by the presented convolutional neural network (CNN), it is tested on two sets of images, containing a previously determined particle fraction. With the classifications of the CNN, a Q3 distribution is calculated. To validate the developed approach in terms of its accuracy it is compared to two other methods as well.