In solution crystallization processes of organic compounds, agglomeration is often an undesired phenomenon because it influences important particle properties such as size, shape, and purity. However, until now, it is still challenging to measure and quantify agglomeration in the process. Kinetics are often determined by fitting the agglomeration kernel together with other kinetic parameters to a measured particle size distribution (PSD). The approach presented decouples agglomeration from particle sizes and crystal growth. By sophisticated image analysis using a deep learning approach, process monitoring of agglomerates is enabled. Extended by modeling and experimental investigation, this allows for a systematic investigation of the agglomeration behavior of the system depending on operation conditions. Here, L-alanine/water is used as a model system. Based on the developed inline image acquisition and instance segmentation, including detection and classification of single crystals and agglomerates, the agglomeration during crystallization is tracked, and the agglomeration kernel is estimated. Further, the agglomeration degree of the final product can be predicted within the limits of the investigated parameters. It is found that the system only tends to agglomerate moderately and agglomeration is reduced when the crystallization time is short, which is defined by the saturation temperature or the cooling rate.