Starch agglomerates are widely applied in the pharmaceutical, agricultural, and food industries. The formation of potato starch tablets and their diametral compression were simulated numerically and verified in a laboratory experiment to analyse the microscopic mechanisms of the compaction and the origins of their breakage strength. Discrete element method (DEM) simulations were performed using EDEM software. Samples comprised of 120,000 spherical particles with radii normally distributed in the range of 5–36 μm were compacted in a cylindrical die with a diameter of 2.5 cm. The linear elastic–plastic constitutive contact model with a parallel bonded-particle model (BPM) was used to model the diametral compression. DEM simulations indicated that the BPM, together with the linear elastic–plastic contact model, could describe the brittle, semi-brittle, or ductile breakage mode, depending on the ratio of the strength to Young’s modulus of the bond and the bond-to-contact elasticity ratio. Experiments confirmed the findings of the DEM simulations and indicated that potato starch (PS) agglomerates can behave as a brittle, semi-brittle, or ductile material, depending on the applied binder. The PS agglomerates without any additives behaved as a semi-brittle material. The addition of 5% of ground sugar resulted in the brittle breakage mode. The addition of 5% gluten resulted in the ductile breakage mode.