The goal of this study was to investigate the changes in oil droplet size in whey protein–stabilized emulsions during the atomization and the subsequent drying step of a spray drying process. For this purpose, experiments were performed in an atomization rig and a pilot spray dryer with two commercial pressure swirl atomizers. By comparing the oil droplet size before atomization, after atomization, and after spray drying, the changes in oil droplet size during each process step were quantified. The effect of oil droplet breakup during atomization was isolated by atomizing emulsions with 1 wt.% oil content and a protein to oil concentration ratio of 0.1. At 100 bar, the Sauter mean diameter of oil droplet size was reduced from 3.13 to 0.61 μm. Directly after breakup, coalescence of the oil droplets was observed for emulsions with a high oil content of 30 wt.%, leading to a droplet size after atomization of 1.15 μm. Increasing the protein to oil concentration ratio to 0.2 reduced coalescence during atomization and oil droplets with a mean diameter of 0.92 μm were obtained. Further coalescence was observed during the drying step: for an oil content of 30 wt.% and a protein to oil concentration ratio of 0.1 the mean droplet size increased to 1.77 μm. Powders produced at high oil contents showed a strong tendency to clump. Comparable effects were observed for a spray drying process with a different nozzle at 250 bar. The results confirm that droplet breakup and coalescence during atomization and coalescence during drying have to be taken into consideration when targeting specific oil droplet sizes in the product. This is relevant for product design in spray drying applications, in which the oil droplet size in the powder or after its redispersion determines product quality and stability.