Whole grain oat has become an increasingly popular baking ingredient. Still, oat baking poses many industrial challenges because the baking quality criteria have not been set for whole grain oat flours, and cultivar variation remains unknown. We aimed to assess the baking quality variation of twenty whole grain oat cultivar samples, and to identify the factors that caused the variation. It was hypothesised that by optimising the water absorption of the dough (i.e., dough yield) by test baking method, the best baking potential could be achieved for all oat cultivar samples. The baking trials were conducted as whole oat baking, without wheat or gluten additions. In most of the samples, good baking quality was obtained by dough yield optimisation. The highest specific volumes (1.9–1.93 mL/g) and best crumb properties were achieved in the samples with the highest optimal dough yields, 205. However, baking quality varied, as all samples could not be baked with good quality at high dough yields. Additionally, small median particle size and high fat content of the oat flours were related to good baking properties of whole grain oat at optimised dough yield (p < 0.05). These findings can benefit the development and the optimisation of industrial oat baking processes.