Tuber size, shape, colorimetric characteristics, and defect susceptibility are all factors that influence the acceptance of new potato cultivars. Despite the importance of these characteristics, our understanding of their inheritance is substantially limited by our inability to precisely measure these features quantitatively on the scale needed to evaluate breeding populations. To alleviate this bottleneck, we developed a low‐cost, semiautomated workflow to capture data and measure each of these characteristics using machine vision. This workflow was applied to assess the phenotypic variation present within 189 F1 progeny of the A08241 breeding population. Machine vision was applied to estimate linear and volumetric tuber size, assess tuber shape characteristics using aspect ratio and biomass profiles, and quantify tuber skin and flesh color; additionally, a deep learning mode was developed to classify the presence of hollow‐heart defect. Our results provide an example of quantitative measurements acquired using machine vision methods that are reliable, heritable, and capable of being used to understand and select multiple traits simultaneously in structured potato breeding populations.