Studies from India and several eastern African countries found that the impact of dairy animal ownership on household nutrition varied greatly, depending on the socio-geographic context. The purpose of this study was to examine the association between livestock ownership and household dietary quality in rural Kolar district, India. We collected data from a household survey in four study villages (n = all 195 households of the four villages) of Kolar district, applying a cross-sectional design. Kendall’s rank correlation coefficient was employed to determine the correlation between milk consumption and other dietary variables. Multivariable logistic regression was used to describe the relationship between dairy animal ownership and household milk consumption. Households owning dairy animals more often had access to irrigation (58.3% vs. 25.2%) and were less often woman-headed (2.4% vs. 22.5%). Household milk consumption was significantly correlated with consumption of vegetable variety, egg, and meat (all p-values < 0.05). After adjusting for multiple confounders, the odds ratio of milk consumption between dairy animal-owning households as compared to other households was 2.11 (95% confidence interval 0.85, 5.45). While dairy animal ownership was found to be associated with improved dietary quality, larger households were in a better position to adopt dairy animals, which, in turn, might contribute to better household nutrition.