Cow milk is under increased scrutiny due to its environmental impact and ethical considerations concerning animal welfare. At the same time, a rising share of consumers is switching to plant-based milk substitutes (abbreviated “plant milk”). The objective of this study was (1) to analyze the product image of plant milk and cow milk and (2) to compare the motivational structure behind the consumption of both product categories. For this purpose, a quantitative survey with Austrian consumers was carried out to analyze the product image of plant milk in comparison to cow milk (n = 1001). The product image analysis revealed that the product image of cow milk is still much better than that of plant milk. Amongst others, cow milk is considered to be healthier, more natural, and better for bones. Product image valuation was dependent on the (non-)consumption of plant milk. Plant milk consumers evaluated plant milk significantly better; they considered plant milk to be much better digestible and allergy-free. The qualitative study using means-end-chain analysis, with two sub-samples of interviewees (plant milk consumers, n = 30, and cow milk consumers, n = 30), identified different motives for the consumption of cow milk and plant milk. Motives that were only reported from cow milk consumers are the origin of milk and the support of small-scale dairy production of farmers. Motives of plant milk consumers were much more diverse and included animal welfare and sustainability aspects.