The identification and validation of biomarkers of food intake (BFIs) is a promising approach to develop more objective and complementary tools to the traditional dietary assessment methods. Concerning dairy, their evaluation in terms of intake is not simple, given the variety of existing foods, making it difficult to establish the association between specific dairy products consumption and the effects on human health, which is also dependent on the study population. Here, we aimed at identifying BFI of both milk (M) and yogurt (Y) in 14 healthy young (20–35 years) and 14 older (65–80 years). After a 3-week run-in period of dairy exclusion from the diet, the subjects acutely consumed 600 ml of M or Y. Metabolomics analyses were conducted on serum samples during the following 6 h (LC-MS and GC-MS). Several metabolites showing increased iAUC after milk or yogurt intake were considered as potential BFI, including lactose (M > Y, 2-fold), galactitol (M > Y, 1.5-fold), galactonate (M > Y, 1.2-fold), sphingosine-1-phosphate (M > Y from 2.1-fold), as well as an annotated disaccharide (Y > M, 3.6-fold). Delayed serum kinetics were also observed after Y compared to M intake lysine (+22 min), phenylalanine (+45 min), tyrosine (+30min), threonine (+38 min) 3-phenyllactic acid (+30 min), lactose (+30 min), galactitol (+45min) and galactonate (+30 min). The statistical significance of certain discriminant metabolites, such as sphingosine-1-phosphate and several free fatty acids, was not maintained in the older group. This could be related to the physiological modifications induced by aging, like dysregulated lipid metabolism, including delayed appearance of dodecanoic acid (+60 min) or altered postprandial appearance of myristic acid (+70% Cmax), 3-dehydroxycarnitine (−26% Cmin), decanoylcarnitine (−51% Cmin) and dodecanoylcarnitine (−40% Cmin). In conclusion, candidate BFI of milk or yogurt could be identified based on the modified postprandial response resulting from the fermentation of milk to yogurt. Moreover, population specificities (e.g., aging) should also be considered in future studies to obtain more accurate and specific BFI.