A large portion of the world’s population has lactose intolerance. Fundamentally, this condition occurs when the small intestine does not produce enough of the lactase enzyme, which digests the disaccharide lactose in milk. Lactose avoiders might unconsciously decide to limit or exclude milk and dairy products from their diets. This group includes people with lactose intolerance, people with an allergy to milk protein, vegans, and those expressing personal preferences. Lactose avoiders are often self-reported as being milk intolerant. In this review, specific amounts of lactose in different types of milk and milk products are presented. The amounts of micro- and macronutrients in them are compared with the daily requirements established by accepted sources. Foods are suggested that can play vital roles in permanently avoiding lactose-containing dairy products, for example, brussels sprouts, as a good source of vitamin B1; kale, as a source of vitamin K; and cereals at breakfast for vitamin B6. Attention is paid to mature cheeses as they are extremely beneficial for health due to their rich vitamin and elemental compositions, and they are also suitable for people with lactose intolerance due to their low lactose content. This information is rarely provided on packaging. In addition, the current state of labeling for the presence of lactose in food and pharmaceutical products is discussed. The term “hidden lactose” is introduced to include added lactose in unexpected foods, drinks, and even medicines.