Early life feeding practices may affect the long-term health of individuals, particularly in terms of the development of non-communicable diseases, such as metabolic and allergic diseases. Accumulating evidence suggests that the interplay of breastfeeding and/or formula feeding followed by the introduction of solids plays a role in the occurrence of non-communicable diseases both in the short and long term. International food allergy guidelines recommend that breastfeeding women do not need to avoid food allergens and do not recommend any infant formula for allergy prevention. Guidelines regarding solid food introduction for food allergy prevention recommend the introduction of well-cooked eggs and peanuts around 4–6 months of age, and not to delay the introduction of other food allergens. There is also an increasing trend to feed infants a plant-based or plant-forward diet and have access to infant formulas based on plant-based ingredients. The use of novel plant-based infant formulas raises a few questions reviewed in this paper: (1) Do fortified, plant-based infant formulas, compliant with US Food and Drug Administration (FDA) regulations and European Food Safety Authority (EFSA) (European) guidelines, support adequate infant growth? (2) Are plant-based infant formulas suitable for the management of cow’s milk allergy? (3) Does feeding with novel, plant-based infant formulas increase the risk of food allergies to the food allergens they contain? (4) Does feeding infants plant-based food allergens in early life increase the risk of allergic and severe allergic reactions? The review of the literature indicated that (1) plant-based formulas supplemented with amino acids and micronutrients to comply with FDA regulations and EFSA guidelines, evaluated in sufficiently powered growth studies, can support adequate growth in infants; (2) currently available plant-based infant formulas are suitable for the management of CMA; (3) an early introduction and continuous intake of food allergens are more likely to prevent food allergies than to increase their risk; and (4) an early introduction of food allergens in young infants is safe.