Human milk (HM) is the golden standard for nutrition of newborn infants. Human milk oligosaccharides (HMOs) are abundantly present in HM and exert multiple beneficial functions, such as support of colonization of the gut microbiota, reduction of pathogenic infections and support of immune development. HMO-composition is during lactation continuously adapted by the mother to accommodate the needs of the neonate. Unfortunately, for many valid reasons not all neonates can be fed with HM and are either totally or partly fed with cow-milk derived infant formulas, which do not contain HMOs. These cow-milk formulas are supplemented with non-digestible carbohydrates (NDCs) that have functional effects similar to that of some HMOs, since production of synthetic HMOs is challenging and still very expensive. However, NDCs cannot substitute all HMO functions. More efficacious NDCs may be developed and customized for specific groups of neonates such as pre-matures and allergy prone infants. Here current knowledge of HMO functions in the neonate in view of possible replacement of HMOs by NDCs in infant formulas is reviewed. Furthermore, methods to expedite identification of suitable NDCs and structure/function relationships are reviewed as in vivo studies in babies are impossible.