Initial establishment of the human gut microbiota is generally believed to occur immediately following birth, involving key gut commensals such as bifidobacteria that are acquired from the mother. The subsequent development of this early gut microbiota is driven and modulated by specific dietary compounds present in human milk that support selective colonization. This represents a very intriguing example of host-microbe coevolution, where both partners are believed to benefit. In recent years, various publications have focused on dissecting microbial infant gut communities and their interaction with their human host, being a determining factor in host physiology and metabolic activities. Such studies have highlighted a reduction of microbial diversity and/or an aberrant microbiota composition, sometimes referred to as dysbiosis, which may manifest itself during the early stage of life, i.e., in infants, or later stages of life. There are growing experimental data that may explain how the early human gut microbiota affects risk factors related to adult health conditions. This concept has fueled the development of various nutritional strategies, many of which are based on probiotics and/or prebiotics, to shape the infant microbiota. In this review, we will present the current state of the art regarding the infant gut microbiota and the role of key commensal microorganisms like bifidobacteria in the establishment of the first microbial communities in the human gut.