HIV infection and undernutrition remain significant public health concerns for infants and children. In infants and children under these conditions, undernutrition is one of the leading causes of death. Proper management of nutrition and related nutrition complications in these groups with increased nutrition needs are prominent challenges, particularly in HIV-prevalent poor-resource environments. Several studies support the complexity of the relation between HIV infection, nutrition, and the immune system. These elements interact and create a vicious circle of poor health outcomes. Recent studies on the use of probiotics as a novel approach to manage microbiome imbalance and gut-mucosal impairment in HIV infection are gaining attention. This new strategy could help to manage dysbiosis and gut-mucosal impairment by reducing immune activation, thereby potentially forestalling unwanted health outcomes in children with HIV. However, existing trials on HIV-infected children are still insufficient. There are also conflicting reports on the dosage and effectiveness of single or multiple micronutrient supplementation in the survival of HIV-infected children with severe acute malnutrition. The WHO has published guidelines that include time of initiation of antiretroviral therapy for HIV-pregnant mothers and their HIV-exposed or HIV-infected children, micronutrient supplementation, dietary formulations, prevention, and management of HIV therapy. However, such guidelines need to be reviewed owing to recent advances in the field of nutrition. There is a need for new intervention studies, practical strategies, and evidence-based guidelines to reduce the disease burden, improve adherence to treatment regimen, and enhance the nutrition, health, and well-being of HIV-infected infants and children. This review provides up-to-date scientific information on current knowledge and existing challenges for nutrition therapy in HIV-infected infants and children. Moreover, it presents new research findings that could be incorporated into current guidelines.