This study aimed to investigate the association between maternal postpartum depressive symptoms, household demographic, socioeconomic, and infant characteristics with infant physical growth, and how these factors correlate to determine latent factors. This study was based on the baseline data of a 6-month randomised controlled trial aimed at providing an egg a day to infants aged 6 to 9-months from a low socioeconomic community in South Africa. Information collected on household demographic, socioeconomic, and infant characteristics was by face-to-face structured interviews, and trained assessors took anthropometric measurements. The Edinburgh Postnatal Depression Scale (EPDS) was used to assess maternal postpartum depressive symptoms. The analysis was based on 428 mother-infant pairs. Total EPDS score and its subscales score were not associated with stunting or underweight risk. However, a three- to four-fold increased risk of stunting and underweight, respectively was observed for premature birth. Low birthweight was associated with an estimated six-fold increased risk of underweight and stunting. Being female was associated with about 50% reduced risk of stunting and underweight. In conclusion, more robust studies are needed to substantiate these findings, with more awareness creation on the consequences of LBW and prematurity on the physical growth of infants from resource-limited settings.