Polybrominated diphenyl ethers (PBDEs) are ubiquitous flame retardants and are environmentally persistent. PBDEs show endocrine disruption, neurotoxicity, and lower birth weight in infants, and their human body burden has become a public health concern. The infants’ exposure begins in the prenatal period and continues via breast milk ingestion, although, little is known about the factors that may influence this exposure. In this study, PBDE levels in Brazilian breast milk were assessed in 200 lactating women. The risk assessment of infants’ exposure to PBDE was performed through the estimated daily intake (EDI) calculation. The geometric mean (GM) of ∑PBDEs levels was 2.33 (0.14–6.05) ng/g wet weight. At least one PBDE congener was detected in the samples, and the 2,2′,4,4′-tetrabromodiphenyl ether (BDE-47) showed a 100% of detection rate (GM of 1.05 ng/g). Location of residence, maternal level education, monthly salary, and race were positively associated with PBDE levels (p < 0.05). The EDI of BDE-47 was higher in Belo Horizonte (8.29 ng/kg/day) than in Viçosa (6.36 ng/kg/day), as well as for the ∑PBDEs (19.77 versus 12.78 ng/kg/day) (p < 0.05). Taking the high detection rate of PBDEs in breast milk and their toxicity, continuous studies on infant exposure, fetal growth, and child neurodevelopment are requested.