The prevalence of early and exclusive breastfeeding in Vietnam remains sub-optimal. The objective of this study was to determine factors associated with early initiation of breastfeeding (EIBF) and exclusive breastfeeding for the first 3 days after birth (EBF3D). We conducted a population-based, cross-sectional survey of 726 mothers with children aged 0–11 months in two provinces and one municipality from May to July 2020. Multinomial logistic regression was used to examine factors associated with EIBF and EBF3D. The prevalence of EIBF was 39.7% and EBF3D 18.0%. The EIBF prevalence is positively associated with immediate and uninterrupted skin-to-skin contact (SSC) for 10–29 min (aOR: 2.55; 95% CI: 1.49, 4.37), 30–59 min (aOR: 4.15; 95% CI: 2.08, 8.27), 60–80 min (aOR: 4.35; 95% CI: 1.50, 12.6), or ≥90 min (aOR: 5.87; 95% CI: 3.14, 10.98). EIBF was negatively associated with cesarean birth (aOR: 0.24; 95% CI: 0.11, 0.51), bringing infant formula to the birth facility (aOR: 0.49; 95% CI: 0.30, 0.78), purchased it after arrival (aOR: 0.37; 95% CI: 0.24, 0.60), or did both (aOR: 0.43; 95% CI: 0.21, 0.89). EBF3D was negatively associated with cesarean section birth (aOR: 0.15; 95% CI: 0.06, 0.39), vaginal birth with episiotomy (aOR: 0.40; 95% CI: 0.18, 0.88), bringing formula to the maternity facility (aOR: 0.03; 95% CI: 0.01, 0.07), purchased it after arrival (aOR: 0.02; 95% CI: 0.01, 0.06) or did both (aOR: 0.04; 95% CI: 0.02, 0.10). Receiving counseling from any source was not significantly associated with early breastfeeding practices. Policy and health service delivery interventions should be directed at eliminating infant formula from birthing environments, reducing unnecessary cesarean sections and episiotomies, providing immediate and uninterrupted SSC for all births, and improving breastfeeding counseling and support.