Background: Human milk oligosaccharides (HMOs) are the third most abundant component of human milk. Various factors may affect the concentration of HMOs, such as the lactation period, Lewis blood type, and the maternal secretor gene status. Objectives: The purpose of this study is to investigate factors associated with HMO concentrations in Chinese populations. Methods: A sub-sample of 481 was randomly selected from a large cross-sectional study in China (n = 6481) conducted in eight provinces (Beijing, Heilongjiang, Shanghai, Yunnan, Gansu, Guangdong, Zhejiang, and Shandong) between 2011 and 2013. HMO concentrations were determined by a high-throughput UPLC-MRM method. Various factors were collected through face-to-face interviews. Anthropometric measurement was conducted by trained staff. Results: Median total HMO concentration was 13.6 g/L, 10.7 g/L, and 6.0 g/L for colostrum, transitional milk, and mature milk, respectively. HMO concentration decreased significantly as the lactation period increased (p < 0.0001). There were significant differences of average total HMO concentration between secretor mothers and non-secretor mothers (secretor 11.3 g/L vs. non-secretor 5.8 g/L, p < 0.0001). There were significant differences of average total HMO concentrations among three Lewis blood types (p = 0.003). Comparing with the concentration of total oligosaccharides of Le+ (a−b+), average of total oligosaccharides concentrations increased by 3.9 (Le+ (a+b−), p = 0.004) and 1.1 g/L (Le− (a−b−), p = 0.049). The volume of breast milk expressed and the province the mother came from affected the concentration of total oligosaccharides (all p < 0.0001). Maternal BMI (p = 0.151), age (p = 0.630), prematurity (p = 0.850), mode of delivery (p = 0.486), infants’ gender (p = 0.685), maternal education level (p = 0.989), maternal occupation (p = 0.568), maternal allergic history (p = 0.370), maternal anemia (p = 0.625), pregnancy-induced hypertension (p = 0.739), gestational diabetes (p = 0.514), and parity (p = 0.098) were not significantly correlated with the concentration of milk oligosaccharides. The concentrations of 2′-fucosyllactose (2′-FL), lacto-N-neotetraose (LNnT), sialyllacto-N-tetraose c (LSTc), lacto-N-fucopentaose I (LNFP-I), disialylated lacto-N-tetraose (DSLNT), difucosyl-para-lacto-N-neohexaose (DFpLNnH), difucosyl-lacto-N-hexaose (DFLNH[a]), and 3-sialyllactose (3′-SL) showed a gradual downward trend, while the concentration of 3-fucosyllactose (3-FL) showed a gradual upward trend among three lactation stages (p < 0.05). Conclusions: The concentration of HMOs changes throughout lactation, and it varies between different HMOs. HMO concentrations differed between lactation stage, maternal secretor gene status, Lewis blood type, volume of breast milk expressed, and the province the mother came from. Prematurity, mode of delivery, parity, infants’ gender, and maternal characteristics did not affect the HMO concentration. Geographical region may be not associated with HMOs concentration in human milk. There may be a mechanism for co-regulation of the secretion of some of the oligosaccharides such as 2′FL vs. 3FL, 2′FL vs. LNnT, and lacto-N-tetraose (LNT).