The immunomodulation of chemically selenylated polysaccharides has been attracting more attention recently, but the corresponding performance of the yam polysaccharides (YPS) with lower selenylation extent remains, thus far, unsolved. In this study, the YPS was selenylated with Na2SeO3 under acidic conditions generated by HNO3 to reach two lower selenylation extents, yielding two selenylated YPSs, namely SeYPS-1 and SeYPS-2 with selenium contents of 715 and 1545 mg/kg, respectively. The results indicated that YPS, SeYPS-1, and SeYPS-2 all had in vitro immuno-modulation when using RAW 264.7 macrophages and murine splenocytes as cell models. In detail, the three polysaccharide samples at dose levels of 5–160 μg/mL showed insignificant cytotoxicity to the macrophages and splenocytes with cell exposure times of 12–24 h, because of the measured values of cell viability larger than 100%. However, Na2SeO3 at dose levels of 1.3–3.25 μg/mL mostly caused obvious cytotoxic effects on the cells, resulting in reduced cell viability values or cell death, efficiently. The results demonstrated that, compared with YPS, both SeYPS-1 and SeYPS-2 at a lower dose level (5 μg/mL) were more active at promoting phagocytosis activity, increasing the CD4+/CD8+ ratio of the T-lymphocyte sub-population in the murine splenocyte, improving cytokine secretion, including interleukin-6 (IL-6), IL-1β, and tumor necrosis factor-α in the macrophages, or increasing interferon-γ secretion, but suppressing IL-4 production in the splenocytes. Consistently, SeYPS-2 has more potential than SeYPS-1 at exerting these assessed bioactivities in the cells. Thus, we conclude that a chemical modification of YPS using trace element Se at a lower selenylation extent could bring about higher immunomodulatory activity towards macrophages and splenocytes, while selenylation extent of YPS is a critical factor used to govern the assessed activity changes of YPS.