Ulcerative colitis (UC), one of the typical inflammatory bowel diseases caused by dysregulated immunity, still requires novel therapeutic medicine with high efficacy and low toxicity. Hericium erinaceus has been widely used to treat different health problems especially gastrointestinal sickness in China for thousands of years. Here, we isolated, purified, and characterized a novel low weight polysaccharide (HEP10, Mw: 9.9 kDa) from the mycelia of H. erinaceus in submerged culture. We explored the therapeutic effect of HEP10 on UC and explored its underlying mechanisms. On one hand, HEP10 suppressed the production of TNF-α, IL-1β, IL-6, inducible iNOS, and COX-2 in LPS challenged murine macrophage RAW264.7 cells, as well as in colons from DSS-induced colitis mice. On the other hand, HEP10 treatment markedly suppressed the activation of NLRP3 inflammasome, NF-κB, AKT, and MAPK pathways. Moreover, HEP10 reversed DSS-induced alternation of the gut community composition and structure by significantly increasing Akkermansia muciniphila and also promoting functional shifts in gut microbiota. Structural equation modeling also highlighted that HEP10 can change widely through gut microbiota. In conclusion, HEP10 has a better prebiotic effect than the crude polysaccharides of H. erinaceus, which can be used as a novel dietary supplement and prebiotic to ameliorate colitis.