A new compound, named arillatanoside E, which was elucidated as 3-O-β-D-glucopyranosyl presenegenin 28-O-β-D-xylopyranosyl-(1 ⟶ 3)-β-D-xylopyranosyl-(1 ⟶ 4)-α-L-rhamnopyranosyl-(1 ⟶ 2)-(4-O-acetyl)-β-D-fucopyranosyl ester, along with 11 known compounds was isolated from the ethanolic extract of the roots of Polygala arillata. The 11 known compounds were identified as oleanolic acid (2), 3′-E-3,4,5-trimethoxy cinnamoyl-6-benzoyl sucrose (3), trans-ferulic acid (4), trans-feruloyl-glucoside (5), feruloyl-glucoside (6), 2,4,6-trimethoxy-1-O-β-D-glycoside (7), 3-methoxy-4-hydroxybenzoic acid (8), monopentadecanoin (9), sinapic acid (10), p-hydroxybenzaldehyde (11), and palmitic acid (12). Among them, seven isolated compounds 1, 2, 4, 5, 7, 8, and 10 exhibited little cytotoxic activity on macrophage RAW 264.7 cells. Then, the inhibitory effects of 7 isolates on nitric oxide (NO) production in lipopolysaccharide-activated macrophages were evaluated. As a result, 3 compounds have significant anti-inflammatory activity, and they were arillatanoside E (1), oleanolic acid (2), and 2,4,6-trimethoxy-1-O-β-D-glycoside (7).