Chinese olive (Canarium album L.) has been highlighted for its remarkable health benefits. We previously showed that the ethyl acetate fraction of Chinese olive (COE) is an effective anti-inflammatory agent. In this study, we used a luciferase-based RAW 264.7 cell platform to detect the transcriptional activity of NF-κB, a key mediator of inflammation, and the promoter activity of its downstream target, COX-2. Through functional-oriented screening using these platforms, we further divided COE into several subfractions. Subsequently, we used silica gel column chromatography for purification, and the active compounds were separated and isolated by thin-layer chromatography (TLC) and high-performance liquid chromatography (HPLC). The structure of the resulting compound with high anti-inflammatory activity was then identified as scoparone. Our results showed that scoparone not only inhibited lipopolysaccharide (LPS)-induced secretion of nitric oxide and suppressed M1 macrophage markers (iNOS, Il-6, Ccl2, and Tnf-α) but also markedly decreased the production of pro-inflammatory cytokines (IL-6, CCL2, and TNF-α). Treatment with scoparone significantly reduced the protein level of TNF-α in LPS-treated bone-marrow-derived macrophages (BMDMs). In addition, scoparone promoted macrophages toward an M2 anti-inflammatory phenotype, as determined by the significantly increased gene expression of M2 macrophage markers (Arg1, Ym1, Mrc1, Il-10, and Cd206) and the protein level of Arg1. This study indicates that COE fruit has high therapeutic potential for various inflammatory diseases as a result of switching the macrophage phenotype from pro-inflammatory M1 to anti-inflammatory M2.