Oral cancer (OC) has been attracted research attention in recent years as result of its high morbidity and mortality. Costunolide (CTD) possesses potential anticancer and bioactive abilities that have been confirmed in several types of cancers. However, its effects on oral cancer remain unclear. This study investigated the potential anticancer ability and underlying mechanisms of CTD in OC in vivo and in vitro. Cell viability and anchorage-independent colony formation assays were performed to examine the antigrowth effects of CTD on OC cells; assessments for migration and invasion of OC cells were conducted by transwell; Cell cycle and apoptosis were investigated by flow cytometry and verified by immunoblotting. The results revealed that CTD suppressed the proliferation, migration and invasion of oral cancer cells effectively and induced cell cycle arrest and apoptosis; regarding the mechanism, CTD bound to AKT directly by binding assay and repressed AKT activities through kinase assay, which thereby downregulating the downstream of AKT. Furthermore, CTD remarkably promotes the generation of reactive oxygen species by flow cytometry assay, leading to cell apoptosis. Notably, CTD strongly suppresses cell-derived xenograft OC tumor growth in an in vivo mouse model. In conclusion, our results suggested that costunolide might prevent progression of OC and promise to be a novel AKT inhibitor.