Among 276 herbal extracts, a methanol extract of Castanopsis cuspidata var. sieboldii stems was selected as an experimental source for novel acetylcholinesterase (AChE) inhibitors. Five compounds were isolated from the extract by activity-guided screening, and their inhibitory activities against butyrylcholinesterase (BChE), monoamine oxidases (MAOs), and β-site amyloid precursor protein cleaving enzyme 1 (BACE-1) were also evaluated. Of these compounds, 4′-O-(α-l-rhamnopyranosyl)-3,3′,4-tri-O-methylellagic acid (3) and 3,3′,4-tri-O-methylellagic acid (4) effectively inhibited AChE with IC50 values of 10.1 and 10.7 µM, respectively. Ellagic acid (5) inhibited AChE (IC50 = 41.7 µM) less than 3 and 4. In addition, 3 effectively inhibited MAO-B (IC50 = 7.27 µM) followed by 5 (IC50 = 9.21 µM). All five compounds weakly inhibited BChE and BACE-1. Compounds 3, 4, and 5 reversibly and competitively inhibited AChE, and were slightly or non-toxic to MDCK cells. The binding energies of 3 and 4 (− 8.5 and − 9.2 kcal/mol, respectively) for AChE were greater than that of 5 (− 8.3 kcal/mol), and 3 and 4 formed a hydrogen bond with Tyr124 in AChE. These results suggest 3 is a dual-targeting inhibitor of AChE and MAO-B, and that these compounds should be viewed as potential therapeutics for the treatment of Alzheimer’s disease.