BackgroundThe medicinal orchid Dendrobium moniliforme contains water-soluble polysaccharides, phenanthrenes, bibenzyl derivatives, and polyphenol compounds. This study explored the antioxidant and cytotoxic activities of D. moniliforme extracts and detected their bioactive compounds.MethodsPlant material was collected from the Daman of Makawanpur district in central Nepal. Plant extracts were prepared from stems using hexane, chloroform, acetone, ethanol and methanol. The total polyphenol content (TPC) in each extract was determined using Folin-Ciocalteu’s reagent and the total flavonoid content (TFC) in each extract was determined using the aluminium chloride method. The in vitro antioxidant and cytotoxic activities of each extract were determined using DPPH (2,2-diphenyl-1-picrylhydrazyl) and MTT (3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide) assays respectively. Gas chromatography and mass spectrometry (GC-MS) analysis was used to detect bioactive compounds.ResultsTPC content was highest (116.65 μg GAE/mg of extract) in D. moniliforme chloroform extract (DMC) and TFC content was highest (116.67 μg QE/mg of extract) in D. moniliforme acetone extract (DMA). D. moniliforme hexane extract (DMH) extract showed the highest percentage of DPPH radical scavenging activity (94.48%), followed closely by D. moniliforme ethanol extract (DME) (94.45%), DMA (93.71%) and DMC (94.35%) at 800 μg/ml concentration. The antioxidant capacities of DMC, DMA, DMH and DME, which were measured in IC50 values, were much lower 42.39 μg/ml, 49.56 μg/ml, 52.68 μg/ml, and 58.77 μg/ml respectively than the IC50 of D. moniliforme methanol extract (DMM) (223.15 μg/ml). DMM at the concentration of 800 μg/ml most inhibited the growth of HeLa cells (78.68%) and DME at the same concentration most inhibited the growth of U251 cells (51.95%). The cytotoxic capacity (IC50) of DMM against HeLa cells was 155.80 μg/ml of extract and that of DME against the U251 cells was 772.50 μg/ml of extract. A number of bioactive compounds were detected in both DME and DMM.ConclusionThe fact that plant extract of D. moniliforme has a number of bioactive compounds which showed antioxidant and cytotoxic activities suggests the potential pharmacological importance of this plant.Electronic supplementary materialThe online version of this article (10.1186/s12906-018-2197-6) contains supplementary material, which is available to authorized users.