The present study was conducted to optimize extraction process for defatted pitaya seed extract (DPSE) adopting response surface methodology (RSM). A five-level central composite design was used to optimize total phenolic content (TPC), total flavonoid content (TFC), ferric reducing antioxidant power (FRAP), and 2,2′-azino-bis (3-ethylbenzothizoline-6-sulfonic acid (ABTS) activities. The independent variables included extraction time (30–60 min), extraction temperature (40–80 °C) and ethanol concentration (60%–80%). Results showed that the quadratic polynomial equations for all models were significant at (p < 0.05), with non-significant lack of fit at p > 0.05 and R2 of more than 0.90. The optimized extraction parameters were established as follows: extraction time of 45 min, extraction temperature of 70 °C and ethanol concentration of 80%. Under these conditions, the recovery of TPC, TFC, and antioxidant activity based on FRAP and ABTS were 128.58 ± 1.61 mg gallic acid equivalent (GAE)/g sample, 9.805 ± 0.69 mg quercetin equivalent (QE)/g sample, 1.23 ± 0.03 mM Fe2+/g sample, and 91.62% ± 0.15, respectively. Ultra-high-performance liquid chromatography-quadrupole time-of-flight mass spectrometry (UPLC-QTOF/MS) analysis identified seven chemical compounds with flavonoids constituting major composition of the DPSE.