Ziziphus jujuba var. spinosa (Bunge) Hu ex H.F.Chow [Rhamnaceae; Ziziphi Spinosae Semen (ZSS)] has attracted extensive attention as the first choice of traditional Chinese medicine in the treatment of insomnia. However, recent studies on the sleep-improving mechanism of ZSS have mainly focused on the role of single components. Thus, to further reveal the potential mechanism of ZSS, an assessment of its multiple constituents is necessary. In this study, ZSS extract (ZSSE) was obtained from ZSS via detailed modern extraction, separation, and purification technologies. The chemical constituents of ZSSE were analyzed by high-performance liquid chromatography–mass spectrometry (HPLC–MS). For in vivo experiments, a rat model of insomnia induced by p-chlorophenylalanine (PCPA) was established to investigate the potential effect and corresponding mechanism of ZSSE on improving sleep. Hematoxylin–eosin staining (HE) results revealed that the drug group showed prominent advantages over the model group in improving sleep. Moreover, the brain levels of γ-aminobutyric acid (GABA), glutamic acid (Glu), 5-hydroxytryptamine (5-HT), and dopamine (DA) were monitored via enzyme-linked immunosorbent assay (ELISA) to further study the sleep-improving mechanism of ZSSE. We found that sleep was effectively improved via upregulation of GABA and 5-HT and downregulation of Glu and DA. In addition, molecular mechanisms of ZSSE in improving sleep were studied by immunohistochemical analysis. The results showed that sleep was improved by regulating the expression levels of GABA receptor subunit alpha-1 (GABAARα1) and GABA acid receptor subunit gamma-2 (GABAARγ2) receptors in the hypothalamus and hippocampus tissue sections. Therefore, this work not only identified the active ingredients of ZSSE but also revealed the potential pharmacological mechanism of ZSSE for improving sleep, which may greatly stimulate the prospective development and application of ZSSE.