Glioma is the most common type of brain tumor, and it has a high mortality rate. Accurate tumor segmentation based on magnetic resonance imaging (MRI) is of great significance for the diagnosis and treatment of brain tumors. Recently, a brain tumor segmentation method based on U-Net has gained considerable attention. However, brain tumor segmentation is a challenging task due to the structural variations and inhomogeneous intensity of tumors. Existing brain tumor segmentation studies have shown that the problems of insufficient down-sampling feature extraction and loss of up-sampling information arise when using U-Net to segment brain tumors. In this study, we proposed an improved U-Net model, SEResU-Net, which combines the deep residual network and the Squeeze-and-Excitation Network (SENet). The deep residual network solves the problem of network degradation so that SEResU-Net can extract more feature information. SENet avoids information loss and enables the network to focus on the useful feature map, which solves the problem of insufficient segmentation accuracy of small-scale brain tumors. Furthermore, we developed a fusion loss function, which is composed of Dice loss and cross-entropy loss, to solve the problems of network convergence and data imbalance. We evaluated the segmentation performance of SEResU-Net on BraTS2018 and BraTS2019, two authoritative MRI brain tumor benchmarks. Experimental results revealed that SEResU-Net outperformed SEU-Net, ResU-Net, and U-Net. For SEResU-Net, the mean Dice similarity coefficients were 0.9373, 0.9108, and 0.8758 for the whole tumor, the tumor core, and the enhanced tumor, which were 7.10%, 11.88%, and 15.33% greater than those of the U-Net benchmark network, respectively. Our findings demonstrate that the proposed SEResU-Net has a competitive effect in segmenting multimodal brain tumors.