Aiming at the problems of low building segmentation accuracy and blurred edges in high-resolution remote sensing images, an improved fully convolutional neural network is proposed based on the SegNet network. First, GELU, which performs well in deep learning tasks, is selected as the activation function to avoid neuron deactivation. Second, the improved residual bottleneck structure is used in the encoding network to extract more building features. Then, skip connections are used to fuse images The low-level and high-level semantic features are used to assist image reconstruction. Finally, an improved edge correction module is connected at the end of the decoding network to further correct the edge details of the building and improve the edge integrity of the building. Experiments are carried out on the Massachusetts building dataset, and the precision rate, recall rate, and F1 value reach 93.5%, 79.3%, and 81.9%, respectively, and the comprehensive evaluation index F1 value is improved by about 5% compared with the basic network.