In order to optimize the technology of the building, the damage identification of the building structure is studied. Firstly, back propagation neural network (BPNN) and information fusion technology are used to build neural network models. Secondly, the established model is trained. Finally, the displacement mode, natural frequency, Modal Assurance Criterion (MAC), and three kinds of information fusion with only one characteristic information are used as input data to analyse the results of BPNN identification damage. The results show that when the natural frequency is used as the sensitive feature of damage, the accuracy is the highest. The difference between the network output value and the expected value is the smallest, the network output is the most stable, and the network recognition effect is the best. The network output of a mixture of two damage depths is compared with the output of a single damage depth. The data of the network training set composed of the feature data with damage depth of 20 mm and 5 mm has higher accuracy and more accurate damage recognition. This research provides a reference for the optimization of building survey technology and has certain practical value.