The monitoring and protection of historic buildings require a highly professional team and material resources. Monitoring and protecting historical architectural features is an urgent issue. According to the theory of biological gene expression, genes are the fundamental units that control and express biological traits. Similarly, the “genes” of historical architecture are the basic units that control historic features. Identifying these historical architecture “genes” involves identifying the main factors that control the historic features. This process is important for monitoring and protecting the historic features. At present, qualitative subjectivity, difficult quantification, poor recognition accuracy, and low reasoning and recognition efficiency exist in the genetic identification of historic buildings. As an example, this article describes Chinese Baroque architecture in Harbin, China, and draws on the principles of biological gene recognition to reference methods of architectural gene recognition in cultural geography and architecture. Improved U-Net models, traditional U-Net models, FCN models, and EfficientNet models that incorporate channel attention mechanisms are used to identify historic building genes, obtaining the optimal intelligent recognition for historical architectural genes based on deep learning. This research shows that the accuracy of an improved U-Net model incorporating a channel attention mechanism is 69%, which is 4%, 7%, and 1% higher than those of the traditional U-Net, FCN, and EfficientNet, respectively. The F1 score of the improved U-Net model reaches 0.654, which is higher than the 0.619 of the traditional U-Net model, 0.645 of the EfficientNet model, and 0.501 of the FCN model. Therefore, the improved U-Net model is the optimal method for identifying historical architecture genes. This research can provide new tools and methods for identifying historical architectural genes.