The high‐rise residential shear wall structure is a crucial component of urban building clusters, while the limited availability of detailed structural information becomes a critical bottleneck in improving the accuracy of seismic performance assessment for high‐rise residential shear wall buildings in urban areas. Based on easily obtainable yet limited structural data at the urban scale, this paper proposes a method to address the shortcomings of existing research on reconstructing hidden structural information and enhance the accuracy of structural seismic performance assessment. It includes a physics‐constrained generative adversarial network module and a fuzzy inference system module to reconstruct the spatial arrangement of shear walls, and material strength grades within buildings, respectively. Validated against two actual buildings, the method outperforms the widely used simplified analysis method at the urban scale, achieving 85.9% accuracy in predicting damage states across various floors.