<p><strong>Abstract.</strong> The survey of building pathologies is focused on reading the state of conservation of the building, composed by the survey of constructive and decorative details, the masonry layering, the crack pattern, the degradation and the color recognition. The drawing of these representations is a time-consuming task, accomplished by manual work by skilled operators who often rely on in-situ analysis and on pictures. In this project three-dimensional an automated method for the condition survey of reinforced concrete spalling has been developed. To realize the automated image-based survey it has been exploited the Mask R-CNN neural network. The training phase has been executed over the original model, providing new examples of images with concrete cover detachments. At the same time, a photogrammetry process involved the images, in order to obtain a point cloud which acts as a reference to a Scan to BIM process. The BIM environment serves as a collector of information, as it owns the ontology to recreate entities and relationships. The information as extracted by neural network and photogrammetry serve to create the pictures which depict the concrete spalling in the BIM environment. A process of projecting information from the images to the BIM recreates the shapes of the pathology on the objects of the model, which becomes a decision support system for the built environment. A case study of a concrete beam bridge in northern Italy demonstrates the validity of the process.</p>