Wire Structure Image-Based 3d Reconstruction Aided by Deep Learning

Kniaz, V. V.; Zheltov, Sergey Yu.; Remondino, Fabio; Knyaz, V. A.; Bordodymov, A.; Gruen, Armin

doi:10.5194/isprs-archives-xliii-b2-2020-435-2020

Cited by 7 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We process a subset of the images by the described above technique, and then use this subset as training data for WireNet neural network model (Kniaz et al, 2020). After training stage, we use WireNet for segmentation of the rest of the images.…”

Section: Framework For Segmentation and Vectorizationmentioning

confidence: 99%

Multi-sensor Data Analysis for Aerial Image Semantic Segmentation and Vectorization

Knyaz,

Kniaz,

Zheltov

et al. 2024

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. One of the urgent and constantly in demand problems is updating maps. Maps, representing geo-information in vector form, have undoubted advantages in compactness and ”readability” compared to aerial photographs. The issue of maps actuality is critically important for rational urban planning, precision farming, the relevance of the cadastre and other geospatial applications. Various sources of data are used for maps updating, with aerial imagery being the main and rich source of information. Automatic processing of aerial photographs makes it possible to efficiently extract vector information, providing operational monitoring and accounting for changes that have appeared. The presented study addresses the problem of multi sensor information fusion in order to obtain accurate vector information. We use aerial images as a main data source and additionally the data of laser scanning and ground survey to increase performance of automatic image semantic segmentation and vectorization. The proposed framework is demonstrated on the task of forest monitoring.

show abstract

Section: Framework For Segmentation and Vectorizationmentioning

confidence: 99%

Multi-sensor Data Analysis for Aerial Image Semantic Segmentation and Vectorization

Knyaz,

Kniaz,

Zheltov

et al. 2024

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Qualitative experimental results demonstrate that the adversarial loss allows to improve the quality of the segmentation in terms of contour accuracy. This improvement results in notable reducing of the root mean square error of the best-fit point-to-point alignment with point cloud of the laser scanning [6] in comparison with first version of WireNet [73] (Section 5.3, Figure 15).…”

Section: Wirenetv2 Model Architecturementioning

confidence: 99%

“…Still, the model was unable to distinguish between foreground and background wire structures in the images. Hence, a new model, based on the HRNetv2, was developed and called WireNet [73] to improve the segmentation of the frontmost and rear parts of the tower.…”

Section: Deep Learning Approachmentioning

confidence: 99%

“…The original WireNet [73] model extended the HRNetv2 with two key contributions: (1) an additional parallel channel for the segmentation of rear structures, (2) a negative log likelihood loss function. While the modified architecture was capable of segmenting images of dense wire structures with sufficient quality, it still suffered from two disadvantages: (i) It had low generalization ability and failed to label wire parts with a similar texture but different structure, such as the upper levels of the tower, if the training dataset included only ground-truth masks of the bottom and middle levels; (ii) The segmentation had soft edges at sharp corners that were caused by the negative log likelihood loss.…”

Section: Wirenetv2 Model Architecturementioning

confidence: 99%

See 1 more Smart Citation

3D Reconstruction of a Complex Grid Structure Combining UAS Images and Deep Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

The latest advances in technical characteristics of unmanned aerial systems (UAS) and their onboard sensors opened the way for smart flying vehicles exploiting new application areas and allowing to perform missions seemed to be impossible before. One of these complicated tasks is the 3D reconstruction and monitoring of large-size, complex, grid-like structures as radio or television towers. Although image-based 3D survey contains a lot of visual and geometrical information useful for making preliminary conclusions on construction health, standard photogrammetric processing fails to perform dense and robust 3D reconstruction of complex large-size mesh structures. The main problem of such objects is repeated and self-occlusive similar elements resulting in false feature matching. This paper presents a method developed for an accurate Multi-View Stereo (MVS) dense 3D reconstruction of the Shukhov Radio Tower in Moscow (Russia) based on UAS photogrammetric survey. A key element for the successful image-based 3D reconstruction is the developed WireNetV2 neural network model for robust automatic semantic segmentation of wire structures. The proposed neural network provides high matching quality due to an accurate masking of the tower elements. The main contributions of the paper are: (1) a deep learning WireNetV2 convolutional neural network model that outperforms the state-of-the-art results of semantic segmentation on a dataset containing images of grid structures of complicated topology with repeated elements, holes, self-occlusions, thus providing robust grid structure masking and, as a result, accurate 3D reconstruction, (2) an advanced image-based pipeline aided by a neural network for the accurate 3D reconstruction of the large-size and complex grid structured, evaluated on UAS imagery of Shukhov radio tower in Moscow.

show abstract

“…To incorporate information about 3D coordinates of the landmark a spatial consistency loss function LSC (A, L0) is introduced. Specifically, similarly to (Kniaz et al, 2020, Knyaz et al, 2020b we add information about predicted location of detecting points as masked image M containing epipolar constrains for NL landmarks detected in the reference image A0.…”

Section: Cl-net Modelmentioning

confidence: 99%

Automatic Anthropological Landmarks Recognition and Measurements

Knyaz

Maksimov

Novikov

et al. 2021

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

Abstract. Many anthropological researches require identification and measurement of craniometric and cephalometric landmarks which provide valuable information about the shape of a head. This information is necessary for morphometric analysis, face approximation, craniafacial identification etc. Traditional techniques use special anthropological tools to perform required measurements, identification of landmarks usually being made by an expert-anthropologist. Modern techniques of optical 3D measurements such as photogrammetry, computer tomography, laser 3D scanning provide new possibilities for acquiring accurate 2D and 3D data of high resolution, thus creating new conditions for anthropological data analysis. Traditional anthropological manual point measurements can be substituted by analysis of accurate textured 3D models, which allow to retrieve more information about studied object and easily to share data for independent analysis. The paper presents the deep learning technique for anthropological landmarks identification and accurate 3D measurements. Photogrammetric methods and their practical implementation in the automatic system for accurate digital 3D reconstruction of anthropological objects are described.

show abstract

Wire Structure Image-Based 3d Reconstruction Aided by Deep Learning

Cited by 7 publications

References 20 publications

Multi-sensor Data Analysis for Aerial Image Semantic Segmentation and Vectorization

Multi-sensor Data Analysis for Aerial Image Semantic Segmentation and Vectorization

3D Reconstruction of a Complex Grid Structure Combining UAS Images and Deep Learning

Automatic Anthropological Landmarks Recognition and Measurements

Contact Info

Product

Resources

About