RoadFormer: Pyramidal deformable vision transformers for road network extraction with remote sensing images

Jiang, Xiaoling; Li, Yinyin; Jiang, Tao; Xie, Jingwei; Wu, Yi‐Long; Cai, Qianfeng; Jiang, Jinhui; Xu, Jiaming; Hui, Zhang

doi:10.1016/j.jag.2022.102987

Cited by 21 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The success of DeformAtt-ViT was attributed to its unique deformable attention module. The deformable attention ensured that the same receptive field was applied to all queries, and the sampling points were learned through the offset network [40]. Moreover, the offset network leverages query features as inputs to generate corresponding offsets for all reference points.…”

Section: Comparison Analysis Between Cnns and Transformersmentioning

confidence: 99%

Identification of Fish Hunger Degree with Deformable Attention Transformer

Wu,

Xu,

et al. 2024

JMSE

View full text Add to dashboard Cite

Feeding is a critical process in aquaculture, as it has a direct impact on the quantity and quality of fish. With advances in convolutional neural network (CNN) and vision transformer (ViT), intelligent feeding has been widely adopted in aquaculture, as the real-time monitoring of fish behavior can lead to better feeding decisions. However, existing models still have the problem of insufficient accuracy in the fish behavior-recognition task. In this study, the largemouth bass (Micropterus salmoides) was selected as the research subject, and three categories (weakly, moderately, and strongly hungry) were defined. We applied the deformable attention to the vision transformer (DeformAtt-ViT) to identify the fish hunger degree. The deformable attention module was extremely powerful in feature extraction because it improved the fixed geometric structure of the receptive fields with data-dependent sparse attention, thereby guiding the model to focus on more important regions. In the experiment, the proposed DeformAtt-ViT was compared with the state-of-the-art transformers. Among them, DeformAtt-ViT achieved optimal performance in terms of accuracy, F1-score, recall, and precision at 95.50%, 94.13%, 95.87%, and 92.45%, respectively. Moreover, a comparative evaluation between DeformAtt-ViT and CNNs was conducted, and DeformAtt-ViT still dominated the others. We further visualized the important pixels that contributed the most to the classification result, enabling the interpretability of the model. As a prerequisite for determining the feed time, the proposed DeformAtt-ViT could identify the aggregation level of the fish and then trigger the feeding machine to be turned on. Also, the feeding machine will stop working when the aggregation disappears. Conclusively, this study was of great significance, as it explored the field of intelligent feeding in aquaculture, enabling precise feeding at a proper time.

show abstract

Section: Comparison Analysis Between Cnns and Transformersmentioning

confidence: 99%

Identification of Fish Hunger Degree with Deformable Attention Transformer

Wu,

Xu,

et al. 2024

JMSE

View full text Add to dashboard Cite

show abstract

“…Current research in this domain is predominantly represented by the work of Kearney et al (2020), who classified pixels of satellite images into forest/rural roads and non-roads using a CNN and training data obtained from vehicle monitoring in the area. Additionally, Jiang et al (Jiang et al, 2022) developed the Roadformer, employing a pyramidal architecture of a deformable vision transformer for road network extraction from satellite images. The deformable vision transformer consistently attends to the most important semantic features, significantly improving performance.…”

Section: Related Workmentioning

confidence: 99%

Forest/rural road network detection and condition monitoring based on satellite imagery and deep semantic segmentation

Kelesakis,

Marthoglou,

Tokmaktsi

et al. 2024

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Sustainable forest and emergency management require comprehensive data on the forest road network and its condition. This paper presents the final framework of the INFOROAD project (https://inforoad.karteco.gr/), which integrates cutting-edge remote sensing and machine learning technologies for automated periodic extraction and monitoring of the forest road network. The framework includes gravel road extraction, road graph extraction, and gravel road condition monitoring, with a focus on the periurban forest in Thessaloniki. The road extraction employs the U-TAE network architecture, with a proposed modification using inverted residual blocks for improved accuracy. Road graph extraction involves creating a graph from road segmentation output or OSM data, enabling efficient road segment analysis. Gravel-road width calculation utilizes road segmentation results and a series of image processing steps, while road condition monitoring employs ML/AI classification algorithms. Worldview 3 high-resolution satellite images and various auxiliary data sources (e.g. DEM) are used as input, including field measurements for the training of classification algorithms. Results showcase the effectiveness of the proposed framework, with gravel road extraction accuracy improved by the modified U-TAE model. Regarding gravel road condition monitoring, algorithms achieving satisfactory results are identified, despite the challenges that arise, due to the significant surface and texture variations in forest and agricultural roads. A WebGIS platform facilitates information presentation, user interaction, and management of geospatial information, supporting various functionalities such as layer management and spatial data visualization. The INFOROAD project represents a significant advancement in leveraging technology for sustainable forest road management and emergency preparedness. Future steps may involve further enhancements and adaptations for improvement of results.

show abstract

“…At present, research in this area is mainly limited to the study of Kearney et al, 5 who manage to classify the pixels of satellite images to those belonging to forest/rural roads and those that do not belong, using a CNN and a set of training data collected from the application of vehicles monitoring of people operating in the area. Jiang et al, 6 developed the Roadformer, using. a pyramidal architecture of a deformable vision transformer, for the extraction of road networks from satellite images.…”

Section: Related Workmentioning

confidence: 99%

A holistic framework for forestry and rural road detection based on satellite imagery and deep semantic segmentation

Kelesakis,

Marthoglou,

GRAMMALIDIS

et al. 2023

Ninth International Conference on Remote Sensing and Geoinformation of the Environment (RSCy2023)

View full text Add to dashboard Cite

The role of forest and rural road network is very important and multifaceted for: a) the exploitation of forests/fields and movement of the staff involved in their management and protection, b) access to existing technical facilities, c) tourism and recreation and d) harvesting and transport of products. Hence, it is important to systematically record and monitor the network status, including its road segment status and important Points of Interest. Since in most cases, such detection and monitoring procedures require manual and tedious procedures, the INFOROAD project (https://inforoad.karteco.gr/) (Project code: KMP6-0079153), implemented under the framework of the Action "Investment Plans of Innovation" of the Operational Program "Central Macedonia 2014 2020", that is co-funded by the European Regional Development Fund and Greece", is developing an innovative methodology and tools for automated extraction, mapping and status monitoring of forest and rural road network, by combining remote sensing data with state-of-the-art deep learning approaches. Specifically, automated procedures will be developed for: a) road extraction, b) road network graph detection and c) road segment condition monitoring, from very high-resolution multispectral satellite images using innovative semantic segmentation and classification based on deep learning. From a research perspective, these problems are very challenging, as forest and agricultural roads -as opposed to asphalt roads -exhibit more significant surface and texture variations, while only few related datasets exist. For this reason, additional data, which is necessary for the training and evaluation of the deep learning algorithms, will also be collected, e.g., by UAVs equipped with RGB cameras that can also provide accurate DSM information, manual inspection or other free sources. The associated dataset that will be created for the pilot application of the project in Seich-Sou forest, Thessaloniki, will be made freely available to boost future research on the field. The display and management of this information by relevant stakeholders (forest authorities, public, etc.) will be performed by an online Geographical Information System (WebGIS).

show abstract

RoadFormer: Pyramidal deformable vision transformers for road network extraction with remote sensing images

Cited by 21 publications

References 37 publications

Identification of Fish Hunger Degree with Deformable Attention Transformer

Identification of Fish Hunger Degree with Deformable Attention Transformer

Forest/rural road network detection and condition monitoring based on satellite imagery and deep semantic segmentation

A holistic framework for forestry and rural road detection based on satellite imagery and deep semantic segmentation

Contact Info

Product

Resources

About