Multi-scale hybrid vision transformer and Sinkhorn tokenizer for sewer defect classification

Haurum, Joakim Bruslund; Madadi, Meysam; Escalera, Sérgio; Moeslund, Thomas B.

doi:10.1016/j.autcon.2022.104614

Cited by 12 publications

(2 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A detector-focused architecture is presented to learn sewer pipe defects and properties, which showed excellent performance on multiple tasks (Haurum et al, 2022). Wang et al devote efforts to evaluate defect severity by detecting and segmenting defects in CCTV images.…”

Section: Related Workmentioning

confidence: 99%

Attention‐guided multiscale neural network for defect detection in sewer pipelines

Wang

Dang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

Sanitary sewer systems are major infrastructures in every modern city, which are essential in protecting water pollution and preventing urban waterlogging. Since the conditions of sewer systems continuously deteriorate over time due to various defects and extrinsic factors, early intervention in the defects is necessary to prolong the service life of the pipelines. However, prior works for defect inspection are limited by accuracy, efficiency, and economic cost. In addition, the current loss functions in object detection approaches are unable to handle the imbalanced data well. To address the above drawbacks, this paper proposes an automatic defect detection framework that accurately identifies and localizes eight types of defects in closed‐circuit television videos based on a deep neural network. First, an effective attention module is introduced and used in the backbone of the detector for better feature extraction. Then, a novel feature fusion mechanism is presented in the neck to alleviate the problem of feature dilution. After that, an efficient loss function that can reasonably adjust the weight of training samples is proposed to tackle the imbalanced data problem. Also, a publicly available dataset is provided for defect detection tasks. The proposed detection framework is robust against the imbalanced data and achieves a state‐of‐the‐art mean average precision of 73.4%, which is potentially applied in realistic sewer defect inspections.

show abstract

Section: Related Workmentioning

confidence: 99%

Attention‐guided multiscale neural network for defect detection in sewer pipelines

Wang

Dang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

show abstract

“…Consequently, their computational complexity grows quadratically with the number of tokens. Therefore, extensive efforts have been made to reduce the token number, and these efforts can be divided into two primary paradigms: token pruning [19], [32]- [34] and token merging [23]- [25], [35]- [37]. E-ViT [20] believes that the importance of tokens is reflected in the <CLS> token.…”

Section: Related Workmentioning

confidence: 99%

Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization

Lin

Shi

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Learning rational behaviors in open-world games like Minecraft remains to be challenging for Reinforcement Learning (RL) research due to the compound challenge of partial observability, high-dimensional visual perception and delayed reward. To address this, we propose JueWu-MC, a sample-efficient hierarchical RL approach equipped with representation learning and imitation learning to deal with perception and exploration. Specifically, our approach includes two levels of hierarchy, where the high-level controller learns a policy to control over options and the low-level workers learn to solve each sub-task. To boost the learning of sub-tasks, we propose a combination of techniques including 1) action-aware representation learning which captures underlying relations between action and representation, 2) discriminator-based self-imitation learning for efficient exploration, and 3) ensemble behavior cloning with consistency filtering for policy robustness. Extensive experiments show that JueWu-MC significantly improves sample efficiency and outperforms a set of baselines by a large margin. Notably, we won the championship of the NeurIPS MineRL 2021 research competition and achieved the highest performance score ever.

show abstract

Agglomerative Token Clustering

Haurum,

Escalera,

Taylor

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multi-scale hybrid vision transformer and Sinkhorn tokenizer for sewer defect classification

Cited by 12 publications

References 66 publications

Attention‐guided multiscale neural network for defect detection in sewer pipelines

Attention‐guided multiscale neural network for defect detection in sewer pipelines

Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization

Agglomerative Token Clustering

Contact Info

Product

Resources

About