PCTNet: depth estimation from single structured light image with a parallel CNN-transformer network

Zhu, Xinjun; Han, Zhiqiang; Zhang, Zhizhi; Song, Limei; Wang, Hongyi; Guo, Qinghua

doi:10.1088/1361-6501/acd136

Cited by 8 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… [ 78 ] 2023 Structured light A parallel CNN transformer network is proposed to achieve an improved depth estimation for structured light images in complex scenes. [ 79 ] 2022 Time-of-Flight (TOF) DELTAR is proposed to enable lightweight Time-of-Flight sensors to measure high-resolution and accurate depth by collaborating with color images. [ 80 ] 2020 Time-of-Flight (TOF) Based on the principle and imaging characteristics of TOF cameras, a single pixel is considered as a continuous Gaussian source, and its differential entropy is proposed as an evaluation parameter.…”

Section: Depth Perception Methods Based On Computer Visionmentioning

confidence: 99%

See 1 more Smart Citation

Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications

Wang,

Li,

2023

Sensors

View full text Add to dashboard Cite

With the rapid development of vision sensing, artificial intelligence, and robotics technology, one of the challenges we face is installing more advanced vision sensors on welding robots to achieve intelligent welding manufacturing and obtain high-quality welding components. Depth perception is one of the bottlenecks in the development of welding sensors. This review provides an assessment of active and passive sensing methods for depth perception and classifies and elaborates on the depth perception mechanisms based on monocular vision, binocular vision, and multi-view vision. It explores the principles and means of using deep learning for depth perception in robotic welding processes. Further, the application of welding robot visual perception in different industrial scenarios is summarized. Finally, the problems and countermeasures of welding robot visual perception technology are analyzed, and developments for the future are proposed. This review has analyzed a total of 2662 articles and cited 152 as references. The potential future research topics are suggested to include deep learning for object detection and recognition, transfer deep learning for welding robot adaptation, developing multi-modal sensor fusion, integrating models and hardware, and performing a comprehensive requirement analysis and system evaluation in collaboration with welding experts to design a multi-modal sensor fusion architecture.

show abstract

Section: Depth Perception Methods Based On Computer Visionmentioning

confidence: 99%

“…Additionally, as the detection distance increases, the accuracy of structured light decreases. To address these issues, current research efforts have employed strategies such as increasing power and changing coding methods [ 77 , 78 , 79 ].…”

Section: Depth Perception Methods Based On Computer Visionmentioning

confidence: 99%

Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications

Wang,

Li,

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Wang et al proposed SwinConvUNet, which employs a self-attention mechanism and the Swin Transformer for fringe-to-depth conversion, aiming to extract both local and global features [35]. Similar to these works, other researchers [36][37][38][39] presented a few end-to-end network models, such as MSUNet++, PCTNet, DF-Dnet, and DCAHINet; they focus on depth recovery through diverse multiscale feature fusion modules. The team in [40] introduced LiteF2DNet, a lightweight deep learning framework designed to reduce network weights, and they tested it on computeraided design (CAD) objects.…”

Section: Related Workmentioning

confidence: 99%

Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches

Nguyen,

Wang

2024

Sensors

View full text Add to dashboard Cite

The field of computer vision has been focusing on achieving accurate three-dimensional (3D) object representations from a single two-dimensional (2D) image through deep artificial neural networks. Recent advancements in 3D shape reconstruction techniques that combine structured light and deep learning show promise in acquiring high-quality geometric information about object surfaces. This paper introduces a new single-shot 3D shape reconstruction method that uses a nonlinear fringe transformation approach through both supervised and unsupervised learning networks. In this method, a deep learning network learns to convert a grayscale fringe input into multiple phase-shifted fringe outputs with different frequencies, which act as an intermediate result for the subsequent 3D reconstruction process using the structured-light fringe projection profilometry technique. Experiments have been conducted to validate the practicality and robustness of the proposed technique. The experimental results demonstrate that the unsupervised learning approach using a deep convolutional generative adversarial network (DCGAN) is superior to the supervised learning approach using UNet in image-to-image generation. The proposed technique’s ability to accurately reconstruct 3D shapes of objects using only a single fringe image opens up vast opportunities for its application across diverse real-world scenarios.

show abstract

“…The radar wave formed by external interference will cause obvious interference to the identification of adverse geologic bodies. (2) Most deep learning algorithms are built based on convolutional neural network (CNN), which tend to pay more attention to the local detailed features of the image and neglect the importance of global information in target inference (Zhou et al 2023, Zhu et al 2023. Adverse geologic bodies often need to be analyzed in conjunction with the overall characteristics of the image.…”

Section: Introductionmentioning

confidence: 99%

A deep learning-based algorithm for intelligent prediction of adverse geologic bodies in tunnels

Mei,

Wang,

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

Aiming at the issues of high subjectivity and low efficiency in the image analysis methods for overcast prediction of tunnel adverse geological bodies, a deep learning-based intelligent prediction algorithm, namely YOLO-SEI (YOLOv8 enhanced by Sim-EFFcinetNet and Interlaced Sparse Self-Attention), is proposed in this paper. Firstly, Sim-EfficientNet with good feature extraction performance and efficiency is proposed as the backbone of YOLOv8 by fusing the SimAM attention and the EfficientNet-v2 network, which improves the model's extraction capability for radar wave features of adverse geologic bodies. Then, a feature fusion module enhanced by Interlaced Sparse Self-Attention is designed to effectively make up for the deficiency of convolutional neural network that is difficult to fully extract the global information of radar images. The experimental results show that the mAP and F1 of YOLO-SEI are 84.87% and 82.28%, respectively, which are higher than other commonly used deep learning models. In addition, YOLO-SEI has smaller storage space (41MB) and faster image processing speed (41.24 f/s), which is suitble for the rapid measurement and prediction of adverse geologic bodies in tunnel excavation construction.

show abstract

PCTNet: depth estimation from single structured light image with a parallel CNN-transformer network

Cited by 8 publications

References 30 publications

Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications

Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications

Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches

A deep learning-based algorithm for intelligent prediction of adverse geologic bodies in tunnels

Contact Info

Product

Resources

About