Coal Mine Rescue Robots Based on Binocular Vision: A Review of the State of the Art

Zhai, Guodong; Zhang, Wentao; Hu, Wenrong; Ji, Zhendong

doi:10.1109/access.2020.3009387

Cited by 41 publications

(25 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…H, W represents the height and width of image. The mask can be calculated as (10). N denotes the number of 1.…”

Section: B Evaluation Metricsmentioning

confidence: 99%

“…If the precisely disparity can be estimated, we can get the exactly depth. Therefore, stereo disparity estimation from a pair of stereo images has drawn more and more attention, which is also widely used in 3D reconstruction [2], [3], augmented reality (AR) [4], [5], self-driving [6], [7] and robotics [8]- [10]. The traditional stereo disparity estimation methods can be divided into global energy function [11], [12] and local similarity [13], [14].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Compact StereoNet: Stereo Disparity Estimation via Knowledge Distillation and Compact Feature Extractor

Gao

Zhou

Li³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Stereo disparity estimation is a difficult and crucial task in computer vision. Although many experimental techniques have been proposed in recent years with the flourishing of deep learning, very few studies take into account the optimization of computational complexity and memory consumption. Most previous works take advantage of stacked 3D convolutional block to generate fine disparity, but with a high computational cost and a large memory consumption. Considering the aforementioned problem, in this paper, we proposed an efficient convolutional neural architecture for stereo disparity estimation. In particular, a compact and efficient multi-scale extractor named MCliqueNet with stacked CliqueBlock was proposed to extract the more refined features for constructing multi-scale cost volume. In order to reduce the computational cost and maintain the accuracy of disparity, we utilized knowledge distillation scheme to transfer contextual features from a teacher network to a student network. Furthermore, we present a novel adaptive Smooth L1 (ASL) Loss for calculating the similarity between the contextual features of the teacher network and those of the student network, resulting in a more robust distillation process. Experimental results have shown that our method achieves competitive performance on the challenging Scene Flow and KITTI benchmarks while maintaining a very fast running speed.

show abstract

“…H, W represents the height and width of image. The mask can be calculated as (10). N denotes the number of 1.…”

Section: B Evaluation Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Compact StereoNet: Stereo Disparity Estimation via Knowledge Distillation and Compact Feature Extractor

Gao

Zhou

Li³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Researchers have developed a variety of depth detection sensors to obtain depth data, such as infrared distance sensor [ 2 ], ultrasonic distance sensor [ 3 ], laser distance sensor [ 4 ]. In recent years, higher-performance depth detectors, such as binocular cameras [ 5 , 6 , 7 , 8 , 9 ], millimeter-wave radar [ 10 , 11 ] and lidar [ 12 , 13 , 14 , 15 , 16 ] have also been equipped on UAVs to complete relevant missions. Researchers also found that the accuracy of depth information can be effectively improved by using several depth detectors simultaneously [ 17 ].…”

Section: Introductionmentioning

confidence: 99%

Monocular Depth Estimation with Self-Supervised Learning for Vineyard Unmanned Agricultural Vehicle

Cui

Feng

Wang

et al. 2022

Sensors

View full text Add to dashboard Cite

To find an economical solution to infer the depth of the surrounding environment of unmanned agricultural vehicles (UAV), a lightweight depth estimation model called MonoDA based on a convolutional neural network is proposed. A series of sequential frames from monocular videos are used to train the model. The model is composed of two subnetworks—the depth estimation subnetwork and the pose estimation subnetwork. The former is a modified version of U-Net that reduces the number of bridges, while the latter takes EfficientNet-B0 as its backbone network to extract the features of sequential frames and predict the pose transformation relations between the frames. The self-supervised strategy is adopted during the training, which means the depth information labels of frames are not needed. Instead, the adjacent frames in the image sequence and the reprojection relation of the pose are used to train the model. Subnetworks’ outputs (depth map and pose relation) are used to reconstruct the input frame, then a self-supervised loss between the reconstructed input and the original input is calculated. Finally, the loss is employed to update the parameters of the two subnetworks through the backward pass. Several experiments are conducted to evaluate the model’s performance, and the results show that MonoDA has competitive accuracy over the KITTI raw dataset as well as our vineyard dataset. Besides, our method also possessed the advantage of non-sensitivity to color. On the computing platform of our UAV’s environment perceptual system NVIDIA JETSON TX2, the model could run at 18.92 FPS. To sum up, our approach provides an economical solution for depth estimation by using monocular cameras, which achieves a good trade-off between accuracy and speed and can be used as a novel auxiliary depth detection paradigm for UAVs.

show abstract

“…The feature extraction can be realized by the visual measurement system after image acquisition and processing of the object. According to the dimension information, visual measurement can also be divided into monocular measurement [8,9] and multi-ocular measurement [10]. Monocular measurement is simple in structure, light in weight and low in cost, but it is difficult to obtain the depth information of objects.…”

Section: Introductionmentioning

confidence: 99%

Pose Determination of the Disc Cutter Holder of Shield Machine Based on Monocular Vision

Peng

Zhu

Zhang

et al. 2022

Sensors

View full text Add to dashboard Cite

The visual measurement system plays a vital role in the disc cutter changing robot of the shield machine, and its accuracy directly determines the success rate of the disc cutter grasping. However, the actual industrial environment with strong noise brings a great challenge to the pose measurement methods. The existing methods are difficult to meet the required accuracy of pose measurement based on machine vision under the disc cutter changing conditions. To solve this problem, we propose a monocular visual pose measurement method consisting of the high precision optimal solution to the PnP problem (OPnP) method and the highly robust distance matching (DM) method. First, the OPnP method is used to calculate the rough pose of the shield machine’s cutter holder, and then the DM method is used to measure its pose accurately. Simulation results show that the proposed monocular measurement method has better accuracy and robustness than the several mainstream PnP methods. The experimental results also show that the maximum error of the proposed method is 0.28° in the direction of rotation and 0.32 mm in the direction of translation, which can meet the measurement accuracy requirement of the vision system of the disc cutter changing robot in practical engineering application.

show abstract

Coal Mine Rescue Robots Based on Binocular Vision: A Review of the State of the Art

Cited by 41 publications

References 84 publications

Compact StereoNet: Stereo Disparity Estimation via Knowledge Distillation and Compact Feature Extractor

Compact StereoNet: Stereo Disparity Estimation via Knowledge Distillation and Compact Feature Extractor

Monocular Depth Estimation with Self-Supervised Learning for Vineyard Unmanned Agricultural Vehicle

Pose Determination of the Disc Cutter Holder of Shield Machine Based on Monocular Vision

Contact Info

Product

Resources

About