Pose Estimation of Excavator Manipulator Based on Monocular Vision Marker System

Zhao, Jiangying; Hu, Yongbiao; Tian, Mingrui

doi:10.3390/s21134478

Cited by 24 publications

(15 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ref. [ 57 ] estimated the orientation and position of excavators using monocular cameras by mounting markers on the excavators. To better understand monocular vision-based 3D localization, Table 1 summarizes the limitations of the localization methods used for aerial objects and ground objects.…”

Section: Related Workmentioning

confidence: 99%

Vision-Based Automated Recognition and 3D Localization Framework for Tower Cranes Using Far-Field Cameras

Wang

Zhang

Yang

et al. 2023

Sensors

View full text Add to dashboard Cite

Tower cranes can cover most of the area of a construction site, which brings significant safety risks, including potential collisions with other entities. To address these issues, it is necessary to obtain accurate and real-time information on the orientation and location of tower cranes and hooks. As a non-invasive sensing method, computer vision-based (CVB) technology is widely applied on construction sites for object detection and three-dimensional (3D) localization. However, most existing methods mainly address the localization on the construction ground plane or rely on specific viewpoints and positions. To address these issues, this study proposes a framework for the real-time recognition and localization of tower cranes and hooks using monocular far-field cameras. The framework consists of four steps: far-field camera autocalibration using feature matching and horizon-line detection, deep learning-based segmentation of tower cranes, geometric feature reconstruction of tower cranes, and 3D localization estimation. The pose estimation of tower cranes using monocular far-field cameras with arbitrary views is the main contribution of this paper. To evaluate the proposed framework, a series of comprehensive experiments were conducted on construction sites in different scenarios and compared with ground-truth data obtained by sensors. The experimental results show that the proposed framework achieves high precision in both crane jib orientation estimation and hook position estimation, thereby contributing to the development of safety management and productivity analysis.

show abstract

Section: Related Workmentioning

confidence: 99%

Vision-Based Automated Recognition and 3D Localization Framework for Tower Cranes Using Far-Field Cameras

Wang

Zhang

Yang

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…With the current approach the excavator is capable of performing construction tasks autonomously, however, operation in GPSdenied environments remains a challenge. Citing similar concerns, [22] propose to place artificial markers on the manipulator and track it using external cameras for excavation tasks. Although low-cost, such an approach is only suitable for tasks where the excavator is either static or operates within a limited workspace as markers need to remain within the detection range of the cameras.…”

Section: Related Workmentioning

confidence: 99%

Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots

Nubert

Khattak

Hutter

2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Enabling autonomous operation of large-scale construction machines, such as excavators, can bring key benefits for human safety and operational opportunities for applications in dangerous and hazardous environments. To facilitate robot autonomy, robust and accurate state-estimation remains a core component to enable these machines for operation in a diverse set of complex environments. In this work, a method for multimodal sensor fusion for robot state-estimation and localization is presented, enabling operation of construction robots in realworld scenarios. The proposed approach presents a graph-based prediction-update loop that combines the benefits of filtering and smoothing in order to provide consistent state estimates at high update rate, while maintaining accurate global localization for large-scale earth-moving excavators. Furthermore, the proposed approach enables a flexible integration of asynchronous sensor measurements and provides consistent pose estimates even during phases of sensor dropout. For this purpose, a dualgraph design for switching between two distinct optimization problems is proposed, directly addressing temporary failure and the subsequent return of global position estimates. The proposed approach is implemented on-board two Menzi Muck walking excavators and validated during real-world tests conducted in representative operational environments.

show abstract

“…In practice, it is impossible to ensure that the optical axis is strictly perpendicular to the measurement plane, necessitating an analysis of the measurement error of the calibration coefficient method. While previous research has focused on error analysis of calibration parameters in visual systems [30][31][32], there has been no related research on error analysis of the calibration coefficient method. To address this gap, this paper systematically analyzes the calibration coefficient method in monocular vision and derives the measurement and error formulars when the optical axis of the camera is not perpendicular to the calibration plane.…”

Section: Introductionmentioning

confidence: 99%

Error analysis of precision measurement with monocular vision

Wang,

Li,

Zhang

et al. 2023

Eng. Res. Express

View full text Add to dashboard Cite

To systematically study the measurement error of the calibration coefficient method when the camera’s optical axis is not perpendicular to the measurement plane, this paper first discusses the principle of error measurement with 2D imaging model, and then derives the distribution relationship of physical points corresponding to evenly distributed pixels in the pixel coordinate system based on the projection transformation matrix of the vision system. Theoretical analysis reveals an equivariance series in the expression of physical point intervals. Considering rotational symmetry of the coordinate axis, this paper analyzes the geometric relationship when the calibration plane rotates around the X axis and derives the mathematical expression for dimensional measurement. Finally, an experimental platform with both industrial and telecentric lenses is proposed and designed. Experimental results demonstrate that the measurement results of the expression derived in this paper are consistent with the traditional coordinate transformation method and the maximum deviation of the measurement is less than 0.06 mm, validating the theoretical analysis presented in this paper and providing theoretical guidance for subsequent practical application.

show abstract

Pose Estimation of Excavator Manipulator Based on Monocular Vision Marker System

Cited by 24 publications

References 37 publications

Vision-Based Automated Recognition and 3D Localization Framework for Tower Cranes Using Far-Field Cameras

Vision-Based Automated Recognition and 3D Localization Framework for Tower Cranes Using Far-Field Cameras

Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots

Error analysis of precision measurement with monocular vision

Contact Info

Product

Resources

About