Fast and accurate obstacle detection of manipulator in complex human–machine interaction workspace

Abstract

In order to solve the problems of a large amount of calculation, poor real-time performance, and insufficient detection accuracy in an existing robot obstacle detection system, this paper proposes an improved lightweight You Only Look Once Version 3 algorithm for obstacle detection and tracking, by combining the DeepSort algorithm with a three-dimensional model velocity estimation method. First, the depthwise separable convolution and convolution kernel pruning methods are used to lighten the network. Second, to meet the requirement of dynamic obstacle detection, the DeepSort method is used to track dynamic obstacles and estimate their moving speed. Third, a method of bounding-sphere modeling and discrete marking is proposed to simplify the obstacle model. The experimental results show that: the model size is reduced by 92.11%, which suggests that the proposed lightweight detection system is more suitable for industrial applications; the detection speed in the actual scene reaches 118 frames per second and the average detection accuracy is 92.6%, while the multiple objects tracking accuracy and multiple objects tracking precision are all above 0.8 in the dynamic obstacle tracking process. Static and dynamic obstacles in the workspace can be detected simultaneously with an average positioning accuracy error of 1.5 cm. The proposed model can not only effectively improve the detection efficiency of obstacles in the manipulator workspace but also enhance the safety of industrial robot systems.

Enhanced robot obstacle avoidance strategy: efficient distance estimation and collision avoidance for hidden robots

“…In figure 6(b), When the repulsive force makes robots far away from the target point, the velocity vector of the control point is opposite to the velocity vector of the obstacle point to ensure that robots move towards the target position while avoiding obstacles. Moreover, the resulting force vector F r is given by the equation: (8) where F c is the control point velocity vector, F t is the attractive force vector and F op is the repulsive force vector defined by (6). In this way, three variables contribute to the direction of the resulting force vector, while the magnitude depends only on the repulsive force vector.…”

“…Human-robot interaction is the focus of future life, including smart factories, and it plays a key role in building new industrial systems and measuring levels of national scientific and technological innovation [1][2][3][4]. As a fundamental characteristic and common technology in the field of robotics, safety has always been a top priority for scholars and enterprises alike [5,6]. Therefore, the method of using external sensors is being studied to construct a safety perception system for robots [7][8][9].…”

Visual cue-aided human supervised robot navigation guidance in photometrically challenging environments using adaptive spatial-feature kernel-guided bilateral LPP

“…In automated HRI environments, one of the main challenges is to perform quality tasks in the presence of complex, unstructured, and dynamically changing environments [4]. Dynamically changing working environments can be arisen from various uncertainties like illumination variation, occlusion, background adversities, spatial movement between the human and the robot, etc [5].…”

Research on real-time perception method of key targets in the comprehensive excavation working face of coal mine

“…Zhang and Zhang [12] proposed a local-global matching network tracking algorithm based on deep learning for the complexity of the underground coal mine scene, the existence of a large number of small targets, and the large variation of tracking target scales, which demonstrated high tracking accuracy for targets such as personnel, helmets, and vehicles in the complex background of coal mine. Cui et al [13] proposed an obstacle detection and tracking algorithm based on improved YOLOv3 with DeepSORT to realize fast tracking and speed estimation of dynamic obstacles for coal mine robots in complex human-robot interaction space. Zhang and Yan [14] jointly used the improved YOLOv5s model with the DeepSORT algorithm to track and count the coal blocks during the transportation operation and realized the abnormal identification of coal block retention and blockage behavior.…”