Grasping Pose Detection for Loose Stacked Object Based on Convolutional Neural Network With Multiple Self-Powered Sensors Information

Yun, Juntong; Du, Jiang; Sun, Yuanyuan; Huang, Li; Tao, Bo; Jiang, Guozhang; Kong, Jianyi; Weng, Yaoqing; Li, Gongfa; Fang, Zifan

doi:10.1109/jsen.2022.3190560

Cited by 39 publications

(34 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the calibration of the fiber-optic stereo compound eye is a nonlinear mapping relationship, so it is difficult to establish an accurate geometric model. Aiming at this problem, this article introduces neural network, [37][38][39][40][41][42][43] which has strong nonlinear mapping ability and robustness, and the computation principle [44][45][46][47][48][49][50] of the optic stereo compound eye is studied in the next part.…”

Section: Calibration Principlementioning

confidence: 99%

Development of neural‐network‐based stereo bionic compound eyes with fiber bundles

Chen

Zhao

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

The compound eye system has many unique advantages, enabling organisms to quickly and accurately obtain the three-dimensional spatial information of the target. Therefore, its bionic applications have great potential in object localization, dynamic tracking, and 3D reconstruction. However, the current bionic compound eye systems still have great difficulties in target detection. Most of the bionic compound eye systems only verify the possibility of three-dimensional (3D) detection, and there is no suitable calibration and detection scheme. In this article, a fiber-optic stereo bionic compound eye is designed, and a simple and rapid calibration method suitable for this system is selected by comparing two effective optical information. Further, the system quantitatively analyzes the detection performance of the target to achieve precise positioning and dynamic trajectory tracking. In the static detection experiment, the average detection errors of the 3D position at the horizontal angle, the elevation angle and the depth are 0.33 • , 0.32 • and 1.84 mm, respectively. In the dynamic trajectory tracking experiment, the circle radius errors in the X-Z plane and Y-Z plane are 0.04 and 2.20 mm, respectively. The proposed system provides a new perspective to understand the nature of the compound eye. K E Y W O R D Sbionic compound eyes 1 , dynamic track tracking 4 , neural network 5 , stereo vision system 2 , three-dimensional detection 3 INTRODUCTIONAs a natural multi-aperture optical system, due to its special physiological structure, 1,2 the compound eyes have the characteristics of small size, high integration, large field of view, 3 high resolution rate, and sensitivity to moving objects. 4 So, it has great advantages in three-dimensional (3D) target detection. The bionic compound eyes are mainly divided into planar compound eye 5-13 and curved compound eye. [14][15][16][17][18][19][20][21] The planar compound eye is small in size and has a fast response, but is limited by the range of the field of view (FOV). The curved compound eye is the closest to the biological compound eye in terms of structure and performance, and it is also the main design idea at present. However, it is difficult to acquire accurate and distortion-free optical images in a large field of view. The nature is very mysterious, the computation principle [22][23][24][25][26][27] of the compound eye is an interesting topic.Research on object detection from existing bionic compound eye shows that such a vision system has the potential for at least qualitative object detection. Richard Hornsey's team proposed a DragonflEye system based on fiber bundle coupling, and realized the tracking of planar target through vanishing point and Z-translation image center calibration techniques. 18,19 The method has a complex structure and is difficult to manufacture, though it does not require calibration of a single sub-eye and has high detection accuracy. Song et al. explored the possibility of orientation and dis-

show abstract

Section: Calibration Principlementioning

confidence: 99%

Development of neural‐network‐based stereo bionic compound eyes with fiber bundles

Chen

Zhao

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…They influence and help each other in many ways [ 14 , 15 , 16 ]. On the one hand, the mapping and positioning of the classical SLAM system are mostly based on the geometric matching at the pixel level, lacking a priori information to help the sensor locate the objects [ 17 , 18 ]. Data association can be improved from the traditional pixel level to the object level with the help of semantic information, which can help improve the positioning accuracy in complex scenes [ 19 , 20 , 21 ].…”

Section: Introductionmentioning

confidence: 99%

Multi-Objective Location and Mapping Based on Deep Learning and Visual Slam

Sun

Yun

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Simultaneous localization and mapping (SLAM) technology can be used to locate and build maps in unknown environments, but the constructed maps often suffer from poor readability and interactivity, and the primary and secondary information in the map cannot be accurately grasped. For intelligent robots to interact in meaningful ways with their environment, they must understand both the geometric and semantic properties of the scene surrounding them. Our proposed method can not only reduce the absolute positional errors (APE) and improve the positioning performance of the system but also construct the object-oriented dense semantic point cloud map and output point cloud model of each object to reconstruct each object in the indoor scene. In fact, eight categories of objects are used for detection and semantic mapping using coco weights in our experiments, and most objects in the actual scene can be reconstructed in theory. Experiments show that the number of points in the point cloud is significantly reduced. The average positioning error of the eight categories of objects in Technical University of Munich (TUM) datasets is very small. The absolute positional error of the camera is also reduced with the introduction of semantic constraints, and the positioning performance of the system is improved. At the same time, our algorithm can segment the point cloud model of objects in the environment with high accuracy.

show abstract

“…As the main carrier of visual information, images play an important role in perception such as target detection and target recognition, which is currently an important branch within the hottest AI areas. [1][2][3][4] The image often contains a lot of valuable information, and the purpose of segmentation is to simplify or change the representation of the image, which makes the image easier to understand and analyze. [5][6][7][8][9] Because of developments in information and communication technology, the instance segmentation assigns a category or other information to each pixel in an image, and pixels with the same information mean that such pixels have similar or identical visual characteristics.…”

Section: Introductionmentioning

confidence: 99%

“…The issue of instance segmentation has always been an essential part of computer vision with respect to the image understanding. As the main carrier of visual information, images play an important role in perception such as target detection and target recognition, which is currently an important branch within the hottest AI areas 1–4 . The image often contains a lot of valuable information, and the purpose of segmentation is to simplify or change the representation of the image, which makes the image easier to understand and analyze 5–9 .…”

Section: Introductionmentioning

confidence: 99%

Large scale instance segmentation of outdoor environment based on improved YOLACT

Zhao

Tong

Sun

et al. 2022

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Instance segmentation is a challenging task that requires both instance-level and pixel-level prediction and it has a wide range of applications in autonomous driving, video analysis, scene understandingand so on. The currently dominant instance segmentation methods have excellent accuracy, but they are slow, and the processing speed will be even less satisfactory if the input is a large-scale image. In order to improve the efficiency and accuracy of instance segmentation of large-scale images, this article modifies the backbone network based on YOLACT network, adds a multi-information fusion module and provides an improved BiFPN method to achieve multi-scale feature fusion, while adding two branches to the first level detector Reti-naNet to achieve instance segmentation. The network model is tested on Cityscapes dataset and the results of the experiments show that the improved instance segmentation network in this article improves the accuracy while ensuring the speed of segmentation. The optimized network model size was reduced by 17% compared to YOLACT, and the mAP, mAP50, and mAP75 were improved by 18.3%, 32.1%, and 24.6%, respectively.

show abstract

Grasping Pose Detection for Loose Stacked Object Based on Convolutional Neural Network With Multiple Self-Powered Sensors Information

Cited by 39 publications

References 57 publications

Development of neural‐network‐based stereo bionic compound eyes with fiber bundles

Development of neural‐network‐based stereo bionic compound eyes with fiber bundles

Multi-Objective Location and Mapping Based on Deep Learning and Visual Slam

Large scale instance segmentation of outdoor environment based on improved YOLACT

Contact Info

Product

Resources

About