Path Planning Aware of Robot’s Center of Mass for Steep Slope Vineyards

Santos, Leonardo D.; Santos, Filipe Neves dos; Mendes, Jorge; Costa, Pedro; Lima, José; Reis, Ricardo; Shinde, Pranjali

doi:10.1017/s0263574719000961

Cited by 41 publications

(25 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to achieve a high-performance detector, that visually recognizes trunks in real-time, a training procedure over the CNNs was performed. To do so, a training dataset was created, using our robotic platform AgRob V16 [48], represented in Fig. 3.…”

Section: Methodsmentioning

confidence: 99%

Visual Trunk Detection Using Transfer Learning and a Deep Learning-Based Coprocessor

et al. 2020

View full text Add to dashboard Cite

Agricultural robotics is nowadays a complex, challenging, and exciting research topic. Some agricultural environments present harsh conditions to robotics operability. In the case of steep slope vineyards, there are several challenges: terrain irregularities, characteristics of illumination, and inaccuracy/unavailability of signals emitted by the Global Navigation Satellite System (GNSS). Under these conditions, robotics navigation becomes a challenging task. To perform these tasks safely and accurately, the extraction of reliable features or landmarks from the surrounding environment is crucial. This work intends to solve this issue, performing accurate, cheap, and fast landmark extraction in steep slope vineyard context. To do so, we used a single camera and an Edge Tensor Processing Unit (TPU) provided by Google's USB Accelerator as a small, high-performance, and low power unit suitable for image classification, object detection, and semantic segmentation. The proposed approach performs object detection using Deep Learning (DL)-based Neural Network (NN) models on this device to detect vine trunks. To train the models, Transfer Learning (TL) is used on several pre-trained versions of MobileNet V1 and MobileNet V2. A benchmark between the two models and the different pre-trained versions is performed. The models are pre-trained in a built in-house dataset, that is publicly available containing 336 different images with approximately 1,600 annotated vine trunks. There are considered two vineyards, one using camera images with the conventional infrared filter and others with an infrablue filter. Results show that this configuration allows a fast vine trunk detection, with MobileNet V2 being the most accurate retrained detector, achieving an overall Average Precision of 52.98%. We briefly compare the proposed approach with the state-of-the-art Tiny YOLO-V3 running on Jetson TX2, showing the outperformance of the adopted system in this work. Additionally, it is also shown that the proposed detectors are suitable for the Localization and Mapping problems.INDEX TERMS Deep learning, transfer learning, convolutional neural networks, tensor processing unit.

show abstract

Section: Methodsmentioning

confidence: 99%

Visual Trunk Detection Using Transfer Learning and a Deep Learning-Based Coprocessor

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In order to acquire images in real vineyard scenarios, we used our robotic platform AgRob V16 [49], which is represented in Figure 4. This robot contains a frontal stereo RGB camera and a frontal thermal camera.…”

Section: Data Acquisitionmentioning

confidence: 99%

Bringing Semantics to the Vineyard: An Approach on Deep Learning-Based Vine Trunk Detection

et al. 2021

View full text Add to dashboard Cite

The development of robotic solutions in unstructured environments brings several challenges, mainly in developing safe and reliable navigation solutions. Agricultural environments are particularly unstructured and, therefore, challenging to the implementation of robotics. An example of this is the mountain vineyards, built-in steep slope hills, which are characterized by satellite signal blockage, terrain irregularities, harsh ground inclinations, and others. All of these factors impose the implementation of precise and reliable navigation algorithms, so that robots can operate safely. This work proposes the detection of semantic natural landmarks that are to be used in Simultaneous Localization and Mapping algorithms. Thus, Deep Learning models were trained and deployed to detect vine trunks. As significant contributions, we made available a novel vine trunk dataset, called VineSet, which was constituted by more than 9000 images and respective annotations for each trunk. VineSet was used to train state-of-the-art Single Shot Multibox Detector models. Additionally, we deployed these models in an Edge-AI fashion and achieve high frame rate execution. Finally, an assisted annotation tool was proposed to make the process of dataset building easier and improve models incrementally. The experiments show that our trained models can detect trunks with an Average Precision up to 84.16% and our assisted annotation tool facilitates the annotation process, even in other areas of agriculture, such as orchards and forests. Additional experiments were performed, where the impact of the amount of training data and the comparison between using Transfer Learning and training from scratch were evaluated. In these cases, some theoretical assumptions were verified.

show abstract

“…AgRob Grid Map to Topologic is a framework developed to deal with big dimensions maps in autonomous robot navigation. As mentioned in our previous work [12], path planning in terrains with large dimensions is complex in terms of memory. This approach automatically divides an occupation grid map into smaller zones and finds the different possible connections between those places.…”

Section: Agrobpp-bridge-agrob Grid Map To Topologicmentioning

confidence: 99%

“…These scenarios present several challenges to autonomous robot navigation: Global Navigation Satellite Systems (GNSS) gets frequently blocked by the hills providing unstable positioning estimations, and the irregular sloppy terrain presents a challenge for path planning algorithms. To tackle some of these challenges, we proposed VineSlam [11] and Agricultural Robotics Path Planning framework (AgRobPP) [12]. An identified limitation in AgRobPP is its memory efficiency.…”

Section: Introductionmentioning

confidence: 99%

Occupancy Grid and Topological Maps Extraction from Satellite Images for Path Planning in Agricultural Robots

et al. 2020

Self Cite

View full text Add to dashboard Cite

Robotics will significantly impact large sectors of the economy with relatively low productivity, such as Agri-Food production. Deploying agricultural robots on the farm is still a challenging task. When it comes to localising the robot, there is a need for a preliminary map, which is obtained from a first robot visit to the farm. Mapping is a semi-autonomous task that requires a human operator to drive the robot throughout the environment using a control pad. Visual and geometric features are used by Simultaneous Localisation and Mapping (SLAM) Algorithms to model and recognise places, and track the robot’s motion. In agricultural fields, this represents a time-consuming operation. This work proposes a novel solution—called AgRoBPP-bridge—to autonomously extract Occupancy Grid and Topological maps from satellites images. These preliminary maps are used by the robot in its first visit, reducing the need of human intervention and making the path planning algorithms more efficient. AgRoBPP-bridge consists of two stages: vineyards row detection and topological map extraction. For vineyards row detection, we explored two approaches, one that is based on conventional machine learning technique, by considering Support Vector Machine with Local Binary Pattern-based features, and another one found in deep learning techniques (ResNET and DenseNET). From the vineyards row detection, we extracted an occupation grid map and, by considering advanced image processing techniques and Voronoi diagrams concept, we obtained a topological map. Our results demonstrated an overall accuracy higher than 85% for detecting vineyards and free paths for robot navigation. The Support Vector Machine (SVM)-based approach demonstrated the best performance in terms of precision and computational resources consumption. AgRoBPP-bridge shows to be a relevant contribution to simplify the deployment of robots in agriculture.

show abstract

Path Planning Aware of Robot’s Center of Mass for Steep Slope Vineyards

Cited by 41 publications

References 21 publications

Visual Trunk Detection Using Transfer Learning and a Deep Learning-Based Coprocessor

Visual Trunk Detection Using Transfer Learning and a Deep Learning-Based Coprocessor

Bringing Semantics to the Vineyard: An Approach on Deep Learning-Based Vine Trunk Detection

Occupancy Grid and Topological Maps Extraction from Satellite Images for Path Planning in Agricultural Robots

Contact Info

Product

Resources

About