Robotic Harvesting of Fruiting Vegetables: A Simulation Approach in V-REP, ROS and MATLAB

Abstract

In modern agriculture, there is a high demand to move from tedious manual harvesting to a continuously automated operation. This chapter reports on designing a simulation and control platform in V-REP, ROS, and MATLAB for experimenting with sensors and manipulators in robotic harvesting of sweet pepper. The objective was to provide a completely simulated environment for improvement of visual servoing task through easy testing and debugging of control algorithms with zero damage risk to the real robot and to the actual equipment. A simulated workspace, including an exact replica of different robot manipulators, sensing mechanisms, and sweet pepper plant, and fruit system was created in V-REP. Image moment method visual servoing with eye-in-hand configuration was implemented in MATLAB, and was tested on four robotic platforms including Fanuc LR Mate 200iD, NOVABOT, multiple linear actuators, and multiple SCARA arms. Data from simulation experiments were used as inputs of the control algorithm in MATLAB, whose outputs were sent back to the simulated workspace and to the actual robots. ROS was used for exchanging data between the simulated environment and the real workspace via its publishand-subscribe architecture. Results provided a framework for experimenting with different sensing and acting scenarios, and verified the performance functionality of the simulator.

Simulation software and virtual environments for acceleration of agricultural robotics: Features highlights and performance comparison

“…Simulation in general refers to the practice of developing and programming virtual models and objects that together are capable of emulating specific tasks, ideas, or a proposal process in the real-world. Computer simulation and control of agricultural robotics require a multidisciplinary knowledge about different software and hardware to create an integrated virtual experiment environment within which the behavior of various objects (i.e., robot models and sensors) and control tasks (i.e., path planning and visual servoing) can be evaluated [1] . Unlike the industrial applications, an agricultural robot interacts with highly variable dynamic environments which necessitate incorporation of horticulture and agronomy science to successfully accomplish a task.…”

“…Models in V-REP are flexible, portable and scalable, meaning that it is possible to modify them, copy from one project scene to another, or resize them in place. If the project requires building a custom robot model which is not available in the simulator (i.e., the manipulators demonstrated in [1,34,35] ), the setups for links, joints and calculation modules such as inverse kinematics necessitates some practice, however, that is the case in any robot simulation software. …”

“…Moreover, V-REP has several calculation modules that can directly operate on one or several scene objects. Figure 7a shows screenshots of a V-REP scene for simulation of sweet pepper robotic harvesting using Fanuc LRMate 200iD 6-DoF manipulator and artificial plants and fruits models [1] . The visual servo control scheme associated with this project is shown in Figure 8.…”

“…V-REP scene [1] visualized in Rviz using vision sensor publisher (robot control and image processing in MATLAB) b. ROS MoveIt and RViz Source: AdaptiveAgroTech.com Source: sweeper-robot.eu …”

“…For the inverse kinematics (IK) task of the manipulator, as well as path planning algorithm for the robot navigation, we used V-REP built-in calculation modules. The details of IK setup in V-REP is available in [1]. Descriptions of the path planning algorithm and corresponding set-ups in V-REP are also available as a video demonstration on the AdaptiveAgroTech Youtube channel.…”