Autonomous detection and sorting of litter using deep learning and soft robotic grippers

Abstract

Road infrastructure is one of the most vital assets of any country. Keeping the road infrastructure clean and unpolluted is important for ensuring road safety and reducing environmental risk. However, roadside litter picking is an extremely laborious, expensive, monotonous and hazardous task. Automating the process would save taxpayers money and reduce the risk for road users and the maintenance crew. This work presents LitterBot, an autonomous robotic system capable of detecting, localizing and classifying common roadside litter. We use a learning-based object detection and segmentation algorithm trained on the TACO dataset for identifying and classifying garbage. We develop a robust modular manipulation framework by using soft robotic grippers and a real-time visual-servoing strategy. This enables the manipulator to pick up objects of variable sizes and shapes even in dynamic environments. The robot achieves greater than 80% classified picking and binning success rates for all experiments; which was validated on a wide variety of test litter objects in static single and cluttered configurations and with dynamically moving test objects. Our results showcase how a deep model trained on an online dataset can be deployed in real-world applications with high accuracy by the appropriate design of a control framework around it.

Advanced Flexible Sensing Technologies for Soft Robots

“…Mounting the sensor on a flexible soft gripper enables real‐time adjustment of the clamping force and effectively prevents the object from slipping, which is important for non‐destructive and safe operation in industrial automation processes.Meanwhile, intelligent algorithms have also been widely used in combination with flexible sensors to improve robot autonomy and intellectuality. [ 294 ] An ionic hydrogel‐based strain and tactile sensor was proposed and integrated into a soft gripper (Figure 9b). Based on a deep learning model, it achieved close to 100% high‐precision recognition of 10 different objects, which could be used for sorting food and chemicals in cryogenic storage and cold chain transportation.…”

Sensing in Soft Robotics

“… 173 The sensors can determine the size and shape of the objects and sort and pack them in boxes in a warehouse, supporting extensive automation of the packaging industry. There has been increased interest to application of soft robots with sensors in sorting of trash 174 and recyclable materials. 175 In this case the use of artificial intelligence in recognition of the shape, color, and size of the objects is very useful.…”

Toward precise force control of soft hand grasping: The viscoelastic parameter estimation of pseudo-soft-rigid-model by applying logarithmic decrement method

“…Model-based controllers depend on analytical models for deriving the controller, while model-free controllers circumvent the need to use kinematic and dynamic models [9]. Model-free controllers focus on using learningbased techniques to perform control without geometric models [10][11][12][13][14][15][16]. However, for most applications, model-based controls have been applicable since simplified models provide enough information to improve control performance significantly compared to the model-free baseline.…”