We present a comprehensive approach to handle perception uncertainty to reduce failure rates in robotic bin-picking. Our focus is on mixed-bins. We identify the main failure modes at various stages of the bin-picking task and present methods to recover from them. If uncertainty in part detection leads to perception failure, then human intervention is invoked. Our approach estimates the confidence in the part match provided by an automated perception system, which is used to detect perception failures. Human intervention is also invoked if uncertainty in estimated part location and orientation leads to a singulation planning failure. We have developed a user interface that enables remote human interventions when necessary. Finally, if uncertainty in part posture in the gripper leads to failure in placing the part with the desired accuracy, sensor-less fine-positioning moves are used to correct the final placement errors. We have developed a fine-positioning planner with a suite of fine-motion strategies that offer different tradeoffs between completion time and postural accuracy at the destination. We report our observations from system characterization experiments with a dual-armed Baxter robot, equipped with a Ensenso three-dimensional camera, to perform bin-picking on mixed-bins.

This paper introduces an approach for decomposing exploration tasks among multiple unmanned surface vehicles (USVs) in congested regions. In order to ensure effective distribution of the workload, the algorithm has to consider the effects of the environmental constraints on the USVs. The performance of a USV is influenced by the surface currents, risk of collision with the civilian traffic, and varying depths due to tides and weather. The team of USVs needs to explore a certain region of the harbor and we need to develop an algorithm to decompose the region of interest into multiple subregions. The algorithm overlays a two-dimensional grid upon a given map to convert it to an occupancy grid, and then proceeds to partition the region of interest among the multiple USVs assigned to explore the region. During partitioning, the rate at which each USV is able to travel varies with the applicable speed limits at the location. The objective is to minimize the time taken for the last USV to finish exploring the assigned area. We use the particle swarm optimization (PSO) method to compute the optimal region partitions. The method is verified by running simulations in different test environments. We also analyze the performance of the developed method in environments where speed restrictions are not known in advance.

Collaborative teams of human operators and mobile ground robots are becoming popular in manufacturing plants to assist humans with a lot of the repetitive tasks such as the packing of related objects into different units, an operation known as kitting. In this paper, we present an ontology to provide a unified representation of all kitting-related tasks, which are decomposed into atomic actions that are either computational involving sensing, perception, planning, and control, or physical involving actuation and manipulation. The ontology is then used in a stochastic integer linear program for optimum partitioning of the atomic tasks between the robots and humans. Preliminary experiments on a single robot, single human case yield promising results where the kitting operations are completed with lower durations and manipulation failure rates using human-robot partnership versus just the human or only the robot. This success is achieved by the robot seeking human assistance for visual perception tasks while performing the other tasks primarily on its own.

Abstract

We present an approach to resolve automated perception failures during bin-picking operations in hybrid assembly cells. Our model exploits complementary strengths of humans and robots. Whereas the robot performs binpicking and proceeds to the subsequent operation like kitting or assembly, a remotely located human assists the robot in critical situations by resolving any automated perception problems encountered during bin-picking. We present the design details of our overall system comprising an automated part recognition system and a remote user interface that allows effective information exchange between the human and the robot that is geared toward solutions that minimize human operator time in resolving the detected perception failures. We use illustrative real robot experiments to show that human-robot information exchange leads to improved bin-picking performance.