Instruction Generation for Assembly Operations Performed by Humans

Abstract

This paper presents the design of an instruction generation system that can be used to automatically generate instructions for complex assembly operations performed by humans on factory shop floors. Multimodal information−text, graphical annotations, and 3D animations− is used to create easy-to-follow instructions. This thereby reduces learning time and eliminates the possibility of assembly errors. An automated motion planning subsystem computes a collision-free path for each part from its initial posture in a crowded scene onto its final posture in the current subassembly. Visualization of this computed motion results in generation of 3D animations. The system also consists of an automated part identification module that enables the human to identify, and pick, the correct part from a set of similar looking parts. The system's ability to automatically translate assembly plans into instructions enables a significant reduction in the time taken to generate instructions and update them in response to design changes. INTRODUCTIONProduct manufacturing involves complex assembly operations that must be performed by humans and/or robots. Within this setting, it is imperative to define the operational roles of the human and the robot appropriately. Whereas robots are superior to humans at handling repetitive tasks like welding and bolting, humans are better at performing tasks like picking, carrying, and placing a wide range of parts without using special fixtures; humans also have a natural ability to handle several assembly equipment easily. However, humans are prone to committing assembly related mistakes. Human workers usually follow a list of instructions to carry out assembly operations on the shop floor. However, poor instructions lead to assembly errors and increased learning time. Therefore, there is a need for effective, and yet, easy-to-follow assembly instructions for humans. Manual generation of such high quality instructions is a time consuming task when utilizing shared setups and tools. This thereby motivates the need for automated generation of instructions for human workers.In this paper, we present the design of an instruction generation system that can be used to automatically generate instructions

Resolving automated perception system failures in bin-picking tasks using assistance from remote human operators

“…The human-robot collaboration based bin-picking described in this paper is one of the key modules required to achieve hybrid work cells for industrial tasks. In our previous work, we have developed other related modules including sequence planning for complex assemblies [27], instruction generation for human operations [28], ensuring human safety [29], and a framework for replanning to recover from errors [30]. As part of the future work, we plan to integrate these individual modules in order to realize the overall operation of the envisioned hybrid cell.…”

ViTroVo: in vitro assembly search for in vivo adaptive operator guidance

“…With this respect, the work of Xiong et al [70] or Morato et al [42] on Rapid-growing Random Trees (RRT) should also be investigated for ViTroVo. Along these lines, we highlight the promising work of Kaipa et al [25], which relied on RRT to procedurally generate animated instructions of a product composed of 71 parts.…”

Design of Hybrid Cells to Facilitate Safe and Efficient Human–Robot Collaboration During Assembly Operations

“…However, poor instructions lead to the human committing mistakes related to the assembly. We address this issue by utilizing an instruction generation system developed in our previous work [39] that creates effective and easy-to-follow assembly instructions for humans. A linearly ordered assembly sequence (result of Sec.…”

“…To address this problem, we utilize a part identification tool developed in Ref. [39] that automatically detects such similarities and present the parts in a manner that enables the human worker to select the correct part. For this purpose, a similarity metric between two parts was constructed based on attributes like part volume, surface area, types of surfaces, and curvature [40,41].…”