Efficient and Consumer-Centered Item Detection and Classification with a Multicamera Network at High Ranges

Abstract

In the EU project SHAREWORK, methods are developed that allow humans and robots to collaborate in an industrial environment. One of the major contributions is a framework for task planning coupled with automated item detection and localization. In this work, we present the methods used for detecting and classifying items on the shop floor. Important in the context of SHAREWORK is the user-friendliness of the methodology. Thus, we renounce heavy-learning-based methods in favor of unsupervised segmentation coupled with lenient machine learning methods for classification. Our algorithm is a combination of established methods adjusted for fast and reliable item detection at high ranges of up to eight meters. In this work, we present the full pipeline from calibration, over segmentation to item classification in the industrial context. The pipeline is validated on a shop floor of 40 sqm and with up to nine different items and assemblies, reaching a mean accuracy of 84% at 0.85 Hz.

Enhanced Cognition for Adaptive Human-Robot Collaboration

“…The environment cognition module of Sharework combines high-level and explainable AI (XAI), and artificial cognition algorithms to enable any factory personnel to implement cognition applications without knowledge about machine learning in general. The module consists of multiple sub-modules applicable to different industrial domains and application scenarios: (i) To supervise and allocate tasks on a widespread work space, typical workshop items are localized and classified with unsupervised segmentation and deep learning techniques [12]. Training the neural networks is made lenient, as training data is directly generated from the segmentation data, and may therefore be applied with low effort on the work cell.…”

“…All sub-modules are implemented in a cognition framework [12] that, on the one hand, deals with accelerating computation by applying efficient data structures such that all methodology is available in real-or online-time depending on the usecase. Fast computation is an inherent feature required to allow further computation based on cognition data.…”

“…For input, a camera is mounted concentrically over the rotary disk, facing the main surface area (see Figure 1(b)). Screw detection is based on tree-learning [19] with a small number of features to maintain a certain level of understandability [12]. The perception module is capable of finding silver and black screws within varying lighting conditions.…”

“…Artificial Intelligence (AI) can play a role in endowing controllers and thus robots with the cognitive skills needed to effectively and reliably behave in such scenarios [10], [11]. Towards this goal, we are investigating the use of novel AI technologies in HRC to allow collaborative robots to: (i) perceive the environment [12], correctly interpret occurring events and situations to build and maintain knowledge about production contexts [13]; (ii) reason about their own capabilities/skills and dynamically contextualize possible actions according to the known state of a production scenario [14], [15] and; (iii) autonomously decide how to interact with the environment and other "actors" (i.e., human operators but even other robots if necessary) in order to carry out tasks and dynamically support production needs [16]- [18].…”

Non-Contact Safety for Stationary Robots Through Optical Entry Detection With a Co-Moving 3D-Camera

“…This can be done (partially) autonomously. The procedure is analogous to our already validated method for discretizing large workspaces, which can be found in [14].…”