Tell Me, What Do You See?—Interpretable Classification of Wiring Harness Branches with Deep Neural Networks

Abstract

In the context of the robotisation of industrial operations related to manipulating deformable linear objects, there is a need for sophisticated machine vision systems, which could classify the wiring harness branches and provide information on where to put them in the assembly process. However, industrial applications require the interpretability of the machine learning system predictions, as the user wants to know the underlying reason for the decision made by the system. We propose several different neural network architectures that are tested on our novel dataset to address this issue. We conducted various experiments to assess the influence of modality, data fusion type, and the impact of data augmentation and pretraining. The outcome of the network is evaluated in terms of the performance and is also equipped with saliency maps, which allow the user to gain in-depth insight into the classifier’s operation, including a way of explaining the responses of the deep neural network and making system predictions interpretable by humans.

Wire Harness Assembly Process Supported by Collaborative Robots: Literature Review and Call for R&D

“…Nguyen and Yoon [19], in "A Novel Vision-based Method for 3D Profile Extraction of Wire Harness in Robotized Assembly Process", developed a method to determine a wire harness profile using a computer vision system to pick and collocate a wire harness in the correct position. Similar research was conducted by Kicki et al [20] in "Tell Me, What Do You See?-Interpretable Classification of Wiring Harness Branches with Deep Neural Networks" to identify different types of wire harnesses with a computer vision system and neural networks to guide a cobot in the wire harness assembly process. They designed algorithms to classify the type of wire harness.…”

“…The use of computer vision systems in the wire harness assembly process can be an excellent opportunity to integrate a cobot into a collaborative assembly process. Nguyen and Yoon [19], Kicki et al [20], and Yumbla et al [21] used computer vision systems in their research works. Nguyen and Yoon [19] and Kicki et al [20] used computer vision to recognise the shape of a wire harness.…”

“…Nguyen and Yoon [19], Kicki et al [20], and Yumbla et al [21] used computer vision systems in their research works. Nguyen and Yoon [19] and Kicki et al [20] used computer vision to recognise the shape of a wire harness. However, Nguyen and Yoon [19] used the recognition so a cobot could work on placing a wire harness on its assembly board.…”

“…The papers from Nguyen and Yoon [19], Kicki et al [20], Yumbla et al [21], and Karaulova et al [27] could be used to implement computer vision systems into different tasks from the wire harnesses assembly process, such as the cable tie collocation or spot-taping.…”

Robotic Manipulation of Elongated and Elastic Objects

“…People encounter and skillfully manipulate DLOs, such as cables, ropes, threads, strings, and hoses. It would be beneficial to give robots similar skills to enable them to perform cable routing [1], [2], knot tying [3], rope untangling [4], belt-drive unit assembly [5], wiring harness assembly in the automotive sector [6], [7], threading the lace through the narrow hole [8], or surgical suturing [9]. A typical approach to manipulating DLOs is to use their models to plan the motion and control commands necessary to rearrange it to the desired state.…”