Digitisation of a moving assembly operation using multiple depth imaging sensors

Abstract

Several manufacturing operations continue to be manual even in today's highly automated industry because the complexity of such operations makes them heavily reliant on human skills, intellect and experience. This work aims to aid the automation of one such operation, the wheel loading operation on the trim and final moving assembly line in automotive production. It proposes a new method that uses multiple low-cost depth imaging sensors, commonly used in gaming, to acquire and digitise key shopfloor data associated with the operation, such as motion characteristics of the vehicle body on the moving conveyor line and the angular positions of alignment features of the parts to be assembled, in order to inform an intelligent automation solution. Experiments are conducted to test the performance of the proposed method across various assembly conditions, and the results are validated against an industry standard method using laser tracking. Some disadvantages of the method are discussed, and suggestions for improvements are suggested. The proposed method has the potential to be adopted to enable the automation of a wide range of moving assembly operations in multiple sectors of the manufacturing industry.

Decision support system enabled by depth imaging sensor data for intelligent automation of moving assemblies

“…The maximum error margin of 2.78 mm in motion tracking and 1.46 mm in alignment feature position measurement are well within the industry standard tolerance of 4 mm (for the wheel and wheel hub type chosen). 23 The proposed DSS only takes this input data from the Kinect sensors and based on pre-defined expert rules, produces logical decisions in real-time for the different phases of the operation. Therefore, the proposed DSS is applicable in real-life vehicle assembly, but rigorous experimentation to establish validity on the actual assembly floor is required before industry adoption.…”

“…Figure 6 shows the operation setup and Figure 7 shows the outcome of alignment feature recognition and measurement of wheel hub motion characteristics for a sinusoidal up-down simulated deviation. The complete details of the experiment are reported in Prabhu et al 23 and summary of data used by the DSS for the wheel loading operation is summarised in Table 2.…”

“…It is evident from the above review that there is a lack of DSSs reported in the literature for automating moving assembly operations that are based on the human cognitive problem solving process. This first-of-its-kind work attempts to develop a DSS based on Rasmussen’s 22 decision ladder, a popular architecture that explains human problem solving process in industrial settings and on the work done by Prabhu et al, 23 to provide real-time data of moving assembly components on the shopfloor using gaming interface sensors. The use of the decision ladder provides the proposed DSS with the key characteristics of flexibility and adaptability that is associated with human decision-making in a dynamic industrial environment.…”

Utilising low cost RGB-D cameras to track the real time progress of a manual assembly sequence

“…In terms of 2D techniques the work of Matas and Obdržálek (2004) appears to be robust as it is able to deal with occlusions, in addition the work of Gupta et al (2014) is insightful as it provides approaches for object contour recognition and data sets creation. The work of Prabhu et al (2015) outlines a method to enable the tracking of the progress of a manual wheel assembly process.…”

Seamless Human–Robot Collaborative Assembly Using Artificial Intelligence and Wearable Devices

“…Identification of tasks being executed by human operators and validation of the assembly progress. Recent research has mainly focused on tracking the position and posture [19] and even fewer approaches perform intention prediction [20,21]. This work advances further by exploiting human and environment tracking to identify the workflow execution status which allows for the triggering of task-optimized support strategies.…”