Integrating Wearable and Camera Based Monitoring in the Digital Twin for Safety Assessment in the Industry 4.0 Era

Boldo, Michele; Bombieri, Nicola; Centomo, Stefano; Marchi, Mirco De; Demrozi, Florenc; Pravadelli, Graziano; Quaglia, Davide; Turetta, Cristian

doi:10.1007/978-3-031-19762-8_13

Cited by 10 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a recognized occupational hazard, and companies and organizations have high interest in minimizing such risks and hazards. In this direction, an automatic system [11] consists of a wearable devices and an NVIDIA Jetson board is proposed in an industrial scenario for monitoring activities of personals and the behaviors of robotic systems.…”

Section: Industrial Use-case: Safe Operation Of Machinesmentioning

confidence: 99%

Model-Driven Engineering in Digital Thread Platforms: A Practical Use Case and Future Challenges

Chaudhary

Guevara

John

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The increasing complexity delivered by the heterogeneity of the cyber-physical systems is being addressed and decoded by edge technologies, IoT development, robotics, digital twin engineering, and AI. Nevertheless, tackling the orchestration of these complex ecosystems has become a challenging problem. Specially the inherent entanglement of the different emerging technologies makes it hard to maintain and scale such ecosystems. In this context, the usage of model-driven engineering as a more abstract form of glue-code, replacing the boilerplate fashion, has improved the software development lifecycle, democratising the access to and use of the aforementioned technologies. In this paper, we present a practical use case in the context of Smart Manufacturing, where we use several platforms as providers of a high-level abstraction layer, as well as security measures, allowing a more efficient system construction and interoperability.

show abstract

Section: Industrial Use-case: Safe Operation Of Machinesmentioning

confidence: 99%

Model-Driven Engineering in Digital Thread Platforms: A Practical Use Case and Future Challenges

Chaudhary

Guevara

John

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…To do this, we analyze, characterize, and model the outputs of the selected indoor RTLS targeting human worker localization and tracking in a reference industrial scenario under realistic operational conditions. In this case, a primary one based on ultra-wideband (UWB) radio and a secondary one based on camera vision (CV) serve as inputs to the DT for performing the advanced control computations for run-time optimization, as well as for carrying out emulated/simulated operational performance predictions based on realistic values and models [11], [12]. As depicted in Fig.…”

Section: Introductionmentioning

confidence: 99%

Experimental Evaluation and Modeling of the Accuracy of Real-Time Locating Systems for Industrial Use

Valiollahi,

Rodriguez,

Zhang

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Real-time locating systems (RTLSs) have proven to be a practical and effective solution for monitoring positions/status of humans and other entities in industrial environments, ensuring safe and efficient automated operations, by responding in real-time to unexpected events, such as the proximity of human workers to autonomous mobile robots (AMRs). This work focuses on evaluating and modeling the performance of two complementary RTLSs targeting human localization in both static and mobile conditions within a realistic industrial environment in operational conditions; with the aim of integrating live positioning information into digital twins (DT) for industrial use. Both the primary RTLS examined, an ultra-wide band (UWB) radio-based system; and the secondary RTLS, an optimized camera vision (CV)-based system with three surveillance cameras introduced as a backup RTLS, exhibited a similar median accuracy of 12-13 cm in static conditions, being the one of the UWB system slightly degraded down to 19 cm in presence of human shadowing. In human mobility conditions, the median accuracy values were further debased by 4 and 13 cm for the UWB and CV systems, respectively, indicating a limited real-time fluctuation, sufficiently bounded to guarantee the safety of human workers based on the readings of either of the primary or secondary RTLS systems. Based on the observed performance, a safety protocol for human detection in operational production scenarios was established, considering operational safety margins around humans of 1-2 m, which could be further leveraged by centralized monitoring and control entities such as industrial digital twins. The localization accuracy of the systems is characterized by means of error functions quantifying the distance to ground truth (GT) points through Gamma distribution functions using maximum likelihood estimates (MLEs). The proposed models are practical for implementation in system level simulators or industrial digital twin tools considering akin industrial environments. The different observations presented along the paper are useful for advanced industrial operation planning and optimization considerations.

show abstract

“…Many research works [2][3][4][5] have already exploited different technologies to track human motion and recognize human gestures in case of repetitive and controlled industrial tasks such as assembly [6] and pick-and-place [2]. However, the operators can perform abrupt gestures different from normal movements due to inattention and unexpected circumstances not directly related to the job task.…”

Section: Introductionmentioning

confidence: 99%

Test–Retest Repeatability of Human Gestures in Manipulation Tasks

et al. 2023

View full text Add to dashboard Cite

The importance of performance excellence and operator’s safety is fundamental not only when operators perform repetitive and controlled industrial tasks, but also in case of abrupt gestures due to inattention and unexpected circumstances. Since optical systems work at frequencies that are too low and they are not able to detect gestures as early as possible, combining the use of wearable magneto-inertial measurement units (MIMUs) with the adoption of deep learning techniques can be useful to instruct the machine about human motion. To improve the initial training phase of neural networks for high classification performance, gesture repeatability over time has to be verified. Since the test–retest approach has been poorly applied based on MIMUs signals in a context of human–machine interaction, the aim of this work was to evaluate the repeatability of pick-and-place gestures composed of both normal and abrupt movements. Overall, results demonstrated an excellent test–retest repeatability for normal movements and a fair-to-good test–retest repeatability for abrupt movements. In addition, results suggested important information about the application of deep learning to identify the types of movements: the test showed how to improve reinforcement learning for the identification of onset gestures, whereas the retest allowed for defining the time necessary to retrain the network.

show abstract

Integrating Wearable and Camera Based Monitoring in the Digital Twin for Safety Assessment in the Industry 4.0 Era

Cited by 10 publications

References 13 publications

Model-Driven Engineering in Digital Thread Platforms: A Practical Use Case and Future Challenges

Model-Driven Engineering in Digital Thread Platforms: A Practical Use Case and Future Challenges

Experimental Evaluation and Modeling of the Accuracy of Real-Time Locating Systems for Industrial Use

Test–Retest Repeatability of Human Gestures in Manipulation Tasks

Contact Info

Product

Resources

About