Predicting Emotion and Engagement of Workers in Order Picking Based on Behavior and Pulse Waves Acquired by Wearable Devices

Kajiwara, Yusuke; Shimauchi, Toshihiko; Kimura, Hitoshi

doi:10.3390/s19010165

Cited by 14 publications

(8 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After constructing Poincaré plots, an SVM was used to classify them into four emotions: happiness, sadness, peacefulness, and fear. Kajiwara et al [ 18 ] developed an application for logistics companies that adopt a manual order-picking system, given that emotions and engagement affect work efficiency and human errors. Specifically, they proposed a method for predicting emotions and engagement during work with a high exercise intensity from behavior and pulse wave data acquired by wearable devices.…”

Section: Related Workmentioning

confidence: 99%

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

Okamoto

Murao

2023

Sensors

View full text Add to dashboard Cite

The electromyogram (EMG) is a waveform representation of the action potential generated by muscle cells using electrodes. EMG acquired using surface electrodes is called surface EMG (sEMG), and it is the acquisition of muscle action potentials transmitted by volume conduction from the skin. Surface electrodes require disposable conductive gel or adhesive tape to be attached to the skin, which is costly to run, and the tape is hard on the skin when it is removed. Muscle activity can be evaluated by acquiring muscle potentials and analyzing quantitative, temporal, and frequency factors. It is also possible to evaluate muscle fatigue because the frequency of the EMG becomes lower as the muscle becomes fatigued. Research on human activity recognition from EMG signals has been actively conducted and applied to systems that support arm and hand functions. This paper proposes a method for recognizing the muscle activity state of the arm using pulse wave data (PPG: Photoplethysmography) and a method for estimating EMG using pulse wave data. This paper assumes that the PPG sensor is worn on the user’s wrist to measure the heart rate. The user also attaches an elastic band to the upper arm, and when the user exerts a force on the arm, the muscles of the upper arm contract. The arteries are then constricted, and the pulse wave measured at the wrist becomes weak. From the change in the pulse wave, the muscle activity of the arm can be recognized and the number of action potentials of the muscle can be estimated. From the evaluation experiment with five subjects, three types of muscle activity were recognized with 80+%, and EMG was estimated with approximately 20% error rate.

show abstract

Section: Related Workmentioning

confidence: 99%

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

Okamoto

Murao

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Goshvarpour et al [1] proposed a method for classifying emotional responses on the basis of electrocardiogram and finger pulse activity. Kajiwara et al [9] focused on the fact that many logistics companies adopt a manual order picking system, and that emotions and engagement affect work efficiency and human errors, and proposed a method for predicting emotions and engagement during work with high exercise intensity on the basis of behavior and pulse waves acquired by wearable devices. Lee et al [11] conducted research on improving the speed of emotion recognition by using PPG signals.…”

Section: Studies Using Pulse Datamentioning

confidence: 99%

disp2ppg: Pulse Wave Generation to PPG Sensor using Display

Fujii

Murao

Matsuhisa

2021

2021 International Symposium on Wearable Computers

View full text Add to dashboard Cite

Wearable devices are often used to record the user's biometric information. Among biometric data, pulse data has been used in methods such as heart rate monitoring and emotion estimation. The most common type of pulse sensor is the photoplethysmogram (PPG), which irradiates a green LED on the skin and measures pulse data from changes in the light reflected through the blood vessels. PPG sensors have been implemented in commercially available wearable devices such as smartwatches. When a smartwatch is worn on an artificial body such as a prosthetic hand or a robotic arm, correct data cannot be acquired because there is no blood flow. In this study, we propose a method that enables the PPG sensor to measure arbitrary pulse data using a display. If this method is successful, it will be possible to input pulse data measured at the junction of the live body and the prosthetic hand to the display, and have the smartwatch attached to the prosthetic hand read the same pulse data. In this paper, we focus on the heart rate and report the results of an experiment in which a target heart rate was input and the display was controlled to determine whether the target heart rate could be obtained by a smartwatch. We implemented a display drawing program and conducted the evaluation using five kinds of smartwatches and four kinds of displays. Results showed that the error between the target heart rate and the heart rate acquired by the smartwatch was within ±3 beats per minute in many cases. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools.

show abstract

“…The AET has been demonstrated effectively in the areas of mine worker safety behavior ( Yang et al, 2020 ), driver driving safety ( Muller et al, 2014 ) and among other areas. Kajiwara verified that emotions can influence the productivity and accuracy of workers in a logistics picking system ( Kajiwara et al, 2019 ). Manzoor developed an agent-based computational social agent model to explore how decisions can be affected by regulating the emotions involved, and how emotions are affected by emotion regulation and contagion ( Manzoor and Treur, 2015 ).…”

Section: Introductionmentioning

confidence: 99%

The Impact of Emotional States on Construction Workers’ Recognition Ability of Safety Hazards Based on Social Cognitive Neuroscience

Chong

et al. 2022

Front. Psychol.

View full text Add to dashboard Cite

The construction industry is one of the most dangerous industries with grave situation owing to high accident rate and mortality rate, which accompanied with a series of security management issues that need to be tackled urgently. The unsafe behavior of construction workers is a critical reason for the high incidence of safety accidents. Affective Events Theory suggests that individual emotional states interfere with individual decisions and behaviors, which means the individual emotional states can significantly influence construction workers’ unsafe behaviors. As the complexity of the construction site environment and the lack of attention to construction workers’ emotions by managers, serious potential emotional problems were planted, resulting in the inability of construction workers to effectively recognize safety hazards, thus leading to safety accidents. Consequently, the study designs a behavioral experiment with E-prime software based on social cognitive neuroscience theories. Forty construction workers’ galvanic skin response signals were collected by a wearable device (HKR-11C+), and the galvanic skin response data were classified into different emotional states with support vector machine (SVM) algorithm. Variance analysis, correlation analysis and regression analysis were used to analyze the influence of emotional states on construction workers’ recognition ability of safety hazards. The research findings indicate that the SVM algorithm could effectively classify galvanic skin response data. The construct ion workers’ the reaction time to safety hazards and emotional valence were negatively correlated, while the accuracy of safety hazards recognition and the perception level of safety hazard separately had an inverted “U” type relationship with emotional valence. For construction workers with more than 20 years of working experience, work experience could effectively reduce the influence of emotional fluctuations on the accuracy of safety hazards identification. This study contributes to the application of physiological measurement techniques in construction safety management and shed a light on improving the theoretical system of safety management.

show abstract

Predicting Emotion and Engagement of Workers in Order Picking Based on Behavior and Pulse Waves Acquired by Wearable Devices

Cited by 14 publications

References 41 publications

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG Values

disp2ppg: Pulse Wave Generation to PPG Sensor using Display

The Impact of Emotional States on Construction Workers’ Recognition Ability of Safety Hazards Based on Social Cognitive Neuroscience

Contact Info

Product

Resources

About