Classifier Level Fusion of Accelerometer and sEMG Signals for Automatic Fitness Activity Diarization

Biagetti, Giorgio; Crippa, Paolo; Falaschetti, Laura; Turchetti, Claudio

doi:10.3390/s18092850

Cited by 16 publications

(10 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is an important factor to take into consideration when choosing a fusion strategy (Castanedo, 2013 ), especially for embedded systems. Therefore, we follow a late fusion approach with a classifier-level fusion, which has been shown to perform better than feature-level fusion for classification tasks (Guo et al, 2014 ; Peng et al, 2016 ; Biagetti et al, 2018 ). It is close to score-level fusion by combining the penultimate layers of the base (unimodal) classifiers in a meta-level (multimodal) classifier that uses the natural complementarity of different modalities to improve the overall classification accuracy.…”

Section: Discussionmentioning

confidence: 99%

Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing

et al. 2020

View full text Add to dashboard Cite

Hand gestures are a form of non-verbal communication used by individuals in conjunction with speech to communicate. Nowadays, with the increasing use of technology, hand-gesture recognition is considered to be an important aspect of Human-Machine Interaction (HMI), allowing the machine to capture and interpret the user's intent and to respond accordingly. The ability to discriminate between human gestures can help in several applications, such as assisted living, healthcare, neuro-rehabilitation, and sports. Recently, multi-sensor data fusion mechanisms have been investigated to improve discrimination accuracy. In this paper, we present a sensor fusion framework that integrates complementary systems: the electromyography (EMG) signal from muscles and visual information. This multi-sensor approach, while improving accuracy and robustness, introduces the disadvantage of high computational cost, which grows exponentially with the number of sensors and the number of measurements. Furthermore, this huge amount of data to process can affect the classification latency which can be crucial in real-case scenarios, such as prosthetic control. Neuromorphic technologies can be deployed to overcome these limitations since they allow real-time processing in parallel at low power consumption. In this paper, we present a fully neuromorphic sensor fusion approach for hand-gesture recognition comprised of an event-based vision sensor and three different neuromorphic processors. In particular, we used the event-based camera, called DVS, and two neuromorphic platforms, Loihi and ODIN + MorphIC. The EMG signals were recorded using traditional electrodes and then converted into spikes to be fed into the chips. We collected a dataset of five gestures from sign language where visual and electromyography signals are synchronized. We compared a fully neuromorphic approach to a baseline implemented using traditional machine learning approaches on a portable GPU system. According to the chip's constraints, we designed specific spiking neural networks (SNNs) for sensor fusion that showed classification accuracy comparable to the software baseline. These neuromorphic alternatives have increased inference time, between 20 and 40%, Ceolini et al. Sensor Fusion Neuromorphic Benchmark with respect to the GPU system but have a significantly smaller energy-delay product (EDP) which makes them between 30× and 600× more efficient. The proposed work represents a new benchmark that moves neuromorphic computing toward a real-world scenario.

show abstract

Section: Discussionmentioning

confidence: 99%

Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing

et al. 2020

View full text Add to dashboard Cite

show abstract

“…We find in the literature two main different strategies for multimodal fusion [50,107]: (1) score-level fusion where data modalities are learned by distinct models then their predictions are fused with another model that provides a final decision, and (2) data-level fusion where modalities are concatenated then learned by a unique model. Our approach can be classified as a classifier-level fusion which is closer to score-level fusion and usually produces better results than feature-level or data-level fusion for classification tasks [108][109][110]. However, it is worth trying to learn the concatenated modalities with one SOM having as much neurons as the two uni-modal SOMs, for a fair comparison.…”

Section: Som Early Data Fusionmentioning

confidence: 99%

Brain-Inspired Self-Organization with Cellular Neuromorphic Computing for Multimodal Unsupervised Learning

2020

View full text Add to dashboard Cite

Cortical plasticity is one of the main features that enable our ability to learn and adapt in our environment. Indeed, the cerebral cortex self-organizes itself through structural and synaptic plasticity mechanisms that are very likely at the basis of an extremely interesting characteristic of the human brain development: the multimodal association. In spite of the diversity of the sensory modalities, like sight, sound and touch, the brain arrives at the same concepts (convergence). Moreover, biological observations show that one modality can activate the internal representation of another modality when both are correlated (divergence). In this work, we propose the Reentrant Self-Organizing Map (ReSOM), a brain-inspired neural system based on the reentry theory using Self-Organizing Maps and Hebbian-like learning. We propose and compare different computational methods for unsupervised learning and inference, then quantify the gain of the ReSOM in a multimodal classification task. The divergence mechanism is used to label one modality based on the other, while the convergence mechanism is used to improve the overall accuracy of the system. We perform our experiments on a constructed written/spoken digits database and a Dynamic Vision Sensor (DVS)/EletroMyoGraphy (EMG) hand gestures database. The proposed model is implemented on a cellular neuromorphic architecture that enables distributed computing with local connectivity. We show the gain of the so-called hardware plasticity induced by the ReSOM, where the system’s topology is not fixed by the user but learned along the system’s experience through self-organization.

show abstract

“…Giorgio Biagetti, Paolo Crippa, Laura Falaschetti and Claudio Turchetti [11], in this research installing a device called surface electromyographic (sEMG) and an accelerometer on the arm of a person doing fitness then sEMG data and accelerometer sent with Wireless Sensor Network devices, then classified according to level Fusion, consequently, the accuracy of the results of the SEMG data transmission and accelerometer is 82.6% of all types of styles during fitness activities, pada research ini data yang dihasilkan adalah sEMG data and accelerometer, in this research the data produced is sEMG data and accelerometer, in this research can be developed using Kalman Filtering and Learning Algorithms. The position of the sensor is like this research, which is on the wrist that takes the movement of the arteries.…”

Section: Related Studiesmentioning

confidence: 99%

ZigBee Radio Frequency (RF) Performance on Raspberry Pi 3 for Internet of Things (IoT) based Blood Pressure Sensors Monitoring

Adi¹,

Kitagawa²

2019

IJACSA

View full text Add to dashboard Cite

Wireless Sensor Network has grown rapidly, e.g. using the Zigbee RF module and combined with the Raspberry Pi 3, a reason at this research is building a Wireless Sensor Network (WSN). this research discusses how sensor nodes work well and how Quality of Service (QoS) from the Sensor node being analyzed and the role of Raspberry Pi 3 as an internet gateway will sending a blood pressure data to the database and displayed in real-time on the internet, from this research it is expected that patients can check the blood pressure from home and don't need to the Hospital even data can be quickly and accurately received by Hospital Officers, doctors, and medical personnel. the purpose of this research is make a prototype to providing a blood pressure (mmHg) real-time data from systolic and diastolic data patient's that determine patients suffering from symptoms of certain diseases, i.e, anemia, symptoms of hypertension and even more chronic diseases. this research discusses how sensor nodes work well and how Quality of Service (QoS) from the Sensor node being analyzed and the role of Raspberry Pi 3 as an internet gateway will sending a blood pressure data to the database and displayed in real-time on the internet. Furthermore, Zigbee has the task of sending Blood pressure (mmHg) data in real-time to the database and then sent to the internet from Zigbee end-device communication to ZigBee coordinator. Zigbee communication at a distance of 5 meters, RSSI simulations show a value of -29 dBm and the experiment shows a value of -40 dBm, at a distance of 100 m, RSSI shows a value of -55 dBm (simulation) and -86 dBm (experiment).

show abstract

Classifier Level Fusion of Accelerometer and sEMG Signals for Automatic Fitness Activity Diarization

Cited by 16 publications

References 37 publications

Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing

Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing

Brain-Inspired Self-Organization with Cellular Neuromorphic Computing for Multimodal Unsupervised Learning

ZigBee Radio Frequency (RF) Performance on Raspberry Pi 3 for Internet of Things (IoT) based Blood Pressure Sensors Monitoring

Contact Info

Product

Resources

About