Feature-Based Information Retrieval of Multimodal Biosignals with a Self-Similarity Matrix: Focus on Automatic Segmentation

Abstract

Biosignal-based technology has been increasingly available in our daily life, being a critical information source. Wearable biosensors have been widely applied in, among others, biometrics, sports, health care, rehabilitation assistance, and edutainment. Continuous data collection from biodevices provides a valuable volume of information, which needs to be curated and prepared before serving machine learning applications. One of the universal preparation steps is data segmentation and labelling/annotation. This work proposes a practical and manageable way to automatically segment and label single-channel or multimodal biosignal data using a self-similarity matrix (SSM) computed with signals’ feature-based representation. Applied to public biosignal datasets and a benchmark for change point detection, the proposed approach delivered lucid visual support in interpreting the biosignals with the SSM while performing accurate automatic segmentation of biosignals with the help of the novelty function and associating the segments grounded on their similarity measures with the similarity profiles. The proposed method performed superior to other algorithms in most cases of a series of automatic biosignal segmentation tasks; of equal appeal is that it provides an intuitive visualization for information retrieval of multimodal biosignals.

Latent space unsupervised semantic segmentation

“…Additionally, future research should explore alternative approaches to segmentation based on the LSMP. A promising starting point would be to investigate the application of techniques used in Rodrigues et al. (2022) , such as novelty search, periodic search, and similarity profile, to the LSMP instead of the feature-based self-similarity matrix.…”

“…As matrix profiles are a core aspect of LS-USS, they will be presented in more detail in Section 3.1. In Rodrigues et al (2022), the authors propose a novel segmentation method that uses a feature-based self-similarity matrix (SSM) to measure the pairwise distance between subsequences of a time series. Unlike previous works, such as Yeh et al (2016) and our own work, where the SSM is based on the raw time series or a latent representation of it, this method selects features from the Time Series Feature Extraction Library (TSFEL) Barandas et al (2020) to construct the SSM.…”

“…As matrix profiles are a core aspect of LS-USS, they will be presented in more detail in Section 3.1 . In Rodrigues et al. (2022) , the authors propose a novel segmentation method that uses a feature-based self-similarity matrix (SSM) to measure the pairwise distance between subsequences of a time series.…”

Heart Rate Variability Based Estimation of Maximal Oxygen Uptake in Athletes Using Supervised Regression Models

“…A single iteration process of greedy forward and backward is represented in the Figure 1 . Rodrigues et al [ 38 ] has proposed a automatic segmentation and labeling approach of multimodal biosignal using Self-Similarity Matrix computed with the signals’ feature-based representation. Hui Liu et al [ 39 ], in his work, used features derived from biosignals for Human Activity Recognition using greedy feature selection.…”

MS2OD: outlier detection using minimum spanning tree and medoid selection

“…In many previous works, researchers mixed the term outlier or anomaly [7,8], sometimes even surprise [9,10], discord [11,12], unusual [13], and novelty [14]. In [15], the authors provided a criterion to distinguish outliers and anomalies for time series: Outlier is mostly used when detecting unwanted data, whereas anomaly has been used when detecting events of interest.…”