Loss Switching Fusion with Similarity Search for Video Classification

Wang, Lei; Huynh, Du Q.; Mansour, Moussa Reda

doi:10.1109/icip.2019.8803051

Cited by 20 publications

(7 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A significant number of research works and studies have been conducted to identify and classify human activities by analyzing the motion from video captured through closed-circuit television (CCTV) or other types of camera systems. Machine learning and deep learning models were widely used in many works to identify anomalies in activities [31,32] along with classification [33][34][35]. Since placing surveillance cameras to observe the residents presents data privacy issues/concerns, sensor-based observation has become popular.…”

Section: Machine Learning-based Human Activity Anomaly Detectionmentioning

confidence: 99%

A Comprehensive Review of Machine Learning Approaches for Anomaly Detection in Smart Homes: Experimental Analysis and Future Directions

Rahman,

Gupta,

Bhatt

et al. 2024

Future Internet

View full text Add to dashboard Cite

Detecting anomalies in human activities is increasingly crucial today, particularly in nuclear family settings, where there may not be constant monitoring of individuals’ health, especially the elderly, during critical periods. Early anomaly detection can prevent from attack scenarios and life-threatening situations. This task becomes notably more complex when multiple ambient sensors are deployed in homes with multiple residents, as opposed to single-resident environments. Additionally, the availability of datasets containing anomalies representing the full spectrum of abnormalities is limited. In our experimental study, we employed eight widely used machine learning and two deep learning classifiers to identify anomalies in human activities. We meticulously generated anomalies, considering all conceivable scenarios. Our findings reveal that the Gated Recurrent Unit (GRU) excels in accurately classifying normal and anomalous activities, while the naïve Bayes classifier demonstrates relatively poor performance among the ten classifiers considered. We conducted various experiments to assess the impact of different training–test splitting ratios, along with a five-fold cross-validation technique, on the performance. Notably, the GRU model consistently outperformed all other classifiers under both conditions. Furthermore, we offer insights into the computational costs associated with these classifiers, encompassing training and prediction phases. Extensive ablation experiments conducted in this study underscore that all these classifiers can effectively be deployed for anomaly detection in two-resident homes.

show abstract

Section: Machine Learning-based Human Activity Anomaly Detectionmentioning

confidence: 99%

A Comprehensive Review of Machine Learning Approaches for Anomaly Detection in Smart Homes: Experimental Analysis and Future Directions

Rahman,

Gupta,

Bhatt

et al. 2024

Future Internet

View full text Add to dashboard Cite

show abstract

“…To tackle these issues, typical feature-based methods use RNNs to encode sequences and measure the distance between corresponding features [34]. Other existing methods [43,45,20] either encode each sequence into features that are invariant to temporal variations [1,26] or adopt alignment for temporal correspondence calibration [38]. However, none of these methods is modeling the aleatoric uncertainty.…”

Section: Related Workmentioning

confidence: 99%

Uncertainty-DTW for Time Series and Sequences

Wang¹,

Koniusz²

2022

Preprint

View full text Add to dashboard Cite

Dynamic Time Warping (DTW) is used for matching pairs of sequences and celebrated in applications such as forecasting the evolution of time series, clustering time series or even matching sequence pairs in few-shot action recognition. The transportation plan of DTW contains a set of paths; each path matches frames between two sequences under a varying degree of time warping, to account for varying temporal intra-class dynamics of actions. However, as DTW is the smallest distance among all paths, it may be affected by the feature uncertainty which varies across time steps/frames. Thus, in this paper, we propose to model the so-called aleatoric uncertainty of a differentiable (soft) version of DTW. To this end, we model the heteroscedastic aleatoric uncertainty of each path by the product of likelihoods from Normal distributions, each capturing variance of pair of frames. (The path distance is the sum of base distances between features of pairs of frames of the path.) The Maximum Likelihood Estimation (MLE) applied to a path yields two terms: (i) a sum of Euclidean distances weighted by the variance inverse, and (ii) a sum of log-variance regularization terms. Thus, our uncertainty-DTW is the smallest weighted path distance among all paths, and the regularization term (penalty for the high uncertainty) is the aggregate of log-variances along the path. The distance and the regularization term can be used in various objectives. We showcase forecasting the evolution of time series, estimating the Fréchet mean of time series, and supervised/unsupervised few-shot action recognition of the articulated human 3D body joints.

show abstract

“…It consists of (i) clustering with k-means for a collection of descriptor vectors from the training set to build so-called visual vocabulary, (ii) assigning each descriptor to its nearest cluster center from the visual dictionary, and (iii) aggregating the one-hot assignment vectors via average pooling. Similar models such as Soft Assignment (SA) [62,33] and Localized Soft Assignment (LcSA) [45,38] use the Component Membership Probability (CMP) of GMM to assign each descriptor with some probability to visual words followed by average or non-linear pooling [38,70].…”

Section: Related Workmentioning

confidence: 99%

Hallucinating IDT Descriptors and I3D Optical Flow Features for Action Recognition With CNNs

Wang

Koniusz

Huynh

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

101

View full text Add to dashboard Cite

In this paper, we revive the use of old-fashioned handcrafted video representations for action recognition and put new life into these techniques via a CNN-based hallucination step. Despite of the use of RGB and optical flow frames, the I3D model (amongst others) thrives on combining its output with the Improved Dense Trajectory (IDT) and extracted with its low-level video descriptors encoded via Bag-of-Words (BoW) and Fisher Vectors (FV). Such a fusion of CNNs and handcrafted representations is timeconsuming due to pre-processing, descriptor extraction, encoding and tuning parameters. Thus, we propose an endto-end trainable network with streams which learn the IDTbased BoW/FV representations at the training stage and are simple to integrate with the I3D model. Specifically, each stream takes I3D feature maps ahead of the last 1D conv. layer and learns to 'translate' these maps to BoW/FV representations. Thus, our model can hallucinate and use such synthesized BoW/FV representations at the testing stage. We show that even features of the entire I3D optical flow stream can be hallucinated thus simplifying the pipeline. Our model saves 20-55h of computations and yields stateof-the-art results on four publicly available datasets.

show abstract

Loss Switching Fusion with Similarity Search for Video Classification

Cited by 20 publications

References 35 publications

A Comprehensive Review of Machine Learning Approaches for Anomaly Detection in Smart Homes: Experimental Analysis and Future Directions

A Comprehensive Review of Machine Learning Approaches for Anomaly Detection in Smart Homes: Experimental Analysis and Future Directions

Uncertainty-DTW for Time Series and Sequences

Hallucinating IDT Descriptors and I3D Optical Flow Features for Action Recognition With CNNs

Contact Info

Product

Resources

About