Self-adapted optimization-based video magnification for revealing subtle changes

Cai, Enjian; Li, Dongsheng; Li, Hong‐Nan; Xue, Zhilin

doi:10.3233/ica-190614

Cited by 11 publications

(16 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Again for monitoring purposes, even Jaad et al 73 propose a future video frame generation-based system to predict the growth of urban areas during the years starting from historical images. Finally, unlike previous works, Cai et al 74 focus on an optimization-based approach of selfadapted video magnification for subtle color and motion amplification that can be useful, for example, to estimate heartbeat by observing blood flow or other vital signs inside video sequences; once again highlighting how video synthesis might be crucial to solve diverse and complex tasks. or skeletons, produces data used as the visible ground truth for amplitudes processed by the student model.…”

Section: Related Workmentioning

confidence: 99%

Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals

Avola,

Cascio,

Cinque

et al. 2022

Preprint

View full text Add to dashboard Cite

The increasing availability of wireless access points (APs) is leading towards human sensing applications based on Wi-Fi signals as support or alternative tools to the widespread visual sensors, where the signals enable to address well-known vision-related problems such as illumination changes or occlusions. Indeed, using image synthesis techniques to translate radio frequencies to the visible spectrum can become essential to obtain otherwise unavailable visual data. This domain-to-domain translation is feasible because both objects and people affect electromagnetic waves, causing radio and optical frequencies variations. In literature, models capable of inferring radio-to-visual features mappings have gained momentum in the last few years since frequency changes can be observed in the radio domain through the channel state information (CSI) of Wi-Fi APs, enabling signal-based feature extraction, e.g., amplitude. On this account, this paper presents a novel two-branch generative neural network that effectively maps radio data into visual features, following a teacher-student design that exploits a cross-modality supervision strategy. The latter conditions signal-based features in the visual domain to completely replace visual data. Once trained, the proposed method synthesizes human silhouette and skeleton videos using exclusively Wi-Fi signals. The approach is evaluated on publicly available data, where it obtains remarkable results for both silhouette and skeleton videos generation, demonstrating the effectiveness of the proposed cross-modality supervision strategy.

show abstract

Section: Related Workmentioning

confidence: 99%

Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals

Avola,

Cascio,

Cinque

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…-Finally, other artificial intelligence models have been developed, especially for camera-based systems and computer vision. Gaussian models [36], semantic technologies [18], intelligent encoders [5], optimization functions [19] or estimation techniques [20] have been reported very recently. All these approaches have the advantage of showing a very good performance and precision, but they are not flexible Table 1 presents and analyzes works on these scenarios.…”

Section: State Of the Artmentioning

confidence: 99%

“…Now we are evaluating the conditional probability of a user u to be executing a certain user action − → U i considered the observed and recognized atomic actions − → A Eq. (19). The user action − → U i maximizing this conditional probability is the recognized user action − → U * Eq.…”

Section: Modeling Phase: User Activities Recognitionmentioning

confidence: 99%

Recognizing human activities in Industry 4.0 scenarios through an analysis-modeling- recognition algorithm and context labels

Bordel

2021

ICA

View full text Add to dashboard Cite

Activity recognition technologies only present a good performance in controlled conditions, where a limited number of actions are allowed. On the contrary, industrial applications are scenarios with real and uncontrolled conditions where thousands of different activities (such as transporting or manufacturing craft products), with an incredible variability, may be developed. In this context, new and enhanced human activity recognition technologies are needed. Therefore, in this paper, a new activity recognition technology, focused on Industry 4.0 scenarios, is proposed. The proposed mechanism consists of different steps, including a first analysis phase where physical signals are processed using moving averages, filters and signal processing techniques, and an atomic recognition step where Dynamic Time Warping technologies and k-nearest neighbors solutions are integrated; a second phase where activities are modeled using generalized Markov models and context labels are recognized using a multi-layer perceptron; and a third step where activities are recognized using the previously created Markov models and context information, formatted as labels. The proposed solution achieves the best recognition rate of 87% which demonstrates the efficacy of the described method. Compared to the state-of-the-art solutions, an improvement up to 10% is reported.

show abstract

“…The superiority of characterizing unstable and small time‐series motions in the phase‐based method relies on motion magnification techniques (Cai et al., 2020; Wadhwa et al., 2013), specifically in the field of computer vision. Video magnification methods can be categorized by two frameworks, Lagrangian and Eulerian.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, an unsupervised machine learning method needs to be explored. In some respect, the optimization‐based magnification method (Cai et al., 2020) performs better than the learning‐based method (T.‐H. Oh et al., 2018), as it adapts filters directly from optimization problems.…”

Section: Introductionmentioning

confidence: 99%

Visualizing and quantifying small and nonstationary structural motions in video measurement

Cai

Zhang

Quek

2022

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

Rapid advancement in vision recording technologies is increasing the importance and production of video data in a wide range of applications. This paper proposed a novel perspective of multifrequency phase inference for characterizing especially challenging nonstationary and often small motions in optical measurement. The model estimates and adjusts the phase information by the multi-frequency phase retrieval, which is derived from the maximum likelihood formulation with block matching 3D sparsity priors. Estimated phase jumps are removed by a robust solution of the 2D phase unwrapping problem. These considerations are supported by applications of dynamic response identification in structural health monitoring. When compared to state-of-the-art techniques, the proposed method readily yielded high-quality magnifications on real videos, with less noise and better anti-noise performance. The proposed method also demonstrated uniformly high skill in extracting clearer time-domain motion estimation of video components.

show abstract

Self-adapted optimization-based video magnification for revealing subtle changes

Cited by 11 publications

References 68 publications

Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals

Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals

Recognizing human activities in Industry 4.0 scenarios through an analysis-modeling- recognition algorithm and context labels

Visualizing and quantifying small and nonstationary structural motions in video measurement

Contact Info

Product

Resources

About