Using EfficientNet-B7 (CNN), Variational Auto Encoder (VAE) and Siamese Twins’ Networks to Evaluate Human Exercises as Super Objects in a TSSCI Images

Abstract

In this article, we introduce a new approach to human movement by defining the movement as a static super object represented by a single two-dimensional image. The described method is applicable in remote healthcare applications, such as physiotherapeutic exercises. It allows researchers to label and describe the entire exercise as a standalone object, isolated from the reference video. This approach allows us to perform various tasks, including detecting similar movements in a video, measuring and comparing movements, generating new similar movements, and defining choreography by controlling specific parameters in the human body skeleton. As a result of the presented approach, we can eliminate the need to label images manually, disregard the problem of finding the start and the end of an exercise, overcome synchronization issues between movements, and perform any deep learning network-based operation that processes super objects in images in general. As part of this article, we will demonstrate two application use cases: one illustrates how to verify and score a fitness exercise. In contrast, the other illustrates how to generate similar movements in the human skeleton space by addressing the challenge of supplying sufficient training data for deep learning applications (DL). A variational auto encoder (VAE) simulator and an EfficientNet-B7 classifier architecture embedded within a Siamese twin neural network are presented in this paper in order to demonstrate the two use cases. These use cases demonstrate the versatility of our innovative concept in measuring, categorizing, inferring human behavior, and generating gestures for other researchers.

Temporal context-aware motion-saliency detection

“…[9][10][11] However, there are applications in which the temporal context of salient motion is as essential as the motion itself, examples including video synopsis, 12,13 object tracking, 14 and action analysis. 15 In these applications, the temporal context of motion is an important semantic feature. Consequently, in this study, we proposed a new type of temporal context-aware motion saliency (TCAMS), including both motion and temporal-contextual information.…”

Classification of Fashion Models’ Walking Styles Using Publicly Available Data, Pose Detection Technology, and Multivariate Analysis: From Past to Current Trendy Walking Styles

“…For example, Hobara et al analyzed publicly available internet broadcasts to determine the running characteristics of able-bodied and amputee sprinters in actual 100 m races at world championships [ 7 ]. Recently, advanced pose detection technologies have been used in various industries, such as sports [ 10 ], healthcare [ 11 ], and entertainment [ 12 ]. Therefore, we concluded that the combination of these cutting-edge technologies, namely, the analysis of publicly available fashion show video resources using pose detection technology and multivariate analysis techniques, can clarify the characteristics of the sophisticated walking styles of the world’s leading fashion models that are modified over time based on the demands of the industry.…”