Motion Images: An Effective Representation of Motion Capture Data for Similarity Search

Eliáš, Petr; Sedmidubský, Jan; Zezula, Pavel

doi:10.1007/978-3-319-25087-8_24

Cited by 13 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the dataset HDM05-14 with 14 motion classes, our proposed scheme outperformed the other existing state-of-the-art approaches [1,62], and it achieved 98.6% accuracy. The addition of keyframes with Normalized Trajectories (NT) substantially contributed to the improved performance of our proposed framework, as is obvious in Table 3.…”

Section: Evaluation On Hdm05-14mentioning

confidence: 82%

Keys for Action: An Efficient Keyframe-Based Approach for 3D Action Recognition Using a Deep Neural Network

Yasin

Hussain

Weber

2020

Sensors

View full text Add to dashboard Cite

In this paper, we propose a novel and efficient framework for 3D action recognition using a deep learning architecture. First, we develop a 3D normalized pose space that consists of only 3D normalized poses, which are generated by discarding translation and orientation information. From these poses, we extract joint features and employ them further in a Deep Neural Network (DNN) in order to learn the action model. The architecture of our DNN consists of two hidden layers with the sigmoid activation function and an output layer with the softmax function. Furthermore, we propose a keyframe extraction methodology through which, from a motion sequence of 3D frames, we efficiently extract the keyframes that contribute substantially to the performance of the action. In this way, we eliminate redundant frames and reduce the length of the motion. More precisely, we ultimately summarize the motion sequence, while preserving the original motion semantics. We only consider the remaining essential informative frames in the process of action recognition, and the proposed pipeline is sufficiently fast and robust as a result. Finally, we evaluate our proposed framework intensively on publicly available benchmark Motion Capture (MoCap) datasets, namely HDM05 and CMU. From our experiments, we reveal that our proposed scheme significantly outperforms other state-of-the-art approaches.2 of 24 inertial sensor-and accelerometer-based systems [7,8]; and hybrid systems [9]. As another domain, 3D motions are reconstructed from different sources, e.g., reconstruction from video or image data [10][11][12][13][14] and reconstruction from accelerometer data [15,16]. In short, motions captured or generated by different sources of means are abundant and contain a lot of hidden knowledge and information that may be exploited further in different types of applications, such as those mentioned above.There exists a variety of action classification methods that are based on different input data; the input may be in the form of simple RGB videos or spatiotemporal joint trajectories acquired by means of a sensor system (e.g., mechanical, magnetic, optic, inertial, non-optic wearable sensors and RGB-D sensors, such as Kinect) or an estimated 3D skeleton from image data or a hybrid system. Although a lot of research has been done in the domain of action recognition, there still exist numerous challenges, such as viewpoint variations, different human body sizes and appearances, and illumination factors that may influence the efficiency and performance of existing algorithms [17]. Moreover, each performing actor has his or her own way and style of executing the same action. The actions may also have many variations in terms of speed and length. In addition to these inter-class variations, extensive intra-class variations make the task more difficult. For example, it is not an easy task to differentiate between jogging and running, walkForward and walkBackward, sitDownChair and sitDownFloor, standUpSitChair and standUpSitFloor, rotateArmBackward and rota...

show abstract

Section: Evaluation On Hdm05-14mentioning

confidence: 82%

Keys for Action: An Efficient Keyframe-Based Approach for 3D Action Recognition Using a Deep Neural Network

Yasin

Hussain

Weber

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Additionally, attempts have been made to classify spatiotemporal joint trajectories by condensing them into an image and using a CNN for classification [20]- [24]. Graphbased architectures that encode spatiotemporal connections have also been developed, with some of them using predefined graph structures based on the natural connectivity of joints [25].…”

Section: Related Work a Segmented Hand Gesture Classificationmentioning

confidence: 99%

TY-Net: Transforming YOLO for Hand Gesture Recognition

Doždor,

Kalafatić,

Ban

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The general idea of this type of approach is to structure the data in order to give them the expected form (a sequence of images) and thus classify these images using standard computer vision methods. Such motion formalisms to represent skeletal sequences by compact image-like inputs were first proposed by Elias et al [25] and extended by Sedmidubsky et al [26] where a special insistence has been given to features representation and data normalization to improve instance indexing.…”

Section: Convolutional Neural Network (Cnn)mentioning

confidence: 99%

“…Since a sequence is represented with a 3-dimensional (300, 14, 2)-shaped tensor, we can easily apply the TSSI normalization [76] on the input and transform the original sequences into a multi-channel redundant image of shape (300, 25,2). A few sequences of pedestrian actions in the TSSI-format are plotted with their ground truth intentions in Figure 7 for illustration.…”

Section: Cartesian Coordinates Features Branchmentioning

confidence: 99%

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

et al. 2020

View full text Add to dashboard Cite

Understanding the behaviors and intentions of humans is still one of the main challenges for vehicle autonomy. More specifically, inferring the intentions and actions of vulnerable actors, namely pedestrians, in complex situations such as urban traffic scenes remains a difficult task and a blocking point towards more automated vehicles. Answering the question “Is the pedestrian going to cross?” is a good starting point in order to advance in the quest to the fifth level of autonomous driving. In this paper, we address the problem of real-time discrete intention prediction of pedestrians in urban traffic environments by linking the dynamics of a pedestrian’s skeleton to an intention. Hence, we propose SPI-Net (Skeleton-based Pedestrian Intention network): a representation-focused multi-branch network combining features from 2D pedestrian body poses for the prediction of pedestrians’ discrete intentions. Experimental results show that SPI-Net achieved 94.4% accuracy in pedestrian crossing prediction on the JAAD data set while being efficient for real-time scenarios since SPI-Net can reach around one inference every 0.25 ms on one GPU (i.e., RTX 2080ti), or every 0.67 ms on one CPU (i.e., Intel Core i7 8700K).

show abstract

Motion Images: An Effective Representation of Motion Capture Data for Similarity Search

Cited by 13 publications

References 10 publications

Keys for Action: An Efficient Keyframe-Based Approach for 3D Action Recognition Using a Deep Neural Network

Keys for Action: An Efficient Keyframe-Based Approach for 3D Action Recognition Using a Deep Neural Network

TY-Net: Transforming YOLO for Hand Gesture Recognition

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

Contact Info

Product

Resources

About