First-Person Daily Activity Recognition With Manipulated Object Proposals and Non-Linear Feature Fusion

Wang, Meng; Luo, Changzhi; Ni, Bingbing; Yuan, Jun; Wang, Jianfeng; Yan, Shuicheng

doi:10.1109/tcsvt.2017.2716819

Cited by 39 publications

(14 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recommender system research, two lines of contributions are most significant to date: 1) pure Collaborative Filtering (CF) techniques such as matrix factorization [27] and its variants [11], and 2) content-or context-aware methods that rely on more complex models such as featurebased embeddings [2] and deep learning [36], [37]. While multimedia recommendation falls into the second category of content-based recommendations, it is more challenging yet popular, due to massive and abundant multimedia (e.g., visual, acoustic and semantic) features in real-world information systems [19], [38], [39].…”

Section: Multimedia Recommendationmentioning

confidence: 99%

Adversarial Training Towards Robust Multimedia Recommender System

Tang

et al. 2020

IEEE Trans. Knowl. Data Eng.

154

View full text Add to dashboard Cite

With the prevalence of multimedia content on the Web, developing recommender solutions that can effectively leverage the rich signal in multimedia data is in urgent need. Owing to the success of deep neural networks in representation learning, recent advance on multimedia recommendation has largely focused on exploring deep learning methods to improve the recommendation accuracy. To date, however, there has been little effort to investigate the robustness of multimedia representation and its impact on the performance of multimedia recommendation. In this paper, we shed light on the robustness of multimedia recommender system. Using the state-of-the-art recommendation framework and deep image features, we demonstrate that the overall system is not robust, such that a small (but purposeful) perturbation on the input image will severely decrease the recommendation accuracy. This implies the possible weakness of multimedia recommender system in predicting user preference, and more importantly, the potential of improvement by enhancing its robustness. To this end, we propose a novel solution named Adversarial Multimedia Recommendation (AMR), which can lead to a more robust multimedia recommender model by using adversarial learning. The idea is to train the model to defend an adversary, which adds perturbations to the target image with the purpose of decreasing the model's accuracy. We conduct experiments on two representative multimedia recommendation tasks, namely, image recommendation and visually-aware product recommendation. Extensive results verify the positive effect of adversarial learning and demonstrate the effectiveness of our AMR method. Source codes are available in https://github.com/duxy-me/AMR.

show abstract

Section: Multimedia Recommendationmentioning

confidence: 99%

Adversarial Training Towards Robust Multimedia Recommender System

Tang

et al. 2020

IEEE Trans. Knowl. Data Eng.

154

View full text Add to dashboard Cite

show abstract

“…The two-stream approach for action recognition is first proposed in [10], in which spatial network acquires single RGB frame feature and temporal network captures the motion pattern between frames with the input of 10 stacked optical flow images. Then a great number of efforts [16], [17], [32] have been made to enhance the twostream network from different perspectives.…”

Section: B Action Recognition With Deep Neural Networkmentioning

confidence: 99%

A Real-Time Action Representation With Temporal Encoding and Deep Compression

Liu

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Deep neural networks have achieved remarkable success for video-based action recognition. However, most of existing approaches cannot be deployed in practice due to the high computational cost. To address this challenge, we propose a new real-time convolutional architecture, called Temporal Convolutional 3D Network (T-C3D), for action representation. T-C3D learns video action representations in a hierarchical multi-granularity manner while obtaining a high process speed. Specifically, we propose a residual 3D Convolutional Neural Network (CNN) to capture complementary information on the appearance of a single frame and the motion between consecutive frames. Based on this CNN, we develop a new temporal encoding method to explore the temporal dynamics of the whole video. Furthermore, we integrate deep compression techniques with T-C3D to further accelerate the deployment of models via reducing the size of the model. By these means, heavy calculations can be avoided when doing the inference, which enables the method to deal with videos beyond real-time speed while keeping promising performance. Our method achieves clear improvements on UCF101 action recognition benchmark against state-of-the-art real-time methods by 5.4% in terms of accuracy and 2 times faster in terms of inference speed with a less than 5MB storage model. We validate our approach by studying its action representation performance on four different benchmarks over three different tasks. Extensive experiments demonstrate comparable recognition performance to the stateof-the-art methods. The source code and the pre-trained models are publicly available at https://github.com/tc3d.

show abstract

“…This segmentation process is a 1D version of the image-based 2D region-growing technique, which is a class of bottom-up image segmentation algorithms used to segment an image into homogeneous regions based on similarity of neighboring gray-level values. Image segmentation methods such as the top-down quad-tree and bottomup region-growing techniques are useful tools that have been utilized in a variety of image processing applications such as image steganography [17], image compression [27], and object detection [28].…”

Section: A Pixogram Segment-growingmentioning

confidence: 99%

The Pixogram: Addressing High Payload Demands for Video Steganography

Rabie

Baziyad

2019

IEEE Access

View full text Add to dashboard Cite

This paper introduces the concept of a pixogram which makes possible a fresh approach to high payload video steganography. The pixogram allows for a new perspective by investigating the temporal changes that take place at the individual pixel level across frames of a video segment. Simply put, a pixogram has the property of converting highly uncorrelated spatial areas of individual frames of a video scene into highly correlated temporal segments by making use of the temporal correlation between the frames of the same scene in a given video segment, thus maximizing the redundant area suitable for hiding in the transform domain. Experimental results demonstrate the effectiveness of this new approach for increased payload capacity while maintaining visual fidelity of the stego-video as compared to competing video steganography schemes.

show abstract

First-Person Daily Activity Recognition With Manipulated Object Proposals and Non-Linear Feature Fusion

Cited by 39 publications

References 43 publications

Adversarial Training Towards Robust Multimedia Recommender System

Adversarial Training Towards Robust Multimedia Recommender System

A Real-Time Action Representation With Temporal Encoding and Deep Compression

The Pixogram: Addressing High Payload Demands for Video Steganography

Contact Info

Product

Resources

About