Part-Activated Deep Reinforcement Learning for Action Prediction

Chen, Lei; Lu, Jiwen; Song, Zhanjie; Zhou, Jie

doi:10.1007/978-3-030-01219-9_26

Cited by 45 publications

(27 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Initial work has already demonstrated the benefits of combining reinforcement learning with RNNs to play Atari ® games 145 . Promising results have also been obtained for visual tracking, 146,147 face recognition, 148 action recognition, 149,150 video captioning, 151 color enhancement, 152 and object detection 153,154 …”

Section: The Role Of Recurrence Beyond Recognitionmentioning

confidence: 99%

“…Initial work has already demonstrated the benefits of combining reinforcement learning with RNNs to play Atari R games. 145 Promising results have also been obtained for visual tracking, 146,147 face recognition, 148 action recognition, 149,150 video captioning, 151 color enhancement, 152 and object detection. 153,154 Another approach to learning structure in the visual world, which does not use explicit labeled examples or a teacher and provides direct rewards/punishment for specific actions, is based on the intuition that predicting what will happen next may be an important principle of computation in the brain.…”

Section: Learning and Plasticitymentioning

confidence: 99%

See 1 more Smart Citation

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Visual perception involves the rapid formation of a coarse image representation at the onset of visual processing, which is iteratively refined by late computational processes. These early versus late time windows approximately map onto feedforward and feedback processes, respectively. State‐of‐the‐art convolutional neural networks, the main engine behind recent machine vision successes, are feedforward architectures. Their successes and limitations provide critical information regarding which visual tasks can be solved by purely feedforward processes and which require feedback mechanisms. We provide an overview of recent work in cognitive neuroscience and machine vision that highlights the possible role of feedback processes for both visual recognition and beyond. We conclude by discussing important open questions for future research.

show abstract

Section: The Role Of Recurrence Beyond Recognitionmentioning

confidence: 99%

Section: Learning and Plasticitymentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

show abstract

“…Our work is most closely related to the ones that use reinforcement learning for action recognition. In [4], the part activation policy of human body parts is learned with RL for action prediction. [40] focuses on skeletonbased action recognition and RL is adopted to distinguish discriminative poses.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…In our case we wish to learn network parameters θ π that maximize the equation (4). The gradient of J(θ π ) is ∇ θπ J(θ π ) = N a=1 u∈U π(u|s; θ π )∇ θπ log π(a|s; θ π )R a .…”

Section: Marl Objectivementioning

confidence: 99%

Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition

Tan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

141

126

View full text Add to dashboard Cite

Video Recognition has drawn great research interest and great progress has been made. A suitable frame sampling strategy can improve the accuracy and efficiency of recognition. However, mainstream solutions generally adopt handcrafted frame sampling strategies for recognition. It could degrade the performance, especially in untrimmed videos, due to the variation of frame-level saliency. To this end, we concentrate on improving untrimmed video classification via developing a learning-based frame sampling strategy. We intuitively formulate the frame sampling procedure as multiple parallel Markov decision processes, each of which aims at picking out a frame/clip by gradually adjusting an initial sampling. Then we propose to solve the problems with multi-agent reinforcement learning (MARL). Our MARL framework is composed of a novel RNN-based context-aware observation network which jointly models context information among nearby agents and historical states of a specific agent, a policy network which generates the probability distribution over a predefined action space at each step and a classification network for reward calculation as well as final recognition. Extensive experimental results show that our MARL-based scheme remarkably outperforms hand-crafted strategies with various 2D and 3D baseline methods. Our single RGB model achieves a comparable performance of ActivityNet v1.3 champion submission with multi-modal multi-model fusion and new state-ofthe-art results on YouTube Birds and YouTube Cars.

show abstract

“…Therefore, we choose the average length for each bone in the first several frames as the reference bone length l ij in Eq. (5). Fig.…”

Section: Results On Kinect Datamentioning

confidence: 99%

Spatio-Temporal Reconstruction for 3D Motion Recovery

Yang

Guo

et al. 2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

This paper addresses the challenge of 3D motion recovery by exploiting the spatio-temporal correlations of corrupted 3D skeleton sequences. We propose a new 3D motion recovery method using spatio-temporal reconstruction, which uses joint low-rank and sparse priors to exploit temporal correlation and an isometric constraint for spatial correlation. The proposed model is formulated as a constrained optimization problem, which is efficiently solved by the augmented Lagrangian method with a Gauss-Newton solver for the subproblem of isometric optimization. Experimental results on the CMU motion capture dataset, Edinburgh dataset and two Kinect datasets demonstrate that the proposed approach achieves better motion recovery than state-of-the-art methods. The proposed method is applicable to Kinect-like skeleton tracking devices and pose estimation methods that cannot provide accurate estimation of complex motions, especially in the presence of occlusion.

show abstract

Part-Activated Deep Reinforcement Learning for Action Prediction

Cited by 45 publications

References 38 publications

Beyond the feedforward sweep: feedback computations in the visual cortex

Beyond the feedforward sweep: feedback computations in the visual cortex

Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition

Spatio-Temporal Reconstruction for 3D Motion Recovery

Contact Info

Product

Resources

About