Tracking as Online Decision-Making: Learning a Policy from Streaming Videos with Reinforcement Learning

Supancic, James Steven; Ramanan, Deva

doi:10.1109/iccv.2017.43

Cited by 103 publications

(48 citation statements)

References 52 publications

(124 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training of the proposed method is stable, and the agents obtain a good strategy immediately because all the agents share the parameters and the gradients are averaged as shown in Eqs. (14) and (16). In other words, the agents can obtain N (> 10 5 ) experience at one backward pass in contrast to usual reinforcement learning where only one experience is obtained at one backward pass.…”

Section: Applications and Resultsmentioning

confidence: 98%

“…The fundamental algorithm of the proposed method and the experimental results of the image denoising, image restoration, and local color enhancement have already been presented in our preliminary study [6]. This paper provides a more detailed explanation and a new application of the pixelRL to saliency-arXiv:1912.07190v1 [cs.CV] 16 Dec 2019 driven image editing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Furuta

Inoue

Yamasaki

2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

This paper tackles a new problem setting: reinforcement learning with pixel-wise rewards (pixelRL) for image processing. After the introduction of the deep Q-network, deep RL has been achieving great success. However, the applications of deep reinforcement learning (RL) for image processing are still limited. Therefore, we extend deep RL to pixelRL for various image processing applications. In pixelRL, each pixel has an agent, and the agent changes the pixel value by taking an action. We also propose an effective learning method for pixelRL that significantly improves the performance by considering not only the future states of the own pixel but also those of the neighbor pixels. The proposed method can be applied to some image processing tasks that require pixel-wise manipulations, where deep RL has never been applied. Besides, it is possible to visualize what kind of operation is employed for each pixel at each iteration, which would help us understand why and how such an operation is chosen. We also believe that our technology can enhance the explainability and interpretability of the deep neural networks. In addition, because the operations executed at each pixels are visualized, we can change or modify the operations if necessary.We apply the proposed method to a variety of image processing tasks: image denoising, image restoration, local color enhancement, and saliency-driven image editing. Our experimental results demonstrate that the proposed method achieves comparable or better performance, compared with the state-ofthe-art methods based on supervised learning. The source code is available on https://github.com/rfuruta/pixelRL.

show abstract

Section: Applications and Resultsmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Furuta

Inoue

Yamasaki

2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…Recently, in the computer vision community there are also preliminary attempts of applying deep RL to traditional tasks, e.g., object localization [37] and region proposal [38]. There are also methods of visual tracking relying on RL [39], [40], [41], [42]. However, they are distinct from our work, as they formulate passive tracking with RL and do not consider camera controls.…”

Section: Reinforcement Learningmentioning

confidence: 99%

End-to-End Active Object Tracking and Its Real-World Deployment via Reinforcement Learning

Luo

Sun

Zhong

et al. 2020

IEEE Trans. Pattern Anal. Mach. Intell.

129

View full text Add to dashboard Cite

We study active object tracking, where a tracker takes visual observations (i.e., frame sequences) as input and produces the corresponding camera control signals as output (e.g., move forward, turn left, etc.). Conventional methods tackle tracking and camera control tasks separately, and the resulting system is difficult to tune jointly. These methods also require significant human efforts for image labeling and expensive trial-and-error system tuning in the real world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning. A ConvNet-LSTM function approximator is adopted for the direct frame-to-action prediction. We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training. The tracker trained in simulators (ViZDoom and Unreal Engine) demonstrates good generalization behaviors in the case of unseen object moving paths, unseen object appearances, unseen backgrounds, and distracting objects. The system is robust and can restore tracking after occasional lost of the target being tracked. We also find that the tracking ability, obtained solely from simulators, can potentially transfer to real-world scenarios. We demonstrate successful examples of such transfer, via experiments over the VOT dataset and the deployment of a real-world robot using the proposed active tracker trained in simulation.

show abstract

“…Employing the deep RL algorithms into computer vision problems could benefit from the experience. In fact, RL has been studied for visual tracking in several recent works [27], [28], [43], [44]. Huang et al succeed in utilizing Q-learning [29] for shallow-level or high-level feature selection [27].…”

Section: Deep Reinforcement Learningmentioning

confidence: 99%

Correlation Filter Selection for Visual Tracking Using Reinforcement Learning

Xie

Xiao

Huang

et al. 2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Correlation filter has been proven to be an effective tool for a number of approaches in visual tracking, particularly for seeking a good balance between tracking accuracy and speed. However, correlation filter based models are susceptible to wrong updates stemming from inaccurate tracking results. To date, little effort has been devoted towards handling the correlation filter update problem. In this paper, we propose a novel approach to address the correlation filter update problem. In our approach, we update and maintain multiple correlation filter models in parallel, and we use deep reinforcement learning for the selection of an optimal correlation filter model among them. To facilitate the decision process in an efficient manner, we propose a decision-net to deal target appearance modeling, which is trained through hundreds of challenging videos using proximal policy optimization and a lightweight learning network. An exhaustive evaluation of the proposed approach on the OTB100 and OTB2013 benchmarks show that the approach is effective enough to achieve the average success rate of 62.3% and the average precision score of 81.2%, both exceeding the performance of traditional correlation filter based trackers.

show abstract

Tracking as Online Decision-Making: Learning a Policy from Streaming Videos with Reinforcement Learning

Cited by 103 publications

References 52 publications

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

End-to-End Active Object Tracking and Its Real-World Deployment via Reinforcement Learning

Correlation Filter Selection for Visual Tracking Using Reinforcement Learning

Contact Info

Product

Resources

About