Collaborative Deep Reinforcement Learning for Image Cropping

Li, Zhuopeng; Zhang, Xiaoyan

doi:10.1109/icme.2019.00052

Cited by 15 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One theoretical concept that must not be neglected in studying RL-based methods is one of societal value. This relates to a broader spectrum of methods under cooperative or collaborative RL (Chan, 2017;Lin et al, 2017). As its naming hints, these methods are derived from the behavior of humans in society.…”

Section: Rl Based Methodsmentioning

confidence: 99%

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Sousa

Henriques

2023

Preprint

View full text Add to dashboard Cite

In the current context of big data, the operational forecasting problems are more and more frequently involving the prediction of collections of multivariate, high-dimensional, related time series. The challenge of forecasting groups of time series can be tackled by fitting a single function to all series (global approach) or assuming each series to be a separate prediction problem and fitting one function to each problem (local approach). Although some global models show promising results against local benchmarks, there is still unclarity on how to best control generalization and structural assumptions by calibrating the exploitation of global patterns with local components. This is heavily reliant on the level of complexity from the dependency relationships exhibited between the series within a collection (e.g. indirect latent, local covariate relationships or noise effects) and other dimensions, such as the heterogeneity of the series and its length. For these reasons, there is an acknowledged need to invest further in developing scalable and data-driven hybrid models. The literature proposes Deep Learning (DL) based models built from different data generating processes (such as Auto-Regressive) which combine global models with classical local models (global-local frameworks). These methods frequently capture globality through factorization models or latent deep components and locality via classical local models or networks. These methods present non-linear modelling capabilities but are still either limited by solid theoretical assumptions (e.g. linearity of the underlying data) or predominantly one-dimensional, i.e., the future predictions for a single dimension mainly depend on past values from that same dimension. Simultaneously, dynamic Reinforcement Learning (RL) based models have been widely explored to optimize the balance of global and local signals in general regression problems, frequently through Q-learning algorithms. These derive from the equilibrium of cooperative strategies or the societal value in multi-agent learning systems. Similar models have been successfully applied to time series forecasting problems (such as stock market predictions), addressing the risk of method generalization under varying conditions. This paper conducted a concise literature review focused on these two research streams (global-local frameworks and RL based models) to optimize the balance between globality and locality in forecasting collections of time series. It focuses on their evolution across time and hints at opportunities to close some of the research gaps by intersecting both propositions. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and achieved a selection of 118 publications since 2000. The main findings revealed that global models have achieved strong expressiveness in capturing the most complex structural patterns while still enabling probabilistic outcomes to be delivered through uncertainty estimates. On the other hand, RL based methods depict great benefits in mitigating the risks of generalization by imprinting contextual diversity when predicting each step ahead for each series. Within those, the adoption of other computational learning or evolutionary-based methods (e.g. Genetic Programming) to improve the parametrization of the learning policies is also highlighted as an area of future work yet to be uncovered.

show abstract

Section: Rl Based Methodsmentioning

confidence: 99%

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Sousa

Henriques

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Aamir [17] constructs a framework for cropping which using graphbased segmentation and grey level adjustment to get more accurate and clear saliency maps. Li and Zhang [18] implement reinforcement learning with two agent. One agent deals with the raw image data and the other deals with the attention information.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement learning cropping method based on comprehensive feature and aesthetics assessment

Zhang

2022

IET Image Processing

View full text Add to dashboard Cite

Automatic image cropping can change the composition to improve the aesthetic quality of the images. Most of the existing automatic image cropping methods based on specific features need to generate a large number of candidate cropping windows. It is very time-consuming and can only produce a limited aspect ratio results. In the face of these situations, a reinforcement learning cropping method based on comprehensive feature and aesthetics assessment is proposed. It does not need to produce a large number of candidate windows. Its gradually cropping mode is more in line with the process of image cropping by human. What is more, the proposed method takes the image aesthetic assessment into consideration. Experimental results show that the proposed method improves the cropping efficiency and achieves excellent cropping effect on the open Flickr Cropping Dataset and CUHK Image Cropping Dataset. The proposed method can overcome the shortages of existing methods.

show abstract

“…Attention-guided Image Cropping: Attention-guided methods [27,25,26,4,11,36,47] assumed that the best crops should preserve visually important content, which is usually determined by the saliency detection methods [38,17]. Usually, the view with the highest average saliency score is selected as the best crop.…”

Section: Related Workmentioning

confidence: 99%

Human-centric Image Cropping with Partition-aware and Content-preserving Features

Zhang¹,

Niu²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Image cropping aims to find visually appealing crops in an image, which is an important yet challenging task. In this paper, we consider a specific and practical application: human-centric image cropping, which focuses on the depiction of a person. To this end, we propose a human-centric image cropping method with two novel feature designs for the candidate crop: partition-aware feature and content-preserving feature. For partition-aware feature, we divide the whole image into nine partitions based on the human bounding box and treat different partitions in a candidate crop differently conditioned on the human information. For content-preserving feature, we predict a heatmap indicating the important content to be included in a good crop, and extract the geometric relation between the heatmap and a candidate crop. Extensive experiments demonstrate that our method can perform favorably against state-of-the-art image cropping methods on human-centric image cropping task. Code is available at https://github.com/bcmi/ Human-Centric-Image-Cropping.

show abstract

Collaborative Deep Reinforcement Learning for Image Cropping

Cited by 15 publications

References 11 publications

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Intersecting reinforcement learning and deep factor methods for optimizing locality and globality in forecasting: a review

Reinforcement learning cropping method based on comprehensive feature and aesthetics assessment

Human-centric Image Cropping with Partition-aware and Content-preserving Features

Contact Info

Product

Resources

About