Hierarchical Cellular Automata for Visual Saliency

Qin, Yao; Feng, Mengyang; Lu, Huchuan; Cottrell, Garrison W.

doi:10.1007/s11263-017-1062-2

Cited by 66 publications

(42 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…13. We used six saliency map estimation methods including both hand-crafted ones (SR [67], VSF [66], and PCA [68]) and deep learningbased ones (NLDF [69]), HCA [70], and PiCANet [71]). The proposed method successfully learns how to enhance/diminish the target/non-target regions.…”

Section: ) Methodmentioning

confidence: 99%

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Furuta

Inoue

Yamasaki

2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

This paper tackles a new problem setting: reinforcement learning with pixel-wise rewards (pixelRL) for image processing. After the introduction of the deep Q-network, deep RL has been achieving great success. However, the applications of deep reinforcement learning (RL) for image processing are still limited. Therefore, we extend deep RL to pixelRL for various image processing applications. In pixelRL, each pixel has an agent, and the agent changes the pixel value by taking an action. We also propose an effective learning method for pixelRL that significantly improves the performance by considering not only the future states of the own pixel but also those of the neighbor pixels. The proposed method can be applied to some image processing tasks that require pixel-wise manipulations, where deep RL has never been applied. Besides, it is possible to visualize what kind of operation is employed for each pixel at each iteration, which would help us understand why and how such an operation is chosen. We also believe that our technology can enhance the explainability and interpretability of the deep neural networks. In addition, because the operations executed at each pixels are visualized, we can change or modify the operations if necessary.We apply the proposed method to a variety of image processing tasks: image denoising, image restoration, local color enhancement, and saliency-driven image editing. Our experimental results demonstrate that the proposed method achieves comparable or better performance, compared with the state-ofthe-art methods based on supervised learning. The source code is available on https://github.com/rfuruta/pixelRL.

show abstract

Section: ) Methodmentioning

confidence: 99%

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Furuta

Inoue

Yamasaki

2020

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…For instance, in the second row of Figure 12, the salient boat was well-separated from the non-salient regions by the SCA+SH+FDAG method, while the SCA+SLIC method also assigned significant saliency values to some non-salient pixels. Complementary to the visual comparison, we also quantitatively evaluated the saliency detection performance of the SCA+SLIC and SCA+SH+FDAG methods in terms of the mean absolute error (MAE) representing the difference between the obtained saliency map and the GT saliency map [2]:…”

Section: Application To Saliency Detectionmentioning

confidence: 99%

Superpixel Segmentation Based on Anisotropic Edge Strength

Wang

Baets

2019

J. Imaging

View full text Add to dashboard Cite

Superpixel segmentation can benefit from the use of an appropriate method to measure edge strength. In this paper, we present such a method based on the first derivative of anisotropic Gaussian kernels. The kernels can capture the position, direction, prominence, and scale of the edge to be detected. We incorporate the anisotropic edge strength into the distance measure between neighboring superpixels, thereby improving the performance of an existing graph-based superpixel segmentation method. Experimental results validate the superiority of our method in generating superpixels over the competing methods. It is also illustrated that the proposed superpixel segmentation method can facilitate subsequent saliency detection.

show abstract

“…More discussions about the values of M can be found in Section 3.1.3. To effectively integrate the results of the multiple scales M, the CCA method [46] was employed, whereby each cell corresponds to a pixel, and the saliency values of all pixels constitute the set of cells' states. For any cell in a saliency map, there should be 5M − 1 neighbors, including pixels with the same coordinates from different saliency maps, in addition to their 4 connected pixels [46].…”

Section: Multiscale Saliencymentioning

confidence: 99%

“…To effectively integrate the results of the multiple scales M, the CCA method [46] was employed, whereby each cell corresponds to a pixel, and the saliency values of all pixels constitute the set of cells' states. For any cell in a saliency map, there should be 5M − 1 neighbors, including pixels with the same coordinates from different saliency maps, in addition to their 4 connected pixels [46]. The saliency value of pixel i in the m-th saliency map at time t stands for its probability to be the foreground F, represented as S (t) m,i , while its possibility to be the background B is denoted as 1 − S (t) m,i .…”

Section: Multiscale Saliencymentioning

confidence: 99%

“…Then, the GWB model [45] is used to consider spatial information, thereby eliminating the impact of land cover surrounding built-up areas, to further improve accuracy and provide reliable training samples for the strong saliency model. After that, the CCA [46] is employed to effectively integrate multiscale saliency maps to optimize the accuracy of the final saliency map. Finally, FODPSO algorithm [47] is used to segment the final saliency map to accurately capture information on built-up areas.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Improved Boosting Learning Saliency Method for Built-Up Areas Extraction in Sentinel-2 Images

2018

View full text Add to dashboard Cite

Built-up areas extraction from satellite images is an important aspect of urban planning and land use; however, this remains a challenging task when using optical satellite images. Existing methods may be limited because of the complex background. In this paper, an improved boosting learning saliency method for built-up area extraction from Sentinel-2 images is proposed. First, the optimal band combination for extracting such areas from Sentinel-2 data is determined; then, a coarse saliency map is generated, based on multiple cues and the geodesic weighted Bayesian (GWB) model, that provides training samples for a strong model; a refined saliency map is subsequently obtained using the strong model. Furthermore, cuboid cellular automata (CCA) is used to integrate multiscale saliency maps for improving the refined saliency map. Then, coarse and refined saliency maps are synthesized to create a final saliency map. Finally, the fractional-order Darwinian particle swarm optimization algorithm (FODPSO) is employed to extract the built-up areas from the final saliency result. Cities in five different types of ecosystems in China (desert, coastal, riverside, valley, and plain) are used to evaluate the proposed method. Analyses of results and comparative analyses with other methods suggest that the proposed method is robust, with good accuracy.

show abstract

Hierarchical Cellular Automata for Visual Saliency

Cited by 66 publications

References 71 publications

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing

Superpixel Segmentation Based on Anisotropic Edge Strength

An Improved Boosting Learning Saliency Method for Built-Up Areas Extraction in Sentinel-2 Images

Contact Info

Product

Resources

About