Understanding deep image representations by inverting them

Mahendran, Aravindh; Vedaldi, Andrea

doi:10.1109/cvpr.2015.7299155

Cited by 1,574 publications

(1,274 citation statements)

References 25 publications

Supporting

Mentioning

1,264

Contrasting

Order By: Relevance

“…Analysis of the DNN-based background subtraction is needed for discussing the characteristics and the issues. Visualization methods for analyzing DNNs are proposed [28][29][30]. The authors visualized features contributing to classification by DNNs.…”

Section: Related Workmentioning

confidence: 99%

Analytics of Deep Neural Network-Based Background Subtraction

et al. 2018

View full text Add to dashboard Cite

Deep neural network-based (DNN-based) background subtraction has demonstrated excellent performance for moving object detection. The DNN-based background subtraction automatically learns the background features from training images and outperforms conventional background modeling based on handcraft features. However, previous works fail to detail why DNNs work well for change detection. This discussion helps to understand the potential of DNNs in background subtraction and to improve DNNs. In this paper, we observe feature maps in all layers of a DNN used in our investigation directly. The DNN provides feature maps with the same resolution as that of the input image. These feature maps help to analyze DNN behaviors because feature maps and the input image can be simultaneously compared. Furthermore, we analyzed important filters for the detection accuracy by removing specific filters from the trained DNN. From the experiments, we found that the DNN consists of subtraction operations in convolutional layers and thresholding operations in bias layers and scene-specific filters are generated to suppress false positives from dynamic backgrounds. In addition, we discuss the characteristics and issues of the DNN based on our observation.

show abstract

Section: Related Workmentioning

confidence: 99%

Analytics of Deep Neural Network-Based Background Subtraction

et al. 2018

View full text Add to dashboard Cite

show abstract

“…The top layers of the network end up-if sufficiently deep-capturing the content of the image, i.e., forming archetypal representations of the objects on which they have been trained: faces, animals, buildings, etc. (Mahendran and Vedaldi, 2014). In a symmetric movement, the potential of deep architectures to encode complex structures such as images or sound can also be used to produce new expressions of these objects: a property that has initiated a new wave of creative applications in the fine arts.…”

Section: From Shallow To Deep Neural Networkmentioning

confidence: 99%

Deep Creations: Intellectual Property and the Automata

Deltorn

2017

Front. Digit. Humanit.

View full text Add to dashboard Cite

The rapid progress of deep neural network architectures is allowing both to automate the production of artworks and to extend the domain of creative expression. As such, it is opening new ground for professional and amateur artists alike. A major asset of these new computer processes is their capacity to derive, from a training phase, a generative model from which new artifacts can be produced. This attribute allows for a wide range of novel applications. New music or paintings in the style of famous artists can be produced at the click of a button, or combined to form new artworks. New graphical compositions can be "hallucinated" by the deep algorithmic models to produce striking, unexpected, visual forms. By the same token, the dependence on preexisting, protected, artworks lays the ground for potential zones of friction with the rights holders of the source data that helped shape the generative model. This articulation, between the popular creative movement initiated by the deep neural architectures and the preexisting rights of the authors, leads to a confrontation between the present legal framework for the protection of artistic creations and the new modalities offered by these new technological objects. The present work will address the conditions of protection of creations generated by deep neural networks under the main copyright regimes.

show abstract

“…In an attempt to better understand the properties of a CNN, some recent vision works have focused on analyzing their internal representations (Szegedy et al 2014;Yosinski et al 2014;Lenc and Vedaldi 2015;Mahendran and Vedaldi 2015;Zeiler and Fergus 2014;Simonyan et al 2014;Agrawal et al 2014;Zhou et al 2015;Eigen et al 2013). Some of these investigated properties of the network, like stability (Szegedy et al 2014), feature transferability (Yosinski et al 2014), equivariance, invariance and equivalence (Lenc and Vedaldi 2015), the ability to reconstruct the input (Mahendran and Vedaldi 2015) and how the number of layers, filters and parameters affects the network performance (Agrawal et al 2014;Eigen et al 2013). Zeiler and Fergus (2014) use deconvolutional networks to visualize locally optimal visual inputs for individual filters.…”

Section: Related Workmentioning

confidence: 99%

Do Semantic Parts Emerge in Convolutional Neural Networks?

2017

View full text Add to dashboard Cite

Semantic object parts can be useful for several visual recognition tasks. Lately, these tasks have been addressed using Convolutional Neural Networks (CNN), achieving outstanding results. In this work we study whether CNNs learn semantic parts in their internal representation. We investigate the responses of convolutional filters and try to associate their stimuli with semantic parts. We perform two extensive quantitative analyses. First, we use groundtruth part bounding-boxes from the PASCAL-Part dataset to determine how many of those semantic parts emerge in the CNN. We explore this emergence for different layers, network depths, and supervision levels. Second, we collect human judgements in order to study what fraction of all filters systematically fire on any semantic part, even if not annotated in PASCAL-Part. Moreover, we explore several connections between discriminative power and semantics. We find out which are the most discriminative filters for object recognition, and analyze whether they respond to semantic parts or to other image patches. We also investigate the other direction: we determine which semantic parts are the most discriminative and whether they correspond to those parts emerging in the network. This enables to gain an even deeper understanding of the role of semantic parts in the network.Communicated by

show abstract

Understanding deep image representations by inverting them

Cited by 1,574 publications

References 25 publications

Analytics of Deep Neural Network-Based Background Subtraction

Analytics of Deep Neural Network-Based Background Subtraction

Deep Creations: Intellectual Property and the Automata

Do Semantic Parts Emerge in Convolutional Neural Networks?

Contact Info

Product

Resources

About