Scene Recognition via Object-to-Scene Class Conversion: End-to-End Training

FOSNet: An End-to-End Trainable Deep Neural Network for Scene Recognition

“…Using object information in an image is the most utilized scene traits for scene recognition [5], [6], [9]. When a particular object appears in an image, the chance of the image belonging to a certain category associated with the object increases.…”

“…Recently, the CCM has learned a relationship between an object feature and scene feature through training and converted object features into scene features [9]. By doing so, the two features in different domains are transformed into the same domain, and the fusion is performed through the elementwise sum operation.…”

“…The CCG is based on a scene traits that when a specific object in an image is found, the scene is very likely to belong to a particular class associated with the object. The CCG is inspired by context gating [31] and the CCM [9]. The concept of CCG is depicted in Fig.…”

“…For a backbone network, the SE-ResNeXt-101 model, which is a combination of ResNeXt [27] with SE-Network [3], was used for Object-Net and PlacesNet in FOSNet. The standard 10-crop testing method [7] is used for comparison with other methods, and an Adi-Red [13] 41.87 -Places365-VGG [28] 55.24 -CCM [9] 56.82 86.92 CNN-SMN [14] 57.1 -SOSF+CFA+GAF [20] 57.27 -Multi-Resolution CNNs [7] 58.3 87.3 Places2-365-CNN [43] 58.93 88.52 SE-Resnet-152 [3] 59 evaluation measurement is the average classification accuracy of 10 crops.…”

“…On the other hand, the CCG combines the two features in an efficient manner, applying the attention concept to the fusion of two features. The CCG is an improved version of the class conversion matrix (CCM) [9], and it also determines the relative importance of the features during training.…”