G2Grad-CAMRL: An Object Detection and Interpretation Model Based on Gradient-Weighted Class Activation Mapping and Reinforcement Learning in Remote Sensing Images

Abstract

Remote sensing images contain important information such as airports, ports, and ships. By extracting remote sensing image features and learning the mapping relationship between image features and text semantic features, the interpretation and description of remote sensing image content can be realized, which has a wide range of application value in military and civil fields such as national defense security, land monitoring, urban planning, disaster mitigation and so on. Aiming at the complex background of remote sensing images and the lack of interpretability of existing target detection models, and the problems in feature extraction between different network structures, different layers, and the accuracy of target classification, we propose an object detection and interpretation model based on Gradient weighted class activation mapping and reinforcement learning. Firstly, ResNet is used as the main backbone network to extract the features of remote sensing images and generate feature graphs. Then, we add the global average pooling layer to obtain the corresponding feature weight vector of the feature graph. The weighted vectors are superimposed to output class activation maps. The reinforcement learning method is used to optimize the generated region generation network. At the same time, we improve the reward function of reinforcement learning to improve the effect of the region generation network. Finally, network dissecting analysis is used to obtain the interpretable semantic concept in the model. Through experiments, the average accuracy is more than 85%. Experimental results in the public remote sensing image description dataset show that the proposed method has high detection accuracy and good description performance for remote sensing images in complex environments.

EfDenseNet: Automated Pulmonary Hypertension Detection Model Based on EfficientNetb0 and DenseNet201 Using CT Images

“…We applied gradient-weighted class activation mapping [27,28] to generate heat maps showing areas of the input CT images that had the most impact on the classification results. The EfDenseNet focused mainly on the pulmonary artery bifurcation in the middle of the images to detect PH (see Figure 7), which aligns with the way radiologists visually examine the image to assess for morphological signs of PH.…”

Predicting depression and suicidal tendencies by analyzing online activities using machine learning in android devices

“…By using the miniImageNet dataset and the omniglot dataset, a conv4 network is designed to extract the class features to make the predictions is proposed in [34]. [35] proposed gradient weighted class activation mapping and reinforcement learning based object detection techniques by following ResNet for remote sensing images which contains important information such as airports, ports and ships. [36] proposed a federated learning approach based on priori knowledge and a bilateral segmentation network for image edge extraction to improve the problem of data islanding.…”