Semantic representation for visual reasoning

Ni, Xubin; Yin, Lirong; Chen, Xiaobing; Liu, Shan; Yang, Bo; Zheng, Wenfeng

doi:10.1051/matecconf/201927702006

Cited by 29 publications

(22 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the former point [1], the general method of extracting semantic representation in the form of the semantic network from general images is perfected. The three elements of the semantic network are node, attribute class, and attribute value.…”

Section: Discussionmentioning

confidence: 99%

“…Visual Question Answering (VQA) combines natural language processing with digital image processing. The general process for solving a VQA problem is to take the image and the corresponding question as input and finally get the answer [1]. The problems which are similar to VQA require more interdependent inference steps to solve.…”

Section: Introductionmentioning

confidence: 99%

“…A method of replacing the visual features of images with the semantic representation of images as a visual reasoning model input is proposed in this paper. This paper's method is based on the baseline model prosed in previous works [1]. We combine the parts and techniques involved in the method and construct weakened image processing and natural language processing.…”

Section: Introductionmentioning

confidence: 99%

“…We combine the parts and techniques involved in the method and construct weakened image processing and natural language processing. Based on the previous point [1], we improve the general method of extracting the general image's semantic representation. After extracting the image's semantic representation, two understanding modules are combined to form a high accuracy coding and decoding model of the representation vectors.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Improving Visual Reasoning Through Semantic Representation

et al. 2021

Self Cite

View full text Add to dashboard Cite

In visual reasoning, the achievement of deep learning significantly improved the accuracy of results. Image features are primarily used as input to get answers. However, the image features are too redundant to learn accurate characterizations within a limited complexity and time. While in the process of human reasoning, abstract description of an image is usually to avoid irrelevant details. Inspired by this, a higher-level representation named semantic representation is introduced. In this paper, a detailed visual reasoning model is proposed. This new model contains an image understanding model based on semantic representation, feature extraction and process model refined with watershed and u-distance method, a feature vector learning model using pyramidal pooling and residual network, and a question understanding model combining problem embedding coding method and machine translation decoding method. The feature vector could better represent the whole image instead of overly focused on specific characteristics. The model using semantic representation as input verifies that more accurate results can be obtained by introducing a highlevel semantic representation. The result also shows that it is feasible and effective to introduce high-level and abstract forms of knowledge representation into deep learning tasks. This study lays a theoretical and experimental foundation for introducing different levels of knowledge representation into deep learning in the future.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improving Visual Reasoning Through Semantic Representation

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The sentence representation module aims to map natural language sentences into a dense vector space under the premise of keeping the sentence expression semantics unchanged. It transforms the complex logical reasoning process into the solution process of similarity between sentences and solving the relationship between sentences by computer [7][8][9].…”

Section: Introductionmentioning

confidence: 99%

Sentence Representation Method Based on Multi-Layer Semantic Network

2021

Self Cite

View full text Add to dashboard Cite

With the development of artificial intelligence, more and more people hope that computers can understand human language through natural language technology, learn to think like human beings, and finally replace human beings to complete the highly difficult tasks with cognitive ability. As the key technology of natural language understanding, sentence representation reasoning technology mainly focuses on the sentence representation method and the reasoning model. Although the performance has been improved, there are still some problems such as incomplete sentence semantic expression, lack of depth of reasoning model, and lack of interpretability of the reasoning process. In this paper, a multi-layer semantic representation network is designed for sentence representation. The multi-attention mechanism obtains the semantic information of different levels of a sentence. The word order information of the sentence is also integrated by adding the relative position mask between words to reduce the uncertainty caused by word order. Finally, the method is verified on the task of text implication recognition and emotion classification. The experimental results show that the multi-layer semantic representation network can promote sentence representation’s accuracy and comprehensiveness.

show abstract