Csrncva: A Model of Cross-Media Semantic Retrieval Based on Neural Computing of Visual and Auditory Sensations

Liu, Yang; Cai, Kai-Yuan; Li, Chun; Zheng, Fang

doi:10.14311/nnw.2018.28.018

Cited by 4 publications

(8 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How to efficiently set the parameter range still needs further exploration. e cross-modal retrieval method based on deep learning makes full use of the powerful feature extraction capabilities of deep learning models, learns the feature representation of different modal data, and then establishes semantic associations between modalities at a high level [17]. Reference [18] proposed a two-stage deep learning method for supervised cross-modal retrieval, extending the traditional norm-related analysis from 2 views to 3 views and conducting supervised learning in two stages.…”

Section: Real-valued Cross-modal Retrieval Methodmentioning

confidence: 99%

A Cross-Modal Image and Text Retrieval Method Based on Efficient Feature Extraction and Interactive Learning CAE

Yin

Chen

2022

Scientific Programming

View full text Add to dashboard Cite

In view of the complexity of the multimodal environment and the existing shallow network structure that cannot achieve high-precision image and text retrieval, a cross-modal image and text retrieval method combining efficient feature extraction and interactive learning convolutional autoencoder (CAE) is proposed. First, the residual network convolution kernel is improved by incorporating two-dimensional principal component analysis (2DPCA) to extract image features and extracting text features through long short-term memory (LSTM) and word vectors to efficiently extract graphic features. Then, based on interactive learning CAE, cross-modal retrieval of images and text is realized. Among them, the image and text features are respectively input to the two input terminals of the dual-modal CAE, and the image-text relationship model is obtained through the interactive learning of the middle layer to realize the image-text retrieval. Finally, based on Flickr30K, MSCOCO, and Pascal VOC 2007 datasets, the proposed method is experimentally demonstrated. The results show that the proposed method can complete accurate image retrieval and text retrieval. Moreover, the mean average precision (MAP) has reached more than 0.3, the area of precision-recall rate (PR) curves are better than other comparison methods, and they are applicable.

show abstract

Section: Real-valued Cross-modal Retrieval Methodmentioning

confidence: 99%

A Cross-Modal Image and Text Retrieval Method Based on Efficient Feature Extraction and Interactive Learning CAE

Yin

Chen

2022

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…The spiking probability is propagating among micro-columns. Micro-columns collect information from lower neighbor micro-columns and disseminate information from upper neighbor micro-columns (Liu et al 2018a). At the same time, it also receives feedback information from LDP, and prediction information from an upper neighbor.…”

Section: Hypothesis 1 (Bmm Hypothesis)mentioning

confidence: 99%

“…For instance, the node is double structure in HTM, RBM, SVM and so on. As shown in Figure 2 C, S mimicking functions from L1 to L4, and simulates memory and spatial patterns process (Liu et al 2018a). T mimicking functions of L5 and L6, and simulates memory and temporal patterns process.…”

Section: Hypothesis 1 (Bmm Hypothesis)mentioning

confidence: 99%

“…(2) Hierarchical network architecture of super-column According to neurocognitive system hierarchical architecture and temporal-spatial locality, super-column architecture also uses hierarchical, multi-level, and bidirectional mapping structure. Super-column composed by a micro-column with principles of "the same layer collaborative" and "hierarchical processing" (Figure 3) (Liu et al 2018a).…”

Section: Bsmentioning

confidence: 99%

“…Studies indicate that the central nervous system is a scale-free and small-world complex network. On the basis of the WMN visualization and document (Liu et al 2018a), we propose a simplified structure model of the whole brain WMN in Figure 7. and PMJ (Perception, Memory, and Judgment) model (Fu et al 2014b) should also deserve our attention.…”

Section: Long Distance Pathways and Neural Circuitsmentioning

confidence: 99%

See 2 more Smart Citations

Research on the Cross-modal Cognitive Neural Computing Framework of the Brain and Mind Inspired Intelligence

Liu

Wei

2022

Preprint

Self Cite

View full text Add to dashboard Cite

To address the problems of scientific theory of brain and mind, common technology and engineering application of nature-inspired intelligence, this paper is focused on research in the semantic-oriented computing framework design for multimedia and multimodal information. The multimedia neural cognitive computing model of brain-inspired computing was designed based on the brain mechanism of nervous system and mind architecture of cognitive system. Furthermore, the semantic-oriented hierarchical cross-modal cognitive neural computing framework of brain-like computing was proposed based on multimedia neural cognitive computing model. Furthermore, the formal description and analysis of cross-modal cognitive neural computing framework was given. It would effectively improve the performance of semantic processing of multimedia and cross-modal information such as target detection, classification and recognition in high-resolution remote sensing image, and has far-reaching significance for exploration and realization brain and mind inspired computing.

show abstract

Multifeature Fusion Tracking Algorithm Based on Self-Associative Memory Learning Mechanism

2022

View full text Add to dashboard Cite

A multi-feature fusion tracking algorithm updated with a self-associative memory learning mechanism is proposed to address the problems of short-time disappearance, re-emergence of the target and instability of single features in the kernelized correlation filtering algorithm. When extracting features, directional gradient histogram features, color features, and scale invariant features are fused instead of single features to collect more features of the target and increase the feature robustness. In the detection stage, the bimodal detection is proposed to judge whether the target model needs updating. Bimodal detection is used to judge the maximum target response in the search domain and predict the location of the target in the next frame. The self-associative memory learning mechanism was added into the updating template, and the original algorithm framework was improved to cope with the change of target model. The new algorithm update is biogenic, can recover fragment information, deal with complex and changeable tracking situation. Simulation experiments were conducted on the OTB50, OTB100, and UAV123 video datasets for the classical and new algorithms. The simulation verified that the proposed tracking algorithm has a high success and accuracy rate, which has research value. The tracking success rate improved by 23.6% and the accuracy rate improved by 18.8%. 15 16 nels (CSK) to improve the computational speed, but the 54 grayscale features used can only adapt to simple environ-55 ments and are susceptible to complex image backgrounds 56 and similar colors of the target backgrounds. Henriques et al. 57 optimized CSK by replacing the Histogram of Oriented Gra-58 dient (HOG) features with grayscale and proposed a ker-59 nelized correlation filter (KCF) algorithm [6], which has 60 the problems of not adapting to large target movements and 61 single feature instability that have not been solved. The color 62 names (CN) tracker [7] uses color attributes in the filter track-63 ing algorithm and adopts an adaptive dimensionality reduc-64 tion strategy to reduce eleven dimensional color features to 65 two dimensional, which improves the algorithm performance 66 while ensuring efficient tracking. The scale adaptive multiple 67 feature (SAMF) [8] tracker based on multi-feature fusion 68 simultaneously fuses the original image grayscale informa-69 tion, color attributes, and HOG multiple features to obtain 70 more robust results. Danelljan et al. proposed a discriminative 71 scale space tracker (DSST) [9] to study the scale problem 72 and propose a solution algorithm. Balancing the weight ratio 73 of the scale and location filters still needs to be studied. For 74 complex illumination, background information, and the use 75 of contextual information [10], [11], [12], [13] have been 76 studied to propose the multiple kernelized correlation filters 77 (MKCF) tracking algorithm, to make full use of the discrim-78 inative invariance of the power spectrums (power spectrums) 79 of various features and further improve the pe...

show abstract

Csrncva: A Model of Cross-Media Semantic Retrieval Based on Neural Computing of Visual and Auditory Sensations

Cited by 4 publications

References 29 publications

A Cross-Modal Image and Text Retrieval Method Based on Efficient Feature Extraction and Interactive Learning CAE

A Cross-Modal Image and Text Retrieval Method Based on Efficient Feature Extraction and Interactive Learning CAE

Research on the Cross-modal Cognitive Neural Computing Framework of the Brain and Mind Inspired Intelligence

Multifeature Fusion Tracking Algorithm Based on Self-Associative Memory Learning Mechanism

Contact Info

Product

Resources

About