A Semantic Text Similarity Model for Double Short Chinese Sequences

Shancheng, Tang; Yunyue, Bai; Ma, Fuying

doi:10.1109/icitbs.2018.00190

Cited by 4 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the EnData dataset, we evaluate the performance of our models on 7 semantic textual similarity (STS) tasks, namely STS 2012-2016 [1,2,3,4,5], STS-Benchmark [7], and SICK-Relatedness [20]. On the other hand, for the CnData dataset, we evaluated the models on the Chinese STS-Benchmark (C-STS-B) [25] and SimCLUE [31]. SimCLUE includes most of the available opensource datasets of semantic similarity and natural language inference in the Chinese domain.…”

Section: Setupsmentioning

confidence: 99%

“…These versions include bertbase-uncased, bert-base-multilingual-uncased, bert-large-uncased as well as stefan-it/albert-large-german-cased and denpa92/bert-basecantonese for German and Cantonese datasets respectively. During the experiments, we fine-tune the IFCL for one epoch and evaluate the models with the verification sets of STS-B or C-STS-B [7,25] after every 151 steps. Our evaluation metric of choice is Spearman's correlation, consistent with previous works.…”

Section: Setupsmentioning

confidence: 99%

See 1 more Smart Citation

Identical and Fraternal Twins: Fine-Grained Semantic Contrastive Learning of Sentence Representations

Xiao,

Li,

Chen

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

The enhancement of unsupervised learning of sentence representations has been significantly achieved by the utility of contrastive learning. This approach clusters the augmented positive instance with the anchor instance to create a desired embedding space. However, relying solely on the contrastive objective can result in sub-optimal outcomes due to its inability to differentiate subtle semantic variations between positive pairs. Specifically, common data augmentation techniques frequently introduce semantic distortion, leading to a semantic margin between the positive pair. While the InfoNCE loss function overlooks the semantic margin and prioritizes similarity maximization between positive pairs during training, leading to the insensitive semantic comprehension ability of the trained model. In this paper, we introduce a novel Identical and Fraternal Twins of Contrastive Learning (named IFTCL) framework, capable of simultaneously adapting to various positive pairs generated by different augmentation techniques. We propose a Twins Loss to preserve the innate margin during training and promote the potential of data enhancement in order to overcome the sub-optimal issue. We also present proof-of-concept experiments combined with the contrastive objective to prove the validity of the proposed Twins Loss. Furthermore, we propose a hippocampus queue mechanism to restore and reuse the negative instances without additional calculation, which further enhances the efficiency and performance of the IFCL. We verify the IFCL framework on nine semantic textual similarity tasks with both English and Chinese datasets, and the experimental results show that IFCL outperforms state-of-the-art methods.

show abstract

Section: Setupsmentioning

confidence: 99%

Section: Setupsmentioning

confidence: 99%

Identical and Fraternal Twins: Fine-Grained Semantic Contrastive Learning of Sentence Representations

Xiao,

Li,

Chen

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…With the continuous development of machine learning technology, the application of text similarity matching algorithm in Chinese text has also been pushed to a new height [9][10][11][12]. Shancheng Tang et al proposed a Chinese short text sequence similarity model based on LSTM, and used the Chinese semantic similarity data set designed by experts for training, so as to overcome the polysemy and semantic ambiguity of Chinese text to some extent [13]. Tao Lei et al proposed a search strategy based on ElasticSearch and semantic similarity matching.…”

Section: Introductionmentioning

confidence: 99%

A multi-dimensional fusion strategy similarity measure method for patent application technology disclosure document

Zhu,

Li,

Hou

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Patent application technology disclosure document is one of the important bases for judging patent novelty and uniqueness. Automated evaluation can effectively solve the problems of long time and strong subjectivity of human evaluation. The text similarity evaluation algorithm based on corpus and deep learning technology has problems such as insufficient amount of cross-library learning data and insufficient core content tendency in the similarity judgment of patent application technology disclosure document, which limits their performance and practical application. In this paper, we propose a similarity evaluation method of patent application technology disclosure document based on multi-dimensional fusion strategy to realize the similarity measurement of patents. Firstly, in the text preprocessing section, word segmentation reconstruction and similarity evaluation optimization strategies based on word frequency and part-of-speech score weighted fusion are proposed. Then, a similarity calculation method of patent application technology disclosure document based on two new mapping spaces of dot matrix and image is proposed to achieve a more diversified comprehensive evaluation. The algorithm was evaluated by using four published text similarity matching datasets (containing 0–5 or 0/1 labels) and a set of patent application technology disclosure documents. Experimental results show that on the published text similarity matching datasets, the similarity evaluation method under the multi-dimensional fusion strategy proposed in this paper has a discrimination accuracy improvement of about 10% compared to traditional vector semantic model, and can match the discriminative ability of lightweight deep learning models without the need for training. At the same time, the discrimination accuracy of the proposed method on the sample dataset of patent application technology disclosure document is superior to traditional vector semantic model (20%) and various deep learning models (1%-8%), and the precision and recall rate are relatively balanced. The visual analysis results on the dataset of the patent application technology disclosure documents also prove the effectiveness and reliability of the similarity calculation method proposed in the dot matrix and image space, which provide a new idea and method for the similarity evaluation between patent application technology disclosure document.

show abstract

Sentiment Analysis of Short Texts Using SVMs and VSMs-Based Multiclass Semantic Classification

Suresh Kumar,

Radha Mani,

Ananth Kumar

et al. 2024

Applied Artificial Intelligence

View full text Add to dashboard Cite

A Semantic Text Similarity Model for Double Short Chinese Sequences

Cited by 4 publications

References 8 publications

Identical and Fraternal Twins: Fine-Grained Semantic Contrastive Learning of Sentence Representations

Identical and Fraternal Twins: Fine-Grained Semantic Contrastive Learning of Sentence Representations

A multi-dimensional fusion strategy similarity measure method for patent application technology disclosure document

Sentiment Analysis of Short Texts Using SVMs and VSMs-Based Multiclass Semantic Classification

Contact Info

Product

Resources

About