Feature Projection for Improved Text Classification

Qin, Qi; Hu, Wenpeng; Liu, Bing

doi:10.18653/v1/2020.acl-main.726

Cited by 53 publications

(29 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DESYR leverages representation learning by exercising a novel dependency-inspired variant of the Poincaré embedding [25]. Furthermore, we aim at amputating the class-invariant features, and obtaining superior class-representing features by incorporating the technique for the feature projection, highlighted by Qin et al [30]. DESYR's backbone comprises of two networks in parallel -the regulation-net (r-net) and the spotlight-net (s-net).…”

Section: Proposed Methodology: Desyrmentioning

confidence: 99%

“…It functions as a network trained in parallel to the s-net. As highlighted in [12,30], we employ a Gradient Reversal Layer (GRL) to capture the class-invariant features. In a nutshell, a gradient-reversal layer can be thought of as a pseudo-functional mapping where the forward and backward propagation are respectively defined by two opposed equations as follows:…”

Section: Regulation Net (R-net)mentioning

confidence: 99%

See 1 more Smart Citation

DESYR: Definition and Syntactic Representation Based Claim Detection on the Web

Sundriyal,

Singh,

Akhtar

et al. 2021

Preprint

View full text Add to dashboard Cite

The formulation of a claim rests at the core of argument mining. To demarcate between a claim and a non-claim is arduous for both humans and machines, owing to latent linguistic variance between the two and the inadequacy of extensive definition-based formalization. Furthermore, the increase in the usage of online social media has resulted in an explosion of unsolicited information on the web presented as informal text. To account for the aforementioned, in this paper, we propose DESYR. It is a framework that intends on annulling the said issues for informal web-based text by leveraging a combination of hierarchical representation learning (dependency-inspired Poincaré embedding), definition-based alignment, and feature projection. We do away with fine-tuning compute-heavy language models in favor of fabricating a more domain-centric but lighter approach. Experimental results indicate that DESYR builds upon the state-of-the-art system across four benchmark claim datasets, most of which were constructed with informal texts. We see an increase of 3 claim-F1 points on the LESA-Twitter dataset, an increase of 1 claim-F1 point and 9 macro-F1 points on the Online Comments (OC) dataset, an increase of 24 claim-F1 points and 17 macro-F1 points on the Web Discourse (WD) dataset, and an increase of 8 claim-F1 points and 5 macro-F1 points on the Micro Texts (MT) dataset. We also perform an extensive analysis of the results. We make a 100-D pre-trained version of our Poincaré-variant along with the source code.

show abstract

Section: Proposed Methodology: Desyrmentioning

confidence: 99%

Section: Regulation Net (R-net)mentioning

confidence: 99%

DESYR: Definition and Syntactic Representation Based Claim Detection on the Web

Sundriyal,

Singh,

Akhtar

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…•Feature Projection (FP): It is a novel approach to improve representation learning through feature projection. Existing features are projected into an orthogonal space (Qin et al, 2020).…”

Section: Comparison With Baselinesmentioning

confidence: 99%

Enhancing Interpretable Clauses Semantically using Pretrained Word Representation

Yadav¹,

Jiao²,

Granmo³

et al. 2021

Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based on propositional logic, which has demonstrated competitive performance in many Natural Language Processing (NLP) tasks, including sentiment analysis, text classification, and Word Sense Disambiguation. To obtain human-level interpretability, legacy TM employs Boolean input features such as bag-of-words (BOW). However, the BOW representation makes it difficult to use any pre-trained information, for instance, word2vec and GloVe word representations. This restriction has constrained the performance of TM compared to deep neural networks (DNNs) in NLP. To reduce the performance gap, in this paper, we propose a novel way of using pre-trained word representations for TM. The approach significantly enhances the performance and interpretability of TM. We achieve this by extracting semantically related words from pre-trained word representations as input features to the TM. Our experiments show that the accuracy of the proposed approach is significantly higher than the previous BOW-based TM, reaching the level of DNN-based models.

show abstract

“…Our work is related to sentiment classification (Liu, 2012), lifelong learning and continual learning. For sentiment classification, recent deep learning models have been shown to outperform traditional methods (Kim, 2014;Devlin et al, 2018;Shen et al, 2018;Qin et al, 2020). However, these models don't retain or transfer the knowledge to new tasks.…”

Section: Related Workmentioning

confidence: 99%

Using the Past Knowledge to Improve Sentiment Classification

Qin

Liu

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

Self Cite

View full text Add to dashboard Cite

This paper studies sentiment classification in the lifelong learning setting that incrementally learns a sequence of sentiment classification tasks. It proposes a new lifelong learning model (called L2PG) that can retain and selectively transfer the knowledge learned in the past to help learn the new task. A key innovation of this proposed model is a novel parameter-gate (p-gate) mechanism that regulates the flow or transfer of the previously learned knowledge to the new task. Specifically, it can selectively use the network parameters (which represent the retained knowledge gained from the previous tasks) to assist the learning of the new task t. Knowledge distillation is also employed in the process to preserve the past knowledge by approximating the network output at the state when task t − 1 was learned. Experimental results show that L2PG outperforms strong baselines, including even multiple task learning.

show abstract

Feature Projection for Improved Text Classification

Cited by 53 publications

References 29 publications

DESYR: Definition and Syntactic Representation Based Claim Detection on the Web

DESYR: Definition and Syntactic Representation Based Claim Detection on the Web

Enhancing Interpretable Clauses Semantically using Pretrained Word Representation

Using the Past Knowledge to Improve Sentiment Classification

Contact Info

Product

Resources

About