Transfer Learning in Sentiment Classification with Deep Neural Networks

Abstract

Cross-domain sentiment classifiers aim to predict the polarity (i.e. sentiment orientation) of target text documents, by reusing a knowledge model learnt from a different source domain. Distinct domains are typically heterogeneous in language, so that transfer learning techniques are advisable to support knowledge transfer from source to target. Deep neural networks have recently reached the state-of-the-art in many NLP tasks, including in-domain sentiment classification, but few of them involve transfer learning and cross-domain sentiment solutions. This paper moves forward the investigation started in a previous work [1], where an unsupervised deep approach for text mining, called Paragraph Vector (PV), achieved cross-domain accuracy equivalent to a method based on Markov Chain (MC), developed ad hoc for crossdomain sentiment classification. In this work, Gated Recurrent Unit (GRU) is included into the previous investigation, showing that memory units are beneficial for cross-domain when enough training data are available. Moreover, the knowledge models learnt from the source domain are tuned on small samples of target instances to foster transfer learning. PV is almost unaffected by fine-tuning, because it is already able to capture word semantics without supervision. On the other hand, fine-tuning boosts the crossdomain performance of GRU. The smaller is the training set used, the greater is the improvement of accuracy.

Unsupervised Descriptive Text Mining for Knowledge Graph Learning

“…As a consequence, the development of OL currently goes hand in hand with that of NLP and advanced machine learning approaches which are essential to extract knowledge from text documents (Domeniconi et al, 2015). Transfer learning to target domains with unlabeled data is becoming increasingly necessary (Domeniconi et al, 2014b;Domeniconi et al, 2014c;Domeniconi et al, 2016b;Domeniconi et al, 2017;Pagliarani et al, 2017;Moro et al, 2018), and is the most frequent case for social messages (Domeniconi et al, 2016b).…”