Enriching Cold Start Personalized Language Model Using Social Network Information

Huang, Yu-Yang; Yan, Rui; Kuo, Tsung-Ting; Lin, Shou-De

doi:10.3115/v1/p14-2100

Cited by 12 publications

(15 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Language model is a fundamental natural language processing problem and has been well studied for many years [Jurafsky and Martin, 2008]. It is natural to extend language models to be personalized or socialized since we know that every person in the world has his/her own speech or writing patterns and can be affected by others.…”

Section: Personalized and Socialized Language Modelsmentioning

confidence: 99%

“…Personalized language models were mainly applied to Web search [Croft et al, 2001;Song et al, 2010;Sontag et al, 2012] or collaborative search (where user groups are clustered based on user behaviors instead of explicit connections) [Sun et al, 2005;Teevan et al, 2009;Xue et al, 2009]. Socialized languages models have recently been developed, which were also applied to search problems, but to social media text search [Vosecky et al, 2014;Huang et al, 2014;.The socialization of language models can incorporate social friends' information as a smoothing factor to improve the language sparsity problem.…”

Section: Personalized and Socialized Language Modelsmentioning

confidence: 99%

“…Users are using social media platforms such as Facebook, Twitter, and Yelp to receive friends' update about their lives and learn knowledge from friends. A well-known effect of social network is the concept of "homophily," which has been developed in psychology [Lazarsfeld and Merton, 1954] and observed in social network [McPherson et al, 2001]. It indicates that users being friends tend to share similar opinions or topics.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Socialized Word Embeddings

Zeng

Yin

Song

et al. 2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Word embeddings have attracted a lot of attention. On social media, each user's language use can be significantly affected by the user's friends. In this paper, we propose a socialized word embedding algorithm which can consider both user's personal characteristics of language use and the user's social relationship on social media. To incorporate personal characteristics, we propose to use a user vector to represent each user. Then for each user, the word embeddings are trained based on each user's corpus by combining the global word vectors and local user vector. To incorporate social relationship, we add a regularization term to impose similarity between two friends. In this way, we can train the global word vectors and user vectors jointly. To demonstrate the effectiveness, we used the latest large-scale Yelp data to train our vectors, and designed several experiments to show how user vectors affect the results.

show abstract

Section: Personalized and Socialized Language Modelsmentioning

confidence: 99%

Section: Personalized and Socialized Language Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Socialized Word Embeddings

Zeng

Yin

Song

et al. 2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…It is therefore possible to train personalized LMs because of the reasonable assumption that users with close friend relationships may share common subject topics, wording habits, and linguistic patterns. Personalized LMs are useful in many aspects [19,20,21]. In the area of speech recognition, personalization of LMs has been proposed and investigated for both N-gram-based LMs [22] and recurrent neural networks (RNNLMs) [23] in the very limited previous works.…”

Section: Introductionmentioning

confidence: 99%

Personalizing universal recurrent neural network language model with user characteristic features by social network crowdsourcing

Tseng¹,

Lee²,

Lee³

2015

2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)

View full text Add to dashboard Cite

With the popularity of mobile devices, personalized speech recognizer becomes more realizable today and highly attractive. Each mobile device is primarily used by a single user, so it's possible to have a personalized recognizer well matching to the characteristics of individual user. Although acoustic model personalization has been investigated for decades, much less work have been reported on personalizing language model, probably because of the difficulties in collecting enough personalized corpora. Previous work used the corpora collected from social networks to solve the problem, but constructing a personalized model for each user is troublesome. In this paper, we propose a universal recurrent neural network language model with user characteristic features, so all users share the same model, except each with different user characteristic features. These user characteristic features can be obtained by crowdsouring over social networks, which include huge quantity of texts posted by users with known friend relationships, who may share some subject topics and wording patterns. The preliminary experiments on Facebook corpus showed that this proposed approach not only drastically reduced the model perplexity, but offered very good improvement in recognition accuracy in n-best rescoring tests. This approach also mitigated the data sparseness problem for personalized language models.

show abstract

“…on the specific tracking task. Social information is incorporated into a factor graph model as features (Huang et al, 2014;Yan et al, 2015). These factor graph model based methods are less efficien…”

Section: Related Workmentioning

confidence: 99%

Tackling Sparsity, the Achilles Heel of Social Networks: Language Model Smoothing via Social Regularization

Yan

Liu

et al. 2015

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Confere

Self Cite

View full text Add to dashboard Cite

Online social networks nowadays have the worldwide prosperity, as they have revolutionized the way for people to discover, to share, and to diffuse information. Social networks are powerful, yet they still have Achilles Heel: extreme data sparsity. Individual posting documents, (e.g., a microblog less than 140 characters), seem to be too sparse to make a difference under various scenarios, while in fact they are quite different. We propose to tackle this specific weakness of social networks by smoothing the posting document language model based on social regularization. We formulate an optimization framework with a social regularizer. Experimental results on the Twitter dataset validate the effectiveness and efficiency of our proposed model.

show abstract

Enriching Cold Start Personalized Language Model Using Social Network Information

Cited by 12 publications

References 19 publications

Socialized Word Embeddings

Socialized Word Embeddings

Personalizing universal recurrent neural network language model with user characteristic features by social network crowdsourcing

Tackling Sparsity, the Achilles Heel of Social Networks: Language Model Smoothing via Social Regularization

Contact Info

Product

Resources

About